2024-09-10T20:54:34.141 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T20:54:34.162 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T20:54:34.248 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899658 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/client-limits} email: hyelloji@redhat.com first_in_suite: false job_id: '7899658' kernel: client: branch: testing flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - responding to mclientcaps\(revoke\) - not advance its oldest_client_tid - failing to advance its oldest client/flush tid - Too many inodes in cache - failing to respond to cache pressure - slow requests are blocked - failing to respond to capability release - MDS cache is too large - \(MDS_CLIENT_OLDEST_TID\) - \(MDS_CACHE_OVERSIZED\) sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 kclient: syntax: v2 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKX3/Vxzmvje760rev3P8sOPUCQzVAT1NO47oyLz8FcyIib2DW5LfEKmzWIeEFsJTg/9YOXmekZ+eIAp7Nxm7bQ= smithi052.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHs+CBX0PV3yOP0028RtPQaSOAuqnP82imiicRokIBQwJ5+ew61XaelpL5yUtUjiBn9cIGMvn7zdXOQrHRlB1JY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_limits teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T20:54:34.249 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T20:54:34.249 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T20:54:34.250 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T20:54:34.251 INFO:teuthology.task.internal:Checking packages... 2024-09-10T20:54:34.271 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T20:54:34.272 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T20:54:34.272 INFO:teuthology.packaging:ref: None 2024-09-10T20:54:34.272 INFO:teuthology.packaging:tag: None 2024-09-10T20:54:34.272 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T20:54:34.272 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:54:34.272 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T20:54:34.421 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854-g8f219c47-1jammy 2024-09-10T20:54:34.422 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T20:54:34.438 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T20:54:34.439 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T20:54:34.453 INFO:teuthology.task.internal:Saving configuration 2024-09-10T20:54:34.469 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T20:54:34.480 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T20:54:34.514 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899658', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 20:48:26.697667', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOjauTeBfM5ZRpOkRHo/8/HAXJ0DkL9z87m/ajpsjdK1'} 2024-09-10T20:54:34.545 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899658', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 20:48:26.696236', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBDOiARi3TqyzYY1bR0B/JdEoh9iARlDPr8Fy1fSmcAb'} 2024-09-10T20:54:34.545 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T20:54:34.555 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-10T20:54:34.555 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-10T20:54:34.555 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T20:54:34.641 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f002dacfbe0>, signals=[15]) 2024-09-10T20:54:34.641 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T20:54:34.647 INFO:teuthology.task.internal:Opening connections... 2024-09-10T20:54:34.648 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T20:54:34.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:54:34.737 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T20:54:34.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:54:34.830 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T20:54:34.837 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-09-10T20:54:34.843 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-09-10T20:54:34.843 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-09-10T20:54:34.891 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-10T20:54:34.891 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Ubuntu" 2024-09-10T20:54:34.891 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="22.04" 2024-09-10T20:54:34.891 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-10T20:54:34.891 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_CODENAME=jammy 2024-09-10T20:54:34.891 INFO:teuthology.orchestra.run.smithi016.stdout:ID=ubuntu 2024-09-10T20:54:34.891 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE=debian 2024-09-10T20:54:34.891 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T20:54:34.891 INFO:teuthology.orchestra.run.smithi016.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T20:54:34.892 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T20:54:34.892 INFO:teuthology.orchestra.run.smithi016.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T20:54:34.892 INFO:teuthology.orchestra.run.smithi016.stdout:UBUNTU_CODENAME=jammy 2024-09-10T20:54:34.892 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-09-10T20:54:34.920 DEBUG:teuthology.orchestra.run.smithi052:> uname -m 2024-09-10T20:54:34.926 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2024-09-10T20:54:34.926 DEBUG:teuthology.orchestra.run.smithi052:> cat /etc/os-release 2024-09-10T20:54:34.973 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-10T20:54:34.973 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="Ubuntu" 2024-09-10T20:54:34.974 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="22.04" 2024-09-10T20:54:34.974 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-10T20:54:34.974 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_CODENAME=jammy 2024-09-10T20:54:34.974 INFO:teuthology.orchestra.run.smithi052.stdout:ID=ubuntu 2024-09-10T20:54:34.974 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE=debian 2024-09-10T20:54:34.974 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T20:54:34.974 INFO:teuthology.orchestra.run.smithi052.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T20:54:34.974 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T20:54:34.974 INFO:teuthology.orchestra.run.smithi052.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T20:54:34.974 INFO:teuthology.orchestra.run.smithi052.stdout:UBUNTU_CODENAME=jammy 2024-09-10T20:54:34.975 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2024-09-10T20:54:35.004 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T20:54:35.014 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T20:54:35.022 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T20:54:35.022 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-09-10T20:54:35.024 DEBUG:teuthology.orchestra.run.smithi052:> test '!' -e /home/ubuntu/cephtest 2024-09-10T20:54:35.029 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T20:54:35.036 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T20:54:35.036 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-09-10T20:54:35.073 DEBUG:teuthology.orchestra.run.smithi052:> test -z $(ls -A /var/lib/ceph) 2024-09-10T20:54:35.087 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T20:54:35.160 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T20:54:35.175 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T20:54:35.176 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-10T20:54:35.176 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-10T20:54:35.176 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T20:54:35.177 DEBUG:teuthology.orchestra.run.smithi052:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T20:54:35.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:54:35.181 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-10T20:54:35.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:54:35.193 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-10T20:54:35.360 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T20:54:35.360 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T20:54:35.361 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-09-10T20:54:35.362 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T20:54:35.362 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T20:54:35.363 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-10T20:54:35.368 INFO:teuthology.orchestra.run.smithi052.stdout:5.15.0-56-generic 2024-09-10T20:54:35.368 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T20:54:35.368 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T20:54:35.369 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-09-10T20:54:35.369 DEBUG:teuthology.orchestra.run.smithi052:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-10T20:54:35.370 INFO:teuthology.orchestra.run.smithi016.stdout:5.15.0-56-generic 2024-09-10T20:54:35.370 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T20:54:35.370 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T20:54:35.371 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-09-10T20:54:35.371 DEBUG:teuthology.orchestra.run.smithi016:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-10T20:54:35.418 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-10T20:54:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-10T20:54:35.550 INFO:teuthology.orchestra.run.smithi052.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-10T20:54:35.550 INFO:teuthology.orchestra.run.smithi052.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-10T20:54:35.551 INFO:teuthology.orchestra.run.smithi052.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-10T20:54:35.551 INFO:teuthology.orchestra.run.smithi052.stderr:hint: 2024-09-10T20:54:35.551 INFO:teuthology.orchestra.run.smithi052.stderr:hint: git config --global init.defaultBranch 2024-09-10T20:54:35.551 INFO:teuthology.orchestra.run.smithi052.stderr:hint: 2024-09-10T20:54:35.551 INFO:teuthology.orchestra.run.smithi052.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-10T20:54:35.551 INFO:teuthology.orchestra.run.smithi052.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-10T20:54:35.551 INFO:teuthology.orchestra.run.smithi052.stderr:hint: 2024-09-10T20:54:35.551 INFO:teuthology.orchestra.run.smithi052.stderr:hint: git branch -m 2024-09-10T20:54:35.551 INFO:teuthology.orchestra.run.smithi052.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-10T20:54:35.554 DEBUG:teuthology.orchestra.run.smithi052:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-10T20:54:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-10T20:54:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-10T20:54:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-10T20:54:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:hint: 2024-09-10T20:54:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:hint: git config --global init.defaultBranch 2024-09-10T20:54:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:hint: 2024-09-10T20:54:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-10T20:54:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-10T20:54:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:hint: 2024-09-10T20:54:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:hint: git branch -m 2024-09-10T20:54:35.566 INFO:teuthology.orchestra.run.smithi016.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-10T20:54:35.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-10T20:54:35.622 DEBUG:teuthology.orchestra.run.smithi052:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-10T20:54:35.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-10T20:54:35.680 DEBUG:teuthology.orchestra.run.smithi052:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-10T20:54:35.692 DEBUG:teuthology.orchestra.run.smithi016:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-10T20:55:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-10T20:55:10.157 INFO:teuthology.orchestra.run.smithi052.stderr: * [new branch] main -> origin/main 2024-09-10T20:55:10.196 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20190312 -> 20190312 2024-09-10T20:55:10.196 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20190416 -> 20190416 2024-09-10T20:55:10.196 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20190514 -> 20190514 2024-09-10T20:55:10.196 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20190618 -> 20190618 2024-09-10T20:55:10.196 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20190717 -> 20190717 2024-09-10T20:55:10.197 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20190815 -> 20190815 2024-09-10T20:55:10.197 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20190923 -> 20190923 2024-09-10T20:55:10.197 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20191022 -> 20191022 2024-09-10T20:55:10.197 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20191215 -> 20191215 2024-09-10T20:55:10.197 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20200122 -> 20200122 2024-09-10T20:55:10.197 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20200316 -> 20200316 2024-09-10T20:55:10.197 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20200421 -> 20200421 2024-09-10T20:55:10.197 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20200519 -> 20200519 2024-09-10T20:55:10.201 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20200619 -> 20200619 2024-09-10T20:55:10.201 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20200721 -> 20200721 2024-09-10T20:55:10.201 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20200817 -> 20200817 2024-09-10T20:55:10.201 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20200918 -> 20200918 2024-09-10T20:55:10.201 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20201022 -> 20201022 2024-09-10T20:55:10.202 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20201118 -> 20201118 2024-09-10T20:55:10.202 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20201218 -> 20201218 2024-09-10T20:55:10.202 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20210208 -> 20210208 2024-09-10T20:55:10.202 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20210315 -> 20210315 2024-09-10T20:55:10.202 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20210511 -> 20210511 2024-09-10T20:55:10.202 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20210716 -> 20210716 2024-09-10T20:55:10.202 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20210818 -> 20210818 2024-09-10T20:55:10.202 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20210919 -> 20210919 2024-09-10T20:55:10.202 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20211027 -> 20211027 2024-09-10T20:55:10.202 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20211216 -> 20211216 2024-09-10T20:55:10.203 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20220209 -> 20220209 2024-09-10T20:55:10.203 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20220310 -> 20220310 2024-09-10T20:55:10.203 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20220411 -> 20220411 2024-09-10T20:55:10.203 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20220509 -> 20220509 2024-09-10T20:55:10.203 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20220610 -> 20220610 2024-09-10T20:55:10.203 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20220708 -> 20220708 2024-09-10T20:55:10.203 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20220815 -> 20220815 2024-09-10T20:55:10.203 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20220913 -> 20220913 2024-09-10T20:55:10.203 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20221012 -> 20221012 2024-09-10T20:55:10.203 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20221109 -> 20221109 2024-09-10T20:55:10.203 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20221214 -> 20221214 2024-09-10T20:55:10.203 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20230117 -> 20230117 2024-09-10T20:55:10.203 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20230210 -> 20230210 2024-09-10T20:55:10.204 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20230310 -> 20230310 2024-09-10T20:55:10.204 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20230404 -> 20230404 2024-09-10T20:55:10.204 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20230515 -> 20230515 2024-09-10T20:55:10.204 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20230625 -> 20230625 2024-09-10T20:55:10.204 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20230804 -> 20230804 2024-09-10T20:55:10.204 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20230919 -> 20230919 2024-09-10T20:55:10.205 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20231030 -> 20231030 2024-09-10T20:55:10.206 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20231111 -> 20231111 2024-09-10T20:55:10.206 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20231211 -> 20231211 2024-09-10T20:55:10.206 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20240115 -> 20240115 2024-09-10T20:55:10.206 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20240220 -> 20240220 2024-09-10T20:55:10.206 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20240312 -> 20240312 2024-09-10T20:55:10.206 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20240410 -> 20240410 2024-09-10T20:55:10.206 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20240513 -> 20240513 2024-09-10T20:55:10.206 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20240610 -> 20240610 2024-09-10T20:55:10.206 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20240709 -> 20240709 2024-09-10T20:55:10.206 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20240811 -> 20240811 2024-09-10T20:55:10.206 INFO:teuthology.orchestra.run.smithi052.stderr: * [new tag] 20240909 -> 20240909 2024-09-10T20:55:12.480 INFO:teuthology.orchestra.run.smithi016.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-10T20:55:12.480 INFO:teuthology.orchestra.run.smithi016.stderr: * [new branch] main -> origin/main 2024-09-10T20:55:12.518 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190312 -> 20190312 2024-09-10T20:55:12.518 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190416 -> 20190416 2024-09-10T20:55:12.519 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190514 -> 20190514 2024-09-10T20:55:12.519 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190618 -> 20190618 2024-09-10T20:55:12.519 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190717 -> 20190717 2024-09-10T20:55:12.519 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190815 -> 20190815 2024-09-10T20:55:12.520 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190923 -> 20190923 2024-09-10T20:55:12.520 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20191022 -> 20191022 2024-09-10T20:55:12.520 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20191215 -> 20191215 2024-09-10T20:55:12.520 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200122 -> 20200122 2024-09-10T20:55:12.520 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200316 -> 20200316 2024-09-10T20:55:12.520 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200421 -> 20200421 2024-09-10T20:55:12.520 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200519 -> 20200519 2024-09-10T20:55:12.521 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200619 -> 20200619 2024-09-10T20:55:12.521 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200721 -> 20200721 2024-09-10T20:55:12.523 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200817 -> 20200817 2024-09-10T20:55:12.523 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200918 -> 20200918 2024-09-10T20:55:12.523 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20201022 -> 20201022 2024-09-10T20:55:12.523 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20201118 -> 20201118 2024-09-10T20:55:12.523 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20201218 -> 20201218 2024-09-10T20:55:12.523 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20210208 -> 20210208 2024-09-10T20:55:12.523 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20210315 -> 20210315 2024-09-10T20:55:12.523 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20210511 -> 20210511 2024-09-10T20:55:12.523 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20210716 -> 20210716 2024-09-10T20:55:12.523 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20210818 -> 20210818 2024-09-10T20:55:12.524 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20210919 -> 20210919 2024-09-10T20:55:12.524 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20211027 -> 20211027 2024-09-10T20:55:12.524 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20211216 -> 20211216 2024-09-10T20:55:12.524 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220209 -> 20220209 2024-09-10T20:55:12.524 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220310 -> 20220310 2024-09-10T20:55:12.524 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220411 -> 20220411 2024-09-10T20:55:12.524 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220509 -> 20220509 2024-09-10T20:55:12.524 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220610 -> 20220610 2024-09-10T20:55:12.524 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220708 -> 20220708 2024-09-10T20:55:12.524 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220815 -> 20220815 2024-09-10T20:55:12.524 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220913 -> 20220913 2024-09-10T20:55:12.527 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20221012 -> 20221012 2024-09-10T20:55:12.527 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20221109 -> 20221109 2024-09-10T20:55:12.527 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20221214 -> 20221214 2024-09-10T20:55:12.527 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230117 -> 20230117 2024-09-10T20:55:12.527 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230210 -> 20230210 2024-09-10T20:55:12.527 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230310 -> 20230310 2024-09-10T20:55:12.527 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230404 -> 20230404 2024-09-10T20:55:12.527 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230515 -> 20230515 2024-09-10T20:55:12.527 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230625 -> 20230625 2024-09-10T20:55:12.527 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230804 -> 20230804 2024-09-10T20:55:12.527 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230919 -> 20230919 2024-09-10T20:55:12.527 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20231030 -> 20231030 2024-09-10T20:55:12.528 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20231111 -> 20231111 2024-09-10T20:55:12.528 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20231211 -> 20231211 2024-09-10T20:55:12.528 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240115 -> 20240115 2024-09-10T20:55:12.528 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240220 -> 20240220 2024-09-10T20:55:12.528 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240312 -> 20240312 2024-09-10T20:55:12.528 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240410 -> 20240410 2024-09-10T20:55:12.528 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240513 -> 20240513 2024-09-10T20:55:12.528 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240610 -> 20240610 2024-09-10T20:55:12.528 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240709 -> 20240709 2024-09-10T20:55:12.528 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240811 -> 20240811 2024-09-10T20:55:12.528 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240909 -> 20240909 2024-09-10T20:55:14.970 INFO:teuthology.orchestra.run.smithi052.stderr:Updating files: 28% (1011/3501) Updating files: 29% (1016/3501) Updating files: 30% (1051/3501) Updating files: 31% (1086/3501) Updating files: 32% (1121/3501) Updating files: 33% (1156/3501) Updating files: 34% (1191/3501) Updating files: 35% (1226/3501) Updating files: 36% (1261/3501) Updating files: 37% (1296/3501) Updating files: 38% (1331/3501) Updating files: 39% (1366/3501) Updating files: 40% (1401/3501) Updating files: 41% (1436/3501) Updating files: 42% (1471/3501) Updating files: 43% (1506/3501) Updating files: 44% (1541/3501) Updating files: 45% (1576/3501) Updating files: 46% (1611/3501) Updating files: 47% (1646/3501) Updating files: 48% (1681/3501) Updating files: 49% (1716/3501) Updating files: 50% (1751/3501) Updating files: 51% (1786/3501) Updating files: 52% (1821/3501) Updating files: 53% (1856/3501) Updating files: 54% (1891/3501) Updating files: 55% (1926/3501) Updating files: 56% (1961/3501) Updating files: 57% (1996/3501) Updating files: 58% (2031/3501) Updating files: 58% (2046/3501) Updating files: 59% (2066/3501) Updating files: 60% (2101/3501) Updating files: 61% (2136/3501) Updating files: 62% (2171/3501) Updating files: 63% (2206/3501) Updating files: 64% (2241/3501) Updating files: 65% (2276/3501) Updating files: 66% (2311/3501) Updating files: 67% (2346/3501) Updating files: 68% (2381/3501) Updating files: 68% (2415/3501) Updating files: 69% (2416/3501) Updating files: 70% (2451/3501) Updating files: 71% (2486/3501) Updating files: 72% (2521/3501) Updating files: 73% (2556/3501) Updating files: 74% (2591/3501) Updating files: 75% (2626/3501) Updating files: 76% (2661/3501) Updating files: 77% (2696/3501) Updating files: 78% (2731/3501) Updating files: 79% (2766/3501) Updating files: 80% (2801/3501) Updating files: 81% (2836/3501) Updating files: 82% (2871/3501) Updating files: 82% (2875/3501) Updating files: 83% (2906/3501) Updating files: 84% (2941/3501) Updating files: 85% (2976/3501) Updating files: 86% (3011/3501) Updating files: 87% (3046/3501) Updating files: 88% (3081/3501) Updating files: 89% (3116/3501) Updating files: 90% (3151/3501) Updating files: 91% (3186/3501) Updating files: 92% (3221/3501) Updating files: 93% (3256/3501) Updating files: 94% (3291/3501) Updating files: 95% (3326/3501) Updating files: 96% (3361/3501) Updating files: 97% (3396/3501) Updating files: 98% (3431/3501) Updating files: 99% (3466/3501) Updating files: 100% (3501/3501) Updating files: 100% (3501/3501), done. 2024-09-10T20:55:14.976 INFO:teuthology.orchestra.run.smithi052.stdout:HEAD is now at ce299c0f Merge branch 'rt1320' into 'main' 2024-09-10T20:55:14.999 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-10T20:55:15.000 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T20:55:15.146 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T20:55:15.146 DEBUG:teuthology.orchestra.run.smithi052:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-10T20:55:15.149 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi052.front.sepia.ceph.com to complete... 2024-09-10T20:55:16.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10 20:55:16 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-10T20:55:16.134 INFO:teuthology.orchestra.run.smithi052.stderr:FINISHED --2024-09-10 20:55:16-- 2024-09-10T20:55:16.134 INFO:teuthology.orchestra.run.smithi052.stderr:Total wall clock time: 1.0s 2024-09-10T20:55:16.134 INFO:teuthology.orchestra.run.smithi052.stderr:Downloaded: 1 files, 43M in 0.8s (51.7 MB/s) 2024-09-10T20:55:16.138 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-10T20:55:16.138 DEBUG:teuthology.orchestra.run.smithi052:> sudo dpkg -i /tmp/linux-image.deb 2024-09-10T20:55:16.378 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-10T20:55:17.299 INFO:teuthology.orchestra.run.smithi016.stderr:Updating files: 28% (1012/3501) Updating files: 29% (1016/3501) Updating files: 30% (1051/3501) Updating files: 31% (1086/3501) Updating files: 32% (1121/3501) Updating files: 33% (1156/3501) Updating files: 34% (1191/3501) Updating files: 35% (1226/3501) Updating files: 36% (1261/3501) Updating files: 37% (1296/3501) Updating files: 38% (1331/3501) Updating files: 39% (1366/3501) Updating files: 40% (1401/3501) Updating files: 41% (1436/3501) Updating files: 42% (1471/3501) Updating files: 43% (1506/3501) Updating files: 44% (1541/3501) Updating files: 45% (1576/3501) Updating files: 46% (1611/3501) Updating files: 47% (1646/3501) Updating files: 48% (1681/3501) Updating files: 49% (1716/3501) Updating files: 50% (1751/3501) Updating files: 51% (1786/3501) Updating files: 52% (1821/3501) Updating files: 53% (1856/3501) Updating files: 54% (1891/3501) Updating files: 55% (1926/3501) Updating files: 56% (1961/3501) Updating files: 57% (1996/3501) Updating files: 58% (2031/3501) Updating files: 58% (2047/3501) Updating files: 59% (2066/3501) Updating files: 60% (2101/3501) Updating files: 61% (2136/3501) Updating files: 62% (2171/3501) Updating files: 63% (2206/3501) Updating files: 64% (2241/3501) Updating files: 65% (2276/3501) Updating files: 66% (2311/3501) Updating files: 67% (2346/3501) Updating files: 68% (2381/3501) Updating files: 69% (2416/3501) Updating files: 70% (2451/3501) Updating files: 71% (2486/3501) Updating files: 72% (2521/3501) Updating files: 73% (2556/3501) Updating files: 74% (2591/3501) Updating files: 75% (2626/3501) Updating files: 76% (2661/3501) Updating files: 77% (2696/3501) Updating files: 78% (2731/3501) Updating files: 79% (2766/3501) Updating files: 80% (2801/3501) Updating files: 81% (2836/3501) Updating files: 82% (2871/3501) Updating files: 82% (2875/3501) Updating files: 83% (2906/3501) Updating files: 84% (2941/3501) Updating files: 85% (2976/3501) Updating files: 86% (3011/3501) Updating files: 87% (3046/3501) Updating files: 88% (3081/3501) Updating files: 89% (3116/3501) Updating files: 90% (3151/3501) Updating files: 91% (3186/3501) Updating files: 92% (3221/3501) Updating files: 93% (3256/3501) Updating files: 94% (3291/3501) Updating files: 95% (3326/3501) Updating files: 96% (3361/3501) Updating files: 97% (3396/3501) Updating files: 98% (3431/3501) Updating files: 99% (3466/3501) Updating files: 100% (3501/3501) Updating files: 100% (3501/3501), done. 2024-09-10T20:55:17.304 INFO:teuthology.orchestra.run.smithi016.stdout:HEAD is now at ce299c0f Merge branch 'rt1320' into 'main' 2024-09-10T20:55:17.327 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T20:55:17.328 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T20:55:17.478 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T20:55:17.478 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-10T20:55:17.481 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi016.front.sepia.ceph.com to complete... 2024-09-10T20:55:18.337 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-10T20:55:18.342 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-10T20:55:18.442 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T20:55:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10 20:55:18 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-10T20:55:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:FINISHED --2024-09-10 20:55:18-- 2024-09-10T20:55:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:Total wall clock time: 1.0s 2024-09-10T20:55:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:Downloaded: 1 files, 43M in 0.8s (51.2 MB/s) 2024-09-10T20:55:18.486 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T20:55:18.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo dpkg -i /tmp/linux-image.deb 2024-09-10T20:55:18.765 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-10T20:55:20.811 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-10T20:55:20.815 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-10T20:55:20.966 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T20:55:22.353 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T20:55:22.463 INFO:teuthology.orchestra.run.smithi052.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:25.166 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T20:55:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:Sourcing file `/etc/default/grub' 2024-09-10T20:55:44.492 INFO:teuthology.orchestra.run.smithi052.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T20:55:44.516 INFO:teuthology.orchestra.run.smithi052.stderr:Generating grub configuration file ... 2024-09-10T20:55:44.850 INFO:teuthology.orchestra.run.smithi052.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:44.879 INFO:teuthology.orchestra.run.smithi052.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:45.214 INFO:teuthology.orchestra.run.smithi052.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T20:55:45.221 INFO:teuthology.orchestra.run.smithi052.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T20:55:45.291 INFO:teuthology.orchestra.run.smithi052.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T20:55:45.297 INFO:teuthology.orchestra.run.smithi052.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T20:55:45.451 INFO:teuthology.orchestra.run.smithi052.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T20:55:45.452 INFO:teuthology.orchestra.run.smithi052.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T20:55:45.452 INFO:teuthology.orchestra.run.smithi052.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T20:55:45.486 INFO:teuthology.orchestra.run.smithi052.stderr:done 2024-09-10T20:55:45.645 DEBUG:teuthology.orchestra.run.smithi052:> dpkg-deb -c /tmp/linux-image.deb 2024-09-10T20:55:45.653 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-10T20:55:45.653 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-10T20:55:45.653 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:45.673 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:45.673 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:45.697 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-10T20:55:45.697 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-10T20:55:45.697 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-10T20:55:45.697 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-10T20:55:45.697 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-10T20:55:45.697 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-10T20:55:45.697 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-10T20:55:45.697 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-10T20:55:45.697 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T20:55:45.698 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-10T20:55:45.698 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-10T20:55:45.698 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-10T20:55:45.698 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-10T20:55:45.698 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-10T20:55:45.698 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T20:55:45.698 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T20:55:45.698 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T20:55:45.698 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T20:55:45.698 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T20:55:45.698 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T20:55:45.698 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T20:55:45.699 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T20:55:45.699 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T20:55:45.699 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T20:55:45.699 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T20:55:45.699 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T20:55:45.699 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T20:55:45.699 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T20:55:45.699 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T20:55:45.699 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T20:55:45.699 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T20:55:45.699 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T20:55:45.699 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T20:55:45.700 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-10T20:55:45.700 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-10T20:55:45.700 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-10T20:55:45.700 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T20:55:45.700 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-10T20:55:45.700 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-10T20:55:45.700 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-10T20:55:45.700 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T20:55:45.701 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T20:55:45.701 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T20:55:45.706 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-10T20:55:45.706 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-10T20:55:45.706 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-10T20:55:45.716 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-10T20:55:45.716 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T20:55:45.716 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T20:55:45.716 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T20:55:45.716 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-10T20:55:45.716 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-10T20:55:45.716 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-10T20:55:45.716 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T20:55:45.716 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T20:55:45.716 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T20:55:45.716 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T20:55:45.717 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T20:55:45.717 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T20:55:45.717 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T20:55:45.717 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T20:55:45.717 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T20:55:45.717 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T20:55:45.717 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T20:55:45.717 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T20:55:45.717 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T20:55:45.717 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T20:55:45.717 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T20:55:45.718 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T20:55:45.718 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T20:55:45.718 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T20:55:45.718 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-10T20:55:45.718 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T20:55:45.718 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-10T20:55:45.718 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T20:55:45.718 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T20:55:45.718 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T20:55:45.718 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T20:55:45.718 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T20:55:45.718 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T20:55:45.719 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T20:55:45.719 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T20:55:45.719 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-10T20:55:45.719 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T20:55:45.719 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T20:55:45.719 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T20:55:45.719 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T20:55:45.719 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T20:55:45.719 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T20:55:45.719 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T20:55:45.719 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T20:55:45.719 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T20:55:45.719 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T20:55:45.720 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T20:55:45.720 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T20:55:45.720 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T20:55:45.720 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T20:55:45.720 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T20:55:45.720 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T20:55:45.720 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T20:55:45.720 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-10T20:55:45.720 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T20:55:45.720 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-10T20:55:45.720 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-10T20:55:45.720 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-10T20:55:45.720 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T20:55:45.721 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T20:55:45.721 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-10T20:55:45.721 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-10T20:55:45.721 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T20:55:45.721 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T20:55:45.721 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T20:55:45.721 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T20:55:45.721 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T20:55:45.721 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-10T20:55:45.721 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T20:55:45.721 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-10T20:55:45.722 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T20:55:45.722 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-10T20:55:45.722 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T20:55:45.722 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T20:55:45.722 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T20:55:45.722 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T20:55:45.722 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T20:55:45.722 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T20:55:45.722 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T20:55:45.722 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T20:55:45.722 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T20:55:45.722 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T20:55:45.722 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T20:55:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T20:55:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T20:55:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T20:55:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T20:55:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T20:55:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T20:55:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T20:55:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-10T20:55:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T20:55:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T20:55:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T20:55:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T20:55:45.723 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T20:55:45.724 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T20:55:45.724 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T20:55:45.724 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T20:55:45.724 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T20:55:45.724 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T20:55:45.724 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T20:55:45.724 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T20:55:45.724 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T20:55:45.724 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-10T20:55:45.724 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T20:55:45.724 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-10T20:55:45.724 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T20:55:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T20:55:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T20:55:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T20:55:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T20:55:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T20:55:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T20:55:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T20:55:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T20:55:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T20:55:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T20:55:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T20:55:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T20:55:45.725 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T20:55:45.726 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T20:55:45.726 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T20:55:45.726 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T20:55:45.726 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T20:55:45.726 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T20:55:45.726 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T20:55:45.726 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-10T20:55:45.726 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T20:55:45.726 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-10T20:55:45.726 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T20:55:45.726 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-10T20:55:45.726 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-10T20:55:45.726 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T20:55:45.727 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-10T20:55:45.727 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T20:55:45.765 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T20:55:45.765 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-10T20:55:45.765 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T20:55:45.765 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T20:55:45.766 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-10T20:55:45.766 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T20:55:45.766 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T20:55:45.766 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T20:55:45.766 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-10T20:55:45.766 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T20:55:45.766 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-10T20:55:45.766 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T20:55:45.766 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-10T20:55:45.766 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-10T20:55:45.766 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T20:55:45.766 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-10T20:55:45.767 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T20:55:45.767 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T20:55:45.767 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T20:55:45.767 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T20:55:45.767 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T20:55:45.767 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-10T20:55:45.767 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T20:55:45.767 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T20:55:45.767 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T20:55:45.767 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T20:55:45.767 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T20:55:45.767 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T20:55:45.767 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T20:55:45.768 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-10T20:55:45.768 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T20:55:45.768 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-10T20:55:45.768 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T20:55:45.768 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T20:55:45.768 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-10T20:55:45.768 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T20:55:45.768 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-10T20:55:45.768 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-10T20:55:45.768 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-10T20:55:45.768 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T20:55:45.768 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T20:55:45.768 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-10T20:55:45.769 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-10T20:55:45.769 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-10T20:55:45.769 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-10T20:55:45.769 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-10T20:55:45.769 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-10T20:55:45.769 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-10T20:55:45.769 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-10T20:55:45.769 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-10T20:55:45.769 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-10T20:55:45.769 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-10T20:55:45.769 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T20:55:45.769 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T20:55:45.770 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T20:55:45.770 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-10T20:55:45.770 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-10T20:55:45.770 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T20:55:45.770 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T20:55:45.770 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-10T20:55:45.770 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-10T20:55:45.770 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-10T20:55:45.770 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T20:55:45.770 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-10T20:55:45.770 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T20:55:45.770 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T20:55:45.771 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T20:55:45.771 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-10T20:55:45.771 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T20:55:45.771 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-10T20:55:45.771 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T20:55:45.771 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-10T20:55:45.771 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-10T20:55:45.771 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T20:55:45.771 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-10T20:55:45.771 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T20:55:45.771 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-10T20:55:45.771 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T20:55:45.771 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T20:55:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T20:55:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T20:55:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T20:55:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T20:55:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T20:55:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T20:55:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T20:55:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T20:55:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T20:55:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T20:55:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T20:55:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-10T20:55:45.772 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-10T20:55:45.773 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-10T20:55:45.773 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-10T20:55:45.773 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-10T20:55:45.773 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-10T20:55:45.773 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-10T20:55:45.773 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-10T20:55:45.773 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-10T20:55:45.773 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-10T20:55:45.773 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-10T20:55:45.773 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-10T20:55:45.773 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T20:55:45.773 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-10T20:55:45.773 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-10T20:55:45.774 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-10T20:55:45.774 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-10T20:55:45.774 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-10T20:55:45.774 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-10T20:55:45.774 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-10T20:55:45.774 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-10T20:55:45.774 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-10T20:55:45.774 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T20:55:45.774 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-10T20:55:45.774 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-10T20:55:45.774 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-10T20:55:45.774 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-10T20:55:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-10T20:55:45.776 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-10T20:55:45.776 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-10T20:55:45.776 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-10T20:55:45.776 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-10T20:55:45.776 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-10T20:55:45.776 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T20:55:45.776 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T20:55:45.776 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T20:55:45.776 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-10T20:55:45.776 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-10T20:55:45.776 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-10T20:55:45.776 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-10T20:55:45.777 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-10T20:55:45.777 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T20:55:45.777 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T20:55:45.777 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T20:55:45.777 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T20:55:45.777 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-10T20:55:45.777 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T20:55:45.777 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-10T20:55:45.777 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T20:55:45.777 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T20:55:45.777 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T20:55:45.777 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T20:55:45.777 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T20:55:45.778 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T20:55:45.778 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T20:55:45.778 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-10T20:55:45.778 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T20:55:45.778 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T20:55:45.778 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T20:55:45.778 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T20:55:45.778 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T20:55:45.778 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-10T20:55:45.778 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T20:55:45.778 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T20:55:45.778 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-10T20:55:45.778 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-10T20:55:45.779 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-10T20:55:45.779 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-10T20:55:45.779 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T20:55:45.779 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-10T20:55:45.779 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T20:55:45.779 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T20:55:45.779 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T20:55:45.779 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T20:55:45.779 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T20:55:45.779 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T20:55:45.779 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T20:55:45.779 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T20:55:45.779 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T20:55:45.780 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T20:55:45.780 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T20:55:45.780 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T20:55:45.780 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T20:55:45.780 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T20:55:45.780 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T20:55:45.780 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T20:55:45.780 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T20:55:45.780 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T20:55:45.780 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T20:55:45.780 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T20:55:45.780 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T20:55:45.781 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T20:55:45.781 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T20:55:45.781 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T20:55:45.781 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T20:55:45.781 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T20:55:45.781 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T20:55:45.781 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T20:55:45.781 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T20:55:45.781 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T20:55:45.781 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T20:55:45.781 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T20:55:45.781 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T20:55:45.782 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T20:55:45.782 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T20:55:45.782 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T20:55:45.782 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T20:55:45.782 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T20:55:45.782 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T20:55:45.782 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-10T20:55:45.782 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T20:55:45.782 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-10T20:55:45.782 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-10T20:55:45.782 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-10T20:55:45.782 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-10T20:55:45.783 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T20:55:45.783 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-10T20:55:45.783 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T20:55:45.783 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T20:55:45.783 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T20:55:45.783 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-10T20:55:45.783 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T20:55:45.783 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T20:55:45.783 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T20:55:45.783 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T20:55:45.783 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T20:55:45.783 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T20:55:45.783 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T20:55:45.784 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T20:55:45.784 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T20:55:45.784 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T20:55:45.784 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T20:55:45.784 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T20:55:45.784 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T20:55:45.784 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T20:55:45.784 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T20:55:45.784 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T20:55:45.784 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T20:55:45.784 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T20:55:45.784 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T20:55:45.785 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T20:55:45.785 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T20:55:45.785 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T20:55:45.785 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T20:55:45.785 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T20:55:45.785 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T20:55:45.785 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-10T20:55:45.785 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-10T20:55:45.785 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-10T20:55:45.785 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T20:55:45.785 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T20:55:45.785 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T20:55:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T20:55:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T20:55:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-10T20:55:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T20:55:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T20:55:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T20:55:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T20:55:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T20:55:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T20:55:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T20:55:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T20:55:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T20:55:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T20:55:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-10T20:55:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T20:55:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T20:55:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T20:55:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T20:55:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T20:55:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T20:55:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T20:55:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T20:55:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-10T20:55:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T20:55:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T20:55:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T20:55:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T20:55:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T20:55:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T20:55:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T20:55:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T20:55:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T20:55:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T20:55:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T20:55:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T20:55:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T20:55:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T20:55:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T20:55:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T20:55:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T20:55:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T20:55:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T20:55:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T20:55:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T20:55:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T20:55:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T20:55:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T20:55:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T20:55:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T20:55:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T20:55:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T20:55:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T20:55:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T20:55:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T20:55:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T20:55:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-10T20:55:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T20:55:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-10T20:55:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T20:55:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T20:55:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T20:55:45.791 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T20:55:45.791 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T20:55:45.791 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T20:55:45.791 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T20:55:45.791 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-10T20:55:45.791 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T20:55:45.791 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T20:55:45.791 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T20:55:45.791 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T20:55:45.791 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T20:55:45.791 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T20:55:45.792 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T20:55:45.792 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T20:55:45.792 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T20:55:45.792 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T20:55:45.792 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T20:55:45.792 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T20:55:45.792 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T20:55:45.792 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T20:55:45.792 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T20:55:45.792 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T20:55:45.792 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T20:55:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T20:55:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T20:55:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T20:55:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T20:55:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T20:55:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T20:55:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T20:55:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T20:55:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T20:55:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T20:55:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T20:55:45.793 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T20:55:45.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T20:55:45.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T20:55:45.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T20:55:45.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T20:55:45.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T20:55:45.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T20:55:45.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T20:55:45.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T20:55:45.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T20:55:45.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T20:55:45.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T20:55:45.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T20:55:45.794 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-10T20:55:45.795 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T20:55:45.796 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-10T20:55:45.797 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T20:55:45.797 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T20:55:45.797 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T20:55:45.797 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-10T20:55:45.797 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T20:55:45.797 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T20:55:45.797 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-10T20:55:45.797 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-10T20:55:45.797 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T20:55:45.797 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-10T20:55:45.797 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T20:55:45.797 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T20:55:45.798 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T20:55:45.798 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T20:55:45.798 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-10T20:55:45.798 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T20:55:45.798 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T20:55:45.798 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T20:55:45.798 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-10T20:55:45.798 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-10T20:55:45.798 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-10T20:55:45.798 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-10T20:55:45.798 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-10T20:55:45.799 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-10T20:55:45.799 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T20:55:45.799 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T20:55:45.799 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T20:55:45.799 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T20:55:45.799 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T20:55:45.799 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T20:55:45.799 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T20:55:45.799 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-10T20:55:45.799 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-10T20:55:45.799 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T20:55:45.799 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-10T20:55:45.799 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T20:55:45.800 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-10T20:55:45.800 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T20:55:45.832 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-10T20:55:45.833 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T20:55:45.833 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-10T20:55:45.833 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-10T20:55:45.833 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-10T20:55:45.833 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-10T20:55:45.833 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-10T20:55:45.833 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-10T20:55:45.833 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T20:55:45.833 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-10T20:55:45.833 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T20:55:45.833 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-10T20:55:45.834 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T20:55:45.834 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-10T20:55:45.834 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T20:55:45.834 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-10T20:55:45.834 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T20:55:45.834 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-10T20:55:45.834 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-10T20:55:45.834 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-10T20:55:45.834 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-10T20:55:45.834 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T20:55:45.834 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T20:55:45.834 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T20:55:45.834 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T20:55:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-10T20:55:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-10T20:55:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-10T20:55:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-10T20:55:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-10T20:55:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T20:55:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-10T20:55:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-10T20:55:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-10T20:55:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-10T20:55:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-10T20:55:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-10T20:55:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-10T20:55:45.836 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-10T20:55:45.836 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-10T20:55:45.836 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-10T20:55:45.836 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-10T20:55:45.836 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-10T20:55:45.836 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-10T20:55:45.836 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-10T20:55:45.836 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-10T20:55:45.836 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-10T20:55:45.836 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-10T20:55:45.836 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-10T20:55:45.837 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-10T20:55:45.837 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-10T20:55:45.837 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-10T20:55:45.837 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T20:55:45.837 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T20:55:45.837 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T20:55:45.837 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T20:55:45.837 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-10T20:55:45.837 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-10T20:55:45.837 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T20:55:45.837 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T20:55:45.837 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T20:55:45.838 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T20:55:45.838 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-10T20:55:45.838 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T20:55:45.838 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-10T20:55:45.838 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T20:55:45.838 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T20:55:45.838 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-10T20:55:45.838 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T20:55:45.838 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-10T20:55:45.838 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-10T20:55:45.838 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-10T20:55:45.838 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T20:55:45.839 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-10T20:55:45.839 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T20:55:45.839 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-10T20:55:45.839 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-10T20:55:45.839 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-10T20:55:45.839 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-10T20:55:45.839 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T20:55:45.839 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-10T20:55:45.839 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-10T20:55:45.839 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-10T20:55:45.839 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T20:55:45.839 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-10T20:55:45.839 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-10T20:55:45.840 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-10T20:55:45.840 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T20:55:45.840 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-10T20:55:45.840 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-10T20:55:45.840 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-10T20:55:45.840 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-10T20:55:45.840 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-10T20:55:45.840 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-10T20:55:45.840 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-10T20:55:45.840 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T20:55:45.840 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-10T20:55:45.840 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-10T20:55:45.841 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-10T20:55:45.841 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-10T20:55:45.841 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T20:55:45.841 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T20:55:45.841 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T20:55:45.841 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T20:55:45.841 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T20:55:45.841 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T20:55:45.841 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T20:55:45.841 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T20:55:45.841 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-10T20:55:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-10T20:55:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T20:55:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T20:55:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-10T20:55:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T20:55:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T20:55:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T20:55:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T20:55:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-10T20:55:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T20:55:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T20:55:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-10T20:55:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T20:55:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T20:55:45.844 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T20:55:45.844 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T20:55:45.844 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T20:55:45.844 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-10T20:55:45.844 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-10T20:55:45.844 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T20:55:45.844 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T20:55:45.844 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T20:55:45.844 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T20:55:45.844 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T20:55:45.844 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T20:55:45.844 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T20:55:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-10T20:55:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-10T20:55:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-10T20:55:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-10T20:55:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-10T20:55:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-10T20:55:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-10T20:55:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-10T20:55:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-10T20:55:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-10T20:55:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-10T20:55:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-10T20:55:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-10T20:55:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T20:55:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-10T20:55:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-10T20:55:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T20:55:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T20:55:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T20:55:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-10T20:55:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-10T20:55:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-10T20:55:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-10T20:55:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-10T20:55:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T20:55:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T20:55:45.847 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T20:55:45.848 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T20:55:45.848 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-10T20:55:45.848 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-10T20:55:45.848 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-10T20:55:45.848 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-10T20:55:45.848 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T20:55:45.848 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-10T20:55:45.848 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-10T20:55:45.848 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-10T20:55:45.848 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-10T20:55:45.848 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-10T20:55:45.848 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T20:55:45.849 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T20:55:45.849 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T20:55:45.849 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T20:55:45.849 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-10T20:55:45.849 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-10T20:55:45.849 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-10T20:55:45.849 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T20:55:45.849 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-10T20:55:45.849 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-10T20:55:45.849 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-10T20:55:45.849 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T20:55:45.849 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T20:55:45.849 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-10T20:55:45.850 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T20:55:45.850 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T20:55:45.850 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-10T20:55:45.850 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T20:55:45.850 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T20:55:45.850 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T20:55:45.850 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-10T20:55:45.850 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-10T20:55:45.850 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-10T20:55:45.850 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T20:55:45.850 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T20:55:45.850 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T20:55:45.851 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T20:55:45.852 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T20:55:45.852 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-10T20:55:45.852 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-10T20:55:45.852 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T20:55:45.852 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T20:55:45.852 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T20:55:45.852 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T20:55:45.852 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-10T20:55:45.852 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T20:55:45.852 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T20:55:45.852 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T20:55:45.852 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T20:55:45.852 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-10T20:55:45.853 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T20:55:45.853 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-10T20:55:45.853 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T20:55:45.853 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-10T20:55:45.853 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T20:55:45.853 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-10T20:55:45.853 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T20:55:45.853 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-10T20:55:45.853 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T20:55:45.853 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T20:55:45.853 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T20:55:45.853 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-10T20:55:45.854 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-10T20:55:45.854 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T20:55:45.854 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-10T20:55:45.854 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T20:55:45.854 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-10T20:55:45.854 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T20:55:45.854 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-10T20:55:45.854 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T20:55:45.854 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T20:55:45.854 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-10T20:55:45.854 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T20:55:45.854 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T20:55:45.855 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-10T20:55:45.855 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T20:55:45.855 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T20:55:45.855 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-10T20:55:45.855 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T20:55:45.855 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T20:55:45.855 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-10T20:55:45.855 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-10T20:55:45.855 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T20:55:45.855 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-10T20:55:45.855 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-10T20:55:45.855 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T20:55:45.856 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-10T20:55:45.856 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T20:55:45.856 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-10T20:55:45.856 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T20:55:45.856 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-10T20:55:45.856 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T20:55:45.856 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-10T20:55:45.856 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T20:55:45.856 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-10T20:55:45.856 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-10T20:55:45.856 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T20:55:45.856 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-10T20:55:45.857 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-10T20:55:45.857 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T20:55:45.857 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T20:55:45.857 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T20:55:45.857 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T20:55:45.857 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T20:55:45.857 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T20:55:45.857 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-10T20:55:45.857 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T20:55:45.857 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T20:55:45.857 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T20:55:45.857 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T20:55:45.858 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T20:55:45.859 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-10T20:55:45.860 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T20:55:45.860 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-10T20:55:45.860 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T20:55:45.860 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T20:55:45.860 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-10T20:55:45.860 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T20:55:45.860 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T20:55:45.860 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T20:55:45.860 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-10T20:55:45.861 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-10T20:55:45.861 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T20:55:45.861 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T20:55:45.861 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-10T20:55:45.861 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-10T20:55:45.861 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T20:55:45.861 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-10T20:55:45.861 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-10T20:55:45.861 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-10T20:55:45.862 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-10T20:55:45.862 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T20:55:45.862 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-10T20:55:45.862 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-10T20:55:45.862 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T20:55:45.862 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-10T20:55:45.862 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T20:55:45.862 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T20:55:45.862 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-10T20:55:45.862 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T20:55:45.862 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-10T20:55:45.862 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T20:55:45.862 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-10T20:55:45.863 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T20:55:45.863 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T20:55:45.863 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-10T20:55:45.863 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-10T20:55:45.863 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T20:55:45.863 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T20:55:45.863 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-10T20:55:45.863 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T20:55:45.940 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-10T20:55:45.940 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T20:55:45.940 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-10T20:55:45.940 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T20:55:45.941 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T20:55:45.941 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T20:55:45.941 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T20:55:45.941 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-10T20:55:45.941 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-10T20:55:45.941 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-10T20:55:45.941 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-10T20:55:45.941 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T20:55:45.941 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-10T20:55:45.941 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T20:55:45.942 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-10T20:55:45.942 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T20:55:45.942 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T20:55:45.942 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-10T20:55:45.942 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T20:55:45.942 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T20:55:45.942 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T20:55:45.942 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T20:55:45.942 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-10T20:55:45.942 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-10T20:55:45.942 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-10T20:55:45.942 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T20:55:45.942 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-10T20:55:45.943 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T20:55:45.943 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-10T20:55:45.943 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T20:55:45.943 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T20:55:45.943 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T20:55:45.943 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T20:55:45.943 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-10T20:55:45.943 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T20:55:45.943 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-10T20:55:45.943 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T20:55:45.943 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-10T20:55:45.943 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T20:55:45.944 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T20:55:45.944 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-10T20:55:45.944 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T20:55:45.944 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T20:55:45.944 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-10T20:55:45.944 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T20:55:45.944 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-10T20:55:45.944 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-10T20:55:45.944 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-10T20:55:45.944 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-10T20:55:45.945 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T20:55:45.945 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-10T20:55:45.945 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T20:55:45.945 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-10T20:55:45.945 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T20:55:45.945 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T20:55:45.945 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-10T20:55:45.945 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T20:55:45.945 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T20:55:45.945 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T20:55:45.945 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-10T20:55:45.945 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T20:55:45.946 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T20:55:45.946 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-10T20:55:45.946 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T20:55:45.946 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T20:55:45.946 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T20:55:45.946 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T20:55:45.946 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T20:55:45.946 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-10T20:55:45.946 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T20:55:45.946 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T20:55:45.946 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-10T20:55:45.946 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T20:55:45.946 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T20:55:45.947 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T20:55:45.947 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T20:55:45.947 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T20:55:45.947 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T20:55:45.947 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T20:55:45.947 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T20:55:45.947 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T20:55:45.947 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T20:55:45.947 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T20:55:45.947 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T20:55:45.947 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T20:55:45.947 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T20:55:45.947 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T20:55:45.948 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T20:55:45.948 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T20:55:45.948 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T20:55:45.948 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T20:55:45.948 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-10T20:55:45.948 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-10T20:55:45.948 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-10T20:55:45.948 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T20:55:45.948 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T20:55:45.948 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T20:55:45.948 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T20:55:45.948 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T20:55:45.948 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T20:55:45.950 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-10T20:55:45.951 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T20:55:45.951 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T20:55:45.951 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T20:55:45.951 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T20:55:45.951 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-10T20:55:45.951 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-10T20:55:45.951 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T20:55:45.951 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-10T20:55:45.951 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-10T20:55:45.951 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-10T20:55:45.951 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T20:55:45.951 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-10T20:55:45.951 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T20:55:45.952 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-10T20:55:45.952 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-10T20:55:45.952 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-10T20:55:45.952 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T20:55:45.952 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T20:55:45.952 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-10T20:55:45.952 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-10T20:55:45.952 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-10T20:55:45.952 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-10T20:55:45.952 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-10T20:55:45.952 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-10T20:55:45.952 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-10T20:55:45.952 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T20:55:45.953 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-10T20:55:45.953 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-10T20:55:45.953 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-10T20:55:45.953 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-10T20:55:45.953 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-10T20:55:45.953 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-10T20:55:45.953 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-10T20:55:45.953 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-10T20:55:45.954 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-10T20:55:45.954 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-10T20:55:45.954 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T20:55:45.954 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T20:55:45.954 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T20:55:45.954 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T20:55:45.954 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T20:55:45.954 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T20:55:45.955 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T20:55:45.955 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T20:55:45.955 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T20:55:45.955 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T20:55:45.955 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T20:55:45.955 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T20:55:45.955 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-10T20:55:45.955 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-10T20:55:45.955 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T20:55:45.955 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-10T20:55:45.955 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T20:55:45.956 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T20:55:45.956 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T20:55:45.956 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T20:55:45.956 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T20:55:45.956 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T20:55:45.956 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T20:55:45.956 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T20:55:45.956 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T20:55:45.956 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T20:55:45.956 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-10T20:55:45.959 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-10T20:55:45.959 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-10T20:55:45.959 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-10T20:55:45.959 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-10T20:55:45.959 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-10T20:55:45.959 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-10T20:55:45.959 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T20:55:45.959 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T20:55:45.959 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-10T20:55:45.959 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-10T20:55:45.959 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T20:55:45.959 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T20:55:45.959 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T20:55:45.960 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T20:55:45.960 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-10T20:55:45.960 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T20:55:45.960 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T20:55:45.960 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-10T20:55:45.960 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-10T20:55:45.960 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-10T20:55:45.960 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T20:55:45.960 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T20:55:45.960 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-10T20:55:45.960 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-10T20:55:45.960 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-10T20:55:45.961 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T20:55:45.961 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-10T20:55:45.961 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-10T20:55:45.961 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-10T20:55:45.961 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T20:55:45.961 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T20:55:45.961 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T20:55:45.961 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T20:55:45.961 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T20:55:45.961 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T20:55:45.961 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T20:55:45.961 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T20:55:45.962 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T20:55:45.962 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T20:55:45.962 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T20:55:45.962 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T20:55:45.962 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T20:55:45.962 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-10T20:55:45.962 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T20:55:45.962 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T20:55:45.962 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T20:55:45.962 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-10T20:55:45.962 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T20:55:45.962 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-10T20:55:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-10T20:55:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-10T20:55:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-10T20:55:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-10T20:55:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T20:55:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-10T20:55:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T20:55:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-10T20:55:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T20:55:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T20:55:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T20:55:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T20:55:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-10T20:55:45.964 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T20:55:45.964 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T20:55:45.964 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T20:55:45.964 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T20:55:45.964 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T20:55:45.964 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-10T20:55:45.964 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T20:55:45.964 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-10T20:55:45.964 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-10T20:55:45.964 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-10T20:55:45.964 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-10T20:55:45.964 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T20:55:45.965 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-10T20:55:45.965 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T20:55:45.965 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T20:55:45.965 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T20:55:45.965 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T20:55:45.965 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T20:55:45.965 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-10T20:55:45.965 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T20:55:45.965 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T20:55:45.966 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-10T20:55:45.966 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T20:55:45.966 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T20:55:45.966 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-10T20:55:45.966 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-10T20:55:45.966 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T20:55:45.966 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-10T20:55:45.966 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T20:55:45.966 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T20:55:45.966 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-10T20:55:45.966 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T20:55:45.966 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-10T20:55:45.967 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T20:55:45.967 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-10T20:55:45.967 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T20:55:45.967 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-10T20:55:45.967 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T20:55:45.967 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T20:55:45.967 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-10T20:55:45.967 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T20:55:45.967 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-10T20:55:45.967 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-10T20:55:45.967 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T20:55:45.967 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-10T20:55:45.967 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T20:55:45.968 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T20:55:45.968 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T20:55:45.968 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T20:55:45.968 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-10T20:55:45.968 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T20:55:45.968 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T20:55:45.968 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-10T20:55:45.968 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T20:55:45.968 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T20:55:45.968 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-10T20:55:45.968 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-10T20:55:45.968 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T20:55:45.969 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T20:55:45.969 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T20:55:45.969 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T20:55:45.969 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-10T20:55:45.969 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T20:55:45.969 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T20:55:45.969 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T20:55:45.969 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-10T20:55:45.969 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T20:55:45.969 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T20:55:45.969 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T20:55:45.969 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-10T20:55:45.970 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T20:55:45.971 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-10T20:55:45.971 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T20:55:45.971 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T20:55:45.971 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-10T20:55:45.971 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-10T20:55:45.971 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-10T20:55:45.971 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-10T20:55:45.971 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-10T20:55:45.971 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-10T20:55:45.971 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T20:55:45.971 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T20:55:45.971 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-10T20:55:45.972 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T20:55:45.972 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-10T20:55:45.972 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T20:55:45.972 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T20:55:45.972 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-10T20:55:45.972 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T20:55:45.972 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-10T20:55:45.972 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T20:55:45.972 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T20:55:45.972 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T20:55:45.972 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T20:55:45.972 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T20:55:45.973 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T20:55:45.973 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T20:55:45.973 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T20:55:45.973 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T20:55:45.973 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T20:55:45.973 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-10T20:55:45.973 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T20:55:45.973 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T20:55:45.973 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T20:55:45.973 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T20:55:45.973 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-10T20:55:45.973 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-10T20:55:45.973 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T20:55:45.974 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-10T20:55:45.974 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-10T20:55:45.974 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T20:55:45.974 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-10T20:55:45.974 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T20:55:45.974 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T20:55:45.974 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T20:55:45.974 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T20:55:45.974 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T20:55:45.974 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T20:55:45.974 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-10T20:55:45.974 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T20:55:45.974 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-10T20:55:45.975 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-10T20:55:45.975 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-10T20:55:45.975 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T20:55:45.975 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T20:55:45.975 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T20:55:45.975 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T20:55:45.975 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T20:55:45.975 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T20:55:45.975 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T20:55:45.975 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T20:55:45.975 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T20:55:45.976 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T20:55:45.976 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T20:55:45.976 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T20:55:45.976 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T20:55:45.976 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T20:55:45.976 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T20:55:45.976 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-10T20:55:45.976 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T20:55:45.976 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T20:55:45.976 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T20:55:45.976 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-10T20:55:45.976 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-10T20:55:45.976 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T20:55:45.977 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-10T20:55:45.977 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-10T20:55:45.977 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T20:55:45.977 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T20:55:45.977 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T20:55:45.977 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-10T20:55:45.977 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-10T20:55:45.977 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-10T20:55:45.977 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T20:55:45.977 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-10T20:55:45.977 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-10T20:55:45.977 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T20:55:45.977 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-10T20:55:45.978 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-10T20:55:45.978 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T20:55:45.978 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-10T20:55:45.978 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-10T20:55:45.978 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-10T20:55:45.978 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-10T20:55:45.978 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-10T20:55:45.978 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T20:55:45.978 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-10T20:55:45.978 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T20:55:45.978 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-10T20:55:45.978 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-10T20:55:45.978 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T20:55:45.979 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T20:55:45.979 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T20:55:45.979 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-10T20:55:45.979 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-10T20:55:45.979 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-10T20:55:45.979 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T20:55:45.979 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-10T20:55:45.979 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T20:55:45.979 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-10T20:55:45.979 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-10T20:55:45.979 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-10T20:55:45.980 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-10T20:55:45.980 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-10T20:55:45.980 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-10T20:55:45.980 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-10T20:55:45.980 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-10T20:55:45.980 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-10T20:55:45.980 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-10T20:55:45.980 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T20:55:45.980 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T20:55:45.980 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T20:55:45.980 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T20:55:45.980 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-10T20:55:45.981 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-10T20:55:45.981 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-10T20:55:45.981 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-10T20:55:45.981 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-10T20:55:45.981 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-10T20:55:45.981 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-10T20:55:45.981 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-10T20:55:45.981 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-10T20:55:45.981 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-10T20:55:45.981 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-10T20:55:45.981 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-10T20:55:45.981 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-10T20:55:45.982 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-10T20:55:45.983 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-10T20:55:45.984 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-10T20:55:45.984 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-10T20:55:45.984 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-10T20:55:45.984 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-10T20:55:45.984 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-10T20:55:45.984 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-10T20:55:45.984 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-10T20:55:45.984 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-10T20:55:45.984 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-10T20:55:45.984 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-10T20:55:45.984 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-10T20:55:45.984 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-10T20:55:45.985 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-10T20:55:45.985 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-10T20:55:45.985 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-10T20:55:45.985 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-10T20:55:45.985 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-10T20:55:45.985 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-10T20:55:45.985 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-10T20:55:45.985 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-10T20:55:45.985 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-10T20:55:45.985 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-10T20:55:45.985 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T20:55:45.986 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-10T20:55:45.986 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-10T20:55:45.986 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-10T20:55:45.986 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-10T20:55:45.986 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T20:55:45.986 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T20:55:45.986 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-10T20:55:45.986 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T20:55:45.986 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T20:55:45.986 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-10T20:55:45.986 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-10T20:55:45.986 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-10T20:55:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T20:55:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T20:55:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T20:55:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T20:55:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T20:55:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T20:55:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T20:55:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T20:55:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-10T20:55:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-10T20:55:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T20:55:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T20:55:45.987 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-10T20:55:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T20:55:45.989 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T20:55:45.990 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T20:55:45.990 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T20:55:45.990 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T20:55:45.990 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T20:55:45.990 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T20:55:45.990 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T20:55:45.990 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T20:55:45.990 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T20:55:45.990 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T20:55:45.990 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T20:55:45.990 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-10T20:55:45.990 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T20:55:45.990 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T20:55:45.991 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-10T20:55:45.991 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T20:55:45.991 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T20:55:45.991 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T20:55:45.991 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T20:55:45.991 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T20:55:45.991 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T20:55:45.991 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T20:55:45.991 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T20:55:45.991 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T20:55:45.991 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T20:55:45.992 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T20:55:45.993 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T20:55:45.994 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T20:55:45.994 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-10T20:55:45.994 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T20:55:45.994 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T20:55:45.994 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T20:55:45.994 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-10T20:55:45.994 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-10T20:55:45.994 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T20:55:45.994 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T20:55:45.994 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-10T20:55:45.995 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T20:55:45.996 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-10T20:55:45.996 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-10T20:55:45.996 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-10T20:55:45.996 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-10T20:55:45.996 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-10T20:55:45.996 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-10T20:55:45.996 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T20:55:45.996 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-10T20:55:45.996 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-10T20:55:45.997 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T20:55:45.998 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T20:55:45.998 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T20:55:45.998 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T20:55:45.998 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T20:55:45.998 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T20:55:45.998 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T20:55:45.998 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T20:55:45.998 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T20:55:45.998 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T20:55:45.998 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T20:55:45.998 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T20:55:45.998 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-10T20:55:45.999 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-10T20:55:45.999 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-10T20:55:45.999 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T20:55:45.999 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T20:55:45.999 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-10T20:55:45.999 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-10T20:55:45.999 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-10T20:55:45.999 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-10T20:55:45.999 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-10T20:55:45.999 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-10T20:55:45.999 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T20:55:45.999 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T20:55:45.999 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T20:55:46.000 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T20:55:46.000 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T20:55:46.000 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T20:55:46.000 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T20:55:46.000 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T20:55:46.000 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T20:55:46.000 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T20:55:46.000 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-10T20:55:46.000 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T20:55:46.000 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-10T20:55:46.001 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-10T20:55:46.001 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T20:55:46.001 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-10T20:55:46.001 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-10T20:55:46.001 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T20:55:46.001 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T20:55:46.001 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-10T20:55:46.001 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-10T20:55:46.001 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-10T20:55:46.001 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T20:55:46.001 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-10T20:55:46.001 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T20:55:46.001 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-10T20:55:46.002 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T20:55:46.002 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T20:55:46.002 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-10T20:55:46.002 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T20:55:46.002 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T20:55:46.002 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T20:55:46.002 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T20:55:46.002 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-10T20:55:46.002 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T20:55:46.002 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-10T20:55:46.002 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-10T20:55:46.002 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T20:55:46.002 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-10T20:55:46.003 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T20:55:46.003 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T20:55:46.003 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T20:55:46.003 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-10T20:55:46.003 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T20:55:46.003 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T20:55:46.003 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-10T20:55:46.003 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-10T20:55:46.003 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T20:55:46.003 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-10T20:55:46.003 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-10T20:55:46.003 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T20:55:46.004 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-10T20:55:46.004 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-10T20:55:46.004 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-10T20:55:46.004 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T20:55:46.004 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-10T20:55:46.004 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-10T20:55:46.004 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-10T20:55:46.004 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-10T20:55:46.004 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-10T20:55:46.004 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T20:55:46.005 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-10T20:55:46.005 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-10T20:55:46.005 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T20:55:46.005 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-10T20:55:46.005 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T20:55:46.005 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-10T20:55:46.005 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T20:55:46.064 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-10T20:55:46.064 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T20:55:46.064 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-10T20:55:46.064 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T20:55:46.065 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-10T20:55:46.065 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-10T20:55:46.065 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-10T20:55:46.065 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T20:55:46.065 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-10T20:55:46.065 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T20:55:46.065 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-10T20:55:46.065 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-10T20:55:46.065 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-10T20:55:46.065 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T20:55:46.065 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-10T20:55:46.065 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T20:55:46.066 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-10T20:55:46.066 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-10T20:55:46.066 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-10T20:55:46.066 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T20:55:46.066 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-10T20:55:46.066 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T20:55:46.066 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-10T20:55:46.066 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T20:55:46.066 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-10T20:55:46.066 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T20:55:46.066 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-10T20:55:46.066 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-10T20:55:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-10T20:55:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T20:55:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-10T20:55:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T20:55:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-10T20:55:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T20:55:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-10T20:55:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T20:55:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-10T20:55:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T20:55:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-10T20:55:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-10T20:55:46.067 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T20:55:46.068 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-10T20:55:46.068 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T20:55:46.068 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-10T20:55:46.068 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T20:55:46.068 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T20:55:46.068 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-10T20:55:46.068 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T20:55:46.068 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T20:55:46.068 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-10T20:55:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T20:55:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T20:55:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-10T20:55:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T20:55:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-10T20:55:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T20:55:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-10T20:55:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T20:55:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T20:55:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T20:55:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T20:55:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T20:55:46.069 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T20:55:46.070 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T20:55:46.070 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T20:55:46.070 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T20:55:46.070 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T20:55:46.070 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T20:55:46.070 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-10T20:55:46.070 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T20:55:46.070 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T20:55:46.070 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T20:55:46.070 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T20:55:46.070 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T20:55:46.070 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T20:55:46.071 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T20:55:46.071 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T20:55:46.071 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T20:55:46.071 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T20:55:46.071 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T20:55:46.071 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T20:55:46.071 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T20:55:46.071 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T20:55:46.071 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T20:55:46.071 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T20:55:46.071 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T20:55:46.071 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T20:55:46.072 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T20:55:46.072 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T20:55:46.072 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T20:55:46.072 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T20:55:46.072 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T20:55:46.072 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T20:55:46.072 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T20:55:46.072 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T20:55:46.072 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T20:55:46.072 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T20:55:46.073 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T20:55:46.073 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T20:55:46.073 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T20:55:46.073 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T20:55:46.073 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T20:55:46.073 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T20:55:46.073 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T20:55:46.073 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T20:55:46.073 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T20:55:46.073 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T20:55:46.073 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T20:55:46.073 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-10T20:55:46.073 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-10T20:55:46.074 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-10T20:55:46.074 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-10T20:55:46.074 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T20:55:46.074 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-10T20:55:46.074 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T20:55:46.074 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-10T20:55:46.074 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T20:55:46.074 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T20:55:46.074 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T20:55:46.074 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T20:55:46.074 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T20:55:46.074 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-10T20:55:46.074 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-10T20:55:46.075 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-10T20:55:46.075 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T20:55:46.075 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-10T20:55:46.075 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T20:55:46.075 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-10T20:55:46.075 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-10T20:55:46.075 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-10T20:55:46.075 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-10T20:55:46.075 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-10T20:55:46.075 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-10T20:55:46.075 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-10T20:55:46.075 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-10T20:55:46.076 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-10T20:55:46.076 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T20:55:46.076 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-10T20:55:46.076 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T20:55:46.076 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-10T20:55:46.076 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-10T20:55:46.076 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T20:55:46.076 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-10T20:55:46.076 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T20:55:46.076 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T20:55:46.076 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-10T20:55:46.076 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T20:55:46.077 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-10T20:55:46.077 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T20:55:46.077 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-10T20:55:46.077 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T20:55:46.077 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-10T20:55:46.077 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T20:55:46.077 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-10T20:55:46.077 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T20:55:46.077 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-10T20:55:46.077 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-10T20:55:46.077 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-10T20:55:46.078 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-10T20:55:46.078 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-10T20:55:46.078 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T20:55:46.078 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-10T20:55:46.078 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T20:55:46.078 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-10T20:55:46.078 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T20:55:46.078 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-10T20:55:46.078 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T20:55:46.078 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T20:55:46.079 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-10T20:55:46.079 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T20:55:46.079 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T20:55:46.079 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-10T20:55:46.079 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T20:55:46.079 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T20:55:46.079 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-10T20:55:46.079 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T20:55:46.079 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-10T20:55:46.079 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-10T20:55:46.079 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-10T20:55:46.079 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-10T20:55:46.080 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-10T20:55:46.080 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T20:55:46.080 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-10T20:55:46.080 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-10T20:55:46.080 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T20:55:46.080 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-10T20:55:46.080 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-10T20:55:46.080 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-10T20:55:46.080 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-10T20:55:46.080 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T20:55:46.081 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T20:55:46.081 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T20:55:46.081 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-10T20:55:46.081 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-10T20:55:46.081 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-10T20:55:46.081 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T20:55:46.081 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-10T20:55:46.081 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-10T20:55:46.081 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T20:55:46.081 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T20:55:46.081 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T20:55:46.082 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T20:55:46.082 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T20:55:46.082 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T20:55:46.082 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T20:55:46.082 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T20:55:46.082 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-10T20:55:46.082 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T20:55:46.082 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T20:55:46.082 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T20:55:46.082 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T20:55:46.083 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T20:55:46.083 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-10T20:55:46.083 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T20:55:46.083 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-10T20:55:46.083 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T20:55:46.083 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T20:55:46.083 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T20:55:46.083 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T20:55:46.084 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-10T20:55:46.084 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T20:55:46.084 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-10T20:55:46.084 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T20:55:46.084 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T20:55:46.084 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-10T20:55:46.084 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T20:55:46.084 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T20:55:46.084 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T20:55:46.085 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T20:55:46.085 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T20:55:46.085 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T20:55:46.085 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T20:55:46.085 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T20:55:46.085 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T20:55:46.085 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T20:55:46.085 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T20:55:46.085 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T20:55:46.085 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T20:55:46.085 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T20:55:46.085 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T20:55:46.086 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T20:55:46.086 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T20:55:46.086 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T20:55:46.086 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T20:55:46.086 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T20:55:46.086 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T20:55:46.086 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T20:55:46.086 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T20:55:46.086 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-10T20:55:46.086 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T20:55:46.087 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-10T20:55:46.087 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T20:55:46.087 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-10T20:55:46.087 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T20:55:46.087 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T20:55:46.087 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T20:55:46.087 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T20:55:46.088 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-10T20:55:46.088 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T20:55:46.088 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T20:55:46.088 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T20:55:46.088 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T20:55:46.088 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T20:55:46.088 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-10T20:55:46.089 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-10T20:55:46.089 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-10T20:55:46.089 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-10T20:55:46.089 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T20:55:46.089 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T20:55:46.089 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T20:55:46.089 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-10T20:55:46.089 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T20:55:46.089 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T20:55:46.089 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T20:55:46.090 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T20:55:46.090 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T20:55:46.090 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T20:55:46.090 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T20:55:46.090 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T20:55:46.090 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T20:55:46.090 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T20:55:46.090 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-10T20:55:46.090 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T20:55:46.091 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T20:55:46.091 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T20:55:46.091 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-10T20:55:46.091 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T20:55:46.091 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-10T20:55:46.091 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T20:55:46.092 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T20:55:46.092 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T20:55:46.092 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-10T20:55:46.092 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T20:55:46.092 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T20:55:46.092 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T20:55:46.092 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T20:55:46.092 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T20:55:46.092 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T20:55:46.092 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T20:55:46.092 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T20:55:46.093 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T20:55:46.093 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-10T20:55:46.093 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T20:55:46.093 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T20:55:46.093 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T20:55:46.093 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T20:55:46.093 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T20:55:46.093 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T20:55:46.093 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T20:55:46.093 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T20:55:46.093 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T20:55:46.093 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T20:55:46.094 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T20:55:46.094 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T20:55:46.094 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T20:55:46.094 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T20:55:46.094 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T20:55:46.094 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T20:55:46.094 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T20:55:46.094 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T20:55:46.094 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T20:55:46.094 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T20:55:46.094 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T20:55:46.094 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T20:55:46.094 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-10T20:55:46.095 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T20:55:46.095 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T20:55:46.095 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T20:55:46.095 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-10T20:55:46.095 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T20:55:46.095 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T20:55:46.095 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T20:55:46.095 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T20:55:46.095 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-10T20:55:46.095 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-10T20:55:46.095 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T20:55:46.095 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-10T20:55:46.095 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T20:55:46.096 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T20:55:46.096 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T20:55:46.096 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T20:55:46.096 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T20:55:46.096 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T20:55:46.096 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T20:55:46.096 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T20:55:46.096 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T20:55:46.096 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T20:55:46.096 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T20:55:46.096 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T20:55:46.096 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T20:55:46.097 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T20:55:46.097 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T20:55:46.097 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T20:55:46.097 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T20:55:46.097 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T20:55:46.097 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T20:55:46.097 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T20:55:46.097 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T20:55:46.097 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T20:55:46.097 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T20:55:46.097 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T20:55:46.097 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T20:55:46.098 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T20:55:46.098 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-10T20:55:46.098 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T20:55:46.098 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-10T20:55:46.098 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T20:55:46.098 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T20:55:46.098 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T20:55:46.098 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T20:55:46.098 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T20:55:46.098 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T20:55:46.098 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T20:55:46.098 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-10T20:55:46.099 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-10T20:55:46.099 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-10T20:55:46.099 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T20:55:46.099 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-10T20:55:46.099 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-10T20:55:46.099 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T20:55:46.099 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-10T20:55:46.099 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T20:55:46.099 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-10T20:55:46.099 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-10T20:55:46.099 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T20:55:46.100 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T20:55:46.100 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T20:55:46.100 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T20:55:46.100 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T20:55:46.100 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T20:55:46.100 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T20:55:46.100 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T20:55:46.100 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T20:55:46.100 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T20:55:46.100 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T20:55:46.101 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T20:55:46.101 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T20:55:46.101 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T20:55:46.101 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T20:55:46.101 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T20:55:46.101 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-10T20:55:46.101 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T20:55:46.101 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T20:55:46.101 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T20:55:46.101 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T20:55:46.101 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T20:55:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T20:55:46.103 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T20:55:46.103 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T20:55:46.103 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T20:55:46.103 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T20:55:46.103 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T20:55:46.103 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T20:55:46.103 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T20:55:46.103 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T20:55:46.103 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T20:55:46.103 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T20:55:46.103 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T20:55:46.103 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T20:55:46.104 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T20:55:46.104 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T20:55:46.104 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T20:55:46.104 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T20:55:46.104 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T20:55:46.104 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T20:55:46.104 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T20:55:46.104 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T20:55:46.104 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T20:55:46.104 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T20:55:46.104 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T20:55:46.104 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-10T20:55:46.105 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T20:55:46.105 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T20:55:46.105 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T20:55:46.105 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T20:55:46.105 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T20:55:46.105 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T20:55:46.105 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T20:55:46.105 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T20:55:46.105 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T20:55:46.105 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T20:55:46.106 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T20:55:46.106 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T20:55:46.106 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T20:55:46.106 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T20:55:46.106 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T20:55:46.106 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-10T20:55:46.106 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T20:55:46.106 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T20:55:46.106 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T20:55:46.106 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T20:55:46.106 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T20:55:46.106 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-10T20:55:46.107 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-10T20:55:46.107 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-10T20:55:46.107 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T20:55:46.107 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T20:55:46.107 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T20:55:46.107 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T20:55:46.107 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T20:55:46.107 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T20:55:46.107 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T20:55:46.107 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T20:55:46.107 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T20:55:46.107 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T20:55:46.108 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T20:55:46.108 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T20:55:46.108 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T20:55:46.108 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T20:55:46.108 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T20:55:46.108 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T20:55:46.108 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T20:55:46.108 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T20:55:46.108 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T20:55:46.108 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T20:55:46.108 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T20:55:46.108 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T20:55:46.109 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T20:55:46.109 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T20:55:46.109 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T20:55:46.109 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T20:55:46.109 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T20:55:46.109 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T20:55:46.109 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T20:55:46.109 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T20:55:46.110 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T20:55:46.110 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T20:55:46.110 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T20:55:46.110 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T20:55:46.110 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T20:55:46.110 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T20:55:46.110 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T20:55:46.110 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T20:55:46.110 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T20:55:46.110 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T20:55:46.110 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T20:55:46.110 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T20:55:46.111 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T20:55:46.111 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T20:55:46.111 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T20:55:46.111 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T20:55:46.111 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T20:55:46.111 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T20:55:46.111 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T20:55:46.111 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T20:55:46.111 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T20:55:46.111 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T20:55:46.111 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T20:55:46.112 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T20:55:46.112 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T20:55:46.112 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T20:55:46.112 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T20:55:46.112 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T20:55:46.112 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T20:55:46.112 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T20:55:46.112 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T20:55:46.112 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T20:55:46.112 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T20:55:46.112 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T20:55:46.113 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T20:55:46.113 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T20:55:46.113 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T20:55:46.113 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T20:55:46.113 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T20:55:46.113 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-10T20:55:46.113 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T20:55:46.113 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T20:55:46.113 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-10T20:55:46.114 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T20:55:46.114 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-10T20:55:46.114 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T20:55:46.114 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-10T20:55:46.114 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T20:55:46.114 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T20:55:46.114 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T20:55:46.114 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T20:55:46.114 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-10T20:55:46.114 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T20:55:46.114 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-10T20:55:46.114 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-10T20:55:46.114 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-10T20:55:46.115 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-10T20:55:46.115 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T20:55:46.115 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T20:55:46.115 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T20:55:46.115 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-10T20:55:46.115 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-10T20:55:46.115 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-10T20:55:46.115 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T20:55:46.115 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T20:55:46.115 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T20:55:46.115 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T20:55:46.115 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T20:55:46.115 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T20:55:46.116 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T20:55:46.116 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T20:55:46.116 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T20:55:46.116 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T20:55:46.116 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T20:55:46.116 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-10T20:55:46.116 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T20:55:46.116 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T20:55:46.116 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T20:55:46.116 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T20:55:46.116 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T20:55:46.116 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T20:55:46.116 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T20:55:46.117 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T20:55:46.117 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T20:55:46.117 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T20:55:46.117 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T20:55:46.117 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T20:55:46.117 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T20:55:46.117 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T20:55:46.117 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-10T20:55:46.118 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T20:55:46.118 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T20:55:46.118 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T20:55:46.118 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T20:55:46.118 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T20:55:46.118 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T20:55:46.119 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T20:55:46.119 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T20:55:46.119 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T20:55:46.119 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T20:55:46.119 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T20:55:46.119 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T20:55:46.119 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T20:55:46.119 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T20:55:46.119 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T20:55:46.120 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T20:55:46.120 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T20:55:46.120 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T20:55:46.120 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-10T20:55:46.120 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T20:55:46.120 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T20:55:46.120 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-10T20:55:46.120 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-10T20:55:46.120 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T20:55:46.120 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T20:55:46.121 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T20:55:46.121 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-10T20:55:46.121 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T20:55:46.121 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-10T20:55:46.121 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T20:55:46.121 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T20:55:46.121 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-10T20:55:46.121 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T20:55:46.121 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-10T20:55:46.121 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T20:55:46.121 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T20:55:46.121 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T20:55:46.122 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T20:55:46.122 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T20:55:46.122 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-10T20:55:46.122 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-10T20:55:46.122 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-10T20:55:46.122 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-10T20:55:46.123 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T20:55:46.123 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-10T20:55:46.123 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T20:55:46.123 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T20:55:46.123 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T20:55:46.124 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T20:55:46.124 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T20:55:46.124 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T20:55:46.124 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T20:55:46.124 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T20:55:46.124 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T20:55:46.124 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T20:55:46.124 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T20:55:46.125 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T20:55:46.125 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T20:55:46.125 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-10T20:55:46.125 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-10T20:55:46.125 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-10T20:55:46.125 INFO:teuthology.orchestra.run.smithi052.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T20:55:46.125 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-10T20:55:46.125 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-10T20:55:46.131 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:46.131 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:46.131 DEBUG:teuthology.orchestra.run.smithi052:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-10T20:55:46.138 INFO:teuthology.orchestra.run.smithi052.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T20:55:46.138 INFO:teuthology.orchestra.run.smithi052.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T20:55:46.138 INFO:teuthology.orchestra.run.smithi052.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T20:55:46.139 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-10T20:55:46.139 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:46.139 DEBUG:teuthology.orchestra.run.smithi052:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-10T20:55:46.183 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi052.front.sepia.ceph.com to complete... 2024-09-10T20:55:46.615 INFO:teuthology.orchestra.run.smithi052.stderr:Sourcing file `/etc/default/grub' 2024-09-10T20:55:46.616 INFO:teuthology.orchestra.run.smithi052.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T20:55:46.628 INFO:teuthology.orchestra.run.smithi052.stderr:Generating grub configuration file ... 2024-09-10T20:55:46.854 INFO:teuthology.orchestra.run.smithi052.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:46.860 INFO:teuthology.orchestra.run.smithi052.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:47.130 INFO:teuthology.orchestra.run.smithi052.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T20:55:47.135 INFO:teuthology.orchestra.run.smithi052.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T20:55:47.178 INFO:teuthology.orchestra.run.smithi052.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T20:55:47.184 INFO:teuthology.orchestra.run.smithi052.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T20:55:47.296 INFO:teuthology.orchestra.run.smithi052.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T20:55:47.302 INFO:teuthology.orchestra.run.smithi052.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T20:55:47.302 INFO:teuthology.orchestra.run.smithi052.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T20:55:47.314 INFO:teuthology.orchestra.run.smithi052.stderr:done 2024-09-10T20:55:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:Sourcing file `/etc/default/grub' 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T20:55:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-10T20:55:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:49.020 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T20:55:49.030 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T20:55:49.098 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T20:55:49.104 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T20:55:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T20:55:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T20:55:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T20:55:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-10T20:55:49.498 DEBUG:teuthology.orchestra.run.smithi016:> dpkg-deb -c /tmp/linux-image.deb 2024-09-10T20:55:49.506 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-10T20:55:49.506 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-10T20:55:49.506 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:49.526 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:49.526 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:49.550 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-10T20:55:49.551 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T20:55:49.552 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T20:55:49.553 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T20:55:49.553 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T20:55:49.553 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T20:55:49.553 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T20:55:49.553 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-10T20:55:49.553 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-10T20:55:49.553 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-10T20:55:49.553 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T20:55:49.553 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-10T20:55:49.553 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-10T20:55:49.553 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-10T20:55:49.553 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T20:55:49.554 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T20:55:49.554 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T20:55:49.570 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-10T20:55:49.571 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-10T20:55:49.571 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-10T20:55:49.571 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-10T20:55:49.571 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T20:55:49.571 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T20:55:49.571 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T20:55:49.571 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-10T20:55:49.571 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-10T20:55:49.571 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-10T20:55:49.571 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T20:55:49.571 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T20:55:49.571 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T20:55:49.571 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T20:55:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T20:55:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T20:55:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T20:55:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T20:55:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T20:55:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T20:55:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T20:55:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T20:55:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T20:55:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T20:55:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T20:55:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T20:55:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T20:55:49.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T20:55:49.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-10T20:55:49.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T20:55:49.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-10T20:55:49.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T20:55:49.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T20:55:49.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T20:55:49.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T20:55:49.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T20:55:49.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T20:55:49.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T20:55:49.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T20:55:49.573 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T20:55:49.574 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T20:55:49.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T20:55:49.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T20:55:49.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T20:55:49.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-10T20:55:49.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T20:55:49.575 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-10T20:55:49.575 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-10T20:55:49.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-10T20:55:49.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T20:55:49.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T20:55:49.575 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-10T20:55:49.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-10T20:55:49.575 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T20:55:49.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T20:55:49.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T20:55:49.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T20:55:49.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T20:55:49.576 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-10T20:55:49.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T20:55:49.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-10T20:55:49.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T20:55:49.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-10T20:55:49.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T20:55:49.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T20:55:49.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T20:55:49.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T20:55:49.577 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T20:55:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T20:55:49.579 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T20:55:49.580 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-10T20:55:49.581 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-10T20:55:49.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T20:55:49.581 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-10T20:55:49.581 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T20:55:49.614 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T20:55:49.614 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-10T20:55:49.614 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T20:55:49.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T20:55:49.615 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-10T20:55:49.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T20:55:49.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T20:55:49.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T20:55:49.615 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-10T20:55:49.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T20:55:49.615 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-10T20:55:49.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T20:55:49.615 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-10T20:55:49.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-10T20:55:49.615 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T20:55:49.615 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-10T20:55:49.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T20:55:49.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T20:55:49.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T20:55:49.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T20:55:49.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T20:55:49.616 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-10T20:55:49.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T20:55:49.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T20:55:49.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T20:55:49.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T20:55:49.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T20:55:49.616 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T20:55:49.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T20:55:49.617 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-10T20:55:49.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T20:55:49.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-10T20:55:49.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T20:55:49.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T20:55:49.617 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-10T20:55:49.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T20:55:49.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-10T20:55:49.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-10T20:55:49.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-10T20:55:49.617 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T20:55:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T20:55:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-10T20:55:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-10T20:55:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-10T20:55:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-10T20:55:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-10T20:55:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-10T20:55:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-10T20:55:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-10T20:55:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-10T20:55:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-10T20:55:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-10T20:55:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T20:55:49.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T20:55:49.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T20:55:49.619 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-10T20:55:49.619 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-10T20:55:49.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T20:55:49.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T20:55:49.619 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-10T20:55:49.619 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-10T20:55:49.619 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-10T20:55:49.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T20:55:49.619 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-10T20:55:49.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T20:55:49.619 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T20:55:49.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T20:55:49.620 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-10T20:55:49.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T20:55:49.620 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-10T20:55:49.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T20:55:49.620 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-10T20:55:49.620 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-10T20:55:49.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T20:55:49.620 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-10T20:55:49.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T20:55:49.620 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-10T20:55:49.620 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T20:55:49.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T20:55:49.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T20:55:49.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T20:55:49.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T20:55:49.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T20:55:49.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T20:55:49.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T20:55:49.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T20:55:49.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T20:55:49.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T20:55:49.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T20:55:49.621 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T20:55:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-10T20:55:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-10T20:55:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-10T20:55:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-10T20:55:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-10T20:55:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-10T20:55:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-10T20:55:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-10T20:55:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-10T20:55:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T20:55:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-10T20:55:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-10T20:55:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-10T20:55:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-10T20:55:49.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-10T20:55:49.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-10T20:55:49.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-10T20:55:49.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-10T20:55:49.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-10T20:55:49.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-10T20:55:49.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-10T20:55:49.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-10T20:55:49.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-10T20:55:49.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-10T20:55:49.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-10T20:55:49.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T20:55:49.624 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-10T20:55:49.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-10T20:55:49.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-10T20:55:49.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-10T20:55:49.625 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-10T20:55:49.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-10T20:55:49.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-10T20:55:49.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T20:55:49.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T20:55:49.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T20:55:49.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-10T20:55:49.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-10T20:55:49.625 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-10T20:55:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-10T20:55:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-10T20:55:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T20:55:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T20:55:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T20:55:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T20:55:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-10T20:55:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T20:55:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-10T20:55:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T20:55:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T20:55:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T20:55:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T20:55:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T20:55:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T20:55:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T20:55:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-10T20:55:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T20:55:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T20:55:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T20:55:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T20:55:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T20:55:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-10T20:55:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T20:55:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T20:55:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-10T20:55:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-10T20:55:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-10T20:55:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-10T20:55:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T20:55:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-10T20:55:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T20:55:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T20:55:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T20:55:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T20:55:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T20:55:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T20:55:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T20:55:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T20:55:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T20:55:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-10T20:55:49.631 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T20:55:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T20:55:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T20:55:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T20:55:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T20:55:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T20:55:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T20:55:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T20:55:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T20:55:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T20:55:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T20:55:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T20:55:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T20:55:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-10T20:55:49.633 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-10T20:55:49.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T20:55:49.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T20:55:49.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T20:55:49.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T20:55:49.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T20:55:49.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-10T20:55:49.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T20:55:49.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T20:55:49.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T20:55:49.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T20:55:49.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T20:55:49.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T20:55:49.634 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T20:55:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T20:55:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T20:55:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T20:55:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-10T20:55:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T20:55:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T20:55:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T20:55:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T20:55:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T20:55:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T20:55:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T20:55:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T20:55:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T20:55:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T20:55:49.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T20:55:49.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T20:55:49.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T20:55:49.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T20:55:49.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T20:55:49.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T20:55:49.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T20:55:49.637 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-10T20:55:49.638 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T20:55:49.639 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T20:55:49.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T20:55:49.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T20:55:49.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T20:55:49.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T20:55:49.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T20:55:49.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T20:55:49.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T20:55:49.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T20:55:49.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T20:55:49.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T20:55:49.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T20:55:49.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T20:55:49.640 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T20:55:49.641 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-10T20:55:49.642 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-10T20:55:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-10T20:55:49.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T20:55:49.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T20:55:49.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T20:55:49.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-10T20:55:49.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T20:55:49.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T20:55:49.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-10T20:55:49.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T20:55:49.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T20:55:49.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T20:55:49.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-10T20:55:49.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T20:55:49.644 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T20:55:49.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-10T20:55:49.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-10T20:55:49.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T20:55:49.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-10T20:55:49.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T20:55:49.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T20:55:49.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T20:55:49.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T20:55:49.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-10T20:55:49.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T20:55:49.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T20:55:49.645 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T20:55:49.646 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-10T20:55:49.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-10T20:55:49.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-10T20:55:49.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-10T20:55:49.646 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-10T20:55:49.646 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-10T20:55:49.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T20:55:49.646 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T20:55:49.670 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T20:55:49.670 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T20:55:49.670 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T20:55:49.670 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T20:55:49.670 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T20:55:49.670 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-10T20:55:49.670 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-10T20:55:49.670 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T20:55:49.670 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-10T20:55:49.671 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T20:55:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-10T20:55:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T20:55:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-10T20:55:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T20:55:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-10T20:55:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T20:55:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-10T20:55:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-10T20:55:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-10T20:55:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-10T20:55:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T20:55:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T20:55:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-10T20:55:49.673 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-10T20:55:49.674 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-10T20:55:49.675 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-10T20:55:49.676 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T20:55:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-10T20:55:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-10T20:55:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-10T20:55:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T20:55:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-10T20:55:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-10T20:55:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-10T20:55:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T20:55:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-10T20:55:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-10T20:55:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-10T20:55:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-10T20:55:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-10T20:55:49.678 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-10T20:55:49.678 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-10T20:55:49.678 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T20:55:49.678 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-10T20:55:49.678 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-10T20:55:49.678 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-10T20:55:49.678 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-10T20:55:49.678 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T20:55:49.678 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T20:55:49.678 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T20:55:49.678 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T20:55:49.678 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-10T20:55:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T20:55:49.680 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T20:55:49.680 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-10T20:55:49.680 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-10T20:55:49.680 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T20:55:49.680 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T20:55:49.680 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T20:55:49.680 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-10T20:55:49.680 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T20:55:49.680 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T20:55:49.680 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-10T20:55:49.680 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T20:55:49.680 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T20:55:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-10T20:55:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-10T20:55:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-10T20:55:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-10T20:55:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-10T20:55:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T20:55:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T20:55:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-10T20:55:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-10T20:55:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-10T20:55:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-10T20:55:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-10T20:55:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-10T20:55:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-10T20:55:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-10T20:55:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T20:55:49.685 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T20:55:49.685 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T20:55:49.685 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T20:55:49.685 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-10T20:55:49.685 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-10T20:55:49.685 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-10T20:55:49.685 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-10T20:55:49.685 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T20:55:49.685 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-10T20:55:49.685 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-10T20:55:49.685 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-10T20:55:49.685 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-10T20:55:49.685 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-10T20:55:49.686 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T20:55:49.686 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T20:55:49.686 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T20:55:49.686 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T20:55:49.686 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-10T20:55:49.686 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-10T20:55:49.686 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-10T20:55:49.686 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T20:55:49.686 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-10T20:55:49.686 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-10T20:55:49.686 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-10T20:55:49.686 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T20:55:49.686 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T20:55:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-10T20:55:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T20:55:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T20:55:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-10T20:55:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T20:55:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T20:55:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T20:55:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-10T20:55:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-10T20:55:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-10T20:55:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T20:55:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T20:55:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T20:55:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T20:55:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-10T20:55:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T20:55:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-10T20:55:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-10T20:55:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T20:55:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-10T20:55:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-10T20:55:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T20:55:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T20:55:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T20:55:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-10T20:55:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T20:55:49.689 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T20:55:49.689 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T20:55:49.689 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-10T20:55:49.689 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-10T20:55:49.689 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T20:55:49.689 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T20:55:49.689 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T20:55:49.689 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T20:55:49.689 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-10T20:55:49.689 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T20:55:49.689 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T20:55:49.689 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T20:55:49.689 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T20:55:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-10T20:55:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T20:55:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-10T20:55:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T20:55:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-10T20:55:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T20:55:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-10T20:55:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T20:55:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-10T20:55:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T20:55:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T20:55:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T20:55:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-10T20:55:49.691 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-10T20:55:49.691 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T20:55:49.691 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-10T20:55:49.691 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T20:55:49.691 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-10T20:55:49.691 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T20:55:49.691 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-10T20:55:49.691 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T20:55:49.691 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T20:55:49.691 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-10T20:55:49.691 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T20:55:49.691 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T20:55:49.691 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T20:55:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-10T20:55:49.693 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T20:55:49.698 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-10T20:55:49.698 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T20:55:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-10T20:55:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T20:55:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-10T20:55:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-10T20:55:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T20:55:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-10T20:55:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-10T20:55:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T20:55:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T20:55:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T20:55:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T20:55:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T20:55:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T20:55:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-10T20:55:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T20:55:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T20:55:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T20:55:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-10T20:55:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-10T20:55:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-10T20:55:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T20:55:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T20:55:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T20:55:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-10T20:55:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-10T20:55:49.701 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-10T20:55:49.701 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T20:55:49.701 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-10T20:55:49.701 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T20:55:49.701 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-10T20:55:49.701 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T20:55:49.701 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-10T20:55:49.701 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T20:55:49.701 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-10T20:55:49.701 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T20:55:49.701 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-10T20:55:49.701 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T20:55:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-10T20:55:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T20:55:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-10T20:55:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T20:55:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-10T20:55:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T20:55:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-10T20:55:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T20:55:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-10T20:55:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T20:55:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-10T20:55:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T20:55:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-10T20:55:49.703 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-10T20:55:49.704 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-10T20:55:49.704 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T20:55:49.704 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-10T20:55:49.704 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-10T20:55:49.704 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T20:55:49.704 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-10T20:55:49.704 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T20:55:49.704 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T20:55:49.704 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-10T20:55:49.704 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T20:55:49.704 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-10T20:55:49.704 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T20:55:49.704 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-10T20:55:49.705 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T20:55:49.705 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T20:55:49.705 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-10T20:55:49.705 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-10T20:55:49.705 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T20:55:49.705 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T20:55:49.705 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-10T20:55:49.705 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T20:55:49.708 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-10T20:55:49.708 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T20:55:49.708 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-10T20:55:49.708 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T20:55:49.708 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T20:55:49.708 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T20:55:49.708 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T20:55:49.708 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-10T20:55:49.708 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-10T20:55:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-10T20:55:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-10T20:55:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T20:55:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-10T20:55:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T20:55:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-10T20:55:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T20:55:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T20:55:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-10T20:55:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T20:55:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T20:55:49.709 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T20:55:49.710 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T20:55:49.710 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-10T20:55:49.710 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-10T20:55:49.710 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-10T20:55:49.710 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T20:55:49.710 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-10T20:55:49.710 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T20:55:49.710 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-10T20:55:49.710 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T20:55:49.710 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T20:55:49.717 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T20:55:49.717 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T20:55:49.717 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-10T20:55:49.718 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T20:55:49.718 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-10T20:55:49.718 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T20:55:49.718 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-10T20:55:49.718 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T20:55:49.718 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T20:55:49.718 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-10T20:55:49.718 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T20:55:49.718 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T20:55:49.718 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-10T20:55:49.718 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T20:55:49.718 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-10T20:55:49.719 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-10T20:55:49.719 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-10T20:55:49.719 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-10T20:55:49.719 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T20:55:49.719 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-10T20:55:49.719 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T20:55:49.719 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-10T20:55:49.719 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T20:55:49.719 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T20:55:49.719 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-10T20:55:49.719 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T20:55:49.719 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T20:55:49.719 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T20:55:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-10T20:55:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T20:55:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T20:55:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-10T20:55:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T20:55:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T20:55:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T20:55:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T20:55:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T20:55:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-10T20:55:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T20:55:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T20:55:49.721 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-10T20:55:49.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T20:55:49.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T20:55:49.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T20:55:49.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T20:55:49.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T20:55:49.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T20:55:49.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T20:55:49.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T20:55:49.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T20:55:49.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T20:55:49.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T20:55:49.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T20:55:49.722 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T20:55:49.722 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T20:55:49.722 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T20:55:49.722 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T20:55:49.722 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T20:55:49.722 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T20:55:49.722 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T20:55:49.722 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-10T20:55:49.722 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-10T20:55:49.722 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-10T20:55:49.722 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T20:55:49.722 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T20:55:49.722 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T20:55:49.723 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T20:55:49.724 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T20:55:49.724 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T20:55:49.724 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T20:55:49.724 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-10T20:55:49.724 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T20:55:49.724 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-10T20:55:49.724 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T20:55:49.739 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-10T20:55:49.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T20:55:49.739 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-10T20:55:49.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T20:55:49.739 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-10T20:55:49.739 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-10T20:55:49.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T20:55:49.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T20:55:49.740 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-10T20:55:49.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T20:55:49.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T20:55:49.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T20:55:49.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T20:55:49.740 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-10T20:55:49.740 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-10T20:55:49.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T20:55:49.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-10T20:55:49.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-10T20:55:49.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-10T20:55:49.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T20:55:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-10T20:55:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T20:55:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-10T20:55:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-10T20:55:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-10T20:55:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T20:55:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T20:55:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-10T20:55:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-10T20:55:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-10T20:55:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-10T20:55:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-10T20:55:49.742 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-10T20:55:49.742 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-10T20:55:49.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T20:55:49.742 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-10T20:55:49.742 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-10T20:55:49.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-10T20:55:49.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-10T20:55:49.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-10T20:55:49.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-10T20:55:49.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-10T20:55:49.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-10T20:55:49.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-10T20:55:49.742 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-10T20:55:49.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T20:55:49.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T20:55:49.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T20:55:49.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T20:55:49.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T20:55:49.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T20:55:49.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T20:55:49.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T20:55:49.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T20:55:49.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T20:55:49.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T20:55:49.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T20:55:49.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-10T20:55:49.744 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-10T20:55:49.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T20:55:49.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-10T20:55:49.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T20:55:49.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T20:55:49.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T20:55:49.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T20:55:49.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T20:55:49.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T20:55:49.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T20:55:49.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T20:55:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T20:55:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T20:55:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T20:55:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-10T20:55:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-10T20:55:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-10T20:55:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-10T20:55:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-10T20:55:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-10T20:55:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-10T20:55:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-10T20:55:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-10T20:55:49.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-10T20:55:49.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-10T20:55:49.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-10T20:55:49.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-10T20:55:49.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-10T20:55:49.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-10T20:55:49.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-10T20:55:49.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-10T20:55:49.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-10T20:55:49.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-10T20:55:49.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-10T20:55:49.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-10T20:55:49.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-10T20:55:49.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-10T20:55:49.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-10T20:55:49.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-10T20:55:49.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-10T20:55:49.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-10T20:55:49.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-10T20:55:49.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-10T20:55:49.747 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-10T20:55:49.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T20:55:49.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T20:55:49.747 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-10T20:55:49.747 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-10T20:55:49.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T20:55:49.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T20:55:49.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T20:55:49.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T20:55:49.748 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-10T20:55:49.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T20:55:49.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T20:55:49.748 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-10T20:55:49.748 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-10T20:55:49.748 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-10T20:55:49.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T20:55:49.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T20:55:49.748 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-10T20:55:49.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-10T20:55:49.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-10T20:55:49.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T20:55:49.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-10T20:55:49.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-10T20:55:49.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-10T20:55:49.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T20:55:49.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T20:55:49.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T20:55:49.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T20:55:49.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T20:55:49.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T20:55:49.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T20:55:49.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T20:55:49.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T20:55:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T20:55:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T20:55:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T20:55:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T20:55:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-10T20:55:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T20:55:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T20:55:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T20:55:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-10T20:55:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T20:55:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-10T20:55:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-10T20:55:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-10T20:55:49.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-10T20:55:49.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-10T20:55:49.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T20:55:49.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-10T20:55:49.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T20:55:49.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-10T20:55:49.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T20:55:49.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T20:55:49.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T20:55:49.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T20:55:49.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-10T20:55:49.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T20:55:49.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T20:55:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T20:55:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T20:55:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T20:55:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-10T20:55:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T20:55:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-10T20:55:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-10T20:55:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-10T20:55:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-10T20:55:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T20:55:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-10T20:55:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T20:55:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T20:55:49.753 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T20:55:49.753 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T20:55:49.753 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T20:55:49.753 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-10T20:55:49.753 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T20:55:49.753 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T20:55:49.753 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-10T20:55:49.753 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T20:55:49.753 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T20:55:49.753 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-10T20:55:49.753 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-10T20:55:49.753 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T20:55:49.753 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-10T20:55:49.754 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T20:55:49.754 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T20:55:49.754 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-10T20:55:49.754 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T20:55:49.754 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-10T20:55:49.754 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T20:55:49.758 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-10T20:55:49.758 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T20:55:49.758 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-10T20:55:49.758 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T20:55:49.758 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T20:55:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-10T20:55:49.760 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T20:55:49.760 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T20:55:49.760 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-10T20:55:49.760 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-10T20:55:49.760 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T20:55:49.760 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T20:55:49.760 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T20:55:49.760 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T20:55:49.760 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-10T20:55:49.760 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T20:55:49.760 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T20:55:49.760 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T20:55:49.760 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-10T20:55:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T20:55:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T20:55:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T20:55:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-10T20:55:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T20:55:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T20:55:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T20:55:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-10T20:55:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T20:55:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-10T20:55:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T20:55:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-10T20:55:49.762 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T20:55:49.762 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T20:55:49.762 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T20:55:49.762 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T20:55:49.762 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-10T20:55:49.762 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T20:55:49.763 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-10T20:55:49.763 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T20:55:49.763 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T20:55:49.764 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-10T20:55:49.764 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-10T20:55:49.764 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-10T20:55:49.764 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-10T20:55:49.764 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-10T20:55:49.764 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-10T20:55:49.764 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T20:55:49.764 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T20:55:49.764 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-10T20:55:49.764 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T20:55:49.764 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-10T20:55:49.764 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T20:55:49.767 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T20:55:49.768 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-10T20:55:49.768 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T20:55:49.769 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-10T20:55:49.769 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T20:55:49.769 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T20:55:49.770 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T20:55:49.770 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T20:55:49.771 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T20:55:49.771 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T20:55:49.771 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T20:55:49.771 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T20:55:49.772 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T20:55:49.773 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T20:55:49.773 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-10T20:55:49.773 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T20:55:49.773 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T20:55:49.773 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T20:55:49.773 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T20:55:49.773 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-10T20:55:49.773 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-10T20:55:49.773 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T20:55:49.774 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-10T20:55:49.774 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-10T20:55:49.774 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T20:55:49.775 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-10T20:55:49.775 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T20:55:49.775 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T20:55:49.775 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T20:55:49.775 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T20:55:49.777 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T20:55:49.777 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T20:55:49.781 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-10T20:55:49.781 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T20:55:49.781 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-10T20:55:49.781 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-10T20:55:49.781 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-10T20:55:49.782 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T20:55:49.782 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T20:55:49.782 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T20:55:49.782 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T20:55:49.782 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T20:55:49.782 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T20:55:49.782 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T20:55:49.782 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T20:55:49.782 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T20:55:49.782 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T20:55:49.782 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T20:55:49.782 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T20:55:49.782 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T20:55:49.783 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T20:55:49.783 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T20:55:49.783 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-10T20:55:49.783 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T20:55:49.783 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T20:55:49.783 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T20:55:49.783 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-10T20:55:49.783 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-10T20:55:49.783 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T20:55:49.783 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-10T20:55:49.783 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-10T20:55:49.783 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T20:55:49.783 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T20:55:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T20:55:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-10T20:55:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-10T20:55:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-10T20:55:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T20:55:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-10T20:55:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-10T20:55:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T20:55:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-10T20:55:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-10T20:55:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T20:55:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-10T20:55:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-10T20:55:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-10T20:55:49.792 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-10T20:55:49.792 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-10T20:55:49.792 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T20:55:49.792 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-10T20:55:49.792 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T20:55:49.792 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-10T20:55:49.792 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-10T20:55:49.792 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T20:55:49.793 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T20:55:49.793 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T20:55:49.793 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-10T20:55:49.793 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-10T20:55:49.793 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-10T20:55:49.793 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T20:55:49.793 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-10T20:55:49.793 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T20:55:49.793 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-10T20:55:49.793 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-10T20:55:49.793 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-10T20:55:49.793 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-10T20:55:49.793 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-10T20:55:49.794 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-10T20:55:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-10T20:55:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-10T20:55:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-10T20:55:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-10T20:55:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-10T20:55:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-10T20:55:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-10T20:55:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-10T20:55:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-10T20:55:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-10T20:55:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-10T20:55:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-10T20:55:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-10T20:55:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-10T20:55:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-10T20:55:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-10T20:55:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-10T20:55:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-10T20:55:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-10T20:55:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-10T20:55:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-10T20:55:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-10T20:55:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-10T20:55:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-10T20:55:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-10T20:55:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-10T20:55:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-10T20:55:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-10T20:55:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-10T20:55:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-10T20:55:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-10T20:55:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-10T20:55:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-10T20:55:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-10T20:55:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-10T20:55:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-10T20:55:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-10T20:55:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-10T20:55:49.798 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-10T20:55:49.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-10T20:55:49.798 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-10T20:55:49.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-10T20:55:49.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-10T20:55:49.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-10T20:55:49.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-10T20:55:49.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-10T20:55:49.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-10T20:55:49.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-10T20:55:49.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-10T20:55:49.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-10T20:55:49.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-10T20:55:49.799 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-10T20:55:49.799 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-10T20:55:49.817 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-10T20:55:49.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-10T20:55:49.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-10T20:55:49.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-10T20:55:49.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-10T20:55:49.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T20:55:49.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-10T20:55:49.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-10T20:55:49.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-10T20:55:49.818 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-10T20:55:49.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T20:55:49.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T20:55:49.818 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-10T20:55:49.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T20:55:49.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T20:55:49.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-10T20:55:49.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-10T20:55:49.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-10T20:55:49.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T20:55:49.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T20:55:49.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T20:55:49.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T20:55:49.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T20:55:49.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T20:55:49.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T20:55:49.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T20:55:49.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-10T20:55:49.820 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-10T20:55:49.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T20:55:49.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T20:55:49.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T20:55:49.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T20:55:49.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T20:55:49.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-10T20:55:49.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T20:55:49.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T20:55:49.820 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-10T20:55:49.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-10T20:55:49.821 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-10T20:55:49.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-10T20:55:49.821 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-10T20:55:49.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T20:55:49.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-10T20:55:49.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-10T20:55:49.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-10T20:55:49.821 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-10T20:55:49.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T20:55:49.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T20:55:49.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T20:55:49.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T20:55:49.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T20:55:49.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T20:55:49.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T20:55:49.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T20:55:49.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T20:55:49.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-10T20:55:49.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T20:55:49.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T20:55:49.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T20:55:49.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T20:55:49.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T20:55:49.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T20:55:49.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T20:55:49.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T20:55:49.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T20:55:49.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T20:55:49.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T20:55:49.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T20:55:49.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T20:55:49.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-10T20:55:49.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T20:55:49.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T20:55:49.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-10T20:55:49.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T20:55:49.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T20:55:49.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T20:55:49.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T20:55:49.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T20:55:49.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T20:55:49.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T20:55:49.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T20:55:49.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T20:55:49.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T20:55:49.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T20:55:49.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T20:55:49.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T20:55:49.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T20:55:49.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T20:55:49.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T20:55:49.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T20:55:49.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T20:55:49.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-10T20:55:49.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T20:55:49.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T20:55:49.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-10T20:55:49.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T20:55:49.825 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-10T20:55:49.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T20:55:49.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T20:55:49.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T20:55:49.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T20:55:49.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T20:55:49.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T20:55:49.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T20:55:49.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T20:55:49.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T20:55:49.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T20:55:49.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T20:55:49.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T20:55:49.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T20:55:49.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T20:55:49.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T20:55:49.826 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-10T20:55:49.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T20:55:49.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T20:55:49.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T20:55:49.827 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-10T20:55:49.827 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-10T20:55:49.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T20:55:49.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T20:55:49.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T20:55:49.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T20:55:49.827 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-10T20:55:49.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T20:55:49.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T20:55:49.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T20:55:49.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T20:55:49.828 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-10T20:55:49.828 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-10T20:55:49.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-10T20:55:49.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-10T20:55:49.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-10T20:55:49.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-10T20:55:49.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-10T20:55:49.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T20:55:49.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-10T20:55:49.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-10T20:55:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-10T20:55:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-10T20:55:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-10T20:55:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-10T20:55:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T20:55:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-10T20:55:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-10T20:55:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-10T20:55:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-10T20:55:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-10T20:55:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-10T20:55:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T20:55:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-10T20:55:49.830 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-10T20:55:49.830 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-10T20:55:49.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T20:55:49.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T20:55:49.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-10T20:55:49.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-10T20:55:49.830 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-10T20:55:49.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T20:55:49.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T20:55:49.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T20:55:49.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T20:55:49.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T20:55:49.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T20:55:49.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T20:55:49.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T20:55:49.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T20:55:49.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T20:55:49.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T20:55:49.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T20:55:49.831 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-10T20:55:49.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-10T20:55:49.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-10T20:55:49.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T20:55:49.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T20:55:49.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-10T20:55:49.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-10T20:55:49.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-10T20:55:49.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-10T20:55:49.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-10T20:55:49.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-10T20:55:49.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T20:55:49.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T20:55:49.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T20:55:49.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T20:55:49.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T20:55:49.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T20:55:49.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T20:55:49.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T20:55:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T20:55:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T20:55:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-10T20:55:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T20:55:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-10T20:55:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-10T20:55:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T20:55:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-10T20:55:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-10T20:55:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T20:55:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T20:55:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-10T20:55:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-10T20:55:49.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-10T20:55:49.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T20:55:49.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-10T20:55:49.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T20:55:49.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-10T20:55:49.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T20:55:49.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T20:55:49.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-10T20:55:49.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T20:55:49.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T20:55:49.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T20:55:49.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T20:55:49.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-10T20:55:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T20:55:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-10T20:55:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-10T20:55:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T20:55:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-10T20:55:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T20:55:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T20:55:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T20:55:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-10T20:55:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T20:55:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T20:55:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-10T20:55:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-10T20:55:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T20:55:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-10T20:55:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-10T20:55:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T20:55:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-10T20:55:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-10T20:55:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-10T20:55:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T20:55:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-10T20:55:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-10T20:55:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-10T20:55:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-10T20:55:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-10T20:55:49.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T20:55:49.837 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-10T20:55:49.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-10T20:55:49.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T20:55:49.837 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-10T20:55:49.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T20:55:49.839 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-10T20:55:49.839 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T20:55:49.839 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-10T20:55:49.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T20:55:49.840 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-10T20:55:49.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T20:55:49.840 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-10T20:55:49.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-10T20:55:49.840 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-10T20:55:49.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T20:55:49.840 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-10T20:55:49.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T20:55:49.840 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-10T20:55:49.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-10T20:55:49.840 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-10T20:55:49.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T20:55:49.841 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-10T20:55:49.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T20:55:49.841 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-10T20:55:49.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-10T20:55:49.841 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-10T20:55:49.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T20:55:49.841 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-10T20:55:49.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T20:55:49.841 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-10T20:55:49.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T20:55:49.841 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-10T20:55:49.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T20:55:49.842 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-10T20:55:49.842 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-10T20:55:49.842 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-10T20:55:49.842 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T20:55:49.842 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-10T20:55:49.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T20:55:49.843 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-10T20:55:49.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T20:55:49.843 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-10T20:55:49.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T20:55:49.843 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-10T20:55:49.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T20:55:49.844 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-10T20:55:49.844 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-10T20:55:49.844 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T20:55:49.844 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-10T20:55:49.844 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T20:55:49.844 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-10T20:55:49.844 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T20:55:49.845 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T20:55:49.846 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-10T20:55:49.846 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T20:55:49.846 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T20:55:49.851 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-10T20:55:49.851 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T20:55:49.851 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T20:55:49.851 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-10T20:55:49.851 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T20:55:49.855 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-10T20:55:49.855 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T20:55:49.860 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-10T20:55:49.860 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T20:55:49.860 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T20:55:49.860 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T20:55:49.860 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T20:55:49.860 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T20:55:49.860 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T20:55:49.860 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T20:55:49.860 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T20:55:49.860 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T20:55:49.860 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T20:55:49.861 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T20:55:49.861 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-10T20:55:49.861 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T20:55:49.861 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T20:55:49.861 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T20:55:49.861 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T20:55:49.861 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T20:55:49.861 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T20:55:49.861 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T20:55:49.861 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T20:55:49.861 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T20:55:49.861 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T20:55:49.861 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T20:55:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T20:55:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T20:55:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T20:55:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T20:55:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T20:55:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T20:55:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T20:55:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T20:55:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T20:55:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T20:55:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T20:55:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T20:55:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T20:55:49.863 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T20:55:49.863 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T20:55:49.863 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T20:55:49.863 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T20:55:49.863 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T20:55:49.863 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T20:55:49.863 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T20:55:49.863 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T20:55:49.863 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T20:55:49.863 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T20:55:49.863 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T20:55:49.863 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T20:55:49.863 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T20:55:49.864 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T20:55:49.864 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T20:55:49.864 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-10T20:55:49.864 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-10T20:55:49.865 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-10T20:55:49.865 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-10T20:55:49.865 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T20:55:49.865 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-10T20:55:49.865 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T20:55:49.865 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-10T20:55:49.865 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T20:55:49.865 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T20:55:49.869 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T20:55:49.869 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T20:55:49.869 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T20:55:49.869 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-10T20:55:49.869 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-10T20:55:49.869 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-10T20:55:49.869 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T20:55:49.870 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-10T20:55:49.870 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T20:55:49.870 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-10T20:55:49.870 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-10T20:55:49.871 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-10T20:55:49.871 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-10T20:55:49.871 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-10T20:55:49.871 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-10T20:55:49.871 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-10T20:55:49.871 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-10T20:55:49.871 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-10T20:55:49.871 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T20:55:49.872 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-10T20:55:49.872 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T20:55:49.872 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-10T20:55:49.872 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-10T20:55:49.872 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T20:55:49.878 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-10T20:55:49.878 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T20:55:49.878 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T20:55:49.879 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-10T20:55:49.879 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T20:55:49.879 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-10T20:55:49.879 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T20:55:49.879 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-10T20:55:49.879 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T20:55:49.879 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-10T20:55:49.880 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T20:55:49.880 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-10T20:55:49.880 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T20:55:49.891 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-10T20:55:49.891 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-10T20:55:49.891 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-10T20:55:49.891 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-10T20:55:49.891 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-10T20:55:49.892 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T20:55:49.892 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-10T20:55:49.892 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T20:55:49.892 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-10T20:55:49.892 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T20:55:49.892 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-10T20:55:49.892 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T20:55:49.892 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T20:55:49.892 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-10T20:55:49.892 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T20:55:49.892 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T20:55:49.892 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-10T20:55:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T20:55:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T20:55:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-10T20:55:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T20:55:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-10T20:55:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-10T20:55:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-10T20:55:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-10T20:55:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-10T20:55:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T20:55:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-10T20:55:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-10T20:55:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T20:55:49.894 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-10T20:55:49.894 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-10T20:55:49.896 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-10T20:55:49.896 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-10T20:55:49.896 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T20:55:49.897 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T20:55:49.897 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T20:55:49.897 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-10T20:55:49.897 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-10T20:55:49.897 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-10T20:55:49.897 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T20:55:49.897 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-10T20:55:49.897 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-10T20:55:49.897 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T20:55:49.897 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T20:55:49.897 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T20:55:49.897 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T20:55:49.898 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T20:55:49.898 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T20:55:49.898 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T20:55:49.898 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T20:55:49.898 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-10T20:55:49.898 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T20:55:49.898 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T20:55:49.898 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T20:55:49.898 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T20:55:49.898 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T20:55:49.898 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-10T20:55:49.898 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T20:55:49.899 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-10T20:55:49.899 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T20:55:49.899 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T20:55:49.899 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T20:55:49.899 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T20:55:49.899 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-10T20:55:49.899 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T20:55:49.901 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-10T20:55:49.901 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T20:55:49.901 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T20:55:49.901 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-10T20:55:49.901 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T20:55:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T20:55:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T20:55:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T20:55:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T20:55:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T20:55:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T20:55:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T20:55:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T20:55:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T20:55:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T20:55:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T20:55:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T20:55:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T20:55:49.903 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T20:55:49.903 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T20:55:49.903 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T20:55:49.903 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T20:55:49.903 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T20:55:49.903 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T20:55:49.903 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T20:55:49.903 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T20:55:49.903 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T20:55:49.903 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-10T20:55:49.903 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T20:55:49.904 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-10T20:55:49.904 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T20:55:49.905 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-10T20:55:49.905 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T20:55:49.905 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T20:55:49.905 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T20:55:49.905 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T20:55:49.905 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-10T20:55:49.905 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T20:55:49.915 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T20:55:49.915 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T20:55:49.915 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T20:55:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T20:55:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-10T20:55:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-10T20:55:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-10T20:55:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-10T20:55:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T20:55:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T20:55:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T20:55:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-10T20:55:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T20:55:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T20:55:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T20:55:49.917 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T20:55:49.917 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T20:55:49.917 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T20:55:49.917 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T20:55:49.917 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T20:55:49.917 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T20:55:49.917 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T20:55:49.917 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-10T20:55:49.917 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T20:55:49.917 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T20:55:49.917 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T20:55:49.917 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-10T20:55:49.917 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T20:55:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-10T20:55:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T20:55:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T20:55:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T20:55:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-10T20:55:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T20:55:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T20:55:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T20:55:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T20:55:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T20:55:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T20:55:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T20:55:49.918 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T20:55:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T20:55:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-10T20:55:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T20:55:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T20:55:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T20:55:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T20:55:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T20:55:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T20:55:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T20:55:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T20:55:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T20:55:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T20:55:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T20:55:49.920 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T20:55:49.920 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T20:55:49.920 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T20:55:49.920 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T20:55:49.920 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T20:55:49.920 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T20:55:49.920 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T20:55:49.920 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T20:55:49.920 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T20:55:49.920 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T20:55:49.920 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T20:55:49.920 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-10T20:55:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T20:55:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T20:55:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T20:55:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-10T20:55:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T20:55:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T20:55:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T20:55:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T20:55:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-10T20:55:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-10T20:55:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T20:55:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-10T20:55:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T20:55:49.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T20:55:49.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T20:55:49.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T20:55:49.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T20:55:49.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T20:55:49.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T20:55:49.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T20:55:49.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T20:55:49.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T20:55:49.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T20:55:49.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T20:55:49.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T20:55:49.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T20:55:49.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T20:55:49.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T20:55:49.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T20:55:49.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T20:55:49.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T20:55:49.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T20:55:49.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T20:55:49.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T20:55:49.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T20:55:49.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T20:55:49.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T20:55:49.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T20:55:49.923 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-10T20:55:49.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T20:55:49.924 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-10T20:55:49.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T20:55:49.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T20:55:49.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T20:55:49.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T20:55:49.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T20:55:49.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T20:55:49.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T20:55:49.924 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-10T20:55:49.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-10T20:55:49.924 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-10T20:55:49.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T20:55:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-10T20:55:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-10T20:55:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T20:55:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-10T20:55:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T20:55:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-10T20:55:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-10T20:55:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T20:55:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T20:55:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T20:55:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T20:55:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T20:55:49.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T20:55:49.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T20:55:49.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T20:55:49.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T20:55:49.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T20:55:49.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T20:55:49.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T20:55:49.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T20:55:49.943 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T20:55:49.943 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T20:55:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T20:55:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-10T20:55:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T20:55:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T20:55:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T20:55:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T20:55:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T20:55:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T20:55:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T20:55:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T20:55:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T20:55:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T20:55:49.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T20:55:49.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T20:55:49.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T20:55:49.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T20:55:49.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T20:55:49.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T20:55:49.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T20:55:49.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T20:55:49.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T20:55:49.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T20:55:49.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T20:55:49.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T20:55:49.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T20:55:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T20:55:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T20:55:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T20:55:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T20:55:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T20:55:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T20:55:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T20:55:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T20:55:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T20:55:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T20:55:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T20:55:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T20:55:49.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T20:55:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T20:55:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T20:55:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T20:55:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T20:55:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T20:55:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T20:55:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-10T20:55:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T20:55:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T20:55:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T20:55:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T20:55:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T20:55:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T20:55:49.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T20:55:49.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T20:55:49.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T20:55:49.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T20:55:49.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T20:55:49.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T20:55:49.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T20:55:49.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T20:55:49.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T20:55:49.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-10T20:55:49.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T20:55:49.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T20:55:49.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T20:55:49.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T20:55:49.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T20:55:49.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-10T20:55:49.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-10T20:55:49.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-10T20:55:49.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T20:55:49.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T20:55:49.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T20:55:49.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T20:55:49.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T20:55:49.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T20:55:49.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T20:55:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T20:55:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T20:55:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T20:55:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T20:55:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T20:55:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T20:55:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T20:55:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T20:55:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T20:55:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T20:55:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T20:55:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T20:55:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T20:55:49.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T20:55:49.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T20:55:49.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T20:55:49.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T20:55:49.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T20:55:49.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T20:55:49.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T20:55:49.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T20:55:49.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T20:55:49.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T20:55:49.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T20:55:49.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T20:55:49.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T20:55:49.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T20:55:49.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T20:55:49.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T20:55:49.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T20:55:49.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T20:55:49.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T20:55:49.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T20:55:49.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T20:55:49.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T20:55:49.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T20:55:49.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T20:55:49.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T20:55:49.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T20:55:49.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T20:55:49.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T20:55:49.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T20:55:49.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T20:55:49.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T20:55:49.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T20:55:49.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T20:55:49.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T20:55:49.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T20:55:49.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T20:55:49.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-10T20:55:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T20:55:49.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T20:55:49.955 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-10T20:55:49.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T20:55:49.955 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-10T20:55:49.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T20:55:49.955 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-10T20:55:49.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T20:55:49.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T20:55:49.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T20:55:49.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T20:55:49.955 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-10T20:55:49.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T20:55:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-10T20:55:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-10T20:55:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-10T20:55:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-10T20:55:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T20:55:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T20:55:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T20:55:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-10T20:55:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-10T20:55:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-10T20:55:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T20:55:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T20:55:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T20:55:49.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T20:55:49.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T20:55:49.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T20:55:49.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T20:55:49.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T20:55:49.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T20:55:49.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T20:55:49.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T20:55:49.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-10T20:55:49.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T20:55:49.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T20:55:49.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T20:55:49.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T20:55:49.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T20:55:49.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T20:55:49.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T20:55:49.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T20:55:49.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T20:55:49.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T20:55:49.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T20:55:49.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T20:55:49.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T20:55:49.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T20:55:49.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-10T20:55:49.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T20:55:49.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T20:55:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T20:55:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T20:55:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T20:55:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T20:55:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T20:55:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T20:55:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T20:55:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T20:55:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T20:55:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T20:55:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T20:55:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T20:55:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T20:55:49.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T20:55:49.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T20:55:49.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T20:55:49.960 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-10T20:55:49.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T20:55:49.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T20:55:49.960 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-10T20:55:49.960 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-10T20:55:49.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T20:55:49.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T20:55:49.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T20:55:49.960 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-10T20:55:49.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T20:55:49.961 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-10T20:55:49.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T20:55:49.964 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T20:55:49.964 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-10T20:55:49.964 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T20:55:49.964 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-10T20:55:49.964 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T20:55:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T20:55:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T20:55:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T20:55:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T20:55:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-10T20:55:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-10T20:55:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-10T20:55:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-10T20:55:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T20:55:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-10T20:55:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T20:55:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T20:55:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T20:55:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T20:55:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T20:55:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T20:55:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T20:55:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T20:55:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T20:55:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T20:55:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T20:55:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T20:55:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T20:55:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-10T20:55:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-10T20:55:49.967 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-10T20:55:49.967 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T20:55:49.967 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-10T20:55:49.967 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-10T20:55:49.971 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:49.971 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:49.971 DEBUG:teuthology.orchestra.run.smithi016:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-10T20:55:50.013 INFO:teuthology.orchestra.run.smithi016.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T20:55:50.013 INFO:teuthology.orchestra.run.smithi016.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T20:55:50.013 INFO:teuthology.orchestra.run.smithi016.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T20:55:50.013 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-10T20:55:50.013 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:50.014 DEBUG:teuthology.orchestra.run.smithi016:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-10T20:55:50.056 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi016.front.sepia.ceph.com to complete... 2024-09-10T20:55:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:Sourcing file `/etc/default/grub' 2024-09-10T20:55:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T20:55:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-10T20:55:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:50.664 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:55:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T20:55:50.929 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T20:55:50.974 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T20:55:50.981 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T20:55:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T20:55:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T20:55:51.121 INFO:teuthology.orchestra.run.smithi016.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T20:55:51.147 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-10T20:56:18.437 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-10T20:56:18.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-09-10T20:56:18.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:56:22.252 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T20:56:22.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-10T20:56:22.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:56:36.925 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.52 2024-09-10T20:56:40.765 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-09-10T20:56:45.929 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-09-10T20:56:45.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:56:49.770 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-10T20:56:49.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:57:45.994 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T20:57:49.782 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T20:57:58.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-09-10T20:57:58.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:57:58.247 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-09-10T20:57:58.970 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-09-10T20:57:58.970 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T20:57:58.970 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-09-10T20:57:59.019 INFO:teuthology.orchestra.run.smithi052.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:57:59.019 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:57:59.019 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T20:57:59.019 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T20:58:00.020 DEBUG:teuthology.parallel:result is None 2024-09-10T20:58:01.783 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-10T20:58:01.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:58:02.007 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-10T20:58:02.781 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-10T20:58:02.781 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T20:58:02.781 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-10T20:58:02.829 INFO:teuthology.orchestra.run.smithi016.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:58:02.829 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T20:58:02.830 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T20:58:02.830 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T20:58:03.830 DEBUG:teuthology.parallel:result is None 2024-09-10T20:58:03.830 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T20:58:03.843 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T20:58:03.843 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T20:58:03.846 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T20:58:03.856 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T20:58:03.864 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T20:58:03.871 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T20:58:03.871 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T20:58:03.897 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T20:58:03.921 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T20:58:03.928 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T20:58:03.928 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T20:58:03.956 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T20:58:04.022 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:58:04.042 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:58:04.044 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:58:04.069 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:58:04.071 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T20:58:04.078 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T20:58:04.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T20:58:04.082 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T20:58:04.104 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T20:58:04.112 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T20:58:04.113 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T20:58:04.149 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T20:58:04.159 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:58:04.211 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:58:04.266 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T20:58:04.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T20:58:04.336 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:58:04.346 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:58:04.396 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T20:58:04.396 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T20:58:04.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-09-10T20:58:04.469 DEBUG:teuthology.orchestra.run.smithi052:> sudo service rsyslog restart 2024-09-10T20:58:04.597 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T20:58:04.604 INFO:teuthology.task.internal:Starting timer... 2024-09-10T20:58:04.605 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T20:58:04.615 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T20:58:04.624 DEBUG:teuthology.task.selinux:Excluding smithi016: OS 'ubuntu' does not support SELinux 2024-09-10T20:58:04.624 DEBUG:teuthology.task.selinux:Excluding smithi052: OS 'ubuntu' does not support SELinux 2024-09-10T20:58:04.624 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T20:58:04.624 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-10T20:58:04.624 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T20:58:04.624 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T20:58:04.663 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T20:58:04.676 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T20:58:04.677 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi052.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T21:06:29.283 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi052.front.sepia.ceph.com')] 2024-09-10T21:06:29.285 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-10T21:06:29.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:06:29.380 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-10T21:06:29.496 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-10T21:06:29.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-09-10T21:06:29.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:06:29.585 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-09-10T21:06:29.711 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-09-10T21:06:29.712 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T21:06:29.723 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T21:06:29.723 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T21:06:29.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:06:29.727 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T21:06:29.727 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:06:29.764 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-10T21:06:29.765 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: Command line: ntpd -gq 2024-09-10T21:06:29.765 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: ---------------------------------------------------- 2024-09-10T21:06:29.765 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: ntp-4 is maintained by Network Time Foundation, 2024-09-10T21:06:29.765 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-10T21:06:29.765 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: corporation. Support and training for ntp-4 are 2024-09-10T21:06:29.765 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: available at https://www.nwtime.org/support 2024-09-10T21:06:29.765 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: ---------------------------------------------------- 2024-09-10T21:06:29.766 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: proto: precision = 0.045 usec (-24) 2024-09-10T21:06:29.766 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: basedate set to 2022-02-04 2024-09-10T21:06:29.766 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: gps base set to 2022-02-06 (week 2196) 2024-09-10T21:06:29.766 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:06:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:06:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:06:29.767 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:06:29.768 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T21:06:29.768 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T21:06:29.768 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T21:06:29.768 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: Listen normally on 3 ens1f0 172.21.15.16:123 2024-09-10T21:06:29.768 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: Listen normally on 4 lo [::1]:123 2024-09-10T21:06:29.768 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1564%4]:123 2024-09-10T21:06:29.768 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:29 ntpd[18349]: Listening on routing socket on fd #22 for interface updates 2024-09-10T21:06:29.798 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-10T21:06:29.798 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: Command line: ntpd -gq 2024-09-10T21:06:29.799 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: ---------------------------------------------------- 2024-09-10T21:06:29.799 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: ntp-4 is maintained by Network Time Foundation, 2024-09-10T21:06:29.799 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-10T21:06:29.799 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: corporation. Support and training for ntp-4 are 2024-09-10T21:06:29.799 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: available at https://www.nwtime.org/support 2024-09-10T21:06:29.799 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: ---------------------------------------------------- 2024-09-10T21:06:29.799 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: proto: precision = 0.044 usec (-24) 2024-09-10T21:06:29.800 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: basedate set to 2022-02-04 2024-09-10T21:06:29.800 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: gps base set to 2022-02-06 (week 2196) 2024-09-10T21:06:29.800 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:06:29.800 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:06:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:06:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:06:29.802 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T21:06:29.802 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T21:06:29.802 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T21:06:29.802 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: Listen normally on 3 ens1f0 172.21.15.52:123 2024-09-10T21:06:29.802 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: Listen normally on 4 lo [::1]:123 2024-09-10T21:06:29.802 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15de%4]:123 2024-09-10T21:06:29.802 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:29 ntpd[18389]: Listening on routing socket on fd #22 for interface updates 2024-09-10T21:06:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:30 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:30 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:30.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:30 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:30.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:30 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:30.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:30 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:30.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:30 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:30.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:30 ntpd[18389]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:30.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:30 ntpd[18389]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:31 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:31 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:31 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:31 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:31 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:31 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:32.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:32.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:32.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:32.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:32.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:32.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:32.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:32.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:32.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:32.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:32.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:32.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:32.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:32.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:32 ntpd[18389]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:33 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:33 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:33 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:33 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:33 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:33 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:33 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:33 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:33 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:33 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:33 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:33 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:33.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:33 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:33.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:33 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:33.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:33 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:33.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:33 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:33.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:33 ntpd[18389]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:33.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:33 ntpd[18389]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:34.765 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:34 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:34.765 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:34 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:34.765 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:34 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:34.765 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:34 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:34.765 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:34 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:34.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:34 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:34.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:34 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:34.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:34 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:34.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:34 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:34.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:34 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:34.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:34 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:34.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:34 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:34.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:34 ntpd[18389]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:34.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:34 ntpd[18389]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:34.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:34 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:34.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:34 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:34.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:34 ntpd[18389]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:34.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:34 ntpd[18389]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:35 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:35 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:35 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:35 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:35 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:35 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:35 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:35 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:35 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:35 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:35 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:35 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:35.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:35 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:35.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:35 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:35.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:35 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:35.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:35 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:35.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:35 ntpd[18389]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:35.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:35 ntpd[18389]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:36.765 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:36 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:36.765 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:36 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:36.765 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:36 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:36.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:36 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:36.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:36 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:36.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:36 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:36.799 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:36.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:36.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:36.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:36.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:36.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:36.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:36.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:36.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:36.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:36.800 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:36.801 INFO:teuthology.orchestra.run.smithi052.stdout:10 Sep 21:06:36 ntpd[18389]: ntpd: time slew -0.003263 s 2024-09-10T21:06:36.801 INFO:teuthology.orchestra.run.smithi052.stdout:ntpd: time slew -0.003263s 2024-09-10T21:06:36.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:36.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T21:06:36.801 INFO:teuthology.orchestra.run.smithi052.stderr:10 Sep 21:06:36 ntpd[18389]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T21:06:36.908 INFO:teuthology.orchestra.run.smithi052.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T21:06:36.908 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================== 2024-09-10T21:06:36.908 INFO:teuthology.orchestra.run.smithi052.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:06:36.908 INFO:teuthology.orchestra.run.smithi052.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:06:36.908 INFO:teuthology.orchestra.run.smithi052.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:06:36.908 INFO:teuthology.orchestra.run.smithi052.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:06:37.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:37 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:37.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:37 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:37.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:37 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:37.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:37 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:37.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:37 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:06:37.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:37 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:06:37.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:37 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:06:37.766 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:37 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:06:37.766 INFO:teuthology.orchestra.run.smithi016.stdout:10 Sep 21:06:37 ntpd[18349]: ntpd: time slew -0.001121 s 2024-09-10T21:06:37.767 INFO:teuthology.orchestra.run.smithi016.stdout:ntpd: time slew -0.001121s 2024-09-10T21:06:37.767 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:37 ntpd[18349]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T21:06:37.767 INFO:teuthology.orchestra.run.smithi016.stderr:10 Sep 21:06:37 ntpd[18349]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T21:06:37.893 INFO:teuthology.orchestra.run.smithi016.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T21:06:37.894 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================== 2024-09-10T21:06:37.894 INFO:teuthology.orchestra.run.smithi016.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:06:37.894 INFO:teuthology.orchestra.run.smithi016.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:06:37.894 INFO:teuthology.orchestra.run.smithi016.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:06:37.894 INFO:teuthology.orchestra.run.smithi016.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:06:37.895 INFO:teuthology.run_tasks:Running task install... 2024-09-10T21:06:37.903 DEBUG:teuthology.task.install:project ceph 2024-09-10T21:06:37.903 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T21:06:37.903 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T21:06:37.903 INFO:teuthology.task.install:Using flavor: default 2024-09-10T21:06:37.908 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T21:06:37.908 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T21:06:37.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo apt-key list | grep Ceph 2024-09-10T21:06:37.909 DEBUG:teuthology.orchestra.run.smithi052:> sudo apt-key list | grep Ceph 2024-09-10T21:06:38.017 INFO:teuthology.orchestra.run.smithi052.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-10T21:06:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-10T21:06:38.070 INFO:teuthology.orchestra.run.smithi052.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T21:06:38.070 INFO:teuthology.orchestra.run.smithi052.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T21:06:38.071 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-10T21:06:38.071 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-10T21:06:38.071 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:06:38.097 INFO:teuthology.orchestra.run.smithi016.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T21:06:38.097 INFO:teuthology.orchestra.run.smithi016.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T21:06:38.098 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-10T21:06:38.098 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-10T21:06:38.099 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:06:38.221 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default/ 2024-09-10T21:06:38.221 INFO:teuthology.task.install.deb:Package version is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:06:38.237 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default/ 2024-09-10T21:06:38.237 INFO:teuthology.task.install.deb:Package version is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:06:38.339 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:06:38.339 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T21:06:38.356 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:06:38.356 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T21:06:38.360 DEBUG:teuthology.orchestra.run.smithi052:> sudo apt-get update 2024-09-10T21:06:38.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo apt-get update 2024-09-10T21:06:38.562 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-10T21:06:38.567 INFO:teuthology.orchestra.run.smithi052.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-10T21:06:38.569 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-10T21:06:38.586 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-10T21:06:38.599 INFO:teuthology.orchestra.run.smithi052.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy InRelease 2024-09-10T21:06:38.613 INFO:teuthology.orchestra.run.smithi016.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy InRelease 2024-09-10T21:06:38.620 INFO:teuthology.orchestra.run.smithi052.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-10T21:06:38.634 INFO:teuthology.orchestra.run.smithi016.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-10T21:06:38.642 INFO:teuthology.orchestra.run.smithi052.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-10T21:06:38.656 INFO:teuthology.orchestra.run.smithi016.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-10T21:06:38.663 INFO:teuthology.orchestra.run.smithi052.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-10T21:06:38.674 INFO:teuthology.orchestra.run.smithi052.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-10T21:06:38.678 INFO:teuthology.orchestra.run.smithi016.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-10T21:06:38.689 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-10T21:06:38.696 INFO:teuthology.orchestra.run.smithi052.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-10T21:06:38.710 INFO:teuthology.orchestra.run.smithi016.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-10T21:06:38.748 INFO:teuthology.orchestra.run.smithi052.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-10T21:06:38.829 INFO:teuthology.orchestra.run.smithi052.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-10T21:06:39.182 INFO:teuthology.orchestra.run.smithi016.stdout:Fetched 29.3 kB in 1s (51.2 kB/s) 2024-09-10T21:06:39.185 INFO:teuthology.orchestra.run.smithi052.stdout:Fetched 29.3 kB in 1s (49.7 kB/s) 2024-09-10T21:06:40.249 INFO:teuthology.orchestra.run.smithi016.stdout:Reading package lists... 2024-09-10T21:06:40.254 INFO:teuthology.orchestra.run.smithi052.stdout:Reading package lists... 2024-09-10T21:06:40.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4854-g8f219c47-1jammy cephadm=19.3.0-4854-g8f219c47-1jammy ceph-mds=19.3.0-4854-g8f219c47-1jammy ceph-mgr=19.3.0-4854-g8f219c47-1jammy ceph-common=19.3.0-4854-g8f219c47-1jammy ceph-fuse=19.3.0-4854-g8f219c47-1jammy ceph-test=19.3.0-4854-g8f219c47-1jammy ceph-volume=19.3.0-4854-g8f219c47-1jammy radosgw=19.3.0-4854-g8f219c47-1jammy python3-rados=19.3.0-4854-g8f219c47-1jammy python3-rgw=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy python3-rbd=19.3.0-4854-g8f219c47-1jammy libcephfs2=19.3.0-4854-g8f219c47-1jammy libcephfs-dev=19.3.0-4854-g8f219c47-1jammy librados2=19.3.0-4854-g8f219c47-1jammy librbd1=19.3.0-4854-g8f219c47-1jammy rbd-fuse=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy cephfs-shell=19.3.0-4854-g8f219c47-1jammy cephfs-top=19.3.0-4854-g8f219c47-1jammy cephfs-mirror=19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:06:40.269 DEBUG:teuthology.orchestra.run.smithi052:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4854-g8f219c47-1jammy cephadm=19.3.0-4854-g8f219c47-1jammy ceph-mds=19.3.0-4854-g8f219c47-1jammy ceph-mgr=19.3.0-4854-g8f219c47-1jammy ceph-common=19.3.0-4854-g8f219c47-1jammy ceph-fuse=19.3.0-4854-g8f219c47-1jammy ceph-test=19.3.0-4854-g8f219c47-1jammy ceph-volume=19.3.0-4854-g8f219c47-1jammy radosgw=19.3.0-4854-g8f219c47-1jammy python3-rados=19.3.0-4854-g8f219c47-1jammy python3-rgw=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy python3-rbd=19.3.0-4854-g8f219c47-1jammy libcephfs2=19.3.0-4854-g8f219c47-1jammy libcephfs-dev=19.3.0-4854-g8f219c47-1jammy librados2=19.3.0-4854-g8f219c47-1jammy librbd1=19.3.0-4854-g8f219c47-1jammy rbd-fuse=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy cephfs-shell=19.3.0-4854-g8f219c47-1jammy cephfs-top=19.3.0-4854-g8f219c47-1jammy cephfs-mirror=19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:06:40.342 INFO:teuthology.orchestra.run.smithi016.stdout:Reading package lists... 2024-09-10T21:06:40.347 INFO:teuthology.orchestra.run.smithi052.stdout:Reading package lists... 2024-09-10T21:06:40.528 INFO:teuthology.orchestra.run.smithi016.stdout:Building dependency tree... 2024-09-10T21:06:40.529 INFO:teuthology.orchestra.run.smithi016.stdout:Reading state information... 2024-09-10T21:06:40.535 INFO:teuthology.orchestra.run.smithi052.stdout:Building dependency tree... 2024-09-10T21:06:40.536 INFO:teuthology.orchestra.run.smithi052.stdout:Reading state information... 2024-09-10T21:06:40.724 INFO:teuthology.orchestra.run.smithi016.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:06:40.724 INFO:teuthology.orchestra.run.smithi016.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:06:40.724 INFO:teuthology.orchestra.run.smithi016.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:06:40.725 INFO:teuthology.orchestra.run.smithi016.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:06:40.725 INFO:teuthology.orchestra.run.smithi016.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:06:40.725 INFO:teuthology.orchestra.run.smithi016.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:06:40.725 INFO:teuthology.orchestra.run.smithi016.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:06:40.725 INFO:teuthology.orchestra.run.smithi016.stdout:The following additional packages will be installed: 2024-09-10T21:06:40.726 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T21:06:40.726 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-10T21:06:40.726 INFO:teuthology.orchestra.run.smithi016.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:06:40.727 INFO:teuthology.orchestra.run.smithi016.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-10T21:06:40.727 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-10T21:06:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-10T21:06:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:06:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:06:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-10T21:06:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-10T21:06:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-10T21:06:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-10T21:06:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-10T21:06:40.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout:Suggested packages: 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-10T21:06:40.731 INFO:teuthology.orchestra.run.smithi016.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-10T21:06:40.732 INFO:teuthology.orchestra.run.smithi016.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-10T21:06:40.732 INFO:teuthology.orchestra.run.smithi016.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T21:06:40.732 INFO:teuthology.orchestra.run.smithi016.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-10T21:06:40.732 INFO:teuthology.orchestra.run.smithi016.stdout:Recommended packages: 2024-09-10T21:06:40.732 INFO:teuthology.orchestra.run.smithi016.stdout: btrfs-tools 2024-09-10T21:06:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:06:40.734 INFO:teuthology.orchestra.run.smithi052.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:06:40.734 INFO:teuthology.orchestra.run.smithi052.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:06:40.734 INFO:teuthology.orchestra.run.smithi052.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:06:40.734 INFO:teuthology.orchestra.run.smithi052.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:06:40.734 INFO:teuthology.orchestra.run.smithi052.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:06:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:06:40.734 INFO:teuthology.orchestra.run.smithi052.stdout:The following additional packages will be installed: 2024-09-10T21:06:40.734 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T21:06:40.734 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-10T21:06:40.734 INFO:teuthology.orchestra.run.smithi052.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:06:40.734 INFO:teuthology.orchestra.run.smithi052.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-10T21:06:40.734 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-10T21:06:40.735 INFO:teuthology.orchestra.run.smithi052.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-10T21:06:40.735 INFO:teuthology.orchestra.run.smithi052.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:06:40.735 INFO:teuthology.orchestra.run.smithi052.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:06:40.735 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-10T21:06:40.735 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-10T21:06:40.735 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-10T21:06:40.735 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-10T21:06:40.735 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-10T21:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-10T21:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T21:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-10T21:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-10T21:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-10T21:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-10T21:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-10T21:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-10T21:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-10T21:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-10T21:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-10T21:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout:Suggested packages: 2024-09-10T21:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-10T21:06:40.737 INFO:teuthology.orchestra.run.smithi052.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-10T21:06:40.737 INFO:teuthology.orchestra.run.smithi052.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-10T21:06:40.737 INFO:teuthology.orchestra.run.smithi052.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-10T21:06:40.737 INFO:teuthology.orchestra.run.smithi052.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-10T21:06:40.737 INFO:teuthology.orchestra.run.smithi052.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T21:06:40.737 INFO:teuthology.orchestra.run.smithi052.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-10T21:06:40.737 INFO:teuthology.orchestra.run.smithi052.stdout:Recommended packages: 2024-09-10T21:06:40.737 INFO:teuthology.orchestra.run.smithi052.stdout: btrfs-tools 2024-09-10T21:06:40.893 INFO:teuthology.orchestra.run.smithi052.stdout:The following NEW packages will be installed: 2024-09-10T21:06:40.893 INFO:teuthology.orchestra.run.smithi052.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T21:06:40.894 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T21:06:40.894 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T21:06:40.894 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-10T21:06:40.894 INFO:teuthology.orchestra.run.smithi052.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:06:40.894 INFO:teuthology.orchestra.run.smithi052.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-10T21:06:40.894 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-10T21:06:40.897 INFO:teuthology.orchestra.run.smithi052.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-10T21:06:40.897 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:06:40.897 INFO:teuthology.orchestra.run.smithi052.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:06:40.897 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-10T21:06:40.897 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-10T21:06:40.897 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-10T21:06:40.897 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-10T21:06:40.898 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-10T21:06:40.898 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-10T21:06:40.898 INFO:teuthology.orchestra.run.smithi052.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-10T21:06:40.898 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-10T21:06:40.898 INFO:teuthology.orchestra.run.smithi052.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T21:06:40.898 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-10T21:06:40.898 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-10T21:06:40.898 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-10T21:06:40.898 INFO:teuthology.orchestra.run.smithi052.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T21:06:40.898 INFO:teuthology.orchestra.run.smithi052.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-10T21:06:40.898 INFO:teuthology.orchestra.run.smithi052.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-10T21:06:40.898 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-10T21:06:40.899 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-10T21:06:40.899 INFO:teuthology.orchestra.run.smithi052.stdout: socat unzip xmlstarlet zip 2024-09-10T21:06:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:The following NEW packages will be installed: 2024-09-10T21:06:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T21:06:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T21:06:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T21:06:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-10T21:06:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-10T21:06:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T21:06:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-10T21:06:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-10T21:06:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-10T21:06:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T21:06:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-10T21:06:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-10T21:06:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-10T21:06:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-10T21:06:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: socat unzip xmlstarlet zip 2024-09-10T21:06:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-10T21:06:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:Need to get 251 MB of archives. 2024-09-10T21:06:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:After this operation, 936 MB of additional disk space will be used. 2024-09-10T21:06:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-10T21:06:40.986 INFO:teuthology.orchestra.run.smithi016.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-10T21:06:40.996 INFO:teuthology.orchestra.run.smithi016.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-10T21:06:41.038 INFO:teuthology.orchestra.run.smithi016.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-10T21:06:41.049 INFO:teuthology.orchestra.run.smithi016.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4854-g8f219c47-1jammy [3,633 kB] 2024-09-10T21:06:41.049 INFO:teuthology.orchestra.run.smithi052.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-10T21:06:41.049 INFO:teuthology.orchestra.run.smithi052.stdout:Need to get 251 MB of archives. 2024-09-10T21:06:41.049 INFO:teuthology.orchestra.run.smithi052.stdout:After this operation, 936 MB of additional disk space will be used. 2024-09-10T21:06:41.050 INFO:teuthology.orchestra.run.smithi052.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4854-g8f219c47-1jammy [3,633 kB] 2024-09-10T21:06:41.059 INFO:teuthology.orchestra.run.smithi016.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-10T21:06:41.064 INFO:teuthology.orchestra.run.smithi016.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-10T21:06:41.076 INFO:teuthology.orchestra.run.smithi016.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-10T21:06:41.093 INFO:teuthology.orchestra.run.smithi052.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-10T21:06:41.115 INFO:teuthology.orchestra.run.smithi016.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-10T21:06:41.117 INFO:teuthology.orchestra.run.smithi016.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-10T21:06:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-10T21:06:41.127 INFO:teuthology.orchestra.run.smithi016.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-10T21:06:41.127 INFO:teuthology.orchestra.run.smithi016.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-10T21:06:41.128 INFO:teuthology.orchestra.run.smithi016.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-10T21:06:41.134 INFO:teuthology.orchestra.run.smithi016.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-10T21:06:41.136 INFO:teuthology.orchestra.run.smithi016.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-10T21:06:41.137 INFO:teuthology.orchestra.run.smithi016.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-10T21:06:41.139 INFO:teuthology.orchestra.run.smithi016.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-10T21:06:41.151 INFO:teuthology.orchestra.run.smithi016.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-10T21:06:41.153 INFO:teuthology.orchestra.run.smithi016.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-10T21:06:41.155 INFO:teuthology.orchestra.run.smithi016.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-10T21:06:41.156 INFO:teuthology.orchestra.run.smithi016.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-10T21:06:41.157 INFO:teuthology.orchestra.run.smithi016.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-10T21:06:41.168 INFO:teuthology.orchestra.run.smithi016.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-10T21:06:41.169 INFO:teuthology.orchestra.run.smithi016.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-10T21:06:41.169 INFO:teuthology.orchestra.run.smithi016.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-10T21:06:41.170 INFO:teuthology.orchestra.run.smithi016.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-10T21:06:41.170 INFO:teuthology.orchestra.run.smithi016.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-10T21:06:41.175 INFO:teuthology.orchestra.run.smithi052.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-10T21:06:41.186 INFO:teuthology.orchestra.run.smithi016.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-10T21:06:41.186 INFO:teuthology.orchestra.run.smithi016.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-10T21:06:41.187 INFO:teuthology.orchestra.run.smithi016.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-10T21:06:41.189 INFO:teuthology.orchestra.run.smithi016.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-10T21:06:41.189 INFO:teuthology.orchestra.run.smithi016.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-10T21:06:41.203 INFO:teuthology.orchestra.run.smithi016.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-10T21:06:41.203 INFO:teuthology.orchestra.run.smithi016.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-10T21:06:41.204 INFO:teuthology.orchestra.run.smithi016.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T21:06:41.205 INFO:teuthology.orchestra.run.smithi016.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-10T21:06:41.205 INFO:teuthology.orchestra.run.smithi016.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-10T21:06:41.220 INFO:teuthology.orchestra.run.smithi016.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-10T21:06:41.221 INFO:teuthology.orchestra.run.smithi016.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-10T21:06:41.221 INFO:teuthology.orchestra.run.smithi052.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-10T21:06:41.221 INFO:teuthology.orchestra.run.smithi016.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-10T21:06:41.226 INFO:teuthology.orchestra.run.smithi016.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-10T21:06:41.226 INFO:teuthology.orchestra.run.smithi016.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-10T21:06:41.237 INFO:teuthology.orchestra.run.smithi016.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-10T21:06:41.238 INFO:teuthology.orchestra.run.smithi016.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-10T21:06:41.239 INFO:teuthology.orchestra.run.smithi016.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-10T21:06:41.240 INFO:teuthology.orchestra.run.smithi016.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-10T21:06:41.241 INFO:teuthology.orchestra.run.smithi016.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-10T21:06:41.242 INFO:teuthology.orchestra.run.smithi052.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4854-g8f219c47-1jammy [3,330 kB] 2024-09-10T21:06:41.254 INFO:teuthology.orchestra.run.smithi016.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-10T21:06:41.255 INFO:teuthology.orchestra.run.smithi016.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-10T21:06:41.257 INFO:teuthology.orchestra.run.smithi016.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-10T21:06:41.257 INFO:teuthology.orchestra.run.smithi016.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-10T21:06:41.258 INFO:teuthology.orchestra.run.smithi016.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-10T21:06:41.260 INFO:teuthology.orchestra.run.smithi016.stdout:Get:54 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4854-g8f219c47-1jammy [3,330 kB] 2024-09-10T21:06:41.271 INFO:teuthology.orchestra.run.smithi016.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-10T21:06:41.316 INFO:teuthology.orchestra.run.smithi016.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-10T21:06:41.316 INFO:teuthology.orchestra.run.smithi016.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-10T21:06:41.318 INFO:teuthology.orchestra.run.smithi016.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T21:06:41.318 INFO:teuthology.orchestra.run.smithi016.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-10T21:06:41.319 INFO:teuthology.orchestra.run.smithi016.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-10T21:06:41.338 INFO:teuthology.orchestra.run.smithi016.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-10T21:06:41.340 INFO:teuthology.orchestra.run.smithi016.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-10T21:06:41.340 INFO:teuthology.orchestra.run.smithi016.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-10T21:06:41.341 INFO:teuthology.orchestra.run.smithi052.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4854-g8f219c47-1jammy [721 kB] 2024-09-10T21:06:41.352 INFO:teuthology.orchestra.run.smithi052.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4854-g8f219c47-1jammy [356 kB] 2024-09-10T21:06:41.357 INFO:teuthology.orchestra.run.smithi016.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-10T21:06:41.357 INFO:teuthology.orchestra.run.smithi016.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-10T21:06:41.358 INFO:teuthology.orchestra.run.smithi016.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-10T21:06:41.358 INFO:teuthology.orchestra.run.smithi052.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4854-g8f219c47-1jammy [32.8 kB] 2024-09-10T21:06:41.358 INFO:teuthology.orchestra.run.smithi016.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-10T21:06:41.359 INFO:teuthology.orchestra.run.smithi052.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4854-g8f219c47-1jammy [180 kB] 2024-09-10T21:06:41.359 INFO:teuthology.orchestra.run.smithi016.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-10T21:06:41.360 INFO:teuthology.orchestra.run.smithi016.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-10T21:06:41.368 INFO:teuthology.orchestra.run.smithi016.stdout:Get:70 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4854-g8f219c47-1jammy [721 kB] 2024-09-10T21:06:41.369 INFO:teuthology.orchestra.run.smithi016.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-10T21:06:41.369 INFO:teuthology.orchestra.run.smithi016.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-10T21:06:41.373 INFO:teuthology.orchestra.run.smithi016.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-10T21:06:41.378 INFO:teuthology.orchestra.run.smithi016.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4854-g8f219c47-1jammy [356 kB] 2024-09-10T21:06:41.384 INFO:teuthology.orchestra.run.smithi016.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4854-g8f219c47-1jammy [32.8 kB] 2024-09-10T21:06:41.385 INFO:teuthology.orchestra.run.smithi016.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4854-g8f219c47-1jammy [180 kB] 2024-09-10T21:06:41.385 INFO:teuthology.orchestra.run.smithi016.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-10T21:06:41.387 INFO:teuthology.orchestra.run.smithi016.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-10T21:06:41.388 INFO:teuthology.orchestra.run.smithi016.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-10T21:06:41.388 INFO:teuthology.orchestra.run.smithi052.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4854-g8f219c47-1jammy [71.2 kB] 2024-09-10T21:06:41.389 INFO:teuthology.orchestra.run.smithi052.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4854-g8f219c47-1jammy [338 kB] 2024-09-10T21:06:41.391 INFO:teuthology.orchestra.run.smithi016.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-10T21:06:41.394 INFO:teuthology.orchestra.run.smithi016.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-10T21:06:41.394 INFO:teuthology.orchestra.run.smithi016.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-10T21:06:41.395 INFO:teuthology.orchestra.run.smithi052.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4854-g8f219c47-1jammy [7,988 kB] 2024-09-10T21:06:41.395 INFO:teuthology.orchestra.run.smithi016.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-10T21:06:41.403 INFO:teuthology.orchestra.run.smithi016.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-10T21:06:41.407 INFO:teuthology.orchestra.run.smithi016.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-10T21:06:41.408 INFO:teuthology.orchestra.run.smithi016.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-10T21:06:41.415 INFO:teuthology.orchestra.run.smithi016.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4854-g8f219c47-1jammy [71.2 kB] 2024-09-10T21:06:41.416 INFO:teuthology.orchestra.run.smithi016.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4854-g8f219c47-1jammy [338 kB] 2024-09-10T21:06:41.420 INFO:teuthology.orchestra.run.smithi016.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-10T21:06:41.420 INFO:teuthology.orchestra.run.smithi016.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T21:06:41.420 INFO:teuthology.orchestra.run.smithi052.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-10T21:06:41.421 INFO:teuthology.orchestra.run.smithi016.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-10T21:06:41.421 INFO:teuthology.orchestra.run.smithi016.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-10T21:06:41.422 INFO:teuthology.orchestra.run.smithi016.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4854-g8f219c47-1jammy [7,988 kB] 2024-09-10T21:06:41.423 INFO:teuthology.orchestra.run.smithi016.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-10T21:06:41.424 INFO:teuthology.orchestra.run.smithi016.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-10T21:06:41.430 INFO:teuthology.orchestra.run.smithi016.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-10T21:06:41.436 INFO:teuthology.orchestra.run.smithi016.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-10T21:06:41.437 INFO:teuthology.orchestra.run.smithi016.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-10T21:06:41.439 INFO:teuthology.orchestra.run.smithi016.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-10T21:06:41.453 INFO:teuthology.orchestra.run.smithi016.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-10T21:06:41.511 INFO:teuthology.orchestra.run.smithi052.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-10T21:06:41.524 INFO:teuthology.orchestra.run.smithi052.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-10T21:06:41.574 INFO:teuthology.orchestra.run.smithi052.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-10T21:06:41.648 INFO:teuthology.orchestra.run.smithi052.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4854-g8f219c47-1jammy [112 kB] 2024-09-10T21:06:41.649 INFO:teuthology.orchestra.run.smithi052.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4854-g8f219c47-1jammy [473 kB] 2024-09-10T21:06:41.655 INFO:teuthology.orchestra.run.smithi052.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4854-g8f219c47-1jammy [28.3 MB] 2024-09-10T21:06:41.681 INFO:teuthology.orchestra.run.smithi016.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4854-g8f219c47-1jammy [112 kB] 2024-09-10T21:06:41.682 INFO:teuthology.orchestra.run.smithi016.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4854-g8f219c47-1jammy [473 kB] 2024-09-10T21:06:41.687 INFO:teuthology.orchestra.run.smithi016.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4854-g8f219c47-1jammy [28.3 MB] 2024-09-10T21:06:41.748 INFO:teuthology.orchestra.run.smithi052.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-10T21:06:41.755 INFO:teuthology.orchestra.run.smithi052.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-10T21:06:41.783 INFO:teuthology.orchestra.run.smithi052.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-10T21:06:41.791 INFO:teuthology.orchestra.run.smithi052.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-10T21:06:41.794 INFO:teuthology.orchestra.run.smithi052.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-10T21:06:41.795 INFO:teuthology.orchestra.run.smithi052.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-10T21:06:41.812 INFO:teuthology.orchestra.run.smithi052.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-10T21:06:41.816 INFO:teuthology.orchestra.run.smithi052.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-10T21:06:41.820 INFO:teuthology.orchestra.run.smithi052.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-10T21:06:41.829 INFO:teuthology.orchestra.run.smithi052.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-10T21:06:41.905 INFO:teuthology.orchestra.run.smithi052.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-10T21:06:41.907 INFO:teuthology.orchestra.run.smithi052.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-10T21:06:41.909 INFO:teuthology.orchestra.run.smithi052.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-10T21:06:41.910 INFO:teuthology.orchestra.run.smithi052.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-10T21:06:41.911 INFO:teuthology.orchestra.run.smithi052.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-10T21:06:41.911 INFO:teuthology.orchestra.run.smithi052.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-10T21:06:41.912 INFO:teuthology.orchestra.run.smithi052.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-10T21:06:41.913 INFO:teuthology.orchestra.run.smithi052.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-10T21:06:41.913 INFO:teuthology.orchestra.run.smithi052.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-10T21:06:41.993 INFO:teuthology.orchestra.run.smithi052.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-10T21:06:41.993 INFO:teuthology.orchestra.run.smithi052.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-10T21:06:41.993 INFO:teuthology.orchestra.run.smithi052.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-10T21:06:42.074 INFO:teuthology.orchestra.run.smithi052.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-10T21:06:42.076 INFO:teuthology.orchestra.run.smithi052.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-10T21:06:42.077 INFO:teuthology.orchestra.run.smithi052.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-10T21:06:42.077 INFO:teuthology.orchestra.run.smithi052.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-10T21:06:42.077 INFO:teuthology.orchestra.run.smithi052.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-10T21:06:42.078 INFO:teuthology.orchestra.run.smithi052.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T21:06:42.079 INFO:teuthology.orchestra.run.smithi052.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-10T21:06:42.156 INFO:teuthology.orchestra.run.smithi052.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-10T21:06:42.157 INFO:teuthology.orchestra.run.smithi052.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-10T21:06:42.157 INFO:teuthology.orchestra.run.smithi052.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-10T21:06:42.238 INFO:teuthology.orchestra.run.smithi052.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-10T21:06:42.242 INFO:teuthology.orchestra.run.smithi052.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-10T21:06:42.242 INFO:teuthology.orchestra.run.smithi052.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-10T21:06:42.243 INFO:teuthology.orchestra.run.smithi052.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-10T21:06:42.244 INFO:teuthology.orchestra.run.smithi052.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-10T21:06:42.245 INFO:teuthology.orchestra.run.smithi052.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-10T21:06:42.246 INFO:teuthology.orchestra.run.smithi052.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-10T21:06:42.319 INFO:teuthology.orchestra.run.smithi052.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-10T21:06:42.320 INFO:teuthology.orchestra.run.smithi052.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-10T21:06:42.320 INFO:teuthology.orchestra.run.smithi052.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-10T21:06:42.400 INFO:teuthology.orchestra.run.smithi052.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-10T21:06:42.400 INFO:teuthology.orchestra.run.smithi052.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-10T21:06:42.401 INFO:teuthology.orchestra.run.smithi052.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-10T21:06:42.403 INFO:teuthology.orchestra.run.smithi052.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-10T21:06:42.511 INFO:teuthology.orchestra.run.smithi052.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-10T21:06:42.512 INFO:teuthology.orchestra.run.smithi052.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-10T21:06:42.514 INFO:teuthology.orchestra.run.smithi052.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T21:06:42.514 INFO:teuthology.orchestra.run.smithi052.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-10T21:06:42.516 INFO:teuthology.orchestra.run.smithi052.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-10T21:06:42.587 INFO:teuthology.orchestra.run.smithi052.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-10T21:06:42.589 INFO:teuthology.orchestra.run.smithi052.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-10T21:06:42.589 INFO:teuthology.orchestra.run.smithi052.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-10T21:06:42.607 INFO:teuthology.orchestra.run.smithi052.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4854-g8f219c47-1jammy [5,206 kB] 2024-09-10T21:06:42.633 INFO:teuthology.orchestra.run.smithi016.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4854-g8f219c47-1jammy [5,206 kB] 2024-09-10T21:06:42.659 INFO:teuthology.orchestra.run.smithi052.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-10T21:06:42.659 INFO:teuthology.orchestra.run.smithi052.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-10T21:06:42.660 INFO:teuthology.orchestra.run.smithi052.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-10T21:06:42.661 INFO:teuthology.orchestra.run.smithi052.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-10T21:06:42.662 INFO:teuthology.orchestra.run.smithi052.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-10T21:06:42.662 INFO:teuthology.orchestra.run.smithi052.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-10T21:06:42.726 INFO:teuthology.orchestra.run.smithi052.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-10T21:06:42.726 INFO:teuthology.orchestra.run.smithi052.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-10T21:06:42.730 INFO:teuthology.orchestra.run.smithi052.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-10T21:06:42.774 INFO:teuthology.orchestra.run.smithi052.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4854-g8f219c47-1jammy [251 kB] 2024-09-10T21:06:42.779 INFO:teuthology.orchestra.run.smithi052.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4854-g8f219c47-1jammy [124 kB] 2024-09-10T21:06:42.781 INFO:teuthology.orchestra.run.smithi052.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4854-g8f219c47-1jammy [1,562 kB] 2024-09-10T21:06:42.798 INFO:teuthology.orchestra.run.smithi016.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4854-g8f219c47-1jammy [251 kB] 2024-09-10T21:06:42.801 INFO:teuthology.orchestra.run.smithi016.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4854-g8f219c47-1jammy [124 kB] 2024-09-10T21:06:42.807 INFO:teuthology.orchestra.run.smithi052.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-10T21:06:42.809 INFO:teuthology.orchestra.run.smithi052.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-10T21:06:42.809 INFO:teuthology.orchestra.run.smithi052.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-10T21:06:42.810 INFO:teuthology.orchestra.run.smithi016.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4854-g8f219c47-1jammy [1,562 kB] 2024-09-10T21:06:42.813 INFO:teuthology.orchestra.run.smithi052.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-10T21:06:42.816 INFO:teuthology.orchestra.run.smithi052.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-10T21:06:42.817 INFO:teuthology.orchestra.run.smithi052.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-10T21:06:42.818 INFO:teuthology.orchestra.run.smithi052.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-10T21:06:42.847 INFO:teuthology.orchestra.run.smithi052.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4854-g8f219c47-1jammy [6,308 kB] 2024-09-10T21:06:42.865 INFO:teuthology.orchestra.run.smithi016.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4854-g8f219c47-1jammy [6,308 kB] 2024-09-10T21:06:42.888 INFO:teuthology.orchestra.run.smithi052.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-10T21:06:42.893 INFO:teuthology.orchestra.run.smithi052.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-10T21:06:42.894 INFO:teuthology.orchestra.run.smithi052.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-10T21:06:42.970 INFO:teuthology.orchestra.run.smithi052.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-10T21:06:42.970 INFO:teuthology.orchestra.run.smithi052.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T21:06:42.971 INFO:teuthology.orchestra.run.smithi052.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-10T21:06:42.972 INFO:teuthology.orchestra.run.smithi052.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-10T21:06:42.973 INFO:teuthology.orchestra.run.smithi052.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-10T21:06:42.974 INFO:teuthology.orchestra.run.smithi052.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-10T21:06:42.981 INFO:teuthology.orchestra.run.smithi052.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-10T21:06:43.051 INFO:teuthology.orchestra.run.smithi052.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4854-g8f219c47-1jammy [23.1 MB] 2024-09-10T21:06:43.051 INFO:teuthology.orchestra.run.smithi052.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-10T21:06:43.052 INFO:teuthology.orchestra.run.smithi052.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-10T21:06:43.054 INFO:teuthology.orchestra.run.smithi052.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-10T21:06:43.069 INFO:teuthology.orchestra.run.smithi016.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4854-g8f219c47-1jammy [23.1 MB] 2024-09-10T21:06:43.133 INFO:teuthology.orchestra.run.smithi052.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-10T21:06:43.813 INFO:teuthology.orchestra.run.smithi052.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4854-g8f219c47-1jammy [14.0 kB] 2024-09-10T21:06:43.813 INFO:teuthology.orchestra.run.smithi052.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4854-g8f219c47-1jammy [880 kB] 2024-09-10T21:06:43.827 INFO:teuthology.orchestra.run.smithi016.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4854-g8f219c47-1jammy [14.0 kB] 2024-09-10T21:06:43.827 INFO:teuthology.orchestra.run.smithi016.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4854-g8f219c47-1jammy [880 kB] 2024-09-10T21:06:43.846 INFO:teuthology.orchestra.run.smithi052.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4854-g8f219c47-1jammy [2,454 kB] 2024-09-10T21:06:43.852 INFO:teuthology.orchestra.run.smithi016.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4854-g8f219c47-1jammy [2,454 kB] 2024-09-10T21:06:43.920 INFO:teuthology.orchestra.run.smithi052.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4854-g8f219c47-1jammy [813 kB] 2024-09-10T21:06:43.928 INFO:teuthology.orchestra.run.smithi016.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4854-g8f219c47-1jammy [813 kB] 2024-09-10T21:06:43.950 INFO:teuthology.orchestra.run.smithi016.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4854-g8f219c47-1jammy [160 kB] 2024-09-10T21:06:43.953 INFO:teuthology.orchestra.run.smithi052.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4854-g8f219c47-1jammy [160 kB] 2024-09-10T21:06:43.956 INFO:teuthology.orchestra.run.smithi052.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4854-g8f219c47-1jammy [61.4 MB] 2024-09-10T21:06:43.968 INFO:teuthology.orchestra.run.smithi016.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4854-g8f219c47-1jammy [61.4 MB] 2024-09-10T21:06:45.988 INFO:teuthology.orchestra.run.smithi052.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4854-g8f219c47-1jammy [8,625 kB] 2024-09-10T21:06:46.002 INFO:teuthology.orchestra.run.smithi016.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4854-g8f219c47-1jammy [8,625 kB] 2024-09-10T21:06:46.253 INFO:teuthology.orchestra.run.smithi052.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4854-g8f219c47-1jammy [14.2 kB] 2024-09-10T21:06:46.254 INFO:teuthology.orchestra.run.smithi052.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4854-g8f219c47-1jammy [55.8 MB] 2024-09-10T21:06:46.265 INFO:teuthology.orchestra.run.smithi016.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4854-g8f219c47-1jammy [14.2 kB] 2024-09-10T21:06:46.265 INFO:teuthology.orchestra.run.smithi016.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4854-g8f219c47-1jammy [55.8 MB] 2024-09-10T21:06:48.084 INFO:teuthology.orchestra.run.smithi052.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4854-g8f219c47-1jammy [130 kB] 2024-09-10T21:06:48.091 INFO:teuthology.orchestra.run.smithi052.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4854-g8f219c47-1jammy [233 kB] 2024-09-10T21:06:48.095 INFO:teuthology.orchestra.run.smithi052.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4854-g8f219c47-1jammy [33.9 kB] 2024-09-10T21:06:48.096 INFO:teuthology.orchestra.run.smithi052.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4854-g8f219c47-1jammy [23.4 kB] 2024-09-10T21:06:48.096 INFO:teuthology.orchestra.run.smithi052.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4854-g8f219c47-1jammy [39.0 kB] 2024-09-10T21:06:48.097 INFO:teuthology.orchestra.run.smithi052.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4854-g8f219c47-1jammy [15.9 MB] 2024-09-10T21:06:48.104 INFO:teuthology.orchestra.run.smithi016.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4854-g8f219c47-1jammy [130 kB] 2024-09-10T21:06:48.105 INFO:teuthology.orchestra.run.smithi016.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4854-g8f219c47-1jammy [233 kB] 2024-09-10T21:06:48.107 INFO:teuthology.orchestra.run.smithi016.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4854-g8f219c47-1jammy [33.9 kB] 2024-09-10T21:06:48.108 INFO:teuthology.orchestra.run.smithi016.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4854-g8f219c47-1jammy [23.4 kB] 2024-09-10T21:06:48.108 INFO:teuthology.orchestra.run.smithi016.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4854-g8f219c47-1jammy [39.0 kB] 2024-09-10T21:06:48.108 INFO:teuthology.orchestra.run.smithi016.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4854-g8f219c47-1jammy [15.9 MB] 2024-09-10T21:06:48.606 INFO:teuthology.orchestra.run.smithi052.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4854-g8f219c47-1jammy [91.2 kB] 2024-09-10T21:06:48.628 INFO:teuthology.orchestra.run.smithi016.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4854-g8f219c47-1jammy [91.2 kB] 2024-09-10T21:06:49.285 INFO:teuthology.orchestra.run.smithi052.stdout:Fetched 251 MB in 8s (32.7 MB/s) 2024-09-10T21:06:49.307 INFO:teuthology.orchestra.run.smithi016.stdout:Fetched 251 MB in 8s (32.6 MB/s) 2024-09-10T21:06:49.440 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-10T21:06:49.440 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-10T21:06:49.507 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-10T21:06:49.508 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-10T21:06:49.511 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-10T21:06:49.511 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-10T21:06:49.555 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-10T21:06:49.555 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-10T21:06:49.816 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T21:06:49.830 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-10T21:06:49.866 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:06:49.942 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T21:06:49.956 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-10T21:06:49.992 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:06:50.285 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-cmd2. 2024-09-10T21:06:50.300 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-10T21:06:50.303 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cmd2. 2024-09-10T21:06:50.317 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-10T21:06:50.352 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-10T21:06:50.361 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-10T21:06:50.697 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-10T21:06:50.712 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-10T21:06:50.731 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-10T21:06:50.746 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-10T21:06:50.797 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:06:50.831 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:06:51.133 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T21:06:51.149 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-10T21:06:51.169 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T21:06:51.184 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-10T21:06:51.192 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:06:51.227 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:06:51.503 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T21:06:51.519 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-10T21:06:51.553 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:06:51.555 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T21:06:51.570 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-10T21:06:51.604 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:06:52.007 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T21:06:52.023 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:06:52.067 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T21:06:52.073 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:06:52.083 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:06:52.134 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:06:52.564 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T21:06:52.579 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:06:52.629 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:06:52.827 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T21:06:52.842 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:06:52.885 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:06:52.940 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T21:06:52.955 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:06:52.998 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:06:53.213 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T21:06:53.229 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:06:53.271 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:06:53.343 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-10T21:06:53.359 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-10T21:06:53.404 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:06:53.616 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-10T21:06:53.631 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-10T21:06:53.674 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:06:53.738 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package librados2. 2024-09-10T21:06:53.754 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../010-librados2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:06:53.796 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:54.019 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package librados2. 2024-09-10T21:06:54.034 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../010-librados2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:06:54.077 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:54.224 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libnbd0. 2024-09-10T21:06:54.240 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-10T21:06:54.274 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-10T21:06:54.472 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libnbd0. 2024-09-10T21:06:54.487 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-10T21:06:54.521 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-10T21:06:54.595 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package librbd1. 2024-09-10T21:06:54.610 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../012-librbd1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:06:54.652 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:54.824 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package librbd1. 2024-09-10T21:06:54.840 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../012-librbd1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:06:54.883 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:55.038 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T21:06:55.054 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:06:55.096 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:55.319 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T21:06:55.330 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:06:55.365 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-rados. 2024-09-10T21:06:55.369 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:55.381 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:06:55.415 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:55.630 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-rados. 2024-09-10T21:06:55.645 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:06:55.680 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:55.752 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T21:06:55.767 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:06:55.810 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:56.007 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T21:06:56.020 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:06:56.057 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:56.063 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T21:06:56.078 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:06:56.121 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:56.310 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T21:06:56.322 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:06:56.393 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:56.432 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T21:06:56.448 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:06:56.490 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:56.704 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T21:06:56.717 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:06:56.754 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:56.829 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T21:06:56.845 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-10T21:06:56.887 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-10T21:06:57.099 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T21:06:57.111 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-10T21:06:57.148 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T21:06:57.149 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-10T21:06:57.164 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:06:57.207 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:57.376 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T21:06:57.388 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:06:57.418 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:06:57.551 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T21:06:57.567 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-10T21:06:57.628 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:06:57.813 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T21:06:57.825 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-10T21:06:57.863 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:06:58.140 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-10T21:06:58.152 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-10T21:06:58.154 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-10T21:06:58.169 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-10T21:06:58.190 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:06:58.204 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:06:58.482 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-10T21:06:58.485 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-10T21:06:58.497 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-10T21:06:58.497 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-10T21:06:58.535 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:06:58.540 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:06:58.830 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package lua5.1. 2024-09-10T21:06:58.842 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-10T21:06:58.871 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:06:58.893 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package lua5.1. 2024-09-10T21:06:58.909 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-10T21:06:58.951 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:06:59.174 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package lua-any. 2024-09-10T21:06:59.187 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-10T21:06:59.224 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-10T21:06:59.246 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package lua-any. 2024-09-10T21:06:59.262 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-10T21:06:59.296 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-10T21:06:59.502 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package zip. 2024-09-10T21:06:59.514 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-10T21:06:59.532 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package zip. 2024-09-10T21:06:59.548 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-10T21:06:59.551 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking zip (3.0-12build2) ... 2024-09-10T21:06:59.590 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking zip (3.0-12build2) ... 2024-09-10T21:06:59.839 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package unzip. 2024-09-10T21:06:59.851 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-10T21:06:59.902 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package unzip. 2024-09-10T21:06:59.917 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-10T21:06:59.919 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:06:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:07:00.300 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package luarocks. 2024-09-10T21:07:00.312 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-10T21:07:00.322 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package luarocks. 2024-09-10T21:07:00.337 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-10T21:07:00.350 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:07:00.372 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:07:00.730 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package librgw2. 2024-09-10T21:07:00.744 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../028-librgw2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:00.750 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package librgw2. 2024-09-10T21:07:00.761 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../028-librgw2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:00.780 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:00.791 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:01.299 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T21:07:01.302 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T21:07:01.314 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:01.314 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:01.352 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:01.357 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:01.714 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-10T21:07:01.718 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-10T21:07:01.726 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-10T21:07:01.734 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-10T21:07:01.763 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:07:01.776 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:07:02.083 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T21:07:02.088 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T21:07:02.098 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:02.103 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:02.141 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:02.146 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:02.503 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-common. 2024-09-10T21:07:02.517 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:02.558 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-common. 2024-09-10T21:07:02.561 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:02.573 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:02.649 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:03.833 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-base. 2024-09-10T21:07:03.847 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:03.901 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:03.963 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-base. 2024-09-10T21:07:03.978 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:04.030 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:04.454 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-10T21:07:04.468 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-10T21:07:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:07:04.551 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-10T21:07:04.566 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-10T21:07:04.625 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:07:04.824 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-cheroot. 2024-09-10T21:07:04.838 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-10T21:07:04.882 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:07:04.920 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cheroot. 2024-09-10T21:07:04.936 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-10T21:07:04.978 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:07:05.152 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-10T21:07:05.165 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-10T21:07:05.201 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:07:05.289 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-10T21:07:05.305 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-10T21:07:05.348 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:07:05.513 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-10T21:07:05.521 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-10T21:07:05.563 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:07:05.667 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-10T21:07:05.683 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-10T21:07:05.725 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:07:05.849 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-10T21:07:05.860 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-10T21:07:05.899 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:07:06.104 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-10T21:07:06.119 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-10T21:07:06.170 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:07:06.201 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-tempora. 2024-09-10T21:07:06.216 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-10T21:07:06.251 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-10T21:07:06.471 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-portend. 2024-09-10T21:07:06.482 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-10T21:07:06.521 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-10T21:07:06.540 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-tempora. 2024-09-10T21:07:06.555 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-10T21:07:06.631 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-10T21:07:06.857 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-10T21:07:06.868 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-10T21:07:06.907 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-10T21:07:06.976 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-portend. 2024-09-10T21:07:06.991 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-10T21:07:07.034 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-10T21:07:07.243 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T21:07:07.254 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-10T21:07:07.293 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:07:07.379 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-10T21:07:07.395 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-10T21:07:07.445 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-10T21:07:07.630 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T21:07:07.641 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-10T21:07:07.680 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-10T21:07:07.823 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T21:07:07.839 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-10T21:07:07.889 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:07:07.983 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T21:07:07.997 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-10T21:07:08.033 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-10T21:07:08.218 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T21:07:08.234 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-10T21:07:08.284 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-10T21:07:08.354 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T21:07:08.365 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-10T21:07:08.411 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:07:08.680 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T21:07:08.695 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-10T21:07:08.722 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-mako. 2024-09-10T21:07:08.734 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-10T21:07:08.754 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-10T21:07:08.772 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:07:09.108 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T21:07:09.108 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T21:07:09.120 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T21:07:09.123 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-10T21:07:09.158 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:07:09.174 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:07:09.411 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T21:07:09.423 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-10T21:07:09.453 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:07:09.485 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-mako. 2024-09-10T21:07:09.501 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-10T21:07:09.543 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:07:09.772 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-webob. 2024-09-10T21:07:09.784 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-10T21:07:09.822 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:07:09.905 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T21:07:09.915 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T21:07:09.955 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:07:10.217 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T21:07:10.224 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T21:07:10.229 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-10T21:07:10.237 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-10T21:07:10.266 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:07:10.278 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:07:10.544 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-webob. 2024-09-10T21:07:10.555 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-10T21:07:10.593 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:07:10.603 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T21:07:10.615 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-10T21:07:10.653 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:07:10.955 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T21:07:10.969 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-10T21:07:10.981 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-paste. 2024-09-10T21:07:10.993 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-10T21:07:11.016 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:07:11.023 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:07:11.325 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T21:07:11.337 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-10T21:07:11.342 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T21:07:11.354 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-10T21:07:11.374 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:07:11.393 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:07:11.654 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T21:07:11.665 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-10T21:07:11.694 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-paste. 2024-09-10T21:07:11.695 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:07:11.706 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-10T21:07:11.744 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:07:12.023 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T21:07:12.035 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-10T21:07:12.064 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T21:07:12.073 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-10T21:07:12.076 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-10T21:07:12.114 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:07:12.384 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T21:07:12.396 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-10T21:07:12.416 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T21:07:12.431 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-10T21:07:12.459 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:07:12.466 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:07:12.778 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T21:07:12.791 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-10T21:07:12.844 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-10T21:07:13.156 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T21:07:13.168 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-10T21:07:13.205 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:07:13.523 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-certifi. 2024-09-10T21:07:13.535 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-10T21:07:13.560 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-certifi. 2024-09-10T21:07:13.573 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-10T21:07:13.648 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-10T21:07:13.656 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-10T21:07:14.201 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-idna. 2024-09-10T21:07:14.213 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-10T21:07:14.242 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:07:14.562 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-urllib3. 2024-09-10T21:07:14.574 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-10T21:07:14.612 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:07:14.664 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-idna. 2024-09-10T21:07:14.677 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-10T21:07:14.722 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:07:14.923 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-requests. 2024-09-10T21:07:14.935 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-10T21:07:14.973 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:07:15.033 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-urllib3. 2024-09-10T21:07:15.045 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-10T21:07:15.083 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:07:15.309 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T21:07:15.321 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-10T21:07:15.359 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:07:15.394 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-requests. 2024-09-10T21:07:15.407 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-10T21:07:15.444 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:07:15.687 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T21:07:15.699 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:15.737 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:15.789 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T21:07:15.802 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-10T21:07:15.830 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:07:16.101 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T21:07:16.109 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T21:07:16.113 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:16.121 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:16.150 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:16.175 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:16.445 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T21:07:16.457 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-10T21:07:16.494 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:07:16.538 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T21:07:16.553 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:16.596 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:16.815 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T21:07:16.826 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-10T21:07:16.864 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-10T21:07:16.916 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T21:07:16.930 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-10T21:07:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:07:17.200 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T21:07:17.212 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:17.219 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T21:07:17.232 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-10T21:07:17.250 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:17.277 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-10T21:07:17.612 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T21:07:17.613 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T21:07:17.624 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:17.627 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:17.653 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:17.705 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:18.075 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T21:07:18.088 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:18.132 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:18.257 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T21:07:18.269 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:18.307 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:18.845 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T21:07:18.859 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:18.895 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:19.177 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph. 2024-09-10T21:07:19.189 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../069-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:19.227 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:19.488 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T21:07:19.500 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:19.529 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:19.640 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph. 2024-09-10T21:07:19.653 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../069-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:19.714 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:19.915 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T21:07:19.927 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:19.974 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:20.034 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T21:07:20.046 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:20.083 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:20.352 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package cephadm. 2024-09-10T21:07:20.364 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../072-cephadm_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:20.410 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:20.420 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T21:07:20.433 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:20.470 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:20.721 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-10T21:07:20.733 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-10T21:07:20.763 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-10T21:07:20.864 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package cephadm. 2024-09-10T21:07:20.877 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../072-cephadm_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:20.914 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:21.066 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-10T21:07:21.077 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:07:21.140 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:07:21.225 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-10T21:07:21.238 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-10T21:07:21.275 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-10T21:07:21.570 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-10T21:07:21.582 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:07:21.620 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:07:22.137 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-babel. 2024-09-10T21:07:22.147 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:07:22.186 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:07:22.523 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T21:07:22.534 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-10T21:07:22.573 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:07:22.633 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-babel. 2024-09-10T21:07:22.645 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:07:22.708 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:07:22.943 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T21:07:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:23.001 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:23.027 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T21:07:23.040 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-10T21:07:23.077 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:07:23.321 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T21:07:23.333 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T21:07:23.371 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T21:07:23.430 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T21:07:23.443 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:23.480 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:23.716 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-routes. 2024-09-10T21:07:23.729 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-10T21:07:23.773 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:07:23.859 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T21:07:23.872 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T21:07:23.908 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T21:07:24.186 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-routes. 2024-09-10T21:07:24.200 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-10T21:07:24.244 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:07:24.294 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T21:07:24.307 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:24.351 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:24.539 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T21:07:24.553 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:24.589 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:26.651 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-10T21:07:26.664 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-10T21:07:26.709 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:07:26.911 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-10T21:07:26.924 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-10T21:07:26.969 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:07:27.187 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T21:07:27.199 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-10T21:07:27.306 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:07:27.439 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T21:07:27.452 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-10T21:07:27.566 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:07:27.660 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-10T21:07:27.673 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-10T21:07:27.709 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:07:27.979 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-10T21:07:27.992 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-10T21:07:28.013 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T21:07:28.026 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-10T21:07:28.037 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:07:28.062 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:07:28.307 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T21:07:28.320 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-10T21:07:28.365 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:07:28.721 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T21:07:28.733 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:28.772 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:29.449 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T21:07:29.462 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:29.499 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:29.783 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T21:07:29.797 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-10T21:07:29.841 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-10T21:07:30.136 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T21:07:30.150 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-10T21:07:30.186 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:07:30.461 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T21:07:30.472 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T21:07:30.474 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-10T21:07:30.486 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-10T21:07:30.518 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-10T21:07:30.530 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:07:30.822 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T21:07:30.835 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-10T21:07:30.867 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T21:07:30.880 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:07:30.881 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-10T21:07:30.925 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-10T21:07:31.158 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T21:07:31.172 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-10T21:07:31.208 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:07:31.287 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T21:07:31.301 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-10T21:07:31.345 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-10T21:07:31.469 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T21:07:31.483 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-10T21:07:31.527 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-10T21:07:31.690 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T21:07:31.704 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-10T21:07:31.749 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:07:31.873 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T21:07:31.886 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-10T21:07:31.931 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-10T21:07:32.052 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T21:07:32.065 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-10T21:07:32.101 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-10T21:07:32.301 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T21:07:32.315 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-10T21:07:32.368 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:07:32.438 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T21:07:32.452 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-10T21:07:32.496 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:07:32.746 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T21:07:32.759 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-10T21:07:32.804 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-10T21:07:33.157 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T21:07:33.171 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-10T21:07:33.202 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T21:07:33.214 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:33.224 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:07:33.252 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:33.571 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T21:07:33.584 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-10T21:07:33.638 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:07:33.908 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T21:07:33.921 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:07:33.957 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:07:33.996 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T21:07:34.009 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:34.046 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:34.333 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T21:07:34.345 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-10T21:07:34.382 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:07:34.577 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package jq. 2024-09-10T21:07:34.594 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:07:34.636 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-10T21:07:34.722 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T21:07:34.736 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:07:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:07:34.897 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package socat. 2024-09-10T21:07:34.911 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-10T21:07:34.946 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:07:35.038 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package jq. 2024-09-10T21:07:35.051 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:07:35.096 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-10T21:07:35.208 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T21:07:35.223 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-10T21:07:35.266 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:07:35.334 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package socat. 2024-09-10T21:07:35.351 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-10T21:07:35.384 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:07:35.654 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T21:07:35.662 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-test. 2024-09-10T21:07:35.669 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-10T21:07:35.676 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:35.721 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:35.745 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:07:36.083 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-test. 2024-09-10T21:07:36.097 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:36.141 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:37.691 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T21:07:37.706 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:37.749 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:38.137 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-10T21:07:38.152 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:38.154 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T21:07:38.169 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:38.195 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:38.203 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:38.498 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-colorama. 2024-09-10T21:07:38.513 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-10T21:07:38.556 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-10T21:07:38.583 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-10T21:07:38.598 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:38.641 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:38.851 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package cephfs-shell. 2024-09-10T21:07:38.865 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:38.908 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:38.969 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-colorama. 2024-09-10T21:07:38.983 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-10T21:07:39.027 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-10T21:07:39.153 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package cephfs-top. 2024-09-10T21:07:39.168 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:39.204 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:39.313 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package cephfs-shell. 2024-09-10T21:07:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:39.488 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T21:07:39.502 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:39.546 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:39.641 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package cephfs-top. 2024-09-10T21:07:39.655 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:07:39.707 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:39.891 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-10T21:07:39.906 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-10T21:07:39.949 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:07:40.044 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T21:07:40.058 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:40.110 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:40.279 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-10T21:07:40.293 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-10T21:07:40.337 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:07:40.488 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-10T21:07:40.502 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-10T21:07:40.554 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:07:40.691 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T21:07:40.705 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-10T21:07:40.741 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:07:40.975 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-10T21:07:40.990 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-10T21:07:41.034 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:07:41.113 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package pkg-config. 2024-09-10T21:07:41.127 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-10T21:07:41.162 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:07:41.416 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-10T21:07:41.430 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-10T21:07:41.474 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:07:41.479 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T21:07:41.494 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-10T21:07:41.545 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:07:41.894 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-10T21:07:41.908 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-10T21:07:41.952 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-10T21:07:41.984 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package pkg-config. 2024-09-10T21:07:41.998 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-10T21:07:42.034 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:07:42.270 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-10T21:07:42.285 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-10T21:07:42.288 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T21:07:42.303 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T21:07:42.328 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:07:42.346 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T21:07:42.675 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T21:07:42.689 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-10T21:07:42.723 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-10T21:07:42.733 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:07:42.738 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-10T21:07:42.781 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-10T21:07:43.011 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T21:07:43.026 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-10T21:07:43.060 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-10T21:07:43.118 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T21:07:43.132 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T21:07:43.176 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T21:07:43.389 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-py. 2024-09-10T21:07:43.403 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-10T21:07:43.471 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-10T21:07:43.521 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T21:07:43.536 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-10T21:07:43.579 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:07:43.783 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pygments. 2024-09-10T21:07:43.798 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-10T21:07:43.841 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:07:43.890 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T21:07:43.905 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-10T21:07:43.983 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-10T21:07:44.327 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-py. 2024-09-10T21:07:44.329 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T21:07:44.342 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-10T21:07:44.343 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-10T21:07:44.384 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-10T21:07:44.387 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:07:44.730 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pygments. 2024-09-10T21:07:44.745 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-10T21:07:44.788 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:07:44.915 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-toml. 2024-09-10T21:07:44.930 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-10T21:07:44.973 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-10T21:07:45.259 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T21:07:45.274 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-10T21:07:45.275 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T21:07:45.291 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-10T21:07:45.317 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:07:45.326 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:07:45.628 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-toml. 2024-09-10T21:07:45.643 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-10T21:07:45.671 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T21:07:45.686 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-10T21:07:45.686 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-10T21:07:45.729 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:07:46.014 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T21:07:46.029 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-10T21:07:46.032 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T21:07:46.047 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-10T21:07:46.072 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:07:46.090 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:07:46.418 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T21:07:46.433 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-10T21:07:46.476 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:07:46.745 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T21:07:46.760 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-10T21:07:46.803 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:07:46.819 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package radosgw. 2024-09-10T21:07:46.833 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../123-radosgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:46.878 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:47.532 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package radosgw. 2024-09-10T21:07:47.547 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../123-radosgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:47.590 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:47.705 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T21:07:47.720 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:47.763 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:48.041 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-10T21:07:48.294 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T21:07:48.308 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:07:48.319 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:07:48.351 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:48.428 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:07:48.638 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-10T21:07:48.674 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-10T21:07:48.926 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:07:49.033 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:07:49.066 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-10T21:07:49.327 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-10T21:07:49.644 INFO:teuthology.orchestra.run.smithi052.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-10T21:07:49.704 INFO:teuthology.orchestra.run.smithi052.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-10T21:07:49.746 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:49.759 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-10T21:07:50.340 INFO:teuthology.orchestra.run.smithi052.stdout:Adding system user cephadm....done 2024-09-10T21:07:50.392 INFO:teuthology.orchestra.run.smithi016.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-10T21:07:50.406 INFO:teuthology.orchestra.run.smithi016.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-10T21:07:50.451 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:50.486 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:07:50.770 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:07:50.945 INFO:teuthology.orchestra.run.smithi016.stdout:Adding system user cephadm....done 2024-09-10T21:07:51.013 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:07:51.066 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:07:51.139 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:07:51.409 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:07:51.524 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T21:07:51.677 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:07:51.803 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:07:51.842 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:07:51.976 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-10T21:07:52.072 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T21:07:52.278 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:07:52.355 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:07:52.481 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-10T21:07:52.688 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-10T21:07:52.800 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:07:52.939 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-10T21:07:53.185 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-10T21:07:53.232 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-10T21:07:53.453 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-10T21:07:53.517 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:07:53.713 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-10T21:07:53.935 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:07:53.980 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:07:54.211 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:07:54.240 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:07:54.471 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:07:54.492 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:07:54.722 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:54.826 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:07:55.023 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:07:55.094 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:07:55.149 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-10T21:07:55.275 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:07:55.395 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:07:55.434 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-10T21:07:55.496 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-10T21:07:55.605 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:07:55.694 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:07:55.731 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-10T21:07:55.820 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:07:56.007 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:07:56.104 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:07:56.142 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:07:56.443 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:07:56.491 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-10T21:07:56.552 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-10T21:07:56.616 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:07:56.742 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-10T21:07:56.845 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-10T21:07:56.873 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-10T21:07:56.954 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:07:57.080 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-10T21:07:57.135 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:07:57.236 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-10T21:07:57.490 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:07:57.504 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:07:57.615 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-10T21:07:57.771 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:07:57.892 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:07:58.064 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up zip (3.0-12build2) ... 2024-09-10T21:07:58.193 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:07:58.208 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:07:58.528 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up zip (3.0-12build2) ... 2024-09-10T21:07:58.637 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:07:58.942 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:07:59.218 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:07:59.355 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:07:59.387 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:07:59.520 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:07:59.632 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:07:59.758 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:07:59.838 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-10T21:07:59.875 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:08:00.331 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-10T21:08:00.599 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:08:00.635 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-10T21:08:01.001 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:08:01.053 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-10T21:08:01.310 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-10T21:08:01.312 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:08:01.611 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:08:01.714 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:08:01.996 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:08:02.023 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-10T21:08:02.247 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:08:02.349 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:08:02.390 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:02.716 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:08:02.717 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:08:02.985 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:08:03.059 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:08:03.111 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:03.429 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:08:03.772 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:08:04.340 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:08:04.574 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:08:04.717 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-10T21:08:04.993 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:08:05.069 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:08:05.119 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:08:05.271 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:08:05.296 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:08:05.422 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-10T21:08:05.564 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:08:05.698 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:08:05.807 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:08:05.840 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-10T21:08:05.917 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:08:06.099 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:08:06.225 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:08:06.226 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:08:06.494 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-10T21:08:06.543 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:08:06.779 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:08:06.819 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-10T21:08:06.880 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:08:07.087 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-10T21:08:07.198 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:08:07.338 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:08:07.457 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-10T21:08:07.581 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-10T21:08:07.759 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-10T21:08:07.848 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:08:07.974 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-10T21:08:08.035 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:08:08.302 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-10T21:08:08.325 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:08:08.452 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:08:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:08:08.688 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-10T21:08:08.761 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:08:08.972 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:08:09.079 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:08:09.089 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:08:09.330 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:08:09.382 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:08:09.448 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:08:09.574 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:08:09.662 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-10T21:08:09.684 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:08:09.900 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:08:09.952 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:08:10.027 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:08:10.077 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:08:10.203 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:08:10.292 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-10T21:08:10.445 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:08:10.563 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:08:10.689 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:08:10.721 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-10T21:08:10.822 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-10T21:08:11.073 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:08:11.099 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:08:11.199 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:08:11.367 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-10T21:08:11.493 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-10T21:08:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:08:11.794 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:08:11.878 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:08:11.920 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:08:12.255 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:08:12.281 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-10T21:08:12.407 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-10T21:08:12.597 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:08:12.692 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-10T21:08:12.959 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-10T21:08:13.002 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:08:13.085 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-10T21:08:13.278 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:08:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-10T21:08:13.655 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T21:08:13.662 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:08:13.931 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:08:13.965 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:08:14.325 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:08:14.332 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T21:08:14.451 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:14.603 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:14.659 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:08:14.720 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:08:15.015 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:15.020 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:08:15.146 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:15.174 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:15.272 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:15.398 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:08:15.484 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-10T21:08:15.751 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:15.793 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:15.877 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:15.928 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:16.003 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-10T21:08:16.029 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:16.233 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:08:16.233 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:08:16.305 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:16.431 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:16.581 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:16.801 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:08:16.801 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:08:16.900 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:17.032 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:17.158 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:17.284 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:17.409 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:08:17.450 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:17.601 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:17.727 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:17.854 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:17.978 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:08:18.780 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:18.906 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:19.032 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:19.158 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:19.309 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:19.315 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:19.435 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:19.442 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:19.609 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:19.735 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:19.853 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:19.962 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:19.980 INFO:teuthology.orchestra.run.smithi052.stdout:Adding group ceph....done 2024-09-10T21:08:20.364 INFO:teuthology.orchestra.run.smithi052.stdout:Adding system user ceph....done 2024-09-10T21:08:20.491 INFO:teuthology.orchestra.run.smithi016.stdout:Adding group ceph....done 2024-09-10T21:08:20.558 INFO:teuthology.orchestra.run.smithi052.stdout:Setting system user ceph properties....done 2024-09-10T21:08:20.578 INFO:teuthology.orchestra.run.smithi052.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T21:08:20.731 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T21:08:20.875 INFO:teuthology.orchestra.run.smithi016.stdout:Adding system user ceph....done 2024-09-10T21:08:21.174 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T21:08:21.432 INFO:teuthology.orchestra.run.smithi016.stdout:Setting system user ceph properties....done 2024-09-10T21:08:21.454 INFO:teuthology.orchestra.run.smithi016.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T21:08:21.604 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T21:08:21.824 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:22.014 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T21:08:22.040 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:08:22.040 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:08:22.659 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:22.728 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:22.810 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:22.943 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:08:22.943 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:08:23.374 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:08:23.374 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:08:23.626 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:23.754 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:23.978 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:24.311 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T21:08:24.313 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:08:24.313 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:08:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:24.947 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:25.217 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:08:25.217 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:08:25.315 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T21:08:25.815 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:26.032 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:26.035 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:08:26.036 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:08:26.262 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:08:26.262 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:08:26.667 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:26.911 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:27.040 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:08:27.040 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:08:27.121 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:08:27.122 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:08:27.660 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:27.770 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:27.786 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:27.946 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:28.099 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:08:28.099 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:08:28.146 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:08:28.146 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:08:28.705 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:28.755 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:28.831 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:28.881 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:28.974 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:29.007 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:29.175 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:29.191 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:08:29.192 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:08:29.553 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:08:29.695 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:08:29.839 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:08:29.842 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:29.985 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:30.102 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:30.237 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:08:30.606 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:08:30.715 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:08:30.867 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:08:31.015 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:08:31.740 INFO:teuthology.orchestra.run.smithi052.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:08:31.745 DEBUG:teuthology.orchestra.run.smithi052:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-10T21:08:31.824 INFO:teuthology.orchestra.run.smithi052.stdout:Reading package lists... 2024-09-10T21:08:32.011 INFO:teuthology.orchestra.run.smithi052.stdout:Building dependency tree... 2024-09-10T21:08:32.012 INFO:teuthology.orchestra.run.smithi052.stdout:Reading state information... 2024-09-10T21:08:32.035 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:08:32.207 INFO:teuthology.orchestra.run.smithi052.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-10T21:08:32.207 INFO:teuthology.orchestra.run.smithi052.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-10T21:08:32.208 INFO:teuthology.orchestra.run.smithi052.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-10T21:08:32.208 INFO:teuthology.orchestra.run.smithi052.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:08:32.208 INFO:teuthology.orchestra.run.smithi052.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:08:32.208 INFO:teuthology.orchestra.run.smithi052.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:08:32.208 INFO:teuthology.orchestra.run.smithi052.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:08:32.209 INFO:teuthology.orchestra.run.smithi052.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:08:32.209 INFO:teuthology.orchestra.run.smithi052.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:08:32.209 INFO:teuthology.orchestra.run.smithi052.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:08:32.209 INFO:teuthology.orchestra.run.smithi052.stdout:The following additional packages will be installed: 2024-09-10T21:08:32.209 INFO:teuthology.orchestra.run.smithi052.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-10T21:08:32.209 INFO:teuthology.orchestra.run.smithi052.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-10T21:08:32.210 INFO:teuthology.orchestra.run.smithi052.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-10T21:08:32.210 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-10T21:08:32.210 INFO:teuthology.orchestra.run.smithi052.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:08:32.211 INFO:teuthology.orchestra.run.smithi052.stdout:Suggested packages: 2024-09-10T21:08:32.211 INFO:teuthology.orchestra.run.smithi052.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-10T21:08:32.211 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-10T21:08:32.212 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-10T21:08:32.269 INFO:teuthology.orchestra.run.smithi052.stdout:The following NEW packages will be installed: 2024-09-10T21:08:32.270 INFO:teuthology.orchestra.run.smithi052.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-10T21:08:32.270 INFO:teuthology.orchestra.run.smithi052.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-10T21:08:32.271 INFO:teuthology.orchestra.run.smithi052.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-10T21:08:32.271 INFO:teuthology.orchestra.run.smithi052.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-10T21:08:32.271 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-10T21:08:32.271 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-10T21:08:32.271 INFO:teuthology.orchestra.run.smithi052.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:08:32.272 INFO:teuthology.orchestra.run.smithi052.stdout:The following packages will be upgraded: 2024-09-10T21:08:32.273 INFO:teuthology.orchestra.run.smithi052.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-10T21:08:32.459 INFO:teuthology.orchestra.run.smithi052.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-10T21:08:32.459 INFO:teuthology.orchestra.run.smithi052.stdout:Need to get 55.8 MB of archives. 2024-09-10T21:08:32.459 INFO:teuthology.orchestra.run.smithi052.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-10T21:08:32.459 INFO:teuthology.orchestra.run.smithi052.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-10T21:08:32.791 INFO:teuthology.orchestra.run.smithi016.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:08:32.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-10T21:08:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:Reading package lists... 2024-09-10T21:08:33.061 INFO:teuthology.orchestra.run.smithi016.stdout:Building dependency tree... 2024-09-10T21:08:33.062 INFO:teuthology.orchestra.run.smithi016.stdout:Reading state information... 2024-09-10T21:08:33.082 INFO:teuthology.orchestra.run.smithi052.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-10T21:08:33.098 INFO:teuthology.orchestra.run.smithi052.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-10T21:08:33.223 INFO:teuthology.orchestra.run.smithi052.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-10T21:08:33.223 INFO:teuthology.orchestra.run.smithi052.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-10T21:08:33.224 INFO:teuthology.orchestra.run.smithi052.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-10T21:08:33.252 INFO:teuthology.orchestra.run.smithi016.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-10T21:08:33.252 INFO:teuthology.orchestra.run.smithi016.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-10T21:08:33.252 INFO:teuthology.orchestra.run.smithi016.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-10T21:08:33.252 INFO:teuthology.orchestra.run.smithi016.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:08:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:08:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:08:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:08:33.253 INFO:teuthology.orchestra.run.smithi052.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-10T21:08:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:08:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:08:33.254 INFO:teuthology.orchestra.run.smithi016.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:08:33.254 INFO:teuthology.orchestra.run.smithi016.stdout:The following additional packages will be installed: 2024-09-10T21:08:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-10T21:08:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-10T21:08:33.255 INFO:teuthology.orchestra.run.smithi052.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-10T21:08:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-10T21:08:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-10T21:08:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:08:33.256 INFO:teuthology.orchestra.run.smithi016.stdout:Suggested packages: 2024-09-10T21:08:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-10T21:08:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-10T21:08:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-10T21:08:33.257 INFO:teuthology.orchestra.run.smithi052.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-10T21:08:33.260 INFO:teuthology.orchestra.run.smithi052.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-10T21:08:33.260 INFO:teuthology.orchestra.run.smithi052.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-10T21:08:33.262 INFO:teuthology.orchestra.run.smithi052.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-10T21:08:33.314 INFO:teuthology.orchestra.run.smithi016.stdout:The following NEW packages will be installed: 2024-09-10T21:08:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-10T21:08:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-10T21:08:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-10T21:08:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-10T21:08:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-10T21:08:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-10T21:08:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:08:33.317 INFO:teuthology.orchestra.run.smithi016.stdout:The following packages will be upgraded: 2024-09-10T21:08:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-10T21:08:33.326 INFO:teuthology.orchestra.run.smithi052.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-10T21:08:33.326 INFO:teuthology.orchestra.run.smithi052.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-10T21:08:33.327 INFO:teuthology.orchestra.run.smithi052.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-10T21:08:33.374 INFO:teuthology.orchestra.run.smithi016.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-10T21:08:33.374 INFO:teuthology.orchestra.run.smithi016.stdout:Need to get 55.8 MB of archives. 2024-09-10T21:08:33.374 INFO:teuthology.orchestra.run.smithi016.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-10T21:08:33.374 INFO:teuthology.orchestra.run.smithi016.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-10T21:08:33.408 INFO:teuthology.orchestra.run.smithi052.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-10T21:08:33.508 INFO:teuthology.orchestra.run.smithi016.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-10T21:08:33.512 INFO:teuthology.orchestra.run.smithi016.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-10T21:08:33.539 INFO:teuthology.orchestra.run.smithi016.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-10T21:08:33.539 INFO:teuthology.orchestra.run.smithi016.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-10T21:08:33.540 INFO:teuthology.orchestra.run.smithi016.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-10T21:08:33.547 INFO:teuthology.orchestra.run.smithi016.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-10T21:08:33.548 INFO:teuthology.orchestra.run.smithi016.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-10T21:08:33.549 INFO:teuthology.orchestra.run.smithi016.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-10T21:08:33.550 INFO:teuthology.orchestra.run.smithi016.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-10T21:08:33.551 INFO:teuthology.orchestra.run.smithi016.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-10T21:08:33.552 INFO:teuthology.orchestra.run.smithi016.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-10T21:08:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-10T21:08:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-10T21:08:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-10T21:08:33.578 INFO:teuthology.orchestra.run.smithi016.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-10T21:08:33.793 INFO:teuthology.orchestra.run.smithi016.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-10T21:08:33.794 INFO:teuthology.orchestra.run.smithi016.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-10T21:08:33.795 INFO:teuthology.orchestra.run.smithi016.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-10T21:08:33.798 INFO:teuthology.orchestra.run.smithi016.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-10T21:08:33.798 INFO:teuthology.orchestra.run.smithi016.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-10T21:08:33.802 INFO:teuthology.orchestra.run.smithi016.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-10T21:08:33.804 INFO:teuthology.orchestra.run.smithi016.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-10T21:08:33.811 INFO:teuthology.orchestra.run.smithi016.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-10T21:08:33.812 INFO:teuthology.orchestra.run.smithi016.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-10T21:08:33.833 INFO:teuthology.orchestra.run.smithi016.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-10T21:08:33.833 INFO:teuthology.orchestra.run.smithi016.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-10T21:08:33.843 INFO:teuthology.orchestra.run.smithi016.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-10T21:08:33.863 INFO:teuthology.orchestra.run.smithi016.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-10T21:08:33.867 INFO:teuthology.orchestra.run.smithi016.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-10T21:08:33.867 INFO:teuthology.orchestra.run.smithi016.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-10T21:08:33.868 INFO:teuthology.orchestra.run.smithi016.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-10T21:08:33.868 INFO:teuthology.orchestra.run.smithi016.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-10T21:08:33.880 INFO:teuthology.orchestra.run.smithi016.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-10T21:08:33.881 INFO:teuthology.orchestra.run.smithi016.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-10T21:08:34.026 INFO:teuthology.orchestra.run.smithi016.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-10T21:08:34.026 INFO:teuthology.orchestra.run.smithi016.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-10T21:08:34.027 INFO:teuthology.orchestra.run.smithi016.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-10T21:08:34.027 INFO:teuthology.orchestra.run.smithi016.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-10T21:08:34.027 INFO:teuthology.orchestra.run.smithi016.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-10T21:08:34.027 INFO:teuthology.orchestra.run.smithi016.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-10T21:08:34.040 INFO:teuthology.orchestra.run.smithi016.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-10T21:08:34.041 INFO:teuthology.orchestra.run.smithi016.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-10T21:08:34.090 INFO:teuthology.orchestra.run.smithi052.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-10T21:08:34.092 INFO:teuthology.orchestra.run.smithi052.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-10T21:08:34.092 INFO:teuthology.orchestra.run.smithi052.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-10T21:08:34.095 INFO:teuthology.orchestra.run.smithi052.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-10T21:08:34.095 INFO:teuthology.orchestra.run.smithi052.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-10T21:08:34.099 INFO:teuthology.orchestra.run.smithi052.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-10T21:08:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-10T21:08:34.149 INFO:teuthology.orchestra.run.smithi052.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-10T21:08:34.149 INFO:teuthology.orchestra.run.smithi052.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-10T21:08:34.188 INFO:teuthology.orchestra.run.smithi052.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-10T21:08:34.232 INFO:teuthology.orchestra.run.smithi052.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-10T21:08:34.242 INFO:teuthology.orchestra.run.smithi052.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-10T21:08:34.319 INFO:teuthology.orchestra.run.smithi052.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-10T21:08:34.322 INFO:teuthology.orchestra.run.smithi052.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-10T21:08:34.323 INFO:teuthology.orchestra.run.smithi052.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-10T21:08:34.324 INFO:teuthology.orchestra.run.smithi052.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-10T21:08:34.324 INFO:teuthology.orchestra.run.smithi052.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-10T21:08:34.341 INFO:teuthology.orchestra.run.smithi052.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-10T21:08:34.347 INFO:teuthology.orchestra.run.smithi052.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-10T21:08:34.443 INFO:teuthology.orchestra.run.smithi016.stdout:Preconfiguring packages ... 2024-09-10T21:08:34.618 INFO:teuthology.orchestra.run.smithi016.stdout:Fetched 55.8 MB in 1s (78.0 MB/s) 2024-09-10T21:08:34.825 INFO:teuthology.orchestra.run.smithi052.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-10T21:08:34.825 INFO:teuthology.orchestra.run.smithi052.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-10T21:08:34.825 INFO:teuthology.orchestra.run.smithi052.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-10T21:08:34.825 INFO:teuthology.orchestra.run.smithi052.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-10T21:08:34.825 INFO:teuthology.orchestra.run.smithi052.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-10T21:08:34.826 INFO:teuthology.orchestra.run.smithi052.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-10T21:08:34.888 INFO:teuthology.orchestra.run.smithi052.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-10T21:08:34.889 INFO:teuthology.orchestra.run.smithi052.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-10T21:08:34.915 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:08:35.066 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:08:35.294 INFO:teuthology.orchestra.run.smithi052.stdout:Preconfiguring packages ... 2024-09-10T21:08:35.441 INFO:teuthology.orchestra.run.smithi052.stdout:Fetched 55.8 MB in 3s (21.4 MB/s) 2024-09-10T21:08:35.693 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-10T21:08:35.769 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:08:35.773 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:08:35.896 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:08:36.631 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-10T21:08:36.799 INFO:teuthology.orchestra.run.smithi016.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-10T21:08:37.054 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package flex. 2024-09-10T21:08:37.130 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:08:37.134 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-10T21:08:37.178 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-10T21:08:37.690 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:08:37.706 INFO:teuthology.orchestra.run.smithi052.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-10T21:08:37.842 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-10T21:08:37.993 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package flex. 2024-09-10T21:08:38.067 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:08:38.070 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-10T21:08:38.116 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-10T21:08:38.260 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:08:38.645 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:08:38.822 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-10T21:08:38.841 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:08:38.845 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:08:38.958 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:08:39.239 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:08:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:08:39.489 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:08:39.811 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:08:39.815 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:08:39.822 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:08:39.938 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:08:40.173 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package bison. 2024-09-10T21:08:40.251 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:08:40.255 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-10T21:08:40.301 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:08:40.309 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:08:40.444 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:08:40.718 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-10T21:08:40.734 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-10T21:08:40.770 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:08:40.776 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:08:41.105 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package dump. 2024-09-10T21:08:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-10T21:08:41.162 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package bison. 2024-09-10T21:08:41.163 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking dump (0.4b47-3) ... 2024-09-10T21:08:41.238 INFO:teuthology.orchestra.run.smithi052.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:08:41.242 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-10T21:08:41.289 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:08:41.450 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package indent. 2024-09-10T21:08:41.467 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:08:41.508 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:08:41.682 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-10T21:08:41.697 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-10T21:08:41.740 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:08:41.786 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-10T21:08:41.802 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-10T21:08:41.836 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:08:42.157 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-10T21:08:42.173 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-10T21:08:42.201 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package dump. 2024-09-10T21:08:42.215 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:08:42.218 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-10T21:08:42.259 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking dump (0.4b47-3) ... 2024-09-10T21:08:42.459 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-10T21:08:42.476 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-10T21:08:42.505 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package indent. 2024-09-10T21:08:42.517 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:08:42.521 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:08:42.563 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:08:42.849 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-10T21:08:42.854 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-10T21:08:42.866 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-10T21:08:42.871 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-10T21:08:42.907 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:08:42.912 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:08:43.140 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-10T21:08:43.157 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-10T21:08:43.190 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:08:43.261 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-10T21:08:43.278 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-10T21:08:43.319 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:08:43.418 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libjson-perl. 2024-09-10T21:08:43.435 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-10T21:08:43.476 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-10T21:08:43.556 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-10T21:08:43.573 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-10T21:08:43.606 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:08:43.821 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-10T21:08:43.838 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-10T21:08:43.879 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:08:43.976 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-10T21:08:43.993 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-10T21:08:44.034 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:08:44.270 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-10T21:08:44.287 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-10T21:08:44.328 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:08:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libjson-perl. 2024-09-10T21:08:44.598 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-10T21:08:44.631 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-10T21:08:44.876 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-10T21:08:44.893 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-10T21:08:44.934 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:08:45.216 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-10T21:08:45.233 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:08:45.274 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:08:45.493 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-10T21:08:45.510 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:08:45.551 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:08:45.830 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-10T21:08:45.846 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:08:45.879 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:08:46.257 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-10T21:08:46.274 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-10T21:08:46.304 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-10T21:08:46.315 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:08:46.320 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:08:46.362 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:08:46.644 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-10T21:08:46.660 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:08:46.673 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-10T21:08:46.690 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:08:46.702 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:08:46.731 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:08:47.009 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-10T21:08:47.013 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-10T21:08:47.026 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:08:47.030 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:08:47.063 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:08:47.093 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:08:47.391 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-10T21:08:47.407 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:08:47.412 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-10T21:08:47.429 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-10T21:08:47.449 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:08:47.479 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:08:47.752 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-10T21:08:47.769 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:08:47.810 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:08:47.832 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-10T21:08:47.849 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:08:47.890 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:08:48.063 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-10T21:08:48.080 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:08:48.121 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:08:48.168 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-10T21:08:48.185 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:08:48.226 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:08:48.546 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-10T21:08:48.558 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-10T21:08:48.563 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:08:48.575 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-10T21:08:48.605 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:08:48.608 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:08:48.924 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-10T21:08:48.936 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package modemmanager. 2024-09-10T21:08:48.941 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:08:48.953 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:08:48.982 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:08:49.003 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:08:49.210 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-10T21:08:49.227 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:08:49.268 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:08:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package network-manager. 2024-09-10T21:08:49.643 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:08:49.693 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:08:49.780 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-10T21:08:49.797 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-10T21:08:49.838 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:08:50.183 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package modemmanager. 2024-09-10T21:08:50.200 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:08:50.241 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:08:50.342 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ppp. 2024-09-10T21:08:50.358 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-10T21:08:50.739 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:08:50.890 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package network-manager. 2024-09-10T21:08:50.907 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:08:50.957 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:08:51.097 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package pptp-linux. 2024-09-10T21:08:51.113 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-10T21:08:51.155 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-10T21:08:51.408 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-10T21:08:51.424 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-10T21:08:51.466 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-10T21:08:51.606 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package ppp. 2024-09-10T21:08:51.622 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-10T21:08:51.769 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-10T21:08:51.786 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-10T21:08:51.827 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql-client-common (238) ... 2024-09-10T21:08:51.990 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:08:52.148 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-10T21:08:52.165 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:08:52.206 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:08:52.369 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package pptp-linux. 2024-09-10T21:08:52.384 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-10T21:08:52.427 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-10T21:08:52.663 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-10T21:08:52.679 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-10T21:08:52.730 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-10T21:08:52.786 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql-common. 2024-09-10T21:08:52.803 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-10T21:08:52.864 INFO:teuthology.orchestra.run.smithi016.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-10T21:08:52.894 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql-common (238) ... 2024-09-10T21:08:53.025 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-10T21:08:53.040 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-10T21:08:53.074 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql-client-common (238) ... 2024-09-10T21:08:53.370 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-10T21:08:53.385 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql-14. 2024-09-10T21:08:53.387 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:08:53.402 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:08:53.428 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:08:53.497 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:08:53.975 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql-common. 2024-09-10T21:08:53.991 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-10T21:08:54.044 INFO:teuthology.orchestra.run.smithi052.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-10T21:08:54.074 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql-common (238) ... 2024-09-10T21:08:54.524 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql-14. 2024-09-10T21:08:54.540 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:08:54.585 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:08:54.638 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql. 2024-09-10T21:08:54.654 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-10T21:08:54.688 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql (14+238) ... 2024-09-10T21:08:54.916 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql-client. 2024-09-10T21:08:54.931 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-10T21:08:55.007 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql-client (14+238) ... 2024-09-10T21:08:55.251 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-10T21:08:55.267 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-10T21:08:55.309 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-10T21:08:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package tcl8.6. 2024-09-10T21:08:55.645 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-10T21:08:55.686 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:08:55.760 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql. 2024-09-10T21:08:55.776 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-10T21:08:55.818 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql (14+238) ... 2024-09-10T21:08:55.931 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package tcl. 2024-09-10T21:08:55.948 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-10T21:08:56.011 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-10T21:08:56.143 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql-client. 2024-09-10T21:08:56.143 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-10T21:08:56.143 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql-client (14+238) ... 2024-09-10T21:08:56.331 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-10T21:08:56.342 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package wpasupplicant. 2024-09-10T21:08:56.348 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-10T21:08:56.359 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-10T21:08:56.390 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-10T21:08:56.400 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:08:56.763 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-10T21:08:56.780 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-10T21:08:56.821 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-10T21:08:57.151 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package tcl8.6. 2024-09-10T21:08:57.167 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-10T21:08:57.209 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:08:57.224 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-10T21:08:57.242 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-10T21:08:57.283 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:08:57.478 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package tcl. 2024-09-10T21:08:57.495 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-10T21:08:57.541 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-10T21:08:57.553 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql-client-common (238) ... 2024-09-10T21:08:57.747 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-10T21:08:57.873 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:08:57.874 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package wpasupplicant. 2024-09-10T21:08:57.892 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-10T21:08:57.932 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:08:58.057 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:08:58.183 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:08:58.309 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:08:58.345 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-10T21:08:58.362 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-10T21:08:58.403 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-10T21:08:58.435 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:08:58.740 INFO:teuthology.orchestra.run.smithi052.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-10T21:08:58.757 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-10T21:08:58.798 INFO:teuthology.orchestra.run.smithi052.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:08:59.102 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql-client-common (238) ... 2024-09-10T21:08:59.312 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-10T21:08:59.393 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-10T21:08:59.438 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:08:59.491 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:08:59.590 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:08:59.707 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:08:59.825 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:08:59.876 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:08:59.876 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:08:59.951 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:09:00.552 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:09:00.670 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:09:00.779 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:09:00.914 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-10T21:09:00.921 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-10T21:09:01.015 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:09:01.065 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:09:01.191 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:09:01.317 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:09:01.415 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:09:01.415 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:09:01.443 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:09:02.277 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:09:02.411 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:09:02.515 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:09:02.562 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:09:02.641 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up dump (0.4b47-3) ... 2024-09-10T21:09:02.688 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-10T21:09:02.736 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-10T21:09:02.801 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:09:02.814 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:09:02.927 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:09:02.940 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:09:03.053 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-10T21:09:03.091 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:09:03.171 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-10T21:09:03.296 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:09:03.300 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:09:03.386 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-10T21:09:03.457 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:09:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:09:03.742 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:09:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:09:03.980 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:09:03.994 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-10T21:09:04.106 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up dump (0.4b47-3) ... 2024-09-10T21:09:04.162 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql-client (14+238) ... 2024-09-10T21:09:04.193 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-10T21:09:04.275 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:09:04.330 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:09:04.401 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:09:04.527 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-10T21:09:04.653 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-10T21:09:04.815 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:09:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:04.903 INFO:teuthology.orchestra.run.smithi016.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-10T21:09:04.903 INFO:teuthology.orchestra.run.smithi016.stdout:which means they are currently configured by ifupdown: 2024-09-10T21:09:04.903 INFO:teuthology.orchestra.run.smithi016.stdout:- ens1f0 2024-09-10T21:09:04.903 INFO:teuthology.orchestra.run.smithi016.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-10T21:09:04.904 INFO:teuthology.orchestra.run.smithi016.stdout:remove their configuration from /etc/network/interfaces. 2024-09-10T21:09:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:04.910 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-10T21:09:04.980 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:09:05.043 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-10T21:09:05.081 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:09:05.258 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:09:05.409 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:09:05.461 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-10T21:09:05.535 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-10T21:09:05.703 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql-client (14+238) ... 2024-09-10T21:09:05.829 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:09:05.898 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-10T21:09:06.417 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:06.417 INFO:teuthology.orchestra.run.smithi052.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-10T21:09:06.417 INFO:teuthology.orchestra.run.smithi052.stdout:which means they are currently configured by ifupdown: 2024-09-10T21:09:06.417 INFO:teuthology.orchestra.run.smithi052.stdout:- ens1f0 2024-09-10T21:09:06.417 INFO:teuthology.orchestra.run.smithi052.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-10T21:09:06.418 INFO:teuthology.orchestra.run.smithi052.stdout:remove their configuration from /etc/network/interfaces. 2024-09-10T21:09:06.418 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:06.549 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-10T21:09:06.633 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:09:06.775 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:09:06.901 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-10T21:09:06.944 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-10T21:09:07.027 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql-common (238) ... 2024-09-10T21:09:07.377 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-10T21:09:08.090 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:09:08.216 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:09:08.240 INFO:teuthology.orchestra.run.smithi016.stdout:Adding user postgres to group ssl-cert 2024-09-10T21:09:08.368 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-10T21:09:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:08.481 INFO:teuthology.orchestra.run.smithi016.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-10T21:09:08.493 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql-common (238) ... 2024-09-10T21:09:08.741 INFO:teuthology.orchestra.run.smithi016.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-10T21:09:08.741 INFO:teuthology.orchestra.run.smithi016.stdout:Removing obsolete dictionary files: 2024-09-10T21:09:09.470 INFO:teuthology.orchestra.run.smithi052.stdout:Adding user postgres to group ssl-cert 2024-09-10T21:09:09.497 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-10T21:09:09.760 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:09.761 INFO:teuthology.orchestra.run.smithi052.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-10T21:09:09.964 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:09:10.008 INFO:teuthology.orchestra.run.smithi052.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-10T21:09:10.008 INFO:teuthology.orchestra.run.smithi052.stdout:Removing obsolete dictionary files: 2024-09-10T21:09:10.312 INFO:teuthology.orchestra.run.smithi016.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-10T21:09:10.338 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-10T21:09:10.396 INFO:teuthology.orchestra.run.smithi016.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-10T21:09:10.396 INFO:teuthology.orchestra.run.smithi016.stdout:This user must also own the server process. 2024-09-10T21:09:10.396 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:10.396 INFO:teuthology.orchestra.run.smithi016.stdout:The database cluster will be initialized with locale "en_US". 2024-09-10T21:09:10.396 INFO:teuthology.orchestra.run.smithi016.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-10T21:09:10.396 INFO:teuthology.orchestra.run.smithi016.stdout:The default text search configuration will be set to "english". 2024-09-10T21:09:10.396 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:10.396 INFO:teuthology.orchestra.run.smithi016.stdout:Data page checksums are disabled. 2024-09-10T21:09:10.396 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:10.397 INFO:teuthology.orchestra.run.smithi016.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-10T21:09:10.397 INFO:teuthology.orchestra.run.smithi016.stdout:creating subdirectories ... ok 2024-09-10T21:09:10.397 INFO:teuthology.orchestra.run.smithi016.stdout:selecting dynamic shared memory implementation ... posix 2024-09-10T21:09:10.421 INFO:teuthology.orchestra.run.smithi016.stdout:selecting default max_connections ... 100 2024-09-10T21:09:10.457 INFO:teuthology.orchestra.run.smithi016.stdout:selecting default shared_buffers ... 128MB 2024-09-10T21:09:10.471 INFO:teuthology.orchestra.run.smithi016.stdout:selecting default time zone ... Etc/UTC 2024-09-10T21:09:10.472 INFO:teuthology.orchestra.run.smithi016.stdout:creating configuration files ... ok 2024-09-10T21:09:10.641 INFO:teuthology.orchestra.run.smithi016.stdout:running bootstrap script ... ok 2024-09-10T21:09:10.728 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-10T21:09:11.272 INFO:teuthology.orchestra.run.smithi016.stdout:performing post-bootstrap initialization ... ok 2024-09-10T21:09:11.320 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:09:11.675 INFO:teuthology.orchestra.run.smithi052.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-10T21:09:11.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-10T21:09:11.742 INFO:teuthology.orchestra.run.smithi052.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-10T21:09:11.742 INFO:teuthology.orchestra.run.smithi052.stdout:This user must also own the server process. 2024-09-10T21:09:11.742 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:11.742 INFO:teuthology.orchestra.run.smithi052.stdout:The database cluster will be initialized with locale "en_US". 2024-09-10T21:09:11.742 INFO:teuthology.orchestra.run.smithi052.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-10T21:09:11.743 INFO:teuthology.orchestra.run.smithi052.stdout:The default text search configuration will be set to "english". 2024-09-10T21:09:11.743 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:11.743 INFO:teuthology.orchestra.run.smithi052.stdout:Data page checksums are disabled. 2024-09-10T21:09:11.743 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:11.743 INFO:teuthology.orchestra.run.smithi052.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-10T21:09:11.743 INFO:teuthology.orchestra.run.smithi052.stdout:creating subdirectories ... ok 2024-09-10T21:09:11.743 INFO:teuthology.orchestra.run.smithi052.stdout:selecting dynamic shared memory implementation ... posix 2024-09-10T21:09:11.768 INFO:teuthology.orchestra.run.smithi052.stdout:selecting default max_connections ... 100 2024-09-10T21:09:11.802 INFO:teuthology.orchestra.run.smithi052.stdout:selecting default shared_buffers ... 128MB 2024-09-10T21:09:11.816 INFO:teuthology.orchestra.run.smithi052.stdout:selecting default time zone ... Etc/UTC 2024-09-10T21:09:11.817 INFO:teuthology.orchestra.run.smithi052.stdout:creating configuration files ... ok 2024-09-10T21:09:11.818 INFO:teuthology.orchestra.run.smithi016.stdout:syncing data to disk ... ok 2024-09-10T21:09:11.987 INFO:teuthology.orchestra.run.smithi052.stdout:running bootstrap script ... ok 2024-09-10T21:09:12.368 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-10T21:09:12.620 INFO:teuthology.orchestra.run.smithi052.stdout:performing post-bootstrap initialization ... ok 2024-09-10T21:09:13.214 INFO:teuthology.orchestra.run.smithi052.stdout:syncing data to disk ... ok 2024-09-10T21:09:13.737 INFO:teuthology.orchestra.run.smithi052.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-10T21:09:15.380 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:09:15.498 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-10T21:09:15.607 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql (14+238) ... 2024-09-10T21:09:15.733 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:09:15.859 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:09:16.127 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:09:16.127 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:09:16.786 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:09:16.894 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:09:16.912 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-10T21:09:16.995 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-10T21:09:17.070 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up postgresql (14+238) ... 2024-09-10T21:09:17.197 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:09:17.306 INFO:teuthology.orchestra.run.smithi052.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:09:17.347 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:09:17.499 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:09:17.566 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:09:17.566 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:09:18.309 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:09:18.409 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-10T21:09:18.720 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:09:18.897 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:09:18.944 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-10T21:09:19.052 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:09:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:09:19.630 DEBUG:teuthology.parallel:result is None 2024-09-10T21:09:20.366 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-10T21:09:20.466 INFO:teuthology.orchestra.run.smithi052.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:09:21.013 INFO:teuthology.orchestra.run.smithi052.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:09:21.020 DEBUG:teuthology.parallel:result is None 2024-09-10T21:09:21.021 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:09:21.180 DEBUG:teuthology.orchestra.run.smithi016:> dpkg-query -W -f '${Version}' ceph 2024-09-10T21:09:21.204 INFO:teuthology.orchestra.run.smithi016.stdout:19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:09:21.205 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:09:21.205 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854-g8f219c47-1jammy is installed. 2024-09-10T21:09:21.207 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:09:21.355 DEBUG:teuthology.orchestra.run.smithi052:> dpkg-query -W -f '${Version}' ceph 2024-09-10T21:09:21.379 INFO:teuthology.orchestra.run.smithi052.stdout:19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:09:21.379 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:09:21.379 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854-g8f219c47-1jammy is installed. 2024-09-10T21:09:21.381 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T21:09:21.381 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:21.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T21:09:21.404 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:21.404 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T21:09:21.441 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T21:09:21.441 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:21.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T21:09:21.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T21:09:21.540 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:21.540 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T21:09:21.560 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T21:09:21.625 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T21:09:21.625 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:21.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T21:09:21.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T21:09:21.711 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:21.711 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T21:09:21.731 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T21:09:21.796 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T21:09:21.797 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:21.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T21:09:21.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T21:09:21.885 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:21.885 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T21:09:21.915 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T21:09:21.999 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T21:09:22.596 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T21:09:22.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2024-09-10T21:09:22.600 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 777 /var/log/ceph 2024-09-10T21:09:22.622 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T21:09:22.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T21:09:22.666 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T21:09:22.690 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T21:09:22.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T21:09:22.734 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T21:09:22.765 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-10T21:09:22.765 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-10T21:09:22.765 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-10T21:09:22.766 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:22.766 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T21:09:22.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T21:09:22.902 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:22.902 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T21:09:22.912 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T21:09:23.009 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T21:09:23.009 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T21:09:23.010 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899658', 'branch': 'wip-hemanth-testing-2024-09-10-0723', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/client-limits}', 'email': 'hyelloji@redhat.com', 'first_in_suite': False, 'job_id': '7899658', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-hemanth-testing-2024-09-10-0723'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-hemanth-testing-2024-09-10-0723', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'owner': 'scheduled_hyelloji@teuthology', 'priority': 100, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6190, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'sleep_before_teardown': 0, 'subset': '479/1024', 'suite': 'fs', 'suite_branch': 'wip-hemanth-testing-2024-09-10-0723', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKX3/Vxzmvje760rev3P8sOPUCQzVAT1NO47oyLz8FcyIib2DW5LfEKmzWIeEFsJTg/9YOXmekZ+eIAp7Nxm7bQ=', 'smithi052.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHs+CBX0PV3yOP0028RtPQaSOAuqnP82imiicRokIBQwJ5+ew61XaelpL5yUtUjiBn9cIGMvn7zdXOQrHRlB1JY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_limits']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_16:59:47', 'tube': 'smithi', 'user': 'hyelloji', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T21:09:23.010 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T21:09:23.014 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T21:09:23.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T21:09:23.070 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T21:09:23.091 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:23.092 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T21:09:23.140 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T21:09:23.140 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-09-10T21:09:23.192 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T21:09:23.192 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:09:23.192 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 898 Links: 1 2024-09-10T21:09:23.192 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:09:23.192 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-10 21:02:36.638092466 +0000 2024-09-10T21:09:23.193 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-10 21:02:36.350095038 +0000 2024-09-10T21:09:23.193 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-10 21:02:36.350095038 +0000 2024-09-10T21:09:23.193 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-10 21:02:36.350095038 +0000 2024-09-10T21:09:23.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T21:09:23.255 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-10T21:09:23.255 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-10T21:09:23.255 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000270539 s, 1.9 MB/s 2024-09-10T21:09:23.257 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T21:09:23.310 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-09-10T21:09:23.360 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T21:09:23.360 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:09:23.360 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 916 Links: 1 2024-09-10T21:09:23.360 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:09:23.361 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-10 21:02:37.174087678 +0000 2024-09-10T21:09:23.361 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-10 21:02:36.882090286 +0000 2024-09-10T21:09:23.361 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-10 21:02:36.882090286 +0000 2024-09-10T21:09:23.361 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-10 21:02:36.882090286 +0000 2024-09-10T21:09:23.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T21:09:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-10T21:09:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-10T21:09:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000276633 s, 1.9 MB/s 2024-09-10T21:09:23.428 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T21:09:23.482 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-09-10T21:09:23.536 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T21:09:23.537 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:09:23.537 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 932 Links: 1 2024-09-10T21:09:23.537 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:09:23.537 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-10 21:02:37.734082675 +0000 2024-09-10T21:09:23.537 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-10 21:02:37.458085141 +0000 2024-09-10T21:09:23.537 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-10 21:02:37.458085141 +0000 2024-09-10T21:09:23.537 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-10 21:02:37.458085141 +0000 2024-09-10T21:09:23.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T21:09:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-10T21:09:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-10T21:09:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000259868 s, 2.0 MB/s 2024-09-10T21:09:23.601 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T21:09:23.658 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-09-10T21:09:23.712 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T21:09:23.712 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:09:23.712 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 950 Links: 1 2024-09-10T21:09:23.712 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:09:23.712 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-10 21:02:38.290077710 +0000 2024-09-10T21:09:23.712 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-10 21:02:38.026080068 +0000 2024-09-10T21:09:23.713 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-10 21:02:38.026080068 +0000 2024-09-10T21:09:23.713 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-10 21:02:38.026080068 +0000 2024-09-10T21:09:23.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T21:09:23.778 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-10T21:09:23.778 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-10T21:09:23.778 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000270739 s, 1.9 MB/s 2024-09-10T21:09:23.780 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T21:09:23.838 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:09:23.838 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:23.838 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T21:09:23.848 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T21:09:23.848 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_1 2024-09-10T21:09:23.903 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T21:09:23.903 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:09:23.903 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 6h/6d Inode: 898 Links: 1 2024-09-10T21:09:23.903 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:09:23.903 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-09-10 21:02:38.505459521 +0000 2024-09-10T21:09:23.903 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-09-10 21:02:38.229461986 +0000 2024-09-10T21:09:23.903 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-09-10 21:02:38.229461986 +0000 2024-09-10T21:09:23.903 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-09-10 21:02:38.229461986 +0000 2024-09-10T21:09:23.904 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T21:09:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-09-10T21:09:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-09-10T21:09:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000268823 s, 1.9 MB/s 2024-09-10T21:09:23.971 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T21:09:24.029 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_2 2024-09-10T21:09:24.078 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T21:09:24.079 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:09:24.079 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 6h/6d Inode: 915 Links: 1 2024-09-10T21:09:24.079 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:09:24.079 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-09-10 21:02:39.069454483 +0000 2024-09-10T21:09:24.079 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-09-10 21:02:38.793456948 +0000 2024-09-10T21:09:24.079 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-09-10 21:02:38.793456948 +0000 2024-09-10T21:09:24.079 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-09-10 21:02:38.793456948 +0000 2024-09-10T21:09:24.079 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T21:09:24.141 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-09-10T21:09:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-09-10T21:09:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000258872 s, 2.0 MB/s 2024-09-10T21:09:24.143 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T21:09:24.197 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_3 2024-09-10T21:09:24.246 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T21:09:24.246 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:09:24.247 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 6h/6d Inode: 934 Links: 1 2024-09-10T21:09:24.247 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:09:24.247 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-09-10 21:02:39.357451911 +0000 2024-09-10T21:09:24.247 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-09-10 21:02:39.353451946 +0000 2024-09-10T21:09:24.247 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-09-10 21:02:39.353451946 +0000 2024-09-10T21:09:24.247 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-09-10 21:02:39.353451946 +0000 2024-09-10T21:09:24.247 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T21:09:24.309 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-09-10T21:09:24.309 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-09-10T21:09:24.309 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000266809 s, 1.9 MB/s 2024-09-10T21:09:24.311 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T21:09:24.365 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_4 2024-09-10T21:09:24.415 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T21:09:24.415 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:09:24.415 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 6h/6d Inode: 951 Links: 1 2024-09-10T21:09:24.415 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:09:24.415 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-09-10 21:02:40.141444907 +0000 2024-09-10T21:09:24.415 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-09-10 21:02:39.873447301 +0000 2024-09-10T21:09:24.415 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-09-10 21:02:39.873447301 +0000 2024-09-10T21:09:24.415 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-09-10 21:02:39.873447301 +0000 2024-09-10T21:09:24.416 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T21:09:24.481 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-09-10T21:09:24.481 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-09-10T21:09:24.481 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000269067 s, 1.9 MB/s 2024-09-10T21:09:24.483 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T21:09:24.540 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:09:24.540 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi052.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T21:09:24.541 INFO:tasks.ceph:Generating config... 2024-09-10T21:09:24.542 INFO:tasks.ceph:[client] pid file = 2024-09-10T21:09:24.542 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-10T21:09:24.542 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-09-10T21:09:24.542 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-10T21:09:24.542 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T21:09:24.542 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T21:09:24.542 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T21:09:24.542 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T21:09:24.542 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T21:09:24.542 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-10T21:09:24.542 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-10T21:09:24.543 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-10T21:09:24.543 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-10T21:09:24.543 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T21:09:24.543 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T21:09:24.543 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-10T21:09:24.543 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-10T21:09:24.543 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T21:09:24.543 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T21:09:24.543 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-10T21:09:24.543 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T21:09:24.543 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T21:09:24.543 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T21:09:24.543 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T21:09:24.544 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T21:09:24.544 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T21:09:24.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T21:09:24.590 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T21:09:24.656 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T21:09:24.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:09:24.771 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16'), ('mon.b', '172.21.15.52'), ('mon.c', '[v2:172.21.15.52:3301,v1:172.21.15.52:6790]')] 2024-09-10T21:09:24.771 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16,172.21.15.52,[v2:172.21.15.52:3301,v1:172.21.15.52:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-10T21:09:24.772 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:24.772 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T21:09:24.819 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --add b 172.21.15.52 --addv c '[v2:172.21.15.52:3301,v1:172.21.15.52:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:24.913 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:24.913 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid eb19be60-6d8f-4816-b778-3d7d6698304d 2024-09-10T21:09:24.913 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-09-10T21:09:24.913 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2024-09-10T21:09:24.913 INFO:teuthology.orchestra.run.smithi016.stdout:fsid eb19be60-6d8f-4816-b778-3d7d6698304d 2024-09-10T21:09:24.913 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2024-09-10T21:09:24.912085+0000 2024-09-10T21:09:24.913 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-10T21:09:24.912085+0000 2024-09-10T21:09:24.913 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 17 (quincy) 2024-09-10T21:09:24.914 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2024-09-10T21:09:24.914 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-10T21:09:24.914 INFO:teuthology.orchestra.run.smithi016.stdout:1: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.b 2024-09-10T21:09:24.914 INFO:teuthology.orchestra.run.smithi016.stdout:2: [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] mon.c 2024-09-10T21:09:24.914 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T21:09:24.916 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T21:09:24.924 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID eb19be60-6d8f-4816-b778-3d7d6698304d... 2024-09-10T21:09:24.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:09:24.974 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:09:25.014 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-10T21:09:25.015 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-09-10T21:09:25.015 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:09:25.015 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-09-10T21:09:25.015 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.015 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-09-10T21:09:25.015 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.015 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-09-10T21:09:25.015 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.015 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-09-10T21:09:25.015 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-09-10T21:09:25.015 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.015 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-09-10T21:09:25.016 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-09-10T21:09:25.016 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.016 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:09:25.016 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-09-10T21:09:25.016 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.016 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:09:25.016 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:09:25.016 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.016 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-09-10T21:09:25.016 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.016 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-09-10T21:09:25.017 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-09-10T21:09:25.017 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:09:25.017 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:09:25.017 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-09-10T21:09:25.017 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:09:25.017 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-09-10T21:09:25.017 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:09:25.017 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:09:25.017 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-09-10T21:09:25.017 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.017 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:09:25.017 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.018 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-09-10T21:09:25.018 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.018 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-09-10T21:09:25.018 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.018 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-09-10T21:09:25.018 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-09-10T21:09:25.018 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-09-10T21:09:25.018 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.52,[v2:172.21.15.52:3301,v1:172.21.15.52:6790]" 2024-09-10T21:09:25.018 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2024-09-10T21:09:25.018 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = eb19be60-6d8f-4816-b778-3d7d6698304d 2024-09-10T21:09:25.018 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-09-10T21:09:25.019 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.020 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: bdev async discard = True 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: bdev enable discard = True 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore allocator = bitmap 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 1/20 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 1/20 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 4/10 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-09-10T21:09:25.021 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:09:25.022 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:09:25.022 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-09-10T21:09:25.022 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.022 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-09-10T21:09:25.022 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-10T21:09:25.022 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-09-10T21:09:25.022 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-10T21:09:25.022 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-10T21:09:25.022 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:09:25.022 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:09:25.022 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-09-10T21:09:25.022 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 30 2024-09-10T21:09:25.022 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:09:25.023 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout: mon down mkfs grace = 300 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-09-10T21:09:25.024 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-09-10T21:09:25.025 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-09-10T21:09:25.025 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-09-10T21:09:25.025 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:09:25.025 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:09:25.025 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-09-10T21:09:25.025 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-09-10T21:09:25.025 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-09-10T21:09:25.025 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-09-10T21:09:25.025 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-10T21:09:25.025 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-10T21:09:25.025 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-10T21:09:25.025 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-10T21:09:25.026 INFO:teuthology.orchestra.run.smithi052.stdout:[global] 2024-09-10T21:09:25.026 INFO:teuthology.orchestra.run.smithi052.stdout: chdir = "" 2024-09-10T21:09:25.026 INFO:teuthology.orchestra.run.smithi052.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:09:25.026 INFO:teuthology.orchestra.run.smithi052.stdout: auth supported = cephx 2024-09-10T21:09:25.026 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.026 INFO:teuthology.orchestra.run.smithi052.stdout: filestore xattr use omap = true 2024-09-10T21:09:25.026 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.026 INFO:teuthology.orchestra.run.smithi052.stdout: mon clock drift allowed = 1.000 2024-09-10T21:09:25.026 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.026 INFO:teuthology.orchestra.run.smithi052.stdout: osd crush chooseleaf type = 0 2024-09-10T21:09:25.027 INFO:teuthology.orchestra.run.smithi052.stdout: auth debug = true 2024-09-10T21:09:25.027 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.027 INFO:teuthology.orchestra.run.smithi052.stdout: ms die on old message = true 2024-09-10T21:09:25.027 INFO:teuthology.orchestra.run.smithi052.stdout: ms die on bug = true 2024-09-10T21:09:25.027 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.027 INFO:teuthology.orchestra.run.smithi052.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:09:25.027 INFO:teuthology.orchestra.run.smithi052.stdout: mon pg warn max object skew = 0 2024-09-10T21:09:25.027 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.027 INFO:teuthology.orchestra.run.smithi052.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:09:25.027 INFO:teuthology.orchestra.run.smithi052.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:09:25.027 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.027 INFO:teuthology.orchestra.run.smithi052.stdout: osd pool default size = 2 2024-09-10T21:09:25.027 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd allow primary affinity = true 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd allow pg remap = true 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on no sortbitwise = false 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: mon warn on too few osds = false 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: mon_allow_pool_size_one = true 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: osd default data pool replay window = 5 2024-09-10T21:09:25.028 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.029 INFO:teuthology.orchestra.run.smithi052.stdout: mon allow pool delete = true 2024-09-10T21:09:25.029 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.029 INFO:teuthology.orchestra.run.smithi052.stdout: mon cluster log file level = debug 2024-09-10T21:09:25.029 INFO:teuthology.orchestra.run.smithi052.stdout: debug asserts on shutdown = true 2024-09-10T21:09:25.029 INFO:teuthology.orchestra.run.smithi052.stdout: mon health detail to clog = false 2024-09-10T21:09:25.029 INFO:teuthology.orchestra.run.smithi052.stdout: mon host = "172.21.15.16,172.21.15.52,[v2:172.21.15.52:3301,v1:172.21.15.52:6790]" 2024-09-10T21:09:25.029 INFO:teuthology.orchestra.run.smithi052.stdout: ms die on skipped message = False 2024-09-10T21:09:25.029 INFO:teuthology.orchestra.run.smithi052.stdout: fsid = eb19be60-6d8f-4816-b778-3d7d6698304d 2024-09-10T21:09:25.029 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.029 INFO:teuthology.orchestra.run.smithi052.stdout:[osd] 2024-09-10T21:09:25.029 INFO:teuthology.orchestra.run.smithi052.stdout: osd journal size = 100 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: osd scrub load threshold = 5.0 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: osd scrub max interval = 600 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: osd recover clone overlap = true 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: osd recovery max chunk = 1048576 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug shutdown = true 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug op order = true 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug verify stray on activate = true 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug trim objects = true 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.030 INFO:teuthology.orchestra.run.smithi052.stdout: osd open classes on start = true 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug pg log writeout = true 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: osd map max advance = 10 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: journal zero on create = true 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: filestore apply finisher threads = 3 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: bdev debug aio = true 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: osd debug misdirected ops = true 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: bdev async discard = True 2024-09-10T21:09:25.031 INFO:teuthology.orchestra.run.smithi052.stdout: bdev enable discard = True 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: bluestore allocator = bitmap 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: bluestore block size = 96636764160 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: bluestore fsck on mount = True 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: debug bluefs = 1/20 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: debug bluestore = 1/20 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: debug osd = 20 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: debug rocksdb = 4/10 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd full ratio = 0.9 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: osd objectstore = bluestore 2024-09-10T21:09:25.032 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout:[mgr] 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout: debug mgr = 20 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout: debug mon = 20 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout: debug auth = 20 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout: mgr/telemetry/nag = false 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout: client mount timeout = 30 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout: debug client = 20 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout:[mon] 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout: debug ms = 1 2024-09-10T21:09:25.033 INFO:teuthology.orchestra.run.smithi052.stdout: debug mon = 20 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: debug paxos = 20 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: debug auth = 20 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: mon data avail warn = 5 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: mon mgr mkfs grace = 240 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd reporter subtree level = osd 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: mon osd prime pg temp = true 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.034 INFO:teuthology.orchestra.run.smithi052.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:09:25.035 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:09:25.035 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:09:25.035 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.035 INFO:teuthology.orchestra.run.smithi052.stdout: # 1m isn't quite enough 2024-09-10T21:09:25.035 INFO:teuthology.orchestra.run.smithi052.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:09:25.035 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.035 INFO:teuthology.orchestra.run.smithi052.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:09:25.035 INFO:teuthology.orchestra.run.smithi052.stdout: mon down mkfs grace = 300 2024-09-10T21:09:25.035 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-09-10T21:09:25.035 INFO:teuthology.orchestra.run.smithi052.stdout:[client] 2024-09-10T21:09:25.035 INFO:teuthology.orchestra.run.smithi052.stdout: rgw cache enabled = true 2024-09-10T21:09:25.035 INFO:teuthology.orchestra.run.smithi052.stdout: rgw enable ops log = true 2024-09-10T21:09:25.035 INFO:teuthology.orchestra.run.smithi052.stdout: rgw enable usage log = true 2024-09-10T21:09:25.036 INFO:teuthology.orchestra.run.smithi052.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:09:25.036 INFO:teuthology.orchestra.run.smithi052.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:09:25.036 INFO:teuthology.orchestra.run.smithi052.stdout: pid file = "" 2024-09-10T21:09:25.036 INFO:teuthology.orchestra.run.smithi052.stdout:[mon.a] 2024-09-10T21:09:25.036 INFO:teuthology.orchestra.run.smithi052.stdout:[mon.b] 2024-09-10T21:09:25.036 INFO:teuthology.orchestra.run.smithi052.stdout:[mon.c] 2024-09-10T21:09:25.036 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.a] 2024-09-10T21:09:25.036 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.c] 2024-09-10T21:09:25.036 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.b] 2024-09-10T21:09:25.036 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.d] 2024-09-10T21:09:25.041 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T21:09:25.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T21:09:25.141 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T21:09:25.141 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:25.141 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T21:09:25.150 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:25.150 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T21:09:25.199 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:09:25.199 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:25.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T21:09:25.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:09:25.279 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:25.279 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:25.331 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:09:25.331 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:25.331 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T21:09:25.331 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:09:25.367 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:25.367 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:25.418 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T21:09:25.418 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T21:09:25.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T21:09:25.473 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T21:09:25.479 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T21:09:25.541 INFO:teuthology.orchestra.run.smithi052.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T21:09:25.559 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T21:09:25.559 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T21:09:25.615 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T21:09:25.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-10T21:09:25.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T21:09:25.738 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T21:09:25.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-10T21:09:25.763 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T21:09:25.826 INFO:teuthology.orchestra.run.smithi052.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T21:09:25.831 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-10T21:09:25.852 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T21:09:25.951 INFO:teuthology.orchestra.run.smithi052.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T21:09:25.955 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-10T21:09:25.975 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T21:09:25.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T21:09:26.020 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-10T21:09:26.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-10T21:09:26.121 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-10T21:09:26.138 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T21:09:26.184 INFO:teuthology.orchestra.run.smithi052.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-10T21:09:26.201 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-10T21:09:26.284 INFO:teuthology.orchestra.run.smithi052.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-10T21:09:26.301 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T21:09:26.302 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi052.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T21:09:26.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T21:09:26.321 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:09:26.321 INFO:tasks.ceph:role: osd.0 2024-09-10T21:09:26.321 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:09:26.322 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T21:09:26.382 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:09:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:09:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:09:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:09:26.383 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:09:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:09:26.383 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:09:26.383 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:09:26.383 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:09:26.383 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:09:26.411 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-10T21:09:26.413 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-10T21:09:26.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T21:09:26.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T21:09:26.542 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:09:26.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:09:26.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T21:09:26.610 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:09:26.610 INFO:tasks.ceph:role: osd.1 2024-09-10T21:09:26.610 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:09:26.610 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T21:09:26.676 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:09:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:09:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:09:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:09:26.677 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:09:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:09:26.677 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:09:26.677 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:09:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:09:26.677 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:09:26.703 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-10T21:09:26.706 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-10T21:09:26.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T21:09:26.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T21:09:26.835 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:09:26.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:09:26.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T21:09:26.899 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:09:26.899 INFO:tasks.ceph:role: osd.2 2024-09-10T21:09:26.899 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:09:26.899 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T21:09:26.963 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:09:26.963 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:09:26.963 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:09:26.963 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:09:26.963 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:09:26.963 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:09:26.963 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:09:26.963 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:09:26.963 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:09:26.963 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:09:26.992 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-10T21:09:27.112 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-10T21:09:27.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T21:09:27.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T21:09:27.238 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:09:27.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:09:27.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-10T21:09:27.302 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:09:27.302 INFO:tasks.ceph:role: osd.3 2024-09-10T21:09:27.303 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:09:27.303 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T21:09:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:09:27.363 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:09:27.363 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:09:27.363 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:09:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:09:27.363 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:09:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:09:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:09:27.363 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:09:27.363 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:09:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-10T21:09:27.395 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-10T21:09:27.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-10T21:09:27.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-10T21:09:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:09:27.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:09:27.524 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:27.760+0000 7f31bb8a27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T21:09:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:27.760+0000 7f31bb8a27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T21:09:27.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:27.760+0000 7f31bb8a27c0 -1 bdev(0x55dee4cb1800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T21:09:27.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:27.760+0000 7f31bb8a27c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T21:09:30.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T21:09:31.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:31.096+0000 7f23232d67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T21:09:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:31.096+0000 7f23232d67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T21:09:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:31.096+0000 7f23232d67c0 -1 bdev(0x5645d8e31800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T21:09:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:31.096+0000 7f23232d67c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T21:09:34.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T21:09:34.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:34.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:34.472+0000 7f8c31d2d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T21:09:34.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:34.472+0000 7f8c31d2d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T21:09:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:34.472+0000 7f8c31d2d7c0 -1 bdev(0x55696af19800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T21:09:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:34.476+0000 7f8c31d2d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T21:09:37.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T21:09:37.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:37.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:37.812+0000 7f2e468c17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-10T21:09:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:37.816+0000 7f2e468c17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-10T21:09:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:37.816+0000 7f2e468c17c0 -1 bdev(0x55b953529800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-10T21:09:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:09:37.816+0000 7f2e468c17c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-10T21:09:41.047 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T21:09:41.069 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-10T21:09:41.091 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:09:41.091 INFO:tasks.ceph:role: osd.4 2024-09-10T21:09:41.091 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:09:41.091 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T21:09:41.155 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:09:41.155 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:09:41.155 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:09:41.155 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:09:41.155 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:09:41.155 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:09:41.155 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:09:41.156 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:09:41.156 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:09:41.156 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:09:41.180 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-09-10T21:09:41.183 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-09-10T21:09:41.183 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-10T21:09:41.252 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-10T21:09:41.309 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:09:41.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:09:41.311 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-10T21:09:41.373 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:09:41.373 INFO:tasks.ceph:role: osd.5 2024-09-10T21:09:41.373 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:09:41.373 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T21:09:41.433 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:09:41.433 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:09:41.433 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:09:41.433 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:09:41.433 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:09:41.433 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:09:41.434 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:09:41.434 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:09:41.434 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:09:41.434 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:09:41.457 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-09-10T21:09:41.460 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-09-10T21:09:41.460 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-10T21:09:41.530 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-10T21:09:41.589 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:09:41.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:09:41.591 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-10T21:09:41.652 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:09:41.653 INFO:tasks.ceph:role: osd.6 2024-09-10T21:09:41.653 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:09:41.653 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T21:09:41.712 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:09:41.712 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:09:41.712 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:09:41.712 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:09:41.712 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:09:41.712 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:09:41.713 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:09:41.713 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:09:41.713 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:09:41.713 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:09:41.736 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-09-10T21:09:41.738 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-09-10T21:09:41.739 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-10T21:09:41.806 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-10T21:09:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:09:41.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:09:41.866 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-10T21:09:41.929 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:09:41.929 INFO:tasks.ceph:role: osd.7 2024-09-10T21:09:41.929 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:09:41.929 DEBUG:teuthology.orchestra.run.smithi052:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T21:09:41.993 INFO:teuthology.orchestra.run.smithi052.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:09:41.993 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:09:41.993 INFO:teuthology.orchestra.run.smithi052.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:09:41.993 INFO:teuthology.orchestra.run.smithi052.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:09:41.993 INFO:teuthology.orchestra.run.smithi052.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:09:41.993 INFO:teuthology.orchestra.run.smithi052.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:09:41.993 INFO:teuthology.orchestra.run.smithi052.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:09:41.994 INFO:teuthology.orchestra.run.smithi052.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:09:41.994 INFO:teuthology.orchestra.run.smithi052.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:09:41.994 INFO:teuthology.orchestra.run.smithi052.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:09:42.020 INFO:teuthology.orchestra.run.smithi052.stdout:Discarding blocks...Done. 2024-09-10T21:09:42.021 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi052.front.sepia.ceph.com -o noatime 2024-09-10T21:09:42.021 DEBUG:teuthology.orchestra.run.smithi052:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-10T21:09:42.090 DEBUG:teuthology.orchestra.run.smithi052:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-10T21:09:42.149 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:09:42.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:09:42.150 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:42.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:42.264+0000 7efdce9e87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-10T21:09:42.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:42.264+0000 7efdce9e87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-10T21:09:42.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:42.264+0000 7efdce9e87c0 -1 bdev(0x55b964525800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-10T21:09:42.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:42.264+0000 7efdce9e87c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-10T21:09:45.521 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T21:09:45.544 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:45.640+0000 7f9c769b67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-10T21:09:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:45.640+0000 7f9c769b67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-10T21:09:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:45.640+0000 7f9c769b67c0 -1 bdev(0x55b2f9e5b800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-10T21:09:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:45.640+0000 7f9c769b67c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-10T21:09:48.877 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T21:09:48.899 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:48.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:48.988+0000 7f83c3e677c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-10T21:09:48.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:48.988+0000 7f83c3e677c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-10T21:09:48.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:48.988+0000 7f83c3e677c0 -1 bdev(0x55a4c4fdd800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-10T21:09:48.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:48.988+0000 7f83c3e677c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-10T21:09:52.240 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-10T21:09:52.262 DEBUG:teuthology.orchestra.run.smithi052:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:52.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:52.356+0000 7f618f6367c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-10T21:09:52.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:52.356+0000 7f618f6367c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-10T21:09:52.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:52.356+0000 7f618f6367c0 -1 bdev(0x56321c997800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-10T21:09:52.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:09:52.356+0000 7f618f6367c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-10T21:09:53.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:09:53.015 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:09:55.573 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T21:09:55.596 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T21:09:55.596 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:55.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-10T21:09:55.619 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:55.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-10T21:09:55.680 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:55.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-10T21:09:55.744 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:55.744 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T21:09:55.812 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:55.812 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T21:09:55.876 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:55.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T21:09:55.940 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:55.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-10T21:09:56.004 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:56.004 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T21:09:56.026 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:56.026 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-10T21:09:56.095 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:56.095 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-10T21:09:56.163 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:56.163 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-10T21:09:56.231 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:56.231 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-10T21:09:56.295 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:56.295 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-10T21:09:56.363 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:56.363 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-10T21:09:56.426 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:56.427 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-10T21:09:56.436 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:09:56.436 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-10T21:09:56.487 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:56.487 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-10T21:09:56.496 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:09:56.497 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-10T21:09:56.546 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T21:09:56.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T21:09:56.549 DEBUG:teuthology.orchestra.run.smithi052:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T21:09:56.594 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.y] 2024-09-10T21:09:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCFteBmeyYxHBAAvBkpTzjPQKAOlHzI1jQ46g== 2024-09-10T21:09:56.594 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-10T21:09:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCFteBmhg+lJBAAWjuYW75BGo2dmyAUi7s9+A== 2024-09-10T21:09:56.594 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-10T21:09:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCFteBmxxoALBAAgkmw91/VP1Uv2WMZsJKAyA== 2024-09-10T21:09:56.594 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2024-09-10T21:09:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCHteBmRoahLRAA32PsUkqtGkeI7tMFRwas5Q== 2024-09-10T21:09:56.594 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2024-09-10T21:09:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCLteBmGmEMBhAAr1yQEf/NkiN4+ezAQ9VR2A== 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCOteBmmMuAHBAAt+G/KApsZymjTLRSmyRbdw== 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCRteBmoQfNMBAA0vnIqlYoI13KNAAfNmJg3w== 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCFteBmMVABIBAAoB72mPyXyUO6Y6k+fMqQQg== 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCFteBmQr7+MBAAOzXhWIirBOoi/7wKnrFgmg== 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCFteBmA4tzOBAAiS+8n+Gokz6yXapVf2ecYg== 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.4] 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCWteBmc/7lDxAADlSi4UpcFJVNdp6YTkpdSw== 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.5] 2024-09-10T21:09:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCZteBmBZ1LJhAAPcbZa1LQu4kNbSsJ/8Z81A== 2024-09-10T21:09:56.596 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.6] 2024-09-10T21:09:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCcteBm7/czOxAAqREnY84+LF1N9dRXOhzz6g== 2024-09-10T21:09:56.596 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.7] 2024-09-10T21:09:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCgteBmG2J8FRAAB+cMFCnf9wJg1R3RD5SUJw== 2024-09-10T21:09:56.596 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-10T21:09:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA== 2024-09-10T21:09:56.596 INFO:teuthology.orchestra.run.smithi016.stdout:[client.2] 2024-09-10T21:09:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBm50o1BxAAIFbVxVqU4rU8eKYpuZjTHA== 2024-09-10T21:09:56.596 INFO:teuthology.orchestra.run.smithi016.stdout:[client.1] 2024-09-10T21:09:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBmuEm4ChAAe08QvhwIcN8ZHa3AoVcYww== 2024-09-10T21:09:56.596 INFO:teuthology.orchestra.run.smithi016.stdout:[client.3] 2024-09-10T21:09:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBmxtu0EBAAmN7+07lDiwjsIC/pNYUlSw== 2024-09-10T21:09:56.608 INFO:teuthology.orchestra.run.smithi052.stdout:[mgr.y] 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCFteBmeyYxHBAAvBkpTzjPQKAOlHzI1jQ46g== 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.a] 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCFteBmhg+lJBAAWjuYW75BGo2dmyAUi7s9+A== 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.c] 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCFteBmxxoALBAAgkmw91/VP1Uv2WMZsJKAyA== 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.0] 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCHteBmRoahLRAA32PsUkqtGkeI7tMFRwas5Q== 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.1] 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCLteBmGmEMBhAAr1yQEf/NkiN4+ezAQ9VR2A== 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.2] 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCOteBmmMuAHBAAt+G/KApsZymjTLRSmyRbdw== 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.3] 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCRteBmoQfNMBAA0vnIqlYoI13KNAAfNmJg3w== 2024-09-10T21:09:56.609 INFO:teuthology.orchestra.run.smithi052.stdout:[mgr.x] 2024-09-10T21:09:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCFteBmMVABIBAAoB72mPyXyUO6Y6k+fMqQQg== 2024-09-10T21:09:56.610 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.b] 2024-09-10T21:09:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCFteBmQr7+MBAAOzXhWIirBOoi/7wKnrFgmg== 2024-09-10T21:09:56.610 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.d] 2024-09-10T21:09:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCFteBmA4tzOBAAiS+8n+Gokz6yXapVf2ecYg== 2024-09-10T21:09:56.610 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.4] 2024-09-10T21:09:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCWteBmc/7lDxAADlSi4UpcFJVNdp6YTkpdSw== 2024-09-10T21:09:56.610 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.5] 2024-09-10T21:09:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCZteBmBZ1LJhAAPcbZa1LQu4kNbSsJ/8Z81A== 2024-09-10T21:09:56.610 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.6] 2024-09-10T21:09:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCcteBm7/czOxAAqREnY84+LF1N9dRXOhzz6g== 2024-09-10T21:09:56.610 INFO:teuthology.orchestra.run.smithi052.stdout:[osd.7] 2024-09-10T21:09:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCgteBmG2J8FRAAB+cMFCnf9wJg1R3RD5SUJw== 2024-09-10T21:09:56.611 INFO:teuthology.orchestra.run.smithi052.stdout:[client.0] 2024-09-10T21:09:56.611 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA== 2024-09-10T21:09:56.611 INFO:teuthology.orchestra.run.smithi052.stdout:[client.2] 2024-09-10T21:09:56.611 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCGteBm50o1BxAAIFbVxVqU4rU8eKYpuZjTHA== 2024-09-10T21:09:56.611 INFO:teuthology.orchestra.run.smithi052.stdout:[client.1] 2024-09-10T21:09:56.611 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCGteBmuEm4ChAAe08QvhwIcN8ZHa3AoVcYww== 2024-09-10T21:09:56.611 INFO:teuthology.orchestra.run.smithi052.stdout:[client.3] 2024-09-10T21:09:56.611 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQCGteBmxtu0EBAAmN7+07lDiwjsIC/pNYUlSw== 2024-09-10T21:09:56.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:09:56.642 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:09:56.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:09:56.717 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:09:56.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:09:56.782 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:09:56.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:56.844 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:56.899 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:56.902 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:56.957 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:56.960 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:57.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:57.018 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:57.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:09:57.077 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:09:57.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:09:57.136 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:09:57.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:09:57.197 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:09:57.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:57.258 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:57.313 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:57.316 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:57.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:57.375 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:57.430 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:57.433 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:09:57.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:09:57.491 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:09:57.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:09:57.552 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:09:57.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:09:57.613 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:09:57.671 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:09:57.674 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:09:57.733 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T21:09:57.806 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T21:09:57.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T21:09:57.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T21:09:57.954 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T21:09:57.974 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T21:09:58.091 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T21:09:58.111 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T21:09:58.174 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T21:09:58.284 DEBUG:teuthology.orchestra.run.smithi052:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T21:09:58.304 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:58.307 DEBUG:teuthology.orchestra.run.smithi052:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:09:58.357 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T21:09:58.358 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T21:09:58.358 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T21:09:58.361 INFO:tasks.ceph.mon.a:Started 2024-09-10T21:09:58.361 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T21:09:58.362 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T21:09:58.400 INFO:tasks.ceph.mon.b:Started 2024-09-10T21:09:58.401 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T21:09:58.401 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T21:09:58.404 INFO:tasks.ceph.mon.c:Started 2024-09-10T21:09:58.404 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T21:09:58.404 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-10T21:09:58.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-10T21:09:58.408 INFO:tasks.ceph.mgr.y:Started 2024-09-10T21:09:58.408 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T21:09:58.408 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T21:09:58.411 INFO:tasks.ceph.mgr.x:Started 2024-09-10T21:09:58.413 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-10T21:09:58.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-10T21:09:58.417 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-10T21:09:58.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-10T21:09:58.466 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-10T21:09:58.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-10T21:09:58.514 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-10T21:09:58.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-10T21:09:58.561 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-10T21:09:58.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-10T21:09:58.605 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-10T21:09:58.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-10T21:09:58.653 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-10T21:09:58.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-10T21:09:58.701 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-10T21:09:58.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-10T21:09:58.750 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-10T21:09:58.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-10T21:09:58.797 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-10T21:09:58.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-10T21:09:58.841 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-10T21:09:58.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-10T21:09:58.890 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-10T21:09:58.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-10T21:09:58.942 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-10T21:09:58.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-10T21:09:58.996 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T21:09:58.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-10T21:09:59.045 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T21:09:59.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-10T21:09:59.090 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-10T21:09:59.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-10T21:09:59.137 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T21:09:59.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T21:09:59.186 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-10T21:09:59.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-10T21:09:59.238 DEBUG:tasks.ceph:set 18 configs 2024-09-10T21:09:59.238 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi016') 2024-09-10T21:10:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.896+0000 7f6027400640 1 -- 172.21.15.16:0/3310202073 <== mon.1 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v3)=0 v3) ==== 109+0+0 (secure 0 0 0) 0x7f6020012db0 con 0x7f60140040c0 2024-09-10T21:10:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.900+0000 7fed42000640 1 -- 172.21.15.16:0/1055997253 <== mon.1 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v4)=0 v4) ==== 115+0+0 (secure 0 0 0) 0x7fed4c0cd260 con 0x7fed300052a0 2024-09-10T21:10:03.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7f6024c00640 1 -- 172.21.15.16:0/3310202073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60140040c0 msgr2=0x7f6030155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:03.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7f6024c00640 1 --2- 172.21.15.16:0/3310202073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60140040c0 0x7f6030155580 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f60300d08b0 tx=0x7f6020014830 comp rx=0 tx=0).stop 2024-09-10T21:10:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7fed50a00640 1 -- 172.21.15.16:0/1055997253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed300052a0 msgr2=0x7fed4c15a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7fed50a00640 1 --2- 172.21.15.16:0/1055997253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed300052a0 0x7fed4c15a0a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed4c073c80 tx=0x7fed44013b70 comp rx=0 tx=0).stop 2024-09-10T21:10:03.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7f6024c00640 1 -- 172.21.15.16:0/3310202073 shutdown_connections 2024-09-10T21:10:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7f6024c00640 1 --2- 172.21.15.16:0/3310202073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6030074280 0x7f60300cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7f6024c00640 1 --2- 172.21.15.16:0/3310202073 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6014004e70 0x7f60300cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7f6024c00640 1 --2- 172.21.15.16:0/3310202073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60140040c0 0x7f6030155580 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7f6024c00640 1 -- 172.21.15.16:0/3310202073 >> 172.21.15.16:0/3310202073 conn(0x7f60300d0a60 msgr2=0x7f60300708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7fed50a00640 1 -- 172.21.15.16:0/1055997253 shutdown_connections 2024-09-10T21:10:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7fed50a00640 1 --2- 172.21.15.16:0/1055997253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed300052a0 0x7fed4c15a0a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7fed50a00640 1 --2- 172.21.15.16:0/1055997253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed30004810 0x7fed4c0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7fed50a00640 1 --2- 172.21.15.16:0/1055997253 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed30003a10 0x7fed4c160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7fed50a00640 1 -- 172.21.15.16:0/1055997253 >> 172.21.15.16:0/1055997253 conn(0x7fed4c0d1ce0 msgr2=0x7fed4c071760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7f6024c00640 1 -- 172.21.15.16:0/3310202073 shutdown_connections 2024-09-10T21:10:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7f6024c00640 1 -- 172.21.15.16:0/3310202073 wait complete. 2024-09-10T21:10:03.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7fed50a00640 1 -- 172.21.15.16:0/1055997253 shutdown_connections 2024-09-10T21:10:03.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.904+0000 7fed50a00640 1 -- 172.21.15.16:0/1055997253 wait complete. 2024-09-10T21:10:03.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.908+0000 7fb252000640 1 -- 172.21.15.16:0/1166810845 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v5)=0 v5) ==== 120+0+0 (secure 0 0 0) 0x7fb248019050 con 0x7fb240004810 2024-09-10T21:10:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.908+0000 7fb260e00640 1 -- 172.21.15.16:0/1166810845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb240004810 msgr2=0x7fb25c0c3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.908+0000 7fb260e00640 1 --2- 172.21.15.16:0/1166810845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb240004810 0x7fb25c0c3990 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb248011ac0 tx=0x7fb248011f80 comp rx=0 tx=0).stop 2024-09-10T21:10:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.908+0000 7fd906400640 1 -- 172.21.15.16:0/2852470495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fd90013bc00 con 0x7fd8e00072d0 2024-09-10T21:10:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.908+0000 7fb260e00640 1 -- 172.21.15.16:0/1166810845 shutdown_connections 2024-09-10T21:10:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.908+0000 7fb260e00640 1 --2- 172.21.15.16:0/1166810845 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb2400052a0 0x7fb25c0b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.908+0000 7fb260e00640 1 --2- 172.21.15.16:0/1166810845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb240004810 0x7fb25c0c3990 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.908+0000 7fb260e00640 1 --2- 172.21.15.16:0/1166810845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb240003a10 0x7fb25c0b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.908+0000 7fb260e00640 1 -- 172.21.15.16:0/1166810845 >> 172.21.15.16:0/1166810845 conn(0x7fb25c0d0c40 msgr2=0x7fb25c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.908+0000 7f86baa00640 1 -- 172.21.15.16:0/4105378713 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v6)=0 v6) ==== 121+0+0 (secure 0 0 0) 0x7f86ac019070 con 0x7f86a8004e70 2024-09-10T21:10:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.912+0000 7fb260e00640 1 -- 172.21.15.16:0/1166810845 shutdown_connections 2024-09-10T21:10:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.912+0000 7fb260e00640 1 -- 172.21.15.16:0/1166810845 wait complete. 2024-09-10T21:10:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.920+0000 7f86c9400640 1 -- 172.21.15.16:0/4105378713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a8004e70 msgr2=0x7f86c415a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.920+0000 7f86c9400640 1 --2- 172.21.15.16:0/4105378713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a8004e70 0x7f86c415a480 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f86ac011ab0 tx=0x7f86ac011f70 comp rx=0 tx=0).stop 2024-09-10T21:10:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.920+0000 7f86c9400640 1 -- 172.21.15.16:0/4105378713 shutdown_connections 2024-09-10T21:10:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.920+0000 7f86c9400640 1 --2- 172.21.15.16:0/4105378713 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f86c406eb10 0x7f86c40b6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.920+0000 7f86c9400640 1 --2- 172.21.15.16:0/4105378713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a8004e70 0x7f86c415a480 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.920+0000 7f86c9400640 1 --2- 172.21.15.16:0/4105378713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86a8003a60 0x7f86c40b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.920+0000 7f86c9400640 1 -- 172.21.15.16:0/4105378713 >> 172.21.15.16:0/4105378713 conn(0x7f86c40d1ce0 msgr2=0x7f86c4162380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.920+0000 7f86c9400640 1 -- 172.21.15.16:0/4105378713 shutdown_connections 2024-09-10T21:10:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.920+0000 7f86c9400640 1 -- 172.21.15.16:0/4105378713 wait complete. 2024-09-10T21:10:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.924+0000 7fd8f7400640 1 -- 172.21.15.16:0/2852470495 <== mon.1 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7fd8f0018000 con 0x7fd8e00072d0 2024-09-10T21:10:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.932+0000 7f7c1ea00640 1 -- 172.21.15.16:0/1529644408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f7c180704a0 con 0x7f7bfc003a10 2024-09-10T21:10:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.936+0000 7fd906400640 1 -- 172.21.15.16:0/2852470495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8e00072d0 msgr2=0x7fd9002043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.936+0000 7fd906400640 1 --2- 172.21.15.16:0/2852470495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8e00072d0 0x7fd9002043d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd8f001c9a0 tx=0x7fd8e4005840 comp rx=0 tx=0).stop 2024-09-10T21:10:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.936+0000 7fd906400640 1 -- 172.21.15.16:0/2852470495 shutdown_connections 2024-09-10T21:10:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.936+0000 7fd906400640 1 --2- 172.21.15.16:0/2852470495 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd8e4004810 0x7fd90013dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.936+0000 7fd906400640 1 --2- 172.21.15.16:0/2852470495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e40039c0 0x7fd90015f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.936+0000 7fd906400640 1 --2- 172.21.15.16:0/2852470495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8e00072d0 0x7fd9002043d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.936+0000 7fd906400640 1 -- 172.21.15.16:0/2852470495 >> 172.21.15.16:0/2852470495 conn(0x7fd9000b2890 msgr2=0x7fd9000b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.936+0000 7fd906400640 1 -- 172.21.15.16:0/2852470495 shutdown_connections 2024-09-10T21:10:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.936+0000 7fd906400640 1 -- 172.21.15.16:0/2852470495 wait complete. 2024-09-10T21:10:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.944+0000 7f7c14c00640 1 -- 172.21.15.16:0/1529644408 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v8)=0 v8) ==== 119+0+0 (secure 0 0 0) 0x7f7c08011cc0 con 0x7f7bfc003a10 2024-09-10T21:10:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.948+0000 7f7c1ea00640 1 -- 172.21.15.16:0/1529644408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bfc003a10 msgr2=0x7f7c18151890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.948+0000 7f7c1ea00640 1 --2- 172.21.15.16:0/1529644408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bfc003a10 0x7f7c18151890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800e590 tx=0x7f7c0800ea50 comp rx=0 tx=0).stop 2024-09-10T21:10:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.948+0000 7f7c1ea00640 1 -- 172.21.15.16:0/1529644408 shutdown_connections 2024-09-10T21:10:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.948+0000 7f7c1ea00640 1 --2- 172.21.15.16:0/1529644408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bfc0052f0 0x7f7c1809d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.948+0000 7f7c1ea00640 1 --2- 172.21.15.16:0/1529644408 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7bfc004810 0x7f7c180c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.948+0000 7f7c1ea00640 1 --2- 172.21.15.16:0/1529644408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bfc003a10 0x7f7c18151890 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800e590 tx=0x7f7c0800ea50 comp rx=0 tx=0).stop 2024-09-10T21:10:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.948+0000 7f7c1ea00640 1 -- 172.21.15.16:0/1529644408 >> 172.21.15.16:0/1529644408 conn(0x7f7c180d1ce0 msgr2=0x7f7c180c4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:03.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.956+0000 7f7c1ea00640 1 -- 172.21.15.16:0/1529644408 shutdown_connections 2024-09-10T21:10:03.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.956+0000 7f7c1ea00640 1 -- 172.21.15.16:0/1529644408 wait complete. 2024-09-10T21:10:03.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.964+0000 7f0616800640 1 -- 172.21.15.16:0/717893140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f05e8004160 con 0x7f05f4004810 2024-09-10T21:10:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.972+0000 7fc8b5600640 1 -- 172.21.15.16:0/3317929887 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(2 keys) ==== 48+0+0 (secure 0 0 0) 0x7fc8b00a9480 con 0x7fc8a0004e70 2024-09-10T21:10:03.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.976+0000 7f2237400640 1 -- 172.21.15.16:0/911786654 <== mon.1 v2:172.21.15.52:3300/0 8 ==== config(2 keys) ==== 48+0+0 (secure 0 0 0) 0x7f2230026cc0 con 0x7f22200061b0 2024-09-10T21:10:03.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.976+0000 7f0607400640 1 -- 172.21.15.16:0/717893140 <== mon.1 v2:172.21.15.52:3300/0 8 ==== config(2 keys) ==== 48+0+0 (secure 0 0 0) 0x7f05fc021800 con 0x7f05f4004810 2024-09-10T21:10:03.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.976+0000 7f0607400640 1 -- 172.21.15.16:0/717893140 <== mon.1 v2:172.21.15.52:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v9)=0 v9) ==== 114+0+0 (secure 0 0 0) 0x7f05fc021af0 con 0x7f05f4004810 2024-09-10T21:10:03.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.976+0000 7f3f96000640 1 -- 172.21.15.16:0/2394265262 <== mon.1 v2:172.21.15.52:3300/0 8 ==== config(2 keys) ==== 48+0+0 (secure 0 0 0) 0x7f3f900b83f0 con 0x7f3f7c0036d0 2024-09-10T21:10:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.980+0000 7f0616800640 1 -- 172.21.15.16:0/717893140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05f4004810 msgr2=0x7f06101e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.980+0000 7f0616800640 1 --2- 172.21.15.16:0/717893140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05f4004810 0x7f06101e4f70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009d80 tx=0x7f05fc011580 comp rx=0 tx=0).stop 2024-09-10T21:10:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.980+0000 7f0616800640 1 -- 172.21.15.16:0/717893140 shutdown_connections 2024-09-10T21:10:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.980+0000 7f0616800640 1 --2- 172.21.15.16:0/717893140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05f40053d0 0x7f06102044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.980+0000 7f0616800640 1 --2- 172.21.15.16:0/717893140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05f4004810 0x7f06101e4f70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.980+0000 7f0616800640 1 --2- 172.21.15.16:0/717893140 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f05f4003a60 0x7f06101502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.980+0000 7f0616800640 1 -- 172.21.15.16:0/717893140 >> 172.21.15.16:0/717893140 conn(0x7f06100cff20 msgr2=0x7f06100ce910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:03.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.980+0000 7f0616800640 1 -- 172.21.15.16:0/717893140 shutdown_connections 2024-09-10T21:10:03.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.980+0000 7f0616800640 1 -- 172.21.15.16:0/717893140 wait complete. 2024-09-10T21:10:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.980+0000 7f223fe00640 1 -- 172.21.15.16:0/911786654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f22140050f0 con 0x7f22200061b0 2024-09-10T21:10:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.988+0000 7f2237400640 1 -- 172.21.15.16:0/911786654 <== mon.1 v2:172.21.15.52:3300/0 9 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v10)=0 v10) ==== 121+0+0 (secure 0 0 0) 0x7f223001c920 con 0x7f22200061b0 2024-09-10T21:10:03.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.988+0000 7f223fe00640 1 -- 172.21.15.16:0/911786654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22200061b0 msgr2=0x7f22400affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:03.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.988+0000 7f223fe00640 1 --2- 172.21.15.16:0/911786654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22200061b0 0x7f22400affc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2230004e20 tx=0x7f22300127b0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.988+0000 7f223fe00640 1 -- 172.21.15.16:0/911786654 shutdown_connections 2024-09-10T21:10:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.988+0000 7f223fe00640 1 --2- 172.21.15.16:0/911786654 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2240154560 0x7f22401549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.988+0000 7f223fe00640 1 --2- 172.21.15.16:0/911786654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22400c7a20 0x7f2240154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.988+0000 7f223fe00640 1 --2- 172.21.15.16:0/911786654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22200061b0 0x7f22400affc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.988+0000 7f223fe00640 1 -- 172.21.15.16:0/911786654 >> 172.21.15.16:0/911786654 conn(0x7f22400d1c70 msgr2=0x7f2240156e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.988+0000 7f223fe00640 1 -- 172.21.15.16:0/911786654 shutdown_connections 2024-09-10T21:10:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.988+0000 7f223fe00640 1 -- 172.21.15.16:0/911786654 wait complete. 2024-09-10T21:10:04.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.996+0000 7fc8c0000640 1 -- 172.21.15.16:0/3317929887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fc88c005110 con 0x7fc8a0004e70 2024-09-10T21:10:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:03.996+0000 7f3fa4e00640 1 -- 172.21.15.16:0/2394265262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f3f70005110 con 0x7f3f7c0036d0 2024-09-10T21:10:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.000+0000 7fc8b5600640 1 -- 172.21.15.16:0/3317929887 <== mon.0 v2:172.21.15.16:3300/0 9 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v11)=0 v11) ==== 123+0+0 (secure 0 0 0) 0x7fc8b0016550 con 0x7fc8a0004e70 2024-09-10T21:10:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.000+0000 7f3f96000640 1 -- 172.21.15.16:0/2394265262 <== mon.1 v2:172.21.15.52:3300/0 9 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7f3f900b7e80 con 0x7f3f7c0036d0 2024-09-10T21:10:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.000+0000 7fc8b5600640 1 -- 172.21.15.16:0/3317929887 <== mon.0 v2:172.21.15.16:3300/0 10 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7fc8b0006660 con 0x7fc8a0004e70 2024-09-10T21:10:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.000+0000 7fc8c0000640 1 -- 172.21.15.16:0/3317929887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a0004e70 msgr2=0x7fc8b815d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.000+0000 7fc8c0000640 1 --2- 172.21.15.16:0/3317929887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a0004e70 0x7fc8b815d2b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8073bb0 tx=0x7fc8a0005840 comp rx=0 tx=0).stop 2024-09-10T21:10:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.000+0000 7fc8c0000640 1 -- 172.21.15.16:0/3317929887 shutdown_connections 2024-09-10T21:10:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.000+0000 7fc8c0000640 1 --2- 172.21.15.16:0/3317929887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8a0004e70 0x7fc8b815d2b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.000+0000 7fc8c0000640 1 --2- 172.21.15.16:0/3317929887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8a0003a60 0x7fc8b815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.000+0000 7fc8c0000640 1 --2- 172.21.15.16:0/3317929887 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc898007370 0x7fc8b8159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.000+0000 7fc8c0000640 1 -- 172.21.15.16:0/3317929887 >> 172.21.15.16:0/3317929887 conn(0x7fc8b80d1ce0 msgr2=0x7fc8b80b7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.000+0000 7fc8c0000640 1 -- 172.21.15.16:0/3317929887 shutdown_connections 2024-09-10T21:10:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.000+0000 7fc8c0000640 1 -- 172.21.15.16:0/3317929887 wait complete. 2024-09-10T21:10:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.004+0000 7f3f96000640 1 -- 172.21.15.16:0/2394265262 <== mon.1 v2:172.21.15.52:3300/0 10 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v12)=0 v12) ==== 125+0+0 (secure 0 0 0) 0x7f3f900c8020 con 0x7f3f7c0036d0 2024-09-10T21:10:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.004+0000 7f3fa4e00640 1 -- 172.21.15.16:0/2394265262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f7c0036d0 msgr2=0x7f3fa02040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.004+0000 7f3fa4e00640 1 --2- 172.21.15.16:0/2394265262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f7c0036d0 0x7f3fa02040e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3f900ad000 tx=0x7f3f9001b090 comp rx=0 tx=0).stop 2024-09-10T21:10:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.004+0000 7f3fa4e00640 1 -- 172.21.15.16:0/2394265262 shutdown_connections 2024-09-10T21:10:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.004+0000 7f3fa4e00640 1 --2- 172.21.15.16:0/2394265262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fa00b2bf0 0x7f3fa01583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.004+0000 7f3fa4e00640 1 --2- 172.21.15.16:0/2394265262 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3fa00b77d0 0x7f3fa00b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.004+0000 7f3fa4e00640 1 --2- 172.21.15.16:0/2394265262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f7c0036d0 0x7f3fa02040e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.004+0000 7f3fa4e00640 1 -- 172.21.15.16:0/2394265262 >> 172.21.15.16:0/2394265262 conn(0x7f3fa00d1ce0 msgr2=0x7f3fa00ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.004+0000 7f3fa4e00640 1 -- 172.21.15.16:0/2394265262 shutdown_connections 2024-09-10T21:10:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.004+0000 7f3fa4e00640 1 -- 172.21.15.16:0/2394265262 wait complete. 2024-09-10T21:10:04.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.560+0000 7f0872800640 1 -- 172.21.15.16:0/3845234436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084c003b40 msgr2=0x7f084c0066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.560+0000 7f0872800640 1 --2- 172.21.15.16:0/3845234436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084c003b40 0x7f084c0066e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f086c0cd710 tx=0x7f085801d730 comp rx=0 tx=0).stop 2024-09-10T21:10:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.560+0000 7f0872800640 1 -- 172.21.15.16:0/3845234436 shutdown_connections 2024-09-10T21:10:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.560+0000 7f0872800640 1 --2- 172.21.15.16:0/3845234436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084c003b40 0x7f084c0066e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.560+0000 7f0872800640 1 -- 172.21.15.16:0/3845234436 >> 172.21.15.16:0/3845234436 conn(0x7f086c0d1ce0 msgr2=0x7f086c0d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.560+0000 7f0872800640 1 -- 172.21.15.16:0/3845234436 shutdown_connections 2024-09-10T21:10:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.560+0000 7f0872800640 1 -- 172.21.15.16:0/3845234436 wait complete. 2024-09-10T21:10:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f0872800640 1 Processor -- start 2024-09-10T21:10:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f0872800640 1 -- start start 2024-09-10T21:10:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f0872800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084c003b40 0x7f086c1f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f0872800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f084c006c20 0x7f086c15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f0872800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f086c15bdc0 0x7f086c158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f0872800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f086c0d04d0 con 0x7f084c003b40 2024-09-10T21:10:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f0872800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f086c0d0350 con 0x7f084c006c20 2024-09-10T21:10:04.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f0872800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f086c0d0650 con 0x7f086c15bdc0 2024-09-10T21:10:04.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084c003b40 0x7f086c1f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:04.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084c003b40 0x7f086c1f5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53444/0 (socket says 172.21.15.16:53444) 2024-09-10T21:10:04.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086b400640 1 -- 172.21.15.16:0/3998513637 learned_addr learned my addr 172.21.15.16:0/3998513637 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:04.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086aa00640 1 --2- 172.21.15.16:0/3998513637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f084c006c20 0x7f086c15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:04.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086be00640 1 --2- 172.21.15.16:0/3998513637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f086c15bdc0 0x7f086c158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086be00640 1 -- 172.21.15.16:0/3998513637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f086c15bdc0 msgr2=0x7f086c158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:10:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086be00640 1 -- 172.21.15.16:0/3998513637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f086c15bdc0 msgr2=0x7f086c158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086be00640 1 --2- 172.21.15.16:0/3998513637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f086c15bdc0 0x7f086c158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086b400640 1 -- 172.21.15.16:0/3998513637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f086c15bdc0 msgr2=0x7f086c158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086be00640 1 --2- 172.21.15.16:0/3998513637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f086c15bdc0 0x7f086c158550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086b400640 1 --2- 172.21.15.16:0/3998513637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f086c15bdc0 0x7f086c158550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086b400640 1 -- 172.21.15.16:0/3998513637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f084c006c20 msgr2=0x7f086c15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086b400640 1 --2- 172.21.15.16:0/3998513637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f084c006c20 0x7f086c15a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086b400640 1 -- 172.21.15.16:0/3998513637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086c15ae00 con 0x7f084c003b40 2024-09-10T21:10:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f086b400640 1 --2- 172.21.15.16:0/3998513637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084c003b40 0x7f086c1f5bd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f085c004c80 tx=0x7f085c014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f0863400640 1 -- 172.21.15.16:0/3998513637 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c012880 con 0x7f084c003b40 2024-09-10T21:10:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f0863400640 1 -- 172.21.15.16:0/3998513637 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7f085c012a20 con 0x7f084c003b40 2024-09-10T21:10:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f0863400640 1 -- 172.21.15.16:0/3998513637 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c012e30 con 0x7f084c003b40 2024-09-10T21:10:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f0872800640 1 -- 172.21.15.16:0/3998513637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086c1f6220 con 0x7f084c003b40 2024-09-10T21:10:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.564+0000 7f0872800640 1 -- 172.21.15.16:0/3998513637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086c1f67e0 con 0x7f084c003b40 2024-09-10T21:10:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.568+0000 7f0863400640 1 -- 172.21.15.16:0/3998513637 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f085c01d3f0 con 0x7f084c003b40 2024-09-10T21:10:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.568+0000 7f0863400640 1 -- 172.21.15.16:0/3998513637 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f085c011b50 con 0x7f084c003b40 2024-09-10T21:10:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.568+0000 7f0872800640 1 -- 172.21.15.16:0/3998513637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f086c1f6b20 con 0x7f084c003b40 2024-09-10T21:10:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.568+0000 7f0863400640 1 -- 172.21.15.16:0/3998513637 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f086c1f6b20 con 0x7f084c003b40 2024-09-10T21:10:04.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.632+0000 7f0872800640 1 -- 172.21.15.16:0/3998513637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f086c0704a0 con 0x7f084c003b40 2024-09-10T21:10:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.632+0000 7f0863400640 1 -- 172.21.15.16:0/3998513637 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v13)=0 v13) ==== 123+0+0 (secure 0 0 0) 0x7f085c019070 con 0x7f084c003b40 2024-09-10T21:10:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.632+0000 7f0863400640 1 -- 172.21.15.16:0/3998513637 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f085c011d50 con 0x7f084c003b40 2024-09-10T21:10:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.636+0000 7f0872800640 1 -- 172.21.15.16:0/3998513637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084c003b40 msgr2=0x7f086c1f5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.636+0000 7f0872800640 1 --2- 172.21.15.16:0/3998513637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084c003b40 0x7f086c1f5bd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f085c004c80 tx=0x7f085c014830 comp rx=0 tx=0).stop 2024-09-10T21:10:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.636+0000 7f0872800640 1 -- 172.21.15.16:0/3998513637 shutdown_connections 2024-09-10T21:10:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.636+0000 7f0872800640 1 --2- 172.21.15.16:0/3998513637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f086c15bdc0 0x7f086c158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.636+0000 7f0872800640 1 --2- 172.21.15.16:0/3998513637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f084c006c20 0x7f086c15a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.636+0000 7f0872800640 1 --2- 172.21.15.16:0/3998513637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084c003b40 0x7f086c1f5bd0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.636+0000 7f0872800640 1 -- 172.21.15.16:0/3998513637 >> 172.21.15.16:0/3998513637 conn(0x7f086c0d1ce0 msgr2=0x7f086c0c7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.636+0000 7f0872800640 1 -- 172.21.15.16:0/3998513637 shutdown_connections 2024-09-10T21:10:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.636+0000 7f0872800640 1 -- 172.21.15.16:0/3998513637 wait complete. 2024-09-10T21:10:04.660 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi016') 2024-09-10T21:10:04.660 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi016') 2024-09-10T21:10:04.661 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-10T21:10:04.661 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-10T21:10:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 -- 172.21.15.16:0/2742577115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b4004810 msgr2=0x7fd7b4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 --2- 172.21.15.16:0/2742577115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b4004810 0x7fd7b4004c90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8005f20 tx=0x7fd7b801e4b0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d1600640 1 -- 172.21.15.16:0/2742577115 reap_dead start 2024-09-10T21:10:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 -- 172.21.15.16:0/2742577115 shutdown_connections 2024-09-10T21:10:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 -- 172.21.15.16:0/2742577115 >> 172.21.15.16:0/2742577115 conn(0x7fd7cc0d0c30 msgr2=0x7fd7cc0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 -- 172.21.15.16:0/2742577115 shutdown_connections 2024-09-10T21:10:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 -- 172.21.15.16:0/2742577115 wait complete. 2024-09-10T21:10:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 Processor -- start 2024-09-10T21:10:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 -- start start 2024-09-10T21:10:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7cc157e70 0x7fd7cc0c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd7cc0c60a0 0x7fd7cc1524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc152a10 0x7fd7cc0aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc0ccbd0 con 0x7fd7cc152a10 2024-09-10T21:10:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc0cca50 con 0x7fd7cc157e70 2024-09-10T21:10:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d3600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd7cc06cbe0 con 0x7fd7cc0c60a0 2024-09-10T21:10:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc152a10 0x7fd7cc0aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d0c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7cc157e70 0x7fd7cc0c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.752+0000 7fd7d1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc152a10 0x7fd7cc0aec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53470/0 (socket says 172.21.15.16:53470) 2024-09-10T21:10:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7d1600640 1 -- 172.21.15.16:0/194707547 learned_addr learned my addr 172.21.15.16:0/194707547 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7cbe00640 1 --2- 172.21.15.16:0/194707547 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd7cc0c60a0 0x7fd7cc1524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7cbe00640 1 -- 172.21.15.16:0/194707547 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd7cc0c60a0 msgr2=0x7fd7cc1524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:10:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7cbe00640 1 -- 172.21.15.16:0/194707547 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd7cc0c60a0 msgr2=0x7fd7cc1524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7cbe00640 1 --2- 172.21.15.16:0/194707547 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd7cc0c60a0 0x7fd7cc1524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7cbe00640 1 --2- 172.21.15.16:0/194707547 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd7cc0c60a0 0x7fd7cc1524d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7d1600640 1 -- 172.21.15.16:0/194707547 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd7cc0c60a0 msgr2=0x7fd7cc1524d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7d1600640 1 --2- 172.21.15.16:0/194707547 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd7cc0c60a0 0x7fd7cc1524d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7d1600640 1 -- 172.21.15.16:0/194707547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7cc157e70 msgr2=0x7fd7cc0c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7d1600640 1 --2- 172.21.15.16:0/194707547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7cc157e70 0x7fd7cc0c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7d1600640 1 -- 172.21.15.16:0/194707547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7cc153200 con 0x7fd7cc152a10 2024-09-10T21:10:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7d1600640 1 --2- 172.21.15.16:0/194707547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc152a10 0x7fd7cc0aec00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac006110 tx=0x7fd7c4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7c8c00640 1 -- 172.21.15.16:0/194707547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4012710 con 0x7fd7cc152a10 2024-09-10T21:10:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7c8c00640 1 -- 172.21.15.16:0/194707547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fd7c4016020 con 0x7fd7cc152a10 2024-09-10T21:10:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7c8c00640 1 -- 172.21.15.16:0/194707547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4012c20 con 0x7fd7cc152a10 2024-09-10T21:10:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7d3600640 1 -- 172.21.15.16:0/194707547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7cc1fbb00 con 0x7fd7cc152a10 2024-09-10T21:10:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7d3600640 1 -- 172.21.15.16:0/194707547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7cc1fc0c0 con 0x7fd7cc152a10 2024-09-10T21:10:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7c8c00640 1 -- 172.21.15.16:0/194707547 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fd7c401b7a0 con 0x7fd7cc152a10 2024-09-10T21:10:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7c8c00640 1 -- 172.21.15.16:0/194707547 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd7c401ac50 con 0x7fd7cc152a10 2024-09-10T21:10:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.756+0000 7fd7d3600640 1 -- 172.21.15.16:0/194707547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7cc0c64a0 con 0x7fd7cc152a10 2024-09-10T21:10:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.760+0000 7fd7c8c00640 1 -- 172.21.15.16:0/194707547 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fd7c4020020 con 0x7fd7cc152a10 2024-09-10T21:10:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.820+0000 7fd7d3600640 1 -- 172.21.15.16:0/194707547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fd7cc06d080 con 0x7fd7cc152a10 2024-09-10T21:10:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.824+0000 7fd7c8c00640 1 -- 172.21.15.16:0/194707547 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7fd7c4012dc0 con 0x7fd7cc152a10 2024-09-10T21:10:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.824+0000 7fd7c8c00640 1 -- 172.21.15.16:0/194707547 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7c401ae50 con 0x7fd7cc152a10 2024-09-10T21:10:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.824+0000 7fd7d3600640 1 -- 172.21.15.16:0/194707547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc152a10 msgr2=0x7fd7cc0aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.824+0000 7fd7d3600640 1 --2- 172.21.15.16:0/194707547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc152a10 0x7fd7cc0aec00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac006110 tx=0x7fd7c4011a90 comp rx=0 tx=0).stop 2024-09-10T21:10:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.824+0000 7fd7d3600640 1 -- 172.21.15.16:0/194707547 shutdown_connections 2024-09-10T21:10:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.824+0000 7fd7d3600640 1 --2- 172.21.15.16:0/194707547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7cc152a10 0x7fd7cc0aec00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.824+0000 7fd7d3600640 1 --2- 172.21.15.16:0/194707547 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd7cc0c60a0 0x7fd7cc1524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.824+0000 7fd7d3600640 1 --2- 172.21.15.16:0/194707547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7cc157e70 0x7fd7cc0c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.824+0000 7fd7d3600640 1 -- 172.21.15.16:0/194707547 >> 172.21.15.16:0/194707547 conn(0x7fd7cc0d0c30 msgr2=0x7fd7cc0afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.824+0000 7fd7d3600640 1 -- 172.21.15.16:0/194707547 shutdown_connections 2024-09-10T21:10:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.824+0000 7fd7d3600640 1 -- 172.21.15.16:0/194707547 wait complete. 2024-09-10T21:10:04.848 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi016') 2024-09-10T21:10:04.849 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi016') 2024-09-10T21:10:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.844+0000 7f29c5e00640 1 -- 172.21.15.16:0/3081294388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a8004be0 msgr2=0x7f29a8004fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.844+0000 7f29c5e00640 1 --2- 172.21.15.16:0/3081294388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a8004be0 0x7f29a8004fe0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f29b4006bf0 tx=0x7f29b401d8b0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.844+0000 7f29c5400640 1 -- 172.21.15.16:0/3081294388 reap_dead start 2024-09-10T21:10:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.844+0000 7f29c5e00640 1 -- 172.21.15.16:0/3081294388 shutdown_connections 2024-09-10T21:10:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.844+0000 7f29c5e00640 1 -- 172.21.15.16:0/3081294388 >> 172.21.15.16:0/3081294388 conn(0x7f29c00d0a60 msgr2=0x7f29c00d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.844+0000 7f29c5e00640 1 -- 172.21.15.16:0/3081294388 shutdown_connections 2024-09-10T21:10:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.844+0000 7f29c5e00640 1 -- 172.21.15.16:0/3081294388 wait complete. 2024-09-10T21:10:04.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c5e00640 1 Processor -- start 2024-09-10T21:10:04.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c5e00640 1 -- start start 2024-09-10T21:10:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c5e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29c00ca4f0 0x7f29c015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c015ac10 0x7f29c00b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c5e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00b4c80 0x7f29c015d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c5e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29c0070410 con 0x7f29c015ac10 2024-09-10T21:10:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c5e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f29c0070290 con 0x7f29c00b4c80 2024-09-10T21:10:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c5e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f29c0070590 con 0x7f29c00ca4f0 2024-09-10T21:10:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c015ac10 0x7f29c00b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c015ac10 0x7f29c00b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53484/0 (socket says 172.21.15.16:53484) 2024-09-10T21:10:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bfe00640 1 -- 172.21.15.16:0/1044198385 learned_addr learned my addr 172.21.15.16:0/1044198385 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c4a00640 1 --2- 172.21.15.16:0/1044198385 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29c00ca4f0 0x7f29c015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c5400640 1 --2- 172.21.15.16:0/1044198385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00b4c80 0x7f29c015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c4a00640 1 -- 172.21.15.16:0/1044198385 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29c00ca4f0 msgr2=0x7f29c015a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:10:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c4a00640 1 -- 172.21.15.16:0/1044198385 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29c00ca4f0 msgr2=0x7f29c015a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c4a00640 1 --2- 172.21.15.16:0/1044198385 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29c00ca4f0 0x7f29c015a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c4a00640 1 --2- 172.21.15.16:0/1044198385 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29c00ca4f0 0x7f29c015a6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bfe00640 1 -- 172.21.15.16:0/1044198385 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29c00ca4f0 msgr2=0x7f29c015a6d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bfe00640 1 --2- 172.21.15.16:0/1044198385 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29c00ca4f0 0x7f29c015a6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bfe00640 1 -- 172.21.15.16:0/1044198385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00b4c80 msgr2=0x7f29c015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bfe00640 1 --2- 172.21.15.16:0/1044198385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00b4c80 0x7f29c015d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bfe00640 1 -- 172.21.15.16:0/1044198385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c01f5770 con 0x7f29c015ac10 2024-09-10T21:10:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bfe00640 1 --2- 172.21.15.16:0/1044198385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c015ac10 0x7f29c00b4740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f29ac006770 tx=0x7f29ac001860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bcc00640 1 -- 172.21.15.16:0/1044198385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac012470 con 0x7f29c015ac10 2024-09-10T21:10:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bcc00640 1 -- 172.21.15.16:0/1044198385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29ac012610 con 0x7f29c015ac10 2024-09-10T21:10:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c5e00640 1 -- 172.21.15.16:0/1044198385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c01f5a30 con 0x7f29c015ac10 2024-09-10T21:10:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bcc00640 1 -- 172.21.15.16:0/1044198385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac012a60 con 0x7f29c015ac10 2024-09-10T21:10:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29c5e00640 1 -- 172.21.15.16:0/1044198385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c01f5ff0 con 0x7f29c015ac10 2024-09-10T21:10:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bcc00640 1 -- 172.21.15.16:0/1044198385 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f29ac012c00 con 0x7f29c015ac10 2024-09-10T21:10:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.848+0000 7f29bcc00640 1 -- 172.21.15.16:0/1044198385 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f29ac01e890 con 0x7f29c015ac10 2024-09-10T21:10:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.852+0000 7f29c5e00640 1 -- 172.21.15.16:0/1044198385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c015b090 con 0x7f29c015ac10 2024-09-10T21:10:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.852+0000 7f29bcc00640 1 -- 172.21.15.16:0/1044198385 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f29ac0134a0 con 0x7f29c015ac10 2024-09-10T21:10:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.912+0000 7f29c5e00640 1 -- 172.21.15.16:0/1044198385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f29c0070af0 con 0x7f29c015ac10 2024-09-10T21:10:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.916+0000 7f29bcc00640 1 -- 172.21.15.16:0/1044198385 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v15)=0 v15) ==== 117+0+0 (secure 0 0 0) 0x7f29ac01ebb0 con 0x7f29c015ac10 2024-09-10T21:10:04.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.920+0000 7f29c5e00640 1 -- 172.21.15.16:0/1044198385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c015ac10 msgr2=0x7f29c00b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.920+0000 7f29c5e00640 1 --2- 172.21.15.16:0/1044198385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c015ac10 0x7f29c00b4740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f29ac006770 tx=0x7f29ac001860 comp rx=0 tx=0).stop 2024-09-10T21:10:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.920+0000 7f29c5e00640 1 -- 172.21.15.16:0/1044198385 shutdown_connections 2024-09-10T21:10:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.920+0000 7f29c5e00640 1 --2- 172.21.15.16:0/1044198385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c00b4c80 0x7f29c015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.920+0000 7f29c5e00640 1 --2- 172.21.15.16:0/1044198385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29c015ac10 0x7f29c00b4740 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.920+0000 7f29c5e00640 1 --2- 172.21.15.16:0/1044198385 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29c00ca4f0 0x7f29c015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.920+0000 7f29c5e00640 1 -- 172.21.15.16:0/1044198385 >> 172.21.15.16:0/1044198385 conn(0x7f29c00d0a60 msgr2=0x7f29c00758a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.920+0000 7f29c5e00640 1 -- 172.21.15.16:0/1044198385 shutdown_connections 2024-09-10T21:10:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.920+0000 7f29c5e00640 1 -- 172.21.15.16:0/1044198385 wait complete. 2024-09-10T21:10:04.951 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi016') 2024-09-10T21:10:04.951 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi016') 2024-09-10T21:10:04.951 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi016') 2024-09-10T21:10:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.988+0000 7f9fcb000640 1 -- 172.21.15.16:0/3148248268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fa8004130 msgr2=0x7f9fa8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.988+0000 7f9fcb000640 1 --2- 172.21.15.16:0/3148248268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fa8004130 0x7f9fa8004810 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40cd710 tx=0x7f9fac01ddd0 comp rx=0 tx=0).stop 2024-09-10T21:10:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.988+0000 7f9fc9000640 1 -- 172.21.15.16:0/3148248268 reap_dead start 2024-09-10T21:10:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.988+0000 7f9fcb000640 1 -- 172.21.15.16:0/3148248268 shutdown_connections 2024-09-10T21:10:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.988+0000 7f9fcb000640 1 -- 172.21.15.16:0/3148248268 >> 172.21.15.16:0/3148248268 conn(0x7f9fc40d1ce0 msgr2=0x7f9fc40d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.988+0000 7f9fcb000640 1 -- 172.21.15.16:0/3148248268 shutdown_connections 2024-09-10T21:10:04.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.988+0000 7f9fcb000640 1 -- 172.21.15.16:0/3148248268 wait complete. 2024-09-10T21:10:04.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.988+0000 7f9fcb000640 1 Processor -- start 2024-09-10T21:10:04.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fcb000640 1 -- start start 2024-09-10T21:10:05.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fcb000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa8004130 0x7f9fc41f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fcb000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc41fc070 0x7f9fc414a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fcb000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc414a8a0 0x7f9fc40ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fcb000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc40d06e0 con 0x7f9fc414a8a0 2024-09-10T21:10:05.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fcb000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc40d0560 con 0x7f9fc41fc070 2024-09-10T21:10:05.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fcb000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc40d0860 con 0x7f9fa8004130 2024-09-10T21:10:05.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc414a8a0 0x7f9fc40ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fbbe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc41fc070 0x7f9fc414a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc3e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa8004130 0x7f9fc41f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc414a8a0 0x7f9fc40ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53494/0 (socket says 172.21.15.16:53494) 2024-09-10T21:10:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc9000640 1 -- 172.21.15.16:0/1246478482 learned_addr learned my addr 172.21.15.16:0/1246478482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc3e00640 1 -- 172.21.15.16:0/1246478482 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa8004130 msgr2=0x7f9fc41f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:10:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc3e00640 1 -- 172.21.15.16:0/1246478482 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa8004130 msgr2=0x7f9fc41f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc3e00640 1 --2- 172.21.15.16:0/1246478482 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa8004130 0x7f9fc41f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc3e00640 1 --2- 172.21.15.16:0/1246478482 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa8004130 0x7f9fc41f7a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc9000640 1 -- 172.21.15.16:0/1246478482 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa8004130 msgr2=0x7f9fc41f7a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc9000640 1 --2- 172.21.15.16:0/1246478482 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa8004130 0x7f9fc41f7a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc9000640 1 -- 172.21.15.16:0/1246478482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc41fc070 msgr2=0x7f9fc414a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc9000640 1 --2- 172.21.15.16:0/1246478482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc41fc070 0x7f9fc414a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc9000640 1 -- 172.21.15.16:0/1246478482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc41f7fa0 con 0x7f9fc414a8a0 2024-09-10T21:10:05.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc9000640 1 --2- 172.21.15.16:0/1246478482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc414a8a0 0x7f9fc40ce630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4160a00 tx=0x7f9fbc0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:05.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc0c00640 1 -- 172.21.15.16:0/1246478482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc012850 con 0x7f9fc414a8a0 2024-09-10T21:10:05.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc0c00640 1 -- 172.21.15.16:0/1246478482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fbc019070 con 0x7f9fc414a8a0 2024-09-10T21:10:05.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fcb000640 1 -- 172.21.15.16:0/1246478482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc414adb0 con 0x7f9fc414a8a0 2024-09-10T21:10:05.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc0c00640 1 -- 172.21.15.16:0/1246478482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc012d60 con 0x7f9fc414a8a0 2024-09-10T21:10:05.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fcb000640 1 -- 172.21.15.16:0/1246478482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc41fb870 con 0x7f9fc414a8a0 2024-09-10T21:10:05.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc0c00640 1 -- 172.21.15.16:0/1246478482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f9fbc01e460 con 0x7f9fc414a8a0 2024-09-10T21:10:05.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fc0c00640 1 -- 172.21.15.16:0/1246478482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9fbc0115f0 con 0x7f9fc414a8a0 2024-09-10T21:10:05.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.992+0000 7f9fcb000640 1 -- 172.21.15.16:0/1246478482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc40ce420 con 0x7f9fc414a8a0 2024-09-10T21:10:05.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:04.996+0000 7f9fc0c00640 1 -- 172.21.15.16:0/1246478482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f9fbc01e8c0 con 0x7f9fc414a8a0 2024-09-10T21:10:05.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.056+0000 7f9fcb000640 1 -- 172.21.15.16:0/1246478482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f9fc40704a0 con 0x7f9fc414a8a0 2024-09-10T21:10:05.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.060+0000 7f9fc0c00640 1 -- 172.21.15.16:0/1246478482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f9fbc01eaa0 con 0x7f9fc414a8a0 2024-09-10T21:10:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.060+0000 7f9fcb000640 1 -- 172.21.15.16:0/1246478482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc414a8a0 msgr2=0x7f9fc40ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.060+0000 7f9fcb000640 1 --2- 172.21.15.16:0/1246478482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc414a8a0 0x7f9fc40ce630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4160a00 tx=0x7f9fbc0019c0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.060+0000 7f9fcb000640 1 -- 172.21.15.16:0/1246478482 shutdown_connections 2024-09-10T21:10:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.060+0000 7f9fcb000640 1 --2- 172.21.15.16:0/1246478482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc414a8a0 0x7f9fc40ce630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.060+0000 7f9fcb000640 1 --2- 172.21.15.16:0/1246478482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc41fc070 0x7f9fc414a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.060+0000 7f9fcb000640 1 --2- 172.21.15.16:0/1246478482 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa8004130 0x7f9fc41f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.060+0000 7f9fcb000640 1 -- 172.21.15.16:0/1246478482 >> 172.21.15.16:0/1246478482 conn(0x7f9fc40d1ce0 msgr2=0x7f9fc40c7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.060+0000 7f9fcb000640 1 -- 172.21.15.16:0/1246478482 shutdown_connections 2024-09-10T21:10:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.060+0000 7f9fcb000640 1 -- 172.21.15.16:0/1246478482 wait complete. 2024-09-10T21:10:05.086 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-09-10T21:10:05.086 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi016') 2024-09-10T21:10:05.086 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-09-10T21:10:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.136+0000 7f5630000640 1 -- 172.21.15.16:0/950188421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560800a9e0 msgr2=0x7f560800cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.136+0000 7f5630000640 1 --2- 172.21.15.16:0/950188421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560800a9e0 0x7f560800cdd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f56280b37a0 tx=0x7f562001c730 comp rx=0 tx=0).stop 2024-09-10T21:10:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.136+0000 7f5630000640 1 -- 172.21.15.16:0/950188421 shutdown_connections 2024-09-10T21:10:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.136+0000 7f5630000640 1 --2- 172.21.15.16:0/950188421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560800a9e0 0x7f560800cdd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.136+0000 7f5630000640 1 --2- 172.21.15.16:0/950188421 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5608003740 0x7f5608005b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.136+0000 7f5630000640 1 -- 172.21.15.16:0/950188421 >> 172.21.15.16:0/950188421 conn(0x7f56280d1ce0 msgr2=0x7f56280d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.140+0000 7f5630000640 1 -- 172.21.15.16:0/950188421 shutdown_connections 2024-09-10T21:10:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.140+0000 7f5630000640 1 -- 172.21.15.16:0/950188421 wait complete. 2024-09-10T21:10:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.140+0000 7f5630000640 1 Processor -- start 2024-09-10T21:10:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.140+0000 7f5630000640 1 -- start start 2024-09-10T21:10:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5630000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5608003740 0x7f56280cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5630000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56080066d0 0x7f56280c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5630000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560800a9e0 0x7f56281f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5630000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56280d06e0 con 0x7f56080066d0 2024-09-10T21:10:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5630000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f56280d0560 con 0x7f560800a9e0 2024-09-10T21:10:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5630000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f56280d0860 con 0x7f5608003740 2024-09-10T21:10:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56080066d0 0x7f56280c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562d600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5608003740 0x7f56280cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56080066d0 0x7f56280c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53506/0 (socket says 172.21.15.16:53506) 2024-09-10T21:10:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562d600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5608003740 0x7f56280cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:45250/0 (socket says 172.21.15.16:45250) 2024-09-10T21:10:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562cc00640 1 -- 172.21.15.16:0/419289695 learned_addr learned my addr 172.21.15.16:0/419289695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562e000640 1 --2- 172.21.15.16:0/419289695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560800a9e0 0x7f56281f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562d600640 1 -- 172.21.15.16:0/419289695 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5608003740 msgr2=0x7f56280cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:10:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562d600640 1 -- 172.21.15.16:0/419289695 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5608003740 msgr2=0x7f56280cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562d600640 1 --2- 172.21.15.16:0/419289695 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5608003740 0x7f56280cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562d600640 1 --2- 172.21.15.16:0/419289695 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5608003740 0x7f56280cab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562cc00640 1 -- 172.21.15.16:0/419289695 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5608003740 msgr2=0x7f56280cab90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562cc00640 1 --2- 172.21.15.16:0/419289695 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5608003740 0x7f56280cab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562cc00640 1 -- 172.21.15.16:0/419289695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560800a9e0 msgr2=0x7f56281f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562cc00640 1 --2- 172.21.15.16:0/419289695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560800a9e0 0x7f56281f7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562cc00640 1 -- 172.21.15.16:0/419289695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56280c9090 con 0x7f56080066d0 2024-09-10T21:10:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f562cc00640 1 --2- 172.21.15.16:0/419289695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56080066d0 0x7f56280c7a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5614011810 tx=0x7f5614011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5625600640 1 -- 172.21.15.16:0/419289695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614010dc0 con 0x7f56080066d0 2024-09-10T21:10:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5625600640 1 -- 172.21.15.16:0/419289695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5614022070 con 0x7f56080066d0 2024-09-10T21:10:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5630000640 1 -- 172.21.15.16:0/419289695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56280c9370 con 0x7f56080066d0 2024-09-10T21:10:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5625600640 1 -- 172.21.15.16:0/419289695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561401d7a0 con 0x7f56080066d0 2024-09-10T21:10:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5630000640 1 -- 172.21.15.16:0/419289695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56280c9930 con 0x7f56080066d0 2024-09-10T21:10:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5625600640 1 -- 172.21.15.16:0/419289695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f561401d940 con 0x7f56080066d0 2024-09-10T21:10:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5625600640 1 -- 172.21.15.16:0/419289695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5614027db0 con 0x7f56080066d0 2024-09-10T21:10:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.144+0000 7f5630000640 1 -- 172.21.15.16:0/419289695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56280ce420 con 0x7f56080066d0 2024-09-10T21:10:05.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.148+0000 7f5625600640 1 -- 172.21.15.16:0/419289695 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f5614027460 con 0x7f56080066d0 2024-09-10T21:10:05.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.232+0000 7f84f3400640 1 -- 172.21.15.16:0/2412962726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84cc003b30 msgr2=0x7f84cc0066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.232+0000 7f84f3400640 1 --2- 172.21.15.16:0/2412962726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84cc003b30 0x7f84cc0066a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0cd710 tx=0x7f84d801d730 comp rx=0 tx=0).stop 2024-09-10T21:10:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.232+0000 7f84f3400640 1 -- 172.21.15.16:0/2412962726 shutdown_connections 2024-09-10T21:10:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.232+0000 7f84f3400640 1 --2- 172.21.15.16:0/2412962726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84cc003b30 0x7f84cc0066a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.232+0000 7f84f3400640 1 --2- 172.21.15.16:0/2412962726 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f84ec160050 0x7f84cc00b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.232+0000 7f84f3400640 1 -- 172.21.15.16:0/2412962726 >> 172.21.15.16:0/2412962726 conn(0x7f84ec0d1ce0 msgr2=0x7f84ec0d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.232+0000 7f84f3400640 1 -- 172.21.15.16:0/2412962726 shutdown_connections 2024-09-10T21:10:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.232+0000 7f84f3400640 1 -- 172.21.15.16:0/2412962726 wait complete. 2024-09-10T21:10:05.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f3400640 1 Processor -- start 2024-09-10T21:10:05.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f3400640 1 -- start start 2024-09-10T21:10:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84cc003b30 0x7f84ec0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f3400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f84cc006be0 0x7f84ec0cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84ec160050 0x7f84ec0b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f3400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec0d06e0 con 0x7f84ec160050 2024-09-10T21:10:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f3400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec0d0560 con 0x7f84cc003b30 2024-09-10T21:10:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f3400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f84ec0d0860 con 0x7f84cc006be0 2024-09-10T21:10:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84ec160050 0x7f84ec0b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f0a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84cc003b30 0x7f84ec0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84ec160050 0x7f84ec0b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53514/0 (socket says 172.21.15.16:53514) 2024-09-10T21:10:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f1400640 1 -- 172.21.15.16:0/3676872405 learned_addr learned my addr 172.21.15.16:0/3676872405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84ebe00640 1 --2- 172.21.15.16:0/3676872405 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f84cc006be0 0x7f84ec0cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84ebe00640 1 -- 172.21.15.16:0/3676872405 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f84cc006be0 msgr2=0x7f84ec0cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:10:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84ebe00640 1 -- 172.21.15.16:0/3676872405 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f84cc006be0 msgr2=0x7f84ec0cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84ebe00640 1 --2- 172.21.15.16:0/3676872405 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f84cc006be0 0x7f84ec0cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84ebe00640 1 --2- 172.21.15.16:0/3676872405 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f84cc006be0 0x7f84ec0cb660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f1400640 1 -- 172.21.15.16:0/3676872405 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f84cc006be0 msgr2=0x7f84ec0cb660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f1400640 1 --2- 172.21.15.16:0/3676872405 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f84cc006be0 0x7f84ec0cb660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f1400640 1 -- 172.21.15.16:0/3676872405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84cc003b30 msgr2=0x7f84ec0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f1400640 1 --2- 172.21.15.16:0/3676872405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84cc003b30 0x7f84ec0caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f1400640 1 -- 172.21.15.16:0/3676872405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ec06d010 con 0x7f84ec160050 2024-09-10T21:10:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f0a00640 1 --2- 172.21.15.16:0/3676872405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84cc003b30 0x7f84ec0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f1400640 1 --2- 172.21.15.16:0/3676872405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84ec160050 0x7f84ec0b5f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0061e0 tx=0x7f84e4013ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84e8c00640 1 -- 172.21.15.16:0/3676872405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4011da0 con 0x7f84ec160050 2024-09-10T21:10:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84e8c00640 1 -- 172.21.15.16:0/3676872405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84e4004b30 con 0x7f84ec160050 2024-09-10T21:10:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84e8c00640 1 -- 172.21.15.16:0/3676872405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4003980 con 0x7f84ec160050 2024-09-10T21:10:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f3400640 1 -- 172.21.15.16:0/3676872405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ec0cb030 con 0x7f84ec160050 2024-09-10T21:10:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84f3400640 1 -- 172.21.15.16:0/3676872405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ec0ca010 con 0x7f84ec160050 2024-09-10T21:10:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.236+0000 7f84e8c00640 1 -- 172.21.15.16:0/3676872405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f84e401a070 con 0x7f84ec160050 2024-09-10T21:10:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.240+0000 7f84e8c00640 1 -- 172.21.15.16:0/3676872405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f84e4024780 con 0x7f84ec160050 2024-09-10T21:10:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.240+0000 7f84f3400640 1 -- 172.21.15.16:0/3676872405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84ec0ce420 con 0x7f84ec160050 2024-09-10T21:10:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.240+0000 7f84e8c00640 1 -- 172.21.15.16:0/3676872405 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f84e4003b20 con 0x7f84ec160050 2024-09-10T21:10:05.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.248+0000 7f5630000640 1 -- 172.21.15.16:0/419289695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f56280704a0 con 0x7f56080066d0 2024-09-10T21:10:05.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.248+0000 7f5625600640 1 -- 172.21.15.16:0/419289695 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f5614027640 con 0x7f56080066d0 2024-09-10T21:10:05.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.252+0000 7f5630000640 1 -- 172.21.15.16:0/419289695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56080066d0 msgr2=0x7f56280c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.252+0000 7f5630000640 1 --2- 172.21.15.16:0/419289695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56080066d0 0x7f56280c7a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5614011810 tx=0x7f5614011cd0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.252+0000 7f5630000640 1 -- 172.21.15.16:0/419289695 shutdown_connections 2024-09-10T21:10:05.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.252+0000 7f5630000640 1 --2- 172.21.15.16:0/419289695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560800a9e0 0x7f56281f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.252+0000 7f5630000640 1 --2- 172.21.15.16:0/419289695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56080066d0 0x7f56280c7a60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.252+0000 7f5630000640 1 --2- 172.21.15.16:0/419289695 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5608003740 0x7f56280cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.252+0000 7f5630000640 1 -- 172.21.15.16:0/419289695 >> 172.21.15.16:0/419289695 conn(0x7f56280d1ce0 msgr2=0x7f56280ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:05.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.252+0000 7f5630000640 1 -- 172.21.15.16:0/419289695 shutdown_connections 2024-09-10T21:10:05.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.252+0000 7f5630000640 1 -- 172.21.15.16:0/419289695 wait complete. 2024-09-10T21:10:05.277 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-10T21:10:05.277 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-10T21:10:05.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.280+0000 7f5f7aa00640 1 -- 172.21.15.16:0/856647359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f50003b60 msgr2=0x7f5f500066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.280+0000 7f5f7aa00640 1 --2- 172.21.15.16:0/856647359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f50003b60 0x7f5f500066b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5f64006bf0 tx=0x7f5f6401d830 comp rx=0 tx=0).stop 2024-09-10T21:10:05.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.280+0000 7f5f7aa00640 1 -- 172.21.15.16:0/856647359 shutdown_connections 2024-09-10T21:10:05.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.280+0000 7f5f7aa00640 1 --2- 172.21.15.16:0/856647359 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f50006bf0 0x7f5f50006fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.280+0000 7f5f7aa00640 1 --2- 172.21.15.16:0/856647359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f50003b60 0x7f5f500066b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.280+0000 7f5f7aa00640 1 -- 172.21.15.16:0/856647359 >> 172.21.15.16:0/856647359 conn(0x7f5f740d1ce0 msgr2=0x7f5f740d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:05.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.280+0000 7f5f7aa00640 1 -- 172.21.15.16:0/856647359 shutdown_connections 2024-09-10T21:10:05.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.280+0000 7f5f7aa00640 1 -- 172.21.15.16:0/856647359 wait complete. 2024-09-10T21:10:05.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f7aa00640 1 Processor -- start 2024-09-10T21:10:05.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f7aa00640 1 -- start start 2024-09-10T21:10:05.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f7aa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f50003b60 0x7f5f74151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f7aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f50006bf0 0x7f5f740b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f7aa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f50007540 0x7f5f740c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f7aa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7406e9b0 con 0x7f5f50006bf0 2024-09-10T21:10:05.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f7aa00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7406e830 con 0x7f5f50003b60 2024-09-10T21:10:05.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f7aa00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5f7406eb30 con 0x7f5f50007540 2024-09-10T21:10:05.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f50006bf0 0x7f5f740b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f73e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f50003b60 0x7f5f74151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f50006bf0 0x7f5f740b5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53530/0 (socket says 172.21.15.16:53530) 2024-09-10T21:10:05.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f6be00640 1 -- 172.21.15.16:0/319257967 learned_addr learned my addr 172.21.15.16:0/319257967 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:05.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f78a00640 1 --2- 172.21.15.16:0/319257967 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f50007540 0x7f5f740c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f78a00640 1 -- 172.21.15.16:0/319257967 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f50007540 msgr2=0x7f5f740c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:10:05.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f78a00640 1 -- 172.21.15.16:0/319257967 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f50007540 msgr2=0x7f5f740c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:05.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f78a00640 1 --2- 172.21.15.16:0/319257967 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f50007540 0x7f5f740c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:05.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f78a00640 1 --2- 172.21.15.16:0/319257967 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f50007540 0x7f5f740c7f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:05.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f6be00640 1 -- 172.21.15.16:0/319257967 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f50007540 msgr2=0x7f5f740c7f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:05.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f6be00640 1 --2- 172.21.15.16:0/319257967 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f50007540 0x7f5f740c7f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f6be00640 1 -- 172.21.15.16:0/319257967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f50003b60 msgr2=0x7f5f74151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f6be00640 1 --2- 172.21.15.16:0/319257967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f50003b60 0x7f5f74151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f6be00640 1 -- 172.21.15.16:0/319257967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f74153030 con 0x7f5f50006bf0 2024-09-10T21:10:05.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f6be00640 1 --2- 172.21.15.16:0/319257967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f50006bf0 0x7f5f740b5b60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c004d80 tx=0x7f5f5c014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:05.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f70c00640 1 -- 172.21.15.16:0/319257967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c012660 con 0x7f5f50006bf0 2024-09-10T21:10:05.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f70c00640 1 -- 172.21.15.16:0/319257967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f5c012820 con 0x7f5f50006bf0 2024-09-10T21:10:05.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f70c00640 1 -- 172.21.15.16:0/319257967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c012c70 con 0x7f5f50006bf0 2024-09-10T21:10:05.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f7aa00640 1 -- 172.21.15.16:0/319257967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f74151aa0 con 0x7f5f50006bf0 2024-09-10T21:10:05.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f7aa00640 1 -- 172.21.15.16:0/319257967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f74160470 con 0x7f5f50006bf0 2024-09-10T21:10:05.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f70c00640 1 -- 172.21.15.16:0/319257967 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f5f5c027440 con 0x7f5f50006bf0 2024-09-10T21:10:05.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.284+0000 7f5f70c00640 1 -- 172.21.15.16:0/319257967 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5f5c01eaf0 con 0x7f5f50006bf0 2024-09-10T21:10:05.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.288+0000 7f5f7aa00640 1 -- 172.21.15.16:0/319257967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f740d0db0 con 0x7f5f50006bf0 2024-09-10T21:10:05.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.288+0000 7f5f70c00640 1 -- 172.21.15.16:0/319257967 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f5f74153030 con 0x7f5f50006bf0 2024-09-10T21:10:05.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.307+0000 7f84f3400640 1 -- 172.21.15.16:0/3676872405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f84ec0b7590 con 0x7f84ec160050 2024-09-10T21:10:05.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.307+0000 7f84e8c00640 1 -- 172.21.15.16:0/3676872405 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v18)=0 v18) ==== 120+0+0 (secure 0 0 0) 0x7f84e4003d00 con 0x7f84ec160050 2024-09-10T21:10:05.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.311+0000 7f84f3400640 1 -- 172.21.15.16:0/3676872405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84ec160050 msgr2=0x7f84ec0b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.311+0000 7f84f3400640 1 --2- 172.21.15.16:0/3676872405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84ec160050 0x7f84ec0b5f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0061e0 tx=0x7f84e4013ce0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.311+0000 7f84f3400640 1 -- 172.21.15.16:0/3676872405 shutdown_connections 2024-09-10T21:10:05.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.311+0000 7f84f3400640 1 --2- 172.21.15.16:0/3676872405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84ec160050 0x7f84ec0b5f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.311+0000 7f84f3400640 1 --2- 172.21.15.16:0/3676872405 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f84cc006be0 0x7f84ec0cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.311+0000 7f84f3400640 1 --2- 172.21.15.16:0/3676872405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84cc003b30 0x7f84ec0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.311+0000 7f84f3400640 1 -- 172.21.15.16:0/3676872405 >> 172.21.15.16:0/3676872405 conn(0x7f84ec0d1ce0 msgr2=0x7f84ec0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:05.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.311+0000 7f84f3400640 1 -- 172.21.15.16:0/3676872405 shutdown_connections 2024-09-10T21:10:05.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.311+0000 7f84f3400640 1 -- 172.21.15.16:0/3676872405 wait complete. 2024-09-10T21:10:05.336 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi016') 2024-09-10T21:10:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.351+0000 7f5f7aa00640 1 -- 172.21.15.16:0/319257967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f5f740704a0 con 0x7f5f50006bf0 2024-09-10T21:10:05.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.355+0000 7f5f70c00640 1 -- 172.21.15.16:0/319257967 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f5f5c012e10 con 0x7f5f50006bf0 2024-09-10T21:10:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.355+0000 7f5f7aa00640 1 -- 172.21.15.16:0/319257967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f50006bf0 msgr2=0x7f5f740b5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.355+0000 7f5f7aa00640 1 --2- 172.21.15.16:0/319257967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f50006bf0 0x7f5f740b5b60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c004d80 tx=0x7f5f5c014830 comp rx=0 tx=0).stop 2024-09-10T21:10:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.355+0000 7f5f7aa00640 1 -- 172.21.15.16:0/319257967 shutdown_connections 2024-09-10T21:10:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.355+0000 7f5f7aa00640 1 --2- 172.21.15.16:0/319257967 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f50007540 0x7f5f740c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.355+0000 7f5f7aa00640 1 --2- 172.21.15.16:0/319257967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f50006bf0 0x7f5f740b5b60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.355+0000 7f5f7aa00640 1 --2- 172.21.15.16:0/319257967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f50003b60 0x7f5f74151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.355+0000 7f5f7aa00640 1 -- 172.21.15.16:0/319257967 >> 172.21.15.16:0/319257967 conn(0x7f5f740d1ce0 msgr2=0x7f5f740ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.355+0000 7f5f7aa00640 1 -- 172.21.15.16:0/319257967 shutdown_connections 2024-09-10T21:10:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.355+0000 7f5f7aa00640 1 -- 172.21.15.16:0/319257967 wait complete. 2024-09-10T21:10:05.381 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi016') 2024-09-10T21:10:05.381 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-09-10T21:10:05.381 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T21:10:05.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T21:10:05.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.563+0000 7f847ea00640 1 -- 172.21.15.16:0/2292408679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84780d02a0 msgr2=0x7f84780af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.563+0000 7f847ea00640 1 --2- 172.21.15.16:0/2292408679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84780d02a0 0x7f84780af690 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8470009c80 tx=0x7f847001cda0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.563+0000 7f847ea00640 1 -- 172.21.15.16:0/2292408679 shutdown_connections 2024-09-10T21:10:05.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.563+0000 7f847ea00640 1 --2- 172.21.15.16:0/2292408679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84780d02a0 0x7f84780af690 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.563+0000 7f847ea00640 1 --2- 172.21.15.16:0/2292408679 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f847815f3f0 0x7f84780b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.563+0000 7f847ea00640 1 -- 172.21.15.16:0/2292408679 >> 172.21.15.16:0/2292408679 conn(0x7f84780d1db0 msgr2=0x7f84780d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:05.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.563+0000 7f847ea00640 1 -- 172.21.15.16:0/2292408679 shutdown_connections 2024-09-10T21:10:05.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.563+0000 7f847ea00640 1 -- 172.21.15.16:0/2292408679 wait complete. 2024-09-10T21:10:05.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.563+0000 7f847ea00640 1 Processor -- start 2024-09-10T21:10:05.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.563+0000 7f847ea00640 1 -- start start 2024-09-10T21:10:05.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84780b2dc0 0x7f84780b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84780d02a0 0x7f8478155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ea00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f847815f3f0 0x7f847806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8478168d10 con 0x7f84780d02a0 2024-09-10T21:10:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ea00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f84780b09c0 con 0x7f84780b2dc0 2024-09-10T21:10:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ea00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8478168e90 con 0x7f847815f3f0 2024-09-10T21:10:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f8477400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84780d02a0 0x7f8478155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f8477400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84780d02a0 0x7f8478155300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53546/0 (socket says 172.21.15.16:53546) 2024-09-10T21:10:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f8477400640 1 -- 172.21.15.16:0/449357194 learned_addr learned my addr 172.21.15.16:0/449357194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f8477e00640 1 --2- 172.21.15.16:0/449357194 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84780b2dc0 0x7f84780b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ca00640 1 --2- 172.21.15.16:0/449357194 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f847815f3f0 0x7f847806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ca00640 1 -- 172.21.15.16:0/449357194 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f847815f3f0 msgr2=0x7f847806ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:10:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ca00640 1 -- 172.21.15.16:0/449357194 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f847815f3f0 msgr2=0x7f847806ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ca00640 1 --2- 172.21.15.16:0/449357194 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f847815f3f0 0x7f847806ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ca00640 1 --2- 172.21.15.16:0/449357194 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f847815f3f0 0x7f847806ee70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f8477400640 1 -- 172.21.15.16:0/449357194 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f847815f3f0 msgr2=0x7f847806ee70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f8477400640 1 --2- 172.21.15.16:0/449357194 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f847815f3f0 0x7f847806ee70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f8477400640 1 -- 172.21.15.16:0/449357194 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84780b2dc0 msgr2=0x7f84780b6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f8477400640 1 --2- 172.21.15.16:0/449357194 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84780b2dc0 0x7f84780b6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f8477400640 1 -- 172.21.15.16:0/449357194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84781f8a50 con 0x7f84780d02a0 2024-09-10T21:10:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f8477400640 1 --2- 172.21.15.16:0/449357194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84780d02a0 0x7f8478155300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f846400c400 tx=0x7f846400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f846fe00640 1 -- 172.21.15.16:0/449357194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464017070 con 0x7f84780d02a0 2024-09-10T21:10:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f846fe00640 1 -- 172.21.15.16:0/449357194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84640053e0 con 0x7f84780d02a0 2024-09-10T21:10:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ea00640 1 -- 172.21.15.16:0/449357194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84781f8d30 con 0x7f84780d02a0 2024-09-10T21:10:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f846fe00640 1 -- 172.21.15.16:0/449357194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464003860 con 0x7f84780d02a0 2024-09-10T21:10:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ea00640 1 -- 172.21.15.16:0/449357194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84781f92f0 con 0x7f84780d02a0 2024-09-10T21:10:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f846fe00640 1 -- 172.21.15.16:0/449357194 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f8464003a00 con 0x7f84780d02a0 2024-09-10T21:10:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f846fe00640 1 -- 172.21.15.16:0/449357194 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f846401c810 con 0x7f84780d02a0 2024-09-10T21:10:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.567+0000 7f847ea00640 1 -- 172.21.15.16:0/449357194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84780af690 con 0x7f84780d02a0 2024-09-10T21:10:05.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.571+0000 7f846fe00640 1 -- 172.21.15.16:0/449357194 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f8464021020 con 0x7f84780d02a0 2024-09-10T21:10:05.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.639+0000 7f847ea00640 1 -- 172.21.15.16:0/449357194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f84780c86b0 con 0x7f84780d02a0 2024-09-10T21:10:05.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.639+0000 7f846fe00640 1 -- 172.21.15.16:0/449357194 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f8464001130 con 0x7f84780d02a0 2024-09-10T21:10:05.647 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-09-10T21:10:05.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.643+0000 7f847ea00640 1 -- 172.21.15.16:0/449357194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84780d02a0 msgr2=0x7f8478155300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.643+0000 7f847ea00640 1 --2- 172.21.15.16:0/449357194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84780d02a0 0x7f8478155300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f846400c400 tx=0x7f846400c8c0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.643+0000 7f847ea00640 1 -- 172.21.15.16:0/449357194 shutdown_connections 2024-09-10T21:10:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.643+0000 7f847ea00640 1 --2- 172.21.15.16:0/449357194 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f847815f3f0 0x7f847806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.643+0000 7f847ea00640 1 --2- 172.21.15.16:0/449357194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84780d02a0 0x7f8478155300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.643+0000 7f847ea00640 1 --2- 172.21.15.16:0/449357194 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84780b2dc0 0x7f84780b6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.643+0000 7f847ea00640 1 -- 172.21.15.16:0/449357194 >> 172.21.15.16:0/449357194 conn(0x7f84780d1db0 msgr2=0x7f847815e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.643+0000 7f847ea00640 1 -- 172.21.15.16:0/449357194 shutdown_connections 2024-09-10T21:10:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:05.643+0000 7f847ea00640 1 -- 172.21.15.16:0/449357194 wait complete. 2024-09-10T21:10:05.667 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T21:10:05.667 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T21:10:05.667 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:10:05.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T21:10:05.687 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:10:05.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T21:10:05.755 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:10:05.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T21:10:05.818 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:10:05.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-10T21:10:05.879 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:10:05.879 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-10T21:10:05.900 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:10:05.900 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-10T21:10:05.961 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:10:05.961 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-10T21:10:06.021 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:10:06.021 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-10T21:10:06.081 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph osd new 30a917bd-7358-4830-b4fc-946e2f5ff128 0 2024-09-10T21:10:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 -- 172.21.15.52:0/1985667141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61e80cfe30 msgr2=0x7f61e80ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 --2- 172.21.15.52:0/1985667141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61e80cfe30 0x7f61e80ce590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009870 tx=0x7f61d801f010 comp rx=0 tx=0).stop 2024-09-10T21:10:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 -- 172.21.15.52:0/1985667141 shutdown_connections 2024-09-10T21:10:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 --2- 172.21.15.52:0/1985667141 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f61e80ce040 0x7f61e80b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 --2- 172.21.15.52:0/1985667141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61e80cfe30 0x7f61e80ce590 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 -- 172.21.15.52:0/1985667141 >> 172.21.15.52:0/1985667141 conn(0x7f61e80d1df0 msgr2=0x7f61e80d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 -- 172.21.15.52:0/1985667141 shutdown_connections 2024-09-10T21:10:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 -- 172.21.15.52:0/1985667141 wait complete. 2024-09-10T21:10:06.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 Processor -- start 2024-09-10T21:10:06.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 -- start start 2024-09-10T21:10:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e80ce040 0x7f61e8159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f61e80cfe30 0x7f61e806e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61e80d0210 0x7f61e806ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61e80d2b00 con 0x7f61e80ce040 2024-09-10T21:10:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f61e80b14f0 con 0x7f61e80d0210 2024-09-10T21:10:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.260+0000 7f61ecc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f61e80cf1e0 con 0x7f61e80cfe30 2024-09-10T21:10:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f61e80cfe30 0x7f61e806e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61e6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61e80d0210 0x7f61e806ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f61e80cfe30 0x7f61e806e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.52:57918/0 (socket says 172.21.15.52:57918) 2024-09-10T21:10:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61e6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61e80d0210 0x7f61e806ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60632/0 (socket says 172.21.15.52:60632) 2024-09-10T21:10:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61dfe00640 1 -- 172.21.15.52:0/3871951512 learned_addr learned my addr 172.21.15.52:0/3871951512 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-09-10T21:10:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61e5e00640 1 --2- 172.21.15.52:0/3871951512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e80ce040 0x7f61e8159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61dfe00640 1 -- 172.21.15.52:0/3871951512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f61e80cfe30 msgr2=0x7f61e806e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:10:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61dfe00640 1 -- 172.21.15.52:0/3871951512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f61e80cfe30 msgr2=0x7f61e806e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61dfe00640 1 --2- 172.21.15.52:0/3871951512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f61e80cfe30 0x7f61e806e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61dfe00640 1 --2- 172.21.15.52:0/3871951512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f61e80cfe30 0x7f61e806e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61e5e00640 1 -- 172.21.15.52:0/3871951512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f61e80cfe30 msgr2=0x7f61e806e9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61e5e00640 1 --2- 172.21.15.52:0/3871951512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f61e80cfe30 0x7f61e806e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61e5e00640 1 -- 172.21.15.52:0/3871951512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61e80d0210 msgr2=0x7f61e806ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61e5e00640 1 --2- 172.21.15.52:0/3871951512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61e80d0210 0x7f61e806ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61e5e00640 1 -- 172.21.15.52:0/3871951512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e815a5e0 con 0x7f61e80ce040 2024-09-10T21:10:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61e6800640 1 --2- 172.21.15.52:0/3871951512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61e80d0210 0x7f61e806ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61e5e00640 1 --2- 172.21.15.52:0/3871951512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e80ce040 0x7f61e8159e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f61d80af000 tx=0x7f61d80a8ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61de000640 1 -- 172.21.15.52:0/3871951512 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d80bf070 con 0x7f61e80ce040 2024-09-10T21:10:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61de000640 1 -- 172.21.15.52:0/3871951512 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61d80052f0 con 0x7f61e80ce040 2024-09-10T21:10:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61de000640 1 -- 172.21.15.52:0/3871951512 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d80b5040 con 0x7f61e80ce040 2024-09-10T21:10:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61ecc00640 1 -- 172.21.15.52:0/3871951512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e815a8c0 con 0x7f61e80ce040 2024-09-10T21:10:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61ecc00640 1 -- 172.21.15.52:0/3871951512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e815d310 con 0x7f61e80ce040 2024-09-10T21:10:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61de000640 1 -- 172.21.15.52:0/3871951512 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f61d80b8070 con 0x7f61e80ce040 2024-09-10T21:10:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61de000640 1 -- 172.21.15.52:0/3871951512 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f61d80b19c0 con 0x7f61e80ce040 2024-09-10T21:10:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.264+0000 7f61ecc00640 1 -- 172.21.15.52:0/3871951512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e80b1710 con 0x7f61e80ce040 2024-09-10T21:10:06.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.268+0000 7f61de000640 1 -- 172.21.15.52:0/3871951512 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f61d80dc950 con 0x7f61e80ce040 2024-09-10T21:10:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.340+0000 7f61ecc00640 1 -- 172.21.15.52:0/3871951512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd new", "uuid": "30a917bd-7358-4830-b4fc-946e2f5ff128", "id": 0} v 0) -- 0x7f61e80b2c00 con 0x7f61e80ce040 2024-09-10T21:10:06.349 INFO:teuthology.orchestra.run.smithi052.stdout:0 2024-09-10T21:10:06.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.340+0000 7f61de000640 1 -- 172.21.15.52:0/3871951512 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "30a917bd-7358-4830-b4fc-946e2f5ff128", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f61d80ca070 con 0x7f61e80ce040 2024-09-10T21:10:06.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.344+0000 7f61ecc00640 1 -- 172.21.15.52:0/3871951512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e80ce040 msgr2=0x7f61e8159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.344+0000 7f61ecc00640 1 --2- 172.21.15.52:0/3871951512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e80ce040 0x7f61e8159e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f61d80af000 tx=0x7f61d80a8ca0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.344+0000 7f61ecc00640 1 -- 172.21.15.52:0/3871951512 shutdown_connections 2024-09-10T21:10:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.344+0000 7f61ecc00640 1 --2- 172.21.15.52:0/3871951512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61e80d0210 0x7f61e806ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.344+0000 7f61ecc00640 1 --2- 172.21.15.52:0/3871951512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f61e80cfe30 0x7f61e806e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.344+0000 7f61ecc00640 1 --2- 172.21.15.52:0/3871951512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61e80ce040 0x7f61e8159e00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.344+0000 7f61ecc00640 1 -- 172.21.15.52:0/3871951512 >> 172.21.15.52:0/3871951512 conn(0x7f61e80d1df0 msgr2=0x7f61e80ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.344+0000 7f61ecc00640 1 -- 172.21.15.52:0/3871951512 shutdown_connections 2024-09-10T21:10:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.344+0000 7f61ecc00640 1 -- 172.21.15.52:0/3871951512 wait complete. 2024-09-10T21:10:06.369 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph osd new d0eef88e-6006-46df-a6d6-59a745737c54 1 2024-09-10T21:10:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.508+0000 7f07d5a00640 1 -- 172.21.15.52:0/2046374898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d00cfa00 msgr2=0x7f07d0074190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.508+0000 7f07d5a00640 1 --2- 172.21.15.52:0/2046374898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d00cfa00 0x7f07d0074190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009bc0 tx=0x7f07c801cba0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.508+0000 7f07d5a00640 1 -- 172.21.15.52:0/2046374898 shutdown_connections 2024-09-10T21:10:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.508+0000 7f07d5a00640 1 --2- 172.21.15.52:0/2046374898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d00cfa00 0x7f07d0074190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.508+0000 7f07d5a00640 1 -- 172.21.15.52:0/2046374898 >> 172.21.15.52:0/2046374898 conn(0x7f07d00b1d40 msgr2=0x7f07d00b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.508+0000 7f07d5a00640 1 -- 172.21.15.52:0/2046374898 shutdown_connections 2024-09-10T21:10:06.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.508+0000 7f07d5a00640 1 -- 172.21.15.52:0/2046374898 wait complete. 2024-09-10T21:10:06.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.508+0000 7f07d5a00640 1 Processor -- start 2024-09-10T21:10:06.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.508+0000 7f07d5a00640 1 -- start start 2024-09-10T21:10:06.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d00c37f0 0x7f07d00c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d00cfa00 0x7f07d00cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d015a420 0x7f07d015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07d0168080 con 0x7f07d00cfa00 2024-09-10T21:10:06.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07d0075150 con 0x7f07d00c37f0 2024-09-10T21:10:06.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f07d0168200 con 0x7f07d015a420 2024-09-10T21:10:06.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07cfe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d00c37f0 0x7f07d00c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d015a420 0x7f07d015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07cfe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d00c37f0 0x7f07d00c9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60644/0 (socket says 172.21.15.52:60644) 2024-09-10T21:10:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07cfe00640 1 -- 172.21.15.52:0/3391016350 learned_addr learned my addr 172.21.15.52:0/3391016350 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-09-10T21:10:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d015a420 0x7f07d015cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.52:57934/0 (socket says 172.21.15.52:57934) 2024-09-10T21:10:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07c7e00640 1 --2- 172.21.15.52:0/3391016350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d00cfa00 0x7f07d00cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5000640 1 -- 172.21.15.52:0/3391016350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d015a420 msgr2=0x7f07d015cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:10:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5000640 1 -- 172.21.15.52:0/3391016350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d015a420 msgr2=0x7f07d015cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5000640 1 --2- 172.21.15.52:0/3391016350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d015a420 0x7f07d015cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5000640 1 --2- 172.21.15.52:0/3391016350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d015a420 0x7f07d015cb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07cfe00640 1 -- 172.21.15.52:0/3391016350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d015a420 msgr2=0x7f07d015cb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07cfe00640 1 --2- 172.21.15.52:0/3391016350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d015a420 0x7f07d015cb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07cfe00640 1 -- 172.21.15.52:0/3391016350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d00cfa00 msgr2=0x7f07d00cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07cfe00640 1 --2- 172.21.15.52:0/3391016350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d00cfa00 0x7f07d00cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07cfe00640 1 -- 172.21.15.52:0/3391016350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d00997b0 con 0x7f07d00c37f0 2024-09-10T21:10:06.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07cfe00640 1 --2- 172.21.15.52:0/3391016350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d00c37f0 0x7f07d00c9a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f07c000c410 tx=0x7f07c000c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:06.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07ccc00640 1 -- 172.21.15.52:0/3391016350 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0017070 con 0x7f07d00c37f0 2024-09-10T21:10:06.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07ccc00640 1 -- 172.21.15.52:0/3391016350 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07c00053c0 con 0x7f07d00c37f0 2024-09-10T21:10:06.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07ccc00640 1 -- 172.21.15.52:0/3391016350 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0003860 con 0x7f07d00c37f0 2024-09-10T21:10:06.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5a00640 1 -- 172.21.15.52:0/3391016350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d0098430 con 0x7f07d00c37f0 2024-09-10T21:10:06.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5a00640 1 -- 172.21.15.52:0/3391016350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d00989f0 con 0x7f07d00c37f0 2024-09-10T21:10:06.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07ccc00640 1 -- 172.21.15.52:0/3391016350 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f07c0003a00 con 0x7f07d00c37f0 2024-09-10T21:10:06.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07ccc00640 1 -- 172.21.15.52:0/3391016350 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f07c001c4d0 con 0x7f07d00c37f0 2024-09-10T21:10:06.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.512+0000 7f07d5a00640 1 -- 172.21.15.52:0/3391016350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d00992e0 con 0x7f07d00c37f0 2024-09-10T21:10:06.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.516+0000 7f07ccc00640 1 -- 172.21.15.52:0/3391016350 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f07c0021070 con 0x7f07d00c37f0 2024-09-10T21:10:06.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.588+0000 7f07d5a00640 1 -- 172.21.15.52:0/3391016350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d0eef88e-6006-46df-a6d6-59a745737c54", "id": 1} v 0) -- 0x7f07d0074190 con 0x7f07d00c37f0 2024-09-10T21:10:06.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.592+0000 7f07ccc00640 1 -- 172.21.15.52:0/3391016350 <== mon.1 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d0eef88e-6006-46df-a6d6-59a745737c54", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f07c0011590 con 0x7f07d00c37f0 2024-09-10T21:10:06.599 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-09-10T21:10:06.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.592+0000 7f07d5a00640 1 -- 172.21.15.52:0/3391016350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d00c37f0 msgr2=0x7f07d00c9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.592+0000 7f07d5a00640 1 --2- 172.21.15.52:0/3391016350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d00c37f0 0x7f07d00c9a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f07c000c410 tx=0x7f07c000c8d0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.592+0000 7f07d5a00640 1 -- 172.21.15.52:0/3391016350 shutdown_connections 2024-09-10T21:10:06.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.592+0000 7f07d5a00640 1 --2- 172.21.15.52:0/3391016350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d015a420 0x7f07d015cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.592+0000 7f07d5a00640 1 --2- 172.21.15.52:0/3391016350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d00cfa00 0x7f07d00cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.592+0000 7f07d5a00640 1 --2- 172.21.15.52:0/3391016350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d00c37f0 0x7f07d00c9a60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.592+0000 7f07d5a00640 1 -- 172.21.15.52:0/3391016350 >> 172.21.15.52:0/3391016350 conn(0x7f07d00b1d40 msgr2=0x7f07d00b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.592+0000 7f07d5a00640 1 -- 172.21.15.52:0/3391016350 shutdown_connections 2024-09-10T21:10:06.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.592+0000 7f07d5a00640 1 -- 172.21.15.52:0/3391016350 wait complete. 2024-09-10T21:10:06.618 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph osd new 81d386aa-827c-4249-9805-b7e752a7918f 2 2024-09-10T21:10:06.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.792+0000 7f8f99c00640 1 -- 172.21.15.52:0/349187193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f78003bf0 msgr2=0x7f8f78003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.792+0000 7f8f99c00640 1 --2- 172.21.15.52:0/349187193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f78003bf0 0x7f8f78003ff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f84006aa0 tx=0x7f8f8401f050 comp rx=0 tx=0).stop 2024-09-10T21:10:06.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.792+0000 7f8f99200640 1 -- 172.21.15.52:0/349187193 reap_dead start 2024-09-10T21:10:06.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.792+0000 7f8f99c00640 1 -- 172.21.15.52:0/349187193 shutdown_connections 2024-09-10T21:10:06.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.792+0000 7f8f99c00640 1 -- 172.21.15.52:0/349187193 >> 172.21.15.52:0/349187193 conn(0x7f8f940d0a80 msgr2=0x7f8f940d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:06.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.792+0000 7f8f99c00640 1 -- 172.21.15.52:0/349187193 shutdown_connections 2024-09-10T21:10:06.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.792+0000 7f8f99c00640 1 -- 172.21.15.52:0/349187193 wait complete. 2024-09-10T21:10:06.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.792+0000 7f8f99c00640 1 Processor -- start 2024-09-10T21:10:06.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.792+0000 7f8f99c00640 1 -- start start 2024-09-10T21:10:06.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f99c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f78003bf0 0x7f8f941f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f99c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f9415db60 0x7f8f940c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f99c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f94158610 0x7f8f94159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f99c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94073e10 con 0x7f8f94158610 2024-09-10T21:10:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f99c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94073c90 con 0x7f8f9415db60 2024-09-10T21:10:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f99c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9415c330 con 0x7f8f78003bf0 2024-09-10T21:10:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f78003bf0 0x7f8f941f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f8be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f9415db60 0x7f8f940c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f78003bf0 0x7f8f941f7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.52:57966/0 (socket says 172.21.15.52:57966) 2024-09-10T21:10:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f93e00640 1 -- 172.21.15.52:0/2017587346 learned_addr learned my addr 172.21.15.52:0/2017587346 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-09-10T21:10:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f99200640 1 --2- 172.21.15.52:0/2017587346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f94158610 0x7f8f94159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f93e00640 1 -- 172.21.15.52:0/2017587346 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f78003bf0 msgr2=0x7f8f941f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:10:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f93e00640 1 -- 172.21.15.52:0/2017587346 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f78003bf0 msgr2=0x7f8f941f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f93e00640 1 --2- 172.21.15.52:0/2017587346 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f78003bf0 0x7f8f941f7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f93e00640 1 --2- 172.21.15.52:0/2017587346 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f78003bf0 0x7f8f941f7c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:06.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f8be00640 1 -- 172.21.15.52:0/2017587346 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f78003bf0 msgr2=0x7f8f941f7c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:06.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f8be00640 1 --2- 172.21.15.52:0/2017587346 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f78003bf0 0x7f8f941f7c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f8be00640 1 -- 172.21.15.52:0/2017587346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f94158610 msgr2=0x7f8f94159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f8be00640 1 --2- 172.21.15.52:0/2017587346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f94158610 0x7f8f94159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f8be00640 1 -- 172.21.15.52:0/2017587346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f941f8150 con 0x7f8f9415db60 2024-09-10T21:10:06.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f99200640 1 --2- 172.21.15.52:0/2017587346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f94158610 0x7f8f94159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:06.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f8be00640 1 --2- 172.21.15.52:0/2017587346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f9415db60 0x7f8f940c7ce0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c005e10 tx=0x7f8f7c011740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:06.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f90c00640 1 -- 172.21.15.52:0/2017587346 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c011e50 con 0x7f8f9415db60 2024-09-10T21:10:06.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f90c00640 1 -- 172.21.15.52:0/2017587346 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f7c018020 con 0x7f8f9415db60 2024-09-10T21:10:06.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f90c00640 1 -- 172.21.15.52:0/2017587346 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c003600 con 0x7f8f9415db60 2024-09-10T21:10:06.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f99c00640 1 -- 172.21.15.52:0/2017587346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f941f8400 con 0x7f8f9415db60 2024-09-10T21:10:06.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f99c00640 1 -- 172.21.15.52:0/2017587346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f941f89c0 con 0x7f8f9415db60 2024-09-10T21:10:06.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f90c00640 1 -- 172.21.15.52:0/2017587346 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f8f7c010040 con 0x7f8f9415db60 2024-09-10T21:10:06.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f90c00640 1 -- 172.21.15.52:0/2017587346 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f8f7c013c70 con 0x7f8f9415db60 2024-09-10T21:10:06.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.796+0000 7f8f99c00640 1 -- 172.21.15.52:0/2017587346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f941409d0 con 0x7f8f9415db60 2024-09-10T21:10:06.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.800+0000 7f8f90c00640 1 -- 172.21.15.52:0/2017587346 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f8f7c022020 con 0x7f8f9415db60 2024-09-10T21:10:06.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.872+0000 7f8f99c00640 1 -- 172.21.15.52:0/2017587346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd new", "uuid": "81d386aa-827c-4249-9805-b7e752a7918f", "id": 2} v 0) -- 0x7f8f94074800 con 0x7f8f9415db60 2024-09-10T21:10:06.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.876+0000 7f8f90c00640 1 -- 172.21.15.52:0/2017587346 <== mon.1 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "81d386aa-827c-4249-9805-b7e752a7918f", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f8f7c003b30 con 0x7f8f9415db60 2024-09-10T21:10:06.885 INFO:teuthology.orchestra.run.smithi052.stdout:2 2024-09-10T21:10:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.876+0000 7f8f99c00640 1 -- 172.21.15.52:0/2017587346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f9415db60 msgr2=0x7f8f940c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.876+0000 7f8f99c00640 1 --2- 172.21.15.52:0/2017587346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f9415db60 0x7f8f940c7ce0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c005e10 tx=0x7f8f7c011740 comp rx=0 tx=0).stop 2024-09-10T21:10:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.876+0000 7f8f99c00640 1 -- 172.21.15.52:0/2017587346 shutdown_connections 2024-09-10T21:10:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.876+0000 7f8f99c00640 1 --2- 172.21.15.52:0/2017587346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f94158610 0x7f8f94159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.876+0000 7f8f99c00640 1 --2- 172.21.15.52:0/2017587346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f9415db60 0x7f8f940c7ce0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.876+0000 7f8f99c00640 1 --2- 172.21.15.52:0/2017587346 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f78003bf0 0x7f8f941f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.876+0000 7f8f99c00640 1 -- 172.21.15.52:0/2017587346 >> 172.21.15.52:0/2017587346 conn(0x7f8f940d0a80 msgr2=0x7f8f940cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.880+0000 7f8f99c00640 1 -- 172.21.15.52:0/2017587346 shutdown_connections 2024-09-10T21:10:06.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:06.880+0000 7f8f99c00640 1 -- 172.21.15.52:0/2017587346 wait complete. 2024-09-10T21:10:06.905 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph osd new eaaf4ff6-2335-43b1-9d7d-34360073abe1 3 2024-09-10T21:10:07.038 INFO:tasks.ceph.mgr.x.smithi052.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T21:10:07.039 INFO:tasks.ceph.mgr.x.smithi052.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T21:10:07.039 INFO:tasks.ceph.mgr.x.smithi052.stderr: from numpy import show_config as show_numpy_config 2024-09-10T21:10:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.040+0000 7fefc5a00640 1 -- 172.21.15.52:0/51956992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc0161430 msgr2=0x7fefc00d1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.040+0000 7fefc5a00640 1 --2- 172.21.15.52:0/51956992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc0161430 0x7fefc00d1aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009900 tx=0x7fefb001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.040+0000 7fefc5a00640 1 -- 172.21.15.52:0/51956992 shutdown_connections 2024-09-10T21:10:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.040+0000 7fefc5a00640 1 --2- 172.21.15.52:0/51956992 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fefc0161810 0x7fefc00d1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.040+0000 7fefc5a00640 1 --2- 172.21.15.52:0/51956992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc0161430 0x7fefc00d1aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.040+0000 7fefc5a00640 1 -- 172.21.15.52:0/51956992 >> 172.21.15.52:0/51956992 conn(0x7fefc00d3ed0 msgr2=0x7fefc00d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.040+0000 7fefc5a00640 1 -- 172.21.15.52:0/51956992 shutdown_connections 2024-09-10T21:10:07.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.040+0000 7fefc5a00640 1 -- 172.21.15.52:0/51956992 wait complete. 2024-09-10T21:10:07.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefc5a00640 1 Processor -- start 2024-09-10T21:10:07.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefc5a00640 1 -- start start 2024-09-10T21:10:07.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefc5a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc00cf480 0x7fefc015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefc5a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fefc0161430 0x7fefc00b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefc5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc0161810 0x7fefc015dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefc5a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefc00d2930 con 0x7fefc0161810 2024-09-10T21:10:07.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefc5a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fefc00d27b0 con 0x7fefc00cf480 2024-09-10T21:10:07.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefc5a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fefc00d0ed0 con 0x7fefc0161430 2024-09-10T21:10:07.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefbea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc00cf480 0x7fefc015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefb7e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fefc0161430 0x7fefc00b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefbea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc00cf480 0x7fefc015a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60686/0 (socket says 172.21.15.52:60686) 2024-09-10T21:10:07.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefbea00640 1 -- 172.21.15.52:0/363427507 learned_addr learned my addr 172.21.15.52:0/363427507 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-09-10T21:10:07.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefbf400640 1 --2- 172.21.15.52:0/363427507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc0161810 0x7fefc015dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefb7e00640 1 -- 172.21.15.52:0/363427507 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fefc0161430 msgr2=0x7fefc00b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:10:07.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefb7e00640 1 -- 172.21.15.52:0/363427507 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fefc0161430 msgr2=0x7fefc00b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:07.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefb7e00640 1 --2- 172.21.15.52:0/363427507 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fefc0161430 0x7fefc00b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:07.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefb7e00640 1 --2- 172.21.15.52:0/363427507 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fefc0161430 0x7fefc00b7a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:07.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefbea00640 1 -- 172.21.15.52:0/363427507 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fefc0161430 msgr2=0x7fefc00b7a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:07.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefbea00640 1 --2- 172.21.15.52:0/363427507 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fefc0161430 0x7fefc00b7a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefbea00640 1 -- 172.21.15.52:0/363427507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc0161810 msgr2=0x7fefc015dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefbea00640 1 --2- 172.21.15.52:0/363427507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc0161810 0x7fefc015dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefbea00640 1 -- 172.21.15.52:0/363427507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc015e770 con 0x7fefc00cf480 2024-09-10T21:10:07.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefbea00640 1 --2- 172.21.15.52:0/363427507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc00cf480 0x7fefc015a550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009850 tx=0x7fefb00a8ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:07.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefb6a00640 1 -- 172.21.15.52:0/363427507 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb00be070 con 0x7fefc00cf480 2024-09-10T21:10:07.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefb6a00640 1 -- 172.21.15.52:0/363427507 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefb0005400 con 0x7fefc00cf480 2024-09-10T21:10:07.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefb6a00640 1 -- 172.21.15.52:0/363427507 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb00b5040 con 0x7fefc00cf480 2024-09-10T21:10:07.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefc5a00640 1 -- 172.21.15.52:0/363427507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc00b6f70 con 0x7fefc00cf480 2024-09-10T21:10:07.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefc5a00640 1 -- 172.21.15.52:0/363427507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc00b7530 con 0x7fefc00cf480 2024-09-10T21:10:07.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefb6a00640 1 -- 172.21.15.52:0/363427507 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fefb00b8030 con 0x7fefc00cf480 2024-09-10T21:10:07.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.044+0000 7fefb6a00640 1 -- 172.21.15.52:0/363427507 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fefb0003c50 con 0x7fefc00cf480 2024-09-10T21:10:07.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.048+0000 7fefc5a00640 1 -- 172.21.15.52:0/363427507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc00b6d80 con 0x7fefc00cf480 2024-09-10T21:10:07.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.048+0000 7fefb6a00640 1 -- 172.21.15.52:0/363427507 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fefb00c8020 con 0x7fefc00cf480 2024-09-10T21:10:07.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.120+0000 7fefc5a00640 1 -- 172.21.15.52:0/363427507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd new", "uuid": "eaaf4ff6-2335-43b1-9d7d-34360073abe1", "id": 3} v 0) -- 0x7fefc0050fd0 con 0x7fefc00cf480 2024-09-10T21:10:07.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.124+0000 7fefb6a00640 1 -- 172.21.15.52:0/363427507 <== mon.1 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "eaaf4ff6-2335-43b1-9d7d-34360073abe1", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fefb0018c30 con 0x7fefc00cf480 2024-09-10T21:10:07.134 INFO:teuthology.orchestra.run.smithi052.stdout:3 2024-09-10T21:10:07.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.128+0000 7fefc5a00640 1 -- 172.21.15.52:0/363427507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc00cf480 msgr2=0x7fefc015a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.128+0000 7fefc5a00640 1 --2- 172.21.15.52:0/363427507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc00cf480 0x7fefc015a550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009850 tx=0x7fefb00a8ce0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.128+0000 7fefc5a00640 1 -- 172.21.15.52:0/363427507 shutdown_connections 2024-09-10T21:10:07.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.128+0000 7fefc5a00640 1 --2- 172.21.15.52:0/363427507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc0161810 0x7fefc015dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.128+0000 7fefc5a00640 1 --2- 172.21.15.52:0/363427507 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fefc0161430 0x7fefc00b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.128+0000 7fefc5a00640 1 --2- 172.21.15.52:0/363427507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefc00cf480 0x7fefc015a550 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.128+0000 7fefc5a00640 1 -- 172.21.15.52:0/363427507 >> 172.21.15.52:0/363427507 conn(0x7fefc00d3ed0 msgr2=0x7fefc01606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.128+0000 7fefc5a00640 1 -- 172.21.15.52:0/363427507 shutdown_connections 2024-09-10T21:10:07.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.128+0000 7fefc5a00640 1 -- 172.21.15.52:0/363427507 wait complete. 2024-09-10T21:10:07.154 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph osd new cb466bfa-3956-40f8-9a8f-89f479318524 4 2024-09-10T21:10:07.197 INFO:tasks.ceph.mgr.y.smithi016.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T21:10:07.197 INFO:tasks.ceph.mgr.y.smithi016.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T21:10:07.197 INFO:tasks.ceph.mgr.y.smithi016.stderr: from numpy import show_config as show_numpy_config 2024-09-10T21:10:07.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.292+0000 7fa640c00640 1 -- 172.21.15.52:0/44558428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa63c15f940 msgr2=0x7fa63c15fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.292+0000 7fa640c00640 1 --2- 172.21.15.52:0/44558428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa63c15f940 0x7fa63c15fd90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa634009bc0 tx=0x7fa63401ca80 comp rx=0 tx=0).stop 2024-09-10T21:10:07.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.292+0000 7fa640c00640 1 -- 172.21.15.52:0/44558428 shutdown_connections 2024-09-10T21:10:07.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.292+0000 7fa640c00640 1 --2- 172.21.15.52:0/44558428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa63c15f940 0x7fa63c15fd90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.292+0000 7fa640c00640 1 --2- 172.21.15.52:0/44558428 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa63c15bdc0 0x7fa63c0d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.292+0000 7fa640c00640 1 -- 172.21.15.52:0/44558428 >> 172.21.15.52:0/44558428 conn(0x7fa63c0d1ce0 msgr2=0x7fa63c0d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:07.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.292+0000 7fa640c00640 1 -- 172.21.15.52:0/44558428 shutdown_connections 2024-09-10T21:10:07.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.292+0000 7fa640c00640 1 -- 172.21.15.52:0/44558428 wait complete. 2024-09-10T21:10:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.292+0000 7fa640c00640 1 Processor -- start 2024-09-10T21:10:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.292+0000 7fa640c00640 1 -- start start 2024-09-10T21:10:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa640c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63c0cd300 0x7fa63c14dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa640c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa63c15bdc0 0x7fa63c0c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa640c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa63c15f940 0x7fa63c070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa640c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c169120 con 0x7fa63c0cd300 2024-09-10T21:10:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa640c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c168fa0 con 0x7fa63c15f940 2024-09-10T21:10:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa640c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa63c1692a0 con 0x7fa63c15bdc0 2024-09-10T21:10:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa639400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa63c15bdc0 0x7fa63c0c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa63a800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa63c15f940 0x7fa63c070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa639400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa63c15bdc0 0x7fa63c0c6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.52:57994/0 (socket says 172.21.15.52:57994) 2024-09-10T21:10:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa639400640 1 -- 172.21.15.52:0/3938986548 learned_addr learned my addr 172.21.15.52:0/3938986548 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-09-10T21:10:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa639e00640 1 --2- 172.21.15.52:0/3938986548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63c0cd300 0x7fa63c14dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa639400640 1 -- 172.21.15.52:0/3938986548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa63c15bdc0 msgr2=0x7fa63c0c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:10:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa639400640 1 -- 172.21.15.52:0/3938986548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa63c15bdc0 msgr2=0x7fa63c0c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa639400640 1 --2- 172.21.15.52:0/3938986548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa63c15bdc0 0x7fa63c0c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa639400640 1 --2- 172.21.15.52:0/3938986548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa63c15bdc0 0x7fa63c0c6ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa63a800640 1 -- 172.21.15.52:0/3938986548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa63c15bdc0 msgr2=0x7fa63c0c6ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa63a800640 1 --2- 172.21.15.52:0/3938986548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa63c15bdc0 0x7fa63c0c6ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa63a800640 1 -- 172.21.15.52:0/3938986548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63c0cd300 msgr2=0x7fa63c14dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa63a800640 1 --2- 172.21.15.52:0/3938986548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63c0cd300 0x7fa63c14dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa63a800640 1 -- 172.21.15.52:0/3938986548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63c0b33f0 con 0x7fa63c15f940 2024-09-10T21:10:07.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa63a800640 1 --2- 172.21.15.52:0/3938986548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa63c15f940 0x7fa63c070de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0cfaa0 tx=0x7fa6340a57d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:07.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa632000640 1 -- 172.21.15.52:0/3938986548 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6340bb070 con 0x7fa63c15f940 2024-09-10T21:10:07.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa632000640 1 -- 172.21.15.52:0/3938986548 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa634004e00 con 0x7fa63c15f940 2024-09-10T21:10:07.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa640c00640 1 -- 172.21.15.52:0/3938986548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63c157880 con 0x7fa63c15f940 2024-09-10T21:10:07.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa632000640 1 -- 172.21.15.52:0/3938986548 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6340b2040 con 0x7fa63c15f940 2024-09-10T21:10:07.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa640c00640 1 -- 172.21.15.52:0/3938986548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa63c1f95a0 con 0x7fa63c15f940 2024-09-10T21:10:07.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa632000640 1 -- 172.21.15.52:0/3938986548 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fa6340b5030 con 0x7fa63c15f940 2024-09-10T21:10:07.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa632000640 1 -- 172.21.15.52:0/3938986548 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fa6340a59d0 con 0x7fa63c15f940 2024-09-10T21:10:07.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.296+0000 7fa640c00640 1 -- 172.21.15.52:0/3938986548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa63c0d2800 con 0x7fa63c15f940 2024-09-10T21:10:07.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.300+0000 7fa632000640 1 -- 172.21.15.52:0/3938986548 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fa634016790 con 0x7fa63c15f940 2024-09-10T21:10:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.372+0000 7fa640c00640 1 -- 172.21.15.52:0/3938986548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cb466bfa-3956-40f8-9a8f-89f479318524", "id": 4} v 0) -- 0x7fa63c0704a0 con 0x7fa63c15f940 2024-09-10T21:10:07.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.376+0000 7fa632000640 1 -- 172.21.15.52:0/3938986548 <== mon.1 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cb466bfa-3956-40f8-9a8f-89f479318524", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fa634004320 con 0x7fa63c15f940 2024-09-10T21:10:07.385 INFO:teuthology.orchestra.run.smithi052.stdout:4 2024-09-10T21:10:07.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.376+0000 7fa640c00640 1 -- 172.21.15.52:0/3938986548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa63c15f940 msgr2=0x7fa63c070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.376+0000 7fa640c00640 1 --2- 172.21.15.52:0/3938986548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa63c15f940 0x7fa63c070de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0cfaa0 tx=0x7fa6340a57d0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.376+0000 7fa640c00640 1 -- 172.21.15.52:0/3938986548 shutdown_connections 2024-09-10T21:10:07.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.376+0000 7fa640c00640 1 --2- 172.21.15.52:0/3938986548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa63c15f940 0x7fa63c070de0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.376+0000 7fa640c00640 1 --2- 172.21.15.52:0/3938986548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa63c15bdc0 0x7fa63c0c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.376+0000 7fa640c00640 1 --2- 172.21.15.52:0/3938986548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63c0cd300 0x7fa63c14dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.376+0000 7fa640c00640 1 -- 172.21.15.52:0/3938986548 >> 172.21.15.52:0/3938986548 conn(0x7fa63c0d1ce0 msgr2=0x7fa63c0b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:07.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.376+0000 7fa640c00640 1 -- 172.21.15.52:0/3938986548 shutdown_connections 2024-09-10T21:10:07.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.376+0000 7fa640c00640 1 -- 172.21.15.52:0/3938986548 wait complete. 2024-09-10T21:10:07.403 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph osd new c47d43ea-abe2-4fe8-af77-699b862309b7 5 2024-09-10T21:10:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.540+0000 7f4d61800640 1 -- 172.21.15.52:0/2192336603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d40004da0 msgr2=0x7f4d40005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.540+0000 7f4d61800640 1 --2- 172.21.15.52:0/2192336603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d40004da0 0x7f4d40005220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400c610 tx=0x7f4d5401d220 comp rx=0 tx=0).stop 2024-09-10T21:10:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.540+0000 7f4d5b400640 1 -- 172.21.15.52:0/2192336603 reap_dead start 2024-09-10T21:10:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.540+0000 7f4d61800640 1 -- 172.21.15.52:0/2192336603 shutdown_connections 2024-09-10T21:10:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.540+0000 7f4d61800640 1 -- 172.21.15.52:0/2192336603 >> 172.21.15.52:0/2192336603 conn(0x7f4d5c0d3ed0 msgr2=0x7f4d5c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.540+0000 7f4d61800640 1 -- 172.21.15.52:0/2192336603 shutdown_connections 2024-09-10T21:10:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.540+0000 7f4d61800640 1 -- 172.21.15.52:0/2192336603 wait complete. 2024-09-10T21:10:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.540+0000 7f4d61800640 1 Processor -- start 2024-09-10T21:10:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d61800640 1 -- start start 2024-09-10T21:10:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d61800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d40004da0 0x7f4d5c0cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d61800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d5c0cd120 0x7f4d5c156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d61800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c1f7fc0 0x7f4d5c1f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d61800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c0d28f0 con 0x7f4d5c1f7fc0 2024-09-10T21:10:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d61800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c0d2770 con 0x7f4d5c0cd120 2024-09-10T21:10:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d61800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4d5c0b4800 con 0x7f4d40004da0 2024-09-10T21:10:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d53e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d5c0cd120 0x7f4d5c156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d5aa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d40004da0 0x7f4d5c0cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d53e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d5c0cd120 0x7f4d5c156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60708/0 (socket says 172.21.15.52:60708) 2024-09-10T21:10:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d53e00640 1 -- 172.21.15.52:0/2708591409 learned_addr learned my addr 172.21.15.52:0/2708591409 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-09-10T21:10:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d5aa00640 1 -- 172.21.15.52:0/2708591409 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d40004da0 msgr2=0x7f4d5c0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:10:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d5aa00640 1 -- 172.21.15.52:0/2708591409 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d40004da0 msgr2=0x7f4d5c0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d5aa00640 1 --2- 172.21.15.52:0/2708591409 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d40004da0 0x7f4d5c0cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d5b400640 1 --2- 172.21.15.52:0/2708591409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c1f7fc0 0x7f4d5c1f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d5aa00640 1 --2- 172.21.15.52:0/2708591409 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d40004da0 0x7f4d5c0cd590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d53e00640 1 -- 172.21.15.52:0/2708591409 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d40004da0 msgr2=0x7f4d5c0cd590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d53e00640 1 --2- 172.21.15.52:0/2708591409 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d40004da0 0x7f4d5c0cd590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d53e00640 1 -- 172.21.15.52:0/2708591409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c1f7fc0 msgr2=0x7f4d5c1f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d53e00640 1 --2- 172.21.15.52:0/2708591409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c1f7fc0 0x7f4d5c1f8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d53e00640 1 -- 172.21.15.52:0/2708591409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5c1568a0 con 0x7f4d5c0cd120 2024-09-10T21:10:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d53e00640 1 --2- 172.21.15.52:0/2708591409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d5c0cd120 0x7f4d5c156360 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d44010ad0 tx=0x7f4d44010f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d52a00640 1 -- 172.21.15.52:0/2708591409 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4400ebf0 con 0x7f4d5c0cd120 2024-09-10T21:10:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d61800640 1 -- 172.21.15.52:0/2708591409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5c0cc2c0 con 0x7f4d5c0cd120 2024-09-10T21:10:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d52a00640 1 -- 172.21.15.52:0/2708591409 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d4400ed90 con 0x7f4d5c0cd120 2024-09-10T21:10:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d52a00640 1 -- 172.21.15.52:0/2708591409 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44005280 con 0x7f4d5c0cd120 2024-09-10T21:10:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d61800640 1 -- 172.21.15.52:0/2708591409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d5c0cc880 con 0x7f4d5c0cd120 2024-09-10T21:10:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d52a00640 1 -- 172.21.15.52:0/2708591409 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f4d440047b0 con 0x7f4d5c0cd120 2024-09-10T21:10:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d52a00640 1 -- 172.21.15.52:0/2708591409 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f4d4401abb0 con 0x7f4d5c0cd120 2024-09-10T21:10:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.544+0000 7f4d61800640 1 -- 172.21.15.52:0/2708591409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d5c0b40f0 con 0x7f4d5c0cd120 2024-09-10T21:10:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.548+0000 7f4d52a00640 1 -- 172.21.15.52:0/2708591409 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f4d4401adb0 con 0x7f4d5c0cd120 2024-09-10T21:10:07.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.632+0000 7f4d61800640 1 -- 172.21.15.52:0/2708591409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c47d43ea-abe2-4fe8-af77-699b862309b7", "id": 5} v 0) -- 0x7f4d5c0d48a0 con 0x7f4d5c0cd120 2024-09-10T21:10:07.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.636+0000 7f4d52a00640 1 -- 172.21.15.52:0/2708591409 <== mon.1 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c47d43ea-abe2-4fe8-af77-699b862309b7", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f4d4401a550 con 0x7f4d5c0cd120 2024-09-10T21:10:07.645 INFO:teuthology.orchestra.run.smithi052.stdout:5 2024-09-10T21:10:07.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.640+0000 7f4d61800640 1 -- 172.21.15.52:0/2708591409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d5c0cd120 msgr2=0x7f4d5c156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.640+0000 7f4d61800640 1 --2- 172.21.15.52:0/2708591409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d5c0cd120 0x7f4d5c156360 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d44010ad0 tx=0x7f4d44010f90 comp rx=0 tx=0).stop 2024-09-10T21:10:07.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.640+0000 7f4d61800640 1 -- 172.21.15.52:0/2708591409 shutdown_connections 2024-09-10T21:10:07.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.640+0000 7f4d61800640 1 --2- 172.21.15.52:0/2708591409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c1f7fc0 0x7f4d5c1f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.640+0000 7f4d61800640 1 --2- 172.21.15.52:0/2708591409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d5c0cd120 0x7f4d5c156360 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.640+0000 7f4d61800640 1 --2- 172.21.15.52:0/2708591409 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d40004da0 0x7f4d5c0cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.640+0000 7f4d61800640 1 -- 172.21.15.52:0/2708591409 >> 172.21.15.52:0/2708591409 conn(0x7f4d5c0d3ed0 msgr2=0x7f4d5c0b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:07.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.640+0000 7f4d61800640 1 -- 172.21.15.52:0/2708591409 shutdown_connections 2024-09-10T21:10:07.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.640+0000 7f4d61800640 1 -- 172.21.15.52:0/2708591409 wait complete. 2024-09-10T21:10:07.664 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph osd new 069eefff-abdc-4750-86b5-f836d3d6c23b 6 2024-09-10T21:10:07.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 -- 172.21.15.52:0/1014625205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facf0004c30 msgr2=0x7facf0005010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 --2- 172.21.15.52:0/1014625205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facf0004c30 0x7facf0005010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7facfc00baf0 tx=0x7facfc01cd80 comp rx=0 tx=0).stop 2024-09-10T21:10:07.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad0b200640 1 -- 172.21.15.52:0/1014625205 reap_dead start 2024-09-10T21:10:07.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 -- 172.21.15.52:0/1014625205 shutdown_connections 2024-09-10T21:10:07.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 -- 172.21.15.52:0/1014625205 >> 172.21.15.52:0/1014625205 conn(0x7fad0c0d1ce0 msgr2=0x7fad0c0d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:07.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 -- 172.21.15.52:0/1014625205 shutdown_connections 2024-09-10T21:10:07.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 -- 172.21.15.52:0/1014625205 wait complete. 2024-09-10T21:10:07.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 Processor -- start 2024-09-10T21:10:07.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 -- start start 2024-09-10T21:10:07.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facf0004c30 0x7fad0c0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c167560 0x7fad0c0cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad0c14ac00 0x7fad0c14b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1602d0 con 0x7fad0c167560 2024-09-10T21:10:07.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c160150 con 0x7fad0c14ac00 2024-09-10T21:10:07.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad11400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fad0c168490 con 0x7facf0004c30 2024-09-10T21:10:07.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad0a800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facf0004c30 0x7fad0c0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad0b200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad0c14ac00 0x7fad0c14b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad0a800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facf0004c30 0x7fad0c0c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.52:58016/0 (socket says 172.21.15.52:58016) 2024-09-10T21:10:07.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad0a800640 1 -- 172.21.15.52:0/2428328864 learned_addr learned my addr 172.21.15.52:0/2428328864 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-09-10T21:10:07.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad03e00640 1 --2- 172.21.15.52:0/2428328864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c167560 0x7fad0c0cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad0a800640 1 -- 172.21.15.52:0/2428328864 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facf0004c30 msgr2=0x7fad0c0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:10:07.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad0a800640 1 -- 172.21.15.52:0/2428328864 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facf0004c30 msgr2=0x7fad0c0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:07.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad0a800640 1 --2- 172.21.15.52:0/2428328864 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facf0004c30 0x7fad0c0c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:07.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.804+0000 7fad0a800640 1 --2- 172.21.15.52:0/2428328864 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facf0004c30 0x7fad0c0c7380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:07.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad0b200640 1 -- 172.21.15.52:0/2428328864 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facf0004c30 msgr2=0x7fad0c0c7380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:07.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad0b200640 1 --2- 172.21.15.52:0/2428328864 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facf0004c30 0x7fad0c0c7380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad0b200640 1 -- 172.21.15.52:0/2428328864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c167560 msgr2=0x7fad0c0cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad0b200640 1 --2- 172.21.15.52:0/2428328864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c167560 0x7fad0c0cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad0b200640 1 -- 172.21.15.52:0/2428328864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c0c7cd0 con 0x7fad0c14ac00 2024-09-10T21:10:07.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad03e00640 1 --2- 172.21.15.52:0/2428328864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c167560 0x7fad0c0cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:07.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad0b200640 1 --2- 172.21.15.52:0/2428328864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad0c14ac00 0x7fad0c14b080 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad0c069ea0 tx=0x7fad04006420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:07.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad02a00640 1 -- 172.21.15.52:0/2428328864 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad040105d0 con 0x7fad0c14ac00 2024-09-10T21:10:07.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad02a00640 1 -- 172.21.15.52:0/2428328864 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad040108c0 con 0x7fad0c14ac00 2024-09-10T21:10:07.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad11400640 1 -- 172.21.15.52:0/2428328864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0c0c7fb0 con 0x7fad0c14ac00 2024-09-10T21:10:07.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad02a00640 1 -- 172.21.15.52:0/2428328864 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04010d10 con 0x7fad0c14ac00 2024-09-10T21:10:07.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad11400640 1 -- 172.21.15.52:0/2428328864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0c0c8570 con 0x7fad0c14ac00 2024-09-10T21:10:07.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad02a00640 1 -- 172.21.15.52:0/2428328864 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fad040117f0 con 0x7fad0c14ac00 2024-09-10T21:10:07.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad02a00640 1 -- 172.21.15.52:0/2428328864 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fad0401a410 con 0x7fad0c14ac00 2024-09-10T21:10:07.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad11400640 1 -- 172.21.15.52:0/2428328864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0c0b4b10 con 0x7fad0c14ac00 2024-09-10T21:10:07.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.808+0000 7fad02a00640 1 -- 172.21.15.52:0/2428328864 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fad04011b10 con 0x7fad0c14ac00 2024-09-10T21:10:07.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.884+0000 7fad11400640 1 -- 172.21.15.52:0/2428328864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd new", "uuid": "069eefff-abdc-4750-86b5-f836d3d6c23b", "id": 6} v 0) -- 0x7fad0c0704a0 con 0x7fad0c14ac00 2024-09-10T21:10:07.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.888+0000 7fad02a00640 1 -- 172.21.15.52:0/2428328864 <== mon.1 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "069eefff-abdc-4750-86b5-f836d3d6c23b", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fad04011cf0 con 0x7fad0c14ac00 2024-09-10T21:10:07.895 INFO:teuthology.orchestra.run.smithi052.stdout:6 2024-09-10T21:10:07.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.888+0000 7fad11400640 1 -- 172.21.15.52:0/2428328864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad0c14ac00 msgr2=0x7fad0c14b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.888+0000 7fad11400640 1 --2- 172.21.15.52:0/2428328864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad0c14ac00 0x7fad0c14b080 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad0c069ea0 tx=0x7fad04006420 comp rx=0 tx=0).stop 2024-09-10T21:10:07.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.888+0000 7fad11400640 1 -- 172.21.15.52:0/2428328864 shutdown_connections 2024-09-10T21:10:07.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.888+0000 7fad11400640 1 --2- 172.21.15.52:0/2428328864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad0c14ac00 0x7fad0c14b080 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.888+0000 7fad11400640 1 --2- 172.21.15.52:0/2428328864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad0c167560 0x7fad0c0cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.888+0000 7fad11400640 1 --2- 172.21.15.52:0/2428328864 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facf0004c30 0x7fad0c0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.888+0000 7fad11400640 1 -- 172.21.15.52:0/2428328864 >> 172.21.15.52:0/2428328864 conn(0x7fad0c0d1ce0 msgr2=0x7fad0c06e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:07.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.888+0000 7fad11400640 1 -- 172.21.15.52:0/2428328864 shutdown_connections 2024-09-10T21:10:07.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:07.888+0000 7fad11400640 1 -- 172.21.15.52:0/2428328864 wait complete. 2024-09-10T21:10:07.913 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --cluster ceph osd new e08188d3-3e07-400b-b9ee-21369f465792 7 2024-09-10T21:10:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.052+0000 7fa735200640 1 -- 172.21.15.52:0/2770442572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73015f940 msgr2=0x7fa73015fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.052+0000 7fa735200640 1 --2- 172.21.15.52:0/2770442572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73015f940 0x7fa73015fd90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa728009c80 tx=0x7fa72801cc80 comp rx=0 tx=0).stop 2024-09-10T21:10:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.052+0000 7fa735200640 1 -- 172.21.15.52:0/2770442572 shutdown_connections 2024-09-10T21:10:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.052+0000 7fa735200640 1 --2- 172.21.15.52:0/2770442572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73015f940 0x7fa73015fd90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.052+0000 7fa735200640 1 --2- 172.21.15.52:0/2770442572 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa73015bdc0 0x7fa7300d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.052+0000 7fa735200640 1 -- 172.21.15.52:0/2770442572 >> 172.21.15.52:0/2770442572 conn(0x7fa7300d1ce0 msgr2=0x7fa7300d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.052+0000 7fa735200640 1 -- 172.21.15.52:0/2770442572 shutdown_connections 2024-09-10T21:10:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.052+0000 7fa735200640 1 -- 172.21.15.52:0/2770442572 wait complete. 2024-09-10T21:10:08.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.052+0000 7fa735200640 1 Processor -- start 2024-09-10T21:10:08.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.052+0000 7fa735200640 1 -- start start 2024-09-10T21:10:08.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa735200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7300cd300 0x7fa73006f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:08.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa735200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73015bdc0 0x7fa7301f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:08.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa735200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73015f940 0x7fa7300719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:08.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa735200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa730169120 con 0x7fa73015f940 2024-09-10T21:10:08.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa735200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa730168fa0 con 0x7fa73015bdc0 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa735200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa7301692a0 con 0x7fa7300cd300 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa72de00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7300cd300 0x7fa73006f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa727e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73015bdc0 0x7fa7301f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa72de00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7300cd300 0x7fa73006f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.52:58040/0 (socket says 172.21.15.52:58040) 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa72de00640 1 -- 172.21.15.52:0/3613878512 learned_addr learned my addr 172.21.15.52:0/3613878512 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa72e800640 1 --2- 172.21.15.52:0/3613878512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73015f940 0x7fa7300719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa72de00640 1 -- 172.21.15.52:0/3613878512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7300cd300 msgr2=0x7fa73006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa72de00640 1 -- 172.21.15.52:0/3613878512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7300cd300 msgr2=0x7fa73006f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa72de00640 1 --2- 172.21.15.52:0/3613878512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7300cd300 0x7fa73006f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa72de00640 1 --2- 172.21.15.52:0/3613878512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7300cd300 0x7fa73006f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa727e00640 1 -- 172.21.15.52:0/3613878512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7300cd300 msgr2=0x7fa73006f8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa727e00640 1 --2- 172.21.15.52:0/3613878512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7300cd300 0x7fa73006f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa727e00640 1 -- 172.21.15.52:0/3613878512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73015f940 msgr2=0x7fa7300719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa727e00640 1 --2- 172.21.15.52:0/3613878512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73015f940 0x7fa7300719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa727e00640 1 -- 172.21.15.52:0/3613878512 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7300b33f0 con 0x7fa73015bdc0 2024-09-10T21:10:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa727e00640 1 --2- 172.21.15.52:0/3613878512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73015bdc0 0x7fa7301f7ea0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa7300d1600 tx=0x7fa71800c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa726000640 1 -- 172.21.15.52:0/3613878512 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718017070 con 0x7fa73015bdc0 2024-09-10T21:10:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa726000640 1 -- 172.21.15.52:0/3613878512 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7180053c0 con 0x7fa73015bdc0 2024-09-10T21:10:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa735200640 1 -- 172.21.15.52:0/3613878512 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7301568f0 con 0x7fa73015bdc0 2024-09-10T21:10:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa726000640 1 -- 172.21.15.52:0/3613878512 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718003840 con 0x7fa73015bdc0 2024-09-10T21:10:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa735200640 1 -- 172.21.15.52:0/3613878512 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa730158550 con 0x7fa73015bdc0 2024-09-10T21:10:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa726000640 1 -- 172.21.15.52:0/3613878512 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fa7180039e0 con 0x7fa73015bdc0 2024-09-10T21:10:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa726000640 1 -- 172.21.15.52:0/3613878512 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fa718003d00 con 0x7fa73015bdc0 2024-09-10T21:10:08.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.056+0000 7fa735200640 1 -- 172.21.15.52:0/3613878512 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7300d2800 con 0x7fa73015bdc0 2024-09-10T21:10:08.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.060+0000 7fa726000640 1 -- 172.21.15.52:0/3613878512 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fa718021020 con 0x7fa73015bdc0 2024-09-10T21:10:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.132+0000 7fa735200640 1 -- 172.21.15.52:0/3613878512 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e08188d3-3e07-400b-b9ee-21369f465792", "id": 7} v 0) -- 0x7fa7300704a0 con 0x7fa73015bdc0 2024-09-10T21:10:08.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.136+0000 7fa726000640 1 -- 172.21.15.52:0/3613878512 <== mon.1 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e08188d3-3e07-400b-b9ee-21369f465792", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fa718011590 con 0x7fa73015bdc0 2024-09-10T21:10:08.145 INFO:teuthology.orchestra.run.smithi052.stdout:7 2024-09-10T21:10:08.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.136+0000 7fa735200640 1 -- 172.21.15.52:0/3613878512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73015bdc0 msgr2=0x7fa7301f7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:08.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.136+0000 7fa735200640 1 --2- 172.21.15.52:0/3613878512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73015bdc0 0x7fa7301f7ea0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa7300d1600 tx=0x7fa71800c820 comp rx=0 tx=0).stop 2024-09-10T21:10:08.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.136+0000 7fa735200640 1 -- 172.21.15.52:0/3613878512 shutdown_connections 2024-09-10T21:10:08.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.136+0000 7fa735200640 1 --2- 172.21.15.52:0/3613878512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73015f940 0x7fa7300719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.136+0000 7fa735200640 1 --2- 172.21.15.52:0/3613878512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73015bdc0 0x7fa7301f7ea0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.136+0000 7fa735200640 1 --2- 172.21.15.52:0/3613878512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7300cd300 0x7fa73006f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.136+0000 7fa735200640 1 -- 172.21.15.52:0/3613878512 >> 172.21.15.52:0/3613878512 conn(0x7fa7300d1ce0 msgr2=0x7fa7300b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:08.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.136+0000 7fa735200640 1 -- 172.21.15.52:0/3613878512 shutdown_connections 2024-09-10T21:10:08.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:08.140+0000 7fa735200640 1 -- 172.21.15.52:0/3613878512 wait complete. 2024-09-10T21:10:08.167 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T21:10:08.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T21:10:08.171 INFO:tasks.ceph.osd.0:Started 2024-09-10T21:10:08.171 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T21:10:08.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T21:10:08.174 INFO:tasks.ceph.osd.1:Started 2024-09-10T21:10:08.174 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T21:10:08.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T21:10:08.177 INFO:tasks.ceph.osd.2:Started 2024-09-10T21:10:08.177 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-10T21:10:08.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-10T21:10:08.181 INFO:tasks.ceph.osd.3:Started 2024-09-10T21:10:08.181 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-10T21:10:08.182 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-10T21:10:08.185 INFO:tasks.ceph.osd.4:Started 2024-09-10T21:10:08.185 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-10T21:10:08.185 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-10T21:10:08.188 INFO:tasks.ceph.osd.5:Started 2024-09-10T21:10:08.188 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-10T21:10:08.188 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-10T21:10:08.191 INFO:tasks.ceph.osd.6:Started 2024-09-10T21:10:08.191 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-10T21:10:08.191 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-10T21:10:08.196 INFO:tasks.ceph.osd.7:Started 2024-09-10T21:10:08.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:10:08.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.363+0000 7f7120000640 1 -- 172.21.15.16:0/2273070683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71180726b0 msgr2=0x7f7118072b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.363+0000 7f7120000640 1 --2- 172.21.15.16:0/2273070683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71180726b0 0x7f7118072b60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7110009bc0 tx=0x7f711001cb10 comp rx=0 tx=0).stop 2024-09-10T21:10:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.363+0000 7f7120000640 1 -- 172.21.15.16:0/2273070683 shutdown_connections 2024-09-10T21:10:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.363+0000 7f7120000640 1 --2- 172.21.15.16:0/2273070683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71180726b0 0x7f7118072b60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.363+0000 7f7120000640 1 --2- 172.21.15.16:0/2273070683 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f71181599d0 0x7f711815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.363+0000 7f7120000640 1 -- 172.21.15.16:0/2273070683 >> 172.21.15.16:0/2273070683 conn(0x7f71180cfb20 msgr2=0x7f71180cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.363+0000 7f7120000640 1 -- 172.21.15.16:0/2273070683 shutdown_connections 2024-09-10T21:10:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.363+0000 7f7120000640 1 -- 172.21.15.16:0/2273070683 wait complete. 2024-09-10T21:10:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7120000640 1 Processor -- start 2024-09-10T21:10:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7120000640 1 -- start start 2024-09-10T21:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7120000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f711806e640 0x7f71180c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7120000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71180726b0 0x7f71180cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7120000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71181599d0 0x7f71180c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7120000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7118168f60 con 0x7f71180726b0 2024-09-10T21:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7120000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7118168de0 con 0x7f71181599d0 2024-09-10T21:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7120000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f71181690e0 con 0x7f711806e640 2024-09-10T21:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71180726b0 0x7f71180cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711d600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f711806e640 0x7f71180c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71180726b0 0x7f71180cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53596/0 (socket says 172.21.15.16:53596) 2024-09-10T21:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711cc00640 1 -- 172.21.15.16:0/2623848626 learned_addr learned my addr 172.21.15.16:0/2623848626 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711e000640 1 --2- 172.21.15.16:0/2623848626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71181599d0 0x7f71180c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711d600640 1 -- 172.21.15.16:0/2623848626 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f711806e640 msgr2=0x7f71180c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:10:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711d600640 1 -- 172.21.15.16:0/2623848626 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f711806e640 msgr2=0x7f71180c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711d600640 1 --2- 172.21.15.16:0/2623848626 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f711806e640 0x7f71180c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711d600640 1 --2- 172.21.15.16:0/2623848626 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f711806e640 0x7f71180c6bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711cc00640 1 -- 172.21.15.16:0/2623848626 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f711806e640 msgr2=0x7f71180c6bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711cc00640 1 --2- 172.21.15.16:0/2623848626 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f711806e640 0x7f71180c6bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711cc00640 1 -- 172.21.15.16:0/2623848626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71181599d0 msgr2=0x7f71180c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711cc00640 1 --2- 172.21.15.16:0/2623848626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71181599d0 0x7f71180c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711cc00640 1 -- 172.21.15.16:0/2623848626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71181587d0 con 0x7f71180726b0 2024-09-10T21:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711e000640 1 --2- 172.21.15.16:0/2623848626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71181599d0 0x7f71180c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f711cc00640 1 --2- 172.21.15.16:0/2623848626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71180726b0 0x7f71180cad40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f710400c450 tx=0x7f710400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7115600640 1 -- 172.21.15.16:0/2623848626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104017070 con 0x7f71180726b0 2024-09-10T21:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7115600640 1 -- 172.21.15.16:0/2623848626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7104003600 con 0x7f71180726b0 2024-09-10T21:10:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7120000640 1 -- 172.21.15.16:0/2623848626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71180c7280 con 0x7f71180726b0 2024-09-10T21:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7115600640 1 -- 172.21.15.16:0/2623848626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104003a50 con 0x7f71180726b0 2024-09-10T21:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7120000640 1 -- 172.21.15.16:0/2623848626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71180c7840 con 0x7f71180726b0 2024-09-10T21:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7115600640 1 -- 172.21.15.16:0/2623848626 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f7104003bf0 con 0x7f71180726b0 2024-09-10T21:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.367+0000 7f7115600640 1 -- 172.21.15.16:0/2623848626 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f710401cb10 con 0x7f71180726b0 2024-09-10T21:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.371+0000 7f7120000640 1 -- 172.21.15.16:0/2623848626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7118072b60 con 0x7f71180726b0 2024-09-10T21:10:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.371+0000 7f7115600640 1 -- 172.21.15.16:0/2623848626 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f7104021070 con 0x7f71180726b0 2024-09-10T21:10:08.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.435+0000 7f7120000640 1 -- 172.21.15.16:0/2623848626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71180cb9f0 con 0x7f71180726b0 2024-09-10T21:10:08.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.435+0000 7f7115600640 1 -- 172.21.15.16:0/2623848626 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f710401c440 con 0x7f71180726b0 2024-09-10T21:10:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:08.441 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:10:08.139455+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:10:08.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.435+0000 7f7120000640 1 -- 172.21.15.16:0/2623848626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71180726b0 msgr2=0x7f71180cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:08.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.435+0000 7f7120000640 1 --2- 172.21.15.16:0/2623848626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71180726b0 0x7f71180cad40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f710400c450 tx=0x7f710400c910 comp rx=0 tx=0).stop 2024-09-10T21:10:08.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.435+0000 7f7120000640 1 -- 172.21.15.16:0/2623848626 shutdown_connections 2024-09-10T21:10:08.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.435+0000 7f7120000640 1 --2- 172.21.15.16:0/2623848626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71181599d0 0x7f71180c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.435+0000 7f7120000640 1 --2- 172.21.15.16:0/2623848626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71180726b0 0x7f71180cad40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.435+0000 7f7120000640 1 --2- 172.21.15.16:0/2623848626 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f711806e640 0x7f71180c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.435+0000 7f7120000640 1 -- 172.21.15.16:0/2623848626 >> 172.21.15.16:0/2623848626 conn(0x7f71180cfb20 msgr2=0x7f7118166070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:08.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.435+0000 7f7120000640 1 -- 172.21.15.16:0/2623848626 shutdown_connections 2024-09-10T21:10:08.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:08.435+0000 7f7120000640 1 -- 172.21.15.16:0/2623848626 wait complete. 2024-09-10T21:10:08.461 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T21:10:08.461 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T21:10:09.267 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T21:10:09.404 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.395+0000 7f7163200640 1 -- 172.21.15.16:0/3205958416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c074220 msgr2=0x7f715c0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:09.404 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.395+0000 7f7163200640 1 --2- 172.21.15.16:0/3205958416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c074220 0x7f715c0746d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7158009bc0 tx=0x7f715801ca80 comp rx=0 tx=0).stop 2024-09-10T21:10:09.404 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.395+0000 7f7163200640 1 -- 172.21.15.16:0/3205958416 shutdown_connections 2024-09-10T21:10:09.404 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.395+0000 7f7163200640 1 --2- 172.21.15.16:0/3205958416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c074220 0x7f715c0746d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.404 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.395+0000 7f7163200640 1 --2- 172.21.15.16:0/3205958416 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f715c15be20 0x7f715c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.405 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.395+0000 7f7163200640 1 -- 172.21.15.16:0/3205958416 >> 172.21.15.16:0/3205958416 conn(0x7f715c0d02a0 msgr2=0x7f715c0d06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:09.405 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.395+0000 7f7163200640 1 -- 172.21.15.16:0/3205958416 shutdown_connections 2024-09-10T21:10:09.405 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.395+0000 7f7163200640 1 -- 172.21.15.16:0/3205958416 wait complete. 2024-09-10T21:10:09.405 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7163200640 1 Processor -- start 2024-09-10T21:10:09.405 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7163200640 1 -- start start 2024-09-10T21:10:09.408 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7163200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c074220 0x7f715c1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.408 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7163200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f715c0b0e70 0x7f715c099310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.408 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7163200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f715c15be20 0x7f715c152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.408 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7163200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f715c168e70 con 0x7f715c074220 2024-09-10T21:10:09.408 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7163200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f715c168cf0 con 0x7f715c0b0e70 2024-09-10T21:10:09.408 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7163200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f715c168ff0 con 0x7f715c15be20 2024-09-10T21:10:09.408 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7161e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c074220 0x7f715c1515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.408 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7161e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c074220 0x7f715c1515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53614/0 (socket says 172.21.15.16:53614) 2024-09-10T21:10:09.408 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7161e00640 1 -- 172.21.15.16:0/3958106745 learned_addr learned my addr 172.21.15.16:0/3958106745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:09.408 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7161400640 1 --2- 172.21.15.16:0/3958106745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f715c0b0e70 0x7f715c099310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.409 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7162800640 1 --2- 172.21.15.16:0/3958106745 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f715c15be20 0x7f715c152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.409 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7161e00640 1 -- 172.21.15.16:0/3958106745 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f715c15be20 msgr2=0x7f715c152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:09.409 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7161e00640 1 --2- 172.21.15.16:0/3958106745 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f715c15be20 0x7f715c152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.409 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7161e00640 1 -- 172.21.15.16:0/3958106745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f715c0b0e70 msgr2=0x7f715c099310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:09.409 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7161e00640 1 --2- 172.21.15.16:0/3958106745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f715c0b0e70 0x7f715c099310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.409 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7161e00640 1 -- 172.21.15.16:0/3958106745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c151ca0 con 0x7f715c074220 2024-09-10T21:10:09.409 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.399+0000 7f7161e00640 1 --2- 172.21.15.16:0/3958106745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c074220 0x7f715c1515a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f715000e7a0 tx=0x7f715000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:09.410 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.403+0000 7f7156000640 1 -- 172.21.15.16:0/3958106745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715000ccc0 con 0x7f715c074220 2024-09-10T21:10:09.410 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.403+0000 7f7156000640 1 -- 172.21.15.16:0/3958106745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7150010430 con 0x7f715c074220 2024-09-10T21:10:09.410 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.403+0000 7f7163200640 1 -- 172.21.15.16:0/3958106745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715c151f80 con 0x7f715c074220 2024-09-10T21:10:09.411 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.403+0000 7f7156000640 1 -- 172.21.15.16:0/3958106745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150010860 con 0x7f715c074220 2024-09-10T21:10:09.411 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.403+0000 7f7163200640 1 -- 172.21.15.16:0/3958106745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715c152540 con 0x7f715c074220 2024-09-10T21:10:09.411 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.403+0000 7f7156000640 1 -- 172.21.15.16:0/3958106745 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f7150010a00 con 0x7f715c074220 2024-09-10T21:10:09.411 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.403+0000 7f7156000640 1 -- 172.21.15.16:0/3958106745 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7150010d20 con 0x7f715c074220 2024-09-10T21:10:09.411 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.403+0000 7f7163200640 1 -- 172.21.15.16:0/3958106745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f715c0746d0 con 0x7f715c074220 2024-09-10T21:10:09.411 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.403+0000 7f7156000640 1 -- 172.21.15.16:0/3958106745 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f7150018c40 con 0x7f715c074220 2024-09-10T21:10:09.453 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.447+0000 7f7156000640 1 -- 172.21.15.16:0/3958106745 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 2) ==== 48237+0+0 (secure 0 0 0) 0x7f7150018420 con 0x7f715c074220 2024-09-10T21:10:09.481 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.475+0000 7f7163200640 1 -- 172.21.15.16:0/3958106745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f715c0b6220 con 0x7f715c074220 2024-09-10T21:10:09.482 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.475+0000 7f7156000640 1 -- 172.21.15.16:0/3958106745 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7150018e20 con 0x7f715c074220 2024-09-10T21:10:09.482 INFO:teuthology.misc.health.smithi016.stdout: 2024-09-10T21:10:09.483 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":10,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:10:08.139455+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:10:09.483 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.475+0000 7f7163200640 1 -- 172.21.15.16:0/3958106745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c074220 msgr2=0x7f715c1515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:09.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.475+0000 7f7163200640 1 --2- 172.21.15.16:0/3958106745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c074220 0x7f715c1515a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f715000e7a0 tx=0x7f715000ec60 comp rx=0 tx=0).stop 2024-09-10T21:10:09.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.475+0000 7f7163200640 1 -- 172.21.15.16:0/3958106745 shutdown_connections 2024-09-10T21:10:09.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.475+0000 7f7163200640 1 --2- 172.21.15.16:0/3958106745 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f715c15be20 0x7f715c152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.475+0000 7f7163200640 1 --2- 172.21.15.16:0/3958106745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f715c0b0e70 0x7f715c099310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.475+0000 7f7163200640 1 --2- 172.21.15.16:0/3958106745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c074220 0x7f715c1515a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.475+0000 7f7163200640 1 -- 172.21.15.16:0/3958106745 >> 172.21.15.16:0/3958106745 conn(0x7f715c0d02a0 msgr2=0x7f715c0702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:09.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.475+0000 7f7163200640 1 -- 172.21.15.16:0/3958106745 shutdown_connections 2024-09-10T21:10:09.484 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:09.475+0000 7f7163200640 1 -- 172.21.15.16:0/3958106745 wait complete. 2024-09-10T21:10:09.504 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T21:10:09.675 INFO:tasks.ceph.osd.6.smithi052.stderr:2024-09-10T21:10:09.668+0000 7fbb1c56e7c0 -1 Falling back to public interface 2024-09-10T21:10:09.796 INFO:tasks.ceph.osd.7.smithi052.stderr:2024-09-10T21:10:09.788+0000 7ff04433c7c0 -1 Falling back to public interface 2024-09-10T21:10:09.845 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-10T21:10:09.839+0000 7f32185ab7c0 -1 Falling back to public interface 2024-09-10T21:10:09.857 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-10T21:10:09.851+0000 7efeeb4a77c0 -1 Falling back to public interface 2024-09-10T21:10:09.879 INFO:tasks.ceph.osd.5.smithi052.stderr:2024-09-10T21:10:09.872+0000 7f66f798f7c0 -1 Falling back to public interface 2024-09-10T21:10:09.921 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-10T21:10:09.915+0000 7f8f451617c0 -1 Falling back to public interface 2024-09-10T21:10:09.945 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-10T21:10:09.939+0000 7feece2f57c0 -1 Falling back to public interface 2024-09-10T21:10:10.187 INFO:tasks.ceph.osd.4.smithi052.stderr:2024-09-10T21:10:10.180+0000 7f8fe202b7c0 -1 Falling back to public interface 2024-09-10T21:10:10.467 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-10T21:10:10.459+0000 7f193c000640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T21:10:10.467 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-10T21:10:10.459+0000 7f193c000640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T21:10:13.943 INFO:tasks.ceph.osd.7.smithi052.stderr:2024-09-10T21:10:13.936+0000 7ff04433c7c0 -1 osd.7 0 log_to_monitors true 2024-09-10T21:10:13.962 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-10T21:10:13.955+0000 7f32185ab7c0 -1 osd.1 0 log_to_monitors true 2024-09-10T21:10:14.127 INFO:tasks.ceph.osd.5.smithi052.stderr:2024-09-10T21:10:14.120+0000 7f66f798f7c0 -1 osd.5 0 log_to_monitors true 2024-09-10T21:10:14.134 INFO:tasks.ceph.osd.6.smithi052.stderr:2024-09-10T21:10:14.128+0000 7fbb1c56e7c0 -1 osd.6 0 log_to_monitors true 2024-09-10T21:10:14.225 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-10T21:10:14.219+0000 7f8f451617c0 -1 osd.3 0 log_to_monitors true 2024-09-10T21:10:14.302 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-10T21:10:14.295+0000 7feece2f57c0 -1 osd.2 0 log_to_monitors true 2024-09-10T21:10:14.322 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-10T21:10:14.315+0000 7efeeb4a77c0 -1 osd.0 0 log_to_monitors true 2024-09-10T21:10:14.553 INFO:tasks.ceph.osd.4.smithi052.stderr:2024-09-10T21:10:14.544+0000 7f8fe202b7c0 -1 osd.4 0 log_to_monitors true 2024-09-10T21:10:16.053 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-10T21:10:16.047+0000 7f3213e00640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T21:10:16.088 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-10T21:10:16.079+0000 7f3209400640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:10:16.106 INFO:tasks.ceph.osd.7.smithi052.stderr:2024-09-10T21:10:16.100+0000 7ff03f200640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T21:10:16.142 INFO:tasks.ceph.osd.7.smithi052.stderr:2024-09-10T21:10:16.136+0000 7ff035200640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:10:16.311 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T21:10:16.503 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.495+0000 7f4b00c00640 1 -- 172.21.15.16:0/2045804366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc0b1800 msgr2=0x7f4afc0b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:16.503 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.495+0000 7f4b00c00640 1 --2- 172.21.15.16:0/2045804366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc0b1800 0x7f4afc0b1c90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009bc0 tx=0x7f4af401cb10 comp rx=0 tx=0).stop 2024-09-10T21:10:16.503 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.495+0000 7f4b00c00640 1 -- 172.21.15.16:0/2045804366 shutdown_connections 2024-09-10T21:10:16.504 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.495+0000 7f4b00c00640 1 --2- 172.21.15.16:0/2045804366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc0b1800 0x7f4afc0b1c90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:16.504 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.495+0000 7f4b00c00640 1 --2- 172.21.15.16:0/2045804366 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4afc0b0ec0 0x7f4afc0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:16.504 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.495+0000 7f4b00c00640 1 -- 172.21.15.16:0/2045804366 >> 172.21.15.16:0/2045804366 conn(0x7f4afc0d1d90 msgr2=0x7f4afc0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:16.504 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.495+0000 7f4b00c00640 1 -- 172.21.15.16:0/2045804366 shutdown_connections 2024-09-10T21:10:16.504 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.495+0000 7f4b00c00640 1 -- 172.21.15.16:0/2045804366 wait complete. 2024-09-10T21:10:16.504 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4b00c00640 1 Processor -- start 2024-09-10T21:10:16.504 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4b00c00640 1 -- start start 2024-09-10T21:10:16.506 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4b00c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4afc0b0ec0 0x7f4afc151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:16.506 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4b00c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc0b1800 0x7f4afc0cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:16.506 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4b00c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc0ce7a0 0x7f4afc0d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:16.507 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4b00c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc1690b0 con 0x7f4afc0b1800 2024-09-10T21:10:16.507 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4b00c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc168f30 con 0x7f4afc0ce7a0 2024-09-10T21:10:16.507 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4b00c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4afc169230 con 0x7f4afc0b0ec0 2024-09-10T21:10:16.507 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4af9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc0b1800 0x7f4afc0cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:16.507 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4af9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc0b1800 0x7f4afc0cfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51378/0 (socket says 172.21.15.16:51378) 2024-09-10T21:10:16.507 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4af9400640 1 -- 172.21.15.16:0/3357338274 learned_addr learned my addr 172.21.15.16:0/3357338274 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:16.507 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4af9e00640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4afc0b0ec0 0x7f4afc151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:16.507 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4afa800640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc0ce7a0 0x7f4afc0d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:16.508 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4af9400640 1 -- 172.21.15.16:0/3357338274 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4afc0b0ec0 msgr2=0x7f4afc151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:16.508 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4af9400640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4afc0b0ec0 0x7f4afc151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:16.509 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4af9400640 1 -- 172.21.15.16:0/3357338274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc0ce7a0 msgr2=0x7f4afc0d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:16.509 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4af9400640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc0ce7a0 0x7f4afc0d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:16.509 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4af9400640 1 -- 172.21.15.16:0/3357338274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4afc154000 con 0x7f4afc0b1800 2024-09-10T21:10:16.509 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4af9400640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc0b1800 0x7f4afc0cfc90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800e9f0 tx=0x7f4ae800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:16.509 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4af2000640 1 -- 172.21.15.16:0/3357338274 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae800cd80 con 0x7f4afc0b1800 2024-09-10T21:10:16.509 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4af2000640 1 -- 172.21.15.16:0/3357338274 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ae8010430 con 0x7f4afc0b1800 2024-09-10T21:10:16.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4b00c00640 1 -- 172.21.15.16:0/3357338274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4afc1542e0 con 0x7f4afc0b1800 2024-09-10T21:10:16.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4b00c00640 1 -- 172.21.15.16:0/3357338274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4afc1548a0 con 0x7f4afc0b1800 2024-09-10T21:10:16.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.499+0000 7f4af2000640 1 -- 172.21.15.16:0/3357338274 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8010860 con 0x7f4afc0b1800 2024-09-10T21:10:16.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.503+0000 7f4b00c00640 1 -- 172.21.15.16:0/3357338274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad40052a0 con 0x7f4afc0b1800 2024-09-10T21:10:16.511 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.503+0000 7f4af2000640 1 -- 172.21.15.16:0/3357338274 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 96338+0+0 (secure 0 0 0) 0x7f4ae8010a00 con 0x7f4afc0b1800 2024-09-10T21:10:16.512 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.503+0000 7f4af2000640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4adc072580 0x7f4adc074a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:16.512 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.503+0000 7f4af2000640 1 -- 172.21.15.16:0/3357338274 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f4ae8092ec0 con 0x7f4afc0b1800 2024-09-10T21:10:16.512 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.503+0000 7f4af9e00640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4adc072580 0x7f4adc074a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:16.513 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.507+0000 7f4af9e00640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4adc072580 0x7f4adc074a40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4aec001870 tx=0x7f4aec001380 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:16.518 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.511+0000 7f4af2000640 1 -- 172.21.15.16:0/3357338274 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4ae805ede0 con 0x7f4afc0b1800 2024-09-10T21:10:16.812 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.803+0000 7f4b00c00640 1 -- 172.21.15.16:0/3357338274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ad4005590 con 0x7f4afc0b1800 2024-09-10T21:10:16.813 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.807+0000 7f4af2000640 1 -- 172.21.15.16:0/3357338274 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7f4ae8063810 con 0x7f4afc0b1800 2024-09-10T21:10:16.814 INFO:teuthology.misc.health.smithi016.stdout: 2024-09-10T21:10:16.814 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":13,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:10:16.000693+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:10:16.817 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.811+0000 7f4b00c00640 1 -- 172.21.15.16:0/3357338274 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4adc072580 msgr2=0x7f4adc074a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:16.817 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.811+0000 7f4b00c00640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4adc072580 0x7f4adc074a40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4aec001870 tx=0x7f4aec001380 comp rx=0 tx=0).stop 2024-09-10T21:10:16.817 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.811+0000 7f4b00c00640 1 -- 172.21.15.16:0/3357338274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc0b1800 msgr2=0x7f4afc0cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:16.817 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.811+0000 7f4b00c00640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc0b1800 0x7f4afc0cfc90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800e9f0 tx=0x7f4ae800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:10:16.818 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.811+0000 7f4b00c00640 1 -- 172.21.15.16:0/3357338274 shutdown_connections 2024-09-10T21:10:16.818 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.811+0000 7f4b00c00640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4adc072580 0x7f4adc074a40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:16.818 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.811+0000 7f4b00c00640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc0ce7a0 0x7f4afc0d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:16.818 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.811+0000 7f4b00c00640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc0b1800 0x7f4afc0cfc90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:16.819 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.811+0000 7f4b00c00640 1 --2- 172.21.15.16:0/3357338274 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4afc0b0ec0 0x7f4afc151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:16.819 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.811+0000 7f4b00c00640 1 -- 172.21.15.16:0/3357338274 >> 172.21.15.16:0/3357338274 conn(0x7f4afc0d1d90 msgr2=0x7f4afc070a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:16.820 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.815+0000 7f4b00c00640 1 -- 172.21.15.16:0/3357338274 shutdown_connections 2024-09-10T21:10:16.820 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:16.815+0000 7f4b00c00640 1 -- 172.21.15.16:0/3357338274 wait complete. 2024-09-10T21:10:16.848 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T21:10:17.479 INFO:tasks.ceph.osd.5.smithi052.stderr:2024-09-10T21:10:17.472+0000 7f66f2800640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T21:10:17.482 INFO:tasks.ceph.osd.6.smithi052.stderr:2024-09-10T21:10:17.476+0000 7fbb17e00640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T21:10:17.487 INFO:tasks.ceph.osd.4.smithi052.stderr:2024-09-10T21:10:17.480+0000 7f8fdc400640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T21:10:17.503 INFO:tasks.ceph.osd.5.smithi052.stderr:2024-09-10T21:10:17.496+0000 7f66e8800640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:10:17.523 INFO:tasks.ceph.osd.4.smithi052.stderr:2024-09-10T21:10:17.516+0000 7f8fd2400640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:10:17.527 INFO:tasks.ceph.osd.6.smithi052.stderr:2024-09-10T21:10:17.520+0000 7fbb0d400640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:10:17.527 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-10T21:10:17.519+0000 7f8f40000640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T21:10:17.550 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-10T21:10:17.543+0000 7feec9c00640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T21:10:17.551 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-10T21:10:17.543+0000 7efee6400640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T21:10:17.557 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-10T21:10:17.551+0000 7f8f36000640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:10:17.573 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-10T21:10:17.567+0000 7feebf200640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:10:17.577 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-10T21:10:17.571+0000 7efedc400640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:10:23.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:10:23.092 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:10:23.655 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T21:10:23.805 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 -- 172.21.15.16:0/22435586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe00005220 msgr2=0x7efe0000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:23.806 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 --2- 172.21.15.16:0/22435586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe00005220 0x7efe0000b9d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efe14006fd0 tx=0x7efe1401caf0 comp rx=0 tx=0).stop 2024-09-10T21:10:23.806 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 -- 172.21.15.16:0/22435586 shutdown_connections 2024-09-10T21:10:23.806 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 --2- 172.21.15.16:0/22435586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe00005220 0x7efe0000b9d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:23.806 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 -- 172.21.15.16:0/22435586 >> 172.21.15.16:0/22435586 conn(0x7efe1c06dcf0 msgr2=0x7efe1c06e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:23.806 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 -- 172.21.15.16:0/22435586 shutdown_connections 2024-09-10T21:10:23.806 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 -- 172.21.15.16:0/22435586 wait complete. 2024-09-10T21:10:23.807 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 Processor -- start 2024-09-10T21:10:23.807 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 -- start start 2024-09-10T21:10:23.808 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe00005220 0x7efe1c143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:23.809 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe1c1f8240 0x7efe1c070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:23.809 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c150430 0x7efe1c1508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:23.809 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c15bfe0 con 0x7efe00005220 2024-09-10T21:10:23.809 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c0d1830 con 0x7efe1c150430 2024-09-10T21:10:23.809 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe21000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7efe1c15c160 con 0x7efe1c1f8240 2024-09-10T21:10:23.809 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe1b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe00005220 0x7efe1c143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:23.809 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe1b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe00005220 0x7efe1c143a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43734/0 (socket says 172.21.15.16:43734) 2024-09-10T21:10:23.809 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe1b400640 1 -- 172.21.15.16:0/928913505 learned_addr learned my addr 172.21.15.16:0/928913505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:23.809 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe13e00640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe1c1f8240 0x7efe1c070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:23.809 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe1be00640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c150430 0x7efe1c1508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:23.809 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe1b400640 1 -- 172.21.15.16:0/928913505 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe1c1f8240 msgr2=0x7efe1c070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:23.810 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe1b400640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe1c1f8240 0x7efe1c070ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:23.810 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe1b400640 1 -- 172.21.15.16:0/928913505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c150430 msgr2=0x7efe1c1508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:23.810 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe1b400640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c150430 0x7efe1c1508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:23.810 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe1b400640 1 -- 172.21.15.16:0/928913505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c1f51b0 con 0x7efe00005220 2024-09-10T21:10:23.810 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.802+0000 7efe1be00640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c150430 0x7efe1c1508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:23.810 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.806+0000 7efe1b400640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe00005220 0x7efe1c143a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efe0c011860 tx=0x7efe0c011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:23.811 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.806+0000 7efe13400640 1 -- 172.21.15.16:0/928913505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c00f9a0 con 0x7efe00005220 2024-09-10T21:10:23.811 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.806+0000 7efe13400640 1 -- 172.21.15.16:0/928913505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe0c00fb40 con 0x7efe00005220 2024-09-10T21:10:23.811 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.806+0000 7efe21000640 1 -- 172.21.15.16:0/928913505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1c1f4630 con 0x7efe00005220 2024-09-10T21:10:23.811 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.806+0000 7efe13400640 1 -- 172.21.15.16:0/928913505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c003600 con 0x7efe00005220 2024-09-10T21:10:23.811 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.806+0000 7efe21000640 1 -- 172.21.15.16:0/928913505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1c1f4bf0 con 0x7efe00005220 2024-09-10T21:10:23.811 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.806+0000 7efe21000640 1 -- 172.21.15.16:0/928913505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1c0b3aa0 con 0x7efe00005220 2024-09-10T21:10:23.811 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.806+0000 7efe13400640 1 -- 172.21.15.16:0/928913505 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efe0c018070 con 0x7efe00005220 2024-09-10T21:10:23.812 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.806+0000 7efe13400640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efdf80725e0 0x7efdf8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:23.812 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.806+0000 7efe13400640 1 -- 172.21.15.16:0/928913505 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7efe0c093ba0 con 0x7efe00005220 2024-09-10T21:10:23.812 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.806+0000 7efe13e00640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efdf80725e0 0x7efdf8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:23.813 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.810+0000 7efe13e00640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efdf80725e0 0x7efdf8074aa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0704b0 tx=0x7efe04006730 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:23.816 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.810+0000 7efe13400640 1 -- 172.21.15.16:0/928913505 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efe0c05f5c0 con 0x7efe00005220 2024-09-10T21:10:23.990 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.986+0000 7efe21000640 1 -- 172.21.15.16:0/928913505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe1c06ff20 con 0x7efe00005220 2024-09-10T21:10:23.991 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.986+0000 7efe13400640 1 -- 172.21.15.16:0/928913505 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12848 (secure 0 0 0) 0x7efe0c05eec0 con 0x7efe00005220 2024-09-10T21:10:23.991 INFO:teuthology.misc.health.smithi016.stdout: 2024-09-10T21:10:23.992 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":17,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:10:20.079230+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:10:23.994 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.990+0000 7efe21000640 1 -- 172.21.15.16:0/928913505 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efdf80725e0 msgr2=0x7efdf8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:23.994 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.990+0000 7efe21000640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efdf80725e0 0x7efdf8074aa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0704b0 tx=0x7efe04006730 comp rx=0 tx=0).stop 2024-09-10T21:10:23.994 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.990+0000 7efe21000640 1 -- 172.21.15.16:0/928913505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe00005220 msgr2=0x7efe1c143a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:23.994 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.990+0000 7efe21000640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe00005220 0x7efe1c143a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efe0c011860 tx=0x7efe0c011d20 comp rx=0 tx=0).stop 2024-09-10T21:10:23.994 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.990+0000 7efe21000640 1 -- 172.21.15.16:0/928913505 shutdown_connections 2024-09-10T21:10:23.994 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.990+0000 7efe21000640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efdf80725e0 0x7efdf8074aa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:23.994 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.990+0000 7efe21000640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c150430 0x7efe1c1508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:23.994 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.990+0000 7efe21000640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe1c1f8240 0x7efe1c070ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:23.994 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.990+0000 7efe21000640 1 --2- 172.21.15.16:0/928913505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe00005220 0x7efe1c143a80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:23.994 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.990+0000 7efe21000640 1 -- 172.21.15.16:0/928913505 >> 172.21.15.16:0/928913505 conn(0x7efe1c06dcf0 msgr2=0x7efe1c1556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:23.994 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.990+0000 7efe21000640 1 -- 172.21.15.16:0/928913505 shutdown_connections 2024-09-10T21:10:23.994 INFO:teuthology.misc.health.smithi016.stderr:2024-09-10T21:10:23.990+0000 7efe21000640 1 -- 172.21.15.16:0/928913505 wait complete. 2024-09-10T21:10:24.011 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-10T21:10:24.011 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T21:10:24.011 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:10:24.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:10:24.015 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:10:24.015 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:10:24.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:10:24.019 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:10:24.019 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:10:24.019 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:10:24.022 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:10:24.023 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:10:24.023 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:10:24.026 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:10:24.026 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-10T21:10:24.026 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:10:24.113 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-10T21:10:24.113 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-10T21:10:24.127 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:10:24.127 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:10:24.129 INFO:tasks.ceph.mds.d.smithi052.stderr:2024-09-10T21:10:24.120+0000 7fa2b7600640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-10T21:10:24.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.178+0000 7f3ed0600640 1 -- 172.21.15.16:0/3371726295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec815f400 msgr2=0x7f3ec8070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:24.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.178+0000 7f3ed0600640 1 --2- 172.21.15.16:0/3371726295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec815f400 0x7f3ec8070fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc005fd0 tx=0x7f3ebc01c700 comp rx=0 tx=0).stop 2024-09-10T21:10:24.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.178+0000 7f3ed0600640 1 -- 172.21.15.16:0/3371726295 shutdown_connections 2024-09-10T21:10:24.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.178+0000 7f3ed0600640 1 --2- 172.21.15.16:0/3371726295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec815f400 0x7f3ec8070fe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:24.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.178+0000 7f3ed0600640 1 -- 172.21.15.16:0/3371726295 >> 172.21.15.16:0/3371726295 conn(0x7f3ec809b130 msgr2=0x7f3ec80d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:24.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.178+0000 7f3ed0600640 1 -- 172.21.15.16:0/3371726295 shutdown_connections 2024-09-10T21:10:24.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.178+0000 7f3ed0600640 1 -- 172.21.15.16:0/3371726295 wait complete. 2024-09-10T21:10:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ed0600640 1 Processor -- start 2024-09-10T21:10:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ed0600640 1 -- start start 2024-09-10T21:10:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ed0600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ec80b0590 0x7f3ec806f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ed0600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ec80b13c0 0x7f3ec814db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ed0600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec815f400 0x7f3ec815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ed0600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec816a800 con 0x7f3ec815f400 2024-09-10T21:10:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ed0600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec80d1260 con 0x7f3ec80b0590 2024-09-10T21:10:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ed0600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec816a980 con 0x7f3ec80b13c0 2024-09-10T21:10:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ece600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec815f400 0x7f3ec815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ecdc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ec80b0590 0x7f3ec806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ece600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec815f400 0x7f3ec815ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43802/0 (socket says 172.21.15.16:43802) 2024-09-10T21:10:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ece600640 1 -- 172.21.15.16:0/4242847887 learned_addr learned my addr 172.21.15.16:0/4242847887 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ecd200640 1 --2- 172.21.15.16:0/4242847887 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ec80b13c0 0x7f3ec814db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ece600640 1 -- 172.21.15.16:0/4242847887 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ec80b13c0 msgr2=0x7f3ec814db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ece600640 1 --2- 172.21.15.16:0/4242847887 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ec80b13c0 0x7f3ec814db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ece600640 1 -- 172.21.15.16:0/4242847887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ec80b0590 msgr2=0x7f3ec806f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ece600640 1 --2- 172.21.15.16:0/4242847887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ec80b0590 0x7f3ec806f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ece600640 1 -- 172.21.15.16:0/4242847887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec806df90 con 0x7f3ec815f400 2024-09-10T21:10:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ece600640 1 --2- 172.21.15.16:0/4242847887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec815f400 0x7f3ec815ba50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000c7b0 tx=0x7f3ec000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ec5600640 1 -- 172.21.15.16:0/4242847887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0004b70 con 0x7f3ec815f400 2024-09-10T21:10:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ec5600640 1 -- 172.21.15.16:0/4242847887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ec0004e00 con 0x7f3ec815f400 2024-09-10T21:10:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ec5600640 1 -- 172.21.15.16:0/4242847887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec000aab0 con 0x7f3ec815f400 2024-09-10T21:10:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ed0600640 1 -- 172.21.15.16:0/4242847887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec806e270 con 0x7f3ec815f400 2024-09-10T21:10:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ed0600640 1 -- 172.21.15.16:0/4242847887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec806e830 con 0x7f3ec815f400 2024-09-10T21:10:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.182+0000 7f3ed0600640 1 -- 172.21.15.16:0/4242847887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec8070fe0 con 0x7f3ec815f400 2024-09-10T21:10:24.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.186+0000 7f3ec5600640 1 -- 172.21.15.16:0/4242847887 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3ec000ac50 con 0x7f3ec815f400 2024-09-10T21:10:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.186+0000 7f3ec5600640 1 --2- 172.21.15.16:0/4242847887 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3ea80725e0 0x7f3ea8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.186+0000 7f3ec5600640 1 -- 172.21.15.16:0/4242847887 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f3ec0092ff0 con 0x7f3ec815f400 2024-09-10T21:10:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.186+0000 7f3ecdc00640 1 --2- 172.21.15.16:0/4242847887 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3ea80725e0 0x7f3ea8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.186+0000 7f3ecdc00640 1 --2- 172.21.15.16:0/4242847887 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3ea80725e0 0x7f3ea8074aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0ae000 tx=0x7f3ebc0b2040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:24.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.190+0000 7f3ec5600640 1 -- 172.21.15.16:0/4242847887 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3ec0020070 con 0x7f3ec815f400 2024-09-10T21:10:24.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.398+0000 7f3ed0600640 1 -- 172.21.15.16:0/4242847887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ec80b0320 con 0x7f3ec815f400 2024-09-10T21:10:24.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.398+0000 7f3ec5600640 1 -- 172.21.15.16:0/4242847887 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7f3ec005e420 con 0x7f3ec815f400 2024-09-10T21:10:24.403 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"btime":"2024-09-10T21:10:24:119475+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2107270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2107270748},{"type":"v1","addr":"172.21.15.16:6836","nonce":2107270748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T21:10:24.403 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-09-10T21:10:24.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.402+0000 7f3ed0600640 1 -- 172.21.15.16:0/4242847887 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3ea80725e0 msgr2=0x7f3ea8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:24.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.402+0000 7f3ed0600640 1 --2- 172.21.15.16:0/4242847887 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3ea80725e0 0x7f3ea8074aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0ae000 tx=0x7f3ebc0b2040 comp rx=0 tx=0).stop 2024-09-10T21:10:24.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.402+0000 7f3ed0600640 1 -- 172.21.15.16:0/4242847887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec815f400 msgr2=0x7f3ec815ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:24.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.402+0000 7f3ed0600640 1 --2- 172.21.15.16:0/4242847887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec815f400 0x7f3ec815ba50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000c7b0 tx=0x7f3ec000cc70 comp rx=0 tx=0).stop 2024-09-10T21:10:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.402+0000 7f3ed0600640 1 -- 172.21.15.16:0/4242847887 shutdown_connections 2024-09-10T21:10:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.402+0000 7f3ed0600640 1 --2- 172.21.15.16:0/4242847887 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3ea80725e0 0x7f3ea8074aa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.402+0000 7f3ed0600640 1 --2- 172.21.15.16:0/4242847887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec815f400 0x7f3ec815ba50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.402+0000 7f3ed0600640 1 --2- 172.21.15.16:0/4242847887 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ec80b13c0 0x7f3ec814db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.402+0000 7f3ed0600640 1 --2- 172.21.15.16:0/4242847887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ec80b0590 0x7f3ec806f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.402+0000 7f3ed0600640 1 -- 172.21.15.16:0/4242847887 >> 172.21.15.16:0/4242847887 conn(0x7f3ec809b130 msgr2=0x7f3ec80b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.402+0000 7f3ed0600640 1 -- 172.21.15.16:0/4242847887 shutdown_connections 2024-09-10T21:10:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:24.402+0000 7f3ed0600640 1 -- 172.21.15.16:0/4242847887 wait complete. 2024-09-10T21:10:26.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:10:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 -- 172.21.15.16:0/2316042805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1110003a60 msgr2=0x7f1110003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 --2- 172.21.15.16:0/2316042805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1110003a60 0x7f1110003e80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f111c001720 tx=0x7f111c01c770 comp rx=0 tx=0).stop 2024-09-10T21:10:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 -- 172.21.15.16:0/2316042805 shutdown_connections 2024-09-10T21:10:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 --2- 172.21.15.16:0/2316042805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1110003a60 0x7f1110003e80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 -- 172.21.15.16:0/2316042805 >> 172.21.15.16:0/2316042805 conn(0x7f112809b130 msgr2=0x7f11280d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 -- 172.21.15.16:0/2316042805 shutdown_connections 2024-09-10T21:10:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 -- 172.21.15.16:0/2316042805 wait complete. 2024-09-10T21:10:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 Processor -- start 2024-09-10T21:10:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 -- start start 2024-09-10T21:10:26.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1110003a60 0x7f11281f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:26.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11280c6640 0x7f11280b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280b5510 0x7f11280cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11280d0490 con 0x7f11280b5510 2024-09-10T21:10:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f11280d1260 con 0x7f11280c6640 2024-09-10T21:10:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f1130200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f11280d0610 con 0x7f1110003a60 2024-09-10T21:10:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f112e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280b5510 0x7f11280cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f112e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280b5510 0x7f11280cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43828/0 (socket says 172.21.15.16:43828) 2024-09-10T21:10:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f112e200640 1 -- 172.21.15.16:0/812399225 learned_addr learned my addr 172.21.15.16:0/812399225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f112ce00640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11280c6640 0x7f11280b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f112d800640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1110003a60 0x7f11281f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f112e200640 1 -- 172.21.15.16:0/812399225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1110003a60 msgr2=0x7f11281f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f112e200640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1110003a60 0x7f11281f66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:26.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f112e200640 1 -- 172.21.15.16:0/812399225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11280c6640 msgr2=0x7f11280b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:26.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f112e200640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11280c6640 0x7f11280b7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:26.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f112e200640 1 -- 172.21.15.16:0/812399225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11280c6e50 con 0x7f11280b5510 2024-09-10T21:10:26.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f112ce00640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11280c6640 0x7f11280b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:26.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f112d800640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1110003a60 0x7f11281f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:26.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.582+0000 7f112e200640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280b5510 0x7f11280cdb50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f11280cf300 tx=0x7f112000eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:26.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.586+0000 7f1125600640 1 -- 172.21.15.16:0/812399225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112000ca30 con 0x7f11280b5510 2024-09-10T21:10:26.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.586+0000 7f1125600640 1 -- 172.21.15.16:0/812399225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1120018070 con 0x7f11280b5510 2024-09-10T21:10:26.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.586+0000 7f1125600640 1 -- 172.21.15.16:0/812399225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120011460 con 0x7f11280b5510 2024-09-10T21:10:26.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.586+0000 7f1130200640 1 -- 172.21.15.16:0/812399225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11280c7130 con 0x7f11280b5510 2024-09-10T21:10:26.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.586+0000 7f1130200640 1 -- 172.21.15.16:0/812399225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11280b5010 con 0x7f11280b5510 2024-09-10T21:10:26.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.586+0000 7f1130200640 1 -- 172.21.15.16:0/812399225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11280af580 con 0x7f11280b5510 2024-09-10T21:10:26.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.586+0000 7f1125600640 1 -- 172.21.15.16:0/812399225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1120011600 con 0x7f11280b5510 2024-09-10T21:10:26.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.586+0000 7f1125600640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11080725e0 0x7f1108074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:26.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.586+0000 7f1125600640 1 -- 172.21.15.16:0/812399225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f11200933b0 con 0x7f11280b5510 2024-09-10T21:10:26.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.586+0000 7f112d800640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11080725e0 0x7f1108074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:26.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.590+0000 7f112d800640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11080725e0 0x7f1108074aa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f111c0ac000 tx=0x7f111c0a8040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:26.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.590+0000 7f1125600640 1 -- 172.21.15.16:0/812399225 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f112005ed40 con 0x7f11280b5510 2024-09-10T21:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.798+0000 7f1130200640 1 -- 172.21.15.16:0/812399225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11280c7300 con 0x7f11280b5510 2024-09-10T21:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.798+0000 7f1125600640 1 -- 172.21.15.16:0/812399225 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3903 (secure 0 0 0) 0x7f112005e640 con 0x7f11280b5510 2024-09-10T21:10:26.805 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:26.805 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"btime":"2024-09-10T21:10:25:120921+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2107270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2107270748},{"type":"v1","addr":"172.21.15.16:6836","nonce":2107270748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/591195258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":591195258},{"type":"v1","addr":"172.21.15.16:6837","nonce":591195258}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2924498152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2924498152},{"type":"v1","addr":"172.21.15.52:6834","nonce":2924498152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3067177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3067177622},{"type":"v1","addr":"172.21.15.52:6835","nonce":3067177622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T21:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2024-09-10T21:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.802+0000 7f1130200640 1 -- 172.21.15.16:0/812399225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11080725e0 msgr2=0x7f1108074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.802+0000 7f1130200640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11080725e0 0x7f1108074aa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f111c0ac000 tx=0x7f111c0a8040 comp rx=0 tx=0).stop 2024-09-10T21:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.802+0000 7f1130200640 1 -- 172.21.15.16:0/812399225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280b5510 msgr2=0x7f11280cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.802+0000 7f1130200640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280b5510 0x7f11280cdb50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f11280cf300 tx=0x7f112000eae0 comp rx=0 tx=0).stop 2024-09-10T21:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.802+0000 7f1130200640 1 -- 172.21.15.16:0/812399225 shutdown_connections 2024-09-10T21:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.802+0000 7f1130200640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11080725e0 0x7f1108074aa0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.802+0000 7f1130200640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11280b5510 0x7f11280cdb50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.802+0000 7f1130200640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11280c6640 0x7f11280b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.802+0000 7f1130200640 1 --2- 172.21.15.16:0/812399225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1110003a60 0x7f11281f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.802+0000 7f1130200640 1 -- 172.21.15.16:0/812399225 >> 172.21.15.16:0/812399225 conn(0x7f112809b130 msgr2=0x7f11280b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:26.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.802+0000 7f1130200640 1 -- 172.21.15.16:0/812399225 shutdown_connections 2024-09-10T21:10:26.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.802+0000 7f1130200640 1 -- 172.21.15.16:0/812399225 wait complete. 2024-09-10T21:10:26.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:10:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.966+0000 7f3388e00640 1 -- 172.21.15.16:0/3290206725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338415bc70 msgr2=0x7f338406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.966+0000 7f3388e00640 1 --2- 172.21.15.16:0/3290206725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338415bc70 0x7f338406ffe0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f336c009900 tx=0x7f336c01c880 comp rx=0 tx=0).stop 2024-09-10T21:10:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.966+0000 7f3388e00640 1 -- 172.21.15.16:0/3290206725 shutdown_connections 2024-09-10T21:10:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.966+0000 7f3388e00640 1 --2- 172.21.15.16:0/3290206725 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3384070520 0x7f33840ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.966+0000 7f3388e00640 1 --2- 172.21.15.16:0/3290206725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338415bc70 0x7f338406ffe0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.966+0000 7f3388e00640 1 -- 172.21.15.16:0/3290206725 >> 172.21.15.16:0/3290206725 conn(0x7f33840cfb20 msgr2=0x7f33840cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.966+0000 7f3388e00640 1 -- 172.21.15.16:0/3290206725 shutdown_connections 2024-09-10T21:10:26.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.966+0000 7f3388e00640 1 -- 172.21.15.16:0/3290206725 wait complete. 2024-09-10T21:10:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3388e00640 1 Processor -- start 2024-09-10T21:10:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3388e00640 1 -- start start 2024-09-10T21:10:26.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3388e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3384070520 0x7f33840c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:26.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3388e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3384154f90 0x7f3384147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:26.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3388e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f338415bc70 0x7f3384151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:26.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3388e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3384167980 con 0x7f3384154f90 2024-09-10T21:10:26.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3388e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33840cb690 con 0x7f338415bc70 2024-09-10T21:10:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3388e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3384167b00 con 0x7f3384070520 2024-09-10T21:10:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3379400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3384154f90 0x7f3384147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3379400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3384154f90 0x7f3384147f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43854/0 (socket says 172.21.15.16:43854) 2024-09-10T21:10:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3379400640 1 -- 172.21.15.16:0/2143743068 learned_addr learned my addr 172.21.15.16:0/2143743068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3381e00640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3384070520 0x7f33840c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3382800640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f338415bc70 0x7f3384151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3379400640 1 -- 172.21.15.16:0/2143743068 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3384070520 msgr2=0x7f33840c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3379400640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3384070520 0x7f33840c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3379400640 1 -- 172.21.15.16:0/2143743068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f338415bc70 msgr2=0x7f3384151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3379400640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f338415bc70 0x7f3384151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3379400640 1 -- 172.21.15.16:0/2143743068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33841f7b50 con 0x7f3384154f90 2024-09-10T21:10:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3379400640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3384154f90 0x7f3384147f30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f336c009850 tx=0x7f336c0a6f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f337aa00640 1 -- 172.21.15.16:0/2143743068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c0b2040 con 0x7f3384154f90 2024-09-10T21:10:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f337aa00640 1 -- 172.21.15.16:0/2143743068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f336c004430 con 0x7f3384154f90 2024-09-10T21:10:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3388e00640 1 -- 172.21.15.16:0/2143743068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33841f7e30 con 0x7f3384154f90 2024-09-10T21:10:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f337aa00640 1 -- 172.21.15.16:0/2143743068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c005240 con 0x7f3384154f90 2024-09-10T21:10:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.970+0000 7f3388e00640 1 -- 172.21.15.16:0/2143743068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3384148470 con 0x7f3384154f90 2024-09-10T21:10:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.974+0000 7f3388e00640 1 -- 172.21.15.16:0/2143743068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33840ca530 con 0x7f3384154f90 2024-09-10T21:10:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.974+0000 7f337aa00640 1 -- 172.21.15.16:0/2143743068 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f336c0053e0 con 0x7f3384154f90 2024-09-10T21:10:26.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.974+0000 7f337aa00640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33600725e0 0x7f3360074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:26.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.974+0000 7f337aa00640 1 -- 172.21.15.16:0/2143743068 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f336c12ca90 con 0x7f3384154f90 2024-09-10T21:10:26.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.974+0000 7f3381e00640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33600725e0 0x7f3360074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:26.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.974+0000 7f3381e00640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33600725e0 0x7f3360074aa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f33740016f0 tx=0x7f3374009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:26.978+0000 7f337aa00640 1 -- 172.21.15.16:0/2143743068 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f336c0164d0 con 0x7f3384154f90 2024-09-10T21:10:27.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.154+0000 7f3388e00640 1 -- 172.21.15.16:0/2143743068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f338415a870 con 0x7f3384154f90 2024-09-10T21:10:27.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.154+0000 7f337aa00640 1 -- 172.21.15.16:0/2143743068 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f336c0f8420 con 0x7f3384154f90 2024-09-10T21:10:27.160 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:27.160 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-10T21:10:27.161 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:10:27.161 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-10T21:10:27.161 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-10T21:10:27.161 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:10:27.161 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-10T21:10:27.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.158+0000 7f3388e00640 1 -- 172.21.15.16:0/2143743068 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33600725e0 msgr2=0x7f3360074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:27.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.158+0000 7f3388e00640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33600725e0 0x7f3360074aa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f33740016f0 tx=0x7f3374009040 comp rx=0 tx=0).stop 2024-09-10T21:10:27.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.158+0000 7f3388e00640 1 -- 172.21.15.16:0/2143743068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3384154f90 msgr2=0x7f3384147f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:27.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.158+0000 7f3388e00640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3384154f90 0x7f3384147f30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f336c009850 tx=0x7f336c0a6f60 comp rx=0 tx=0).stop 2024-09-10T21:10:27.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.158+0000 7f3388e00640 1 -- 172.21.15.16:0/2143743068 shutdown_connections 2024-09-10T21:10:27.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.158+0000 7f3388e00640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33600725e0 0x7f3360074aa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:27.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.158+0000 7f3388e00640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f338415bc70 0x7f3384151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:27.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.158+0000 7f3388e00640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3384154f90 0x7f3384147f30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:27.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.158+0000 7f3388e00640 1 --2- 172.21.15.16:0/2143743068 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3384070520 0x7f33840c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:27.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.158+0000 7f3388e00640 1 -- 172.21.15.16:0/2143743068 >> 172.21.15.16:0/2143743068 conn(0x7f33840cfb20 msgr2=0x7f3384152770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:27.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.158+0000 7f3388e00640 1 -- 172.21.15.16:0/2143743068 shutdown_connections 2024-09-10T21:10:27.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.158+0000 7f3388e00640 1 -- 172.21.15.16:0/2143743068 wait complete. 2024-09-10T21:10:27.181 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:10:27.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:10:27.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.326+0000 7fb674400640 1 -- 172.21.15.16:0/3784968473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb66c0d28b0 msgr2=0x7fb66c0cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:27.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.326+0000 7fb674400640 1 --2- 172.21.15.16:0/3784968473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb66c0d28b0 0x7fb66c0cece0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb66c074cd0 tx=0x7fb65801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:10:27.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.326+0000 7fb674400640 1 -- 172.21.15.16:0/3784968473 shutdown_connections 2024-09-10T21:10:27.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.326+0000 7fb674400640 1 --2- 172.21.15.16:0/3784968473 >> v2:172.21.15.52:3300/0 conn(0x7fb66c0cf220 0x7fb66c0cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:27.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.326+0000 7fb674400640 1 --2- 172.21.15.16:0/3784968473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb66c0d28b0 0x7fb66c0cece0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.326+0000 7fb674400640 1 --2- 172.21.15.16:0/3784968473 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb66c15bf00 0x7fb66c0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.326+0000 7fb674400640 1 -- 172.21.15.16:0/3784968473 >> 172.21.15.16:0/3784968473 conn(0x7fb66c0d1d90 msgr2=0x7fb66c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.326+0000 7fb674400640 1 -- 172.21.15.16:0/3784968473 shutdown_connections 2024-09-10T21:10:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.326+0000 7fb674400640 1 -- 172.21.15.16:0/3784968473 wait complete. 2024-09-10T21:10:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb674400640 1 Processor -- start 2024-09-10T21:10:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb674400640 1 -- start start 2024-09-10T21:10:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb674400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb66c0cf220 0x7fb66c1f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb674400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb66c0d28b0 0x7fb66c1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb674400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb66c15bf00 0x7fb66c15b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb674400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c0b27a0 con 0x7fb66c15bf00 2024-09-10T21:10:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb674400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c0b2620 con 0x7fb66c0cf220 2024-09-10T21:10:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb674400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb66c0b2920 con 0x7fb66c0d28b0 2024-09-10T21:10:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb672400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb66c15bf00 0x7fb66c15b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb671000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb66c0d28b0 0x7fb66c1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb671a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb66c0cf220 0x7fb66c1f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb672400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb66c15bf00 0x7fb66c15b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43876/0 (socket says 172.21.15.16:43876) 2024-09-10T21:10:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb672400640 1 -- 172.21.15.16:0/2587992871 learned_addr learned my addr 172.21.15.16:0/2587992871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb672400640 1 -- 172.21.15.16:0/2587992871 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb66c0d28b0 msgr2=0x7fb66c1528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb672400640 1 --2- 172.21.15.16:0/2587992871 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb66c0d28b0 0x7fb66c1528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb672400640 1 -- 172.21.15.16:0/2587992871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb66c0cf220 msgr2=0x7fb66c1f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb672400640 1 --2- 172.21.15.16:0/2587992871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb66c0cf220 0x7fb66c1f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb672400640 1 -- 172.21.15.16:0/2587992871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66c1f9040 con 0x7fb66c15bf00 2024-09-10T21:10:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb671000640 1 --2- 172.21.15.16:0/2587992871 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb66c0d28b0 0x7fb66c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb672400640 1 --2- 172.21.15.16:0/2587992871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb66c15bf00 0x7fb66c15b010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb66400ccb0 tx=0x7fb66400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb669600640 1 -- 172.21.15.16:0/2587992871 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66400f450 con 0x7fb66c15bf00 2024-09-10T21:10:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb669600640 1 -- 172.21.15.16:0/2587992871 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb66400f700 con 0x7fb66c15bf00 2024-09-10T21:10:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb674400640 1 -- 172.21.15.16:0/2587992871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66c1f9320 con 0x7fb66c15bf00 2024-09-10T21:10:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.330+0000 7fb669600640 1 -- 172.21.15.16:0/2587992871 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66400fb50 con 0x7fb66c15bf00 2024-09-10T21:10:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.334+0000 7fb674400640 1 -- 172.21.15.16:0/2587992871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66c0c8f00 con 0x7fb66c15bf00 2024-09-10T21:10:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.334+0000 7fb674400640 1 -- 172.21.15.16:0/2587992871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb66c0c9100 con 0x7fb66c15bf00 2024-09-10T21:10:27.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.334+0000 7fb669600640 1 -- 172.21.15.16:0/2587992871 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb664020030 con 0x7fb66c15bf00 2024-09-10T21:10:27.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.334+0000 7fb669600640 1 --2- 172.21.15.16:0/2587992871 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb64c0725e0 0x7fb64c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:27.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.334+0000 7fb669600640 1 -- 172.21.15.16:0/2587992871 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fb664093640 con 0x7fb66c15bf00 2024-09-10T21:10:27.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.334+0000 7fb671a00640 1 --2- 172.21.15.16:0/2587992871 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb64c0725e0 0x7fb64c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:27.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.334+0000 7fb671a00640 1 --2- 172.21.15.16:0/2587992871 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb64c0725e0 0x7fb64c074aa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb66c1f89c0 tx=0x7fb66000a250 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:27.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.338+0000 7fb669600640 1 -- 172.21.15.16:0/2587992871 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb664014030 con 0x7fb66c15bf00 2024-09-10T21:10:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:27.510+0000 7fb674400640 1 -- 172.21.15.16:0/2587992871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb66c0b2e40 con 0x7fb66c15bf00 2024-09-10T21:10:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.134+0000 7fb669600640 1 -- 172.21.15.16:0/2587992871 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fb66405ea10 con 0x7fb66c15bf00 2024-09-10T21:10:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-10T21:10:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.138+0000 7fb674400640 1 -- 172.21.15.16:0/2587992871 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb64c0725e0 msgr2=0x7fb64c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.138+0000 7fb674400640 1 --2- 172.21.15.16:0/2587992871 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb64c0725e0 0x7fb64c074aa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb66c1f89c0 tx=0x7fb66000a250 comp rx=0 tx=0).stop 2024-09-10T21:10:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.138+0000 7fb674400640 1 -- 172.21.15.16:0/2587992871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb66c15bf00 msgr2=0x7fb66c15b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.138+0000 7fb674400640 1 --2- 172.21.15.16:0/2587992871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb66c15bf00 0x7fb66c15b010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb66400ccb0 tx=0x7fb66400a620 comp rx=0 tx=0).stop 2024-09-10T21:10:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.138+0000 7fb674400640 1 -- 172.21.15.16:0/2587992871 shutdown_connections 2024-09-10T21:10:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.138+0000 7fb674400640 1 --2- 172.21.15.16:0/2587992871 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb64c0725e0 0x7fb64c074aa0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.138+0000 7fb674400640 1 --2- 172.21.15.16:0/2587992871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb66c15bf00 0x7fb66c15b010 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.138+0000 7fb674400640 1 --2- 172.21.15.16:0/2587992871 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb66c0d28b0 0x7fb66c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.138+0000 7fb674400640 1 --2- 172.21.15.16:0/2587992871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb66c0cf220 0x7fb66c1f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.138+0000 7fb674400640 1 -- 172.21.15.16:0/2587992871 >> 172.21.15.16:0/2587992871 conn(0x7fb66c0d1d90 msgr2=0x7fb66c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:28.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.138+0000 7fb674400640 1 -- 172.21.15.16:0/2587992871 shutdown_connections 2024-09-10T21:10:28.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.138+0000 7fb674400640 1 -- 172.21.15.16:0/2587992871 wait complete. 2024-09-10T21:10:28.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:10:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 -- 172.21.15.16:0/3418533585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8acc0cbee0 msgr2=0x7f8acc0cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 --2- 172.21.15.16:0/3418533585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8acc0cbee0 0x7f8acc0cc390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009900 tx=0x7f8ac401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:10:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 -- 172.21.15.16:0/3418533585 shutdown_connections 2024-09-10T21:10:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 --2- 172.21.15.16:0/3418533585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8acc0cbee0 0x7f8acc0cc390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 --2- 172.21.15.16:0/3418533585 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8acc06f0e0 0x7f8acc06f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 -- 172.21.15.16:0/3418533585 >> 172.21.15.16:0/3418533585 conn(0x7f8acc0b16a0 msgr2=0x7f8acc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 -- 172.21.15.16:0/3418533585 shutdown_connections 2024-09-10T21:10:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 -- 172.21.15.16:0/3418533585 wait complete. 2024-09-10T21:10:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 Processor -- start 2024-09-10T21:10:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 -- start start 2024-09-10T21:10:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8acc06f0e0 0x7f8acc15ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8acc0cbb00 0x7f8acc15c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8acc0cbee0 0x7f8acc15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc168d00 con 0x7f8acc0cbee0 2024-09-10T21:10:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc168b80 con 0x7f8acc06f0e0 2024-09-10T21:10:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ad0e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8acc168e80 con 0x7f8acc0cbb00 2024-09-10T21:10:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8aca800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8acc0cbee0 0x7f8acc15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8aca800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8acc0cbee0 0x7f8acc15c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43896/0 (socket says 172.21.15.16:43896) 2024-09-10T21:10:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8aca800640 1 -- 172.21.15.16:0/192544260 learned_addr learned my addr 172.21.15.16:0/192544260 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ac1400640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8acc0cbb00 0x7f8acc15c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ac9e00640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8acc06f0e0 0x7f8acc15ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8aca800640 1 -- 172.21.15.16:0/192544260 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8acc0cbb00 msgr2=0x7f8acc15c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8aca800640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8acc0cbb00 0x7f8acc15c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8aca800640 1 -- 172.21.15.16:0/192544260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8acc06f0e0 msgr2=0x7f8acc15ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8aca800640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8acc06f0e0 0x7f8acc15ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8aca800640 1 -- 172.21.15.16:0/192544260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc15d1c0 con 0x7f8acc0cbee0 2024-09-10T21:10:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.326+0000 7f8ac9e00640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8acc06f0e0 0x7f8acc15ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.330+0000 7f8aca800640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8acc0cbee0 0x7f8acc15c8e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40b0040 tx=0x7f8ac40a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.330+0000 7f8ac2a00640 1 -- 172.21.15.16:0/192544260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40bb070 con 0x7f8acc0cbee0 2024-09-10T21:10:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.330+0000 7f8ac2a00640 1 -- 172.21.15.16:0/192544260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ac4005070 con 0x7f8acc0cbee0 2024-09-10T21:10:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.330+0000 7f8ac2a00640 1 -- 172.21.15.16:0/192544260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40b2040 con 0x7f8acc0cbee0 2024-09-10T21:10:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.330+0000 7f8ad0e00640 1 -- 172.21.15.16:0/192544260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8acc140750 con 0x7f8acc0cbee0 2024-09-10T21:10:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.330+0000 7f8ad0e00640 1 -- 172.21.15.16:0/192544260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8acc13a840 con 0x7f8acc0cbee0 2024-09-10T21:10:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.330+0000 7f8ad0e00640 1 -- 172.21.15.16:0/192544260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8acc06f520 con 0x7f8acc0cbee0 2024-09-10T21:10:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.330+0000 7f8ac2a00640 1 -- 172.21.15.16:0/192544260 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8ac40054a0 con 0x7f8acc0cbee0 2024-09-10T21:10:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.334+0000 7f8ac2a00640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8aa8072610 0x7f8aa8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.334+0000 7f8ac2a00640 1 -- 172.21.15.16:0/192544260 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f8ac412c710 con 0x7f8acc0cbee0 2024-09-10T21:10:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.334+0000 7f8ac9e00640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8aa8072610 0x7f8aa8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.334+0000 7f8ac9e00640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8aa8072610 0x7f8aa8074ad0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8abc001e50 tx=0x7f8abc009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.338+0000 7f8ac2a00640 1 -- 172.21.15.16:0/192544260 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ac40164d0 con 0x7f8acc0cbee0 2024-09-10T21:10:28.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:28.522+0000 7f8ad0e00640 1 -- 172.21.15.16:0/192544260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8acc0adcc0 con 0x7f8acc0cbee0 2024-09-10T21:10:29.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.142+0000 7f8ac2a00640 1 -- 172.21.15.16:0/192544260 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7f8ac40f7f30 con 0x7f8acc0cbee0 2024-09-10T21:10:29.146 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-10T21:10:29.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.142+0000 7f8ad0e00640 1 -- 172.21.15.16:0/192544260 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8aa8072610 msgr2=0x7f8aa8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:29.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.142+0000 7f8ad0e00640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8aa8072610 0x7f8aa8074ad0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8abc001e50 tx=0x7f8abc009040 comp rx=0 tx=0).stop 2024-09-10T21:10:29.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.142+0000 7f8ad0e00640 1 -- 172.21.15.16:0/192544260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8acc0cbee0 msgr2=0x7f8acc15c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:29.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.142+0000 7f8ad0e00640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8acc0cbee0 0x7f8acc15c8e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40b0040 tx=0x7f8ac40a57f0 comp rx=0 tx=0).stop 2024-09-10T21:10:29.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.146+0000 7f8ad0e00640 1 -- 172.21.15.16:0/192544260 shutdown_connections 2024-09-10T21:10:29.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.146+0000 7f8ad0e00640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8aa8072610 0x7f8aa8074ad0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:29.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.146+0000 7f8ad0e00640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8acc0cbee0 0x7f8acc15c8e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:29.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.146+0000 7f8ad0e00640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8acc0cbb00 0x7f8acc15c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:29.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.146+0000 7f8ad0e00640 1 --2- 172.21.15.16:0/192544260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8acc06f0e0 0x7f8acc15ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:29.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.146+0000 7f8ad0e00640 1 -- 172.21.15.16:0/192544260 >> 172.21.15.16:0/192544260 conn(0x7f8acc0b16a0 msgr2=0x7f8acc0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:29.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.146+0000 7f8ad0e00640 1 -- 172.21.15.16:0/192544260 shutdown_connections 2024-09-10T21:10:29.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.146+0000 7f8ad0e00640 1 -- 172.21.15.16:0/192544260 wait complete. 2024-09-10T21:10:29.192 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:10:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 -- 172.21.15.16:0/2420319826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4815e490 msgr2=0x7fae4806f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 --2- 172.21.15.16:0/2420319826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4815e490 0x7fae4806f800 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae4000ad90 tx=0x7fae4001dcb0 comp rx=0 tx=0).stop 2024-09-10T21:10:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 -- 172.21.15.16:0/2420319826 shutdown_connections 2024-09-10T21:10:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 --2- 172.21.15.16:0/2420319826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4815e490 0x7fae4806f800 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 --2- 172.21.15.16:0/2420319826 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fae48151210 0x7fae480726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 -- 172.21.15.16:0/2420319826 >> 172.21.15.16:0/2420319826 conn(0x7fae480b16a0 msgr2=0x7fae480b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 -- 172.21.15.16:0/2420319826 shutdown_connections 2024-09-10T21:10:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 -- 172.21.15.16:0/2420319826 wait complete. 2024-09-10T21:10:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 Processor -- start 2024-09-10T21:10:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 -- start start 2024-09-10T21:10:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48072bf0 0x7fae4814b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fae48151210 0x7fae481526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae4815e490 0x7fae48152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae480705d0 con 0x7fae48072bf0 2024-09-10T21:10:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae48070450 con 0x7fae4815e490 2024-09-10T21:10:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae4ce00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fae48070750 con 0x7fae48151210 2024-09-10T21:10:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48072bf0 0x7fae4814b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48072bf0 0x7fae4814b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43918/0 (socket says 172.21.15.16:43918) 2024-09-10T21:10:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae45e00640 1 -- 172.21.15.16:0/2971480720 learned_addr learned my addr 172.21.15.16:0/2971480720 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae46800640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae4815e490 0x7fae48152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae3fe00640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fae48151210 0x7fae481526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae45e00640 1 -- 172.21.15.16:0/2971480720 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fae48151210 msgr2=0x7fae481526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae45e00640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fae48151210 0x7fae481526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae45e00640 1 -- 172.21.15.16:0/2971480720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae4815e490 msgr2=0x7fae48152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae45e00640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae4815e490 0x7fae48152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.338+0000 7fae45e00640 1 -- 172.21.15.16:0/2971480720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae481518c0 con 0x7fae48072bf0 2024-09-10T21:10:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.342+0000 7fae45e00640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48072bf0 0x7fae4814b800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae3800d950 tx=0x7fae3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.342+0000 7fae3e000640 1 -- 172.21.15.16:0/2971480720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38017070 con 0x7fae48072bf0 2024-09-10T21:10:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.342+0000 7fae3e000640 1 -- 172.21.15.16:0/2971480720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae38009d20 con 0x7fae48072bf0 2024-09-10T21:10:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.342+0000 7fae3e000640 1 -- 172.21.15.16:0/2971480720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae380106c0 con 0x7fae48072bf0 2024-09-10T21:10:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.342+0000 7fae4ce00640 1 -- 172.21.15.16:0/2971480720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae48151b60 con 0x7fae48072bf0 2024-09-10T21:10:29.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.342+0000 7fae4ce00640 1 -- 172.21.15.16:0/2971480720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae48152120 con 0x7fae48072bf0 2024-09-10T21:10:29.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.342+0000 7fae4ce00640 1 -- 172.21.15.16:0/2971480720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae480726b0 con 0x7fae48072bf0 2024-09-10T21:10:29.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.342+0000 7fae3e000640 1 -- 172.21.15.16:0/2971480720 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fae38010860 con 0x7fae48072bf0 2024-09-10T21:10:29.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.342+0000 7fae3e000640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fae24072610 0x7fae24074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:29.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.346+0000 7fae3e000640 1 -- 172.21.15.16:0/2971480720 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7fae38093aa0 con 0x7fae48072bf0 2024-09-10T21:10:29.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.346+0000 7fae3fe00640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fae24072610 0x7fae24074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:29.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.346+0000 7fae3fe00640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fae24072610 0x7fae24074ad0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae48072970 tx=0x7fae30009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:29.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.346+0000 7fae3e000640 1 -- 172.21.15.16:0/2971480720 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fae3805f1c0 con 0x7fae48072bf0 2024-09-10T21:10:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:29.558+0000 7fae4ce00640 1 -- 172.21.15.16:0/2971480720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fae480b1f20 con 0x7fae48072bf0 2024-09-10T21:10:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.154+0000 7fae3e000640 1 -- 172.21.15.16:0/2971480720 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fae3805eac0 con 0x7fae48072bf0 2024-09-10T21:10:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T21:10:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.170+0000 7fae4ce00640 1 -- 172.21.15.16:0/2971480720 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fae24072610 msgr2=0x7fae24074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.170+0000 7fae4ce00640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fae24072610 0x7fae24074ad0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae48072970 tx=0x7fae30009040 comp rx=0 tx=0).stop 2024-09-10T21:10:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.170+0000 7fae4ce00640 1 -- 172.21.15.16:0/2971480720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48072bf0 msgr2=0x7fae4814b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.170+0000 7fae4ce00640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48072bf0 0x7fae4814b800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae3800d950 tx=0x7fae3800de10 comp rx=0 tx=0).stop 2024-09-10T21:10:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.174+0000 7fae4ce00640 1 -- 172.21.15.16:0/2971480720 shutdown_connections 2024-09-10T21:10:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.174+0000 7fae4ce00640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fae24072610 0x7fae24074ad0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.174+0000 7fae4ce00640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae4815e490 0x7fae48152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.174+0000 7fae4ce00640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fae48151210 0x7fae481526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.174+0000 7fae4ce00640 1 --2- 172.21.15.16:0/2971480720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48072bf0 0x7fae4814b800 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.174+0000 7fae4ce00640 1 -- 172.21.15.16:0/2971480720 >> 172.21.15.16:0/2971480720 conn(0x7fae480b16a0 msgr2=0x7fae48159fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.174+0000 7fae4ce00640 1 -- 172.21.15.16:0/2971480720 shutdown_connections 2024-09-10T21:10:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.174+0000 7fae4ce00640 1 -- 172.21.15.16:0/2971480720 wait complete. 2024-09-10T21:10:30.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:10:30.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.358+0000 7f9eb3200640 1 -- 172.21.15.16:0/143732500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9eac0b13c0 msgr2=0x7f9eac071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:30.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.358+0000 7f9eb3200640 1 --2- 172.21.15.16:0/143732500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9eac0b13c0 0x7f9eac071520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009840 tx=0x7f9e9401c620 comp rx=0 tx=0).stop 2024-09-10T21:10:30.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.358+0000 7f9eb3200640 1 -- 172.21.15.16:0/143732500 shutdown_connections 2024-09-10T21:10:30.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.358+0000 7f9eb3200640 1 --2- 172.21.15.16:0/143732500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9eac0b13c0 0x7f9eac071520 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.358+0000 7f9eb3200640 1 --2- 172.21.15.16:0/143732500 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eac15f400 0x7f9eac070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.358+0000 7f9eb3200640 1 -- 172.21.15.16:0/143732500 >> 172.21.15.16:0/143732500 conn(0x7f9eac09b130 msgr2=0x7f9eac0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:30.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.358+0000 7f9eb3200640 1 -- 172.21.15.16:0/143732500 shutdown_connections 2024-09-10T21:10:30.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.358+0000 7f9eb3200640 1 -- 172.21.15.16:0/143732500 wait complete. 2024-09-10T21:10:30.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb3200640 1 Processor -- start 2024-09-10T21:10:30.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb3200640 1 -- start start 2024-09-10T21:10:30.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb3200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eac0b13c0 0x7f9eac153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:30.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb3200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9eac0cfc80 0x7f9eac0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:30.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac15f400 0x7f9eac0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:30.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac169770 con 0x7f9eac15f400 2024-09-10T21:10:30.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb3200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1695f0 con 0x7f9eac0cfc80 2024-09-10T21:10:30.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb3200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9eac1698f0 con 0x7f9eac0b13c0 2024-09-10T21:10:30.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac15f400 0x7f9eac0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:30.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac15f400 0x7f9eac0ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43924/0 (socket says 172.21.15.16:43924) 2024-09-10T21:10:30.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb1200640 1 -- 172.21.15.16:0/3137875128 learned_addr learned my addr 172.21.15.16:0/3137875128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:30.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eabe00640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eac0b13c0 0x7f9eac153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:30.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9ea3e00640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9eac0cfc80 0x7f9eac0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:30.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb1200640 1 -- 172.21.15.16:0/3137875128 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eac0b13c0 msgr2=0x7f9eac153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:30.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb1200640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eac0b13c0 0x7f9eac153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb1200640 1 -- 172.21.15.16:0/3137875128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9eac0cfc80 msgr2=0x7f9eac0cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:30.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb1200640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9eac0cfc80 0x7f9eac0cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb1200640 1 -- 172.21.15.16:0/3137875128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac0b4c50 con 0x7f9eac15f400 2024-09-10T21:10:30.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb1200640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac15f400 0x7f9eac0ce4a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400cfd0 tx=0x7f9ea4001fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:30.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9ea8c00640 1 -- 172.21.15.16:0/3137875128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4016020 con 0x7f9eac15f400 2024-09-10T21:10:30.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9ea8c00640 1 -- 172.21.15.16:0/3137875128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ea400ab60 con 0x7f9eac15f400 2024-09-10T21:10:30.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9eb3200640 1 -- 172.21.15.16:0/3137875128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac153e90 con 0x7f9eac15f400 2024-09-10T21:10:30.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.362+0000 7f9ea8c00640 1 -- 172.21.15.16:0/3137875128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4012420 con 0x7f9eac15f400 2024-09-10T21:10:30.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.366+0000 7f9eb3200640 1 -- 172.21.15.16:0/3137875128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac164cb0 con 0x7f9eac15f400 2024-09-10T21:10:30.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.366+0000 7f9eb3200640 1 -- 172.21.15.16:0/3137875128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eac0d00d0 con 0x7f9eac15f400 2024-09-10T21:10:30.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.366+0000 7f9ea8c00640 1 -- 172.21.15.16:0/3137875128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9ea40125c0 con 0x7f9eac15f400 2024-09-10T21:10:30.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.366+0000 7f9ea8c00640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e880725e0 0x7f9e88074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:30.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.366+0000 7f9ea8c00640 1 -- 172.21.15.16:0/3137875128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9ea4093960 con 0x7f9eac15f400 2024-09-10T21:10:30.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.366+0000 7f9eabe00640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e880725e0 0x7f9e88074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:30.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.366+0000 7f9eabe00640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e880725e0 0x7f9e88074aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0095b0 tx=0x7f9e9c005f40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:30.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.370+0000 7f9ea8c00640 1 -- 172.21.15.16:0/3137875128 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9ea4020020 con 0x7f9eac15f400 2024-09-10T21:10:30.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.550+0000 7f9eb3200640 1 -- 172.21.15.16:0/3137875128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9eac0b1f00 con 0x7f9eac15f400 2024-09-10T21:10:30.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.550+0000 7f9ea8c00640 1 -- 172.21.15.16:0/3137875128 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16634 (secure 0 0 0) 0x7f9ea405e950 con 0x7f9eac15f400 2024-09-10T21:10:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:30.556 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:10:29.560224+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:10:27.523741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:10:28.533379+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:10:30.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.554+0000 7f9eb3200640 1 -- 172.21.15.16:0/3137875128 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e880725e0 msgr2=0x7f9e88074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:30.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.554+0000 7f9eb3200640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e880725e0 0x7f9e88074aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0095b0 tx=0x7f9e9c005f40 comp rx=0 tx=0).stop 2024-09-10T21:10:30.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.554+0000 7f9eb3200640 1 -- 172.21.15.16:0/3137875128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac15f400 msgr2=0x7f9eac0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:30.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.554+0000 7f9eb3200640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac15f400 0x7f9eac0ce4a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400cfd0 tx=0x7f9ea4001fe0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.554+0000 7f9eb3200640 1 -- 172.21.15.16:0/3137875128 shutdown_connections 2024-09-10T21:10:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.554+0000 7f9eb3200640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e880725e0 0x7f9e88074aa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.554+0000 7f9eb3200640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac15f400 0x7f9eac0ce4a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.554+0000 7f9eb3200640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9eac0cfc80 0x7f9eac0cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.554+0000 7f9eb3200640 1 --2- 172.21.15.16:0/3137875128 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eac0b13c0 0x7f9eac153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.554+0000 7f9eb3200640 1 -- 172.21.15.16:0/3137875128 >> 172.21.15.16:0/3137875128 conn(0x7f9eac09b130 msgr2=0x7f9eac15d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.554+0000 7f9eb3200640 1 -- 172.21.15.16:0/3137875128 shutdown_connections 2024-09-10T21:10:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.554+0000 7f9eb3200640 1 -- 172.21.15.16:0/3137875128 wait complete. 2024-09-10T21:10:30.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:10:30.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.718+0000 7f1155800640 1 -- 172.21.15.16:0/4294607498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11340051d0 msgr2=0x7f113400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:30.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.718+0000 7f1155800640 1 --2- 172.21.15.16:0/4294607498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11340051d0 0x7f113400b9d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1148001dc0 tx=0x7f114801cb30 comp rx=0 tx=0).stop 2024-09-10T21:10:30.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.718+0000 7f1155800640 1 -- 172.21.15.16:0/4294607498 shutdown_connections 2024-09-10T21:10:30.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.718+0000 7f1155800640 1 --2- 172.21.15.16:0/4294607498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11340051d0 0x7f113400b9d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.718+0000 7f1155800640 1 -- 172.21.15.16:0/4294607498 >> 172.21.15.16:0/4294607498 conn(0x7f115009b130 msgr2=0x7f11500d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:30.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.718+0000 7f1155800640 1 -- 172.21.15.16:0/4294607498 shutdown_connections 2024-09-10T21:10:30.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.718+0000 7f1155800640 1 -- 172.21.15.16:0/4294607498 wait complete. 2024-09-10T21:10:30.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1155800640 1 Processor -- start 2024-09-10T21:10:30.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1155800640 1 -- start start 2024-09-10T21:10:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1155800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11340051d0 0x7f11500c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1155800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11500b7670 0x7f115015c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1155800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f11500c6f60 0x7f11500c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1155800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11500d0710 con 0x7f11340051d0 2024-09-10T21:10:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1155800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f11500d0590 con 0x7f11500b7670 2024-09-10T21:10:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1155800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f11500d0890 con 0x7f11500c6f60 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f114ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11340051d0 0x7f11500c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f114ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11340051d0 0x7f11500c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43934/0 (socket says 172.21.15.16:43934) 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f114ea00640 1 -- 172.21.15.16:0/4162377921 learned_addr learned my addr 172.21.15.16:0/4162377921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1147e00640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11500b7670 0x7f115015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f114ea00640 1 -- 172.21.15.16:0/4162377921 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f11500c6f60 msgr2=0x7f11500c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f114ea00640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f11500c6f60 0x7f11500c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f114ea00640 1 -- 172.21.15.16:0/4162377921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11500b7670 msgr2=0x7f115015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f114ea00640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11500b7670 0x7f115015c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f114ea00640 1 -- 172.21.15.16:0/4162377921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115015c980 con 0x7f11340051d0 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1147e00640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11500b7670 0x7f115015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f114ea00640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11340051d0 0x7f11500c57e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f114000e5c0 tx=0x7f1140001960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1146a00640 1 -- 172.21.15.16:0/4162377921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140015020 con 0x7f11340051d0 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1146a00640 1 -- 172.21.15.16:0/4162377921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f114000ca00 con 0x7f11340051d0 2024-09-10T21:10:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1155800640 1 -- 172.21.15.16:0/4162377921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115009f9e0 con 0x7f11340051d0 2024-09-10T21:10:30.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1146a00640 1 -- 172.21.15.16:0/4162377921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114000ce50 con 0x7f11340051d0 2024-09-10T21:10:30.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.722+0000 7f1155800640 1 -- 172.21.15.16:0/4162377921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11500c61a0 con 0x7f11340051d0 2024-09-10T21:10:30.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.726+0000 7f1155800640 1 -- 172.21.15.16:0/4162377921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1150071000 con 0x7f11340051d0 2024-09-10T21:10:30.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.726+0000 7f1146a00640 1 -- 172.21.15.16:0/4162377921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1140011460 con 0x7f11340051d0 2024-09-10T21:10:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.726+0000 7f1146a00640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f112c072610 0x7f112c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.726+0000 7f1146a00640 1 -- 172.21.15.16:0/4162377921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1140093ac0 con 0x7f11340051d0 2024-09-10T21:10:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.726+0000 7f1147e00640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f112c072610 0x7f112c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.726+0000 7f1147e00640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f112c072610 0x7f112c074ad0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1138009470 tx=0x7f11380091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:30.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.730+0000 7f1146a00640 1 -- 172.21.15.16:0/4162377921 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f114001f020 con 0x7f11340051d0 2024-09-10T21:10:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.906+0000 7f1155800640 1 -- 172.21.15.16:0/4162377921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11500af490 con 0x7f11340051d0 2024-09-10T21:10:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.906+0000 7f1146a00640 1 -- 172.21.15.16:0/4162377921 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16634 (secure 0 0 0) 0x7f114005eab0 con 0x7f11340051d0 2024-09-10T21:10:30.914 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:30.914 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:10:29.560224+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:10:27.523741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:10:28.533379+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:10:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.910+0000 7f1155800640 1 -- 172.21.15.16:0/4162377921 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f112c072610 msgr2=0x7f112c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.910+0000 7f1155800640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f112c072610 0x7f112c074ad0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1138009470 tx=0x7f11380091e0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.910+0000 7f1155800640 1 -- 172.21.15.16:0/4162377921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11340051d0 msgr2=0x7f11500c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.910+0000 7f1155800640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11340051d0 0x7f11500c57e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f114000e5c0 tx=0x7f1140001960 comp rx=0 tx=0).stop 2024-09-10T21:10:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.910+0000 7f1155800640 1 -- 172.21.15.16:0/4162377921 shutdown_connections 2024-09-10T21:10:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.910+0000 7f1155800640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f112c072610 0x7f112c074ad0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.910+0000 7f1155800640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f11500c6f60 0x7f11500c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.910+0000 7f1155800640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11500b7670 0x7f115015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.910+0000 7f1155800640 1 --2- 172.21.15.16:0/4162377921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11340051d0 0x7f11500c57e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.910+0000 7f1155800640 1 -- 172.21.15.16:0/4162377921 >> 172.21.15.16:0/4162377921 conn(0x7f115009b130 msgr2=0x7f11500b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.910+0000 7f1155800640 1 -- 172.21.15.16:0/4162377921 shutdown_connections 2024-09-10T21:10:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:30.910+0000 7f1155800640 1 -- 172.21.15.16:0/4162377921 wait complete. 2024-09-10T21:10:30.933 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:10:31.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.078+0000 7f472d200640 1 -- 172.21.15.16:0/2574676030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472815f280 msgr2=0x7f472806d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:31.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.078+0000 7f472d200640 1 --2- 172.21.15.16:0/2574676030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472815f280 0x7f472806d5f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f47280cbb70 tx=0x7f471801c7d0 comp rx=0 tx=0).stop 2024-09-10T21:10:31.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.078+0000 7f472d200640 1 -- 172.21.15.16:0/2574676030 shutdown_connections 2024-09-10T21:10:31.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.078+0000 7f472d200640 1 --2- 172.21.15.16:0/2574676030 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f47280b3fc0 0x7f4728073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:31.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.078+0000 7f472d200640 1 --2- 172.21.15.16:0/2574676030 >> v2:172.21.15.52:3300/0 conn(0x7f47280d1750 0x7f47280b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:31.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.078+0000 7f472d200640 1 --2- 172.21.15.16:0/2574676030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472815f280 0x7f472806d5f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:31.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.078+0000 7f472d200640 1 -- 172.21.15.16:0/2574676030 >> 172.21.15.16:0/2574676030 conn(0x7f47280d0c30 msgr2=0x7f47280d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:31.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.078+0000 7f472d200640 1 -- 172.21.15.16:0/2574676030 shutdown_connections 2024-09-10T21:10:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.078+0000 7f472d200640 1 -- 172.21.15.16:0/2574676030 wait complete. 2024-09-10T21:10:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.078+0000 7f472d200640 1 Processor -- start 2024-09-10T21:10:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.078+0000 7f472d200640 1 -- start start 2024-09-10T21:10:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f472d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47280b3fc0 0x7f47281660e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f472d200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47280d1750 0x7f472814fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f472d200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f472815f280 0x7f47280ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f472d200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4728070080 con 0x7f47280b3fc0 2024-09-10T21:10:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f472d200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f472806ff00 con 0x7f47280d1750 2024-09-10T21:10:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f472d200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4728070200 con 0x7f472815f280 2024-09-10T21:10:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f4725e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47280b3fc0 0x7f47281660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f4725e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47280b3fc0 0x7f47281660e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43944/0 (socket says 172.21.15.16:43944) 2024-09-10T21:10:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f4725e00640 1 -- 172.21.15.16:0/1947213159 learned_addr learned my addr 172.21.15.16:0/1947213159 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f4725400640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47280d1750 0x7f472814fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f4725e00640 1 -- 172.21.15.16:0/1947213159 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f472815f280 msgr2=0x7f47280ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f4725e00640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f472815f280 0x7f47280ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f4725e00640 1 -- 172.21.15.16:0/1947213159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47280d1750 msgr2=0x7f472814fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f4725e00640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47280d1750 0x7f472814fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f4725e00640 1 -- 172.21.15.16:0/1947213159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47281501a0 con 0x7f47280b3fc0 2024-09-10T21:10:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f4725400640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47280d1750 0x7f472814fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f4725e00640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47280b3fc0 0x7f47281660e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f47180b0040 tx=0x7f47180a6c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f471e000640 1 -- 172.21.15.16:0/1947213159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47180ba020 con 0x7f47280b3fc0 2024-09-10T21:10:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f471e000640 1 -- 172.21.15.16:0/1947213159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4718004330 con 0x7f47280b3fc0 2024-09-10T21:10:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f472d200640 1 -- 172.21.15.16:0/1947213159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4728164770 con 0x7f47280b3fc0 2024-09-10T21:10:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f471e000640 1 -- 172.21.15.16:0/1947213159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47180b2040 con 0x7f47280b3fc0 2024-09-10T21:10:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f472d200640 1 -- 172.21.15.16:0/1947213159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47280c8fa0 con 0x7f47280b3fc0 2024-09-10T21:10:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.082+0000 7f472d200640 1 -- 172.21.15.16:0/1947213159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4728164c90 con 0x7f47280b3fc0 2024-09-10T21:10:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.086+0000 7f471e000640 1 -- 172.21.15.16:0/1947213159 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4718003860 con 0x7f47280b3fc0 2024-09-10T21:10:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.086+0000 7f471e000640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f47080725e0 0x7f4708074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.086+0000 7f471e000640 1 -- 172.21.15.16:0/1947213159 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f471812ca90 con 0x7f47280b3fc0 2024-09-10T21:10:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.086+0000 7f4725400640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f47080725e0 0x7f4708074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.086+0000 7f4725400640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f47080725e0 0x7f4708074aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4714005fd0 tx=0x7f4714005da0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.090+0000 7f471e000640 1 -- 172.21.15.16:0/1947213159 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f47180a6e20 con 0x7f47280b3fc0 2024-09-10T21:10:31.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:31.302+0000 7f472d200640 1 -- 172.21.15.16:0/1947213159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f47280b3a80 con 0x7f47280b3fc0 2024-09-10T21:10:32.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.170+0000 7f471e000640 1 -- 172.21.15.16:0/1947213159 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f47180164b0 con 0x7f47280b3fc0 2024-09-10T21:10:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.174+0000 7f472d200640 1 -- 172.21.15.16:0/1947213159 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f47080725e0 msgr2=0x7f4708074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.174+0000 7f472d200640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f47080725e0 0x7f4708074aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4714005fd0 tx=0x7f4714005da0 comp rx=0 tx=0).stop 2024-09-10T21:10:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.174+0000 7f472d200640 1 -- 172.21.15.16:0/1947213159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47280b3fc0 msgr2=0x7f47281660e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.174+0000 7f472d200640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47280b3fc0 0x7f47281660e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f47180b0040 tx=0x7f47180a6c30 comp rx=0 tx=0).stop 2024-09-10T21:10:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.174+0000 7f472d200640 1 -- 172.21.15.16:0/1947213159 shutdown_connections 2024-09-10T21:10:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.174+0000 7f472d200640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f47080725e0 0x7f4708074aa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.174+0000 7f472d200640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f472815f280 0x7f47280ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.174+0000 7f472d200640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47280d1750 0x7f472814fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.174+0000 7f472d200640 1 --2- 172.21.15.16:0/1947213159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47280b3fc0 0x7f47281660e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.174+0000 7f472d200640 1 -- 172.21.15.16:0/1947213159 >> 172.21.15.16:0/1947213159 conn(0x7f47280d0c30 msgr2=0x7f472806d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.174+0000 7f472d200640 1 -- 172.21.15.16:0/1947213159 shutdown_connections 2024-09-10T21:10:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.174+0000 7f472d200640 1 -- 172.21.15.16:0/1947213159 wait complete. 2024-09-10T21:10:32.202 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:10:32.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:10:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.350+0000 7fb3c4200640 1 -- 172.21.15.16:0/3151987598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc160050 msgr2=0x7fb3bc0d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.350+0000 7fb3c4200640 1 --2- 172.21.15.16:0/3151987598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc160050 0x7fb3bc0d2800 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009bc0 tx=0x7fb3b401ca80 comp rx=0 tx=0).stop 2024-09-10T21:10:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.350+0000 7fb3c4200640 1 -- 172.21.15.16:0/3151987598 shutdown_connections 2024-09-10T21:10:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.350+0000 7fb3c4200640 1 --2- 172.21.15.16:0/3151987598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc160050 0x7fb3bc0d2800 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.350+0000 7fb3c4200640 1 -- 172.21.15.16:0/3151987598 >> 172.21.15.16:0/3151987598 conn(0x7fb3bc0d1ce0 msgr2=0x7fb3bc0d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.350+0000 7fb3c4200640 1 -- 172.21.15.16:0/3151987598 shutdown_connections 2024-09-10T21:10:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.350+0000 7fb3c4200640 1 -- 172.21.15.16:0/3151987598 wait complete. 2024-09-10T21:10:32.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c4200640 1 Processor -- start 2024-09-10T21:10:32.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c4200640 1 -- start start 2024-09-10T21:10:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c4200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3bc0cd300 0x7fb3bc166a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c4200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc15bdc0 0x7fb3bc0b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c4200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3bc160050 0x7fb3bc149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c4200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc16a300 con 0x7fb3bc15bdc0 2024-09-10T21:10:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c4200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc16a180 con 0x7fb3bc0cd300 2024-09-10T21:10:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c4200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb3bc16a480 con 0x7fb3bc160050 2024-09-10T21:10:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc15bdc0 0x7fb3bc0b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc15bdc0 0x7fb3bc0b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38264/0 (socket says 172.21.15.16:38264) 2024-09-10T21:10:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c0e00640 1 -- 172.21.15.16:0/3262280170 learned_addr learned my addr 172.21.15.16:0/3262280170 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c1800640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3bc0cd300 0x7fb3bc166a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c2200640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3bc160050 0x7fb3bc149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c0e00640 1 -- 172.21.15.16:0/3262280170 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3bc160050 msgr2=0x7fb3bc149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c0e00640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3bc160050 0x7fb3bc149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c0e00640 1 -- 172.21.15.16:0/3262280170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3bc0cd300 msgr2=0x7fb3bc166a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c0e00640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3bc0cd300 0x7fb3bc166a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c0e00640 1 -- 172.21.15.16:0/3262280170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc165f80 con 0x7fb3bc15bdc0 2024-09-10T21:10:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c0e00640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc15bdc0 0x7fb3bc0b6850 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c400 tx=0x7fb3a800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3b9600640 1 -- 172.21.15.16:0/3262280170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8011790 con 0x7fb3bc15bdc0 2024-09-10T21:10:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3b9600640 1 -- 172.21.15.16:0/3262280170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3a8017070 con 0x7fb3bc15bdc0 2024-09-10T21:10:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3b9600640 1 -- 172.21.15.16:0/3262280170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8011d00 con 0x7fb3bc15bdc0 2024-09-10T21:10:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c4200640 1 -- 172.21.15.16:0/3262280170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc164ee0 con 0x7fb3bc15bdc0 2024-09-10T21:10:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.354+0000 7fb3c4200640 1 -- 172.21.15.16:0/3262280170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc165780 con 0x7fb3bc15bdc0 2024-09-10T21:10:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.358+0000 7fb3c4200640 1 -- 172.21.15.16:0/3262280170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3bc0d2800 con 0x7fb3bc15bdc0 2024-09-10T21:10:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.358+0000 7fb3b9600640 1 -- 172.21.15.16:0/3262280170 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb3a801c460 con 0x7fb3bc15bdc0 2024-09-10T21:10:32.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.358+0000 7fb3b9600640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb39c0725e0 0x7fb39c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:32.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.358+0000 7fb3b9600640 1 -- 172.21.15.16:0/3262280170 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb3a8093090 con 0x7fb3bc15bdc0 2024-09-10T21:10:32.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.358+0000 7fb3c1800640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb39c0725e0 0x7fb39c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:32.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.358+0000 7fb3c1800640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb39c0725e0 0x7fb39c074aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0004e30 tx=0x7fb3b0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:32.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.362+0000 7fb3b9600640 1 -- 172.21.15.16:0/3262280170 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb3bc165f80 con 0x7fb3bc15bdc0 2024-09-10T21:10:32.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:32.570+0000 7fb3c4200640 1 -- 172.21.15.16:0/3262280170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb3bc0704a0 con 0x7fb3bc15bdc0 2024-09-10T21:10:33.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.174+0000 7fb3b9600640 1 -- 172.21.15.16:0/3262280170 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fb3a805e760 con 0x7fb3bc15bdc0 2024-09-10T21:10:33.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.174+0000 7fb3c4200640 1 -- 172.21.15.16:0/3262280170 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb39c0725e0 msgr2=0x7fb39c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:33.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.174+0000 7fb3c4200640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb39c0725e0 0x7fb39c074aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0004e30 tx=0x7fb3b0009040 comp rx=0 tx=0).stop 2024-09-10T21:10:33.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.174+0000 7fb3c4200640 1 -- 172.21.15.16:0/3262280170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc15bdc0 msgr2=0x7fb3bc0b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.174+0000 7fb3c4200640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc15bdc0 0x7fb3bc0b6850 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c400 tx=0x7fb3a800c8c0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.174+0000 7fb3c4200640 1 -- 172.21.15.16:0/3262280170 shutdown_connections 2024-09-10T21:10:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.174+0000 7fb3c4200640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb39c0725e0 0x7fb39c074aa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.174+0000 7fb3c4200640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3bc160050 0x7fb3bc149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.174+0000 7fb3c4200640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc15bdc0 0x7fb3bc0b6850 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.174+0000 7fb3c4200640 1 --2- 172.21.15.16:0/3262280170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3bc0cd300 0x7fb3bc166a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.174+0000 7fb3c4200640 1 -- 172.21.15.16:0/3262280170 >> 172.21.15.16:0/3262280170 conn(0x7fb3bc0d1ce0 msgr2=0x7fb3bc15af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.174+0000 7fb3c4200640 1 -- 172.21.15.16:0/3262280170 shutdown_connections 2024-09-10T21:10:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.174+0000 7fb3c4200640 1 -- 172.21.15.16:0/3262280170 wait complete. 2024-09-10T21:10:33.203 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:10:33.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:10:33.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.350+0000 7f02fb200640 1 -- 172.21.15.16:0/849778302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8004810 msgr2=0x7f02d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:33.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.350+0000 7f02fb200640 1 --2- 172.21.15.16:0/849778302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8004810 0x7f02d8004c90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009b30 tx=0x7f02dc01d210 comp rx=0 tx=0).stop 2024-09-10T21:10:33.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.350+0000 7f02f9200640 1 -- 172.21.15.16:0/849778302 reap_dead start 2024-09-10T21:10:33.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.350+0000 7f02fb200640 1 -- 172.21.15.16:0/849778302 shutdown_connections 2024-09-10T21:10:33.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.350+0000 7f02fb200640 1 -- 172.21.15.16:0/849778302 >> 172.21.15.16:0/849778302 conn(0x7f02f40d3f30 msgr2=0x7f02f40d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:33.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.350+0000 7f02fb200640 1 -- 172.21.15.16:0/849778302 shutdown_connections 2024-09-10T21:10:33.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.350+0000 7f02fb200640 1 -- 172.21.15.16:0/849778302 wait complete. 2024-09-10T21:10:33.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.350+0000 7f02fb200640 1 Processor -- start 2024-09-10T21:10:33.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.350+0000 7f02fb200640 1 -- start start 2024-09-10T21:10:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02fb200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02d8004810 0x7f02f4156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02fb200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02f4157280 0x7f02f40cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02fb200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02f40cda30 0x7f02f40cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02fb200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4168de0 con 0x7f02f40cda30 2024-09-10T21:10:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02fb200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f02f40d2b70 con 0x7f02f4157280 2024-09-10T21:10:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02fb200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f02f4168f60 con 0x7f02d8004810 2024-09-10T21:10:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02f40cda30 0x7f02f40cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02f40cda30 0x7f02f40cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38288/0 (socket says 172.21.15.16:38288) 2024-09-10T21:10:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f9200640 1 -- 172.21.15.16:0/2884024192 learned_addr learned my addr 172.21.15.16:0/2884024192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02ebe00640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02f4157280 0x7f02f40cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f3e00640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02d8004810 0x7f02f4156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f9200640 1 -- 172.21.15.16:0/2884024192 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02d8004810 msgr2=0x7f02f4156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f9200640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02d8004810 0x7f02f4156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f9200640 1 -- 172.21.15.16:0/2884024192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02f4157280 msgr2=0x7f02f40cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f9200640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02f4157280 0x7f02f40cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f9200640 1 -- 172.21.15.16:0/2884024192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f40b0b90 con 0x7f02f40cda30 2024-09-10T21:10:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f9200640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02f40cda30 0x7f02f40cbbf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f02f40b5150 tx=0x7f02ec010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f0c00640 1 -- 172.21.15.16:0/2884024192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec00ee10 con 0x7f02f40cda30 2024-09-10T21:10:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f0c00640 1 -- 172.21.15.16:0/2884024192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02ec015070 con 0x7f02f40cda30 2024-09-10T21:10:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02fb200640 1 -- 172.21.15.16:0/2884024192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f40cc8d0 con 0x7f02f40cda30 2024-09-10T21:10:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f0c00640 1 -- 172.21.15.16:0/2884024192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec003d10 con 0x7f02f40cda30 2024-09-10T21:10:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02fb200640 1 -- 172.21.15.16:0/2884024192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f40cce90 con 0x7f02f40cda30 2024-09-10T21:10:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02fb200640 1 -- 172.21.15.16:0/2884024192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f40cd0d0 con 0x7f02f40cda30 2024-09-10T21:10:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.354+0000 7f02f0c00640 1 -- 172.21.15.16:0/2884024192 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f02ec024060 con 0x7f02f40cda30 2024-09-10T21:10:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.358+0000 7f02f0c00640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f02d0072610 0x7f02d0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.358+0000 7f02f0c00640 1 -- 172.21.15.16:0/2884024192 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f02ec093660 con 0x7f02f40cda30 2024-09-10T21:10:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.358+0000 7f02f3e00640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f02d0072610 0x7f02d0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.358+0000 7f02f3e00640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f02d0072610 0x7f02d0074ad0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f02e40049f0 tx=0x7f02e4006040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.362+0000 7f02f0c00640 1 -- 172.21.15.16:0/2884024192 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f02f40cd0d0 con 0x7f02f40cda30 2024-09-10T21:10:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.570+0000 7f02fb200640 1 -- 172.21.15.16:0/2884024192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02f40d1b00 con 0x7f02f40cda30 2024-09-10T21:10:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.570+0000 7f02f0c00640 1 -- 172.21.15.16:0/2884024192 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f02ec05e650 con 0x7f02f40cda30 2024-09-10T21:10:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:33.575 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-10T21:10:33:173828+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2107270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2107270748},{"type":"v1","addr":"172.21.15.16:6836","nonce":2107270748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/591195258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":591195258},{"type":"v1","addr":"172.21.15.16:6837","nonce":591195258}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2924498152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2924498152},{"type":"v1","addr":"172.21.15.52:6834","nonce":2924498152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:10:29.560192+0000","modified":"2024-09-10T21:10:32.575127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4356":{"gid":4356,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/3067177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3067177622},{"type":"v1","addr":"172.21.15.52:6835","nonce":3067177622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4356,"qdb_cluster":[4356]},"id":1}]} 2024-09-10T21:10:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-10T21:10:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.574+0000 7f02fb200640 1 -- 172.21.15.16:0/2884024192 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f02d0072610 msgr2=0x7f02d0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.574+0000 7f02fb200640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f02d0072610 0x7f02d0074ad0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f02e40049f0 tx=0x7f02e4006040 comp rx=0 tx=0).stop 2024-09-10T21:10:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.574+0000 7f02fb200640 1 -- 172.21.15.16:0/2884024192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02f40cda30 msgr2=0x7f02f40cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.574+0000 7f02fb200640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02f40cda30 0x7f02f40cbbf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f02f40b5150 tx=0x7f02ec010f20 comp rx=0 tx=0).stop 2024-09-10T21:10:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.574+0000 7f02fb200640 1 -- 172.21.15.16:0/2884024192 shutdown_connections 2024-09-10T21:10:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.574+0000 7f02fb200640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f02d0072610 0x7f02d0074ad0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.574+0000 7f02fb200640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02f40cda30 0x7f02f40cbbf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.574+0000 7f02fb200640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02f4157280 0x7f02f40cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.574+0000 7f02fb200640 1 --2- 172.21.15.16:0/2884024192 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02d8004810 0x7f02f4156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.574+0000 7f02fb200640 1 -- 172.21.15.16:0/2884024192 >> 172.21.15.16:0/2884024192 conn(0x7f02f40d3f30 msgr2=0x7f02f4167950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.574+0000 7f02fb200640 1 -- 172.21.15.16:0/2884024192 shutdown_connections 2024-09-10T21:10:33.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.574+0000 7f02fb200640 1 -- 172.21.15.16:0/2884024192 wait complete. 2024-09-10T21:10:33.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:10:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.738+0000 7f951f400640 1 -- 172.21.15.16:0/1156814240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180cf7d0 msgr2=0x7f95180d2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.738+0000 7f951f400640 1 --2- 172.21.15.16:0/1156814240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180cf7d0 0x7f95180d2710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9510009bc0 tx=0x7f951001cb70 comp rx=0 tx=0).stop 2024-09-10T21:10:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.738+0000 7f951f400640 1 -- 172.21.15.16:0/1156814240 shutdown_connections 2024-09-10T21:10:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.738+0000 7f951f400640 1 --2- 172.21.15.16:0/1156814240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180cf7d0 0x7f95180d2710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.738+0000 7f951f400640 1 --2- 172.21.15.16:0/1156814240 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f951815e200 0x7f95180ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.738+0000 7f951f400640 1 -- 172.21.15.16:0/1156814240 >> 172.21.15.16:0/1156814240 conn(0x7f95180d3f70 msgr2=0x7f95180d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.738+0000 7f951f400640 1 -- 172.21.15.16:0/1156814240 shutdown_connections 2024-09-10T21:10:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.738+0000 7f951f400640 1 -- 172.21.15.16:0/1156814240 wait complete. 2024-09-10T21:10:33.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.738+0000 7f951f400640 1 Processor -- start 2024-09-10T21:10:33.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951f400640 1 -- start start 2024-09-10T21:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180cf7d0 0x7f95180cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951f400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f95180d4b00 0x7f9518150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951815e200 0x7f9518154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951f400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f951816ae30 con 0x7f951815e200 2024-09-10T21:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951f400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f95180d3a30 con 0x7f95180cf7d0 2024-09-10T21:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951f400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f951816afb0 con 0x7f95180d4b00 2024-09-10T21:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951815e200 0x7f9518154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951ca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180cf7d0 0x7f95180cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951815e200 0x7f9518154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38304/0 (socket says 172.21.15.16:38304) 2024-09-10T21:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951d400640 1 -- 172.21.15.16:0/4249761204 learned_addr learned my addr 172.21.15.16:0/4249761204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f9517e00640 1 --2- 172.21.15.16:0/4249761204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f95180d4b00 0x7f9518150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951d400640 1 -- 172.21.15.16:0/4249761204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f95180d4b00 msgr2=0x7f9518150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951d400640 1 --2- 172.21.15.16:0/4249761204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f95180d4b00 0x7f9518150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951d400640 1 -- 172.21.15.16:0/4249761204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180cf7d0 msgr2=0x7f95180cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951d400640 1 --2- 172.21.15.16:0/4249761204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180cf7d0 0x7f95180cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951d400640 1 -- 172.21.15.16:0/4249761204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9518151290 con 0x7f951815e200 2024-09-10T21:10:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951d400640 1 --2- 172.21.15.16:0/4249761204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951815e200 0x7f9518154190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f951001c5e0 tx=0x7f9510005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f9514c00640 1 -- 172.21.15.16:0/4249761204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95100b2040 con 0x7f951815e200 2024-09-10T21:10:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f9514c00640 1 -- 172.21.15.16:0/4249761204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95100bb070 con 0x7f951815e200 2024-09-10T21:10:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951f400640 1 -- 172.21.15.16:0/4249761204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95181fb840 con 0x7f951815e200 2024-09-10T21:10:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f9514c00640 1 -- 172.21.15.16:0/4249761204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9510004390 con 0x7f951815e200 2024-09-10T21:10:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951f400640 1 -- 172.21.15.16:0/4249761204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95181fad60 con 0x7f951815e200 2024-09-10T21:10:33.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.742+0000 7f951f400640 1 -- 172.21.15.16:0/4249761204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95180d2710 con 0x7f951815e200 2024-09-10T21:10:33.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.746+0000 7f9514c00640 1 -- 172.21.15.16:0/4249761204 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f95100b5070 con 0x7f951815e200 2024-09-10T21:10:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.746+0000 7f9514c00640 1 --2- 172.21.15.16:0/4249761204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f94f80725e0 0x7f94f8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.746+0000 7f9514c00640 1 -- 172.21.15.16:0/4249761204 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f951012d670 con 0x7f951815e200 2024-09-10T21:10:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.746+0000 7f951ca00640 1 --2- 172.21.15.16:0/4249761204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f94f80725e0 0x7f94f8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.746+0000 7f951ca00640 1 --2- 172.21.15.16:0/4249761204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f94f80725e0 0x7f94f8074aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f950c009b40 tx=0x7f950c0091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:33.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.750+0000 7f9514c00640 1 -- 172.21.15.16:0/4249761204 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9518151290 con 0x7f951815e200 2024-09-10T21:10:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.926+0000 7f951f400640 1 -- 172.21.15.16:0/4249761204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95180cfe20 con 0x7f951815e200 2024-09-10T21:10:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.926+0000 7f9514c00640 1 -- 172.21.15.16:0/4249761204 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f9510016790 con 0x7f951815e200 2024-09-10T21:10:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:33.933 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:10:31.158120+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:10:27.523741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:10:28.533379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:10:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.930+0000 7f951f400640 1 -- 172.21.15.16:0/4249761204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f94f80725e0 msgr2=0x7f94f8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.930+0000 7f951f400640 1 --2- 172.21.15.16:0/4249761204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f94f80725e0 0x7f94f8074aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f950c009b40 tx=0x7f950c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.930+0000 7f951f400640 1 -- 172.21.15.16:0/4249761204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951815e200 msgr2=0x7f9518154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.930+0000 7f951f400640 1 --2- 172.21.15.16:0/4249761204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951815e200 0x7f9518154190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f951001c5e0 tx=0x7f9510005a80 comp rx=0 tx=0).stop 2024-09-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.930+0000 7f951f400640 1 -- 172.21.15.16:0/4249761204 shutdown_connections 2024-09-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.930+0000 7f951f400640 1 --2- 172.21.15.16:0/4249761204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f94f80725e0 0x7f94f8074aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.930+0000 7f951f400640 1 --2- 172.21.15.16:0/4249761204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951815e200 0x7f9518154190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.930+0000 7f951f400640 1 --2- 172.21.15.16:0/4249761204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f95180d4b00 0x7f9518150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.930+0000 7f951f400640 1 --2- 172.21.15.16:0/4249761204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95180cf7d0 0x7f95180cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.930+0000 7f951f400640 1 -- 172.21.15.16:0/4249761204 >> 172.21.15.16:0/4249761204 conn(0x7f95180d3f70 msgr2=0x7f95180b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.930+0000 7f951f400640 1 -- 172.21.15.16:0/4249761204 shutdown_connections 2024-09-10T21:10:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:33.930+0000 7f951f400640 1 -- 172.21.15.16:0/4249761204 wait complete. 2024-09-10T21:10:33.954 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:10:33.955 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:10:34.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 -- 172.21.15.16:0/3114470122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97cc004d50 msgr2=0x7f97cc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:34.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 --2- 172.21.15.16:0/3114470122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97cc004d50 0x7f97cc0051d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009eb0 tx=0x7f97dc01d5f0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ea400640 1 -- 172.21.15.16:0/3114470122 reap_dead start 2024-09-10T21:10:34.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 -- 172.21.15.16:0/3114470122 shutdown_connections 2024-09-10T21:10:34.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 -- 172.21.15.16:0/3114470122 >> 172.21.15.16:0/3114470122 conn(0x7f97e409b130 msgr2=0x7f97e40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:34.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 -- 172.21.15.16:0/3114470122 shutdown_connections 2024-09-10T21:10:34.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 -- 172.21.15.16:0/3114470122 wait complete. 2024-09-10T21:10:34.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 Processor -- start 2024-09-10T21:10:34.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 -- start start 2024-09-10T21:10:34.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97cc004d50 0x7f97e4160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:34.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97e4159c10 0x7f97e406f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:34.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e406f790 0x7f97e40ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:34.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97e4166f50 con 0x7f97e406f790 2024-09-10T21:10:34.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97e4166dd0 con 0x7f97cc004d50 2024-09-10T21:10:34.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ec400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f97e41670d0 con 0x7f97e4159c10 2024-09-10T21:10:34.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ea400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e406f790 0x7f97e40ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:34.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97e9a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97cc004d50 0x7f97e4160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:34.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97e9a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97cc004d50 0x7f97e4160d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:52716/0 (socket says 172.21.15.16:52716) 2024-09-10T21:10:34.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97e9a00640 1 -- 172.21.15.16:0/1431372184 learned_addr learned my addr 172.21.15.16:0/1431372184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:34.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ea400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e406f790 0x7f97e40ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38310/0 (socket says 172.21.15.16:38310) 2024-09-10T21:10:34.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97e9000640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97e4159c10 0x7f97e406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:34.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97e9000640 1 -- 172.21.15.16:0/1431372184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97cc004d50 msgr2=0x7f97e4160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:34.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97e9000640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97cc004d50 0x7f97e4160d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97e9000640 1 -- 172.21.15.16:0/1431372184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e406f790 msgr2=0x7f97e40ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:34.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97e9000640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e406f790 0x7f97e40ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97e9000640 1 -- 172.21.15.16:0/1431372184 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e41fb8c0 con 0x7f97e4159c10 2024-09-10T21:10:34.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97ea400640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e406f790 0x7f97e40ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:34.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.098+0000 7f97e9000640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97e4159c10 0x7f97e406f250 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f97e409b860 tx=0x7f97d0011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:34.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.102+0000 7f97e1600640 1 -- 172.21.15.16:0/1431372184 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d000f970 con 0x7f97e4159c10 2024-09-10T21:10:34.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.102+0000 7f97e1600640 1 -- 172.21.15.16:0/1431372184 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97d000fb10 con 0x7f97e4159c10 2024-09-10T21:10:34.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.102+0000 7f97e1600640 1 -- 172.21.15.16:0/1431372184 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0003600 con 0x7f97e4159c10 2024-09-10T21:10:34.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.102+0000 7f97ec400640 1 -- 172.21.15.16:0/1431372184 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e41fbba0 con 0x7f97e4159c10 2024-09-10T21:10:34.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.102+0000 7f97ec400640 1 -- 172.21.15.16:0/1431372184 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e41fc160 con 0x7f97e4159c10 2024-09-10T21:10:34.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.102+0000 7f97ec400640 1 -- 172.21.15.16:0/1431372184 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e40afaa0 con 0x7f97e4159c10 2024-09-10T21:10:34.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.102+0000 7f97e1600640 1 -- 172.21.15.16:0/1431372184 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f97d0018070 con 0x7f97e4159c10 2024-09-10T21:10:34.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.102+0000 7f97e1600640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f97c4072610 0x7f97c4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:34.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.102+0000 7f97e1600640 1 -- 172.21.15.16:0/1431372184 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f97d0094290 con 0x7f97e4159c10 2024-09-10T21:10:34.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.106+0000 7f97e9a00640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f97c4072610 0x7f97c4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:34.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.106+0000 7f97e9a00640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f97c4072610 0x7f97c4074ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97d8001a20 tx=0x7f97d8001090 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:34.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.106+0000 7f97e1600640 1 -- 172.21.15.16:0/1431372184 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f97d005f390 con 0x7f97e4159c10 2024-09-10T21:10:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.278+0000 7f97ec400640 1 -- 172.21.15.16:0/1431372184 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97e40cd6c0 con 0x7f97c4072610 2024-09-10T21:10:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.286+0000 7f97e1600640 1 -- 172.21.15.16:0/1431372184 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241772 (secure 0 0 0) 0x7f97e40cd6c0 con 0x7f97c4072610 2024-09-10T21:10:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:34.294 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:10:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.290+0000 7f97ec400640 1 -- 172.21.15.16:0/1431372184 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f97c4072610 msgr2=0x7f97c4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.290+0000 7f97ec400640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f97c4072610 0x7f97c4074ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97d8001a20 tx=0x7f97d8001090 comp rx=0 tx=0).stop 2024-09-10T21:10:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.290+0000 7f97ec400640 1 -- 172.21.15.16:0/1431372184 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97e4159c10 msgr2=0x7f97e406f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.290+0000 7f97ec400640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97e4159c10 0x7f97e406f250 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f97e409b860 tx=0x7f97d0011d10 comp rx=0 tx=0).stop 2024-09-10T21:10:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.290+0000 7f97ec400640 1 -- 172.21.15.16:0/1431372184 shutdown_connections 2024-09-10T21:10:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.290+0000 7f97ec400640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f97c4072610 0x7f97c4074ad0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.290+0000 7f97ec400640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e406f790 0x7f97e40ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.290+0000 7f97ec400640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97e4159c10 0x7f97e406f250 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.290+0000 7f97ec400640 1 --2- 172.21.15.16:0/1431372184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97cc004d50 0x7f97e4160d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.290+0000 7f97ec400640 1 -- 172.21.15.16:0/1431372184 >> 172.21.15.16:0/1431372184 conn(0x7f97e409b130 msgr2=0x7f97e4070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.290+0000 7f97ec400640 1 -- 172.21.15.16:0/1431372184 shutdown_connections 2024-09-10T21:10:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.290+0000 7f97ec400640 1 -- 172.21.15.16:0/1431372184 wait complete. 2024-09-10T21:10:34.312 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-10T21:10:33.457312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590492,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252336,"kb_used_data":2576,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722384,"statfs":{"total":773094113280,"available":772835721216,"internally_reserved":0,"allocated":2637824,"data_stored":1636509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":124,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.319533"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168511+0000","last_change":"2024-09-10T21:10:30.180534+0000","last_active":"2024-09-10T21:10:31.168511+0000","last_peered":"2024-09-10T21:10:31.168511+0000","last_clean":"2024-09-10T21:10:31.168511+0000","last_became_active":"2024-09-10T21:10:30.180196+0000","last_became_peered":"2024-09-10T21:10:30.180196+0000","last_unstale":"2024-09-10T21:10:31.168511+0000","last_undegraded":"2024-09-10T21:10:31.168511+0000","last_fullsized":"2024-09-10T21:10:31.168511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:04.038591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446465+0000","last_change":"2024-09-10T21:10:30.169268+0000","last_active":"2024-09-10T21:10:31.446465+0000","last_peered":"2024-09-10T21:10:31.446465+0000","last_clean":"2024-09-10T21:10:31.446465+0000","last_became_active":"2024-09-10T21:10:30.168869+0000","last_became_peered":"2024-09-10T21:10:30.168869+0000","last_unstale":"2024-09-10T21:10:31.446465+0000","last_undegraded":"2024-09-10T21:10:31.446465+0000","last_fullsized":"2024-09-10T21:10:31.446465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:06.070406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167488+0000","last_change":"2024-09-10T21:10:30.168344+0000","last_active":"2024-09-10T21:10:31.167488+0000","last_peered":"2024-09-10T21:10:31.167488+0000","last_clean":"2024-09-10T21:10:31.167488+0000","last_became_active":"2024-09-10T21:10:30.167966+0000","last_became_peered":"2024-09-10T21:10:30.167966+0000","last_unstale":"2024-09-10T21:10:31.167488+0000","last_undegraded":"2024-09-10T21:10:31.167488+0000","last_fullsized":"2024-09-10T21:10:31.167488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:38.100207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446249+0000","last_change":"2024-09-10T21:10:30.171833+0000","last_active":"2024-09-10T21:10:31.446249+0000","last_peered":"2024-09-10T21:10:31.446249+0000","last_clean":"2024-09-10T21:10:31.446249+0000","last_became_active":"2024-09-10T21:10:30.171461+0000","last_became_peered":"2024-09-10T21:10:30.171461+0000","last_unstale":"2024-09-10T21:10:31.446249+0000","last_undegraded":"2024-09-10T21:10:31.446249+0000","last_fullsized":"2024-09-10T21:10:31.446249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:32.018476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167929+0000","last_change":"2024-09-10T21:10:30.171980+0000","last_active":"2024-09-10T21:10:31.167929+0000","last_peered":"2024-09-10T21:10:31.167929+0000","last_clean":"2024-09-10T21:10:31.167929+0000","last_became_active":"2024-09-10T21:10:30.167136+0000","last_became_peered":"2024-09-10T21:10:30.167136+0000","last_unstale":"2024-09-10T21:10:31.167929+0000","last_undegraded":"2024-09-10T21:10:31.167929+0000","last_fullsized":"2024-09-10T21:10:31.167929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:33.142099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.144466+0000","last_change":"2024-09-10T21:10:28.144466+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.144466+0000","last_undegraded":"2024-09-10T21:10:28.144466+0000","last_fullsized":"2024-09-10T21:10:28.144466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.145128+0000","last_change":"2024-09-10T21:10:28.145128+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.145128+0000","last_undegraded":"2024-09-10T21:10:28.145128+0000","last_fullsized":"2024-09-10T21:10:28.145128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.145380+0000","last_change":"2024-09-10T21:10:28.145380+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.145380+0000","last_undegraded":"2024-09-10T21:10:28.145380+0000","last_fullsized":"2024-09-10T21:10:28.145380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.143282+0000","last_change":"2024-09-10T21:10:28.143282+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.143282+0000","last_undegraded":"2024-09-10T21:10:28.143282+0000","last_fullsized":"2024-09-10T21:10:28.143282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168376+0000","last_change":"2024-09-10T21:10:29.170975+0000","last_active":"2024-09-10T21:10:31.168376+0000","last_peered":"2024-09-10T21:10:31.168376+0000","last_clean":"2024-09-10T21:10:31.168376+0000","last_became_active":"2024-09-10T21:10:29.170623+0000","last_became_peered":"2024-09-10T21:10:29.170623+0000","last_unstale":"2024-09-10T21:10:31.168376+0000","last_undegraded":"2024-09-10T21:10:31.168376+0000","last_fullsized":"2024-09-10T21:10:31.168376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.148219+0000","last_change":"2024-09-10T21:10:28.148219+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.148219+0000","last_undegraded":"2024-09-10T21:10:28.148219+0000","last_fullsized":"2024-09-10T21:10:28.148219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446376+0000","last_change":"2024-09-10T21:10:29.171850+0000","last_active":"2024-09-10T21:10:31.446376+0000","last_peered":"2024-09-10T21:10:31.446376+0000","last_clean":"2024-09-10T21:10:31.446376+0000","last_became_active":"2024-09-10T21:10:29.171589+0000","last_became_peered":"2024-09-10T21:10:29.171589+0000","last_unstale":"2024-09-10T21:10:31.446376+0000","last_undegraded":"2024-09-10T21:10:31.446376+0000","last_fullsized":"2024-09-10T21:10:31.446376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:43.143812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.146602+0000","last_change":"2024-09-10T21:10:28.146602+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.146602+0000","last_undegraded":"2024-09-10T21:10:28.146602+0000","last_fullsized":"2024-09-10T21:10:28.146602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446251+0000","last_change":"2024-09-10T21:10:29.178312+0000","last_active":"2024-09-10T21:10:31.446251+0000","last_peered":"2024-09-10T21:10:31.446251+0000","last_clean":"2024-09-10T21:10:31.446251+0000","last_became_active":"2024-09-10T21:10:29.178044+0000","last_became_peered":"2024-09-10T21:10:29.178044+0000","last_unstale":"2024-09-10T21:10:31.446251+0000","last_undegraded":"2024-09-10T21:10:31.446251+0000","last_fullsized":"2024-09-10T21:10:31.446251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:41.871122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.143960+0000","last_change":"2024-09-10T21:10:28.143960+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.143960+0000","last_undegraded":"2024-09-10T21:10:28.143960+0000","last_fullsized":"2024-09-10T21:10:28.143960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.146446+0000","last_change":"2024-09-10T21:10:28.146446+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.146446+0000","last_undegraded":"2024-09-10T21:10:28.146446+0000","last_fullsized":"2024-09-10T21:10:28.146446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167103+0000","last_change":"2024-09-10T21:10:30.169299+0000","last_active":"2024-09-10T21:10:31.167103+0000","last_peered":"2024-09-10T21:10:31.167103+0000","last_clean":"2024-09-10T21:10:31.167103+0000","last_became_active":"2024-09-10T21:10:30.169013+0000","last_became_peered":"2024-09-10T21:10:30.169013+0000","last_unstale":"2024-09-10T21:10:31.167103+0000","last_undegraded":"2024-09-10T21:10:31.167103+0000","last_fullsized":"2024-09-10T21:10:31.167103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:40.546116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.138943+0000","last_change":"2024-09-10T21:10:28.138943+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.138943+0000","last_undegraded":"2024-09-10T21:10:28.138943+0000","last_fullsized":"2024-09-10T21:10:28.138943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.148023+0000","last_change":"2024-09-10T21:10:28.148023+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.148023+0000","last_undegraded":"2024-09-10T21:10:28.148023+0000","last_fullsized":"2024-09-10T21:10:28.148023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.139515+0000","last_change":"2024-09-10T21:10:28.139515+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.139515+0000","last_undegraded":"2024-09-10T21:10:28.139515+0000","last_fullsized":"2024-09-10T21:10:28.139515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.144525+0000","last_change":"2024-09-10T21:10:28.144525+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.144525+0000","last_undegraded":"2024-09-10T21:10:28.144525+0000","last_fullsized":"2024-09-10T21:10:28.144525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.145343+0000","last_change":"2024-09-10T21:10:28.145343+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.145343+0000","last_undegraded":"2024-09-10T21:10:28.145343+0000","last_fullsized":"2024-09-10T21:10:28.145343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167202+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:10:31.167202+0000","last_peered":"2024-09-10T21:10:31.167202+0000","last_clean":"2024-09-10T21:10:31.167202+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:10:31.167202+0000","last_undegraded":"2024-09-10T21:10:31.167202+0000","last_fullsized":"2024-09-10T21:10:31.167202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141728+0000","last_change":"2024-09-10T21:10:28.141728+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141728+0000","last_undegraded":"2024-09-10T21:10:28.141728+0000","last_fullsized":"2024-09-10T21:10:28.141728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141939+0000","last_change":"2024-09-10T21:10:28.141939+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141939+0000","last_undegraded":"2024-09-10T21:10:28.141939+0000","last_fullsized":"2024-09-10T21:10:28.141939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446253+0000","last_change":"2024-09-10T21:10:29.170636+0000","last_active":"2024-09-10T21:10:31.446253+0000","last_peered":"2024-09-10T21:10:31.446253+0000","last_clean":"2024-09-10T21:10:31.446253+0000","last_became_active":"2024-09-10T21:10:29.170342+0000","last_became_peered":"2024-09-10T21:10:29.170342+0000","last_unstale":"2024-09-10T21:10:31.446253+0000","last_undegraded":"2024-09-10T21:10:31.446253+0000","last_fullsized":"2024-09-10T21:10:31.446253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141731+0000","last_change":"2024-09-10T21:10:28.141731+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141731+0000","last_undegraded":"2024-09-10T21:10:28.141731+0000","last_fullsized":"2024-09-10T21:10:28.141731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.144620+0000","last_change":"2024-09-10T21:10:28.144620+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.144620+0000","last_undegraded":"2024-09-10T21:10:28.144620+0000","last_fullsized":"2024-09-10T21:10:28.144620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141746+0000","last_change":"2024-09-10T21:10:28.141746+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141746+0000","last_undegraded":"2024-09-10T21:10:28.141746+0000","last_fullsized":"2024-09-10T21:10:28.141746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.138748+0000","last_change":"2024-09-10T21:10:28.138748+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.138748+0000","last_undegraded":"2024-09-10T21:10:28.138748+0000","last_fullsized":"2024-09-10T21:10:28.138748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141755+0000","last_change":"2024-09-10T21:10:28.141755+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141755+0000","last_undegraded":"2024-09-10T21:10:28.141755+0000","last_fullsized":"2024-09-10T21:10:28.141755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.147248+0000","last_change":"2024-09-10T21:10:28.147248+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.147248+0000","last_undegraded":"2024-09-10T21:10:28.147248+0000","last_fullsized":"2024-09-10T21:10:28.147248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.138724+0000","last_change":"2024-09-10T21:10:28.138724+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.138724+0000","last_undegraded":"2024-09-10T21:10:28.138724+0000","last_fullsized":"2024-09-10T21:10:28.138724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.146663+0000","last_change":"2024-09-10T21:10:28.146663+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.146663+0000","last_undegraded":"2024-09-10T21:10:28.146663+0000","last_fullsized":"2024-09-10T21:10:28.146663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446431+0000","last_change":"2024-09-10T21:10:30.176046+0000","last_active":"2024-09-10T21:10:31.446431+0000","last_peered":"2024-09-10T21:10:31.446431+0000","last_clean":"2024-09-10T21:10:31.446431+0000","last_became_active":"2024-09-10T21:10:30.175784+0000","last_became_peered":"2024-09-10T21:10:30.175784+0000","last_unstale":"2024-09-10T21:10:31.446431+0000","last_undegraded":"2024-09-10T21:10:31.446431+0000","last_fullsized":"2024-09-10T21:10:31.446431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:37.972763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.143681+0000","last_change":"2024-09-10T21:10:28.143681+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.143681+0000","last_undegraded":"2024-09-10T21:10:28.143681+0000","last_fullsized":"2024-09-10T21:10:28.143681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.148838+0000","last_change":"2024-09-10T21:10:28.148838+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.148838+0000","last_undegraded":"2024-09-10T21:10:28.148838+0000","last_fullsized":"2024-09-10T21:10:28.148838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167690+0000","last_change":"2024-09-10T21:10:30.179424+0000","last_active":"2024-09-10T21:10:31.167690+0000","last_peered":"2024-09-10T21:10:31.167690+0000","last_clean":"2024-09-10T21:10:31.167690+0000","last_became_active":"2024-09-10T21:10:30.178557+0000","last_became_peered":"2024-09-10T21:10:30.178557+0000","last_unstale":"2024-09-10T21:10:31.167690+0000","last_undegraded":"2024-09-10T21:10:31.167690+0000","last_fullsized":"2024-09-10T21:10:31.167690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:14.118820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.147855+0000","last_change":"2024-09-10T21:10:28.147855+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.147855+0000","last_undegraded":"2024-09-10T21:10:28.147855+0000","last_fullsized":"2024-09-10T21:10:28.147855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.145534+0000","last_change":"2024-09-10T21:10:28.145534+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.145534+0000","last_undegraded":"2024-09-10T21:10:28.145534+0000","last_fullsized":"2024-09-10T21:10:28.145534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.138794+0000","last_change":"2024-09-10T21:10:28.138794+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.138794+0000","last_undegraded":"2024-09-10T21:10:28.138794+0000","last_fullsized":"2024-09-10T21:10:28.138794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":124,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":753664,"data_stored":637388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":806912,"data_stored":662625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:10:34.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:10:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.502+0000 7fc807800640 1 -- 172.21.15.16:0/234996982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8004da0 msgr2=0x7fc7e8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.502+0000 7fc807800640 1 --2- 172.21.15.16:0/234996982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8004da0 0x7fc7e8005200 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc800074ce0 tx=0x7fc7f801d590 comp rx=0 tx=0).stop 2024-09-10T21:10:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.502+0000 7fc805800640 1 -- 172.21.15.16:0/234996982 reap_dead start 2024-09-10T21:10:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.502+0000 7fc807800640 1 -- 172.21.15.16:0/234996982 shutdown_connections 2024-09-10T21:10:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.502+0000 7fc807800640 1 -- 172.21.15.16:0/234996982 >> 172.21.15.16:0/234996982 conn(0x7fc80009b130 msgr2=0x7fc8000d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.502+0000 7fc807800640 1 -- 172.21.15.16:0/234996982 shutdown_connections 2024-09-10T21:10:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.502+0000 7fc807800640 1 -- 172.21.15.16:0/234996982 wait complete. 2024-09-10T21:10:34.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc807800640 1 Processor -- start 2024-09-10T21:10:34.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc807800640 1 -- start start 2024-09-10T21:10:34.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc807800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8004da0 0x7fc8000cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:34.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc807800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc800154790 0x7fc8000c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:34.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc807800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8000c9390 0x7fc800140c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:34.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc807800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc800167bd0 con 0x7fc7e8004da0 2024-09-10T21:10:34.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc807800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8000b0760 con 0x7fc8000c9390 2024-09-10T21:10:34.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc807800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc800167d50 con 0x7fc800154790 2024-09-10T21:10:34.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc804e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8004da0 0x7fc8000cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:34.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc804e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8004da0 0x7fc8000cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38340/0 (socket says 172.21.15.16:38340) 2024-09-10T21:10:34.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc804e00640 1 -- 172.21.15.16:0/3169903268 learned_addr learned my addr 172.21.15.16:0/3169903268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:34.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc7ffe00640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc800154790 0x7fc8000c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:34.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc805800640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8000c9390 0x7fc800140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc804e00640 1 -- 172.21.15.16:0/3169903268 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc800154790 msgr2=0x7fc8000c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc804e00640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc800154790 0x7fc8000c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc804e00640 1 -- 172.21.15.16:0/3169903268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8000c9390 msgr2=0x7fc800140c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc804e00640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8000c9390 0x7fc800140c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc804e00640 1 -- 172.21.15.16:0/3169903268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8001536f0 con 0x7fc7e8004da0 2024-09-10T21:10:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc804e00640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8004da0 0x7fc8000cba70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4001380 tx=0x7fc7f400e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc7fcc00640 1 -- 172.21.15.16:0/3169903268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4015070 con 0x7fc7e8004da0 2024-09-10T21:10:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc7fcc00640 1 -- 172.21.15.16:0/3169903268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7f4010980 con 0x7fc7e8004da0 2024-09-10T21:10:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc7fcc00640 1 -- 172.21.15.16:0/3169903268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4010dd0 con 0x7fc7e8004da0 2024-09-10T21:10:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc807800640 1 -- 172.21.15.16:0/3169903268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8001539d0 con 0x7fc7e8004da0 2024-09-10T21:10:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc807800640 1 -- 172.21.15.16:0/3169903268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc800153f90 con 0x7fc7e8004da0 2024-09-10T21:10:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.506+0000 7fc807800640 1 -- 172.21.15.16:0/3169903268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8000af580 con 0x7fc7e8004da0 2024-09-10T21:10:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.510+0000 7fc7fcc00640 1 -- 172.21.15.16:0/3169903268 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc7f4011460 con 0x7fc7e8004da0 2024-09-10T21:10:34.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.510+0000 7fc7fcc00640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7e0072610 0x7fc7e0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:34.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.510+0000 7fc7fcc00640 1 -- 172.21.15.16:0/3169903268 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc7f40937c0 con 0x7fc7e8004da0 2024-09-10T21:10:34.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.510+0000 7fc7ffe00640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7e0072610 0x7fc7e0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:34.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.510+0000 7fc7ffe00640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7e0072610 0x7fc7e0074ad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc8001f9ed0 tx=0x7fc7ec006d70 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:34.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.514+0000 7fc7fcc00640 1 -- 172.21.15.16:0/3169903268 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc7f405e840 con 0x7fc7e8004da0 2024-09-10T21:10:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.686+0000 7fc807800640 1 -- 172.21.15.16:0/3169903268 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc800074ae0 con 0x7fc7e0072610 2024-09-10T21:10:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.694+0000 7fc7fcc00640 1 -- 172.21.15.16:0/3169903268 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241772 (secure 0 0 0) 0x7fc800074ae0 con 0x7fc7e0072610 2024-09-10T21:10:34.700 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:10:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.694+0000 7fc807800640 1 -- 172.21.15.16:0/3169903268 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7e0072610 msgr2=0x7fc7e0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.694+0000 7fc807800640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7e0072610 0x7fc7e0074ad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc8001f9ed0 tx=0x7fc7ec006d70 comp rx=0 tx=0).stop 2024-09-10T21:10:34.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.694+0000 7fc807800640 1 -- 172.21.15.16:0/3169903268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8004da0 msgr2=0x7fc8000cba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:34.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.694+0000 7fc807800640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8004da0 0x7fc8000cba70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4001380 tx=0x7fc7f400e9a0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.694+0000 7fc807800640 1 -- 172.21.15.16:0/3169903268 shutdown_connections 2024-09-10T21:10:34.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.694+0000 7fc807800640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7e0072610 0x7fc7e0074ad0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.694+0000 7fc807800640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8000c9390 0x7fc800140c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.694+0000 7fc807800640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc800154790 0x7fc8000c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.694+0000 7fc807800640 1 --2- 172.21.15.16:0/3169903268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8004da0 0x7fc8000cba70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.694+0000 7fc807800640 1 -- 172.21.15.16:0/3169903268 >> 172.21.15.16:0/3169903268 conn(0x7fc80009b130 msgr2=0x7fc800158dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:34.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.694+0000 7fc807800640 1 -- 172.21.15.16:0/3169903268 shutdown_connections 2024-09-10T21:10:34.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.694+0000 7fc807800640 1 -- 172.21.15.16:0/3169903268 wait complete. 2024-09-10T21:10:34.718 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-10T21:10:33.457312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590492,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252336,"kb_used_data":2576,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722384,"statfs":{"total":773094113280,"available":772835721216,"internally_reserved":0,"allocated":2637824,"data_stored":1636509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":124,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.319533"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168511+0000","last_change":"2024-09-10T21:10:30.180534+0000","last_active":"2024-09-10T21:10:31.168511+0000","last_peered":"2024-09-10T21:10:31.168511+0000","last_clean":"2024-09-10T21:10:31.168511+0000","last_became_active":"2024-09-10T21:10:30.180196+0000","last_became_peered":"2024-09-10T21:10:30.180196+0000","last_unstale":"2024-09-10T21:10:31.168511+0000","last_undegraded":"2024-09-10T21:10:31.168511+0000","last_fullsized":"2024-09-10T21:10:31.168511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:04.038591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446465+0000","last_change":"2024-09-10T21:10:30.169268+0000","last_active":"2024-09-10T21:10:31.446465+0000","last_peered":"2024-09-10T21:10:31.446465+0000","last_clean":"2024-09-10T21:10:31.446465+0000","last_became_active":"2024-09-10T21:10:30.168869+0000","last_became_peered":"2024-09-10T21:10:30.168869+0000","last_unstale":"2024-09-10T21:10:31.446465+0000","last_undegraded":"2024-09-10T21:10:31.446465+0000","last_fullsized":"2024-09-10T21:10:31.446465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:06.070406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167488+0000","last_change":"2024-09-10T21:10:30.168344+0000","last_active":"2024-09-10T21:10:31.167488+0000","last_peered":"2024-09-10T21:10:31.167488+0000","last_clean":"2024-09-10T21:10:31.167488+0000","last_became_active":"2024-09-10T21:10:30.167966+0000","last_became_peered":"2024-09-10T21:10:30.167966+0000","last_unstale":"2024-09-10T21:10:31.167488+0000","last_undegraded":"2024-09-10T21:10:31.167488+0000","last_fullsized":"2024-09-10T21:10:31.167488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:38.100207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446249+0000","last_change":"2024-09-10T21:10:30.171833+0000","last_active":"2024-09-10T21:10:31.446249+0000","last_peered":"2024-09-10T21:10:31.446249+0000","last_clean":"2024-09-10T21:10:31.446249+0000","last_became_active":"2024-09-10T21:10:30.171461+0000","last_became_peered":"2024-09-10T21:10:30.171461+0000","last_unstale":"2024-09-10T21:10:31.446249+0000","last_undegraded":"2024-09-10T21:10:31.446249+0000","last_fullsized":"2024-09-10T21:10:31.446249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:32.018476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167929+0000","last_change":"2024-09-10T21:10:30.171980+0000","last_active":"2024-09-10T21:10:31.167929+0000","last_peered":"2024-09-10T21:10:31.167929+0000","last_clean":"2024-09-10T21:10:31.167929+0000","last_became_active":"2024-09-10T21:10:30.167136+0000","last_became_peered":"2024-09-10T21:10:30.167136+0000","last_unstale":"2024-09-10T21:10:31.167929+0000","last_undegraded":"2024-09-10T21:10:31.167929+0000","last_fullsized":"2024-09-10T21:10:31.167929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:33.142099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.144466+0000","last_change":"2024-09-10T21:10:28.144466+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.144466+0000","last_undegraded":"2024-09-10T21:10:28.144466+0000","last_fullsized":"2024-09-10T21:10:28.144466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.145128+0000","last_change":"2024-09-10T21:10:28.145128+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.145128+0000","last_undegraded":"2024-09-10T21:10:28.145128+0000","last_fullsized":"2024-09-10T21:10:28.145128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.145380+0000","last_change":"2024-09-10T21:10:28.145380+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.145380+0000","last_undegraded":"2024-09-10T21:10:28.145380+0000","last_fullsized":"2024-09-10T21:10:28.145380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.143282+0000","last_change":"2024-09-10T21:10:28.143282+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.143282+0000","last_undegraded":"2024-09-10T21:10:28.143282+0000","last_fullsized":"2024-09-10T21:10:28.143282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168376+0000","last_change":"2024-09-10T21:10:29.170975+0000","last_active":"2024-09-10T21:10:31.168376+0000","last_peered":"2024-09-10T21:10:31.168376+0000","last_clean":"2024-09-10T21:10:31.168376+0000","last_became_active":"2024-09-10T21:10:29.170623+0000","last_became_peered":"2024-09-10T21:10:29.170623+0000","last_unstale":"2024-09-10T21:10:31.168376+0000","last_undegraded":"2024-09-10T21:10:31.168376+0000","last_fullsized":"2024-09-10T21:10:31.168376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.148219+0000","last_change":"2024-09-10T21:10:28.148219+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.148219+0000","last_undegraded":"2024-09-10T21:10:28.148219+0000","last_fullsized":"2024-09-10T21:10:28.148219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446376+0000","last_change":"2024-09-10T21:10:29.171850+0000","last_active":"2024-09-10T21:10:31.446376+0000","last_peered":"2024-09-10T21:10:31.446376+0000","last_clean":"2024-09-10T21:10:31.446376+0000","last_became_active":"2024-09-10T21:10:29.171589+0000","last_became_peered":"2024-09-10T21:10:29.171589+0000","last_unstale":"2024-09-10T21:10:31.446376+0000","last_undegraded":"2024-09-10T21:10:31.446376+0000","last_fullsized":"2024-09-10T21:10:31.446376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:43.143812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.146602+0000","last_change":"2024-09-10T21:10:28.146602+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.146602+0000","last_undegraded":"2024-09-10T21:10:28.146602+0000","last_fullsized":"2024-09-10T21:10:28.146602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446251+0000","last_change":"2024-09-10T21:10:29.178312+0000","last_active":"2024-09-10T21:10:31.446251+0000","last_peered":"2024-09-10T21:10:31.446251+0000","last_clean":"2024-09-10T21:10:31.446251+0000","last_became_active":"2024-09-10T21:10:29.178044+0000","last_became_peered":"2024-09-10T21:10:29.178044+0000","last_unstale":"2024-09-10T21:10:31.446251+0000","last_undegraded":"2024-09-10T21:10:31.446251+0000","last_fullsized":"2024-09-10T21:10:31.446251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:41.871122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.143960+0000","last_change":"2024-09-10T21:10:28.143960+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.143960+0000","last_undegraded":"2024-09-10T21:10:28.143960+0000","last_fullsized":"2024-09-10T21:10:28.143960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.146446+0000","last_change":"2024-09-10T21:10:28.146446+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.146446+0000","last_undegraded":"2024-09-10T21:10:28.146446+0000","last_fullsized":"2024-09-10T21:10:28.146446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167103+0000","last_change":"2024-09-10T21:10:30.169299+0000","last_active":"2024-09-10T21:10:31.167103+0000","last_peered":"2024-09-10T21:10:31.167103+0000","last_clean":"2024-09-10T21:10:31.167103+0000","last_became_active":"2024-09-10T21:10:30.169013+0000","last_became_peered":"2024-09-10T21:10:30.169013+0000","last_unstale":"2024-09-10T21:10:31.167103+0000","last_undegraded":"2024-09-10T21:10:31.167103+0000","last_fullsized":"2024-09-10T21:10:31.167103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:40.546116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.138943+0000","last_change":"2024-09-10T21:10:28.138943+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.138943+0000","last_undegraded":"2024-09-10T21:10:28.138943+0000","last_fullsized":"2024-09-10T21:10:28.138943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.148023+0000","last_change":"2024-09-10T21:10:28.148023+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.148023+0000","last_undegraded":"2024-09-10T21:10:28.148023+0000","last_fullsized":"2024-09-10T21:10:28.148023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.139515+0000","last_change":"2024-09-10T21:10:28.139515+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.139515+0000","last_undegraded":"2024-09-10T21:10:28.139515+0000","last_fullsized":"2024-09-10T21:10:28.139515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.144525+0000","last_change":"2024-09-10T21:10:28.144525+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.144525+0000","last_undegraded":"2024-09-10T21:10:28.144525+0000","last_fullsized":"2024-09-10T21:10:28.144525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.145343+0000","last_change":"2024-09-10T21:10:28.145343+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.145343+0000","last_undegraded":"2024-09-10T21:10:28.145343+0000","last_fullsized":"2024-09-10T21:10:28.145343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167202+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:10:31.167202+0000","last_peered":"2024-09-10T21:10:31.167202+0000","last_clean":"2024-09-10T21:10:31.167202+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:10:31.167202+0000","last_undegraded":"2024-09-10T21:10:31.167202+0000","last_fullsized":"2024-09-10T21:10:31.167202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141728+0000","last_change":"2024-09-10T21:10:28.141728+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141728+0000","last_undegraded":"2024-09-10T21:10:28.141728+0000","last_fullsized":"2024-09-10T21:10:28.141728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141939+0000","last_change":"2024-09-10T21:10:28.141939+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141939+0000","last_undegraded":"2024-09-10T21:10:28.141939+0000","last_fullsized":"2024-09-10T21:10:28.141939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446253+0000","last_change":"2024-09-10T21:10:29.170636+0000","last_active":"2024-09-10T21:10:31.446253+0000","last_peered":"2024-09-10T21:10:31.446253+0000","last_clean":"2024-09-10T21:10:31.446253+0000","last_became_active":"2024-09-10T21:10:29.170342+0000","last_became_peered":"2024-09-10T21:10:29.170342+0000","last_unstale":"2024-09-10T21:10:31.446253+0000","last_undegraded":"2024-09-10T21:10:31.446253+0000","last_fullsized":"2024-09-10T21:10:31.446253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141731+0000","last_change":"2024-09-10T21:10:28.141731+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141731+0000","last_undegraded":"2024-09-10T21:10:28.141731+0000","last_fullsized":"2024-09-10T21:10:28.141731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.144620+0000","last_change":"2024-09-10T21:10:28.144620+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.144620+0000","last_undegraded":"2024-09-10T21:10:28.144620+0000","last_fullsized":"2024-09-10T21:10:28.144620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141746+0000","last_change":"2024-09-10T21:10:28.141746+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141746+0000","last_undegraded":"2024-09-10T21:10:28.141746+0000","last_fullsized":"2024-09-10T21:10:28.141746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.138748+0000","last_change":"2024-09-10T21:10:28.138748+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.138748+0000","last_undegraded":"2024-09-10T21:10:28.138748+0000","last_fullsized":"2024-09-10T21:10:28.138748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141755+0000","last_change":"2024-09-10T21:10:28.141755+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141755+0000","last_undegraded":"2024-09-10T21:10:28.141755+0000","last_fullsized":"2024-09-10T21:10:28.141755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.147248+0000","last_change":"2024-09-10T21:10:28.147248+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.147248+0000","last_undegraded":"2024-09-10T21:10:28.147248+0000","last_fullsized":"2024-09-10T21:10:28.147248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.138724+0000","last_change":"2024-09-10T21:10:28.138724+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.138724+0000","last_undegraded":"2024-09-10T21:10:28.138724+0000","last_fullsized":"2024-09-10T21:10:28.138724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.146663+0000","last_change":"2024-09-10T21:10:28.146663+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.146663+0000","last_undegraded":"2024-09-10T21:10:28.146663+0000","last_fullsized":"2024-09-10T21:10:28.146663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446431+0000","last_change":"2024-09-10T21:10:30.176046+0000","last_active":"2024-09-10T21:10:31.446431+0000","last_peered":"2024-09-10T21:10:31.446431+0000","last_clean":"2024-09-10T21:10:31.446431+0000","last_became_active":"2024-09-10T21:10:30.175784+0000","last_became_peered":"2024-09-10T21:10:30.175784+0000","last_unstale":"2024-09-10T21:10:31.446431+0000","last_undegraded":"2024-09-10T21:10:31.446431+0000","last_fullsized":"2024-09-10T21:10:31.446431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:37.972763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.143681+0000","last_change":"2024-09-10T21:10:28.143681+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.143681+0000","last_undegraded":"2024-09-10T21:10:28.143681+0000","last_fullsized":"2024-09-10T21:10:28.143681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.148838+0000","last_change":"2024-09-10T21:10:28.148838+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.148838+0000","last_undegraded":"2024-09-10T21:10:28.148838+0000","last_fullsized":"2024-09-10T21:10:28.148838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167690+0000","last_change":"2024-09-10T21:10:30.179424+0000","last_active":"2024-09-10T21:10:31.167690+0000","last_peered":"2024-09-10T21:10:31.167690+0000","last_clean":"2024-09-10T21:10:31.167690+0000","last_became_active":"2024-09-10T21:10:30.178557+0000","last_became_peered":"2024-09-10T21:10:30.178557+0000","last_unstale":"2024-09-10T21:10:31.167690+0000","last_undegraded":"2024-09-10T21:10:31.167690+0000","last_fullsized":"2024-09-10T21:10:31.167690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:14.118820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.147855+0000","last_change":"2024-09-10T21:10:28.147855+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.147855+0000","last_undegraded":"2024-09-10T21:10:28.147855+0000","last_fullsized":"2024-09-10T21:10:28.147855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.145534+0000","last_change":"2024-09-10T21:10:28.145534+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.145534+0000","last_undegraded":"2024-09-10T21:10:28.145534+0000","last_fullsized":"2024-09-10T21:10:28.145534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.138794+0000","last_change":"2024-09-10T21:10:28.138794+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.138794+0000","last_undegraded":"2024-09-10T21:10:28.138794+0000","last_fullsized":"2024-09-10T21:10:28.138794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":124,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":753664,"data_stored":637388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":806912,"data_stored":662625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:10:34.722 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:10:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 -- 172.21.15.16:0/2752482329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1c0d06d0 msgr2=0x7fbf1c0aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 --2- 172.21.15.16:0/2752482329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1c0d06d0 0x7fbf1c0aed30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009c80 tx=0x7fbf1401cce0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 -- 172.21.15.16:0/2752482329 shutdown_connections 2024-09-10T21:10:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 --2- 172.21.15.16:0/2752482329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1c0d06d0 0x7fbf1c0aed30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 --2- 172.21.15.16:0/2752482329 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf1c15f350 0x7fbf1c0cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 -- 172.21.15.16:0/2752482329 >> 172.21.15.16:0/2752482329 conn(0x7fbf1c0d1d80 msgr2=0x7fbf1c0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 -- 172.21.15.16:0/2752482329 shutdown_connections 2024-09-10T21:10:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 -- 172.21.15.16:0/2752482329 wait complete. 2024-09-10T21:10:34.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 Processor -- start 2024-09-10T21:10:34.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 -- start start 2024-09-10T21:10:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1c0cfb60 0x7fbf1c1f9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf1c0d06d0 0x7fbf1c0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf1c15f350 0x7fbf1c071630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c168c50 con 0x7fbf1c0cfb60 2024-09-10T21:10:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c168ad0 con 0x7fbf1c15f350 2024-09-10T21:10:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf22e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1c168dd0 con 0x7fbf1c0d06d0 2024-09-10T21:10:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1c0cfb60 0x7fbf1c1f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1c0cfb60 0x7fbf1c1f9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38366/0 (socket says 172.21.15.16:38366) 2024-09-10T21:10:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf1be00640 1 -- 172.21.15.16:0/1516339537 learned_addr learned my addr 172.21.15.16:0/1516339537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf13e00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf1c0d06d0 0x7fbf1c0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.906+0000 7fbf20e00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf1c15f350 0x7fbf1c071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.910+0000 7fbf1be00640 1 -- 172.21.15.16:0/1516339537 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf1c0d06d0 msgr2=0x7fbf1c0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.910+0000 7fbf1be00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf1c0d06d0 0x7fbf1c0cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.910+0000 7fbf1be00640 1 -- 172.21.15.16:0/1516339537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf1c15f350 msgr2=0x7fbf1c071630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.910+0000 7fbf1be00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf1c15f350 0x7fbf1c071630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.910+0000 7fbf1be00640 1 -- 172.21.15.16:0/1516339537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1c0c91a0 con 0x7fbf1c0cfb60 2024-09-10T21:10:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.910+0000 7fbf1be00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1c0cfb60 0x7fbf1c1f9610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00b410 tx=0x7fbf0c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.910+0000 7fbf18c00640 1 -- 172.21.15.16:0/1516339537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c015020 con 0x7fbf1c0cfb60 2024-09-10T21:10:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.910+0000 7fbf18c00640 1 -- 172.21.15.16:0/1516339537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf0c011920 con 0x7fbf1c0cfb60 2024-09-10T21:10:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.910+0000 7fbf18c00640 1 -- 172.21.15.16:0/1516339537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c011d70 con 0x7fbf1c0cfb60 2024-09-10T21:10:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.910+0000 7fbf22e00640 1 -- 172.21.15.16:0/1516339537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1c0c9480 con 0x7fbf1c0cfb60 2024-09-10T21:10:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.910+0000 7fbf22e00640 1 -- 172.21.15.16:0/1516339537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1c0c9d20 con 0x7fbf1c0cfb60 2024-09-10T21:10:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.910+0000 7fbf22e00640 1 -- 172.21.15.16:0/1516339537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf1c0c9f20 con 0x7fbf1c0cfb60 2024-09-10T21:10:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.910+0000 7fbf18c00640 1 -- 172.21.15.16:0/1516339537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbf0c01a400 con 0x7fbf1c0cfb60 2024-09-10T21:10:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.914+0000 7fbf18c00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbef80725e0 0x7fbef8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.914+0000 7fbf18c00640 1 -- 172.21.15.16:0/1516339537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbf0c093510 con 0x7fbf1c0cfb60 2024-09-10T21:10:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.914+0000 7fbf13e00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbef80725e0 0x7fbef8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.914+0000 7fbf13e00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbef80725e0 0x7fbef8074aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0d0150 tx=0x7fbf04009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:34.918+0000 7fbf18c00640 1 -- 172.21.15.16:0/1516339537 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbf1c0c9f20 con 0x7fbf1c0cfb60 2024-09-10T21:10:35.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.086+0000 7fbf22e00640 1 -- 172.21.15.16:0/1516339537 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf1c0c9650 con 0x7fbef80725e0 2024-09-10T21:10:35.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.090+0000 7fbf18c00640 1 -- 172.21.15.16:0/1516339537 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241772 (secure 0 0 0) 0x7fbf1c0c9650 con 0x7fbef80725e0 2024-09-10T21:10:35.097 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:35.098 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:10:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.094+0000 7fbf22e00640 1 -- 172.21.15.16:0/1516339537 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbef80725e0 msgr2=0x7fbef8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.094+0000 7fbf22e00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbef80725e0 0x7fbef8074aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0d0150 tx=0x7fbf04009040 comp rx=0 tx=0).stop 2024-09-10T21:10:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.094+0000 7fbf22e00640 1 -- 172.21.15.16:0/1516339537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1c0cfb60 msgr2=0x7fbf1c1f9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.094+0000 7fbf22e00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1c0cfb60 0x7fbf1c1f9610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00b410 tx=0x7fbf0c00b8d0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.094+0000 7fbf22e00640 1 -- 172.21.15.16:0/1516339537 shutdown_connections 2024-09-10T21:10:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.094+0000 7fbf22e00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbef80725e0 0x7fbef8074aa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.094+0000 7fbf22e00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf1c15f350 0x7fbf1c071630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.094+0000 7fbf22e00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf1c0d06d0 0x7fbf1c0cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.094+0000 7fbf22e00640 1 --2- 172.21.15.16:0/1516339537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1c0cfb60 0x7fbf1c1f9610 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.094+0000 7fbf22e00640 1 -- 172.21.15.16:0/1516339537 >> 172.21.15.16:0/1516339537 conn(0x7fbf1c0d1d80 msgr2=0x7fbf1c0c85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.094+0000 7fbf22e00640 1 -- 172.21.15.16:0/1516339537 shutdown_connections 2024-09-10T21:10:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.094+0000 7fbf22e00640 1 -- 172.21.15.16:0/1516339537 wait complete. 2024-09-10T21:10:35.118 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-10T21:10:33.457312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590492,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252336,"kb_used_data":2576,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722384,"statfs":{"total":773094113280,"available":772835721216,"internally_reserved":0,"allocated":2637824,"data_stored":1636509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":124,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.319533"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168511+0000","last_change":"2024-09-10T21:10:30.180534+0000","last_active":"2024-09-10T21:10:31.168511+0000","last_peered":"2024-09-10T21:10:31.168511+0000","last_clean":"2024-09-10T21:10:31.168511+0000","last_became_active":"2024-09-10T21:10:30.180196+0000","last_became_peered":"2024-09-10T21:10:30.180196+0000","last_unstale":"2024-09-10T21:10:31.168511+0000","last_undegraded":"2024-09-10T21:10:31.168511+0000","last_fullsized":"2024-09-10T21:10:31.168511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:04.038591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446465+0000","last_change":"2024-09-10T21:10:30.169268+0000","last_active":"2024-09-10T21:10:31.446465+0000","last_peered":"2024-09-10T21:10:31.446465+0000","last_clean":"2024-09-10T21:10:31.446465+0000","last_became_active":"2024-09-10T21:10:30.168869+0000","last_became_peered":"2024-09-10T21:10:30.168869+0000","last_unstale":"2024-09-10T21:10:31.446465+0000","last_undegraded":"2024-09-10T21:10:31.446465+0000","last_fullsized":"2024-09-10T21:10:31.446465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:06.070406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167488+0000","last_change":"2024-09-10T21:10:30.168344+0000","last_active":"2024-09-10T21:10:31.167488+0000","last_peered":"2024-09-10T21:10:31.167488+0000","last_clean":"2024-09-10T21:10:31.167488+0000","last_became_active":"2024-09-10T21:10:30.167966+0000","last_became_peered":"2024-09-10T21:10:30.167966+0000","last_unstale":"2024-09-10T21:10:31.167488+0000","last_undegraded":"2024-09-10T21:10:31.167488+0000","last_fullsized":"2024-09-10T21:10:31.167488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:38.100207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446249+0000","last_change":"2024-09-10T21:10:30.171833+0000","last_active":"2024-09-10T21:10:31.446249+0000","last_peered":"2024-09-10T21:10:31.446249+0000","last_clean":"2024-09-10T21:10:31.446249+0000","last_became_active":"2024-09-10T21:10:30.171461+0000","last_became_peered":"2024-09-10T21:10:30.171461+0000","last_unstale":"2024-09-10T21:10:31.446249+0000","last_undegraded":"2024-09-10T21:10:31.446249+0000","last_fullsized":"2024-09-10T21:10:31.446249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:32.018476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167929+0000","last_change":"2024-09-10T21:10:30.171980+0000","last_active":"2024-09-10T21:10:31.167929+0000","last_peered":"2024-09-10T21:10:31.167929+0000","last_clean":"2024-09-10T21:10:31.167929+0000","last_became_active":"2024-09-10T21:10:30.167136+0000","last_became_peered":"2024-09-10T21:10:30.167136+0000","last_unstale":"2024-09-10T21:10:31.167929+0000","last_undegraded":"2024-09-10T21:10:31.167929+0000","last_fullsized":"2024-09-10T21:10:31.167929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:33.142099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.144466+0000","last_change":"2024-09-10T21:10:28.144466+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.144466+0000","last_undegraded":"2024-09-10T21:10:28.144466+0000","last_fullsized":"2024-09-10T21:10:28.144466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.145128+0000","last_change":"2024-09-10T21:10:28.145128+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.145128+0000","last_undegraded":"2024-09-10T21:10:28.145128+0000","last_fullsized":"2024-09-10T21:10:28.145128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.145380+0000","last_change":"2024-09-10T21:10:28.145380+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.145380+0000","last_undegraded":"2024-09-10T21:10:28.145380+0000","last_fullsized":"2024-09-10T21:10:28.145380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.143282+0000","last_change":"2024-09-10T21:10:28.143282+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.143282+0000","last_undegraded":"2024-09-10T21:10:28.143282+0000","last_fullsized":"2024-09-10T21:10:28.143282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168376+0000","last_change":"2024-09-10T21:10:29.170975+0000","last_active":"2024-09-10T21:10:31.168376+0000","last_peered":"2024-09-10T21:10:31.168376+0000","last_clean":"2024-09-10T21:10:31.168376+0000","last_became_active":"2024-09-10T21:10:29.170623+0000","last_became_peered":"2024-09-10T21:10:29.170623+0000","last_unstale":"2024-09-10T21:10:31.168376+0000","last_undegraded":"2024-09-10T21:10:31.168376+0000","last_fullsized":"2024-09-10T21:10:31.168376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.148219+0000","last_change":"2024-09-10T21:10:28.148219+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.148219+0000","last_undegraded":"2024-09-10T21:10:28.148219+0000","last_fullsized":"2024-09-10T21:10:28.148219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446376+0000","last_change":"2024-09-10T21:10:29.171850+0000","last_active":"2024-09-10T21:10:31.446376+0000","last_peered":"2024-09-10T21:10:31.446376+0000","last_clean":"2024-09-10T21:10:31.446376+0000","last_became_active":"2024-09-10T21:10:29.171589+0000","last_became_peered":"2024-09-10T21:10:29.171589+0000","last_unstale":"2024-09-10T21:10:31.446376+0000","last_undegraded":"2024-09-10T21:10:31.446376+0000","last_fullsized":"2024-09-10T21:10:31.446376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:43.143812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.146602+0000","last_change":"2024-09-10T21:10:28.146602+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.146602+0000","last_undegraded":"2024-09-10T21:10:28.146602+0000","last_fullsized":"2024-09-10T21:10:28.146602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446251+0000","last_change":"2024-09-10T21:10:29.178312+0000","last_active":"2024-09-10T21:10:31.446251+0000","last_peered":"2024-09-10T21:10:31.446251+0000","last_clean":"2024-09-10T21:10:31.446251+0000","last_became_active":"2024-09-10T21:10:29.178044+0000","last_became_peered":"2024-09-10T21:10:29.178044+0000","last_unstale":"2024-09-10T21:10:31.446251+0000","last_undegraded":"2024-09-10T21:10:31.446251+0000","last_fullsized":"2024-09-10T21:10:31.446251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:41.871122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.143960+0000","last_change":"2024-09-10T21:10:28.143960+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.143960+0000","last_undegraded":"2024-09-10T21:10:28.143960+0000","last_fullsized":"2024-09-10T21:10:28.143960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.146446+0000","last_change":"2024-09-10T21:10:28.146446+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.146446+0000","last_undegraded":"2024-09-10T21:10:28.146446+0000","last_fullsized":"2024-09-10T21:10:28.146446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167103+0000","last_change":"2024-09-10T21:10:30.169299+0000","last_active":"2024-09-10T21:10:31.167103+0000","last_peered":"2024-09-10T21:10:31.167103+0000","last_clean":"2024-09-10T21:10:31.167103+0000","last_became_active":"2024-09-10T21:10:30.169013+0000","last_became_peered":"2024-09-10T21:10:30.169013+0000","last_unstale":"2024-09-10T21:10:31.167103+0000","last_undegraded":"2024-09-10T21:10:31.167103+0000","last_fullsized":"2024-09-10T21:10:31.167103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:40.546116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.138943+0000","last_change":"2024-09-10T21:10:28.138943+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.138943+0000","last_undegraded":"2024-09-10T21:10:28.138943+0000","last_fullsized":"2024-09-10T21:10:28.138943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.148023+0000","last_change":"2024-09-10T21:10:28.148023+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.148023+0000","last_undegraded":"2024-09-10T21:10:28.148023+0000","last_fullsized":"2024-09-10T21:10:28.148023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.139515+0000","last_change":"2024-09-10T21:10:28.139515+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.139515+0000","last_undegraded":"2024-09-10T21:10:28.139515+0000","last_fullsized":"2024-09-10T21:10:28.139515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.144525+0000","last_change":"2024-09-10T21:10:28.144525+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.144525+0000","last_undegraded":"2024-09-10T21:10:28.144525+0000","last_fullsized":"2024-09-10T21:10:28.144525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.145343+0000","last_change":"2024-09-10T21:10:28.145343+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.145343+0000","last_undegraded":"2024-09-10T21:10:28.145343+0000","last_fullsized":"2024-09-10T21:10:28.145343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167202+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:10:31.167202+0000","last_peered":"2024-09-10T21:10:31.167202+0000","last_clean":"2024-09-10T21:10:31.167202+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:10:31.167202+0000","last_undegraded":"2024-09-10T21:10:31.167202+0000","last_fullsized":"2024-09-10T21:10:31.167202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141728+0000","last_change":"2024-09-10T21:10:28.141728+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141728+0000","last_undegraded":"2024-09-10T21:10:28.141728+0000","last_fullsized":"2024-09-10T21:10:28.141728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141939+0000","last_change":"2024-09-10T21:10:28.141939+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141939+0000","last_undegraded":"2024-09-10T21:10:28.141939+0000","last_fullsized":"2024-09-10T21:10:28.141939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446253+0000","last_change":"2024-09-10T21:10:29.170636+0000","last_active":"2024-09-10T21:10:31.446253+0000","last_peered":"2024-09-10T21:10:31.446253+0000","last_clean":"2024-09-10T21:10:31.446253+0000","last_became_active":"2024-09-10T21:10:29.170342+0000","last_became_peered":"2024-09-10T21:10:29.170342+0000","last_unstale":"2024-09-10T21:10:31.446253+0000","last_undegraded":"2024-09-10T21:10:31.446253+0000","last_fullsized":"2024-09-10T21:10:31.446253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141731+0000","last_change":"2024-09-10T21:10:28.141731+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141731+0000","last_undegraded":"2024-09-10T21:10:28.141731+0000","last_fullsized":"2024-09-10T21:10:28.141731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.144620+0000","last_change":"2024-09-10T21:10:28.144620+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.144620+0000","last_undegraded":"2024-09-10T21:10:28.144620+0000","last_fullsized":"2024-09-10T21:10:28.144620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141746+0000","last_change":"2024-09-10T21:10:28.141746+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141746+0000","last_undegraded":"2024-09-10T21:10:28.141746+0000","last_fullsized":"2024-09-10T21:10:28.141746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.138748+0000","last_change":"2024-09-10T21:10:28.138748+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.138748+0000","last_undegraded":"2024-09-10T21:10:28.138748+0000","last_fullsized":"2024-09-10T21:10:28.138748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.141755+0000","last_change":"2024-09-10T21:10:28.141755+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.141755+0000","last_undegraded":"2024-09-10T21:10:28.141755+0000","last_fullsized":"2024-09-10T21:10:28.141755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.147248+0000","last_change":"2024-09-10T21:10:28.147248+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.147248+0000","last_undegraded":"2024-09-10T21:10:28.147248+0000","last_fullsized":"2024-09-10T21:10:28.147248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.138724+0000","last_change":"2024-09-10T21:10:28.138724+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.138724+0000","last_undegraded":"2024-09-10T21:10:28.138724+0000","last_fullsized":"2024-09-10T21:10:28.138724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.146663+0000","last_change":"2024-09-10T21:10:28.146663+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.146663+0000","last_undegraded":"2024-09-10T21:10:28.146663+0000","last_fullsized":"2024-09-10T21:10:28.146663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446431+0000","last_change":"2024-09-10T21:10:30.176046+0000","last_active":"2024-09-10T21:10:31.446431+0000","last_peered":"2024-09-10T21:10:31.446431+0000","last_clean":"2024-09-10T21:10:31.446431+0000","last_became_active":"2024-09-10T21:10:30.175784+0000","last_became_peered":"2024-09-10T21:10:30.175784+0000","last_unstale":"2024-09-10T21:10:31.446431+0000","last_undegraded":"2024-09-10T21:10:31.446431+0000","last_fullsized":"2024-09-10T21:10:31.446431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:37.972763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.143681+0000","last_change":"2024-09-10T21:10:28.143681+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.143681+0000","last_undegraded":"2024-09-10T21:10:28.143681+0000","last_fullsized":"2024-09-10T21:10:28.143681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.148838+0000","last_change":"2024-09-10T21:10:28.148838+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.148838+0000","last_undegraded":"2024-09-10T21:10:28.148838+0000","last_fullsized":"2024-09-10T21:10:28.148838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167690+0000","last_change":"2024-09-10T21:10:30.179424+0000","last_active":"2024-09-10T21:10:31.167690+0000","last_peered":"2024-09-10T21:10:31.167690+0000","last_clean":"2024-09-10T21:10:31.167690+0000","last_became_active":"2024-09-10T21:10:30.178557+0000","last_became_peered":"2024-09-10T21:10:30.178557+0000","last_unstale":"2024-09-10T21:10:31.167690+0000","last_undegraded":"2024-09-10T21:10:31.167690+0000","last_fullsized":"2024-09-10T21:10:31.167690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:14.118820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.147855+0000","last_change":"2024-09-10T21:10:28.147855+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.147855+0000","last_undegraded":"2024-09-10T21:10:28.147855+0000","last_fullsized":"2024-09-10T21:10:28.147855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.145534+0000","last_change":"2024-09-10T21:10:28.145534+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.145534+0000","last_undegraded":"2024-09-10T21:10:28.145534+0000","last_fullsized":"2024-09-10T21:10:28.145534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:10:29.137082+0000","last_change":"2024-09-10T21:10:29.137082+0000","last_active":"2024-09-10T21:10:29.137082+0000","last_peered":"2024-09-10T21:10:29.137082+0000","last_clean":"2024-09-10T21:10:29.137082+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:29.137082+0000","last_undegraded":"2024-09-10T21:10:29.137082+0000","last_fullsized":"2024-09-10T21:10:29.137082+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T21:10:28.138794+0000","last_change":"2024-09-10T21:10:28.138794+0000","last_active":"2024-09-10T21:10:28.130271+0000","last_peered":"2024-09-10T21:10:28.130271+0000","last_clean":"2024-09-10T21:10:28.130271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:10:28.138794+0000","last_undegraded":"2024-09-10T21:10:28.138794+0000","last_fullsized":"2024-09-10T21:10:28.138794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":124,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":753664,"data_stored":637388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":806912,"data_stored":662625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:10:35.122 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-10T21:10:35.122 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.122 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-10T21:10:35.122 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.122 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-10T21:10:35.122 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.123 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-10T21:10:35.123 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.123 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-10T21:10:35.123 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.123 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-10T21:10:35.123 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.124 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-10T21:10:35.124 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.124 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-10T21:10:35.124 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.124 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-10T21:10:35.124 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.124 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-10T21:10:35.124 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.124 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-10T21:10:35.125 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.125 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-10T21:10:35.125 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.125 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-10T21:10:35.125 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.125 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-10T21:10:35.125 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.125 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-10T21:10:35.125 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.125 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-10T21:10:35.125 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.126 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-10T21:10:35.126 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.126 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-10T21:10:35.126 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.126 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-09-10T21:10:35.126 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.126 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-10T21:10:35.126 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.126 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-10T21:10:35.127 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.127 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-10T21:10:35.127 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.127 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-10T21:10:35.127 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.127 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-10T21:10:35.127 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.127 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-10T21:10:35.127 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.127 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-10T21:10:35.127 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.128 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-10T21:10:35.128 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.128 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-10T21:10:35.128 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.128 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1b is not active+clean 2024-09-10T21:10:35.128 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.144466+0000', 'last_change': '2024-09-10T21:10:28.144466+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.144466+0000', 'last_undegraded': '2024-09-10T21:10:28.144466+0000', 'last_fullsized': '2024-09-10T21:10:28.144466+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:10:35.128 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-10T21:10:35.128 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.128 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1a is not active+clean 2024-09-10T21:10:35.128 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.145128+0000', 'last_change': '2024-09-10T21:10:28.145128+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.145128+0000', 'last_undegraded': '2024-09-10T21:10:28.145128+0000', 'last_fullsized': '2024-09-10T21:10:28.145128+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:10:35.129 INFO:tasks.ceph.ceph_manager.ceph:PG 2.19 is not active+clean 2024-09-10T21:10:35.129 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.145380+0000', 'last_change': '2024-09-10T21:10:28.145380+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.145380+0000', 'last_undegraded': '2024-09-10T21:10:28.145380+0000', 'last_fullsized': '2024-09-10T21:10:28.145380+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:10:35.129 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-10T21:10:35.129 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.129 INFO:tasks.ceph.ceph_manager.ceph:PG 2.18 is not active+clean 2024-09-10T21:10:35.129 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.143282+0000', 'last_change': '2024-09-10T21:10:28.143282+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.143282+0000', 'last_undegraded': '2024-09-10T21:10:28.143282+0000', 'last_fullsized': '2024-09-10T21:10:28.143282+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:10:35.129 INFO:tasks.ceph.ceph_manager.ceph:PG 3.19 is not active+clean 2024-09-10T21:10:35.129 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.130 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-10T21:10:35.130 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.130 INFO:tasks.ceph.ceph_manager.ceph:PG 2.16 is not active+clean 2024-09-10T21:10:35.130 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.148219+0000', 'last_change': '2024-09-10T21:10:28.148219+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.148219+0000', 'last_undegraded': '2024-09-10T21:10:28.148219+0000', 'last_fullsized': '2024-09-10T21:10:28.148219+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:10:35.130 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-10T21:10:35.130 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.130 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-10T21:10:35.130 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.130 INFO:tasks.ceph.ceph_manager.ceph:PG 2.14 is not active+clean 2024-09-10T21:10:35.130 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.146602+0000', 'last_change': '2024-09-10T21:10:28.146602+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.146602+0000', 'last_undegraded': '2024-09-10T21:10:28.146602+0000', 'last_fullsized': '2024-09-10T21:10:28.146602+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:10:35.130 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-10T21:10:35.131 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.131 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-10T21:10:35.131 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.131 INFO:tasks.ceph.ceph_manager.ceph:PG 2.12 is not active+clean 2024-09-10T21:10:35.131 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.143960+0000', 'last_change': '2024-09-10T21:10:28.143960+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.143960+0000', 'last_undegraded': '2024-09-10T21:10:28.143960+0000', 'last_fullsized': '2024-09-10T21:10:28.143960+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:10:35.131 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-10T21:10:35.131 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.131 INFO:tasks.ceph.ceph_manager.ceph:PG 2.11 is not active+clean 2024-09-10T21:10:35.131 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.146446+0000', 'last_change': '2024-09-10T21:10:28.146446+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.146446+0000', 'last_undegraded': '2024-09-10T21:10:28.146446+0000', 'last_fullsized': '2024-09-10T21:10:28.146446+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:10:35.131 INFO:tasks.ceph.ceph_manager.ceph:PG 2.10 is not active+clean 2024-09-10T21:10:35.131 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.138943+0000', 'last_change': '2024-09-10T21:10:28.138943+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.138943+0000', 'last_undegraded': '2024-09-10T21:10:28.138943+0000', 'last_fullsized': '2024-09-10T21:10:28.138943+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:10:35.132 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-10T21:10:35.132 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.132 INFO:tasks.ceph.ceph_manager.ceph:PG 2.f is not active+clean 2024-09-10T21:10:35.132 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.148023+0000', 'last_change': '2024-09-10T21:10:28.148023+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.148023+0000', 'last_undegraded': '2024-09-10T21:10:28.148023+0000', 'last_fullsized': '2024-09-10T21:10:28.148023+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:10:35.132 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-10T21:10:35.132 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.132 INFO:tasks.ceph.ceph_manager.ceph:PG 2.e is not active+clean 2024-09-10T21:10:35.132 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.139515+0000', 'last_change': '2024-09-10T21:10:28.139515+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.139515+0000', 'last_undegraded': '2024-09-10T21:10:28.139515+0000', 'last_fullsized': '2024-09-10T21:10:28.139515+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:10:35.132 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-10T21:10:35.133 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.133 INFO:tasks.ceph.ceph_manager.ceph:PG 2.d is not active+clean 2024-09-10T21:10:35.133 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.144525+0000', 'last_change': '2024-09-10T21:10:28.144525+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.144525+0000', 'last_undegraded': '2024-09-10T21:10:28.144525+0000', 'last_fullsized': '2024-09-10T21:10:28.144525+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:10:35.133 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-10T21:10:35.133 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.133 INFO:tasks.ceph.ceph_manager.ceph:PG 2.c is not active+clean 2024-09-10T21:10:35.133 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.145343+0000', 'last_change': '2024-09-10T21:10:28.145343+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.145343+0000', 'last_undegraded': '2024-09-10T21:10:28.145343+0000', 'last_fullsized': '2024-09-10T21:10:28.145343+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:10:35.133 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-10T21:10:35.133 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.133 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2024-09-10T21:10:35.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.141728+0000', 'last_change': '2024-09-10T21:10:28.141728+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.141728+0000', 'last_undegraded': '2024-09-10T21:10:28.141728+0000', 'last_fullsized': '2024-09-10T21:10:28.141728+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:10:35.134 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2 is not active+clean 2024-09-10T21:10:35.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.134 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-09-10T21:10:35.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.141939+0000', 'last_change': '2024-09-10T21:10:28.141939+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.141939+0000', 'last_undegraded': '2024-09-10T21:10:28.141939+0000', 'last_fullsized': '2024-09-10T21:10:28.141939+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:10:35.134 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-10T21:10:35.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.134 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-10T21:10:35.134 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.134 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2024-09-10T21:10:35.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.141731+0000', 'last_change': '2024-09-10T21:10:28.141731+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.141731+0000', 'last_undegraded': '2024-09-10T21:10:28.141731+0000', 'last_fullsized': '2024-09-10T21:10:28.141731+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:10:35.135 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-10T21:10:35.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.135 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2024-09-10T21:10:35.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.144620+0000', 'last_change': '2024-09-10T21:10:28.144620+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.144620+0000', 'last_undegraded': '2024-09-10T21:10:28.144620+0000', 'last_fullsized': '2024-09-10T21:10:28.144620+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:10:35.135 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-10T21:10:35.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.135 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-09-10T21:10:35.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.141746+0000', 'last_change': '2024-09-10T21:10:28.141746+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.141746+0000', 'last_undegraded': '2024-09-10T21:10:28.141746+0000', 'last_fullsized': '2024-09-10T21:10:28.141746+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:10:35.135 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-10T21:10:35.135 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.136 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2024-09-10T21:10:35.136 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.138748+0000', 'last_change': '2024-09-10T21:10:28.138748+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.138748+0000', 'last_undegraded': '2024-09-10T21:10:28.138748+0000', 'last_fullsized': '2024-09-10T21:10:28.138748+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:10:35.136 INFO:tasks.ceph.ceph_manager.ceph:PG 3.7 is not active+clean 2024-09-10T21:10:35.136 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.136 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2024-09-10T21:10:35.136 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.141755+0000', 'last_change': '2024-09-10T21:10:28.141755+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.141755+0000', 'last_undegraded': '2024-09-10T21:10:28.141755+0000', 'last_fullsized': '2024-09-10T21:10:28.141755+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:10:35.136 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-10T21:10:35.136 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.136 INFO:tasks.ceph.ceph_manager.ceph:PG 2.8 is not active+clean 2024-09-10T21:10:35.136 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.147248+0000', 'last_change': '2024-09-10T21:10:28.147248+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.147248+0000', 'last_undegraded': '2024-09-10T21:10:28.147248+0000', 'last_fullsized': '2024-09-10T21:10:28.147248+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:10:35.136 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-10T21:10:35.136 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.137 INFO:tasks.ceph.ceph_manager.ceph:PG 2.9 is not active+clean 2024-09-10T21:10:35.137 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.138724+0000', 'last_change': '2024-09-10T21:10:28.138724+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.138724+0000', 'last_undegraded': '2024-09-10T21:10:28.138724+0000', 'last_fullsized': '2024-09-10T21:10:28.138724+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:10:35.137 INFO:tasks.ceph.ceph_manager.ceph:PG 3.8 is not active+clean 2024-09-10T21:10:35.137 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.137 INFO:tasks.ceph.ceph_manager.ceph:PG 2.a is not active+clean 2024-09-10T21:10:35.137 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.146663+0000', 'last_change': '2024-09-10T21:10:28.146663+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.146663+0000', 'last_undegraded': '2024-09-10T21:10:28.146663+0000', 'last_fullsized': '2024-09-10T21:10:28.146663+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:10:35.137 INFO:tasks.ceph.ceph_manager.ceph:PG 2.b is not active+clean 2024-09-10T21:10:35.137 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.143681+0000', 'last_change': '2024-09-10T21:10:28.143681+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.143681+0000', 'last_undegraded': '2024-09-10T21:10:28.143681+0000', 'last_fullsized': '2024-09-10T21:10:28.143681+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:10:35.137 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-10T21:10:35.137 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.138 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-10T21:10:35.138 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.138 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1c is not active+clean 2024-09-10T21:10:35.138 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.148838+0000', 'last_change': '2024-09-10T21:10:28.148838+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.148838+0000', 'last_undegraded': '2024-09-10T21:10:28.148838+0000', 'last_fullsized': '2024-09-10T21:10:28.148838+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:10:35.138 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1d is not active+clean 2024-09-10T21:10:35.138 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.147855+0000', 'last_change': '2024-09-10T21:10:28.147855+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.147855+0000', 'last_undegraded': '2024-09-10T21:10:28.147855+0000', 'last_fullsized': '2024-09-10T21:10:28.147855+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:10:35.138 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-10T21:10:35.138 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.138 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1e is not active+clean 2024-09-10T21:10:35.138 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.145534+0000', 'last_change': '2024-09-10T21:10:28.145534+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.145534+0000', 'last_undegraded': '2024-09-10T21:10:28.145534+0000', 'last_fullsized': '2024-09-10T21:10:28.145534+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:10:35.138 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-10T21:10:35.139 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:10:29.137082+0000', 'last_change': '2024-09-10T21:10:29.137082+0000', 'last_active': '2024-09-10T21:10:29.137082+0000', 'last_peered': '2024-09-10T21:10:29.137082+0000', 'last_clean': '2024-09-10T21:10:29.137082+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:29.137082+0000', 'last_undegraded': '2024-09-10T21:10:29.137082+0000', 'last_fullsized': '2024-09-10T21:10:29.137082+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:29.137082+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:10:35.139 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1f is not active+clean 2024-09-10T21:10:35.139 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:10:28.138794+0000', 'last_change': '2024-09-10T21:10:28.138794+0000', 'last_active': '2024-09-10T21:10:28.130271+0000', 'last_peered': '2024-09-10T21:10:28.130271+0000', 'last_clean': '2024-09-10T21:10:28.130271+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:10:28.138794+0000', 'last_undegraded': '2024-09-10T21:10:28.138794+0000', 'last_fullsized': '2024-09-10T21:10:28.138794+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'last_clean_scrub_stamp': '2024-09-10T21:10:28.130271+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:10:35.139 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T21:10:35.140 INFO:tasks.ceph.mds.d.smithi052.stderr:2024-09-10T21:10:35.128+0000 7fa2afe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:10:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 -- 172.21.15.16:0/4202247268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a98161410 msgr2=0x7f4a980b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 --2- 172.21.15.16:0/4202247268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a98161410 0x7f4a980b2990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009840 tx=0x7f4a8801efd0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 -- 172.21.15.16:0/4202247268 shutdown_connections 2024-09-10T21:10:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 --2- 172.21.15.16:0/4202247268 >> v2:172.21.15.52:3300/0 conn(0x7f4a980cea00 0x7f4a9806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 --2- 172.21.15.16:0/4202247268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a98161410 0x7f4a980b2990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 -- 172.21.15.16:0/4202247268 >> 172.21.15.16:0/4202247268 conn(0x7f4a980d1d00 msgr2=0x7f4a980d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 -- 172.21.15.16:0/4202247268 shutdown_connections 2024-09-10T21:10:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 -- 172.21.15.16:0/4202247268 wait complete. 2024-09-10T21:10:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 Processor -- start 2024-09-10T21:10:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 -- start start 2024-09-10T21:10:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4a980b2ed0 0x7f4a980cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a980cea00 0x7f4a980cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a98161410 0x7f4a98153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9806e3b0 con 0x7f4a98161410 2024-09-10T21:10:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9806e230 con 0x7f4a980cea00 2024-09-10T21:10:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a97200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9806e530 con 0x7f4a980b2ed0 2024-09-10T21:10:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a96800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a98161410 0x7f4a98153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.306+0000 7f4a96800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a98161410 0x7f4a98153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38378/0 (socket says 172.21.15.16:38378) 2024-09-10T21:10:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a96800640 1 -- 172.21.15.16:0/1173080792 learned_addr learned my addr 172.21.15.16:0/1173080792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a8d400640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a980cea00 0x7f4a980cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a95e00640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4a980b2ed0 0x7f4a980cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a96800640 1 -- 172.21.15.16:0/1173080792 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4a980b2ed0 msgr2=0x7f4a980cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a96800640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4a980b2ed0 0x7f4a980cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a96800640 1 -- 172.21.15.16:0/1173080792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a980cea00 msgr2=0x7f4a980cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a96800640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a980cea00 0x7f4a980cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a96800640 1 -- 172.21.15.16:0/1173080792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a981f7f70 con 0x7f4a98161410 2024-09-10T21:10:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a96800640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a98161410 0x7f4a98153b30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000e9b0 tx=0x7f4a9000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a8ea00640 1 -- 172.21.15.16:0/1173080792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90004d30 con 0x7f4a98161410 2024-09-10T21:10:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a8ea00640 1 -- 172.21.15.16:0/1173080792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a9000cb20 con 0x7f4a98161410 2024-09-10T21:10:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a8ea00640 1 -- 172.21.15.16:0/1173080792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90005020 con 0x7f4a98161410 2024-09-10T21:10:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a97200640 1 -- 172.21.15.16:0/1173080792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a981f8250 con 0x7f4a98161410 2024-09-10T21:10:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a97200640 1 -- 172.21.15.16:0/1173080792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a981f8810 con 0x7f4a98161410 2024-09-10T21:10:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a97200640 1 -- 172.21.15.16:0/1173080792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9806dcf0 con 0x7f4a98161410 2024-09-10T21:10:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.310+0000 7f4a8ea00640 1 -- 172.21.15.16:0/1173080792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4a90022060 con 0x7f4a98161410 2024-09-10T21:10:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.314+0000 7f4a8ea00640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a740725e0 0x7f4a74074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.314+0000 7f4a8ea00640 1 -- 172.21.15.16:0/1173080792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4a90093760 con 0x7f4a98161410 2024-09-10T21:10:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.314+0000 7f4a95e00640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a740725e0 0x7f4a74074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.314+0000 7f4a95e00640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a740725e0 0x7f4a74074aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4a880b2000 tx=0x7f4a8801d810 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.318+0000 7f4a8ea00640 1 -- 172.21.15.16:0/1173080792 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4a90016030 con 0x7f4a98161410 2024-09-10T21:10:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.550+0000 7f4a97200640 1 -- 172.21.15.16:0/1173080792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4a980b2360 con 0x7f4a98161410 2024-09-10T21:10:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.550+0000 7f4a8ea00640 1 -- 172.21.15.16:0/1173080792 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1214 (secure 0 0 0) 0x7f4a9005e7e0 con 0x7f4a98161410 2024-09-10T21:10:35.556 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:35.556 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":30,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1726002618,"num_in_osds":8,"osd_in_since":1726002608,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":56},{"state_name":"creating+peering","count":28},{"state_name":"active+clean","count":13}],"num_pgs":97,"num_pools":3,"num_objects":6,"data_bytes":590492,"bytes_used":258392064,"bytes_avail":772835721216,"bytes_total":773094113280,"unknown_pgs_ratio":0.57731956243515015,"inactive_pgs_ratio":0.28865978121757507,"write_bytes_sec":1539,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":8,"btime":"2024-09-10T21:10:33:173828+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4356}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-10T21:10:03.681249+0000","services":{}},"progress_events":{}} 2024-09-10T21:10:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.554+0000 7f4a97200640 1 -- 172.21.15.16:0/1173080792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a740725e0 msgr2=0x7f4a74074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.554+0000 7f4a97200640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a740725e0 0x7f4a74074aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4a880b2000 tx=0x7f4a8801d810 comp rx=0 tx=0).stop 2024-09-10T21:10:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.554+0000 7f4a97200640 1 -- 172.21.15.16:0/1173080792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a98161410 msgr2=0x7f4a98153b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.554+0000 7f4a97200640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a98161410 0x7f4a98153b30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000e9b0 tx=0x7f4a9000ee70 comp rx=0 tx=0).stop 2024-09-10T21:10:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.554+0000 7f4a97200640 1 -- 172.21.15.16:0/1173080792 shutdown_connections 2024-09-10T21:10:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.554+0000 7f4a97200640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a740725e0 0x7f4a74074aa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.554+0000 7f4a97200640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a98161410 0x7f4a98153b30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.554+0000 7f4a97200640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a980cea00 0x7f4a980cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.554+0000 7f4a97200640 1 --2- 172.21.15.16:0/1173080792 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4a980b2ed0 0x7f4a980cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.554+0000 7f4a97200640 1 -- 172.21.15.16:0/1173080792 >> 172.21.15.16:0/1173080792 conn(0x7f4a980d1d00 msgr2=0x7f4a98071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.554+0000 7f4a97200640 1 -- 172.21.15.16:0/1173080792 shutdown_connections 2024-09-10T21:10:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.554+0000 7f4a97200640 1 -- 172.21.15.16:0/1173080792 wait complete. 2024-09-10T21:10:35.577 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T21:10:35.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:10:35.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.718+0000 7fb1d5e00640 1 -- 172.21.15.16:0/3859934527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d00b0b40 msgr2=0x7fb1d00d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:35.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.718+0000 7fb1d5e00640 1 --2- 172.21.15.16:0/3859934527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d00b0b40 0x7fb1d00d0ad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0073f60 tx=0x7fb1c801cd30 comp rx=0 tx=0).stop 2024-09-10T21:10:35.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.718+0000 7fb1d5e00640 1 -- 172.21.15.16:0/3859934527 shutdown_connections 2024-09-10T21:10:35.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.718+0000 7fb1d5e00640 1 --2- 172.21.15.16:0/3859934527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d00b0b40 0x7fb1d00d0ad0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.718+0000 7fb1d5e00640 1 --2- 172.21.15.16:0/3859934527 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1d0071230 0x7fb1d00d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.718+0000 7fb1d5e00640 1 --2- 172.21.15.16:0/3859934527 >> v2:172.21.15.52:3300/0 conn(0x7fb1d0161620 0x7fb1d0070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.718+0000 7fb1d5e00640 1 -- 172.21.15.16:0/3859934527 >> 172.21.15.16:0/3859934527 conn(0x7fb1d009b130 msgr2=0x7fb1d00d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:35.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.718+0000 7fb1d5e00640 1 -- 172.21.15.16:0/3859934527 shutdown_connections 2024-09-10T21:10:35.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.718+0000 7fb1d5e00640 1 -- 172.21.15.16:0/3859934527 wait complete. 2024-09-10T21:10:35.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1d5e00640 1 Processor -- start 2024-09-10T21:10:35.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1d5e00640 1 -- start start 2024-09-10T21:10:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1d5e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0071230 0x7fb1d00b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1d5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d00b0b40 0x7fb1d00c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1d5e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1d0161620 0x7fb1d014dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1d5e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d006e020 con 0x7fb1d00b0b40 2024-09-10T21:10:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1d5e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d00cf380 con 0x7fb1d0071230 2024-09-10T21:10:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1d5e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d006e1a0 con 0x7fb1d0161620 2024-09-10T21:10:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1c7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d00b0b40 0x7fb1d00c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1c7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d00b0b40 0x7fb1d00c5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38390/0 (socket says 172.21.15.16:38390) 2024-09-10T21:10:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1c7e00640 1 -- 172.21.15.16:0/841676686 learned_addr learned my addr 172.21.15.16:0/841676686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1cea00640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0071230 0x7fb1d00b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1cf400640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1d0161620 0x7fb1d014dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1c7e00640 1 -- 172.21.15.16:0/841676686 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1d0161620 msgr2=0x7fb1d014dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1c7e00640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1d0161620 0x7fb1d014dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1c7e00640 1 -- 172.21.15.16:0/841676686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0071230 msgr2=0x7fb1d00b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1c7e00640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0071230 0x7fb1d00b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1c7e00640 1 -- 172.21.15.16:0/841676686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d00b4200 con 0x7fb1d00b0b40 2024-09-10T21:10:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1c7e00640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d00b0b40 0x7fb1d00c5ad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800d700 tx=0x7fb1b800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1c7400640 1 -- 172.21.15.16:0/841676686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8017070 con 0x7fb1d00b0b40 2024-09-10T21:10:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1c7400640 1 -- 172.21.15.16:0/841676686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1b8005310 con 0x7fb1d00b0b40 2024-09-10T21:10:35.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1c7400640 1 -- 172.21.15.16:0/841676686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8009820 con 0x7fb1d00b0b40 2024-09-10T21:10:35.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1d5e00640 1 -- 172.21.15.16:0/841676686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d00b44e0 con 0x7fb1d00b0b40 2024-09-10T21:10:35.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1d5e00640 1 -- 172.21.15.16:0/841676686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d00b2fd0 con 0x7fb1d00b0b40 2024-09-10T21:10:35.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.722+0000 7fb1d5e00640 1 -- 172.21.15.16:0/841676686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d0070cf0 con 0x7fb1d00b0b40 2024-09-10T21:10:35.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.726+0000 7fb1c7400640 1 -- 172.21.15.16:0/841676686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb1b80099c0 con 0x7fb1d00b0b40 2024-09-10T21:10:35.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.726+0000 7fb1c7400640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1ac0725e0 0x7fb1ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:35.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.726+0000 7fb1c7400640 1 -- 172.21.15.16:0/841676686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb1b80932c0 con 0x7fb1d00b0b40 2024-09-10T21:10:35.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.726+0000 7fb1cea00640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1ac0725e0 0x7fb1ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:35.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.726+0000 7fb1cea00640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1ac0725e0 0x7fb1ac074aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00c8bc0 tx=0x7fb1c000a250 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:35.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.730+0000 7fb1c7400640 1 -- 172.21.15.16:0/841676686 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb1b805e340 con 0x7fb1d00b0b40 2024-09-10T21:10:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.902+0000 7fb1d5e00640 1 -- 172.21.15.16:0/841676686 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1d00c8bf0 con 0x7fb1ac0725e0 2024-09-10T21:10:35.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.906+0000 7fb1c7400640 1 -- 172.21.15.16:0/841676686 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251932 (secure 0 0 0) 0x7fb1d00c8bf0 con 0x7fb1ac0725e0 2024-09-10T21:10:35.915 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:35.916 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:10:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.910+0000 7fb1d5e00640 1 -- 172.21.15.16:0/841676686 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1ac0725e0 msgr2=0x7fb1ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.910+0000 7fb1d5e00640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1ac0725e0 0x7fb1ac074aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00c8bc0 tx=0x7fb1c000a250 comp rx=0 tx=0).stop 2024-09-10T21:10:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.910+0000 7fb1d5e00640 1 -- 172.21.15.16:0/841676686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d00b0b40 msgr2=0x7fb1d00c5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.910+0000 7fb1d5e00640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d00b0b40 0x7fb1d00c5ad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800d700 tx=0x7fb1b800dbc0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.910+0000 7fb1d5e00640 1 -- 172.21.15.16:0/841676686 shutdown_connections 2024-09-10T21:10:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.910+0000 7fb1d5e00640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1ac0725e0 0x7fb1ac074aa0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.910+0000 7fb1d5e00640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1d0161620 0x7fb1d014dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.910+0000 7fb1d5e00640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d00b0b40 0x7fb1d00c5ad0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.910+0000 7fb1d5e00640 1 --2- 172.21.15.16:0/841676686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d0071230 0x7fb1d00b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.910+0000 7fb1d5e00640 1 -- 172.21.15.16:0/841676686 >> 172.21.15.16:0/841676686 conn(0x7fb1d009b130 msgr2=0x7fb1d0160820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.910+0000 7fb1d5e00640 1 -- 172.21.15.16:0/841676686 shutdown_connections 2024-09-10T21:10:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:35.910+0000 7fb1d5e00640 1 -- 172.21.15.16:0/841676686 wait complete. 2024-09-10T21:10:35.935 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-10T21:10:35.457720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252400,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722320,"statfs":{"total":773094113280,"available":772835655680,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.312868"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179857+0000","last_change":"2024-09-10T21:10:30.197837+0000","last_active":"2024-09-10T21:10:31.179857+0000","last_peered":"2024-09-10T21:10:31.179857+0000","last_clean":"2024-09-10T21:10:31.179857+0000","last_became_active":"2024-09-10T21:10:30.197476+0000","last_became_peered":"2024-09-10T21:10:30.197476+0000","last_unstale":"2024-09-10T21:10:31.179857+0000","last_undegraded":"2024-09-10T21:10:31.179857+0000","last_fullsized":"2024-09-10T21:10:31.179857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:31.264063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166547+0000","last_change":"2024-09-10T21:10:30.162505+0000","last_active":"2024-09-10T21:10:31.166547+0000","last_peered":"2024-09-10T21:10:31.166547+0000","last_clean":"2024-09-10T21:10:31.166547+0000","last_became_active":"2024-09-10T21:10:30.162220+0000","last_became_peered":"2024-09-10T21:10:30.162220+0000","last_unstale":"2024-09-10T21:10:31.166547+0000","last_undegraded":"2024-09-10T21:10:31.166547+0000","last_fullsized":"2024-09-10T21:10:31.166547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:28.836761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166183+0000","last_change":"2024-09-10T21:10:30.166240+0000","last_active":"2024-09-10T21:10:31.166183+0000","last_peered":"2024-09-10T21:10:31.166183+0000","last_clean":"2024-09-10T21:10:31.166183+0000","last_became_active":"2024-09-10T21:10:30.165827+0000","last_became_peered":"2024-09-10T21:10:30.165827+0000","last_unstale":"2024-09-10T21:10:31.166183+0000","last_undegraded":"2024-09-10T21:10:31.166183+0000","last_fullsized":"2024-09-10T21:10:31.166183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:31.342353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166786+0000","last_change":"2024-09-10T21:10:30.163365+0000","last_active":"2024-09-10T21:10:31.166786+0000","last_peered":"2024-09-10T21:10:31.166786+0000","last_clean":"2024-09-10T21:10:31.166786+0000","last_became_active":"2024-09-10T21:10:30.163034+0000","last_became_peered":"2024-09-10T21:10:30.163034+0000","last_unstale":"2024-09-10T21:10:31.166786+0000","last_undegraded":"2024-09-10T21:10:31.166786+0000","last_fullsized":"2024-09-10T21:10:31.166786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:36.337660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179172+0000","last_change":"2024-09-10T21:10:30.174443+0000","last_active":"2024-09-10T21:10:31.179172+0000","last_peered":"2024-09-10T21:10:31.179172+0000","last_clean":"2024-09-10T21:10:31.179172+0000","last_became_active":"2024-09-10T21:10:30.173975+0000","last_became_peered":"2024-09-10T21:10:30.173975+0000","last_unstale":"2024-09-10T21:10:31.179172+0000","last_undegraded":"2024-09-10T21:10:31.179172+0000","last_fullsized":"2024-09-10T21:10:31.179172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:37.732853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166683+0000","last_change":"2024-09-10T21:10:30.166122+0000","last_active":"2024-09-10T21:10:31.166683+0000","last_peered":"2024-09-10T21:10:31.166683+0000","last_clean":"2024-09-10T21:10:31.166683+0000","last_became_active":"2024-09-10T21:10:30.165627+0000","last_became_peered":"2024-09-10T21:10:30.165627+0000","last_unstale":"2024-09-10T21:10:31.166683+0000","last_undegraded":"2024-09-10T21:10:31.166683+0000","last_fullsized":"2024-09-10T21:10:31.166683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:39.070890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178907+0000","last_change":"2024-09-10T21:10:30.173891+0000","last_active":"2024-09-10T21:10:31.178907+0000","last_peered":"2024-09-10T21:10:31.178907+0000","last_clean":"2024-09-10T21:10:31.178907+0000","last_became_active":"2024-09-10T21:10:30.173513+0000","last_became_peered":"2024-09-10T21:10:30.173513+0000","last_unstale":"2024-09-10T21:10:31.178907+0000","last_undegraded":"2024-09-10T21:10:31.178907+0000","last_fullsized":"2024-09-10T21:10:31.178907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:49.396588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.165773+0000","last_change":"2024-09-10T21:10:30.173739+0000","last_active":"2024-09-10T21:10:31.165773+0000","last_peered":"2024-09-10T21:10:31.165773+0000","last_clean":"2024-09-10T21:10:31.165773+0000","last_became_active":"2024-09-10T21:10:30.173401+0000","last_became_peered":"2024-09-10T21:10:30.173401+0000","last_unstale":"2024-09-10T21:10:31.165773+0000","last_undegraded":"2024-09-10T21:10:31.165773+0000","last_fullsized":"2024-09-10T21:10:31.165773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:43.846568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168511+0000","last_change":"2024-09-10T21:10:30.180534+0000","last_active":"2024-09-10T21:10:31.168511+0000","last_peered":"2024-09-10T21:10:31.168511+0000","last_clean":"2024-09-10T21:10:31.168511+0000","last_became_active":"2024-09-10T21:10:30.180196+0000","last_became_peered":"2024-09-10T21:10:30.180196+0000","last_unstale":"2024-09-10T21:10:31.168511+0000","last_undegraded":"2024-09-10T21:10:31.168511+0000","last_fullsized":"2024-09-10T21:10:31.168511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:04.038591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179548+0000","last_change":"2024-09-10T21:10:30.191472+0000","last_active":"2024-09-10T21:10:31.179548+0000","last_peered":"2024-09-10T21:10:31.179548+0000","last_clean":"2024-09-10T21:10:31.179548+0000","last_became_active":"2024-09-10T21:10:30.182657+0000","last_became_peered":"2024-09-10T21:10:30.182657+0000","last_unstale":"2024-09-10T21:10:31.179548+0000","last_undegraded":"2024-09-10T21:10:31.179548+0000","last_fullsized":"2024-09-10T21:10:31.179548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:42.385122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179302+0000","last_change":"2024-09-10T21:10:30.196473+0000","last_active":"2024-09-10T21:10:31.179302+0000","last_peered":"2024-09-10T21:10:31.179302+0000","last_clean":"2024-09-10T21:10:31.179302+0000","last_became_active":"2024-09-10T21:10:30.195712+0000","last_became_peered":"2024-09-10T21:10:30.195712+0000","last_unstale":"2024-09-10T21:10:31.179302+0000","last_undegraded":"2024-09-10T21:10:31.179302+0000","last_fullsized":"2024-09-10T21:10:31.179302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:14.592947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179252+0000","last_change":"2024-09-10T21:10:30.187870+0000","last_active":"2024-09-10T21:10:31.179252+0000","last_peered":"2024-09-10T21:10:31.179252+0000","last_clean":"2024-09-10T21:10:31.179252+0000","last_became_active":"2024-09-10T21:10:30.186791+0000","last_became_peered":"2024-09-10T21:10:30.186791+0000","last_unstale":"2024-09-10T21:10:31.179252+0000","last_undegraded":"2024-09-10T21:10:31.179252+0000","last_fullsized":"2024-09-10T21:10:31.179252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:49.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707773+0000","last_change":"2024-09-10T21:10:30.174269+0000","last_active":"2024-09-10T21:10:31.707773+0000","last_peered":"2024-09-10T21:10:31.707773+0000","last_clean":"2024-09-10T21:10:31.707773+0000","last_became_active":"2024-09-10T21:10:30.173714+0000","last_became_peered":"2024-09-10T21:10:30.173714+0000","last_unstale":"2024-09-10T21:10:31.707773+0000","last_undegraded":"2024-09-10T21:10:31.707773+0000","last_fullsized":"2024-09-10T21:10:31.707773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:30.207193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707685+0000","last_change":"2024-09-10T21:10:30.172867+0000","last_active":"2024-09-10T21:10:31.707685+0000","last_peered":"2024-09-10T21:10:31.707685+0000","last_clean":"2024-09-10T21:10:31.707685+0000","last_became_active":"2024-09-10T21:10:30.172301+0000","last_became_peered":"2024-09-10T21:10:30.172301+0000","last_unstale":"2024-09-10T21:10:31.707685+0000","last_undegraded":"2024-09-10T21:10:31.707685+0000","last_fullsized":"2024-09-10T21:10:31.707685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:32.205639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.165740+0000","last_change":"2024-09-10T21:10:30.181460+0000","last_active":"2024-09-10T21:10:31.165740+0000","last_peered":"2024-09-10T21:10:31.165740+0000","last_clean":"2024-09-10T21:10:31.165740+0000","last_became_active":"2024-09-10T21:10:30.175816+0000","last_became_peered":"2024-09-10T21:10:30.175816+0000","last_unstale":"2024-09-10T21:10:31.165740+0000","last_undegraded":"2024-09-10T21:10:31.165740+0000","last_fullsized":"2024-09-10T21:10:31.165740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:26.624987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708880+0000","last_change":"2024-09-10T21:10:30.185809+0000","last_active":"2024-09-10T21:10:31.708880+0000","last_peered":"2024-09-10T21:10:31.708880+0000","last_clean":"2024-09-10T21:10:31.708880+0000","last_became_active":"2024-09-10T21:10:30.185485+0000","last_became_peered":"2024-09-10T21:10:30.185485+0000","last_unstale":"2024-09-10T21:10:31.708880+0000","last_undegraded":"2024-09-10T21:10:31.708880+0000","last_fullsized":"2024-09-10T21:10:31.708880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:16.358565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709494+0000","last_change":"2024-09-10T21:10:30.196400+0000","last_active":"2024-09-10T21:10:31.709494+0000","last_peered":"2024-09-10T21:10:31.709494+0000","last_clean":"2024-09-10T21:10:31.709494+0000","last_became_active":"2024-09-10T21:10:30.194199+0000","last_became_peered":"2024-09-10T21:10:30.194199+0000","last_unstale":"2024-09-10T21:10:31.709494+0000","last_undegraded":"2024-09-10T21:10:31.709494+0000","last_fullsized":"2024-09-10T21:10:31.709494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:21.510129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446465+0000","last_change":"2024-09-10T21:10:30.169268+0000","last_active":"2024-09-10T21:10:31.446465+0000","last_peered":"2024-09-10T21:10:31.446465+0000","last_clean":"2024-09-10T21:10:31.446465+0000","last_became_active":"2024-09-10T21:10:30.168869+0000","last_became_peered":"2024-09-10T21:10:30.168869+0000","last_unstale":"2024-09-10T21:10:31.446465+0000","last_undegraded":"2024-09-10T21:10:31.446465+0000","last_fullsized":"2024-09-10T21:10:31.446465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:06.070406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709259+0000","last_change":"2024-09-10T21:10:30.180496+0000","last_active":"2024-09-10T21:10:31.709259+0000","last_peered":"2024-09-10T21:10:31.709259+0000","last_clean":"2024-09-10T21:10:31.709259+0000","last_became_active":"2024-09-10T21:10:30.180199+0000","last_became_peered":"2024-09-10T21:10:30.180199+0000","last_unstale":"2024-09-10T21:10:31.709259+0000","last_undegraded":"2024-09-10T21:10:31.709259+0000","last_fullsized":"2024-09-10T21:10:31.709259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:36.737204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179263+0000","last_change":"2024-09-10T21:10:30.187710+0000","last_active":"2024-09-10T21:10:31.179263+0000","last_peered":"2024-09-10T21:10:31.179263+0000","last_clean":"2024-09-10T21:10:31.179263+0000","last_became_active":"2024-09-10T21:10:30.186422+0000","last_became_peered":"2024-09-10T21:10:30.186422+0000","last_unstale":"2024-09-10T21:10:31.179263+0000","last_undegraded":"2024-09-10T21:10:31.179263+0000","last_fullsized":"2024-09-10T21:10:31.179263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:37.171374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173731+0000","last_change":"2024-09-10T21:10:30.157845+0000","last_active":"2024-09-10T21:10:31.173731+0000","last_peered":"2024-09-10T21:10:31.173731+0000","last_clean":"2024-09-10T21:10:31.173731+0000","last_became_active":"2024-09-10T21:10:30.155515+0000","last_became_peered":"2024-09-10T21:10:30.155515+0000","last_unstale":"2024-09-10T21:10:31.173731+0000","last_undegraded":"2024-09-10T21:10:31.173731+0000","last_fullsized":"2024-09-10T21:10:31.173731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:36.354773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167488+0000","last_change":"2024-09-10T21:10:30.168344+0000","last_active":"2024-09-10T21:10:31.167488+0000","last_peered":"2024-09-10T21:10:31.167488+0000","last_clean":"2024-09-10T21:10:31.167488+0000","last_became_active":"2024-09-10T21:10:30.167966+0000","last_became_peered":"2024-09-10T21:10:30.167966+0000","last_unstale":"2024-09-10T21:10:31.167488+0000","last_undegraded":"2024-09-10T21:10:31.167488+0000","last_fullsized":"2024-09-10T21:10:31.167488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:38.100207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166637+0000","last_change":"2024-09-10T21:10:30.159521+0000","last_active":"2024-09-10T21:10:31.166637+0000","last_peered":"2024-09-10T21:10:31.166637+0000","last_clean":"2024-09-10T21:10:31.166637+0000","last_became_active":"2024-09-10T21:10:30.159269+0000","last_became_peered":"2024-09-10T21:10:30.159269+0000","last_unstale":"2024-09-10T21:10:31.166637+0000","last_undegraded":"2024-09-10T21:10:31.166637+0000","last_fullsized":"2024-09-10T21:10:31.166637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:21.433157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708679+0000","last_change":"2024-09-10T21:10:30.182227+0000","last_active":"2024-09-10T21:10:31.708679+0000","last_peered":"2024-09-10T21:10:31.708679+0000","last_clean":"2024-09-10T21:10:31.708679+0000","last_became_active":"2024-09-10T21:10:30.178359+0000","last_became_peered":"2024-09-10T21:10:30.178359+0000","last_unstale":"2024-09-10T21:10:31.708679+0000","last_undegraded":"2024-09-10T21:10:31.708679+0000","last_fullsized":"2024-09-10T21:10:31.708679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:50.308163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166494+0000","last_change":"2024-09-10T21:10:30.170127+0000","last_active":"2024-09-10T21:10:31.166494+0000","last_peered":"2024-09-10T21:10:31.166494+0000","last_clean":"2024-09-10T21:10:31.166494+0000","last_became_active":"2024-09-10T21:10:30.169781+0000","last_became_peered":"2024-09-10T21:10:30.169781+0000","last_unstale":"2024-09-10T21:10:31.166494+0000","last_undegraded":"2024-09-10T21:10:31.166494+0000","last_fullsized":"2024-09-10T21:10:31.166494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:16.729840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167717+0000","last_change":"2024-09-10T21:10:30.161842+0000","last_active":"2024-09-10T21:10:31.167717+0000","last_peered":"2024-09-10T21:10:31.167717+0000","last_clean":"2024-09-10T21:10:31.167717+0000","last_became_active":"2024-09-10T21:10:30.160146+0000","last_became_peered":"2024-09-10T21:10:30.160146+0000","last_unstale":"2024-09-10T21:10:31.167717+0000","last_undegraded":"2024-09-10T21:10:31.167717+0000","last_fullsized":"2024-09-10T21:10:31.167717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:06.600402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707965+0000","last_change":"2024-09-10T21:10:30.176930+0000","last_active":"2024-09-10T21:10:31.707965+0000","last_peered":"2024-09-10T21:10:31.707965+0000","last_clean":"2024-09-10T21:10:31.707965+0000","last_became_active":"2024-09-10T21:10:30.175522+0000","last_became_peered":"2024-09-10T21:10:30.175522+0000","last_unstale":"2024-09-10T21:10:31.707965+0000","last_undegraded":"2024-09-10T21:10:31.707965+0000","last_fullsized":"2024-09-10T21:10:31.707965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:36.354773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707870+0000","last_change":"2024-09-10T21:10:30.179884+0000","last_active":"2024-09-10T21:10:31.707870+0000","last_peered":"2024-09-10T21:10:31.707870+0000","last_clean":"2024-09-10T21:10:31.707870+0000","last_became_active":"2024-09-10T21:10:30.177866+0000","last_became_peered":"2024-09-10T21:10:30.177866+0000","last_unstale":"2024-09-10T21:10:31.707870+0000","last_undegraded":"2024-09-10T21:10:31.707870+0000","last_fullsized":"2024-09-10T21:10:31.707870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:57.957611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166067+0000","last_change":"2024-09-10T21:10:30.169322+0000","last_active":"2024-09-10T21:10:31.166067+0000","last_peered":"2024-09-10T21:10:31.166067+0000","last_clean":"2024-09-10T21:10:31.166067+0000","last_became_active":"2024-09-10T21:10:30.169030+0000","last_became_peered":"2024-09-10T21:10:30.169030+0000","last_unstale":"2024-09-10T21:10:31.166067+0000","last_undegraded":"2024-09-10T21:10:31.166067+0000","last_fullsized":"2024-09-10T21:10:31.166067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:41.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708760+0000","last_change":"2024-09-10T21:10:30.194536+0000","last_active":"2024-09-10T21:10:31.708760+0000","last_peered":"2024-09-10T21:10:31.708760+0000","last_clean":"2024-09-10T21:10:31.708760+0000","last_became_active":"2024-09-10T21:10:30.188848+0000","last_became_peered":"2024-09-10T21:10:30.188848+0000","last_unstale":"2024-09-10T21:10:31.708760+0000","last_undegraded":"2024-09-10T21:10:31.708760+0000","last_fullsized":"2024-09-10T21:10:31.708760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:31.264063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446249+0000","last_change":"2024-09-10T21:10:30.171833+0000","last_active":"2024-09-10T21:10:31.446249+0000","last_peered":"2024-09-10T21:10:31.446249+0000","last_clean":"2024-09-10T21:10:31.446249+0000","last_became_active":"2024-09-10T21:10:30.171461+0000","last_became_peered":"2024-09-10T21:10:30.171461+0000","last_unstale":"2024-09-10T21:10:31.446249+0000","last_undegraded":"2024-09-10T21:10:31.446249+0000","last_fullsized":"2024-09-10T21:10:31.446249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:32.018476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166344+0000","last_change":"2024-09-10T21:10:30.183415+0000","last_active":"2024-09-10T21:10:31.166344+0000","last_peered":"2024-09-10T21:10:31.166344+0000","last_clean":"2024-09-10T21:10:31.166344+0000","last_became_active":"2024-09-10T21:10:30.183093+0000","last_became_peered":"2024-09-10T21:10:30.183093+0000","last_unstale":"2024-09-10T21:10:31.166344+0000","last_undegraded":"2024-09-10T21:10:31.166344+0000","last_fullsized":"2024-09-10T21:10:31.166344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:30.558298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167929+0000","last_change":"2024-09-10T21:10:30.171980+0000","last_active":"2024-09-10T21:10:31.167929+0000","last_peered":"2024-09-10T21:10:31.167929+0000","last_clean":"2024-09-10T21:10:31.167929+0000","last_became_active":"2024-09-10T21:10:30.167136+0000","last_became_peered":"2024-09-10T21:10:30.167136+0000","last_unstale":"2024-09-10T21:10:31.167929+0000","last_undegraded":"2024-09-10T21:10:31.167929+0000","last_fullsized":"2024-09-10T21:10:31.167929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:33.142099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166187+0000","last_change":"2024-09-10T21:10:29.171062+0000","last_active":"2024-09-10T21:10:31.166187+0000","last_peered":"2024-09-10T21:10:31.166187+0000","last_clean":"2024-09-10T21:10:31.166187+0000","last_became_active":"2024-09-10T21:10:29.170765+0000","last_became_peered":"2024-09-10T21:10:29.170765+0000","last_unstale":"2024-09-10T21:10:31.166187+0000","last_undegraded":"2024-09-10T21:10:31.166187+0000","last_fullsized":"2024-09-10T21:10:31.166187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.425174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173821+0000","last_change":"2024-09-10T21:10:30.174660+0000","last_active":"2024-09-10T21:10:31.173821+0000","last_peered":"2024-09-10T21:10:31.173821+0000","last_clean":"2024-09-10T21:10:31.173821+0000","last_became_active":"2024-09-10T21:10:30.174372+0000","last_became_peered":"2024-09-10T21:10:30.174372+0000","last_unstale":"2024-09-10T21:10:31.173821+0000","last_undegraded":"2024-09-10T21:10:31.173821+0000","last_fullsized":"2024-09-10T21:10:31.173821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:59.722000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708981+0000","last_change":"2024-09-10T21:10:29.171541+0000","last_active":"2024-09-10T21:10:31.708981+0000","last_peered":"2024-09-10T21:10:31.708981+0000","last_clean":"2024-09-10T21:10:31.708981+0000","last_became_active":"2024-09-10T21:10:29.171269+0000","last_became_peered":"2024-09-10T21:10:29.171269+0000","last_unstale":"2024-09-10T21:10:31.708981+0000","last_undegraded":"2024-09-10T21:10:31.708981+0000","last_fullsized":"2024-09-10T21:10:31.708981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:10.046262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707383+0000","last_change":"2024-09-10T21:10:29.166482+0000","last_active":"2024-09-10T21:10:31.707383+0000","last_peered":"2024-09-10T21:10:31.707383+0000","last_clean":"2024-09-10T21:10:31.707383+0000","last_became_active":"2024-09-10T21:10:29.166091+0000","last_became_peered":"2024-09-10T21:10:29.166091+0000","last_unstale":"2024-09-10T21:10:31.707383+0000","last_undegraded":"2024-09-10T21:10:31.707383+0000","last_fullsized":"2024-09-10T21:10:31.707383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.425174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166297+0000","last_change":"2024-09-10T21:10:30.183180+0000","last_active":"2024-09-10T21:10:31.166297+0000","last_peered":"2024-09-10T21:10:31.166297+0000","last_clean":"2024-09-10T21:10:31.166297+0000","last_became_active":"2024-09-10T21:10:30.182908+0000","last_became_peered":"2024-09-10T21:10:30.182908+0000","last_unstale":"2024-09-10T21:10:31.166297+0000","last_undegraded":"2024-09-10T21:10:31.166297+0000","last_fullsized":"2024-09-10T21:10:31.166297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:37.697114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708748+0000","last_change":"2024-09-10T21:10:29.167898+0000","last_active":"2024-09-10T21:10:31.708748+0000","last_peered":"2024-09-10T21:10:31.708748+0000","last_clean":"2024-09-10T21:10:31.708748+0000","last_became_active":"2024-09-10T21:10:29.167498+0000","last_became_peered":"2024-09-10T21:10:29.167498+0000","last_unstale":"2024-09-10T21:10:31.708748+0000","last_undegraded":"2024-09-10T21:10:31.708748+0000","last_fullsized":"2024-09-10T21:10:31.708748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:50.337610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173782+0000","last_change":"2024-09-10T21:10:30.161329+0000","last_active":"2024-09-10T21:10:31.173782+0000","last_peered":"2024-09-10T21:10:31.173782+0000","last_clean":"2024-09-10T21:10:31.173782+0000","last_became_active":"2024-09-10T21:10:30.160309+0000","last_became_peered":"2024-09-10T21:10:30.160309+0000","last_unstale":"2024-09-10T21:10:31.173782+0000","last_undegraded":"2024-09-10T21:10:31.173782+0000","last_fullsized":"2024-09-10T21:10:31.173782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:14.235339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168376+0000","last_change":"2024-09-10T21:10:29.170975+0000","last_active":"2024-09-10T21:10:31.168376+0000","last_peered":"2024-09-10T21:10:31.168376+0000","last_clean":"2024-09-10T21:10:31.168376+0000","last_became_active":"2024-09-10T21:10:29.170623+0000","last_became_peered":"2024-09-10T21:10:29.170623+0000","last_unstale":"2024-09-10T21:10:31.168376+0000","last_undegraded":"2024-09-10T21:10:31.168376+0000","last_fullsized":"2024-09-10T21:10:31.168376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167790+0000","last_change":"2024-09-10T21:10:30.160525+0000","last_active":"2024-09-10T21:10:31.167790+0000","last_peered":"2024-09-10T21:10:31.167790+0000","last_clean":"2024-09-10T21:10:31.167790+0000","last_became_active":"2024-09-10T21:10:30.159639+0000","last_became_peered":"2024-09-10T21:10:30.159639+0000","last_unstale":"2024-09-10T21:10:31.167790+0000","last_undegraded":"2024-09-10T21:10:31.167790+0000","last_fullsized":"2024-09-10T21:10:31.167790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:46.116954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179609+0000","last_change":"2024-09-10T21:10:29.175134+0000","last_active":"2024-09-10T21:10:31.179609+0000","last_peered":"2024-09-10T21:10:31.179609+0000","last_clean":"2024-09-10T21:10:31.179609+0000","last_became_active":"2024-09-10T21:10:29.174808+0000","last_became_peered":"2024-09-10T21:10:29.174808+0000","last_unstale":"2024-09-10T21:10:31.179609+0000","last_undegraded":"2024-09-10T21:10:31.179609+0000","last_fullsized":"2024-09-10T21:10:31.179609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:52.586551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166756+0000","last_change":"2024-09-10T21:10:30.167445+0000","last_active":"2024-09-10T21:10:31.166756+0000","last_peered":"2024-09-10T21:10:31.166756+0000","last_clean":"2024-09-10T21:10:31.166756+0000","last_became_active":"2024-09-10T21:10:30.167163+0000","last_became_peered":"2024-09-10T21:10:30.167163+0000","last_unstale":"2024-09-10T21:10:31.166756+0000","last_undegraded":"2024-09-10T21:10:31.166756+0000","last_fullsized":"2024-09-10T21:10:31.166756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:59.291659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446376+0000","last_change":"2024-09-10T21:10:29.171850+0000","last_active":"2024-09-10T21:10:31.446376+0000","last_peered":"2024-09-10T21:10:31.446376+0000","last_clean":"2024-09-10T21:10:31.446376+0000","last_became_active":"2024-09-10T21:10:29.171589+0000","last_became_peered":"2024-09-10T21:10:29.171589+0000","last_unstale":"2024-09-10T21:10:31.446376+0000","last_undegraded":"2024-09-10T21:10:31.446376+0000","last_fullsized":"2024-09-10T21:10:31.446376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:43.143812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708938+0000","last_change":"2024-09-10T21:10:30.162228+0000","last_active":"2024-09-10T21:10:31.708938+0000","last_peered":"2024-09-10T21:10:31.708938+0000","last_clean":"2024-09-10T21:10:31.708938+0000","last_became_active":"2024-09-10T21:10:30.161901+0000","last_became_peered":"2024-09-10T21:10:30.161901+0000","last_unstale":"2024-09-10T21:10:31.708938+0000","last_undegraded":"2024-09-10T21:10:31.708938+0000","last_fullsized":"2024-09-10T21:10:31.708938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:12.489819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167129+0000","last_change":"2024-09-10T21:10:29.164653+0000","last_active":"2024-09-10T21:10:31.167129+0000","last_peered":"2024-09-10T21:10:31.167129+0000","last_clean":"2024-09-10T21:10:31.167129+0000","last_became_active":"2024-09-10T21:10:29.164338+0000","last_became_peered":"2024-09-10T21:10:29.164338+0000","last_unstale":"2024-09-10T21:10:31.167129+0000","last_undegraded":"2024-09-10T21:10:31.167129+0000","last_fullsized":"2024-09-10T21:10:31.167129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:24.324180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179343+0000","last_change":"2024-09-10T21:10:30.191023+0000","last_active":"2024-09-10T21:10:31.179343+0000","last_peered":"2024-09-10T21:10:31.179343+0000","last_clean":"2024-09-10T21:10:31.179343+0000","last_became_active":"2024-09-10T21:10:30.190264+0000","last_became_peered":"2024-09-10T21:10:30.190264+0000","last_unstale":"2024-09-10T21:10:31.179343+0000","last_undegraded":"2024-09-10T21:10:31.179343+0000","last_fullsized":"2024-09-10T21:10:31.179343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:08.381083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446251+0000","last_change":"2024-09-10T21:10:29.178312+0000","last_active":"2024-09-10T21:10:31.446251+0000","last_peered":"2024-09-10T21:10:31.446251+0000","last_clean":"2024-09-10T21:10:31.446251+0000","last_became_active":"2024-09-10T21:10:29.178044+0000","last_became_peered":"2024-09-10T21:10:29.178044+0000","last_unstale":"2024-09-10T21:10:31.446251+0000","last_undegraded":"2024-09-10T21:10:31.446251+0000","last_fullsized":"2024-09-10T21:10:31.446251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:41.871122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707682+0000","last_change":"2024-09-10T21:10:30.181507+0000","last_active":"2024-09-10T21:10:31.707682+0000","last_peered":"2024-09-10T21:10:31.707682+0000","last_clean":"2024-09-10T21:10:31.707682+0000","last_became_active":"2024-09-10T21:10:30.177520+0000","last_became_peered":"2024-09-10T21:10:30.177520+0000","last_unstale":"2024-09-10T21:10:31.707682+0000","last_undegraded":"2024-09-10T21:10:31.707682+0000","last_fullsized":"2024-09-10T21:10:31.707682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:56.567997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709035+0000","last_change":"2024-09-10T21:10:29.166737+0000","last_active":"2024-09-10T21:10:31.709035+0000","last_peered":"2024-09-10T21:10:31.709035+0000","last_clean":"2024-09-10T21:10:31.709035+0000","last_became_active":"2024-09-10T21:10:29.166301+0000","last_became_peered":"2024-09-10T21:10:29.166301+0000","last_unstale":"2024-09-10T21:10:31.709035+0000","last_undegraded":"2024-09-10T21:10:31.709035+0000","last_fullsized":"2024-09-10T21:10:31.709035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:52.586551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709252+0000","last_change":"2024-09-10T21:10:30.180470+0000","last_active":"2024-09-10T21:10:31.709252+0000","last_peered":"2024-09-10T21:10:31.709252+0000","last_clean":"2024-09-10T21:10:31.709252+0000","last_became_active":"2024-09-10T21:10:30.179447+0000","last_became_peered":"2024-09-10T21:10:30.179447+0000","last_unstale":"2024-09-10T21:10:31.709252+0000","last_undegraded":"2024-09-10T21:10:31.709252+0000","last_fullsized":"2024-09-10T21:10:31.709252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:36.701958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178781+0000","last_change":"2024-09-10T21:10:29.169914+0000","last_active":"2024-09-10T21:10:31.178781+0000","last_peered":"2024-09-10T21:10:31.178781+0000","last_clean":"2024-09-10T21:10:31.178781+0000","last_became_active":"2024-09-10T21:10:29.169617+0000","last_became_peered":"2024-09-10T21:10:29.169617+0000","last_unstale":"2024-09-10T21:10:31.178781+0000","last_undegraded":"2024-09-10T21:10:31.178781+0000","last_fullsized":"2024-09-10T21:10:31.178781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:48.286592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167103+0000","last_change":"2024-09-10T21:10:30.169299+0000","last_active":"2024-09-10T21:10:31.167103+0000","last_peered":"2024-09-10T21:10:31.167103+0000","last_clean":"2024-09-10T21:10:31.167103+0000","last_became_active":"2024-09-10T21:10:30.169013+0000","last_became_peered":"2024-09-10T21:10:30.169013+0000","last_unstale":"2024-09-10T21:10:31.167103+0000","last_undegraded":"2024-09-10T21:10:31.167103+0000","last_fullsized":"2024-09-10T21:10:31.167103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:40.546116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173457+0000","last_change":"2024-09-10T21:10:29.162746+0000","last_active":"2024-09-10T21:10:31.173457+0000","last_peered":"2024-09-10T21:10:31.173457+0000","last_clean":"2024-09-10T21:10:31.173457+0000","last_became_active":"2024-09-10T21:10:29.162466+0000","last_became_peered":"2024-09-10T21:10:29.162466+0000","last_unstale":"2024-09-10T21:10:31.173457+0000","last_undegraded":"2024-09-10T21:10:31.173457+0000","last_fullsized":"2024-09-10T21:10:31.173457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.425174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708924+0000","last_change":"2024-09-10T21:10:30.162565+0000","last_active":"2024-09-10T21:10:31.708924+0000","last_peered":"2024-09-10T21:10:31.708924+0000","last_clean":"2024-09-10T21:10:31.708924+0000","last_became_active":"2024-09-10T21:10:30.161927+0000","last_became_peered":"2024-09-10T21:10:30.161927+0000","last_unstale":"2024-09-10T21:10:31.708924+0000","last_undegraded":"2024-09-10T21:10:31.708924+0000","last_fullsized":"2024-09-10T21:10:31.708924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:43.633279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.180274+0000","last_change":"2024-09-10T21:10:29.170318+0000","last_active":"2024-09-10T21:10:31.180274+0000","last_peered":"2024-09-10T21:10:31.180274+0000","last_clean":"2024-09-10T21:10:31.180274+0000","last_became_active":"2024-09-10T21:10:29.169096+0000","last_became_peered":"2024-09-10T21:10:29.169096+0000","last_unstale":"2024-09-10T21:10:31.180274+0000","last_undegraded":"2024-09-10T21:10:31.180274+0000","last_fullsized":"2024-09-10T21:10:31.180274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:01.835985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167829+0000","last_change":"2024-09-10T21:10:30.176716+0000","last_active":"2024-09-10T21:10:31.167829+0000","last_peered":"2024-09-10T21:10:31.167829+0000","last_clean":"2024-09-10T21:10:31.167829+0000","last_became_active":"2024-09-10T21:10:30.175255+0000","last_became_peered":"2024-09-10T21:10:30.175255+0000","last_unstale":"2024-09-10T21:10:31.167829+0000","last_undegraded":"2024-09-10T21:10:31.167829+0000","last_fullsized":"2024-09-10T21:10:31.167829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.859164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173973+0000","last_change":"2024-09-10T21:10:29.163718+0000","last_active":"2024-09-10T21:10:31.173973+0000","last_peered":"2024-09-10T21:10:31.173973+0000","last_clean":"2024-09-10T21:10:31.173973+0000","last_became_active":"2024-09-10T21:10:29.163328+0000","last_became_peered":"2024-09-10T21:10:29.163328+0000","last_unstale":"2024-09-10T21:10:31.173973+0000","last_undegraded":"2024-09-10T21:10:31.173973+0000","last_fullsized":"2024-09-10T21:10:31.173973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:57.741500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.180052+0000","last_change":"2024-09-10T21:10:30.183206+0000","last_active":"2024-09-10T21:10:31.180052+0000","last_peered":"2024-09-10T21:10:31.180052+0000","last_clean":"2024-09-10T21:10:31.180052+0000","last_became_active":"2024-09-10T21:10:30.182950+0000","last_became_peered":"2024-09-10T21:10:30.182950+0000","last_unstale":"2024-09-10T21:10:31.180052+0000","last_undegraded":"2024-09-10T21:10:31.180052+0000","last_fullsized":"2024-09-10T21:10:31.180052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:45.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166843+0000","last_change":"2024-09-10T21:10:29.165292+0000","last_active":"2024-09-10T21:10:31.166843+0000","last_peered":"2024-09-10T21:10:31.166843+0000","last_clean":"2024-09-10T21:10:31.166843+0000","last_became_active":"2024-09-10T21:10:29.164991+0000","last_became_peered":"2024-09-10T21:10:29.164991+0000","last_unstale":"2024-09-10T21:10:31.166843+0000","last_undegraded":"2024-09-10T21:10:31.166843+0000","last_fullsized":"2024-09-10T21:10:31.166843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:57.741500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179462+0000","last_change":"2024-09-10T21:10:30.188067+0000","last_active":"2024-09-10T21:10:31.179462+0000","last_peered":"2024-09-10T21:10:31.179462+0000","last_clean":"2024-09-10T21:10:31.179462+0000","last_became_active":"2024-09-10T21:10:30.187079+0000","last_became_peered":"2024-09-10T21:10:30.187079+0000","last_unstale":"2024-09-10T21:10:31.179462+0000","last_undegraded":"2024-09-10T21:10:31.179462+0000","last_fullsized":"2024-09-10T21:10:31.179462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:16.358565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707972+0000","last_change":"2024-09-10T21:10:29.168326+0000","last_active":"2024-09-10T21:10:31.707972+0000","last_peered":"2024-09-10T21:10:31.707972+0000","last_clean":"2024-09-10T21:10:31.707972+0000","last_became_active":"2024-09-10T21:10:29.168048+0000","last_became_peered":"2024-09-10T21:10:29.168048+0000","last_unstale":"2024-09-10T21:10:31.707972+0000","last_undegraded":"2024-09-10T21:10:31.707972+0000","last_fullsized":"2024-09-10T21:10:31.707972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:56.871372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709184+0000","last_change":"2024-09-10T21:10:30.179705+0000","last_active":"2024-09-10T21:10:31.709184+0000","last_peered":"2024-09-10T21:10:31.709184+0000","last_clean":"2024-09-10T21:10:31.709184+0000","last_became_active":"2024-09-10T21:10:30.175366+0000","last_became_peered":"2024-09-10T21:10:30.175366+0000","last_unstale":"2024-09-10T21:10:31.709184+0000","last_undegraded":"2024-09-10T21:10:31.709184+0000","last_fullsized":"2024-09-10T21:10:31.709184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:44.960366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167202+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:10:31.167202+0000","last_peered":"2024-09-10T21:10:31.167202+0000","last_clean":"2024-09-10T21:10:31.167202+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:10:31.167202+0000","last_undegraded":"2024-09-10T21:10:31.167202+0000","last_fullsized":"2024-09-10T21:10:31.167202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708999+0000","last_change":"2024-09-10T21:10:29.172283+0000","last_active":"2024-09-10T21:10:31.708999+0000","last_peered":"2024-09-10T21:10:31.708999+0000","last_clean":"2024-09-10T21:10:31.708999+0000","last_became_active":"2024-09-10T21:10:29.170550+0000","last_became_peered":"2024-09-10T21:10:29.170550+0000","last_unstale":"2024-09-10T21:10:31.708999+0000","last_undegraded":"2024-09-10T21:10:31.708999+0000","last_fullsized":"2024-09-10T21:10:31.708999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:09.439008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173734+0000","last_change":"2024-09-10T21:10:30.160275+0000","last_active":"2024-09-10T21:10:31.173734+0000","last_peered":"2024-09-10T21:10:31.173734+0000","last_clean":"2024-09-10T21:10:31.173734+0000","last_became_active":"2024-09-10T21:10:30.159817+0000","last_became_peered":"2024-09-10T21:10:30.159817+0000","last_unstale":"2024-09-10T21:10:31.173734+0000","last_undegraded":"2024-09-10T21:10:31.173734+0000","last_fullsized":"2024-09-10T21:10:31.173734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.698693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708902+0000","last_change":"2024-09-10T21:10:29.164034+0000","last_active":"2024-09-10T21:10:31.708902+0000","last_peered":"2024-09-10T21:10:31.708902+0000","last_clean":"2024-09-10T21:10:31.708902+0000","last_became_active":"2024-09-10T21:10:29.163755+0000","last_became_peered":"2024-09-10T21:10:29.163755+0000","last_unstale":"2024-09-10T21:10:31.708902+0000","last_undegraded":"2024-09-10T21:10:31.708902+0000","last_fullsized":"2024-09-10T21:10:31.708902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:10.200363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707745+0000","last_change":"2024-09-10T21:10:30.171490+0000","last_active":"2024-09-10T21:10:31.707745+0000","last_peered":"2024-09-10T21:10:31.707745+0000","last_clean":"2024-09-10T21:10:31.707745+0000","last_became_active":"2024-09-10T21:10:30.170892+0000","last_became_peered":"2024-09-10T21:10:30.170892+0000","last_unstale":"2024-09-10T21:10:31.707745+0000","last_undegraded":"2024-09-10T21:10:31.707745+0000","last_fullsized":"2024-09-10T21:10:31.707745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:36.425140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446253+0000","last_change":"2024-09-10T21:10:29.170636+0000","last_active":"2024-09-10T21:10:31.446253+0000","last_peered":"2024-09-10T21:10:31.446253+0000","last_clean":"2024-09-10T21:10:31.446253+0000","last_became_active":"2024-09-10T21:10:29.170342+0000","last_became_peered":"2024-09-10T21:10:29.170342+0000","last_unstale":"2024-09-10T21:10:31.446253+0000","last_undegraded":"2024-09-10T21:10:31.446253+0000","last_fullsized":"2024-09-10T21:10:31.446253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166047+0000","last_change":"2024-09-10T21:10:30.164807+0000","last_active":"2024-09-10T21:10:31.166047+0000","last_peered":"2024-09-10T21:10:31.166047+0000","last_clean":"2024-09-10T21:10:31.166047+0000","last_became_active":"2024-09-10T21:10:30.164545+0000","last_became_peered":"2024-09-10T21:10:30.164545+0000","last_unstale":"2024-09-10T21:10:31.166047+0000","last_undegraded":"2024-09-10T21:10:31.166047+0000","last_fullsized":"2024-09-10T21:10:31.166047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:35.647295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709113+0000","last_change":"2024-09-10T21:10:29.173240+0000","last_active":"2024-09-10T21:10:31.709113+0000","last_peered":"2024-09-10T21:10:31.709113+0000","last_clean":"2024-09-10T21:10:31.709113+0000","last_became_active":"2024-09-10T21:10:29.172991+0000","last_became_peered":"2024-09-10T21:10:29.172991+0000","last_unstale":"2024-09-10T21:10:31.709113+0000","last_undegraded":"2024-09-10T21:10:31.709113+0000","last_fullsized":"2024-09-10T21:10:31.709113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:00.278081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178965+0000","last_change":"2024-09-10T21:10:30.189010+0000","last_active":"2024-09-10T21:10:31.178965+0000","last_peered":"2024-09-10T21:10:31.178965+0000","last_clean":"2024-09-10T21:10:31.178965+0000","last_became_active":"2024-09-10T21:10:30.188712+0000","last_became_peered":"2024-09-10T21:10:30.188712+0000","last_unstale":"2024-09-10T21:10:31.178965+0000","last_undegraded":"2024-09-10T21:10:31.178965+0000","last_fullsized":"2024-09-10T21:10:31.178965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:04.853111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.165941+0000","last_change":"2024-09-10T21:10:29.173419+0000","last_active":"2024-09-10T21:10:31.165941+0000","last_peered":"2024-09-10T21:10:31.165941+0000","last_clean":"2024-09-10T21:10:31.165941+0000","last_became_active":"2024-09-10T21:10:29.173104+0000","last_became_peered":"2024-09-10T21:10:29.173104+0000","last_unstale":"2024-09-10T21:10:31.165941+0000","last_undegraded":"2024-09-10T21:10:31.165941+0000","last_fullsized":"2024-09-10T21:10:31.165941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:25.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167265+0000","last_change":"2024-09-10T21:10:30.169171+0000","last_active":"2024-09-10T21:10:31.167265+0000","last_peered":"2024-09-10T21:10:31.167265+0000","last_clean":"2024-09-10T21:10:31.167265+0000","last_became_active":"2024-09-10T21:10:30.168866+0000","last_became_peered":"2024-09-10T21:10:30.168866+0000","last_unstale":"2024-09-10T21:10:31.167265+0000","last_undegraded":"2024-09-10T21:10:31.167265+0000","last_fullsized":"2024-09-10T21:10:31.167265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:45.633966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709233+0000","last_change":"2024-09-10T21:10:29.170135+0000","last_active":"2024-09-10T21:10:31.709233+0000","last_peered":"2024-09-10T21:10:31.709233+0000","last_clean":"2024-09-10T21:10:31.709233+0000","last_became_active":"2024-09-10T21:10:29.169894+0000","last_became_peered":"2024-09-10T21:10:29.169894+0000","last_unstale":"2024-09-10T21:10:31.709233+0000","last_undegraded":"2024-09-10T21:10:31.709233+0000","last_fullsized":"2024-09-10T21:10:31.709233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:10.057351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707993+0000","last_change":"2024-09-10T21:10:30.171973+0000","last_active":"2024-09-10T21:10:31.707993+0000","last_peered":"2024-09-10T21:10:31.707993+0000","last_clean":"2024-09-10T21:10:31.707993+0000","last_became_active":"2024-09-10T21:10:30.171622+0000","last_became_peered":"2024-09-10T21:10:30.171622+0000","last_unstale":"2024-09-10T21:10:31.707993+0000","last_undegraded":"2024-09-10T21:10:31.707993+0000","last_fullsized":"2024-09-10T21:10:31.707993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:06.330104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174157+0000","last_change":"2024-09-10T21:10:29.165107+0000","last_active":"2024-09-10T21:10:31.174157+0000","last_peered":"2024-09-10T21:10:31.174157+0000","last_clean":"2024-09-10T21:10:31.174157+0000","last_became_active":"2024-09-10T21:10:29.164798+0000","last_became_peered":"2024-09-10T21:10:29.164798+0000","last_unstale":"2024-09-10T21:10:31.174157+0000","last_undegraded":"2024-09-10T21:10:31.174157+0000","last_fullsized":"2024-09-10T21:10:31.174157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:04.055586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174286+0000","last_change":"2024-09-10T21:10:30.173086+0000","last_active":"2024-09-10T21:10:31.174286+0000","last_peered":"2024-09-10T21:10:31.174286+0000","last_clean":"2024-09-10T21:10:31.174286+0000","last_became_active":"2024-09-10T21:10:30.172730+0000","last_became_peered":"2024-09-10T21:10:30.172730+0000","last_unstale":"2024-09-10T21:10:31.174286+0000","last_undegraded":"2024-09-10T21:10:31.174286+0000","last_fullsized":"2024-09-10T21:10:31.174286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:50.512074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709420+0000","last_change":"2024-09-10T21:10:29.179231+0000","last_active":"2024-09-10T21:10:31.709420+0000","last_peered":"2024-09-10T21:10:31.709420+0000","last_clean":"2024-09-10T21:10:31.709420+0000","last_became_active":"2024-09-10T21:10:29.178937+0000","last_became_peered":"2024-09-10T21:10:29.178937+0000","last_unstale":"2024-09-10T21:10:31.709420+0000","last_undegraded":"2024-09-10T21:10:31.709420+0000","last_fullsized":"2024-09-10T21:10:31.709420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:41.528182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174168+0000","last_change":"2024-09-10T21:10:30.167226+0000","last_active":"2024-09-10T21:10:31.174168+0000","last_peered":"2024-09-10T21:10:31.174168+0000","last_clean":"2024-09-10T21:10:31.174168+0000","last_became_active":"2024-09-10T21:10:30.166959+0000","last_became_peered":"2024-09-10T21:10:30.166959+0000","last_unstale":"2024-09-10T21:10:31.174168+0000","last_undegraded":"2024-09-10T21:10:31.174168+0000","last_fullsized":"2024-09-10T21:10:31.174168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:43.225225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178611+0000","last_change":"2024-09-10T21:10:29.175872+0000","last_active":"2024-09-10T21:10:31.178611+0000","last_peered":"2024-09-10T21:10:31.178611+0000","last_clean":"2024-09-10T21:10:31.178611+0000","last_became_active":"2024-09-10T21:10:29.175659+0000","last_became_peered":"2024-09-10T21:10:29.175659+0000","last_unstale":"2024-09-10T21:10:31.178611+0000","last_undegraded":"2024-09-10T21:10:31.178611+0000","last_fullsized":"2024-09-10T21:10:31.178611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:21.990689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708884+0000","last_change":"2024-09-10T21:10:30.179720+0000","last_active":"2024-09-10T21:10:31.708884+0000","last_peered":"2024-09-10T21:10:31.708884+0000","last_clean":"2024-09-10T21:10:31.708884+0000","last_became_active":"2024-09-10T21:10:30.179435+0000","last_became_peered":"2024-09-10T21:10:30.179435+0000","last_unstale":"2024-09-10T21:10:31.708884+0000","last_undegraded":"2024-09-10T21:10:31.708884+0000","last_fullsized":"2024-09-10T21:10:31.708884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:45.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173726+0000","last_change":"2024-09-10T21:10:29.165142+0000","last_active":"2024-09-10T21:10:31.173726+0000","last_peered":"2024-09-10T21:10:31.173726+0000","last_clean":"2024-09-10T21:10:31.173726+0000","last_became_active":"2024-09-10T21:10:29.164732+0000","last_became_peered":"2024-09-10T21:10:29.164732+0000","last_unstale":"2024-09-10T21:10:31.173726+0000","last_undegraded":"2024-09-10T21:10:31.173726+0000","last_fullsized":"2024-09-10T21:10:31.173726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:26.261468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173644+0000","last_change":"2024-09-10T21:10:30.164096+0000","last_active":"2024-09-10T21:10:31.173644+0000","last_peered":"2024-09-10T21:10:31.173644+0000","last_clean":"2024-09-10T21:10:31.173644+0000","last_became_active":"2024-09-10T21:10:30.163474+0000","last_became_peered":"2024-09-10T21:10:30.163474+0000","last_unstale":"2024-09-10T21:10:31.173644+0000","last_undegraded":"2024-09-10T21:10:31.173644+0000","last_fullsized":"2024-09-10T21:10:31.173644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:57.957611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707812+0000","last_change":"2024-09-10T21:10:29.166991+0000","last_active":"2024-09-10T21:10:31.707812+0000","last_peered":"2024-09-10T21:10:31.707812+0000","last_clean":"2024-09-10T21:10:31.707812+0000","last_became_active":"2024-09-10T21:10:29.166675+0000","last_became_peered":"2024-09-10T21:10:29.166675+0000","last_unstale":"2024-09-10T21:10:31.707812+0000","last_undegraded":"2024-09-10T21:10:31.707812+0000","last_fullsized":"2024-09-10T21:10:31.707812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:25.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446431+0000","last_change":"2024-09-10T21:10:30.176046+0000","last_active":"2024-09-10T21:10:31.446431+0000","last_peered":"2024-09-10T21:10:31.446431+0000","last_clean":"2024-09-10T21:10:31.446431+0000","last_became_active":"2024-09-10T21:10:30.175784+0000","last_became_peered":"2024-09-10T21:10:30.175784+0000","last_unstale":"2024-09-10T21:10:31.446431+0000","last_undegraded":"2024-09-10T21:10:31.446431+0000","last_fullsized":"2024-09-10T21:10:31.446431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:37.972763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708858+0000","last_change":"2024-09-10T21:10:29.172106+0000","last_active":"2024-09-10T21:10:31.708858+0000","last_peered":"2024-09-10T21:10:31.708858+0000","last_clean":"2024-09-10T21:10:31.708858+0000","last_became_active":"2024-09-10T21:10:29.167132+0000","last_became_peered":"2024-09-10T21:10:29.167132+0000","last_unstale":"2024-09-10T21:10:31.708858+0000","last_undegraded":"2024-09-10T21:10:31.708858+0000","last_fullsized":"2024-09-10T21:10:31.708858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:48.493501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708898+0000","last_change":"2024-09-10T21:10:30.176689+0000","last_active":"2024-09-10T21:10:31.708898+0000","last_peered":"2024-09-10T21:10:31.708898+0000","last_clean":"2024-09-10T21:10:31.708898+0000","last_became_active":"2024-09-10T21:10:30.176361+0000","last_became_peered":"2024-09-10T21:10:30.176361+0000","last_unstale":"2024-09-10T21:10:31.708898+0000","last_undegraded":"2024-09-10T21:10:31.708898+0000","last_fullsized":"2024-09-10T21:10:31.708898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:55.720343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709088+0000","last_change":"2024-09-10T21:10:30.180387+0000","last_active":"2024-09-10T21:10:31.709088+0000","last_peered":"2024-09-10T21:10:31.709088+0000","last_clean":"2024-09-10T21:10:31.709088+0000","last_became_active":"2024-09-10T21:10:30.180100+0000","last_became_peered":"2024-09-10T21:10:30.180100+0000","last_unstale":"2024-09-10T21:10:31.709088+0000","last_undegraded":"2024-09-10T21:10:31.709088+0000","last_fullsized":"2024-09-10T21:10:31.709088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:05.939523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179323+0000","last_change":"2024-09-10T21:10:29.178074+0000","last_active":"2024-09-10T21:10:31.179323+0000","last_peered":"2024-09-10T21:10:31.179323+0000","last_clean":"2024-09-10T21:10:31.179323+0000","last_became_active":"2024-09-10T21:10:29.177794+0000","last_became_peered":"2024-09-10T21:10:29.177794+0000","last_unstale":"2024-09-10T21:10:31.179323+0000","last_undegraded":"2024-09-10T21:10:31.179323+0000","last_fullsized":"2024-09-10T21:10:31.179323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:18.267264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167690+0000","last_change":"2024-09-10T21:10:30.179424+0000","last_active":"2024-09-10T21:10:31.167690+0000","last_peered":"2024-09-10T21:10:31.167690+0000","last_clean":"2024-09-10T21:10:31.167690+0000","last_became_active":"2024-09-10T21:10:30.178557+0000","last_became_peered":"2024-09-10T21:10:30.178557+0000","last_unstale":"2024-09-10T21:10:31.167690+0000","last_undegraded":"2024-09-10T21:10:31.167690+0000","last_fullsized":"2024-09-10T21:10:31.167690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:14.118820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179366+0000","last_change":"2024-09-10T21:10:29.169287+0000","last_active":"2024-09-10T21:10:31.179366+0000","last_peered":"2024-09-10T21:10:31.179366+0000","last_clean":"2024-09-10T21:10:31.179366+0000","last_became_active":"2024-09-10T21:10:29.169010+0000","last_became_peered":"2024-09-10T21:10:29.169010+0000","last_unstale":"2024-09-10T21:10:31.179366+0000","last_undegraded":"2024-09-10T21:10:31.179366+0000","last_fullsized":"2024-09-10T21:10:31.179366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:41.528182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166555+0000","last_change":"2024-09-10T21:10:30.166099+0000","last_active":"2024-09-10T21:10:31.166555+0000","last_peered":"2024-09-10T21:10:31.166555+0000","last_clean":"2024-09-10T21:10:31.166555+0000","last_became_active":"2024-09-10T21:10:30.165762+0000","last_became_peered":"2024-09-10T21:10:30.165762+0000","last_unstale":"2024-09-10T21:10:31.166555+0000","last_undegraded":"2024-09-10T21:10:31.166555+0000","last_fullsized":"2024-09-10T21:10:31.166555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:54.002262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166762+0000","last_change":"2024-09-10T21:10:29.167583+0000","last_active":"2024-09-10T21:10:31.166762+0000","last_peered":"2024-09-10T21:10:31.166762+0000","last_clean":"2024-09-10T21:10:31.166762+0000","last_became_active":"2024-09-10T21:10:29.162012+0000","last_became_peered":"2024-09-10T21:10:29.162012+0000","last_unstale":"2024-09-10T21:10:31.166762+0000","last_undegraded":"2024-09-10T21:10:31.166762+0000","last_fullsized":"2024-09-10T21:10:31.166762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:04.055586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166801+0000","last_change":"2024-09-10T21:10:30.162898+0000","last_active":"2024-09-10T21:10:31.166801+0000","last_peered":"2024-09-10T21:10:31.166801+0000","last_clean":"2024-09-10T21:10:31.166801+0000","last_became_active":"2024-09-10T21:10:30.162536+0000","last_became_peered":"2024-09-10T21:10:30.162536+0000","last_unstale":"2024-09-10T21:10:31.166801+0000","last_undegraded":"2024-09-10T21:10:31.166801+0000","last_fullsized":"2024-09-10T21:10:31.166801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:09.399612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174191+0000","last_change":"2024-09-10T21:10:29.166448+0000","last_active":"2024-09-10T21:10:31.174191+0000","last_peered":"2024-09-10T21:10:31.174191+0000","last_clean":"2024-09-10T21:10:31.174191+0000","last_became_active":"2024-09-10T21:10:29.166153+0000","last_became_peered":"2024-09-10T21:10:29.166153+0000","last_unstale":"2024-09-10T21:10:31.174191+0000","last_undegraded":"2024-09-10T21:10:31.174191+0000","last_fullsized":"2024-09-10T21:10:31.174191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:08.960569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":811008,"data_stored":662647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":74471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":72769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":217088,"data_stored":72243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":806912,"data_stored":662625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:10:38.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:10:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 -- 172.21.15.16:0/3862484986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44161900 msgr2=0x7f6e440d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 --2- 172.21.15.16:0/3862484986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44161900 0x7f6e440d0ee0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009900 tx=0x7f6e3001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 -- 172.21.15.16:0/3862484986 shutdown_connections 2024-09-10T21:10:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 --2- 172.21.15.16:0/3862484986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44161900 0x7f6e440d0ee0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 -- 172.21.15.16:0/3862484986 >> 172.21.15.16:0/3862484986 conn(0x7f6e440d1d80 msgr2=0x7f6e440d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 -- 172.21.15.16:0/3862484986 shutdown_connections 2024-09-10T21:10:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 -- 172.21.15.16:0/3862484986 wait complete. 2024-09-10T21:10:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 Processor -- start 2024-09-10T21:10:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 -- start start 2024-09-10T21:10:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6e440b2290 0x7f6e4414d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44161520 0x7f6e440b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e44161900 0x7f6e440d0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4416c000 con 0x7f6e44161520 2024-09-10T21:10:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4416be80 con 0x7f6e44161900 2024-09-10T21:10:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.098+0000 7f6e4a600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6e4416c180 con 0x7f6e440b2290 2024-09-10T21:10:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e42a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44161520 0x7f6e440b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e42a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44161520 0x7f6e440b70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38402/0 (socket says 172.21.15.16:38402) 2024-09-10T21:10:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e42a00640 1 -- 172.21.15.16:0/66080548 learned_addr learned my addr 172.21.15.16:0/66080548 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e43400640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6e440b2290 0x7f6e4414d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e43e00640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e44161900 0x7f6e440d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e42a00640 1 -- 172.21.15.16:0/66080548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6e440b2290 msgr2=0x7f6e4414d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e42a00640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6e440b2290 0x7f6e4414d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e42a00640 1 -- 172.21.15.16:0/66080548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e44161900 msgr2=0x7f6e440d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e42a00640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e44161900 0x7f6e440d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e42a00640 1 -- 172.21.15.16:0/66080548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4414d770 con 0x7f6e44161520 2024-09-10T21:10:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e43e00640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e44161900 0x7f6e440d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e42a00640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44161520 0x7f6e440b70a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009850 tx=0x7f6e300a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e3b400640 1 -- 172.21.15.16:0/66080548 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e300b2040 con 0x7f6e44161520 2024-09-10T21:10:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e3b400640 1 -- 172.21.15.16:0/66080548 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e300044b0 con 0x7f6e44161520 2024-09-10T21:10:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e3b400640 1 -- 172.21.15.16:0/66080548 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30003860 con 0x7f6e44161520 2024-09-10T21:10:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e4a600640 1 -- 172.21.15.16:0/66080548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e440b79f0 con 0x7f6e44161520 2024-09-10T21:10:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e4a600640 1 -- 172.21.15.16:0/66080548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4415e080 con 0x7f6e44161520 2024-09-10T21:10:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e4a600640 1 -- 172.21.15.16:0/66080548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e440b26c0 con 0x7f6e44161520 2024-09-10T21:10:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.102+0000 7f6e3b400640 1 -- 172.21.15.16:0/66080548 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6e30003a00 con 0x7f6e44161520 2024-09-10T21:10:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.106+0000 7f6e3b400640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6e24072610 0x7f6e24074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.106+0000 7f6e3b400640 1 -- 172.21.15.16:0/66080548 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6e3012d3b0 con 0x7f6e44161520 2024-09-10T21:10:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.106+0000 7f6e43400640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6e24072610 0x7f6e24074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.106+0000 7f6e43400640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6e24072610 0x7f6e24074ad0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6e34007fd0 tx=0x7f6e34009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.110+0000 7f6e3b400640 1 -- 172.21.15.16:0/66080548 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6e300164d0 con 0x7f6e44161520 2024-09-10T21:10:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.282+0000 7f6e4a600640 1 -- 172.21.15.16:0/66080548 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e440cfe50 con 0x7f6e24072610 2024-09-10T21:10:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.286+0000 7f6e3b400640 1 -- 172.21.15.16:0/66080548 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251932 (secure 0 0 0) 0x7f6e440cfe50 con 0x7f6e24072610 2024-09-10T21:10:39.294 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:10:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.290+0000 7f6e4a600640 1 -- 172.21.15.16:0/66080548 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6e24072610 msgr2=0x7f6e24074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.290+0000 7f6e4a600640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6e24072610 0x7f6e24074ad0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6e34007fd0 tx=0x7f6e34009040 comp rx=0 tx=0).stop 2024-09-10T21:10:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.290+0000 7f6e4a600640 1 -- 172.21.15.16:0/66080548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44161520 msgr2=0x7f6e440b70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.290+0000 7f6e4a600640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44161520 0x7f6e440b70a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009850 tx=0x7f6e300a5700 comp rx=0 tx=0).stop 2024-09-10T21:10:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.290+0000 7f6e4a600640 1 -- 172.21.15.16:0/66080548 shutdown_connections 2024-09-10T21:10:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.290+0000 7f6e4a600640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6e24072610 0x7f6e24074ad0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.290+0000 7f6e4a600640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e44161900 0x7f6e440d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.290+0000 7f6e4a600640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e44161520 0x7f6e440b70a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.290+0000 7f6e4a600640 1 --2- 172.21.15.16:0/66080548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6e440b2290 0x7f6e4414d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.290+0000 7f6e4a600640 1 -- 172.21.15.16:0/66080548 >> 172.21.15.16:0/66080548 conn(0x7f6e440d1d80 msgr2=0x7f6e44159400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.290+0000 7f6e4a600640 1 -- 172.21.15.16:0/66080548 shutdown_connections 2024-09-10T21:10:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.290+0000 7f6e4a600640 1 -- 172.21.15.16:0/66080548 wait complete. 2024-09-10T21:10:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-10T21:10:37.458143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252400,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722320,"statfs":{"total":773094113280,"available":772835655680,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001448"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179857+0000","last_change":"2024-09-10T21:10:30.197837+0000","last_active":"2024-09-10T21:10:31.179857+0000","last_peered":"2024-09-10T21:10:31.179857+0000","last_clean":"2024-09-10T21:10:31.179857+0000","last_became_active":"2024-09-10T21:10:30.197476+0000","last_became_peered":"2024-09-10T21:10:30.197476+0000","last_unstale":"2024-09-10T21:10:31.179857+0000","last_undegraded":"2024-09-10T21:10:31.179857+0000","last_fullsized":"2024-09-10T21:10:31.179857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:31.264063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166547+0000","last_change":"2024-09-10T21:10:30.162505+0000","last_active":"2024-09-10T21:10:31.166547+0000","last_peered":"2024-09-10T21:10:31.166547+0000","last_clean":"2024-09-10T21:10:31.166547+0000","last_became_active":"2024-09-10T21:10:30.162220+0000","last_became_peered":"2024-09-10T21:10:30.162220+0000","last_unstale":"2024-09-10T21:10:31.166547+0000","last_undegraded":"2024-09-10T21:10:31.166547+0000","last_fullsized":"2024-09-10T21:10:31.166547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:28.836761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166183+0000","last_change":"2024-09-10T21:10:30.166240+0000","last_active":"2024-09-10T21:10:31.166183+0000","last_peered":"2024-09-10T21:10:31.166183+0000","last_clean":"2024-09-10T21:10:31.166183+0000","last_became_active":"2024-09-10T21:10:30.165827+0000","last_became_peered":"2024-09-10T21:10:30.165827+0000","last_unstale":"2024-09-10T21:10:31.166183+0000","last_undegraded":"2024-09-10T21:10:31.166183+0000","last_fullsized":"2024-09-10T21:10:31.166183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:31.342353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166786+0000","last_change":"2024-09-10T21:10:30.163365+0000","last_active":"2024-09-10T21:10:31.166786+0000","last_peered":"2024-09-10T21:10:31.166786+0000","last_clean":"2024-09-10T21:10:31.166786+0000","last_became_active":"2024-09-10T21:10:30.163034+0000","last_became_peered":"2024-09-10T21:10:30.163034+0000","last_unstale":"2024-09-10T21:10:31.166786+0000","last_undegraded":"2024-09-10T21:10:31.166786+0000","last_fullsized":"2024-09-10T21:10:31.166786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:36.337660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179172+0000","last_change":"2024-09-10T21:10:30.174443+0000","last_active":"2024-09-10T21:10:31.179172+0000","last_peered":"2024-09-10T21:10:31.179172+0000","last_clean":"2024-09-10T21:10:31.179172+0000","last_became_active":"2024-09-10T21:10:30.173975+0000","last_became_peered":"2024-09-10T21:10:30.173975+0000","last_unstale":"2024-09-10T21:10:31.179172+0000","last_undegraded":"2024-09-10T21:10:31.179172+0000","last_fullsized":"2024-09-10T21:10:31.179172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:37.732853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166683+0000","last_change":"2024-09-10T21:10:30.166122+0000","last_active":"2024-09-10T21:10:31.166683+0000","last_peered":"2024-09-10T21:10:31.166683+0000","last_clean":"2024-09-10T21:10:31.166683+0000","last_became_active":"2024-09-10T21:10:30.165627+0000","last_became_peered":"2024-09-10T21:10:30.165627+0000","last_unstale":"2024-09-10T21:10:31.166683+0000","last_undegraded":"2024-09-10T21:10:31.166683+0000","last_fullsized":"2024-09-10T21:10:31.166683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:39.070890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178907+0000","last_change":"2024-09-10T21:10:30.173891+0000","last_active":"2024-09-10T21:10:31.178907+0000","last_peered":"2024-09-10T21:10:31.178907+0000","last_clean":"2024-09-10T21:10:31.178907+0000","last_became_active":"2024-09-10T21:10:30.173513+0000","last_became_peered":"2024-09-10T21:10:30.173513+0000","last_unstale":"2024-09-10T21:10:31.178907+0000","last_undegraded":"2024-09-10T21:10:31.178907+0000","last_fullsized":"2024-09-10T21:10:31.178907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:49.396588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.165773+0000","last_change":"2024-09-10T21:10:30.173739+0000","last_active":"2024-09-10T21:10:31.165773+0000","last_peered":"2024-09-10T21:10:31.165773+0000","last_clean":"2024-09-10T21:10:31.165773+0000","last_became_active":"2024-09-10T21:10:30.173401+0000","last_became_peered":"2024-09-10T21:10:30.173401+0000","last_unstale":"2024-09-10T21:10:31.165773+0000","last_undegraded":"2024-09-10T21:10:31.165773+0000","last_fullsized":"2024-09-10T21:10:31.165773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:43.846568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168511+0000","last_change":"2024-09-10T21:10:30.180534+0000","last_active":"2024-09-10T21:10:31.168511+0000","last_peered":"2024-09-10T21:10:31.168511+0000","last_clean":"2024-09-10T21:10:31.168511+0000","last_became_active":"2024-09-10T21:10:30.180196+0000","last_became_peered":"2024-09-10T21:10:30.180196+0000","last_unstale":"2024-09-10T21:10:31.168511+0000","last_undegraded":"2024-09-10T21:10:31.168511+0000","last_fullsized":"2024-09-10T21:10:31.168511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:04.038591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179548+0000","last_change":"2024-09-10T21:10:30.191472+0000","last_active":"2024-09-10T21:10:31.179548+0000","last_peered":"2024-09-10T21:10:31.179548+0000","last_clean":"2024-09-10T21:10:31.179548+0000","last_became_active":"2024-09-10T21:10:30.182657+0000","last_became_peered":"2024-09-10T21:10:30.182657+0000","last_unstale":"2024-09-10T21:10:31.179548+0000","last_undegraded":"2024-09-10T21:10:31.179548+0000","last_fullsized":"2024-09-10T21:10:31.179548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:42.385122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179302+0000","last_change":"2024-09-10T21:10:30.196473+0000","last_active":"2024-09-10T21:10:31.179302+0000","last_peered":"2024-09-10T21:10:31.179302+0000","last_clean":"2024-09-10T21:10:31.179302+0000","last_became_active":"2024-09-10T21:10:30.195712+0000","last_became_peered":"2024-09-10T21:10:30.195712+0000","last_unstale":"2024-09-10T21:10:31.179302+0000","last_undegraded":"2024-09-10T21:10:31.179302+0000","last_fullsized":"2024-09-10T21:10:31.179302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:14.592947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179252+0000","last_change":"2024-09-10T21:10:30.187870+0000","last_active":"2024-09-10T21:10:31.179252+0000","last_peered":"2024-09-10T21:10:31.179252+0000","last_clean":"2024-09-10T21:10:31.179252+0000","last_became_active":"2024-09-10T21:10:30.186791+0000","last_became_peered":"2024-09-10T21:10:30.186791+0000","last_unstale":"2024-09-10T21:10:31.179252+0000","last_undegraded":"2024-09-10T21:10:31.179252+0000","last_fullsized":"2024-09-10T21:10:31.179252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:49.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707773+0000","last_change":"2024-09-10T21:10:30.174269+0000","last_active":"2024-09-10T21:10:31.707773+0000","last_peered":"2024-09-10T21:10:31.707773+0000","last_clean":"2024-09-10T21:10:31.707773+0000","last_became_active":"2024-09-10T21:10:30.173714+0000","last_became_peered":"2024-09-10T21:10:30.173714+0000","last_unstale":"2024-09-10T21:10:31.707773+0000","last_undegraded":"2024-09-10T21:10:31.707773+0000","last_fullsized":"2024-09-10T21:10:31.707773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:30.207193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707685+0000","last_change":"2024-09-10T21:10:30.172867+0000","last_active":"2024-09-10T21:10:31.707685+0000","last_peered":"2024-09-10T21:10:31.707685+0000","last_clean":"2024-09-10T21:10:31.707685+0000","last_became_active":"2024-09-10T21:10:30.172301+0000","last_became_peered":"2024-09-10T21:10:30.172301+0000","last_unstale":"2024-09-10T21:10:31.707685+0000","last_undegraded":"2024-09-10T21:10:31.707685+0000","last_fullsized":"2024-09-10T21:10:31.707685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:32.205639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.165740+0000","last_change":"2024-09-10T21:10:30.181460+0000","last_active":"2024-09-10T21:10:31.165740+0000","last_peered":"2024-09-10T21:10:31.165740+0000","last_clean":"2024-09-10T21:10:31.165740+0000","last_became_active":"2024-09-10T21:10:30.175816+0000","last_became_peered":"2024-09-10T21:10:30.175816+0000","last_unstale":"2024-09-10T21:10:31.165740+0000","last_undegraded":"2024-09-10T21:10:31.165740+0000","last_fullsized":"2024-09-10T21:10:31.165740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:26.624987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708880+0000","last_change":"2024-09-10T21:10:30.185809+0000","last_active":"2024-09-10T21:10:31.708880+0000","last_peered":"2024-09-10T21:10:31.708880+0000","last_clean":"2024-09-10T21:10:31.708880+0000","last_became_active":"2024-09-10T21:10:30.185485+0000","last_became_peered":"2024-09-10T21:10:30.185485+0000","last_unstale":"2024-09-10T21:10:31.708880+0000","last_undegraded":"2024-09-10T21:10:31.708880+0000","last_fullsized":"2024-09-10T21:10:31.708880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:16.358565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709494+0000","last_change":"2024-09-10T21:10:30.196400+0000","last_active":"2024-09-10T21:10:31.709494+0000","last_peered":"2024-09-10T21:10:31.709494+0000","last_clean":"2024-09-10T21:10:31.709494+0000","last_became_active":"2024-09-10T21:10:30.194199+0000","last_became_peered":"2024-09-10T21:10:30.194199+0000","last_unstale":"2024-09-10T21:10:31.709494+0000","last_undegraded":"2024-09-10T21:10:31.709494+0000","last_fullsized":"2024-09-10T21:10:31.709494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:21.510129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446465+0000","last_change":"2024-09-10T21:10:30.169268+0000","last_active":"2024-09-10T21:10:31.446465+0000","last_peered":"2024-09-10T21:10:31.446465+0000","last_clean":"2024-09-10T21:10:31.446465+0000","last_became_active":"2024-09-10T21:10:30.168869+0000","last_became_peered":"2024-09-10T21:10:30.168869+0000","last_unstale":"2024-09-10T21:10:31.446465+0000","last_undegraded":"2024-09-10T21:10:31.446465+0000","last_fullsized":"2024-09-10T21:10:31.446465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:06.070406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709259+0000","last_change":"2024-09-10T21:10:30.180496+0000","last_active":"2024-09-10T21:10:31.709259+0000","last_peered":"2024-09-10T21:10:31.709259+0000","last_clean":"2024-09-10T21:10:31.709259+0000","last_became_active":"2024-09-10T21:10:30.180199+0000","last_became_peered":"2024-09-10T21:10:30.180199+0000","last_unstale":"2024-09-10T21:10:31.709259+0000","last_undegraded":"2024-09-10T21:10:31.709259+0000","last_fullsized":"2024-09-10T21:10:31.709259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:36.737204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179263+0000","last_change":"2024-09-10T21:10:30.187710+0000","last_active":"2024-09-10T21:10:31.179263+0000","last_peered":"2024-09-10T21:10:31.179263+0000","last_clean":"2024-09-10T21:10:31.179263+0000","last_became_active":"2024-09-10T21:10:30.186422+0000","last_became_peered":"2024-09-10T21:10:30.186422+0000","last_unstale":"2024-09-10T21:10:31.179263+0000","last_undegraded":"2024-09-10T21:10:31.179263+0000","last_fullsized":"2024-09-10T21:10:31.179263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:37.171374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173731+0000","last_change":"2024-09-10T21:10:30.157845+0000","last_active":"2024-09-10T21:10:31.173731+0000","last_peered":"2024-09-10T21:10:31.173731+0000","last_clean":"2024-09-10T21:10:31.173731+0000","last_became_active":"2024-09-10T21:10:30.155515+0000","last_became_peered":"2024-09-10T21:10:30.155515+0000","last_unstale":"2024-09-10T21:10:31.173731+0000","last_undegraded":"2024-09-10T21:10:31.173731+0000","last_fullsized":"2024-09-10T21:10:31.173731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:36.354773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167488+0000","last_change":"2024-09-10T21:10:30.168344+0000","last_active":"2024-09-10T21:10:31.167488+0000","last_peered":"2024-09-10T21:10:31.167488+0000","last_clean":"2024-09-10T21:10:31.167488+0000","last_became_active":"2024-09-10T21:10:30.167966+0000","last_became_peered":"2024-09-10T21:10:30.167966+0000","last_unstale":"2024-09-10T21:10:31.167488+0000","last_undegraded":"2024-09-10T21:10:31.167488+0000","last_fullsized":"2024-09-10T21:10:31.167488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:38.100207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166637+0000","last_change":"2024-09-10T21:10:30.159521+0000","last_active":"2024-09-10T21:10:31.166637+0000","last_peered":"2024-09-10T21:10:31.166637+0000","last_clean":"2024-09-10T21:10:31.166637+0000","last_became_active":"2024-09-10T21:10:30.159269+0000","last_became_peered":"2024-09-10T21:10:30.159269+0000","last_unstale":"2024-09-10T21:10:31.166637+0000","last_undegraded":"2024-09-10T21:10:31.166637+0000","last_fullsized":"2024-09-10T21:10:31.166637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:21.433157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708679+0000","last_change":"2024-09-10T21:10:30.182227+0000","last_active":"2024-09-10T21:10:31.708679+0000","last_peered":"2024-09-10T21:10:31.708679+0000","last_clean":"2024-09-10T21:10:31.708679+0000","last_became_active":"2024-09-10T21:10:30.178359+0000","last_became_peered":"2024-09-10T21:10:30.178359+0000","last_unstale":"2024-09-10T21:10:31.708679+0000","last_undegraded":"2024-09-10T21:10:31.708679+0000","last_fullsized":"2024-09-10T21:10:31.708679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:50.308163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166494+0000","last_change":"2024-09-10T21:10:30.170127+0000","last_active":"2024-09-10T21:10:31.166494+0000","last_peered":"2024-09-10T21:10:31.166494+0000","last_clean":"2024-09-10T21:10:31.166494+0000","last_became_active":"2024-09-10T21:10:30.169781+0000","last_became_peered":"2024-09-10T21:10:30.169781+0000","last_unstale":"2024-09-10T21:10:31.166494+0000","last_undegraded":"2024-09-10T21:10:31.166494+0000","last_fullsized":"2024-09-10T21:10:31.166494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:16.729840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167717+0000","last_change":"2024-09-10T21:10:30.161842+0000","last_active":"2024-09-10T21:10:31.167717+0000","last_peered":"2024-09-10T21:10:31.167717+0000","last_clean":"2024-09-10T21:10:31.167717+0000","last_became_active":"2024-09-10T21:10:30.160146+0000","last_became_peered":"2024-09-10T21:10:30.160146+0000","last_unstale":"2024-09-10T21:10:31.167717+0000","last_undegraded":"2024-09-10T21:10:31.167717+0000","last_fullsized":"2024-09-10T21:10:31.167717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:06.600402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707965+0000","last_change":"2024-09-10T21:10:30.176930+0000","last_active":"2024-09-10T21:10:31.707965+0000","last_peered":"2024-09-10T21:10:31.707965+0000","last_clean":"2024-09-10T21:10:31.707965+0000","last_became_active":"2024-09-10T21:10:30.175522+0000","last_became_peered":"2024-09-10T21:10:30.175522+0000","last_unstale":"2024-09-10T21:10:31.707965+0000","last_undegraded":"2024-09-10T21:10:31.707965+0000","last_fullsized":"2024-09-10T21:10:31.707965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:36.354773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707870+0000","last_change":"2024-09-10T21:10:30.179884+0000","last_active":"2024-09-10T21:10:31.707870+0000","last_peered":"2024-09-10T21:10:31.707870+0000","last_clean":"2024-09-10T21:10:31.707870+0000","last_became_active":"2024-09-10T21:10:30.177866+0000","last_became_peered":"2024-09-10T21:10:30.177866+0000","last_unstale":"2024-09-10T21:10:31.707870+0000","last_undegraded":"2024-09-10T21:10:31.707870+0000","last_fullsized":"2024-09-10T21:10:31.707870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:57.957611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166067+0000","last_change":"2024-09-10T21:10:30.169322+0000","last_active":"2024-09-10T21:10:31.166067+0000","last_peered":"2024-09-10T21:10:31.166067+0000","last_clean":"2024-09-10T21:10:31.166067+0000","last_became_active":"2024-09-10T21:10:30.169030+0000","last_became_peered":"2024-09-10T21:10:30.169030+0000","last_unstale":"2024-09-10T21:10:31.166067+0000","last_undegraded":"2024-09-10T21:10:31.166067+0000","last_fullsized":"2024-09-10T21:10:31.166067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:41.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708760+0000","last_change":"2024-09-10T21:10:30.194536+0000","last_active":"2024-09-10T21:10:31.708760+0000","last_peered":"2024-09-10T21:10:31.708760+0000","last_clean":"2024-09-10T21:10:31.708760+0000","last_became_active":"2024-09-10T21:10:30.188848+0000","last_became_peered":"2024-09-10T21:10:30.188848+0000","last_unstale":"2024-09-10T21:10:31.708760+0000","last_undegraded":"2024-09-10T21:10:31.708760+0000","last_fullsized":"2024-09-10T21:10:31.708760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:31.264063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446249+0000","last_change":"2024-09-10T21:10:30.171833+0000","last_active":"2024-09-10T21:10:31.446249+0000","last_peered":"2024-09-10T21:10:31.446249+0000","last_clean":"2024-09-10T21:10:31.446249+0000","last_became_active":"2024-09-10T21:10:30.171461+0000","last_became_peered":"2024-09-10T21:10:30.171461+0000","last_unstale":"2024-09-10T21:10:31.446249+0000","last_undegraded":"2024-09-10T21:10:31.446249+0000","last_fullsized":"2024-09-10T21:10:31.446249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:32.018476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166344+0000","last_change":"2024-09-10T21:10:30.183415+0000","last_active":"2024-09-10T21:10:31.166344+0000","last_peered":"2024-09-10T21:10:31.166344+0000","last_clean":"2024-09-10T21:10:31.166344+0000","last_became_active":"2024-09-10T21:10:30.183093+0000","last_became_peered":"2024-09-10T21:10:30.183093+0000","last_unstale":"2024-09-10T21:10:31.166344+0000","last_undegraded":"2024-09-10T21:10:31.166344+0000","last_fullsized":"2024-09-10T21:10:31.166344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:30.558298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167929+0000","last_change":"2024-09-10T21:10:30.171980+0000","last_active":"2024-09-10T21:10:31.167929+0000","last_peered":"2024-09-10T21:10:31.167929+0000","last_clean":"2024-09-10T21:10:31.167929+0000","last_became_active":"2024-09-10T21:10:30.167136+0000","last_became_peered":"2024-09-10T21:10:30.167136+0000","last_unstale":"2024-09-10T21:10:31.167929+0000","last_undegraded":"2024-09-10T21:10:31.167929+0000","last_fullsized":"2024-09-10T21:10:31.167929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:33.142099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166187+0000","last_change":"2024-09-10T21:10:29.171062+0000","last_active":"2024-09-10T21:10:31.166187+0000","last_peered":"2024-09-10T21:10:31.166187+0000","last_clean":"2024-09-10T21:10:31.166187+0000","last_became_active":"2024-09-10T21:10:29.170765+0000","last_became_peered":"2024-09-10T21:10:29.170765+0000","last_unstale":"2024-09-10T21:10:31.166187+0000","last_undegraded":"2024-09-10T21:10:31.166187+0000","last_fullsized":"2024-09-10T21:10:31.166187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.425174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173821+0000","last_change":"2024-09-10T21:10:30.174660+0000","last_active":"2024-09-10T21:10:31.173821+0000","last_peered":"2024-09-10T21:10:31.173821+0000","last_clean":"2024-09-10T21:10:31.173821+0000","last_became_active":"2024-09-10T21:10:30.174372+0000","last_became_peered":"2024-09-10T21:10:30.174372+0000","last_unstale":"2024-09-10T21:10:31.173821+0000","last_undegraded":"2024-09-10T21:10:31.173821+0000","last_fullsized":"2024-09-10T21:10:31.173821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:59.722000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708981+0000","last_change":"2024-09-10T21:10:29.171541+0000","last_active":"2024-09-10T21:10:31.708981+0000","last_peered":"2024-09-10T21:10:31.708981+0000","last_clean":"2024-09-10T21:10:31.708981+0000","last_became_active":"2024-09-10T21:10:29.171269+0000","last_became_peered":"2024-09-10T21:10:29.171269+0000","last_unstale":"2024-09-10T21:10:31.708981+0000","last_undegraded":"2024-09-10T21:10:31.708981+0000","last_fullsized":"2024-09-10T21:10:31.708981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:10.046262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707383+0000","last_change":"2024-09-10T21:10:29.166482+0000","last_active":"2024-09-10T21:10:31.707383+0000","last_peered":"2024-09-10T21:10:31.707383+0000","last_clean":"2024-09-10T21:10:31.707383+0000","last_became_active":"2024-09-10T21:10:29.166091+0000","last_became_peered":"2024-09-10T21:10:29.166091+0000","last_unstale":"2024-09-10T21:10:31.707383+0000","last_undegraded":"2024-09-10T21:10:31.707383+0000","last_fullsized":"2024-09-10T21:10:31.707383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.425174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166297+0000","last_change":"2024-09-10T21:10:30.183180+0000","last_active":"2024-09-10T21:10:31.166297+0000","last_peered":"2024-09-10T21:10:31.166297+0000","last_clean":"2024-09-10T21:10:31.166297+0000","last_became_active":"2024-09-10T21:10:30.182908+0000","last_became_peered":"2024-09-10T21:10:30.182908+0000","last_unstale":"2024-09-10T21:10:31.166297+0000","last_undegraded":"2024-09-10T21:10:31.166297+0000","last_fullsized":"2024-09-10T21:10:31.166297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:37.697114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708748+0000","last_change":"2024-09-10T21:10:29.167898+0000","last_active":"2024-09-10T21:10:31.708748+0000","last_peered":"2024-09-10T21:10:31.708748+0000","last_clean":"2024-09-10T21:10:31.708748+0000","last_became_active":"2024-09-10T21:10:29.167498+0000","last_became_peered":"2024-09-10T21:10:29.167498+0000","last_unstale":"2024-09-10T21:10:31.708748+0000","last_undegraded":"2024-09-10T21:10:31.708748+0000","last_fullsized":"2024-09-10T21:10:31.708748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:50.337610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173782+0000","last_change":"2024-09-10T21:10:30.161329+0000","last_active":"2024-09-10T21:10:31.173782+0000","last_peered":"2024-09-10T21:10:31.173782+0000","last_clean":"2024-09-10T21:10:31.173782+0000","last_became_active":"2024-09-10T21:10:30.160309+0000","last_became_peered":"2024-09-10T21:10:30.160309+0000","last_unstale":"2024-09-10T21:10:31.173782+0000","last_undegraded":"2024-09-10T21:10:31.173782+0000","last_fullsized":"2024-09-10T21:10:31.173782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:14.235339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168376+0000","last_change":"2024-09-10T21:10:29.170975+0000","last_active":"2024-09-10T21:10:31.168376+0000","last_peered":"2024-09-10T21:10:31.168376+0000","last_clean":"2024-09-10T21:10:31.168376+0000","last_became_active":"2024-09-10T21:10:29.170623+0000","last_became_peered":"2024-09-10T21:10:29.170623+0000","last_unstale":"2024-09-10T21:10:31.168376+0000","last_undegraded":"2024-09-10T21:10:31.168376+0000","last_fullsized":"2024-09-10T21:10:31.168376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167790+0000","last_change":"2024-09-10T21:10:30.160525+0000","last_active":"2024-09-10T21:10:31.167790+0000","last_peered":"2024-09-10T21:10:31.167790+0000","last_clean":"2024-09-10T21:10:31.167790+0000","last_became_active":"2024-09-10T21:10:30.159639+0000","last_became_peered":"2024-09-10T21:10:30.159639+0000","last_unstale":"2024-09-10T21:10:31.167790+0000","last_undegraded":"2024-09-10T21:10:31.167790+0000","last_fullsized":"2024-09-10T21:10:31.167790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:46.116954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179609+0000","last_change":"2024-09-10T21:10:29.175134+0000","last_active":"2024-09-10T21:10:31.179609+0000","last_peered":"2024-09-10T21:10:31.179609+0000","last_clean":"2024-09-10T21:10:31.179609+0000","last_became_active":"2024-09-10T21:10:29.174808+0000","last_became_peered":"2024-09-10T21:10:29.174808+0000","last_unstale":"2024-09-10T21:10:31.179609+0000","last_undegraded":"2024-09-10T21:10:31.179609+0000","last_fullsized":"2024-09-10T21:10:31.179609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:52.586551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166756+0000","last_change":"2024-09-10T21:10:30.167445+0000","last_active":"2024-09-10T21:10:31.166756+0000","last_peered":"2024-09-10T21:10:31.166756+0000","last_clean":"2024-09-10T21:10:31.166756+0000","last_became_active":"2024-09-10T21:10:30.167163+0000","last_became_peered":"2024-09-10T21:10:30.167163+0000","last_unstale":"2024-09-10T21:10:31.166756+0000","last_undegraded":"2024-09-10T21:10:31.166756+0000","last_fullsized":"2024-09-10T21:10:31.166756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:59.291659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446376+0000","last_change":"2024-09-10T21:10:29.171850+0000","last_active":"2024-09-10T21:10:31.446376+0000","last_peered":"2024-09-10T21:10:31.446376+0000","last_clean":"2024-09-10T21:10:31.446376+0000","last_became_active":"2024-09-10T21:10:29.171589+0000","last_became_peered":"2024-09-10T21:10:29.171589+0000","last_unstale":"2024-09-10T21:10:31.446376+0000","last_undegraded":"2024-09-10T21:10:31.446376+0000","last_fullsized":"2024-09-10T21:10:31.446376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:43.143812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708938+0000","last_change":"2024-09-10T21:10:30.162228+0000","last_active":"2024-09-10T21:10:31.708938+0000","last_peered":"2024-09-10T21:10:31.708938+0000","last_clean":"2024-09-10T21:10:31.708938+0000","last_became_active":"2024-09-10T21:10:30.161901+0000","last_became_peered":"2024-09-10T21:10:30.161901+0000","last_unstale":"2024-09-10T21:10:31.708938+0000","last_undegraded":"2024-09-10T21:10:31.708938+0000","last_fullsized":"2024-09-10T21:10:31.708938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:12.489819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167129+0000","last_change":"2024-09-10T21:10:29.164653+0000","last_active":"2024-09-10T21:10:31.167129+0000","last_peered":"2024-09-10T21:10:31.167129+0000","last_clean":"2024-09-10T21:10:31.167129+0000","last_became_active":"2024-09-10T21:10:29.164338+0000","last_became_peered":"2024-09-10T21:10:29.164338+0000","last_unstale":"2024-09-10T21:10:31.167129+0000","last_undegraded":"2024-09-10T21:10:31.167129+0000","last_fullsized":"2024-09-10T21:10:31.167129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:24.324180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179343+0000","last_change":"2024-09-10T21:10:30.191023+0000","last_active":"2024-09-10T21:10:31.179343+0000","last_peered":"2024-09-10T21:10:31.179343+0000","last_clean":"2024-09-10T21:10:31.179343+0000","last_became_active":"2024-09-10T21:10:30.190264+0000","last_became_peered":"2024-09-10T21:10:30.190264+0000","last_unstale":"2024-09-10T21:10:31.179343+0000","last_undegraded":"2024-09-10T21:10:31.179343+0000","last_fullsized":"2024-09-10T21:10:31.179343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:08.381083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446251+0000","last_change":"2024-09-10T21:10:29.178312+0000","last_active":"2024-09-10T21:10:31.446251+0000","last_peered":"2024-09-10T21:10:31.446251+0000","last_clean":"2024-09-10T21:10:31.446251+0000","last_became_active":"2024-09-10T21:10:29.178044+0000","last_became_peered":"2024-09-10T21:10:29.178044+0000","last_unstale":"2024-09-10T21:10:31.446251+0000","last_undegraded":"2024-09-10T21:10:31.446251+0000","last_fullsized":"2024-09-10T21:10:31.446251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:41.871122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707682+0000","last_change":"2024-09-10T21:10:30.181507+0000","last_active":"2024-09-10T21:10:31.707682+0000","last_peered":"2024-09-10T21:10:31.707682+0000","last_clean":"2024-09-10T21:10:31.707682+0000","last_became_active":"2024-09-10T21:10:30.177520+0000","last_became_peered":"2024-09-10T21:10:30.177520+0000","last_unstale":"2024-09-10T21:10:31.707682+0000","last_undegraded":"2024-09-10T21:10:31.707682+0000","last_fullsized":"2024-09-10T21:10:31.707682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:56.567997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709035+0000","last_change":"2024-09-10T21:10:29.166737+0000","last_active":"2024-09-10T21:10:31.709035+0000","last_peered":"2024-09-10T21:10:31.709035+0000","last_clean":"2024-09-10T21:10:31.709035+0000","last_became_active":"2024-09-10T21:10:29.166301+0000","last_became_peered":"2024-09-10T21:10:29.166301+0000","last_unstale":"2024-09-10T21:10:31.709035+0000","last_undegraded":"2024-09-10T21:10:31.709035+0000","last_fullsized":"2024-09-10T21:10:31.709035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:52.586551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709252+0000","last_change":"2024-09-10T21:10:30.180470+0000","last_active":"2024-09-10T21:10:31.709252+0000","last_peered":"2024-09-10T21:10:31.709252+0000","last_clean":"2024-09-10T21:10:31.709252+0000","last_became_active":"2024-09-10T21:10:30.179447+0000","last_became_peered":"2024-09-10T21:10:30.179447+0000","last_unstale":"2024-09-10T21:10:31.709252+0000","last_undegraded":"2024-09-10T21:10:31.709252+0000","last_fullsized":"2024-09-10T21:10:31.709252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:36.701958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178781+0000","last_change":"2024-09-10T21:10:29.169914+0000","last_active":"2024-09-10T21:10:31.178781+0000","last_peered":"2024-09-10T21:10:31.178781+0000","last_clean":"2024-09-10T21:10:31.178781+0000","last_became_active":"2024-09-10T21:10:29.169617+0000","last_became_peered":"2024-09-10T21:10:29.169617+0000","last_unstale":"2024-09-10T21:10:31.178781+0000","last_undegraded":"2024-09-10T21:10:31.178781+0000","last_fullsized":"2024-09-10T21:10:31.178781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:48.286592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167103+0000","last_change":"2024-09-10T21:10:30.169299+0000","last_active":"2024-09-10T21:10:31.167103+0000","last_peered":"2024-09-10T21:10:31.167103+0000","last_clean":"2024-09-10T21:10:31.167103+0000","last_became_active":"2024-09-10T21:10:30.169013+0000","last_became_peered":"2024-09-10T21:10:30.169013+0000","last_unstale":"2024-09-10T21:10:31.167103+0000","last_undegraded":"2024-09-10T21:10:31.167103+0000","last_fullsized":"2024-09-10T21:10:31.167103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:40.546116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173457+0000","last_change":"2024-09-10T21:10:29.162746+0000","last_active":"2024-09-10T21:10:31.173457+0000","last_peered":"2024-09-10T21:10:31.173457+0000","last_clean":"2024-09-10T21:10:31.173457+0000","last_became_active":"2024-09-10T21:10:29.162466+0000","last_became_peered":"2024-09-10T21:10:29.162466+0000","last_unstale":"2024-09-10T21:10:31.173457+0000","last_undegraded":"2024-09-10T21:10:31.173457+0000","last_fullsized":"2024-09-10T21:10:31.173457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.425174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708924+0000","last_change":"2024-09-10T21:10:30.162565+0000","last_active":"2024-09-10T21:10:31.708924+0000","last_peered":"2024-09-10T21:10:31.708924+0000","last_clean":"2024-09-10T21:10:31.708924+0000","last_became_active":"2024-09-10T21:10:30.161927+0000","last_became_peered":"2024-09-10T21:10:30.161927+0000","last_unstale":"2024-09-10T21:10:31.708924+0000","last_undegraded":"2024-09-10T21:10:31.708924+0000","last_fullsized":"2024-09-10T21:10:31.708924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:43.633279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.180274+0000","last_change":"2024-09-10T21:10:29.170318+0000","last_active":"2024-09-10T21:10:31.180274+0000","last_peered":"2024-09-10T21:10:31.180274+0000","last_clean":"2024-09-10T21:10:31.180274+0000","last_became_active":"2024-09-10T21:10:29.169096+0000","last_became_peered":"2024-09-10T21:10:29.169096+0000","last_unstale":"2024-09-10T21:10:31.180274+0000","last_undegraded":"2024-09-10T21:10:31.180274+0000","last_fullsized":"2024-09-10T21:10:31.180274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:01.835985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167829+0000","last_change":"2024-09-10T21:10:30.176716+0000","last_active":"2024-09-10T21:10:31.167829+0000","last_peered":"2024-09-10T21:10:31.167829+0000","last_clean":"2024-09-10T21:10:31.167829+0000","last_became_active":"2024-09-10T21:10:30.175255+0000","last_became_peered":"2024-09-10T21:10:30.175255+0000","last_unstale":"2024-09-10T21:10:31.167829+0000","last_undegraded":"2024-09-10T21:10:31.167829+0000","last_fullsized":"2024-09-10T21:10:31.167829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.859164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173973+0000","last_change":"2024-09-10T21:10:29.163718+0000","last_active":"2024-09-10T21:10:31.173973+0000","last_peered":"2024-09-10T21:10:31.173973+0000","last_clean":"2024-09-10T21:10:31.173973+0000","last_became_active":"2024-09-10T21:10:29.163328+0000","last_became_peered":"2024-09-10T21:10:29.163328+0000","last_unstale":"2024-09-10T21:10:31.173973+0000","last_undegraded":"2024-09-10T21:10:31.173973+0000","last_fullsized":"2024-09-10T21:10:31.173973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:57.741500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.180052+0000","last_change":"2024-09-10T21:10:30.183206+0000","last_active":"2024-09-10T21:10:31.180052+0000","last_peered":"2024-09-10T21:10:31.180052+0000","last_clean":"2024-09-10T21:10:31.180052+0000","last_became_active":"2024-09-10T21:10:30.182950+0000","last_became_peered":"2024-09-10T21:10:30.182950+0000","last_unstale":"2024-09-10T21:10:31.180052+0000","last_undegraded":"2024-09-10T21:10:31.180052+0000","last_fullsized":"2024-09-10T21:10:31.180052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:45.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166843+0000","last_change":"2024-09-10T21:10:29.165292+0000","last_active":"2024-09-10T21:10:31.166843+0000","last_peered":"2024-09-10T21:10:31.166843+0000","last_clean":"2024-09-10T21:10:31.166843+0000","last_became_active":"2024-09-10T21:10:29.164991+0000","last_became_peered":"2024-09-10T21:10:29.164991+0000","last_unstale":"2024-09-10T21:10:31.166843+0000","last_undegraded":"2024-09-10T21:10:31.166843+0000","last_fullsized":"2024-09-10T21:10:31.166843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:57.741500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179462+0000","last_change":"2024-09-10T21:10:30.188067+0000","last_active":"2024-09-10T21:10:31.179462+0000","last_peered":"2024-09-10T21:10:31.179462+0000","last_clean":"2024-09-10T21:10:31.179462+0000","last_became_active":"2024-09-10T21:10:30.187079+0000","last_became_peered":"2024-09-10T21:10:30.187079+0000","last_unstale":"2024-09-10T21:10:31.179462+0000","last_undegraded":"2024-09-10T21:10:31.179462+0000","last_fullsized":"2024-09-10T21:10:31.179462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:16.358565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707972+0000","last_change":"2024-09-10T21:10:29.168326+0000","last_active":"2024-09-10T21:10:31.707972+0000","last_peered":"2024-09-10T21:10:31.707972+0000","last_clean":"2024-09-10T21:10:31.707972+0000","last_became_active":"2024-09-10T21:10:29.168048+0000","last_became_peered":"2024-09-10T21:10:29.168048+0000","last_unstale":"2024-09-10T21:10:31.707972+0000","last_undegraded":"2024-09-10T21:10:31.707972+0000","last_fullsized":"2024-09-10T21:10:31.707972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:56.871372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709184+0000","last_change":"2024-09-10T21:10:30.179705+0000","last_active":"2024-09-10T21:10:31.709184+0000","last_peered":"2024-09-10T21:10:31.709184+0000","last_clean":"2024-09-10T21:10:31.709184+0000","last_became_active":"2024-09-10T21:10:30.175366+0000","last_became_peered":"2024-09-10T21:10:30.175366+0000","last_unstale":"2024-09-10T21:10:31.709184+0000","last_undegraded":"2024-09-10T21:10:31.709184+0000","last_fullsized":"2024-09-10T21:10:31.709184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:44.960366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167202+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:10:31.167202+0000","last_peered":"2024-09-10T21:10:31.167202+0000","last_clean":"2024-09-10T21:10:31.167202+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:10:31.167202+0000","last_undegraded":"2024-09-10T21:10:31.167202+0000","last_fullsized":"2024-09-10T21:10:31.167202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708999+0000","last_change":"2024-09-10T21:10:29.172283+0000","last_active":"2024-09-10T21:10:31.708999+0000","last_peered":"2024-09-10T21:10:31.708999+0000","last_clean":"2024-09-10T21:10:31.708999+0000","last_became_active":"2024-09-10T21:10:29.170550+0000","last_became_peered":"2024-09-10T21:10:29.170550+0000","last_unstale":"2024-09-10T21:10:31.708999+0000","last_undegraded":"2024-09-10T21:10:31.708999+0000","last_fullsized":"2024-09-10T21:10:31.708999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:09.439008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173734+0000","last_change":"2024-09-10T21:10:30.160275+0000","last_active":"2024-09-10T21:10:31.173734+0000","last_peered":"2024-09-10T21:10:31.173734+0000","last_clean":"2024-09-10T21:10:31.173734+0000","last_became_active":"2024-09-10T21:10:30.159817+0000","last_became_peered":"2024-09-10T21:10:30.159817+0000","last_unstale":"2024-09-10T21:10:31.173734+0000","last_undegraded":"2024-09-10T21:10:31.173734+0000","last_fullsized":"2024-09-10T21:10:31.173734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.698693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708902+0000","last_change":"2024-09-10T21:10:29.164034+0000","last_active":"2024-09-10T21:10:31.708902+0000","last_peered":"2024-09-10T21:10:31.708902+0000","last_clean":"2024-09-10T21:10:31.708902+0000","last_became_active":"2024-09-10T21:10:29.163755+0000","last_became_peered":"2024-09-10T21:10:29.163755+0000","last_unstale":"2024-09-10T21:10:31.708902+0000","last_undegraded":"2024-09-10T21:10:31.708902+0000","last_fullsized":"2024-09-10T21:10:31.708902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:10.200363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707745+0000","last_change":"2024-09-10T21:10:30.171490+0000","last_active":"2024-09-10T21:10:31.707745+0000","last_peered":"2024-09-10T21:10:31.707745+0000","last_clean":"2024-09-10T21:10:31.707745+0000","last_became_active":"2024-09-10T21:10:30.170892+0000","last_became_peered":"2024-09-10T21:10:30.170892+0000","last_unstale":"2024-09-10T21:10:31.707745+0000","last_undegraded":"2024-09-10T21:10:31.707745+0000","last_fullsized":"2024-09-10T21:10:31.707745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:36.425140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446253+0000","last_change":"2024-09-10T21:10:29.170636+0000","last_active":"2024-09-10T21:10:31.446253+0000","last_peered":"2024-09-10T21:10:31.446253+0000","last_clean":"2024-09-10T21:10:31.446253+0000","last_became_active":"2024-09-10T21:10:29.170342+0000","last_became_peered":"2024-09-10T21:10:29.170342+0000","last_unstale":"2024-09-10T21:10:31.446253+0000","last_undegraded":"2024-09-10T21:10:31.446253+0000","last_fullsized":"2024-09-10T21:10:31.446253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166047+0000","last_change":"2024-09-10T21:10:30.164807+0000","last_active":"2024-09-10T21:10:31.166047+0000","last_peered":"2024-09-10T21:10:31.166047+0000","last_clean":"2024-09-10T21:10:31.166047+0000","last_became_active":"2024-09-10T21:10:30.164545+0000","last_became_peered":"2024-09-10T21:10:30.164545+0000","last_unstale":"2024-09-10T21:10:31.166047+0000","last_undegraded":"2024-09-10T21:10:31.166047+0000","last_fullsized":"2024-09-10T21:10:31.166047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:35.647295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709113+0000","last_change":"2024-09-10T21:10:29.173240+0000","last_active":"2024-09-10T21:10:31.709113+0000","last_peered":"2024-09-10T21:10:31.709113+0000","last_clean":"2024-09-10T21:10:31.709113+0000","last_became_active":"2024-09-10T21:10:29.172991+0000","last_became_peered":"2024-09-10T21:10:29.172991+0000","last_unstale":"2024-09-10T21:10:31.709113+0000","last_undegraded":"2024-09-10T21:10:31.709113+0000","last_fullsized":"2024-09-10T21:10:31.709113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:00.278081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178965+0000","last_change":"2024-09-10T21:10:30.189010+0000","last_active":"2024-09-10T21:10:31.178965+0000","last_peered":"2024-09-10T21:10:31.178965+0000","last_clean":"2024-09-10T21:10:31.178965+0000","last_became_active":"2024-09-10T21:10:30.188712+0000","last_became_peered":"2024-09-10T21:10:30.188712+0000","last_unstale":"2024-09-10T21:10:31.178965+0000","last_undegraded":"2024-09-10T21:10:31.178965+0000","last_fullsized":"2024-09-10T21:10:31.178965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:04.853111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.165941+0000","last_change":"2024-09-10T21:10:29.173419+0000","last_active":"2024-09-10T21:10:31.165941+0000","last_peered":"2024-09-10T21:10:31.165941+0000","last_clean":"2024-09-10T21:10:31.165941+0000","last_became_active":"2024-09-10T21:10:29.173104+0000","last_became_peered":"2024-09-10T21:10:29.173104+0000","last_unstale":"2024-09-10T21:10:31.165941+0000","last_undegraded":"2024-09-10T21:10:31.165941+0000","last_fullsized":"2024-09-10T21:10:31.165941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:25.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167265+0000","last_change":"2024-09-10T21:10:30.169171+0000","last_active":"2024-09-10T21:10:31.167265+0000","last_peered":"2024-09-10T21:10:31.167265+0000","last_clean":"2024-09-10T21:10:31.167265+0000","last_became_active":"2024-09-10T21:10:30.168866+0000","last_became_peered":"2024-09-10T21:10:30.168866+0000","last_unstale":"2024-09-10T21:10:31.167265+0000","last_undegraded":"2024-09-10T21:10:31.167265+0000","last_fullsized":"2024-09-10T21:10:31.167265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:45.633966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709233+0000","last_change":"2024-09-10T21:10:29.170135+0000","last_active":"2024-09-10T21:10:31.709233+0000","last_peered":"2024-09-10T21:10:31.709233+0000","last_clean":"2024-09-10T21:10:31.709233+0000","last_became_active":"2024-09-10T21:10:29.169894+0000","last_became_peered":"2024-09-10T21:10:29.169894+0000","last_unstale":"2024-09-10T21:10:31.709233+0000","last_undegraded":"2024-09-10T21:10:31.709233+0000","last_fullsized":"2024-09-10T21:10:31.709233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:10.057351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707993+0000","last_change":"2024-09-10T21:10:30.171973+0000","last_active":"2024-09-10T21:10:31.707993+0000","last_peered":"2024-09-10T21:10:31.707993+0000","last_clean":"2024-09-10T21:10:31.707993+0000","last_became_active":"2024-09-10T21:10:30.171622+0000","last_became_peered":"2024-09-10T21:10:30.171622+0000","last_unstale":"2024-09-10T21:10:31.707993+0000","last_undegraded":"2024-09-10T21:10:31.707993+0000","last_fullsized":"2024-09-10T21:10:31.707993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:06.330104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174157+0000","last_change":"2024-09-10T21:10:29.165107+0000","last_active":"2024-09-10T21:10:31.174157+0000","last_peered":"2024-09-10T21:10:31.174157+0000","last_clean":"2024-09-10T21:10:31.174157+0000","last_became_active":"2024-09-10T21:10:29.164798+0000","last_became_peered":"2024-09-10T21:10:29.164798+0000","last_unstale":"2024-09-10T21:10:31.174157+0000","last_undegraded":"2024-09-10T21:10:31.174157+0000","last_fullsized":"2024-09-10T21:10:31.174157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:04.055586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174286+0000","last_change":"2024-09-10T21:10:30.173086+0000","last_active":"2024-09-10T21:10:31.174286+0000","last_peered":"2024-09-10T21:10:31.174286+0000","last_clean":"2024-09-10T21:10:31.174286+0000","last_became_active":"2024-09-10T21:10:30.172730+0000","last_became_peered":"2024-09-10T21:10:30.172730+0000","last_unstale":"2024-09-10T21:10:31.174286+0000","last_undegraded":"2024-09-10T21:10:31.174286+0000","last_fullsized":"2024-09-10T21:10:31.174286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:50.512074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709420+0000","last_change":"2024-09-10T21:10:29.179231+0000","last_active":"2024-09-10T21:10:31.709420+0000","last_peered":"2024-09-10T21:10:31.709420+0000","last_clean":"2024-09-10T21:10:31.709420+0000","last_became_active":"2024-09-10T21:10:29.178937+0000","last_became_peered":"2024-09-10T21:10:29.178937+0000","last_unstale":"2024-09-10T21:10:31.709420+0000","last_undegraded":"2024-09-10T21:10:31.709420+0000","last_fullsized":"2024-09-10T21:10:31.709420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:41.528182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174168+0000","last_change":"2024-09-10T21:10:30.167226+0000","last_active":"2024-09-10T21:10:31.174168+0000","last_peered":"2024-09-10T21:10:31.174168+0000","last_clean":"2024-09-10T21:10:31.174168+0000","last_became_active":"2024-09-10T21:10:30.166959+0000","last_became_peered":"2024-09-10T21:10:30.166959+0000","last_unstale":"2024-09-10T21:10:31.174168+0000","last_undegraded":"2024-09-10T21:10:31.174168+0000","last_fullsized":"2024-09-10T21:10:31.174168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:43.225225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178611+0000","last_change":"2024-09-10T21:10:29.175872+0000","last_active":"2024-09-10T21:10:31.178611+0000","last_peered":"2024-09-10T21:10:31.178611+0000","last_clean":"2024-09-10T21:10:31.178611+0000","last_became_active":"2024-09-10T21:10:29.175659+0000","last_became_peered":"2024-09-10T21:10:29.175659+0000","last_unstale":"2024-09-10T21:10:31.178611+0000","last_undegraded":"2024-09-10T21:10:31.178611+0000","last_fullsized":"2024-09-10T21:10:31.178611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:21.990689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708884+0000","last_change":"2024-09-10T21:10:30.179720+0000","last_active":"2024-09-10T21:10:31.708884+0000","last_peered":"2024-09-10T21:10:31.708884+0000","last_clean":"2024-09-10T21:10:31.708884+0000","last_became_active":"2024-09-10T21:10:30.179435+0000","last_became_peered":"2024-09-10T21:10:30.179435+0000","last_unstale":"2024-09-10T21:10:31.708884+0000","last_undegraded":"2024-09-10T21:10:31.708884+0000","last_fullsized":"2024-09-10T21:10:31.708884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:45.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173726+0000","last_change":"2024-09-10T21:10:29.165142+0000","last_active":"2024-09-10T21:10:31.173726+0000","last_peered":"2024-09-10T21:10:31.173726+0000","last_clean":"2024-09-10T21:10:31.173726+0000","last_became_active":"2024-09-10T21:10:29.164732+0000","last_became_peered":"2024-09-10T21:10:29.164732+0000","last_unstale":"2024-09-10T21:10:31.173726+0000","last_undegraded":"2024-09-10T21:10:31.173726+0000","last_fullsized":"2024-09-10T21:10:31.173726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:26.261468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173644+0000","last_change":"2024-09-10T21:10:30.164096+0000","last_active":"2024-09-10T21:10:31.173644+0000","last_peered":"2024-09-10T21:10:31.173644+0000","last_clean":"2024-09-10T21:10:31.173644+0000","last_became_active":"2024-09-10T21:10:30.163474+0000","last_became_peered":"2024-09-10T21:10:30.163474+0000","last_unstale":"2024-09-10T21:10:31.173644+0000","last_undegraded":"2024-09-10T21:10:31.173644+0000","last_fullsized":"2024-09-10T21:10:31.173644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:57.957611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707812+0000","last_change":"2024-09-10T21:10:29.166991+0000","last_active":"2024-09-10T21:10:31.707812+0000","last_peered":"2024-09-10T21:10:31.707812+0000","last_clean":"2024-09-10T21:10:31.707812+0000","last_became_active":"2024-09-10T21:10:29.166675+0000","last_became_peered":"2024-09-10T21:10:29.166675+0000","last_unstale":"2024-09-10T21:10:31.707812+0000","last_undegraded":"2024-09-10T21:10:31.707812+0000","last_fullsized":"2024-09-10T21:10:31.707812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:25.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446431+0000","last_change":"2024-09-10T21:10:30.176046+0000","last_active":"2024-09-10T21:10:31.446431+0000","last_peered":"2024-09-10T21:10:31.446431+0000","last_clean":"2024-09-10T21:10:31.446431+0000","last_became_active":"2024-09-10T21:10:30.175784+0000","last_became_peered":"2024-09-10T21:10:30.175784+0000","last_unstale":"2024-09-10T21:10:31.446431+0000","last_undegraded":"2024-09-10T21:10:31.446431+0000","last_fullsized":"2024-09-10T21:10:31.446431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:37.972763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708858+0000","last_change":"2024-09-10T21:10:29.172106+0000","last_active":"2024-09-10T21:10:31.708858+0000","last_peered":"2024-09-10T21:10:31.708858+0000","last_clean":"2024-09-10T21:10:31.708858+0000","last_became_active":"2024-09-10T21:10:29.167132+0000","last_became_peered":"2024-09-10T21:10:29.167132+0000","last_unstale":"2024-09-10T21:10:31.708858+0000","last_undegraded":"2024-09-10T21:10:31.708858+0000","last_fullsized":"2024-09-10T21:10:31.708858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:48.493501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708898+0000","last_change":"2024-09-10T21:10:30.176689+0000","last_active":"2024-09-10T21:10:31.708898+0000","last_peered":"2024-09-10T21:10:31.708898+0000","last_clean":"2024-09-10T21:10:31.708898+0000","last_became_active":"2024-09-10T21:10:30.176361+0000","last_became_peered":"2024-09-10T21:10:30.176361+0000","last_unstale":"2024-09-10T21:10:31.708898+0000","last_undegraded":"2024-09-10T21:10:31.708898+0000","last_fullsized":"2024-09-10T21:10:31.708898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:55.720343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709088+0000","last_change":"2024-09-10T21:10:30.180387+0000","last_active":"2024-09-10T21:10:31.709088+0000","last_peered":"2024-09-10T21:10:31.709088+0000","last_clean":"2024-09-10T21:10:31.709088+0000","last_became_active":"2024-09-10T21:10:30.180100+0000","last_became_peered":"2024-09-10T21:10:30.180100+0000","last_unstale":"2024-09-10T21:10:31.709088+0000","last_undegraded":"2024-09-10T21:10:31.709088+0000","last_fullsized":"2024-09-10T21:10:31.709088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:05.939523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179323+0000","last_change":"2024-09-10T21:10:29.178074+0000","last_active":"2024-09-10T21:10:31.179323+0000","last_peered":"2024-09-10T21:10:31.179323+0000","last_clean":"2024-09-10T21:10:31.179323+0000","last_became_active":"2024-09-10T21:10:29.177794+0000","last_became_peered":"2024-09-10T21:10:29.177794+0000","last_unstale":"2024-09-10T21:10:31.179323+0000","last_undegraded":"2024-09-10T21:10:31.179323+0000","last_fullsized":"2024-09-10T21:10:31.179323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:18.267264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167690+0000","last_change":"2024-09-10T21:10:30.179424+0000","last_active":"2024-09-10T21:10:31.167690+0000","last_peered":"2024-09-10T21:10:31.167690+0000","last_clean":"2024-09-10T21:10:31.167690+0000","last_became_active":"2024-09-10T21:10:30.178557+0000","last_became_peered":"2024-09-10T21:10:30.178557+0000","last_unstale":"2024-09-10T21:10:31.167690+0000","last_undegraded":"2024-09-10T21:10:31.167690+0000","last_fullsized":"2024-09-10T21:10:31.167690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:14.118820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179366+0000","last_change":"2024-09-10T21:10:29.169287+0000","last_active":"2024-09-10T21:10:31.179366+0000","last_peered":"2024-09-10T21:10:31.179366+0000","last_clean":"2024-09-10T21:10:31.179366+0000","last_became_active":"2024-09-10T21:10:29.169010+0000","last_became_peered":"2024-09-10T21:10:29.169010+0000","last_unstale":"2024-09-10T21:10:31.179366+0000","last_undegraded":"2024-09-10T21:10:31.179366+0000","last_fullsized":"2024-09-10T21:10:31.179366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:41.528182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166555+0000","last_change":"2024-09-10T21:10:30.166099+0000","last_active":"2024-09-10T21:10:31.166555+0000","last_peered":"2024-09-10T21:10:31.166555+0000","last_clean":"2024-09-10T21:10:31.166555+0000","last_became_active":"2024-09-10T21:10:30.165762+0000","last_became_peered":"2024-09-10T21:10:30.165762+0000","last_unstale":"2024-09-10T21:10:31.166555+0000","last_undegraded":"2024-09-10T21:10:31.166555+0000","last_fullsized":"2024-09-10T21:10:31.166555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:54.002262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166762+0000","last_change":"2024-09-10T21:10:29.167583+0000","last_active":"2024-09-10T21:10:31.166762+0000","last_peered":"2024-09-10T21:10:31.166762+0000","last_clean":"2024-09-10T21:10:31.166762+0000","last_became_active":"2024-09-10T21:10:29.162012+0000","last_became_peered":"2024-09-10T21:10:29.162012+0000","last_unstale":"2024-09-10T21:10:31.166762+0000","last_undegraded":"2024-09-10T21:10:31.166762+0000","last_fullsized":"2024-09-10T21:10:31.166762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:04.055586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166801+0000","last_change":"2024-09-10T21:10:30.162898+0000","last_active":"2024-09-10T21:10:31.166801+0000","last_peered":"2024-09-10T21:10:31.166801+0000","last_clean":"2024-09-10T21:10:31.166801+0000","last_became_active":"2024-09-10T21:10:30.162536+0000","last_became_peered":"2024-09-10T21:10:30.162536+0000","last_unstale":"2024-09-10T21:10:31.166801+0000","last_undegraded":"2024-09-10T21:10:31.166801+0000","last_fullsized":"2024-09-10T21:10:31.166801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:09.399612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174191+0000","last_change":"2024-09-10T21:10:29.166448+0000","last_active":"2024-09-10T21:10:31.174191+0000","last_peered":"2024-09-10T21:10:31.174191+0000","last_clean":"2024-09-10T21:10:31.174191+0000","last_became_active":"2024-09-10T21:10:29.166153+0000","last_became_peered":"2024-09-10T21:10:29.166153+0000","last_unstale":"2024-09-10T21:10:31.174191+0000","last_undegraded":"2024-09-10T21:10:31.174191+0000","last_fullsized":"2024-09-10T21:10:31.174191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:08.960569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":811008,"data_stored":662647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":74471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":72769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":217088,"data_stored":72243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":806912,"data_stored":662625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:10:39.318 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:10:39.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-10T21:10:39.319 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T21:10:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.498+0000 7f2b62800640 1 -- 172.21.15.16:0/1692981781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b5c0cb150 msgr2=0x7f2b5c0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.498+0000 7f2b62800640 1 --2- 172.21.15.16:0/1692981781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b5c0cb150 0x7f2b5c0cff20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009bc0 tx=0x7f2b5401cb70 comp rx=0 tx=0).stop 2024-09-10T21:10:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.498+0000 7f2b62800640 1 -- 172.21.15.16:0/1692981781 shutdown_connections 2024-09-10T21:10:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.498+0000 7f2b62800640 1 --2- 172.21.15.16:0/1692981781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b5c0cb150 0x7f2b5c0cff20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.498+0000 7f2b62800640 1 -- 172.21.15.16:0/1692981781 >> 172.21.15.16:0/1692981781 conn(0x7f2b5c0d0b00 msgr2=0x7f2b5c0d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:39.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.498+0000 7f2b62800640 1 -- 172.21.15.16:0/1692981781 shutdown_connections 2024-09-10T21:10:39.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.498+0000 7f2b62800640 1 -- 172.21.15.16:0/1692981781 wait complete. 2024-09-10T21:10:39.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b62800640 1 Processor -- start 2024-09-10T21:10:39.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b62800640 1 -- start start 2024-09-10T21:10:39.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b62800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b5c071780 0x7f2b5c0a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:39.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b62800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b5c0cb150 0x7f2b5c0af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:39.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b62800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b5c0c3be0 0x7f2b5c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:39.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b62800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c15a4e0 con 0x7f2b5c0cb150 2024-09-10T21:10:39.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b62800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c15a360 con 0x7f2b5c0c3be0 2024-09-10T21:10:39.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b62800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5c15a660 con 0x7f2b5c071780 2024-09-10T21:10:39.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b60a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b5c0cb150 0x7f2b5c0af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b60a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b5c0cb150 0x7f2b5c0af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38414/0 (socket says 172.21.15.16:38414) 2024-09-10T21:10:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b60a00640 1 -- 172.21.15.16:0/3369818992 learned_addr learned my addr 172.21.15.16:0/3369818992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b61400640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b5c071780 0x7f2b5c0a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b61e00640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b5c0c3be0 0x7f2b5c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b60a00640 1 -- 172.21.15.16:0/3369818992 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b5c071780 msgr2=0x7f2b5c0a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b60a00640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b5c071780 0x7f2b5c0a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b60a00640 1 -- 172.21.15.16:0/3369818992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b5c0c3be0 msgr2=0x7f2b5c15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b60a00640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b5c0c3be0 0x7f2b5c15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b60a00640 1 -- 172.21.15.16:0/3369818992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5c09fb60 con 0x7f2b5c0cb150 2024-09-10T21:10:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b61400640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b5c071780 0x7f2b5c0a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b61e00640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b5c0c3be0 0x7f2b5c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b60a00640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b5c0cb150 0x7f2b5c0af8c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800c4b0 tx=0x7f2b4800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b59600640 1 -- 172.21.15.16:0/3369818992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b480052d0 con 0x7f2b5c0cb150 2024-09-10T21:10:39.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b59600640 1 -- 172.21.15.16:0/3369818992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b48005470 con 0x7f2b5c0cb150 2024-09-10T21:10:39.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b62800640 1 -- 172.21.15.16:0/3369818992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5c09fe40 con 0x7f2b5c0cb150 2024-09-10T21:10:39.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b59600640 1 -- 172.21.15.16:0/3369818992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48012850 con 0x7f2b5c0cb150 2024-09-10T21:10:39.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b62800640 1 -- 172.21.15.16:0/3369818992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b5c0a0400 con 0x7f2b5c0cb150 2024-09-10T21:10:39.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.502+0000 7f2b62800640 1 -- 172.21.15.16:0/3369818992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b5c0b0300 con 0x7f2b5c0cb150 2024-09-10T21:10:39.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.506+0000 7f2b59600640 1 -- 172.21.15.16:0/3369818992 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b48003600 con 0x7f2b5c0cb150 2024-09-10T21:10:39.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.506+0000 7f2b59600640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b3c072610 0x7f2b3c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:39.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.506+0000 7f2b59600640 1 -- 172.21.15.16:0/3369818992 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2b48092e80 con 0x7f2b5c0cb150 2024-09-10T21:10:39.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.506+0000 7f2b61400640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b3c072610 0x7f2b3c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:39.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.506+0000 7f2b61400640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b3c072610 0x7f2b3c074ad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2b50000f70 tx=0x7f2b50009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:39.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.510+0000 7f2b59600640 1 -- 172.21.15.16:0/3369818992 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b4805e430 con 0x7f2b5c0cb150 2024-09-10T21:10:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b62800640 1 -- 172.21.15.16:0/3369818992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f2b5c0d19d0 con 0x7f2b5c0cb150 2024-09-10T21:10:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b59600640 1 -- 172.21.15.16:0/3369818992 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7f2b4805dd30 con 0x7f2b5c0cb150 2024-09-10T21:10:39.690 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:39.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b62800640 1 -- 172.21.15.16:0/3369818992 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b3c072610 msgr2=0x7f2b3c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:39.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b62800640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b3c072610 0x7f2b3c074ad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2b50000f70 tx=0x7f2b50009040 comp rx=0 tx=0).stop 2024-09-10T21:10:39.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b62800640 1 -- 172.21.15.16:0/3369818992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b5c0cb150 msgr2=0x7f2b5c0af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b62800640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b5c0cb150 0x7f2b5c0af8c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800c4b0 tx=0x7f2b4800c970 comp rx=0 tx=0).stop 2024-09-10T21:10:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b62800640 1 -- 172.21.15.16:0/3369818992 shutdown_connections 2024-09-10T21:10:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b62800640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b3c072610 0x7f2b3c074ad0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b62800640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b5c0c3be0 0x7f2b5c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b62800640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b5c0cb150 0x7f2b5c0af8c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b62800640 1 --2- 172.21.15.16:0/3369818992 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b5c071780 0x7f2b5c0a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b62800640 1 -- 172.21.15.16:0/3369818992 >> 172.21.15.16:0/3369818992 conn(0x7f2b5c0d0b00 msgr2=0x7f2b5c0b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b62800640 1 -- 172.21.15.16:0/3369818992 shutdown_connections 2024-09-10T21:10:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.686+0000 7f2b62800640 1 -- 172.21.15.16:0/3369818992 wait complete. 2024-09-10T21:10:39.712 INFO:teuthology.orchestra.run.smithi016.stdout:[{"version":27,"timestamp":"2024-09-10T21:10:17.553862+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"4226.427518"}]}] 2024-09-10T21:10:39.712 INFO:tasks.ceph_manager:config epoch is 27 2024-09-10T21:10:39.712 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T21:10:39.712 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T21:10:39.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T21:10:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 -- 172.21.15.16:0/2653479469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23400d06b0 msgr2=0x7f23400700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 --2- 172.21.15.16:0/2653479469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23400d06b0 0x7f23400700a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2340074fa0 tx=0x7f232801c770 comp rx=0 tx=0).stop 2024-09-10T21:10:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 -- 172.21.15.16:0/2653479469 shutdown_connections 2024-09-10T21:10:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 --2- 172.21.15.16:0/2653479469 >> v2:172.21.15.52:3300/0 conn(0x7f23400705e0 0x7f23400ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 --2- 172.21.15.16:0/2653479469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23400d06b0 0x7f23400700a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 -- 172.21.15.16:0/2653479469 >> 172.21.15.16:0/2653479469 conn(0x7f23400cfb20 msgr2=0x7f23400cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 -- 172.21.15.16:0/2653479469 shutdown_connections 2024-09-10T21:10:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 -- 172.21.15.16:0/2653479469 wait complete. 2024-09-10T21:10:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 Processor -- start 2024-09-10T21:10:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 -- start start 2024-09-10T21:10:39.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23400705e0 0x7f23401f9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:39.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23400d06b0 0x7f2340150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:39.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f234015a270 0x7f2340156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:39.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2340167bc0 con 0x7f23400705e0 2024-09-10T21:10:39.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2340167a40 con 0x7f23400d06b0 2024-09-10T21:10:39.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f2347200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2340167d40 con 0x7f234015a270 2024-09-10T21:10:39.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f233fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23400705e0 0x7f23401f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:39.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f233fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23400705e0 0x7f23401f9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38434/0 (socket says 172.21.15.16:38434) 2024-09-10T21:10:39.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.854+0000 7f233fe00640 1 -- 172.21.15.16:0/1935671633 learned_addr learned my addr 172.21.15.16:0/1935671633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:39.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f2337400640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23400d06b0 0x7f2340150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:39.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f2345200640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f234015a270 0x7f2340156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:39.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f233fe00640 1 -- 172.21.15.16:0/1935671633 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f234015a270 msgr2=0x7f2340156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:39.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f233fe00640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f234015a270 0x7f2340156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f233fe00640 1 -- 172.21.15.16:0/1935671633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23400d06b0 msgr2=0x7f2340150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f233fe00640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23400d06b0 0x7f2340150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f233fe00640 1 -- 172.21.15.16:0/1935671633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23401574b0 con 0x7f23400705e0 2024-09-10T21:10:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f2345200640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f234015a270 0x7f2340156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f233fe00640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23400705e0 0x7f23401f9950 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2330001ec0 tx=0x7f233000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f233cc00640 1 -- 172.21.15.16:0/1935671633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330017070 con 0x7f23400705e0 2024-09-10T21:10:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f233cc00640 1 -- 172.21.15.16:0/1935671633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2330003600 con 0x7f23400705e0 2024-09-10T21:10:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f233cc00640 1 -- 172.21.15.16:0/1935671633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330003a50 con 0x7f23400705e0 2024-09-10T21:10:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f2347200640 1 -- 172.21.15.16:0/1935671633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23400c8450 con 0x7f23400705e0 2024-09-10T21:10:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f2347200640 1 -- 172.21.15.16:0/1935671633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23400c8710 con 0x7f23400705e0 2024-09-10T21:10:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f2347200640 1 -- 172.21.15.16:0/1935671633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23400ca170 con 0x7f23400705e0 2024-09-10T21:10:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f233cc00640 1 -- 172.21.15.16:0/1935671633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2330003bf0 con 0x7f23400705e0 2024-09-10T21:10:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.858+0000 7f233cc00640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f231c072610 0x7f231c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.862+0000 7f233cc00640 1 -- 172.21.15.16:0/1935671633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2330092890 con 0x7f23400705e0 2024-09-10T21:10:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.862+0000 7f2337400640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f231c072610 0x7f231c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.862+0000 7f2337400640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f231c072610 0x7f231c074ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f23280af000 tx=0x7f232801afb0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:39.862+0000 7f233cc00640 1 -- 172.21.15.16:0/1935671633 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f23401574b0 con 0x7f23400705e0 2024-09-10T21:10:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.078+0000 7f2347200640 1 -- 172.21.15.16:0/1935671633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f23400b5aa0 con 0x7f23400705e0 2024-09-10T21:10:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.082+0000 7f233cc00640 1 -- 172.21.15.16:0/1935671633 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+184981 (secure 0 0 0) 0x7f233005df60 con 0x7f23400705e0 2024-09-10T21:10:40.090 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.086+0000 7f2347200640 1 -- 172.21.15.16:0/1935671633 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f231c072610 msgr2=0x7f231c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.086+0000 7f2347200640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f231c072610 0x7f231c074ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f23280af000 tx=0x7f232801afb0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.086+0000 7f2347200640 1 -- 172.21.15.16:0/1935671633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23400705e0 msgr2=0x7f23401f9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.086+0000 7f2347200640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23400705e0 0x7f23401f9950 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2330001ec0 tx=0x7f233000c890 comp rx=0 tx=0).stop 2024-09-10T21:10:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.086+0000 7f2347200640 1 -- 172.21.15.16:0/1935671633 shutdown_connections 2024-09-10T21:10:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.086+0000 7f2347200640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f231c072610 0x7f231c074ad0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.086+0000 7f2347200640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f234015a270 0x7f2340156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.086+0000 7f2347200640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23400d06b0 0x7f2340150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.086+0000 7f2347200640 1 --2- 172.21.15.16:0/1935671633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23400705e0 0x7f23401f9950 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.086+0000 7f2347200640 1 -- 172.21.15.16:0/1935671633 >> 172.21.15.16:0/1935671633 conn(0x7f23400cfb20 msgr2=0x7f2340159090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.086+0000 7f2347200640 1 -- 172.21.15.16:0/1935671633 shutdown_connections 2024-09-10T21:10:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.086+0000 7f2347200640 1 -- 172.21.15.16:0/1935671633 wait complete. 2024-09-10T21:10:40.110 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"flags":0,"active_gid":4137,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":462868332},{"type":"v1","addr":"172.21.15.16:6801","nonce":462868332}]},"active_addr":"172.21.15.16:6801/462868332","active_change":"2024-09-10T21:10:09.446776+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4117,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":301071164}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2924537483}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1691883432}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2832406077}]}]} 2024-09-10T21:10:40.114 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T21:10:40.115 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T21:10:40.115 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:10:40.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 -- 172.21.15.16:0/1915850455 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e140d0700 msgr2=0x7f9e140aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:40.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 --2- 172.21.15.16:0/1915850455 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e140d0700 0x7f9e140aed30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009c80 tx=0x7f9e0c01cce0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 -- 172.21.15.16:0/1915850455 shutdown_connections 2024-09-10T21:10:40.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 --2- 172.21.15.16:0/1915850455 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e140d0700 0x7f9e140aed30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 --2- 172.21.15.16:0/1915850455 >> v2:172.21.15.52:3300/0 conn(0x7f9e1415f350 0x7f9e140cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 -- 172.21.15.16:0/1915850455 >> 172.21.15.16:0/1915850455 conn(0x7f9e140d1d80 msgr2=0x7f9e140d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:40.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 -- 172.21.15.16:0/1915850455 shutdown_connections 2024-09-10T21:10:40.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 -- 172.21.15.16:0/1915850455 wait complete. 2024-09-10T21:10:40.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 Processor -- start 2024-09-10T21:10:40.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 -- start start 2024-09-10T21:10:40.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e140cfb60 0x7f9e140b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:40.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e140d0700 0x7f9e1406efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:40.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e1415f350 0x7f9e14156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:40.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e14168ad0 con 0x7f9e1415f350 2024-09-10T21:10:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9e140b0020 con 0x7f9e140cfb60 2024-09-10T21:10:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e19a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9e14168c50 con 0x7f9e140d0700 2024-09-10T21:10:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e13400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e1415f350 0x7f9e14156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e12a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e140cfb60 0x7f9e140b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e13400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e1415f350 0x7f9e14156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38442/0 (socket says 172.21.15.16:38442) 2024-09-10T21:10:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e13400640 1 -- 172.21.15.16:0/1603002278 learned_addr learned my addr 172.21.15.16:0/1603002278 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e0a000640 1 --2- 172.21.15.16:0/1603002278 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e140d0700 0x7f9e1406efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:40.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e13400640 1 -- 172.21.15.16:0/1603002278 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e140d0700 msgr2=0x7f9e1406efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:40.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e13400640 1 --2- 172.21.15.16:0/1603002278 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e140d0700 0x7f9e1406efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e13400640 1 -- 172.21.15.16:0/1603002278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e140cfb60 msgr2=0x7f9e140b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:40.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e13400640 1 --2- 172.21.15.16:0/1603002278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e140cfb60 0x7f9e140b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e13400640 1 -- 172.21.15.16:0/1603002278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e141fb710 con 0x7f9e1415f350 2024-09-10T21:10:40.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.258+0000 7f9e13400640 1 --2- 172.21.15.16:0/1603002278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e1415f350 0x7f9e14156640 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c01b390 tx=0x7f9e0c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:40.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.262+0000 7f9e0b400640 1 -- 172.21.15.16:0/1603002278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0bb070 con 0x7f9e1415f350 2024-09-10T21:10:40.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.262+0000 7f9e0b400640 1 -- 172.21.15.16:0/1603002278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e0c004d40 con 0x7f9e1415f350 2024-09-10T21:10:40.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.262+0000 7f9e0b400640 1 -- 172.21.15.16:0/1603002278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0b2040 con 0x7f9e1415f350 2024-09-10T21:10:40.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.262+0000 7f9e19a00640 1 -- 172.21.15.16:0/1603002278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e141fb9f0 con 0x7f9e1415f350 2024-09-10T21:10:40.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.262+0000 7f9e19a00640 1 -- 172.21.15.16:0/1603002278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e141fbfb0 con 0x7f9e1415f350 2024-09-10T21:10:40.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.262+0000 7f9e19a00640 1 -- 172.21.15.16:0/1603002278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e140aed30 con 0x7f9e1415f350 2024-09-10T21:10:40.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.262+0000 7f9e0b400640 1 -- 172.21.15.16:0/1603002278 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9e0c0b5030 con 0x7f9e1415f350 2024-09-10T21:10:40.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.266+0000 7f9e0b400640 1 --2- 172.21.15.16:0/1603002278 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9df00725e0 0x7f9df0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:40.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.266+0000 7f9e0b400640 1 -- 172.21.15.16:0/1603002278 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9e0c12cdc0 con 0x7f9e1415f350 2024-09-10T21:10:40.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.266+0000 7f9e12a00640 1 --2- 172.21.15.16:0/1603002278 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9df00725e0 0x7f9df0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:40.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.266+0000 7f9e12a00640 1 --2- 172.21.15.16:0/1603002278 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9df00725e0 0x7f9df0074aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009b40 tx=0x7f9e040091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:40.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.270+0000 7f9e0b400640 1 -- 172.21.15.16:0/1603002278 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9e0c016990 con 0x7f9e1415f350 2024-09-10T21:10:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.442+0000 7f9e19a00640 1 -- 172.21.15.16:0/1603002278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e140b0480 con 0x7f9e1415f350 2024-09-10T21:10:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.442+0000 7f9e0b400640 1 -- 172.21.15.16:0/1603002278 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f9e0c0f7e60 con 0x7f9e1415f350 2024-09-10T21:10:40.449 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:40.449 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:10:31.158120+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:10:27.523741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:10:28.533379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:10:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.446+0000 7f9e19a00640 1 -- 172.21.15.16:0/1603002278 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9df00725e0 msgr2=0x7f9df0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.446+0000 7f9e19a00640 1 --2- 172.21.15.16:0/1603002278 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9df00725e0 0x7f9df0074aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009b40 tx=0x7f9e040091e0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.446+0000 7f9e19a00640 1 -- 172.21.15.16:0/1603002278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e1415f350 msgr2=0x7f9e14156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.446+0000 7f9e19a00640 1 --2- 172.21.15.16:0/1603002278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e1415f350 0x7f9e14156640 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c01b390 tx=0x7f9e0c0092e0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.446+0000 7f9e19a00640 1 -- 172.21.15.16:0/1603002278 shutdown_connections 2024-09-10T21:10:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.446+0000 7f9e19a00640 1 --2- 172.21.15.16:0/1603002278 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9df00725e0 0x7f9df0074aa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.446+0000 7f9e19a00640 1 --2- 172.21.15.16:0/1603002278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e1415f350 0x7f9e14156640 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.446+0000 7f9e19a00640 1 --2- 172.21.15.16:0/1603002278 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e140d0700 0x7f9e1406efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.446+0000 7f9e19a00640 1 --2- 172.21.15.16:0/1603002278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e140cfb60 0x7f9e140b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.446+0000 7f9e19a00640 1 -- 172.21.15.16:0/1603002278 >> 172.21.15.16:0/1603002278 conn(0x7f9e140d1d80 msgr2=0x7f9e140b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.446+0000 7f9e19a00640 1 -- 172.21.15.16:0/1603002278 shutdown_connections 2024-09-10T21:10:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.446+0000 7f9e19a00640 1 -- 172.21.15.16:0/1603002278 wait complete. 2024-09-10T21:10:40.471 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T21:10:40.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:10:40.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.614+0000 7fd584a00640 1 -- 172.21.15.16:0/1073303466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58015f400 msgr2=0x7fd580070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:40.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.614+0000 7fd584a00640 1 --2- 172.21.15.16:0/1073303466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58015f400 0x7fd580070fe0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd5700098f0 tx=0x7fd57001c7c0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.614+0000 7fd584a00640 1 -- 172.21.15.16:0/1073303466 shutdown_connections 2024-09-10T21:10:40.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.614+0000 7fd584a00640 1 --2- 172.21.15.16:0/1073303466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58015f400 0x7fd580070fe0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.614+0000 7fd584a00640 1 -- 172.21.15.16:0/1073303466 >> 172.21.15.16:0/1073303466 conn(0x7fd58009b130 msgr2=0x7fd5800d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:40.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.614+0000 7fd584a00640 1 -- 172.21.15.16:0/1073303466 shutdown_connections 2024-09-10T21:10:40.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.614+0000 7fd584a00640 1 -- 172.21.15.16:0/1073303466 wait complete. 2024-09-10T21:10:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.614+0000 7fd584a00640 1 Processor -- start 2024-09-10T21:10:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.614+0000 7fd584a00640 1 -- start start 2024-09-10T21:10:40.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd584a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd5800b0590 0x7fd58014db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:40.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd584a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5800b13c0 0x7fd5800c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd584a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58015f400 0x7fd58015d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd584a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd58016a800 con 0x7fd5800b13c0 2024-09-10T21:10:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd584a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd5800d1260 con 0x7fd58015f400 2024-09-10T21:10:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd584a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd58016a980 con 0x7fd5800b0590 2024-09-10T21:10:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd577e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5800b13c0 0x7fd5800c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd577e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5800b13c0 0x7fd5800c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38466/0 (socket says 172.21.15.16:38466) 2024-09-10T21:10:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd577e00640 1 -- 172.21.15.16:0/1468712758 learned_addr learned my addr 172.21.15.16:0/1468712758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd57de00640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd5800b0590 0x7fd58014db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd57e800640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58015f400 0x7fd58015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd577e00640 1 -- 172.21.15.16:0/1468712758 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd5800b0590 msgr2=0x7fd58014db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd577e00640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd5800b0590 0x7fd58014db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd577e00640 1 -- 172.21.15.16:0/1468712758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58015f400 msgr2=0x7fd58015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:40.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd577e00640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58015f400 0x7fd58015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd577e00640 1 -- 172.21.15.16:0/1468712758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5800b0a30 con 0x7fd5800b13c0 2024-09-10T21:10:40.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd577e00640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5800b13c0 0x7fd5800c9cd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd56800c500 tx=0x7fd56800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:40.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd576000640 1 -- 172.21.15.16:0/1468712758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568017070 con 0x7fd5800b13c0 2024-09-10T21:10:40.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd576000640 1 -- 172.21.15.16:0/1468712758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd568003600 con 0x7fd5800b13c0 2024-09-10T21:10:40.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd576000640 1 -- 172.21.15.16:0/1468712758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568003a90 con 0x7fd5800b13c0 2024-09-10T21:10:40.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd584a00640 1 -- 172.21.15.16:0/1468712758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5800cb9b0 con 0x7fd5800b13c0 2024-09-10T21:10:40.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd584a00640 1 -- 172.21.15.16:0/1468712758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5800cbc20 con 0x7fd5800b13c0 2024-09-10T21:10:40.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd584a00640 1 -- 172.21.15.16:0/1468712758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd58015ec30 con 0x7fd5800b13c0 2024-09-10T21:10:40.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.618+0000 7fd576000640 1 -- 172.21.15.16:0/1468712758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd568003c30 con 0x7fd5800b13c0 2024-09-10T21:10:40.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.622+0000 7fd576000640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd55c072610 0x7fd55c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:40.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.622+0000 7fd576000640 1 -- 172.21.15.16:0/1468712758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd568092a70 con 0x7fd5800b13c0 2024-09-10T21:10:40.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.622+0000 7fd57de00640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd55c072610 0x7fd55c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:40.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.622+0000 7fd57de00640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd55c072610 0x7fd55c074ad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd5700ae000 tx=0x7fd57001b000 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:40.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.626+0000 7fd576000640 1 -- 172.21.15.16:0/1468712758 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd56805e200 con 0x7fd5800b13c0 2024-09-10T21:10:40.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.798+0000 7fd584a00640 1 -- 172.21.15.16:0/1468712758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5800ce790 con 0x7fd5800b13c0 2024-09-10T21:10:40.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.802+0000 7fd576000640 1 -- 172.21.15.16:0/1468712758 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7fd56805db00 con 0x7fd5800b13c0 2024-09-10T21:10:40.806 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:40.806 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:10:31.158120+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:10:27.523741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:10:28.533379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:10:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.802+0000 7fd584a00640 1 -- 172.21.15.16:0/1468712758 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd55c072610 msgr2=0x7fd55c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.802+0000 7fd584a00640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd55c072610 0x7fd55c074ad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd5700ae000 tx=0x7fd57001b000 comp rx=0 tx=0).stop 2024-09-10T21:10:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.802+0000 7fd584a00640 1 -- 172.21.15.16:0/1468712758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5800b13c0 msgr2=0x7fd5800c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.802+0000 7fd584a00640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5800b13c0 0x7fd5800c9cd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd56800c500 tx=0x7fd56800c9c0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.802+0000 7fd584a00640 1 -- 172.21.15.16:0/1468712758 shutdown_connections 2024-09-10T21:10:40.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.802+0000 7fd584a00640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd55c072610 0x7fd55c074ad0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.802+0000 7fd584a00640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd58015f400 0x7fd58015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.802+0000 7fd584a00640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5800b13c0 0x7fd5800c9cd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.802+0000 7fd584a00640 1 --2- 172.21.15.16:0/1468712758 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd5800b0590 0x7fd58014db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:40.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.802+0000 7fd584a00640 1 -- 172.21.15.16:0/1468712758 >> 172.21.15.16:0/1468712758 conn(0x7fd58009b130 msgr2=0x7fd5800cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:40.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.802+0000 7fd584a00640 1 -- 172.21.15.16:0/1468712758 shutdown_connections 2024-09-10T21:10:40.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:40.802+0000 7fd584a00640 1 -- 172.21.15.16:0/1468712758 wait complete. 2024-09-10T21:10:40.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T21:10:40.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T21:10:40.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T21:10:40.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T21:10:40.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T21:10:40.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T21:10:40.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T21:10:40.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T21:10:41.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.082+0000 7fc881000640 1 -- 172.21.15.16:0/2775395171 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc87c0b13c0 msgr2=0x7fc87c0afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.082+0000 7fc881000640 1 --2- 172.21.15.16:0/2775395171 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc87c0b13c0 0x7fc87c0afac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc874009c80 tx=0x7fc87401cc80 comp rx=0 tx=0).stop 2024-09-10T21:10:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.082+0000 7ff172e00640 1 -- 172.21.15.16:0/655322339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c0d2a20 msgr2=0x7ff16c0cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7ff16aa00640 1 -- 172.21.15.16:0/655322339 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1640b2070 con 0x7ff16c0d2a20 2024-09-10T21:10:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7ff172e00640 1 --2- 172.21.15.16:0/655322339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c0d2a20 0x7ff16c0cfce0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0b7e90 tx=0x7ff16401c9a0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7fc881000640 1 -- 172.21.15.16:0/2775395171 shutdown_connections 2024-09-10T21:10:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7fc881000640 1 --2- 172.21.15.16:0/2775395171 >> v2:172.21.15.16:3300/0 conn(0x7fc87c0716f0 0x7fc87c0af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7fc881000640 1 --2- 172.21.15.16:0/2775395171 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc87c0b13c0 0x7fc87c0afac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7fc881000640 1 -- 172.21.15.16:0/2775395171 >> 172.21.15.16:0/2775395171 conn(0x7fc87c09b130 msgr2=0x7fc87c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7ff172e00640 1 -- 172.21.15.16:0/655322339 shutdown_connections 2024-09-10T21:10:41.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7ff172e00640 1 --2- 172.21.15.16:0/655322339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c0d2a20 0x7ff16c0cfce0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7ff172e00640 1 --2- 172.21.15.16:0/655322339 >> v2:172.21.15.52:3300/0 conn(0x7ff16c15f700 0x7ff16c160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7ff172e00640 1 -- 172.21.15.16:0/655322339 >> 172.21.15.16:0/655322339 conn(0x7ff16c0d1de0 msgr2=0x7ff16c0d21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7ff172e00640 1 -- 172.21.15.16:0/655322339 shutdown_connections 2024-09-10T21:10:41.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7fc881000640 1 -- 172.21.15.16:0/2775395171 shutdown_connections 2024-09-10T21:10:41.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7ff172e00640 1 -- 172.21.15.16:0/655322339 wait complete. 2024-09-10T21:10:41.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7fc881000640 1 -- 172.21.15.16:0/2775395171 wait complete. 2024-09-10T21:10:41.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7f9fdb400640 1 -- 172.21.15.16:0/1283058715 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fbc004810 msgr2=0x7f9fbc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.086+0000 7f9fdb400640 1 --2- 172.21.15.16:0/1283058715 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fbc004810 0x7f9fbc004c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0001ae0 tx=0x7f9fc001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7f9fdb400640 1 -- 172.21.15.16:0/1283058715 shutdown_connections 2024-09-10T21:10:41.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7f9fdb400640 1 --2- 172.21.15.16:0/1283058715 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fbc004810 0x7f9fbc004c90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7f9fdb400640 1 -- 172.21.15.16:0/1283058715 >> 172.21.15.16:0/1283058715 conn(0x7f9fd40cfb20 msgr2=0x7f9fd40cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc881000640 1 Processor -- start 2024-09-10T21:10:41.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7f9fdb400640 1 -- 172.21.15.16:0/1283058715 shutdown_connections 2024-09-10T21:10:41.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7ff172e00640 1 Processor -- start 2024-09-10T21:10:41.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc881000640 1 -- start start 2024-09-10T21:10:41.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7ff172e00640 1 -- start start 2024-09-10T21:10:41.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc881000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc87c0716f0 0x7fc87c0c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc881000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87c0b13c0 0x7fc87c0c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc881000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c15f400 0x7fc87c15e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc881000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c168fc0 con 0x7fc87c15f400 2024-09-10T21:10:41.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc881000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c168e40 con 0x7fc87c0b13c0 2024-09-10T21:10:41.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc881000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc87c169140 con 0x7fc87c0716f0 2024-09-10T21:10:41.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7f1857200640 1 -- 172.21.15.16:0/1325151937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18580d03d0 msgr2=0x7f18580d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7f1857200640 1 --2- 172.21.15.16:0/1325151937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18580d03d0 0x7f18580d1f60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1848009840 tx=0x7f184801c820 comp rx=0 tx=0).stop 2024-09-10T21:10:41.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc879400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87c0b13c0 0x7fc87c0c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc87a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c15f400 0x7fc87c15e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc87a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c15f400 0x7fc87c15e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38516/0 (socket says 172.21.15.16:38516) 2024-09-10T21:10:41.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc87a800640 1 -- 172.21.15.16:0/2914461226 learned_addr learned my addr 172.21.15.16:0/2914461226 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7ff172e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c0d2a20 0x7ff16c152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7ff172e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff16c15f320 0x7ff16c0b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7ff172e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff16c15f700 0x7ff16c14edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7ff172e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c169490 con 0x7ff16c0d2a20 2024-09-10T21:10:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7ff172e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c169310 con 0x7ff16c15f320 2024-09-10T21:10:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7ff172e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff16c169610 con 0x7ff16c15f700 2024-09-10T21:10:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc87a800640 1 -- 172.21.15.16:0/2914461226 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc87c0716f0 msgr2=0x7fc87c0c5770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc87a800640 1 --2- 172.21.15.16:0/2914461226 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc87c0716f0 0x7fc87c0c5770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc87a800640 1 -- 172.21.15.16:0/2914461226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87c0b13c0 msgr2=0x7fc87c0c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc87a800640 1 --2- 172.21.15.16:0/2914461226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc87c0b13c0 0x7fc87c0c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7fc87a800640 1 -- 172.21.15.16:0/2914461226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c0c5cb0 con 0x7fc87c15f400 2024-09-10T21:10:41.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7feeb3c00640 1 -- 172.21.15.16:0/98505951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeac15ae90 msgr2=0x7feeac0cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7feeb3c00640 1 --2- 172.21.15.16:0/98505951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeac15ae90 0x7feeac0cfbf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feeac15faa0 tx=0x7fee9c01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7f1857200640 1 -- 172.21.15.16:0/1325151937 shutdown_connections 2024-09-10T21:10:41.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7f1857200640 1 --2- 172.21.15.16:0/1325151937 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18580d3910 0x7f18580d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7f1857200640 1 --2- 172.21.15.16:0/1325151937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18580d03d0 0x7f18580d1f60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7f1857200640 1 -- 172.21.15.16:0/1325151937 >> 172.21.15.16:0/1325151937 conn(0x7f18580d4060 msgr2=0x7f18580d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.090+0000 7f9fdb400640 1 -- 172.21.15.16:0/1283058715 wait complete. 2024-09-10T21:10:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f9fdb400640 1 Processor -- start 2024-09-10T21:10:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7ff16be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c0d2a20 0x7ff16c152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f9fdb400640 1 -- start start 2024-09-10T21:10:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f1857200640 1 -- 172.21.15.16:0/1325151937 shutdown_connections 2024-09-10T21:10:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f1857200640 1 -- 172.21.15.16:0/1325151937 wait complete. 2024-09-10T21:10:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7fc87a800640 1 --2- 172.21.15.16:0/2914461226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c15f400 0x7fc87c15e140 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc87401cab0 tx=0x7fc8740a5a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7ff170e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff16c15f700 0x7ff16c14edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7ff170e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff16c15f700 0x7ff16c14edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:43274/0 (socket says 172.21.15.16:43274) 2024-09-10T21:10:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7ff170e00640 1 -- 172.21.15.16:0/3367395785 learned_addr learned my addr 172.21.15.16:0/3367395785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7feeb3c00640 1 -- 172.21.15.16:0/98505951 shutdown_connections 2024-09-10T21:10:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7feeb3c00640 1 --2- 172.21.15.16:0/98505951 >> v2:172.21.15.52:3300/0 conn(0x7feeac073ed0 0x7feeac0ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7feeb3c00640 1 --2- 172.21.15.16:0/98505951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeac15ae90 0x7feeac0cfbf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7feeb3c00640 1 -- 172.21.15.16:0/98505951 >> 172.21.15.16:0/98505951 conn(0x7feeac0d0c30 msgr2=0x7feeac0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7feeb3c00640 1 -- 172.21.15.16:0/98505951 shutdown_connections 2024-09-10T21:10:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7feeb3c00640 1 -- 172.21.15.16:0/98505951 wait complete. 2024-09-10T21:10:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7fc872000640 1 -- 172.21.15.16:0/2914461226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8740bb070 con 0x7fc87c15f400 2024-09-10T21:10:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7fc872000640 1 -- 172.21.15.16:0/2914461226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc874004d60 con 0x7fc87c15f400 2024-09-10T21:10:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7fc881000640 1 -- 172.21.15.16:0/2914461226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87c0c6f30 con 0x7fc87c15f400 2024-09-10T21:10:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7fc872000640 1 -- 172.21.15.16:0/2914461226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8740b2040 con 0x7fc87c15f400 2024-09-10T21:10:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7fc881000640 1 -- 172.21.15.16:0/2914461226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87c1ff610 con 0x7fc87c15f400 2024-09-10T21:10:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7ff16b400640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff16c15f320 0x7ff16c0b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7ff16be00640 1 -- 172.21.15.16:0/3367395785 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff16c15f700 msgr2=0x7ff16c14edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7ff16be00640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff16c15f700 0x7ff16c14edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7ff16be00640 1 -- 172.21.15.16:0/3367395785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff16c15f320 msgr2=0x7ff16c0b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f9fdb400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fbc004810 0x7f9fd42051b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7ff16be00640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff16c15f320 0x7ff16c0b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7ff16be00640 1 -- 172.21.15.16:0/3367395785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16c06f820 con 0x7ff16c0d2a20 2024-09-10T21:10:41.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7ff170e00640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff16c15f700 0x7ff16c14edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f1857200640 1 Processor -- start 2024-09-10T21:10:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f9fdb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd42056f0 0x7f9fd406f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f1857200640 1 -- start start 2024-09-10T21:10:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f9fd8a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fbc004810 0x7f9fd42051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd42056f0 0x7f9fd406f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd42056f0 0x7f9fd406f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38542/0 (socket says 172.21.15.16:38542) 2024-09-10T21:10:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f9fd3e00640 1 -- 172.21.15.16:0/497163231 learned_addr learned my addr 172.21.15.16:0/497163231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f9fdb400640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fd414e8c0 0x7f9fd406f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd415b070 con 0x7f9fd42056f0 2024-09-10T21:10:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd415aef0 con 0x7f9fd414e8c0 2024-09-10T21:10:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd415b1f0 con 0x7f9fbc004810 2024-09-10T21:10:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fd9400640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fd414e8c0 0x7f9fd406f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7fc881000640 1 -- 172.21.15.16:0/2914461226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc87c070fe0 con 0x7fc87c15f400 2024-09-10T21:10:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.094+0000 7feeb3c00640 1 Processor -- start 2024-09-10T21:10:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7feeb3c00640 1 -- start start 2024-09-10T21:10:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fd9400640 1 -- 172.21.15.16:0/497163231 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fbc004810 msgr2=0x7f9fd42051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1857200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18580d03d0 0x7f185816af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fd9400640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fbc004810 0x7f9fd42051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fd9400640 1 -- 172.21.15.16:0/497163231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd42056f0 msgr2=0x7f9fd406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fd9400640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd42056f0 0x7f9fd406f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fd9400640 1 -- 172.21.15.16:0/497163231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd414ee60 con 0x7f9fd414e8c0 2024-09-10T21:10:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fd8a00640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fbc004810 0x7f9fd42051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1857200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18580d4d10 0x7f18580b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1857200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18580b4670 0x7f18580a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1857200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1858177ea0 con 0x7f18580d4d10 2024-09-10T21:10:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1857200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1858177d20 con 0x7f18580d03d0 2024-09-10T21:10:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1857200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f1858178020 con 0x7f18580b4670 2024-09-10T21:10:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1855e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18580d03d0 0x7f185816af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7ff16be00640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c0d2a20 0x7ff16c152630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00c730 tx=0x7ff15c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fd9400640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fd414e8c0 0x7f9fd406f950 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4076480 tx=0x7f9fcc00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1855e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18580d03d0 0x7f185816af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:52950/0 (socket says 172.21.15.16:52950) 2024-09-10T21:10:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1855e00640 1 -- 172.21.15.16:0/1199696154 learned_addr learned my addr 172.21.15.16:0/1199696154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fd0c00640 1 -- 172.21.15.16:0/497163231 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00eea0 con 0x7f9fd414e8c0 2024-09-10T21:10:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fd0c00640 1 -- 172.21.15.16:0/497163231 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fcc018070 con 0x7f9fd414e8c0 2024-09-10T21:10:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fd0c00640 1 -- 172.21.15.16:0/497163231 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc011460 con 0x7f9fd414e8c0 2024-09-10T21:10:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1855e00640 1 -- 172.21.15.16:0/1199696154 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18580b4670 msgr2=0x7f18580a0690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1855e00640 1 --2- 172.21.15.16:0/1199696154 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18580b4670 0x7f18580a0690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1855e00640 1 -- 172.21.15.16:0/1199696154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18580d4d10 msgr2=0x7f18580b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1855e00640 1 --2- 172.21.15.16:0/1199696154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18580d4d10 0x7f18580b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f1855e00640 1 -- 172.21.15.16:0/1199696154 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185816b4a0 con 0x7f18580d03d0 2024-09-10T21:10:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7feeb3c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeac073ed0 0x7feeac0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7feeb3c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feeac0d0130 0x7feeac0c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7feeb3c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeac15ae90 0x7feeac0c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7fc872000640 1 -- 172.21.15.16:0/2914461226 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc8740b5030 con 0x7fc87c15f400 2024-09-10T21:10:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd414f140 con 0x7f9fd414e8c0 2024-09-10T21:10:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd4204c90 con 0x7f9fd414e8c0 2024-09-10T21:10:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7feeabe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feeac0d0130 0x7feeac0c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7feeabe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feeac0d0130 0x7feeac0c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:43304/0 (socket says 172.21.15.16:43304) 2024-09-10T21:10:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7feeb1c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeac15ae90 0x7feeac0c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7feeb1c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeac15ae90 0x7feeac0c64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:52960/0 (socket says 172.21.15.16:52960) 2024-09-10T21:10:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7feeabe00640 1 -- 172.21.15.16:0/2812884087 learned_addr learned my addr 172.21.15.16:0/2812884087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feeac06cf70 con 0x7feeac073ed0 2024-09-10T21:10:41.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fd3e00640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd42056f0 0x7f9fd406f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:41.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9fd414f660 con 0x7f9fd414e8c0 2024-09-10T21:10:41.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feeac06cdf0 con 0x7feeac15ae90 2024-09-10T21:10:41.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7feeac06d0f0 con 0x7feeac0d0130 2024-09-10T21:10:41.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.098+0000 7fc872000640 1 --2- 172.21.15.16:0/2914461226 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc85c0726b0 0x7fc85c074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7fc872000640 1 -- 172.21.15.16:0/2914461226 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc87412d1b0 con 0x7fc87c15f400 2024-09-10T21:10:41.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7feeb1c00640 1 -- 172.21.15.16:0/2812884087 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feeac0d0130 msgr2=0x7feeac0c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7feeb1c00640 1 --2- 172.21.15.16:0/2812884087 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feeac0d0130 0x7feeac0c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7feeb1c00640 1 -- 172.21.15.16:0/2812884087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeac073ed0 msgr2=0x7feeac0c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7feeb1c00640 1 --2- 172.21.15.16:0/2812884087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeac073ed0 0x7feeac0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7f1855e00640 1 --2- 172.21.15.16:0/1199696154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18580d03d0 0x7f185816af60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f184801aed0 tx=0x7f1848005ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7feeb1c00640 1 -- 172.21.15.16:0/2812884087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeac0c6a30 con 0x7feeac15ae90 2024-09-10T21:10:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7fc872000640 1 --2- 172.21.15.16:0/2914461226 >> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] conn(0x7fc85c07a850 0x7fc85c07cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7fc872000640 1 -- 172.21.15.16:0/2914461226 --> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8740b0e70 con 0x7fc85c07a850 2024-09-10T21:10:41.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7feeabe00640 1 --2- 172.21.15.16:0/2812884087 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feeac0d0130 0x7feeac0c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:41.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7fc879400640 1 --2- 172.21.15.16:0/2914461226 >> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] conn(0x7fc85c07a850 0x7fc85c07cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7f184e000640 1 -- 172.21.15.16:0/1199696154 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18480b2040 con 0x7f18580d03d0 2024-09-10T21:10:41.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7f184e000640 1 -- 172.21.15.16:0/1199696154 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18480a5c90 con 0x7f18580d03d0 2024-09-10T21:10:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7f184e000640 1 -- 172.21.15.16:0/1199696154 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848005220 con 0x7f18580d03d0 2024-09-10T21:10:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7f1857200640 1 -- 172.21.15.16:0/1199696154 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1858204ca0 con 0x7f18580d03d0 2024-09-10T21:10:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7f1857200640 1 -- 172.21.15.16:0/1199696154 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1858205260 con 0x7f18580d03d0 2024-09-10T21:10:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7fc879e00640 1 --2- 172.21.15.16:0/2914461226 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc85c0726b0 0x7fc85c074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7feeb1c00640 1 --2- 172.21.15.16:0/2812884087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeac15ae90 0x7feeac0c64f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feea400ea30 tx=0x7feea400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7fc879400640 1 --2- 172.21.15.16:0/2914461226 >> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] conn(0x7fc85c07a850 0x7fc85c07cc70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7f9fd0c00640 1 -- 172.21.15.16:0/497163231 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9fcc011600 con 0x7f9fd414e8c0 2024-09-10T21:10:41.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7f1843400640 1 -- 172.21.15.16:0/1199696154 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f182c001040 con 0x7f18580d03d0 2024-09-10T21:10:41.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7feea8c00640 1 -- 172.21.15.16:0/2812884087 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea400ce40 con 0x7feeac15ae90 2024-09-10T21:10:41.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7fc872000640 1 -- 172.21.15.16:0/2914461226 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc874134050 con 0x7fc87c15f400 2024-09-10T21:10:41.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7feea8c00640 1 -- 172.21.15.16:0/2812884087 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feea4004c20 con 0x7feeac15ae90 2024-09-10T21:10:41.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7fc879e00640 1 --2- 172.21.15.16:0/2914461226 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc85c0726b0 0x7fc85c074b70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc86c003bd0 tx=0x7fc86c0079c0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeac0c87d0 con 0x7feeac15ae90 2024-09-10T21:10:41.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7feea8c00640 1 -- 172.21.15.16:0/2812884087 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4005020 con 0x7feeac15ae90 2024-09-10T21:10:41.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeac15a1d0 con 0x7feeac15ae90 2024-09-10T21:10:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.102+0000 7f9fd0c00640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fb40725e0 0x7f9fb4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f9fd0c00640 1 -- 172.21.15.16:0/497163231 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9fcc0942a0 con 0x7f9fd414e8c0 2024-09-10T21:10:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f9fd8a00640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fb40725e0 0x7f9fb4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f9fd0c00640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] conn(0x7f9fb407a780 0x7f9fb407cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f9fd0c00640 1 -- 172.21.15.16:0/497163231 --> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9fcc027e50 con 0x7f9fb407a780 2024-09-10T21:10:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f9fd0c00640 1 -- 172.21.15.16:0/497163231 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9fcc094650 con 0x7f9fd414e8c0 2024-09-10T21:10:41.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feeac0c8580 con 0x7feeac15ae90 2024-09-10T21:10:41.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f9fd8a00640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fb40725e0 0x7f9fb4074aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4076820 tx=0x7f9fc8005d00 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f9fd3e00640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] conn(0x7f9fb407a780 0x7f9fb407cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7ff163e00640 1 -- 172.21.15.16:0/3367395785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c017070 con 0x7ff16c0d2a20 2024-09-10T21:10:41.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7ff163e00640 1 -- 172.21.15.16:0/3367395785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff15c005270 con 0x7ff16c0d2a20 2024-09-10T21:10:41.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7ff163e00640 1 -- 172.21.15.16:0/3367395785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00a8a0 con 0x7ff16c0d2a20 2024-09-10T21:10:41.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7ff172e00640 1 -- 172.21.15.16:0/3367395785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16c06fb00 con 0x7ff16c0d2a20 2024-09-10T21:10:41.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7ff172e00640 1 -- 172.21.15.16:0/3367395785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16c15e4b0 con 0x7ff16c0d2a20 2024-09-10T21:10:41.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7fc872000640 1 -- 172.21.15.16:0/2914461226 <== osd.1 v2:172.21.15.16:6802/4097098606 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc8740b0e70 con 0x7fc85c07a850 2024-09-10T21:10:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7fc59f200640 1 -- 172.21.15.16:0/3530555155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc584005240 msgr2=0x7fc5840059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7fc59f200640 1 --2- 172.21.15.16:0/3530555155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc584005240 0x7fc5840059a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00d0180 tx=0x7fc59001d230 comp rx=0 tx=0).stop 2024-09-10T21:10:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7fc59f200640 1 -- 172.21.15.16:0/3530555155 shutdown_connections 2024-09-10T21:10:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f9fd3e00640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] conn(0x7f9fb407a780 0x7f9fb407cba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7fc59f200640 1 --2- 172.21.15.16:0/3530555155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc584005240 0x7fc5840059a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7ff172e00640 1 -- 172.21.15.16:0/3367395785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff16c15e890 con 0x7ff16c0d2a20 2024-09-10T21:10:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7fc59f200640 1 -- 172.21.15.16:0/3530555155 >> 172.21.15.16:0/3530555155 conn(0x7fc5a00b1d60 msgr2=0x7fc5a00b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7feea8c00640 1 -- 172.21.15.16:0/2812884087 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feea4020030 con 0x7feeac15ae90 2024-09-10T21:10:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f184e000640 1 -- 172.21.15.16:0/1199696154 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f18480b5070 con 0x7f18580d03d0 2024-09-10T21:10:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f184e000640 1 --2- 172.21.15.16:0/1199696154 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18380725e0 0x7f1838074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f184e000640 1 -- 172.21.15.16:0/1199696154 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f184812c9d0 con 0x7f18580d03d0 2024-09-10T21:10:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f184e000640 1 --2- 172.21.15.16:0/1199696154 >> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] conn(0x7f183807a800 0x7f183807cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f184e000640 1 -- 172.21.15.16:0/1199696154 --> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1848009480 con 0x7f183807a800 2024-09-10T21:10:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.106+0000 7f184e000640 1 -- 172.21.15.16:0/1199696154 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f18481312a0 con 0x7f18580d03d0 2024-09-10T21:10:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f1855400640 1 --2- 172.21.15.16:0/1199696154 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18380725e0 0x7f1838074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f770cc00640 1 -- 172.21.15.16:0/2906202105 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76ec003c40 msgr2=0x7f76ec004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f770cc00640 1 --2- 172.21.15.16:0/2906202105 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76ec003c40 0x7f76ec004020 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7700001f80 tx=0x7f770001d9e0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f770cc00640 1 -- 172.21.15.16:0/2906202105 shutdown_connections 2024-09-10T21:10:41.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f770cc00640 1 --2- 172.21.15.16:0/2906202105 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76ec003c40 0x7f76ec004020 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f770cc00640 1 -- 172.21.15.16:0/2906202105 >> 172.21.15.16:0/2906202105 conn(0x7f77080d0d50 msgr2=0x7f77080d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f770cc00640 1 -- 172.21.15.16:0/2906202105 shutdown_connections 2024-09-10T21:10:41.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f770cc00640 1 -- 172.21.15.16:0/2906202105 wait complete. 2024-09-10T21:10:41.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7feea8c00640 1 --2- 172.21.15.16:0/2812884087 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fee8c0726b0 0x7fee8c074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f1856800640 1 --2- 172.21.15.16:0/1199696154 >> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] conn(0x7f183807a800 0x7f183807cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f9fd0c00640 1 -- 172.21.15.16:0/497163231 <== osd.0 v2:172.21.15.16:6810/1824652127 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9fcc027e50 con 0x7f9fb407a780 2024-09-10T21:10:41.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f1855400640 1 --2- 172.21.15.16:0/1199696154 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18380725e0 0x7f1838074aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18440058d0 tx=0x7f1844005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7feea8c00640 1 -- 172.21.15.16:0/2812884087 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feea4093c50 con 0x7feeac15ae90 2024-09-10T21:10:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7fc59f200640 1 -- 172.21.15.16:0/3530555155 shutdown_connections 2024-09-10T21:10:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f1856800640 1 --2- 172.21.15.16:0/1199696154 >> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] conn(0x7f183807a800 0x7f183807cc20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f770cc00640 1 Processor -- start 2024-09-10T21:10:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7f770cc00640 1 -- start start 2024-09-10T21:10:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7fc59f200640 1 -- 172.21.15.16:0/3530555155 wait complete. 2024-09-10T21:10:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7ff163e00640 1 -- 172.21.15.16:0/3367395785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff15c00aa40 con 0x7ff16c0d2a20 2024-09-10T21:10:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f184e000640 1 -- 172.21.15.16:0/1199696154 <== osd.4 v2:172.21.15.52:6824/447485120 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1848009480 con 0x7f183807a800 2024-09-10T21:10:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7faca7000640 1 -- 172.21.15.16:0/1900063804 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faca00cfc80 msgr2=0x7faca00d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7faca7000640 1 --2- 172.21.15.16:0/1900063804 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faca00cfc80 0x7faca00d00f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fac90009870 tx=0x7fac9001c860 comp rx=0 tx=0).stop 2024-09-10T21:10:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f770cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec003c40 0x7f77080c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f770cc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77081fd870 0x7f77080ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7faca7000640 1 -- 172.21.15.16:0/1900063804 shutdown_connections 2024-09-10T21:10:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7faca7000640 1 --2- 172.21.15.16:0/1900063804 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faca00cfc80 0x7faca00d00f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7faca7000640 1 -- 172.21.15.16:0/1900063804 >> 172.21.15.16:0/1900063804 conn(0x7faca009b130 msgr2=0x7faca00d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f770cc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77081fcf30 0x7f77081ef950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f770cc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77080cbcc0 con 0x7f76ec003c40 2024-09-10T21:10:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f770cc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f77080cbb40 con 0x7f77081fcf30 2024-09-10T21:10:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f770cc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f77080cbe40 con 0x7f77081fd870 2024-09-10T21:10:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7faca7000640 1 -- 172.21.15.16:0/1900063804 shutdown_connections 2024-09-10T21:10:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7ff163e00640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff14c072630 0x7ff14c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7ff163e00640 1 -- 172.21.15.16:0/3367395785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff15c092730 con 0x7ff16c0d2a20 2024-09-10T21:10:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7ff163e00640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] conn(0x7ff14c07a850 0x7ff14c07cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7ff163e00640 1 -- 172.21.15.16:0/3367395785 --> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff14c07d320 con 0x7ff14c07a850 2024-09-10T21:10:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7ff163e00640 1 -- 172.21.15.16:0/3367395785 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff15c099050 con 0x7ff16c0d2a20 2024-09-10T21:10:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f7705e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec003c40 0x7f77080c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f7705e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec003c40 0x7f77080c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38574/0 (socket says 172.21.15.16:38574) 2024-09-10T21:10:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f7705e00640 1 -- 172.21.15.16:0/1241951982 learned_addr learned my addr 172.21.15.16:0/1241951982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f7705400640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77081fd870 0x7f77080ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7feeb1200640 1 --2- 172.21.15.16:0/2812884087 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fee8c0726b0 0x7fee8c074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7feea8c00640 1 --2- 172.21.15.16:0/2812884087 >> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] conn(0x7fee8c07a930 0x7fee8c07cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7feea8c00640 1 -- 172.21.15.16:0/2812884087 --> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feea401fe80 con 0x7fee8c07a930 2024-09-10T21:10:41.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f7706800640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77081fcf30 0x7f77081ef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.110+0000 7feea8c00640 1 -- 172.21.15.16:0/2812884087 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7feea40992a0 con 0x7feeac15ae90 2024-09-10T21:10:41.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7feeabe00640 1 --2- 172.21.15.16:0/2812884087 >> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] conn(0x7fee8c07a930 0x7fee8c07cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7feeb1200640 1 --2- 172.21.15.16:0/2812884087 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fee8c0726b0 0x7fee8c074b70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0ae000 tx=0x7fee9c019660 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7feeabe00640 1 --2- 172.21.15.16:0/2812884087 >> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] conn(0x7fee8c07a930 0x7fee8c07cd50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7ff170e00640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] conn(0x7ff14c07a850 0x7ff14c07cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f7706800640 1 -- 172.21.15.16:0/1241951982 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77081fd870 msgr2=0x7f77080ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f7706800640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77081fd870 0x7f77080ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f7706800640 1 -- 172.21.15.16:0/1241951982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec003c40 msgr2=0x7f77080c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f7706800640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec003c40 0x7f77080c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f7706800640 1 -- 172.21.15.16:0/1241951982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77080c7f50 con 0x7f77081fcf30 2024-09-10T21:10:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f7706800640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77081fcf30 0x7f77081ef950 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f77080b3850 tx=0x7f76f4011740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f76fe000640 1 -- 172.21.15.16:0/1241951982 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4011e60 con 0x7f77081fcf30 2024-09-10T21:10:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f76fe000640 1 -- 172.21.15.16:0/1241951982 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76f4018020 con 0x7f77081fcf30 2024-09-10T21:10:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f76fe000640 1 -- 172.21.15.16:0/1241951982 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4005350 con 0x7f77081fcf30 2024-09-10T21:10:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7ff170e00640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] conn(0x7ff14c07a850 0x7ff14c07cc70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f770cc00640 1 -- 172.21.15.16:0/1241951982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77080c8230 con 0x7f77081fcf30 2024-09-10T21:10:41.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7f770cc00640 1 -- 172.21.15.16:0/1241951982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77080c87f0 con 0x7f77081fcf30 2024-09-10T21:10:41.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7f770cc00640 1 -- 172.21.15.16:0/1241951982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7708073dd0 con 0x7f77081fcf30 2024-09-10T21:10:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.114+0000 7faca7000640 1 -- 172.21.15.16:0/1900063804 wait complete. 2024-09-10T21:10:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7faca7000640 1 Processor -- start 2024-09-10T21:10:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7faca7000640 1 -- start start 2024-09-10T21:10:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7ff16b400640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff14c072630 0x7ff14c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7ff163e00640 1 -- 172.21.15.16:0/3367395785 <== osd.2 v2:172.21.15.16:6826/3506045643 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff14c07d320 con 0x7ff14c07a850 2024-09-10T21:10:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7ff16b400640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff14c072630 0x7ff14c074af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff16c166e50 tx=0x7ff158009060 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7fc59f200640 1 Processor -- start 2024-09-10T21:10:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7feea8c00640 1 -- 172.21.15.16:0/2812884087 <== osd.3 v2:172.21.15.16:6818/2694071396 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feea401fe80 con 0x7fee8c07a930 2024-09-10T21:10:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7faca7000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006e280 0x7faca015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7faca7000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faca00b0b40 0x7faca0207530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7faca7000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faca00cfc80 0x7faca00b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7fac9f400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faca00b0b40 0x7faca0207530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7faca7000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faca0160a80 con 0x7faca00cfc80 2024-09-10T21:10:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7fac9fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006e280 0x7faca015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7faca7000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faca0160900 con 0x7faca006e280 2024-09-10T21:10:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7fac9f400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faca00b0b40 0x7faca0207530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:43340/0 (socket says 172.21.15.16:43340) 2024-09-10T21:10:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7faca7000640 1 -- 172.21.15.16:0/1171934444 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7faca0160c00 con 0x7faca00b0b40 2024-09-10T21:10:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7fac9f400640 1 -- 172.21.15.16:0/1171934444 learned_addr learned my addr 172.21.15.16:0/1171934444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7faca5000640 1 --2- 172.21.15.16:0/1171934444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faca00cfc80 0x7faca00b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7fac9f400640 1 -- 172.21.15.16:0/1171934444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006e280 msgr2=0x7faca015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7fac9f400640 1 --2- 172.21.15.16:0/1171934444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006e280 0x7faca015d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7fac9f400640 1 -- 172.21.15.16:0/1171934444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faca00cfc80 msgr2=0x7faca00b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7fac9f400640 1 --2- 172.21.15.16:0/1171934444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faca00cfc80 0x7faca00b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.118+0000 7fac9f400640 1 -- 172.21.15.16:0/1171934444 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca00cd3e0 con 0x7faca00b0b40 2024-09-10T21:10:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7fac9f400640 1 --2- 172.21.15.16:0/1171934444 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faca00b0b40 0x7faca0207530 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00c400 tx=0x7fac8c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7f76fe000640 1 -- 172.21.15.16:0/1241951982 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f76f4010040 con 0x7f77081fcf30 2024-09-10T21:10:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7fac97e00640 1 -- 172.21.15.16:0/1171934444 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c004220 con 0x7faca00b0b40 2024-09-10T21:10:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7fac97e00640 1 -- 172.21.15.16:0/1171934444 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac8c005030 con 0x7faca00b0b40 2024-09-10T21:10:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7fac97e00640 1 -- 172.21.15.16:0/1171934444 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c005500 con 0x7faca00b0b40 2024-09-10T21:10:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7faca7000640 1 -- 172.21.15.16:0/1171934444 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca00b4330 con 0x7faca00b0b40 2024-09-10T21:10:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7fc59f200640 1 -- start start 2024-09-10T21:10:41.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7faca7000640 1 -- 172.21.15.16:0/1171934444 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faca015ce70 con 0x7faca00b0b40 2024-09-10T21:10:41.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7fc59f200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc584005240 0x7fc5a01f8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7fc59f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a0202640 0x7fc5a015cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7fc59f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a015d2e0 0x7fc5a02015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7faca7000640 1 -- 172.21.15.16:0/1171934444 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac78001040 con 0x7faca00b0b40 2024-09-10T21:10:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7f76fe000640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76e80726b0 0x7f76e8074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7f76fe000640 1 -- 172.21.15.16:0/1241951982 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f76f4096690 con 0x7f77081fcf30 2024-09-10T21:10:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7f76fe000640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] conn(0x7f76e807a930 0x7f76e807cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7f76fe000640 1 -- 172.21.15.16:0/1241951982 --> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76e807d400 con 0x7f76e807a930 2024-09-10T21:10:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7f76fe000640 1 -- 172.21.15.16:0/1241951982 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f76f4096a80 con 0x7f77081fcf30 2024-09-10T21:10:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7fc59d400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a0202640 0x7fc5a015cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.122+0000 7fc59d400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a0202640 0x7fc5a015cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:53004/0 (socket says 172.21.15.16:53004) 2024-09-10T21:10:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7f7705400640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] conn(0x7f76e807a930 0x7f76e807cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7f7705e00640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76e80726b0 0x7f76e8074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59de00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc584005240 0x7fc5a01f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59de00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc584005240 0x7fc5a01f8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:43348/0 (socket says 172.21.15.16:43348) 2024-09-10T21:10:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59d400640 1 -- 172.21.15.16:0/1857696312 learned_addr learned my addr 172.21.15.16:0/1857696312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59f200640 1 -- 172.21.15.16:0/1857696312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a00cffb0 con 0x7fc5a015d2e0 2024-09-10T21:10:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59f200640 1 -- 172.21.15.16:0/1857696312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a00cfe30 con 0x7fc5a0202640 2024-09-10T21:10:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59f200640 1 -- 172.21.15.16:0/1857696312 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a00b6170 con 0x7fc584005240 2024-09-10T21:10:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59e800640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a015d2e0 0x7fc5a02015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59de00640 1 -- 172.21.15.16:0/1857696312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a0202640 msgr2=0x7fc5a015cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59de00640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a0202640 0x7fc5a015cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59de00640 1 -- 172.21.15.16:0/1857696312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a015d2e0 msgr2=0x7fc5a02015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59de00640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a015d2e0 0x7fc5a02015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59de00640 1 -- 172.21.15.16:0/1857696312 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a0202360 con 0x7fc584005240 2024-09-10T21:10:41.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59e800640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a015d2e0 0x7fc5a02015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:41.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59d400640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a0202640 0x7fc5a015cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:41.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7f7705400640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] conn(0x7f76e807a930 0x7f76e807cd50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7f7705e00640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76e80726b0 0x7f76e8074b70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7700001810 tx=0x7f77000b2040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59de00640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc584005240 0x7fc5a01f8090 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc598010a40 tx=0x7fc598010e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fac97e00640 1 -- 172.21.15.16:0/1171934444 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fac8c004670 con 0x7faca00b0b40 2024-09-10T21:10:41.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7f76fe000640 1 -- 172.21.15.16:0/1241951982 <== osd.7 v2:172.21.15.52:6808/480240003 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76e807d400 con 0x7f76e807a930 2024-09-10T21:10:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc596000640 1 -- 172.21.15.16:0/1857696312 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59800ece0 con 0x7fc584005240 2024-09-10T21:10:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc596000640 1 -- 172.21.15.16:0/1857696312 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc59800ee80 con 0x7fc584005240 2024-09-10T21:10:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59f200640 1 -- 172.21.15.16:0/1857696312 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a0157930 con 0x7fc584005240 2024-09-10T21:10:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc59f200640 1 -- 172.21.15.16:0/1857696312 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a015d760 con 0x7fc584005240 2024-09-10T21:10:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.126+0000 7fc596000640 1 -- 172.21.15.16:0/1857696312 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59800ece0 con 0x7fc584005240 2024-09-10T21:10:41.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.130+0000 7fac97e00640 1 --2- 172.21.15.16:0/1171934444 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fac800726b0 0x7fac80074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.130+0000 7fac9fe00640 1 --2- 172.21.15.16:0/1171934444 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fac800726b0 0x7fac80074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.130+0000 7fac97e00640 1 -- 172.21.15.16:0/1171934444 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fac8c0930c0 con 0x7faca00b0b40 2024-09-10T21:10:41.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.130+0000 7fc59f200640 1 -- 172.21.15.16:0/1857696312 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5a0157720 con 0x7fc584005240 2024-09-10T21:10:41.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.130+0000 7fc596000640 1 -- 172.21.15.16:0/1857696312 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc59800f040 con 0x7fc584005240 2024-09-10T21:10:41.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.130+0000 7fac9fe00640 1 --2- 172.21.15.16:0/1171934444 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fac800726b0 0x7fac80074b70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faca00510a0 tx=0x7fac98009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.130+0000 7fac97e00640 1 --2- 172.21.15.16:0/1171934444 >> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] conn(0x7fac8007a930 0x7fac8007cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.130+0000 7fac97e00640 1 -- 172.21.15.16:0/1171934444 --> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac8c003e70 con 0x7fac8007a930 2024-09-10T21:10:41.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.130+0000 7fac97e00640 1 -- 172.21.15.16:0/1171934444 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fac8c0934b0 con 0x7faca00b0b40 2024-09-10T21:10:41.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.130+0000 7faca5000640 1 --2- 172.21.15.16:0/1171934444 >> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] conn(0x7fac8007a930 0x7fac8007cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.130+0000 7faca5000640 1 --2- 172.21.15.16:0/1171934444 >> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] conn(0x7fac8007a930 0x7fac8007cd50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.134+0000 7fac97e00640 1 -- 172.21.15.16:0/1171934444 <== osd.6 v2:172.21.15.52:6800/2640674683 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac8c003e70 con 0x7fac8007a930 2024-09-10T21:10:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.134+0000 7fc596000640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc57c0727e0 0x7fc57c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.134+0000 7fc596000640 1 -- 172.21.15.16:0/1857696312 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc598093b70 con 0x7fc584005240 2024-09-10T21:10:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.134+0000 7fc596000640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] conn(0x7fc57c07a9e0 0x7fc57c07ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.134+0000 7fc59d400640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc57c0727e0 0x7fc57c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.134+0000 7fc596000640 1 -- 172.21.15.16:0/1857696312 --> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc598007e90 con 0x7fc57c07a9e0 2024-09-10T21:10:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.134+0000 7fc59e800640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] conn(0x7fc57c07a9e0 0x7fc57c07ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.134+0000 7fc596000640 1 -- 172.21.15.16:0/1857696312 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc598094000 con 0x7fc584005240 2024-09-10T21:10:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.138+0000 7fc59e800640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] conn(0x7fc57c07a9e0 0x7fc57c07ce00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.138+0000 7fc596000640 1 -- 172.21.15.16:0/1857696312 <== osd.5 v2:172.21.15.52:6816/1432062268 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc598007e90 con 0x7fc57c07a9e0 2024-09-10T21:10:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.138+0000 7fc59d400640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc57c0727e0 0x7fc57c074ca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5a009e520 tx=0x7fc5880016f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.142+0000 7ff172e00640 1 -- 172.21.15.16:0/3367395785 --> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff16c074af0 con 0x7ff14c07a850 2024-09-10T21:10:41.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.142+0000 7f1843400640 1 -- 172.21.15.16:0/1199696154 --> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f182c002de0 con 0x7f183807a800 2024-09-10T21:10:41.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f184e000640 1 -- 172.21.15.16:0/1199696154 <== osd.4 v2:172.21.15.52:6824/447485120 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f182c002de0 con 0x7f183807a800 2024-09-10T21:10:41.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7ff163e00640 1 -- 172.21.15.16:0/3367395785 <== osd.2 v2:172.21.15.16:6826/3506045643 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff16c074af0 con 0x7ff14c07a850 2024-09-10T21:10:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 --> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9fd40c7710 con 0x7f9fb407a780 2024-09-10T21:10:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7fc881000640 1 -- 172.21.15.16:0/2914461226 --> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc87c152840 con 0x7fc85c07a850 2024-09-10T21:10:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7ff172e00640 1 -- 172.21.15.16:0/3367395785 >> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] conn(0x7ff14c07a850 msgr2=0x7ff14c07cc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7ff172e00640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] conn(0x7ff14c07a850 0x7ff14c07cc70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f1843400640 1 -- 172.21.15.16:0/1199696154 >> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] conn(0x7f183807a800 msgr2=0x7f183807cc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f1843400640 1 --2- 172.21.15.16:0/1199696154 >> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] conn(0x7f183807a800 0x7f183807cc20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f1843400640 1 -- 172.21.15.16:0/1199696154 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18380725e0 msgr2=0x7f1838074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f1843400640 1 --2- 172.21.15.16:0/1199696154 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18380725e0 0x7f1838074aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18440058d0 tx=0x7f1844005860 comp rx=0 tx=0).stop 2024-09-10T21:10:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f1843400640 1 -- 172.21.15.16:0/1199696154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18580d03d0 msgr2=0x7f185816af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f1843400640 1 --2- 172.21.15.16:0/1199696154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18580d03d0 0x7f185816af60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f184801aed0 tx=0x7f1848005ca0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7ff172e00640 1 -- 172.21.15.16:0/3367395785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff14c072630 msgr2=0x7ff14c074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7ff172e00640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff14c072630 0x7ff14c074af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff16c166e50 tx=0x7ff158009060 comp rx=0 tx=0).stop 2024-09-10T21:10:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7ff172e00640 1 -- 172.21.15.16:0/3367395785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c0d2a20 msgr2=0x7ff16c152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7ff172e00640 1 --2- 172.21.15.16:0/3367395785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c0d2a20 0x7ff16c152630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00c730 tx=0x7ff15c00cbf0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f1856800640 1 -- 172.21.15.16:0/1199696154 reap_dead start 2024-09-10T21:10:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7ff170e00640 1 -- 172.21.15.16:0/3367395785 reap_dead start 2024-09-10T21:10:41.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7ff172e00640 1 -- 172.21.15.16:0/3367395785 shutdown_connections 2024-09-10T21:10:41.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7ff172e00640 1 -- 172.21.15.16:0/3367395785 >> 172.21.15.16:0/3367395785 conn(0x7ff16c0d1de0 msgr2=0x7ff16c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7ff172e00640 1 -- 172.21.15.16:0/3367395785 shutdown_connections 2024-09-10T21:10:41.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7ff172e00640 1 -- 172.21.15.16:0/3367395785 wait complete. 2024-09-10T21:10:41.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f9fd0c00640 1 -- 172.21.15.16:0/497163231 <== osd.0 v2:172.21.15.16:6810/1824652127 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9fd40c7710 con 0x7f9fb407a780 2024-09-10T21:10:41.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7fc872000640 1 -- 172.21.15.16:0/2914461226 <== osd.1 v2:172.21.15.16:6802/4097098606 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc87c152840 con 0x7fc85c07a850 2024-09-10T21:10:41.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f1843400640 1 -- 172.21.15.16:0/1199696154 shutdown_connections 2024-09-10T21:10:41.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f1843400640 1 -- 172.21.15.16:0/1199696154 >> 172.21.15.16:0/1199696154 conn(0x7f18580d4060 msgr2=0x7f1858175040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f1843400640 1 -- 172.21.15.16:0/1199696154 shutdown_connections 2024-09-10T21:10:41.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f1843400640 1 -- 172.21.15.16:0/1199696154 wait complete. 2024-09-10T21:10:41.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 >> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] conn(0x7f9fb407a780 msgr2=0x7f9fb407cba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.146+0000 7f9fdb400640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] conn(0x7f9fb407a780 0x7f9fb407cba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7fc881000640 1 -- 172.21.15.16:0/2914461226 >> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] conn(0x7fc85c07a850 msgr2=0x7fc85c07cc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7fc881000640 1 --2- 172.21.15.16:0/2914461226 >> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] conn(0x7fc85c07a850 0x7fc85c07cc70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7fc881000640 1 -- 172.21.15.16:0/2914461226 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc85c0726b0 msgr2=0x7fc85c074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7fc881000640 1 --2- 172.21.15.16:0/2914461226 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc85c0726b0 0x7fc85c074b70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc86c003bd0 tx=0x7fc86c0079c0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7fc881000640 1 -- 172.21.15.16:0/2914461226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c15f400 msgr2=0x7fc87c15e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7fc881000640 1 --2- 172.21.15.16:0/2914461226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c15f400 0x7fc87c15e140 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc87401cab0 tx=0x7fc8740a5a00 comp rx=0 tx=0).stop 2024-09-10T21:10:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7fc87a800640 1 -- 172.21.15.16:0/2914461226 reap_dead start 2024-09-10T21:10:41.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7fc881000640 1 -- 172.21.15.16:0/2914461226 shutdown_connections 2024-09-10T21:10:41.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7fc881000640 1 -- 172.21.15.16:0/2914461226 >> 172.21.15.16:0/2914461226 conn(0x7fc87c09b130 msgr2=0x7fc87c0cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7fc881000640 1 -- 172.21.15.16:0/2914461226 shutdown_connections 2024-09-10T21:10:41.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7fc881000640 1 -- 172.21.15.16:0/2914461226 wait complete. 2024-09-10T21:10:41.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fb40725e0 msgr2=0x7f9fb4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 --> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feeac159d40 con 0x7fee8c07a930 2024-09-10T21:10:41.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7f9fdb400640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fb40725e0 0x7f9fb4074aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4076820 tx=0x7f9fc8005d00 comp rx=0 tx=0).stop 2024-09-10T21:10:41.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fd414e8c0 msgr2=0x7f9fd406f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7f9fdb400640 1 --2- 172.21.15.16:0/497163231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fd414e8c0 0x7f9fd406f950 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4076480 tx=0x7f9fcc00ed70 comp rx=0 tx=0).stop 2024-09-10T21:10:41.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7feea8c00640 1 -- 172.21.15.16:0/2812884087 <== osd.3 v2:172.21.15.16:6818/2694071396 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feeac159d40 con 0x7fee8c07a930 2024-09-10T21:10:41.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.150+0000 7f9fd9400640 1 -- 172.21.15.16:0/497163231 reap_dead start 2024-09-10T21:10:41.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.154+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 shutdown_connections 2024-09-10T21:10:41.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.154+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 >> 172.21.15.16:0/497163231 conn(0x7f9fd40cfb20 msgr2=0x7f9fd415a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.154+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 shutdown_connections 2024-09-10T21:10:41.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.154+0000 7f9fdb400640 1 -- 172.21.15.16:0/497163231 wait complete. 2024-09-10T21:10:41.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.154+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 >> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] conn(0x7fee8c07a930 msgr2=0x7fee8c07cd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.154+0000 7feeb3c00640 1 --2- 172.21.15.16:0/2812884087 >> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] conn(0x7fee8c07a930 0x7fee8c07cd50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.154+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fee8c0726b0 msgr2=0x7fee8c074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.154+0000 7feeb3c00640 1 --2- 172.21.15.16:0/2812884087 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fee8c0726b0 0x7fee8c074b70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0ae000 tx=0x7fee9c019660 comp rx=0 tx=0).stop 2024-09-10T21:10:41.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.154+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeac15ae90 msgr2=0x7feeac0c64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.154+0000 7feeb3c00640 1 --2- 172.21.15.16:0/2812884087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feeac15ae90 0x7feeac0c64f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feea400ea30 tx=0x7feea400eef0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.154+0000 7feeb1c00640 1 -- 172.21.15.16:0/2812884087 reap_dead start 2024-09-10T21:10:41.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.158+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 shutdown_connections 2024-09-10T21:10:41.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.158+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 >> 172.21.15.16:0/2812884087 conn(0x7feeac0d0c30 msgr2=0x7feeac0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.158+0000 7f770cc00640 1 -- 172.21.15.16:0/1241951982 --> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f77080d0ae0 con 0x7f76e807a930 2024-09-10T21:10:41.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.158+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 shutdown_connections 2024-09-10T21:10:41.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.158+0000 7feeb3c00640 1 -- 172.21.15.16:0/2812884087 wait complete. 2024-09-10T21:10:41.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.158+0000 7f76fe000640 1 -- 172.21.15.16:0/1241951982 <== osd.7 v2:172.21.15.52:6808/480240003 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f77080d0ae0 con 0x7f76e807a930 2024-09-10T21:10:41.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.158+0000 7f770cc00640 1 -- 172.21.15.16:0/1241951982 >> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] conn(0x7f76e807a930 msgr2=0x7f76e807cd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.158+0000 7f770cc00640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] conn(0x7f76e807a930 0x7f76e807cd50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.162+0000 7f770cc00640 1 -- 172.21.15.16:0/1241951982 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76e80726b0 msgr2=0x7f76e8074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.162+0000 7faca7000640 1 -- 172.21.15.16:0/1171934444 --> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac78002de0 con 0x7fac8007a930 2024-09-10T21:10:41.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.162+0000 7f770cc00640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76e80726b0 0x7f76e8074b70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7700001810 tx=0x7f77000b2040 comp rx=0 tx=0).stop 2024-09-10T21:10:41.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.162+0000 7f770cc00640 1 -- 172.21.15.16:0/1241951982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77081fcf30 msgr2=0x7f77081ef950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.162+0000 7f770cc00640 1 --2- 172.21.15.16:0/1241951982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77081fcf30 0x7f77081ef950 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f77080b3850 tx=0x7f76f4011740 comp rx=0 tx=0).stop 2024-09-10T21:10:41.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.162+0000 7fac97e00640 1 -- 172.21.15.16:0/1171934444 <== osd.6 v2:172.21.15.52:6800/2640674683 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fac78002de0 con 0x7fac8007a930 2024-09-10T21:10:41.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.162+0000 7f7706800640 1 -- 172.21.15.16:0/1241951982 reap_dead start 2024-09-10T21:10:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.162+0000 7faca7000640 1 -- 172.21.15.16:0/1171934444 >> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] conn(0x7fac8007a930 msgr2=0x7fac8007cd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.162+0000 7faca7000640 1 --2- 172.21.15.16:0/1171934444 >> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] conn(0x7fac8007a930 0x7fac8007cd50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.166+0000 7faca7000640 1 -- 172.21.15.16:0/1171934444 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fac800726b0 msgr2=0x7fac80074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.166+0000 7faca7000640 1 --2- 172.21.15.16:0/1171934444 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fac800726b0 0x7fac80074b70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faca00510a0 tx=0x7fac98009040 comp rx=0 tx=0).stop 2024-09-10T21:10:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.166+0000 7faca7000640 1 -- 172.21.15.16:0/1171934444 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faca00b0b40 msgr2=0x7faca0207530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.166+0000 7faca7000640 1 --2- 172.21.15.16:0/1171934444 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faca00b0b40 0x7faca0207530 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00c400 tx=0x7fac8c00c8c0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.166+0000 7faca5000640 1 -- 172.21.15.16:0/1171934444 reap_dead start 2024-09-10T21:10:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.166+0000 7faca7000640 1 -- 172.21.15.16:0/1171934444 shutdown_connections 2024-09-10T21:10:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.166+0000 7faca7000640 1 -- 172.21.15.16:0/1171934444 >> 172.21.15.16:0/1171934444 conn(0x7faca009b130 msgr2=0x7faca0071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.166+0000 7faca7000640 1 -- 172.21.15.16:0/1171934444 shutdown_connections 2024-09-10T21:10:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.166+0000 7faca7000640 1 -- 172.21.15.16:0/1171934444 wait complete. 2024-09-10T21:10:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.170+0000 7f770cc00640 1 -- 172.21.15.16:0/1241951982 shutdown_connections 2024-09-10T21:10:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.170+0000 7f770cc00640 1 -- 172.21.15.16:0/1241951982 >> 172.21.15.16:0/1241951982 conn(0x7f77080d0d50 msgr2=0x7f77080b61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.170+0000 7f770cc00640 1 -- 172.21.15.16:0/1241951982 shutdown_connections 2024-09-10T21:10:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.170+0000 7f770cc00640 1 -- 172.21.15.16:0/1241951982 wait complete. 2024-09-10T21:10:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.170+0000 7fc59f200640 1 -- 172.21.15.16:0/1857696312 --> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5a00744f0 con 0x7fc57c07a9e0 2024-09-10T21:10:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.170+0000 7fc596000640 1 -- 172.21.15.16:0/1857696312 <== osd.5 v2:172.21.15.52:6816/1432062268 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc5a00744f0 con 0x7fc57c07a9e0 2024-09-10T21:10:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.170+0000 7fc58f400640 1 -- 172.21.15.16:0/1857696312 >> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] conn(0x7fc57c07a9e0 msgr2=0x7fc57c07ce00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.170+0000 7fc58f400640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] conn(0x7fc57c07a9e0 0x7fc57c07ce00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.170+0000 7fc58f400640 1 -- 172.21.15.16:0/1857696312 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc57c0727e0 msgr2=0x7fc57c074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.170+0000 7fc58f400640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc57c0727e0 0x7fc57c074ca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5a009e520 tx=0x7fc5880016f0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.170+0000 7fc58f400640 1 -- 172.21.15.16:0/1857696312 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc584005240 msgr2=0x7fc5a01f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.170+0000 7fc58f400640 1 --2- 172.21.15.16:0/1857696312 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc584005240 0x7fc5a01f8090 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc598010a40 tx=0x7fc598010e40 comp rx=0 tx=0).stop 2024-09-10T21:10:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.174+0000 7fc59e800640 1 -- 172.21.15.16:0/1857696312 reap_dead start 2024-09-10T21:10:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.174+0000 7fc58f400640 1 -- 172.21.15.16:0/1857696312 shutdown_connections 2024-09-10T21:10:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.174+0000 7fc58f400640 1 -- 172.21.15.16:0/1857696312 >> 172.21.15.16:0/1857696312 conn(0x7fc5a00b1d60 msgr2=0x7fc5a00b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.178+0000 7fc58f400640 1 -- 172.21.15.16:0/1857696312 shutdown_connections 2024-09-10T21:10:41.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.178+0000 7fc58f400640 1 -- 172.21.15.16:0/1857696312 wait complete. 2024-09-10T21:10:41.193 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-10T21:10:41.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T21:10:41.197 INFO:teuthology.orchestra.run.smithi016.stdout:60129542150 2024-09-10T21:10:41.198 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-09-10T21:10:41.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T21:10:41.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T21:10:41.201 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-09-10T21:10:41.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T21:10:41.205 INFO:teuthology.orchestra.run.smithi016.stdout:64424509446 2024-09-10T21:10:41.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T21:10:41.211 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-09-10T21:10:41.212 INFO:teuthology.orchestra.run.smithi016.stdout:60129542151 2024-09-10T21:10:41.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T21:10:41.213 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T21:10:41.216 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-10T21:10:41.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T21:10:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.438+0000 7fe1dfa00640 1 -- 172.21.15.16:0/224806716 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1d80cc690 msgr2=0x7fe1d81581a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.438+0000 7fe1dfa00640 1 --2- 172.21.15.16:0/224806716 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1d80cc690 0x7fe1d81581a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009930 tx=0x7fe1c401c8d0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.438+0000 7fe1dfa00640 1 -- 172.21.15.16:0/224806716 shutdown_connections 2024-09-10T21:10:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.438+0000 7fe1dfa00640 1 --2- 172.21.15.16:0/224806716 >> v2:172.21.15.52:3300/0 conn(0x7fe1d80b4550 0x7fe1d8157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.438+0000 7fe1dfa00640 1 --2- 172.21.15.16:0/224806716 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1d80cc690 0x7fe1d81581a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.438+0000 7fe1dfa00640 1 -- 172.21.15.16:0/224806716 >> 172.21.15.16:0/224806716 conn(0x7fe1d806cbe0 msgr2=0x7fe1d806cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.438+0000 7fe1dfa00640 1 -- 172.21.15.16:0/224806716 shutdown_connections 2024-09-10T21:10:41.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.438+0000 7fe1dfa00640 1 -- 172.21.15.16:0/224806716 wait complete. 2024-09-10T21:10:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.442+0000 7fe1dfa00640 1 Processor -- start 2024-09-10T21:10:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.442+0000 7fe1dfa00640 1 -- start start 2024-09-10T21:10:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1dfa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1d80b4550 0x7fe1d815b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1dfa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1d80cc2b0 0x7fe1d8142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1dfa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d80cc690 0x7fe1d8157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1d7e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1d80cc2b0 0x7fe1d8142390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1d7e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1d80cc2b0 0x7fe1d8142390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:43356/0 (socket says 172.21.15.16:43356) 2024-09-10T21:10:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1d7e00640 1 -- 172.21.15.16:0/3213307328 learned_addr learned my addr 172.21.15.16:0/3213307328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1dd000640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1d80b4550 0x7fe1d815b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1dfa00640 1 -- 172.21.15.16:0/3213307328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d80ce6b0 con 0x7fe1d80cc690 2024-09-10T21:10:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1dfa00640 1 -- 172.21.15.16:0/3213307328 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d80ce530 con 0x7fe1d80b4550 2024-09-10T21:10:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1dfa00640 1 -- 172.21.15.16:0/3213307328 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d816f470 con 0x7fe1d80cc2b0 2024-09-10T21:10:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1dda00640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d80cc690 0x7fe1d8157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1d7e00640 1 -- 172.21.15.16:0/3213307328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1d80b4550 msgr2=0x7fe1d815b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1d7e00640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1d80b4550 0x7fe1d815b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1d7e00640 1 -- 172.21.15.16:0/3213307328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d80cc690 msgr2=0x7fe1d8157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1d7e00640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d80cc690 0x7fe1d8157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1d7e00640 1 -- 172.21.15.16:0/3213307328 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d815b6c0 con 0x7fe1d80cc2b0 2024-09-10T21:10:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1dd000640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1d80b4550 0x7fe1d815b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1d7e00640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1d80cc2b0 0x7fe1d8142390 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800d980 tx=0x7fe1c800de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1d4c00640 1 -- 172.21.15.16:0/3213307328 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8017070 con 0x7fe1d80cc2b0 2024-09-10T21:10:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1d4c00640 1 -- 172.21.15.16:0/3213307328 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1c8009d20 con 0x7fe1d80cc2b0 2024-09-10T21:10:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.446+0000 7fe1dfa00640 1 -- 172.21.15.16:0/3213307328 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d815b9a0 con 0x7fe1d80cc2b0 2024-09-10T21:10:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.450+0000 7fe1dfa00640 1 -- 172.21.15.16:0/3213307328 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d81428d0 con 0x7fe1d80cc2b0 2024-09-10T21:10:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.450+0000 7fe1d4c00640 1 -- 172.21.15.16:0/3213307328 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80106c0 con 0x7fe1d80cc2b0 2024-09-10T21:10:41.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.450+0000 7fe1dfa00640 1 -- 172.21.15.16:0/3213307328 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b00052a0 con 0x7fe1d80cc2b0 2024-09-10T21:10:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.454+0000 7fdb8be00640 1 -- 172.21.15.16:0/2523359782 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb8c0705f0 msgr2=0x7fdb8c143280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.454+0000 7fdb8be00640 1 --2- 172.21.15.16:0/2523359782 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb8c0705f0 0x7fdb8c143280 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009870 tx=0x7fdb7c01f050 comp rx=0 tx=0).stop 2024-09-10T21:10:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.454+0000 7fdb8be00640 1 -- 172.21.15.16:0/2523359782 shutdown_connections 2024-09-10T21:10:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.454+0000 7fdb8be00640 1 --2- 172.21.15.16:0/2523359782 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb8c0705f0 0x7fdb8c143280 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.454+0000 7fdb8be00640 1 -- 172.21.15.16:0/2523359782 >> 172.21.15.16:0/2523359782 conn(0x7fdb8c0b1710 msgr2=0x7fdb8c0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.454+0000 7fe1d4c00640 1 -- 172.21.15.16:0/3213307328 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe1c8010860 con 0x7fe1d80cc2b0 2024-09-10T21:10:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.458+0000 7fdb8be00640 1 -- 172.21.15.16:0/2523359782 shutdown_connections 2024-09-10T21:10:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.458+0000 7fe1d4c00640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe1b80726b0 0x7fe1b8074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.458+0000 7fe1d4c00640 1 -- 172.21.15.16:0/3213307328 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe1c80935a0 con 0x7fe1d80cc2b0 2024-09-10T21:10:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.458+0000 7f0158600640 1 -- 172.21.15.16:0/1002061471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01500b3390 msgr2=0x7f015015db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.458+0000 7f0158600640 1 --2- 172.21.15.16:0/1002061471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01500b3390 0x7f015015db10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f013c009840 tx=0x7f013c01c820 comp rx=0 tx=0).stop 2024-09-10T21:10:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.458+0000 7fe1dd000640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe1b80726b0 0x7fe1b8074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.458+0000 7f0158600640 1 -- 172.21.15.16:0/1002061471 shutdown_connections 2024-09-10T21:10:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.458+0000 7f0158600640 1 --2- 172.21.15.16:0/1002061471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01500b3390 0x7f015015db10 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f013c009840 tx=0x7f013c01c820 comp rx=0 tx=0).stop 2024-09-10T21:10:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.458+0000 7f0158600640 1 --2- 172.21.15.16:0/1002061471 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f015015e050 0x7f01500d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.458+0000 7f0158600640 1 -- 172.21.15.16:0/1002061471 >> 172.21.15.16:0/1002061471 conn(0x7f015009d150 msgr2=0x7f01500d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.458+0000 7fdb8be00640 1 -- 172.21.15.16:0/2523359782 wait complete. 2024-09-10T21:10:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.458+0000 7fdb8be00640 1 Processor -- start 2024-09-10T21:10:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fe1d4c00640 1 -- 172.21.15.16:0/3213307328 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe1c805e6a0 con 0x7fe1d80cc2b0 2024-09-10T21:10:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fe1dd000640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe1b80726b0 0x7fe1b8074b70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00012f0 tx=0x7fe1d0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8be00640 1 -- start start 2024-09-10T21:10:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb8c0705f0 0x7fdb8c131b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c072540 0x7fdb8c0b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8be00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb8c1437c0 0x7fdb8c1f7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7f0158600640 1 -- 172.21.15.16:0/1002061471 shutdown_connections 2024-09-10T21:10:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c06ec10 con 0x7fdb8c072540 2024-09-10T21:10:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8be00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c072e20 con 0x7fdb8c0705f0 2024-09-10T21:10:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8be00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fdb8c06ed90 con 0x7fdb8c1437c0 2024-09-10T21:10:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8aa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb8c0705f0 0x7fdb8c131b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8aa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb8c0705f0 0x7fdb8c131b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:53034/0 (socket says 172.21.15.16:53034) 2024-09-10T21:10:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8aa00640 1 -- 172.21.15.16:0/1151149139 learned_addr learned my addr 172.21.15.16:0/1151149139 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8b400640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb8c1437c0 0x7fdb8c1f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8a000640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c072540 0x7fdb8c0b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8b400640 1 -- 172.21.15.16:0/1151149139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb8c0705f0 msgr2=0x7fdb8c131b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8b400640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb8c0705f0 0x7fdb8c131b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8b400640 1 -- 172.21.15.16:0/1151149139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c072540 msgr2=0x7fdb8c0b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8b400640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c072540 0x7fdb8c0b01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.462+0000 7fdb8b400640 1 -- 172.21.15.16:0/1151149139 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8c132050 con 0x7fdb8c1437c0 2024-09-10T21:10:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.466+0000 7f0158600640 1 -- 172.21.15.16:0/1002061471 wait complete. 2024-09-10T21:10:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.466+0000 7fdb8b400640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb8c1437c0 0x7fdb8c1f7370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400caa0 tx=0x7fdb8400cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.466+0000 7fdb8a000640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c072540 0x7fdb8c0b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.466+0000 7fdb82a00640 1 -- 172.21.15.16:0/1151149139 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84016020 con 0x7fdb8c1437c0 2024-09-10T21:10:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.466+0000 7fdb82a00640 1 -- 172.21.15.16:0/1151149139 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb8400a930 con 0x7fdb8c1437c0 2024-09-10T21:10:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.466+0000 7fdb82a00640 1 -- 172.21.15.16:0/1151149139 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8400ad60 con 0x7fdb8c1437c0 2024-09-10T21:10:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.466+0000 7fdb8be00640 1 -- 172.21.15.16:0/1151149139 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8c159900 con 0x7fdb8c1437c0 2024-09-10T21:10:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.466+0000 7fdb8be00640 1 -- 172.21.15.16:0/1151149139 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8c12f0c0 con 0x7fdb8c1437c0 2024-09-10T21:10:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.466+0000 7f0158600640 1 Processor -- start 2024-09-10T21:10:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7fdb8be00640 1 -- 172.21.15.16:0/1151149139 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb640052a0 con 0x7fdb8c1437c0 2024-09-10T21:10:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7fdb82a00640 1 -- 172.21.15.16:0/1151149139 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdb84012420 con 0x7fdb8c1437c0 2024-09-10T21:10:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7fd1fbe00640 1 -- 172.21.15.16:0/3473416279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1f401c370 msgr2=0x7fd1f4006650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7fd1fbe00640 1 --2- 172.21.15.16:0/3473416279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1f401c370 0x7fd1f4006650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc06bad0 tx=0x7fd1fc098000 comp rx=0 tx=0).stop 2024-09-10T21:10:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7fd1fbe00640 1 -- 172.21.15.16:0/3473416279 shutdown_connections 2024-09-10T21:10:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7fd1fbe00640 1 --2- 172.21.15.16:0/3473416279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1f401c370 0x7fd1f4006650 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7fd1fbe00640 1 --2- 172.21.15.16:0/3473416279 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd1f4036700 0x7fd1f4006110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7fd1fbe00640 1 -- 172.21.15.16:0/3473416279 >> 172.21.15.16:0/3473416279 conn(0x7fd1f4054570 msgr2=0x7fd1f4054980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7f0158600640 1 -- start start 2024-09-10T21:10:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7f0158600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01500d08f0 0x7f01500cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7f0158600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015015e050 0x7f0150156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7f0158600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0150157d40 0x7f01500b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7f0158600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f015016f690 con 0x7f015015e050 2024-09-10T21:10:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7f0158600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f015016f510 con 0x7f0150157d40 2024-09-10T21:10:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7f0158600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f015016f810 con 0x7f01500d08f0 2024-09-10T21:10:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7f0155c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01500d08f0 0x7f01500cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7f0155c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01500d08f0 0x7f01500cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:43420/0 (socket says 172.21.15.16:43420) 2024-09-10T21:10:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7f0155c00640 1 -- 172.21.15.16:0/1881533557 learned_addr learned my addr 172.21.15.16:0/1881533557 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7f0155c00640 1 -- 172.21.15.16:0/1881533557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0150157d40 msgr2=0x7f01500b6d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7f0155c00640 1 --2- 172.21.15.16:0/1881533557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0150157d40 0x7f01500b6d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7f0155c00640 1 -- 172.21.15.16:0/1881533557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015015e050 msgr2=0x7f0150156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7f0155c00640 1 --2- 172.21.15.16:0/1881533557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015015e050 0x7f0150156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7f0155c00640 1 -- 172.21.15.16:0/1881533557 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0150158240 con 0x7f01500d08f0 2024-09-10T21:10:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7fdb82a00640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdb6c0726e0 0x7fdb6c074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.470+0000 7fdb82a00640 1 -- 172.21.15.16:0/1151149139 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdb84093290 con 0x7fdb8c1437c0 2024-09-10T21:10:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7f0155c00640 1 --2- 172.21.15.16:0/1881533557 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01500d08f0 0x7f01500cd8d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f014400cba0 tx=0x7f014400cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7fd1fbe00640 1 -- 172.21.15.16:0/3473416279 shutdown_connections 2024-09-10T21:10:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7fd1fbe00640 1 -- 172.21.15.16:0/3473416279 wait complete. 2024-09-10T21:10:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7fdb8aa00640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdb6c0726e0 0x7fdb6c074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7fdb8aa00640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdb6c0726e0 0x7fdb6c074ba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009870 tx=0x7fdb7c01d890 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7fd1fbe00640 1 Processor -- start 2024-09-10T21:10:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7fd1fbe00640 1 -- start start 2024-09-10T21:10:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1fbe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd1f400ba10 0x7fd1f40de280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f401c370 0x7fd1f4180d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1fbe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1f4036700 0x7fd1f4183b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1fbe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f40eb2e0 con 0x7fd1f401c370 2024-09-10T21:10:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1fbe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f40eb160 con 0x7fd1f4036700 2024-09-10T21:10:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1fbe00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f40eb460 con 0x7fd1f400ba10 2024-09-10T21:10:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1faa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd1f400ba10 0x7fd1f40de280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1faa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd1f400ba10 0x7fd1f40de280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:43424/0 (socket says 172.21.15.16:43424) 2024-09-10T21:10:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7f7246000640 1 -- 172.21.15.16:0/1984618279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72400b5510 msgr2=0x7f72400b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7f7246000640 1 --2- 172.21.15.16:0/1984618279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72400b5510 0x7f72400b3960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f72400b4aa0 tx=0x7f723001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f7246000640 1 -- 172.21.15.16:0/1984618279 shutdown_connections 2024-09-10T21:10:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f7246000640 1 --2- 172.21.15.16:0/1984618279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72400b5510 0x7f72400b3960 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f7246000640 1 --2- 172.21.15.16:0/1984618279 >> v2:172.21.15.16:3300/0 conn(0x7f7240161810 0x7f72400d2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f7246000640 1 --2- 172.21.15.16:0/1984618279 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7240161430 0x7f72400d1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f7246000640 1 -- 172.21.15.16:0/1984618279 >> 172.21.15.16:0/1984618279 conn(0x7f72400d4000 msgr2=0x7f72400d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f7246000640 1 -- 172.21.15.16:0/1984618279 shutdown_connections 2024-09-10T21:10:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fdb82a00640 1 -- 172.21.15.16:0/1151149139 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdb84020030 con 0x7fdb8c1437c0 2024-09-10T21:10:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f403ca00640 1 -- 172.21.15.16:0/2718756054 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f403815e200 msgr2=0x7f40380ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f403ca00640 1 --2- 172.21.15.16:0/2718756054 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f403815e200 0x7f40380ceb80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4028009840 tx=0x7f402801c830 comp rx=0 tx=0).stop 2024-09-10T21:10:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7f014d600640 1 -- 172.21.15.16:0/1881533557 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144016020 con 0x7f01500d08f0 2024-09-10T21:10:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7f014d600640 1 -- 172.21.15.16:0/1881533557 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f014400ab40 con 0x7f01500d08f0 2024-09-10T21:10:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7f014d600640 1 -- 172.21.15.16:0/1881533557 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144012420 con 0x7f01500d08f0 2024-09-10T21:10:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7f0158600640 1 -- 172.21.15.16:0/1881533557 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0150158520 con 0x7f01500d08f0 2024-09-10T21:10:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.474+0000 7f0158600640 1 -- 172.21.15.16:0/1881533557 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0150157040 con 0x7f01500d08f0 2024-09-10T21:10:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1faa00640 1 -- 172.21.15.16:0/691556675 learned_addr learned my addr 172.21.15.16:0/691556675 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1fb400640 1 --2- 172.21.15.16:0/691556675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1f4036700 0x7fd1f4183b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1faa00640 1 -- 172.21.15.16:0/691556675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1f4036700 msgr2=0x7fd1f4183b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1faa00640 1 --2- 172.21.15.16:0/691556675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1f4036700 0x7fd1f4183b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1faa00640 1 -- 172.21.15.16:0/691556675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f401c370 msgr2=0x7fd1f4180d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1faa00640 1 --2- 172.21.15.16:0/691556675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f401c370 0x7fd1f4180d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1faa00640 1 -- 172.21.15.16:0/691556675 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f41840b0 con 0x7fd1f400ba10 2024-09-10T21:10:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fca79400640 1 -- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0332b0 msgr2=0x7fca6c0077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fca79400640 1 --2- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0332b0 0x7fca6c0077b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fca6c010bd0 tx=0x7fca6001c820 comp rx=0 tx=0).stop 2024-09-10T21:10:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1faa00640 1 --2- 172.21.15.16:0/691556675 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd1f400ba10 0x7fd1f40de280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c430 tx=0x7fd1ec00c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f0142a00640 1 -- 172.21.15.16:0/1881533557 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01500d1c60 con 0x7f01500d08f0 2024-09-10T21:10:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f7246000640 1 -- 172.21.15.16:0/1984618279 wait complete. 2024-09-10T21:10:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1f2a00640 1 -- 172.21.15.16:0/691556675 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec006810 con 0x7fd1f400ba10 2024-09-10T21:10:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f403ca00640 1 -- 172.21.15.16:0/2718756054 shutdown_connections 2024-09-10T21:10:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1f2a00640 1 -- 172.21.15.16:0/691556675 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1ec0069b0 con 0x7fd1f400ba10 2024-09-10T21:10:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1fbe00640 1 -- 172.21.15.16:0/691556675 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f401e9a0 con 0x7fd1f400ba10 2024-09-10T21:10:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f403ca00640 1 --2- 172.21.15.16:0/2718756054 >> v2:172.21.15.16:3300/0 conn(0x7f40380cf7d0 0x7f40380d2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1f2a00640 1 -- 172.21.15.16:0/691556675 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec006de0 con 0x7fd1f400ba10 2024-09-10T21:10:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7fd1fbe00640 1 -- 172.21.15.16:0/691556675 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f401ef60 con 0x7fd1f400ba10 2024-09-10T21:10:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f7246000640 1 Processor -- start 2024-09-10T21:10:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f403ca00640 1 --2- 172.21.15.16:0/2718756054 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f403815e200 0x7f40380ceb80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f403ca00640 1 -- 172.21.15.16:0/2718756054 >> 172.21.15.16:0/2718756054 conn(0x7f40380d3f70 msgr2=0x7f40380d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f403ca00640 1 -- 172.21.15.16:0/2718756054 shutdown_connections 2024-09-10T21:10:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.478+0000 7f403ca00640 1 -- 172.21.15.16:0/2718756054 wait complete. 2024-09-10T21:10:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.482+0000 7f014d600640 1 -- 172.21.15.16:0/1881533557 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f014401b400 con 0x7f01500d08f0 2024-09-10T21:10:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.482+0000 7fd1fbe00640 1 -- 172.21.15.16:0/691556675 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f4035ff0 con 0x7fd1f400ba10 2024-09-10T21:10:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.482+0000 7f014d600640 1 --2- 172.21.15.16:0/1881533557 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f01300725e0 0x7f0130074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.482+0000 7f014d600640 1 -- 172.21.15.16:0/1881533557 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0144093850 con 0x7f01500d08f0 2024-09-10T21:10:41.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.482+0000 7f0155200640 1 --2- 172.21.15.16:0/1881533557 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f01300725e0 0x7f0130074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.482+0000 7f403ca00640 1 Processor -- start 2024-09-10T21:10:41.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.482+0000 7f0155200640 1 --2- 172.21.15.16:0/1881533557 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f01300725e0 0x7f0130074aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f013c0af000 tx=0x7f013c0195c0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.482+0000 7fd1f2a00640 1 -- 172.21.15.16:0/691556675 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd1ec0379f0 con 0x7fd1f400ba10 2024-09-10T21:10:41.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.482+0000 7f7246000640 1 -- start start 2024-09-10T21:10:41.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.482+0000 7f7246000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72400b5510 0x7f72401513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f7246000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7240161430 0x7f724015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f7246000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7240161810 0x7f724015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f403ca00640 1 -- start start 2024-09-10T21:10:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7fca79400640 1 -- 172.21.15.16:0/850546280 shutdown_connections 2024-09-10T21:10:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7fca79400640 1 --2- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0332b0 0x7fca6c0077b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7fca79400640 1 --2- 172.21.15.16:0/850546280 >> v2:172.21.15.52:3300/0 conn(0x7fca6c0e69b0 0x7fca6c0e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7fca79400640 1 -- 172.21.15.16:0/850546280 >> 172.21.15.16:0/850546280 conn(0x7fca6c056720 msgr2=0x7fca6c056b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7fca79400640 1 -- 172.21.15.16:0/850546280 shutdown_connections 2024-09-10T21:10:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7fca79400640 1 -- 172.21.15.16:0/850546280 wait complete. 2024-09-10T21:10:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f723f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72400b5510 0x7f72401513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f723f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72400b5510 0x7f72401513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:53090/0 (socket says 172.21.15.16:53090) 2024-09-10T21:10:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f723ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7240161430 0x7f724015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f723f400640 1 -- 172.21.15.16:0/3921925339 learned_addr learned my addr 172.21.15.16:0/3921925339 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f7246000640 1 -- 172.21.15.16:0/3921925339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72400b2200 con 0x7f7240161430 2024-09-10T21:10:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7fd1f2a00640 1 --2- 172.21.15.16:0/691556675 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd1dc072630 0x7fd1dc074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7fd1f2a00640 1 -- 172.21.15.16:0/691556675 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd1ec092f90 con 0x7fd1f400ba10 2024-09-10T21:10:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f7246000640 1 -- 172.21.15.16:0/3921925339 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f72400b2080 con 0x7f72400b5510 2024-09-10T21:10:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f7246000640 1 -- 172.21.15.16:0/3921925339 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f72400b2380 con 0x7f7240161810 2024-09-10T21:10:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f723fe00640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7240161810 0x7f724015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f723fe00640 1 -- 172.21.15.16:0/3921925339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72400b5510 msgr2=0x7f72401513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f403ca00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40380cf7d0 0x7f40380ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f403ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40380d4b00 0x7f40380ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f403ca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f403815e200 0x7f40380cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f403ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f403816afb0 con 0x7f40380d4b00 2024-09-10T21:10:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f403ca00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f403816ae30 con 0x7f403815e200 2024-09-10T21:10:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f403ca00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f403816b130 con 0x7f40380cf7d0 2024-09-10T21:10:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7fd1fa000640 1 --2- 172.21.15.16:0/691556675 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd1dc072630 0x7fd1dc074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f723fe00640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72400b5510 0x7f72401513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f723fe00640 1 -- 172.21.15.16:0/3921925339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7240161430 msgr2=0x7f724015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f723fe00640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7240161430 0x7f724015bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f723fe00640 1 -- 172.21.15.16:0/3921925339 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72400ce130 con 0x7f7240161810 2024-09-10T21:10:41.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f723fe00640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7240161810 0x7f724015c050 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f72400d2f80 tx=0x7f7230005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f4035e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40380cf7d0 0x7f40380ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f4036800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f403815e200 0x7f40380cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f4036800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f403815e200 0x7f40380cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:53102/0 (socket says 172.21.15.16:53102) 2024-09-10T21:10:41.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f4036800640 1 -- 172.21.15.16:0/3428058078 learned_addr learned my addr 172.21.15.16:0/3428058078 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f8820a00640 1 -- 172.21.15.16:0/3866613265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f881c0ca880 msgr2=0x7f881c15aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f8820a00640 1 --2- 172.21.15.16:0/3866613265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f881c0ca880 0x7f881c15aef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f881c0af550 tx=0x7f880801c800 comp rx=0 tx=0).stop 2024-09-10T21:10:41.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f014d600640 1 -- 172.21.15.16:0/1881533557 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0144020020 con 0x7f01500d08f0 2024-09-10T21:10:41.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7fd1fa000640 1 --2- 172.21.15.16:0/691556675 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd1dc072630 0x7fd1dc074af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80058d0 tx=0x7fd1e8005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f723ea00640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7240161430 0x7f724015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:41.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f4035400640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40380d4b00 0x7f40380ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f8820a00640 1 -- 172.21.15.16:0/3866613265 shutdown_connections 2024-09-10T21:10:41.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f8820a00640 1 --2- 172.21.15.16:0/3866613265 >> v2:172.21.15.16:3300/0 conn(0x7f881c0cd6f0 0x7f881c072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f8820a00640 1 --2- 172.21.15.16:0/3866613265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f881c0ca880 0x7f881c15aef0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f8820a00640 1 -- 172.21.15.16:0/3866613265 >> 172.21.15.16:0/3866613265 conn(0x7f881c0cfb20 msgr2=0x7f881c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f4035400640 1 -- 172.21.15.16:0/3428058078 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40380cf7d0 msgr2=0x7f40380ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f4035400640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40380cf7d0 0x7f40380ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f4035400640 1 -- 172.21.15.16:0/3428058078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f403815e200 msgr2=0x7f40380cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f4035400640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f403815e200 0x7f40380cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f4035400640 1 -- 172.21.15.16:0/3428058078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40380cbb90 con 0x7f40380d4b00 2024-09-10T21:10:41.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7f4036800640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f403815e200 0x7f40380cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:41.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.486+0000 7fca79400640 1 Processor -- start 2024-09-10T21:10:41.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f8820a00640 1 -- 172.21.15.16:0/3866613265 shutdown_connections 2024-09-10T21:10:41.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f4035400640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40380d4b00 0x7f40380ca0e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4024007fd0 tx=0x7f402400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f402e000640 1 -- 172.21.15.16:0/3428058078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024011790 con 0x7f40380d4b00 2024-09-10T21:10:41.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f402e000640 1 -- 172.21.15.16:0/3428058078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4024017070 con 0x7f40380d4b00 2024-09-10T21:10:41.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f402e000640 1 -- 172.21.15.16:0/3428058078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024011ce0 con 0x7f40380d4b00 2024-09-10T21:10:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f403ca00640 1 -- 172.21.15.16:0/3428058078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40380cbe70 con 0x7f40380d4b00 2024-09-10T21:10:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f403ca00640 1 -- 172.21.15.16:0/3428058078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40380cc710 con 0x7f40380d4b00 2024-09-10T21:10:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f8820a00640 1 -- 172.21.15.16:0/3866613265 wait complete. 2024-09-10T21:10:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7fca79400640 1 -- start start 2024-09-10T21:10:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7fca79400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca6c0332b0 0x7fca6c0d9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7fca79400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fca6c0e69b0 0x7fca6c0338c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f403ca00640 1 -- 172.21.15.16:0/3428058078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40380cc910 con 0x7f40380d4b00 2024-09-10T21:10:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7fca73e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca6c0332b0 0x7fca6c0d9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7fca73e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca6c0332b0 0x7fca6c0d9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:53106/0 (socket says 172.21.15.16:53106) 2024-09-10T21:10:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7fca73e00640 1 -- 172.21.15.16:0/778919038 learned_addr learned my addr 172.21.15.16:0/778919038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7fca79400640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0e72d0 0x7fca6c033e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f7237400640 1 -- 172.21.15.16:0/3921925339 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72300bc070 con 0x7f7240161810 2024-09-10T21:10:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f7237400640 1 -- 172.21.15.16:0/3921925339 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7230004490 con 0x7f7240161810 2024-09-10T21:10:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f7237400640 1 -- 172.21.15.16:0/3921925339 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72300b2040 con 0x7f7240161810 2024-09-10T21:10:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f7246000640 1 -- 172.21.15.16:0/3921925339 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7240151d50 con 0x7f7240161810 2024-09-10T21:10:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7fd1f2a00640 1 -- 172.21.15.16:0/691556675 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd1f4005c50 con 0x7fd1f400ba10 2024-09-10T21:10:41.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7f7246000640 1 -- 172.21.15.16:0/3921925339 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f72400caa50 con 0x7f7240161810 2024-09-10T21:10:41.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.490+0000 7fca73400640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fca6c0e69b0 0x7fca6c0338c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca78a00640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0e72d0 0x7fca6c033e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca79400640 1 -- 172.21.15.16:0/778919038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c0370b0 con 0x7fca6c0e72d0 2024-09-10T21:10:41.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca79400640 1 -- 172.21.15.16:0/778919038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c036f30 con 0x7fca6c0332b0 2024-09-10T21:10:41.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca79400640 1 -- 172.21.15.16:0/778919038 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fca6c037230 con 0x7fca6c0e69b0 2024-09-10T21:10:41.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca73e00640 1 -- 172.21.15.16:0/778919038 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fca6c0e69b0 msgr2=0x7fca6c0338c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca73e00640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fca6c0e69b0 0x7fca6c0338c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca73e00640 1 -- 172.21.15.16:0/778919038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0e72d0 msgr2=0x7fca6c033e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7f8820a00640 1 Processor -- start 2024-09-10T21:10:41.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7f7246000640 1 -- 172.21.15.16:0/3921925339 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72400b3960 con 0x7f7240161810 2024-09-10T21:10:41.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca73e00640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0e72d0 0x7fca6c033e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca73e00640 1 -- 172.21.15.16:0/778919038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c0dbc70 con 0x7fca6c0332b0 2024-09-10T21:10:41.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7f8820a00640 1 -- start start 2024-09-10T21:10:41.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7f402e000640 1 -- 172.21.15.16:0/3428058078 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f402401c460 con 0x7f40380d4b00 2024-09-10T21:10:41.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7f402e000640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40180725e0 0x7f4018074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7f402e000640 1 -- 172.21.15.16:0/3428058078 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4024092fc0 con 0x7f40380d4b00 2024-09-10T21:10:41.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca73400640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fca6c0e69b0 0x7fca6c0338c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:41.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca78a00640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0e72d0 0x7fca6c033e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:41.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7f7237400640 1 -- 172.21.15.16:0/3921925339 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f72300b5070 con 0x7f7240161810 2024-09-10T21:10:41.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7f4035e00640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40180725e0 0x7f4018074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca73e00640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca6c0332b0 0x7fca6c0d9b70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fca6800b4a0 tx=0x7fca6800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca67e00640 1 -- 172.21.15.16:0/778919038 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68014070 con 0x7fca6c0332b0 2024-09-10T21:10:41.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca79400640 1 -- 172.21.15.16:0/778919038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c0db0b0 con 0x7fca6c0332b0 2024-09-10T21:10:41.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca79400640 1 -- 172.21.15.16:0/778919038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c0db8d0 con 0x7fca6c0332b0 2024-09-10T21:10:41.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca67e00640 1 -- 172.21.15.16:0/778919038 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca68010610 con 0x7fca6c0332b0 2024-09-10T21:10:41.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7fca67e00640 1 -- 172.21.15.16:0/778919038 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68010a40 con 0x7fca6c0332b0 2024-09-10T21:10:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.494+0000 7f8820a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f881c0ca880 0x7f881c14a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8820a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f881c0cd6f0 0x7f881c1f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8820a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f881c157be0 0x7f881c14de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8820a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f881c070920 con 0x7f881c157be0 2024-09-10T21:10:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8820a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f881c0707a0 con 0x7f881c0cd6f0 2024-09-10T21:10:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8820a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f881c167460 con 0x7f881c0ca880 2024-09-10T21:10:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8819e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f881c0ca880 0x7f881c14a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8819e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f881c0ca880 0x7f881c14a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:43450/0 (socket says 172.21.15.16:43450) 2024-09-10T21:10:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8819e00640 1 -- 172.21.15.16:0/3515642108 learned_addr learned my addr 172.21.15.16:0/3515642108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:41.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8819e00640 1 -- 172.21.15.16:0/3515642108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f881c0cd6f0 msgr2=0x7f881c1f6ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:10:41.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f4035e00640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40180725e0 0x7f4018074aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f40280af000 tx=0x7f402801b070 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f7237400640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7220072630 0x7f7220074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f881a800640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f881c157be0 0x7f881c14de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8819e00640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f881c0cd6f0 0x7f881c1f6ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8819e00640 1 -- 172.21.15.16:0/3515642108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f881c157be0 msgr2=0x7f881c14de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8819e00640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f881c157be0 0x7f881c14de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8819e00640 1 -- 172.21.15.16:0/3515642108 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c1f8390 con 0x7f881c0ca880 2024-09-10T21:10:41.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f881a800640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f881c157be0 0x7f881c14de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:41.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f723f400640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7220072630 0x7f7220074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f8819e00640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f881c0ca880 0x7f881c14a510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f880c00c400 tx=0x7f880c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f7237400640 1 -- 172.21.15.16:0/3921925339 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f723012c8a0 con 0x7f7240161810 2024-09-10T21:10:41.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.498+0000 7f723f400640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7220072630 0x7f7220074af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f72400b9c70 tx=0x7f722c009210 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7f8812000640 1 -- 172.21.15.16:0/3515642108 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c011790 con 0x7f881c0ca880 2024-09-10T21:10:41.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7f8812000640 1 -- 172.21.15.16:0/3515642108 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f880c017070 con 0x7f881c0ca880 2024-09-10T21:10:41.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7f8812000640 1 -- 172.21.15.16:0/3515642108 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c011cc0 con 0x7f881c0ca880 2024-09-10T21:10:41.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7f8820a00640 1 -- 172.21.15.16:0/3515642108 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881c14aa50 con 0x7f881c0ca880 2024-09-10T21:10:41.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7f7237400640 1 -- 172.21.15.16:0/3921925339 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7230136080 con 0x7f7240161810 2024-09-10T21:10:41.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7f402e000640 1 -- 172.21.15.16:0/3428058078 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f40380cc910 con 0x7f40380d4b00 2024-09-10T21:10:41.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7fca79400640 1 -- 172.21.15.16:0/778919038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca500052a0 con 0x7fca6c0332b0 2024-09-10T21:10:41.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7fca67e00640 1 -- 172.21.15.16:0/778919038 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fca68010be0 con 0x7fca6c0332b0 2024-09-10T21:10:41.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7fca67e00640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fca540726b0 0x7fca54074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7fca67e00640 1 -- 172.21.15.16:0/778919038 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fca68092cb0 con 0x7fca6c0332b0 2024-09-10T21:10:41.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7f8820a00640 1 -- 172.21.15.16:0/3515642108 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f881c1f7af0 con 0x7f881c0ca880 2024-09-10T21:10:41.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7f8820a00640 1 -- 172.21.15.16:0/3515642108 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881c06a430 con 0x7f881c0ca880 2024-09-10T21:10:41.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7f8812000640 1 -- 172.21.15.16:0/3515642108 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f880c01c460 con 0x7f881c0ca880 2024-09-10T21:10:41.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.502+0000 7fca73400640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fca540726b0 0x7fca54074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.506+0000 7fca73400640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fca540726b0 0x7fca54074b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fca600af000 tx=0x7fca600a7040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.506+0000 7f8812000640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f87fc0725e0 0x7f87fc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:41.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.506+0000 7f8812000640 1 -- 172.21.15.16:0/3515642108 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f880c0927f0 con 0x7f881c0ca880 2024-09-10T21:10:41.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.506+0000 7f8819400640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f87fc0725e0 0x7f87fc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:41.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.510+0000 7f8819400640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f87fc0725e0 0x7f87fc074aa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88080af000 tx=0x7f880801b040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:41.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.510+0000 7fca67e00640 1 -- 172.21.15.16:0/778919038 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fca6805ddb0 con 0x7fca6c0332b0 2024-09-10T21:10:41.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.510+0000 7f8812000640 1 -- 172.21.15.16:0/3515642108 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f880c095030 con 0x7f881c0ca880 2024-09-10T21:10:41.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.842+0000 7fe1dfa00640 1 -- 172.21.15.16:0/3213307328 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe1b0005590 con 0x7fe1d80cc2b0 2024-09-10T21:10:41.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.842+0000 7fe1d4c00640 1 -- 172.21.15.16:0/3213307328 <== mon.2 v2:172.21.15.52:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe1c80630d0 con 0x7fe1d80cc2b0 2024-09-10T21:10:41.849 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-10T21:10:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.846+0000 7fe1dfa00640 1 -- 172.21.15.16:0/3213307328 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe1b80726b0 msgr2=0x7fe1b8074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.846+0000 7fe1dfa00640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe1b80726b0 0x7fe1b8074b70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00012f0 tx=0x7fe1d0009040 comp rx=0 tx=0).stop 2024-09-10T21:10:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.846+0000 7fe1dfa00640 1 -- 172.21.15.16:0/3213307328 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1d80cc2b0 msgr2=0x7fe1d8142390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.846+0000 7fe1dfa00640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1d80cc2b0 0x7fe1d8142390 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800d980 tx=0x7fe1c800de40 comp rx=0 tx=0).stop 2024-09-10T21:10:41.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.846+0000 7fe1dfa00640 1 -- 172.21.15.16:0/3213307328 shutdown_connections 2024-09-10T21:10:41.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.846+0000 7fe1dfa00640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe1b80726b0 0x7fe1b8074b70 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00012f0 tx=0x7fe1d0009040 comp rx=0 tx=0).stop 2024-09-10T21:10:41.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.846+0000 7fe1dfa00640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d80cc690 0x7fe1d8157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.846+0000 7fe1dfa00640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1d80cc2b0 0x7fe1d8142390 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.846+0000 7fe1dfa00640 1 --2- 172.21.15.16:0/3213307328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1d80b4550 0x7fe1d815b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.846+0000 7fe1dfa00640 1 -- 172.21.15.16:0/3213307328 >> 172.21.15.16:0/3213307328 conn(0x7fe1d806cbe0 msgr2=0x7fe1d815ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.846+0000 7fe1dfa00640 1 -- 172.21.15.16:0/3213307328 shutdown_connections 2024-09-10T21:10:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.850+0000 7fe1dfa00640 1 -- 172.21.15.16:0/3213307328 wait complete. 2024-09-10T21:10:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.850+0000 7fdb8be00640 1 -- 172.21.15.16:0/1151149139 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdb64005590 con 0x7fdb8c1437c0 2024-09-10T21:10:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.854+0000 7fdb82a00640 1 -- 172.21.15.16:0/1151149139 <== mon.2 v2:172.21.15.52:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdb8405e950 con 0x7fdb8c1437c0 2024-09-10T21:10:41.861 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-09-10T21:10:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.858+0000 7f0142a00640 1 -- 172.21.15.16:0/1881533557 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f015016c1a0 con 0x7f01500d08f0 2024-09-10T21:10:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.858+0000 7f014d600640 1 -- 172.21.15.16:0/1881533557 <== mon.2 v2:172.21.15.52:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f014405efc0 con 0x7f01500d08f0 2024-09-10T21:10:41.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.858+0000 7fdb8be00640 1 -- 172.21.15.16:0/1151149139 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdb6c0726e0 msgr2=0x7fdb6c074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.858+0000 7fdb8be00640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdb6c0726e0 0x7fdb6c074ba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009870 tx=0x7fdb7c01d890 comp rx=0 tx=0).stop 2024-09-10T21:10:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.858+0000 7fdb8be00640 1 -- 172.21.15.16:0/1151149139 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb8c1437c0 msgr2=0x7fdb8c1f7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.858+0000 7fdb8be00640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb8c1437c0 0x7fdb8c1f7370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400caa0 tx=0x7fdb8400cb90 comp rx=0 tx=0).stop 2024-09-10T21:10:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7fdb8be00640 1 -- 172.21.15.16:0/1151149139 shutdown_connections 2024-09-10T21:10:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7fdb8be00640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdb6c0726e0 0x7fdb6c074ba0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7fdb8be00640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb8c1437c0 0x7fdb8c1f7370 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7fdb8be00640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c072540 0x7fdb8c0b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7fdb8be00640 1 --2- 172.21.15.16:0/1151149139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb8c0705f0 0x7fdb8c131b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7fdb8be00640 1 -- 172.21.15.16:0/1151149139 >> 172.21.15.16:0/1151149139 conn(0x7fdb8c0b1710 msgr2=0x7fdb8c0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.866 INFO:teuthology.orchestra.run.smithi016.stdout:60129542150 2024-09-10T21:10:41.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7fd1fbe00640 1 -- 172.21.15.16:0/691556675 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd1f4053630 con 0x7fd1f400ba10 2024-09-10T21:10:41.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7f0142a00640 1 -- 172.21.15.16:0/1881533557 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f01300725e0 msgr2=0x7f0130074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7f0142a00640 1 --2- 172.21.15.16:0/1881533557 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f01300725e0 0x7f0130074aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f013c0af000 tx=0x7f013c0195c0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7f0142a00640 1 -- 172.21.15.16:0/1881533557 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01500d08f0 msgr2=0x7f01500cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7f0142a00640 1 --2- 172.21.15.16:0/1881533557 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01500d08f0 0x7f01500cd8d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f014400cba0 tx=0x7f014400cc90 comp rx=0 tx=0).stop 2024-09-10T21:10:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7fdb8be00640 1 -- 172.21.15.16:0/1151149139 shutdown_connections 2024-09-10T21:10:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7fd1f2a00640 1 -- 172.21.15.16:0/691556675 <== mon.2 v2:172.21.15.52:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd1ec05e8f0 con 0x7fd1f400ba10 2024-09-10T21:10:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7fdb8be00640 1 -- 172.21.15.16:0/1151149139 wait complete. 2024-09-10T21:10:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7f0142a00640 1 -- 172.21.15.16:0/1881533557 shutdown_connections 2024-09-10T21:10:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7f0142a00640 1 --2- 172.21.15.16:0/1881533557 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f01300725e0 0x7f0130074aa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7f0142a00640 1 --2- 172.21.15.16:0/1881533557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0150157d40 0x7f01500b6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7f0142a00640 1 --2- 172.21.15.16:0/1881533557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015015e050 0x7f0150156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7f0142a00640 1 --2- 172.21.15.16:0/1881533557 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01500d08f0 0x7f01500cd8d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7f0142a00640 1 -- 172.21.15.16:0/1881533557 >> 172.21.15.16:0/1881533557 conn(0x7f015009d150 msgr2=0x7f015016c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.869 INFO:teuthology.orchestra.run.smithi016.stdout:64424509446 2024-09-10T21:10:41.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7f0142a00640 1 -- 172.21.15.16:0/1881533557 shutdown_connections 2024-09-10T21:10:41.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7f0142a00640 1 -- 172.21.15.16:0/1881533557 wait complete. 2024-09-10T21:10:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.862+0000 7fd1fbe00640 1 -- 172.21.15.16:0/691556675 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd1dc072630 msgr2=0x7fd1dc074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.866+0000 7fd1fbe00640 1 --2- 172.21.15.16:0/691556675 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd1dc072630 0x7fd1dc074af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80058d0 tx=0x7fd1e8005860 comp rx=0 tx=0).stop 2024-09-10T21:10:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.866+0000 7fd1fbe00640 1 -- 172.21.15.16:0/691556675 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd1f400ba10 msgr2=0x7fd1f40de280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.866+0000 7fd1fbe00640 1 --2- 172.21.15.16:0/691556675 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd1f400ba10 0x7fd1f40de280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c430 tx=0x7fd1ec00c8f0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.866+0000 7fd1fbe00640 1 -- 172.21.15.16:0/691556675 shutdown_connections 2024-09-10T21:10:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.866+0000 7fd1fbe00640 1 --2- 172.21.15.16:0/691556675 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd1dc072630 0x7fd1dc074af0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.866+0000 7fd1fbe00640 1 --2- 172.21.15.16:0/691556675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1f4036700 0x7fd1f4183b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.866+0000 7fd1fbe00640 1 --2- 172.21.15.16:0/691556675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f401c370 0x7fd1f4180d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.866+0000 7fd1fbe00640 1 --2- 172.21.15.16:0/691556675 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd1f400ba10 0x7fd1f40de280 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.866+0000 7fd1fbe00640 1 -- 172.21.15.16:0/691556675 >> 172.21.15.16:0/691556675 conn(0x7fd1f4054570 msgr2=0x7fd1f4033ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.866+0000 7fd1fbe00640 1 -- 172.21.15.16:0/691556675 shutdown_connections 2024-09-10T21:10:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.866+0000 7fd1fbe00640 1 -- 172.21.15.16:0/691556675 wait complete. 2024-09-10T21:10:41.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.870+0000 7f403ca00640 1 -- 172.21.15.16:0/3428058078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f40380b9bb0 con 0x7f40380d4b00 2024-09-10T21:10:41.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.870+0000 7f7246000640 1 -- 172.21.15.16:0/3921925339 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7240160990 con 0x7f7240161810 2024-09-10T21:10:41.877 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-09-10T21:10:41.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.870+0000 7f402e000640 1 -- 172.21.15.16:0/3428058078 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f402405e600 con 0x7f40380d4b00 2024-09-10T21:10:41.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.874+0000 7f7237400640 1 -- 172.21.15.16:0/3921925339 <== mon.2 v2:172.21.15.52:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f72300164d0 con 0x7f7240161810 2024-09-10T21:10:41.880 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-09-10T21:10:41.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.874+0000 7f403ca00640 1 -- 172.21.15.16:0/3428058078 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40180725e0 msgr2=0x7f4018074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.874+0000 7f403ca00640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40180725e0 0x7f4018074aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f40280af000 tx=0x7f402801b070 comp rx=0 tx=0).stop 2024-09-10T21:10:41.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.874+0000 7f403ca00640 1 -- 172.21.15.16:0/3428058078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40380d4b00 msgr2=0x7f40380ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.874+0000 7f403ca00640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40380d4b00 0x7f40380ca0e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4024007fd0 tx=0x7f402400c890 comp rx=0 tx=0).stop 2024-09-10T21:10:41.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.874+0000 7f403ca00640 1 -- 172.21.15.16:0/3428058078 shutdown_connections 2024-09-10T21:10:41.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.874+0000 7f403ca00640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40180725e0 0x7f4018074aa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.874+0000 7f403ca00640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f403815e200 0x7f40380cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.874+0000 7f403ca00640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40380d4b00 0x7f40380ca0e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.874+0000 7f403ca00640 1 --2- 172.21.15.16:0/3428058078 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40380cf7d0 0x7f40380ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.874+0000 7f403ca00640 1 -- 172.21.15.16:0/3428058078 >> 172.21.15.16:0/3428058078 conn(0x7f40380d3f70 msgr2=0x7f40380b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.874+0000 7f403ca00640 1 -- 172.21.15.16:0/3428058078 shutdown_connections 2024-09-10T21:10:41.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.874+0000 7f403ca00640 1 -- 172.21.15.16:0/3428058078 wait complete. 2024-09-10T21:10:41.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.878+0000 7f7246000640 1 -- 172.21.15.16:0/3921925339 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7220072630 msgr2=0x7f7220074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.878+0000 7f7246000640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7220072630 0x7f7220074af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f72400b9c70 tx=0x7f722c009210 comp rx=0 tx=0).stop 2024-09-10T21:10:41.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.878+0000 7f7246000640 1 -- 172.21.15.16:0/3921925339 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7240161810 msgr2=0x7f724015c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.878+0000 7f7246000640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7240161810 0x7f724015c050 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f72400d2f80 tx=0x7f7230005700 comp rx=0 tx=0).stop 2024-09-10T21:10:41.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.878+0000 7f7246000640 1 -- 172.21.15.16:0/3921925339 shutdown_connections 2024-09-10T21:10:41.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.878+0000 7f7246000640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7220072630 0x7f7220074af0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.878+0000 7f7246000640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7240161810 0x7f724015c050 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.878+0000 7f7246000640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7240161430 0x7f724015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.878+0000 7f7246000640 1 --2- 172.21.15.16:0/3921925339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72400b5510 0x7f72401513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.878+0000 7f7246000640 1 -- 172.21.15.16:0/3921925339 >> 172.21.15.16:0/3921925339 conn(0x7f72400d4000 msgr2=0x7f72400d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.878+0000 7f7246000640 1 -- 172.21.15.16:0/3921925339 shutdown_connections 2024-09-10T21:10:41.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.878+0000 7f7246000640 1 -- 172.21.15.16:0/3921925339 wait complete. 2024-09-10T21:10:41.893 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.2 2024-09-10T21:10:41.893 DEBUG:teuthology.parallel:result is None 2024-09-10T21:10:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.890+0000 7fca79400640 1 -- 172.21.15.16:0/778919038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fca50005590 con 0x7fca6c0332b0 2024-09-10T21:10:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.890+0000 7f8820a00640 1 -- 172.21.15.16:0/3515642108 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f881c15aef0 con 0x7f881c0ca880 2024-09-10T21:10:41.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.890+0000 7fca67e00640 1 -- 172.21.15.16:0/778919038 <== mon.1 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fca680627e0 con 0x7fca6c0332b0 2024-09-10T21:10:41.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.890+0000 7f8812000640 1 -- 172.21.15.16:0/3515642108 <== mon.2 v2:172.21.15.52:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f881c0cf7d0 con 0x7f881c0ca880 2024-09-10T21:10:41.895 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-10T21:10:41.896 INFO:teuthology.orchestra.run.smithi016.stdout:60129542151 2024-09-10T21:10:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7f8820a00640 1 -- 172.21.15.16:0/3515642108 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f87fc0725e0 msgr2=0x7f87fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7f8820a00640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f87fc0725e0 0x7f87fc074aa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88080af000 tx=0x7f880801b040 comp rx=0 tx=0).stop 2024-09-10T21:10:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7f8820a00640 1 -- 172.21.15.16:0/3515642108 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f881c0ca880 msgr2=0x7f881c14a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7f8820a00640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f881c0ca880 0x7f881c14a510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f880c00c400 tx=0x7f880c00c8c0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7fca79400640 1 -- 172.21.15.16:0/778919038 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fca540726b0 msgr2=0x7fca54074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7fca79400640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fca540726b0 0x7fca54074b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fca600af000 tx=0x7fca600a7040 comp rx=0 tx=0).stop 2024-09-10T21:10:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7fca79400640 1 -- 172.21.15.16:0/778919038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca6c0332b0 msgr2=0x7fca6c0d9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7fca79400640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca6c0332b0 0x7fca6c0d9b70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fca6800b4a0 tx=0x7fca6800b960 comp rx=0 tx=0).stop 2024-09-10T21:10:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7f8820a00640 1 -- 172.21.15.16:0/3515642108 shutdown_connections 2024-09-10T21:10:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7f8820a00640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f87fc0725e0 0x7f87fc074aa0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7f8820a00640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f881c157be0 0x7f881c14de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7f8820a00640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f881c0cd6f0 0x7f881c1f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7f8820a00640 1 --2- 172.21.15.16:0/3515642108 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f881c0ca880 0x7f881c14a510 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7f8820a00640 1 -- 172.21.15.16:0/3515642108 >> 172.21.15.16:0/3515642108 conn(0x7f881c0cfb20 msgr2=0x7f881c151890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7fca79400640 1 -- 172.21.15.16:0/778919038 shutdown_connections 2024-09-10T21:10:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7fca79400640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fca540726b0 0x7fca54074b70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7f8820a00640 1 -- 172.21.15.16:0/3515642108 shutdown_connections 2024-09-10T21:10:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7fca79400640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca6c0e72d0 0x7fca6c033e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7fca79400640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fca6c0e69b0 0x7fca6c0338c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7fca79400640 1 --2- 172.21.15.16:0/778919038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca6c0332b0 0x7fca6c0d9b70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:41.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7fca79400640 1 -- 172.21.15.16:0/778919038 >> 172.21.15.16:0/778919038 conn(0x7fca6c056720 msgr2=0x7fca6c0e4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:41.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7f8820a00640 1 -- 172.21.15.16:0/3515642108 wait complete. 2024-09-10T21:10:41.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7fca79400640 1 -- 172.21.15.16:0/778919038 shutdown_connections 2024-09-10T21:10:41.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:41.894+0000 7fca79400640 1 -- 172.21.15.16:0/778919038 wait complete. 2024-09-10T21:10:41.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2024-09-10T21:10:41.912 DEBUG:teuthology.parallel:result is None 2024-09-10T21:10:41.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-09-10T21:10:41.914 DEBUG:teuthology.parallel:result is None 2024-09-10T21:10:41.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2024-09-10T21:10:41.915 DEBUG:teuthology.parallel:result is None 2024-09-10T21:10:41.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-09-10T21:10:41.917 DEBUG:teuthology.parallel:result is None 2024-09-10T21:10:41.920 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-09-10T21:10:41.920 DEBUG:teuthology.parallel:result is None 2024-09-10T21:10:41.936 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.7 2024-09-10T21:10:41.937 DEBUG:teuthology.parallel:result is None 2024-09-10T21:10:41.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.5 2024-09-10T21:10:41.938 DEBUG:teuthology.parallel:result is None 2024-09-10T21:10:41.938 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:10:41.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:10:42.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 -- 172.21.15.16:0/1225507913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63600d2820 msgr2=0x7f63600b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 --2- 172.21.15.16:0/1225507913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63600d2820 0x7f63600b1600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6348009900 tx=0x7f634801c850 comp rx=0 tx=0).stop 2024-09-10T21:10:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 -- 172.21.15.16:0/1225507913 shutdown_connections 2024-09-10T21:10:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 --2- 172.21.15.16:0/1225507913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63600d2820 0x7f63600b1600 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 --2- 172.21.15.16:0/1225507913 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f63600b1b40 0x7f63600b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 -- 172.21.15.16:0/1225507913 >> 172.21.15.16:0/1225507913 conn(0x7f63600d1d00 msgr2=0x7f63600d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 -- 172.21.15.16:0/1225507913 shutdown_connections 2024-09-10T21:10:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 -- 172.21.15.16:0/1225507913 wait complete. 2024-09-10T21:10:42.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 Processor -- start 2024-09-10T21:10:42.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 -- start start 2024-09-10T21:10:42.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63600b1b40 0x7f636015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:42.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f63600cfd40 0x7f63600cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:42.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63600d2820 0x7f63600c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:42.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63600b0620 con 0x7f63600b1b40 2024-09-10T21:10:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f63600b04a0 con 0x7f63600d2820 2024-09-10T21:10:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635fe00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f63600b07a0 con 0x7f63600cfd40 2024-09-10T21:10:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63600b1b40 0x7f636015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63600b1b40 0x7f636015d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36314/0 (socket says 172.21.15.16:36314) 2024-09-10T21:10:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.122+0000 7f635ea00640 1 -- 172.21.15.16:0/403966987 learned_addr learned my addr 172.21.15.16:0/403966987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f6356000640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f63600cfd40 0x7f63600cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f635f400640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63600d2820 0x7f63600c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f635ea00640 1 -- 172.21.15.16:0/403966987 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f63600cfd40 msgr2=0x7f63600cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f635ea00640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f63600cfd40 0x7f63600cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f635ea00640 1 -- 172.21.15.16:0/403966987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63600d2820 msgr2=0x7f63600c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f635ea00640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63600d2820 0x7f63600c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f635ea00640 1 -- 172.21.15.16:0/403966987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63600c8f00 con 0x7f63600b1b40 2024-09-10T21:10:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f635ea00640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63600b1b40 0x7f636015d8b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f635000c500 tx=0x7f635000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f6357e00640 1 -- 172.21.15.16:0/403966987 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350017070 con 0x7f63600b1b40 2024-09-10T21:10:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f6357e00640 1 -- 172.21.15.16:0/403966987 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6350003600 con 0x7f63600b1b40 2024-09-10T21:10:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f635fe00640 1 -- 172.21.15.16:0/403966987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636015ddf0 con 0x7f63600b1b40 2024-09-10T21:10:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f6357e00640 1 -- 172.21.15.16:0/403966987 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350003a90 con 0x7f63600b1b40 2024-09-10T21:10:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f635fe00640 1 -- 172.21.15.16:0/403966987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63600c91a0 con 0x7f63600b1b40 2024-09-10T21:10:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f635fe00640 1 -- 172.21.15.16:0/403966987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63600c95d0 con 0x7f63600b1b40 2024-09-10T21:10:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.126+0000 7f6357e00640 1 -- 172.21.15.16:0/403966987 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6350003c30 con 0x7f63600b1b40 2024-09-10T21:10:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.130+0000 7f6357e00640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f633c0725e0 0x7f633c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.130+0000 7f6357e00640 1 -- 172.21.15.16:0/403966987 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f63500929c0 con 0x7f63600b1b40 2024-09-10T21:10:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.130+0000 7f6356000640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f633c0725e0 0x7f633c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:42.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.130+0000 7f6356000640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f633c0725e0 0x7f633c074aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f63480af000 tx=0x7f634801b090 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:42.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.134+0000 7f6357e00640 1 -- 172.21.15.16:0/403966987 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f635005db00 con 0x7f63600b1b40 2024-09-10T21:10:42.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.302+0000 7f635fe00640 1 -- 172.21.15.16:0/403966987 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63600b1fb0 con 0x7f633c0725e0 2024-09-10T21:10:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.306+0000 7f6357e00640 1 -- 172.21.15.16:0/403966987 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f63600b1fb0 con 0x7f633c0725e0 2024-09-10T21:10:42.314 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:42.315 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:10:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.310+0000 7f635fe00640 1 -- 172.21.15.16:0/403966987 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f633c0725e0 msgr2=0x7f633c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.310+0000 7f635fe00640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f633c0725e0 0x7f633c074aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f63480af000 tx=0x7f634801b090 comp rx=0 tx=0).stop 2024-09-10T21:10:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.310+0000 7f635fe00640 1 -- 172.21.15.16:0/403966987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63600b1b40 msgr2=0x7f636015d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.310+0000 7f635fe00640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63600b1b40 0x7f636015d8b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f635000c500 tx=0x7f635000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.310+0000 7f635fe00640 1 -- 172.21.15.16:0/403966987 shutdown_connections 2024-09-10T21:10:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.310+0000 7f635fe00640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f633c0725e0 0x7f633c074aa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.310+0000 7f635fe00640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63600d2820 0x7f63600c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.310+0000 7f635fe00640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f63600cfd40 0x7f63600cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.310+0000 7f635fe00640 1 --2- 172.21.15.16:0/403966987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63600b1b40 0x7f636015d8b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.310+0000 7f635fe00640 1 -- 172.21.15.16:0/403966987 >> 172.21.15.16:0/403966987 conn(0x7f63600d1d00 msgr2=0x7f63600d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.310+0000 7f635fe00640 1 -- 172.21.15.16:0/403966987 shutdown_connections 2024-09-10T21:10:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.310+0000 7f635fe00640 1 -- 172.21.15.16:0/403966987 wait complete. 2024-09-10T21:10:42.335 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-10T21:10:41.459095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252400,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722320,"statfs":{"total":773094113280,"available":772835655680,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.293839"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179857+0000","last_change":"2024-09-10T21:10:30.197837+0000","last_active":"2024-09-10T21:10:31.179857+0000","last_peered":"2024-09-10T21:10:31.179857+0000","last_clean":"2024-09-10T21:10:31.179857+0000","last_became_active":"2024-09-10T21:10:30.197476+0000","last_became_peered":"2024-09-10T21:10:30.197476+0000","last_unstale":"2024-09-10T21:10:31.179857+0000","last_undegraded":"2024-09-10T21:10:31.179857+0000","last_fullsized":"2024-09-10T21:10:31.179857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:31.264063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166547+0000","last_change":"2024-09-10T21:10:30.162505+0000","last_active":"2024-09-10T21:10:31.166547+0000","last_peered":"2024-09-10T21:10:31.166547+0000","last_clean":"2024-09-10T21:10:31.166547+0000","last_became_active":"2024-09-10T21:10:30.162220+0000","last_became_peered":"2024-09-10T21:10:30.162220+0000","last_unstale":"2024-09-10T21:10:31.166547+0000","last_undegraded":"2024-09-10T21:10:31.166547+0000","last_fullsized":"2024-09-10T21:10:31.166547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:28.836761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166183+0000","last_change":"2024-09-10T21:10:30.166240+0000","last_active":"2024-09-10T21:10:31.166183+0000","last_peered":"2024-09-10T21:10:31.166183+0000","last_clean":"2024-09-10T21:10:31.166183+0000","last_became_active":"2024-09-10T21:10:30.165827+0000","last_became_peered":"2024-09-10T21:10:30.165827+0000","last_unstale":"2024-09-10T21:10:31.166183+0000","last_undegraded":"2024-09-10T21:10:31.166183+0000","last_fullsized":"2024-09-10T21:10:31.166183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:31.342353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166786+0000","last_change":"2024-09-10T21:10:30.163365+0000","last_active":"2024-09-10T21:10:31.166786+0000","last_peered":"2024-09-10T21:10:31.166786+0000","last_clean":"2024-09-10T21:10:31.166786+0000","last_became_active":"2024-09-10T21:10:30.163034+0000","last_became_peered":"2024-09-10T21:10:30.163034+0000","last_unstale":"2024-09-10T21:10:31.166786+0000","last_undegraded":"2024-09-10T21:10:31.166786+0000","last_fullsized":"2024-09-10T21:10:31.166786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:36.337660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179172+0000","last_change":"2024-09-10T21:10:30.174443+0000","last_active":"2024-09-10T21:10:31.179172+0000","last_peered":"2024-09-10T21:10:31.179172+0000","last_clean":"2024-09-10T21:10:31.179172+0000","last_became_active":"2024-09-10T21:10:30.173975+0000","last_became_peered":"2024-09-10T21:10:30.173975+0000","last_unstale":"2024-09-10T21:10:31.179172+0000","last_undegraded":"2024-09-10T21:10:31.179172+0000","last_fullsized":"2024-09-10T21:10:31.179172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:37.732853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166683+0000","last_change":"2024-09-10T21:10:30.166122+0000","last_active":"2024-09-10T21:10:31.166683+0000","last_peered":"2024-09-10T21:10:31.166683+0000","last_clean":"2024-09-10T21:10:31.166683+0000","last_became_active":"2024-09-10T21:10:30.165627+0000","last_became_peered":"2024-09-10T21:10:30.165627+0000","last_unstale":"2024-09-10T21:10:31.166683+0000","last_undegraded":"2024-09-10T21:10:31.166683+0000","last_fullsized":"2024-09-10T21:10:31.166683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:39.070890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178907+0000","last_change":"2024-09-10T21:10:30.173891+0000","last_active":"2024-09-10T21:10:31.178907+0000","last_peered":"2024-09-10T21:10:31.178907+0000","last_clean":"2024-09-10T21:10:31.178907+0000","last_became_active":"2024-09-10T21:10:30.173513+0000","last_became_peered":"2024-09-10T21:10:30.173513+0000","last_unstale":"2024-09-10T21:10:31.178907+0000","last_undegraded":"2024-09-10T21:10:31.178907+0000","last_fullsized":"2024-09-10T21:10:31.178907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:49.396588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.165773+0000","last_change":"2024-09-10T21:10:30.173739+0000","last_active":"2024-09-10T21:10:31.165773+0000","last_peered":"2024-09-10T21:10:31.165773+0000","last_clean":"2024-09-10T21:10:31.165773+0000","last_became_active":"2024-09-10T21:10:30.173401+0000","last_became_peered":"2024-09-10T21:10:30.173401+0000","last_unstale":"2024-09-10T21:10:31.165773+0000","last_undegraded":"2024-09-10T21:10:31.165773+0000","last_fullsized":"2024-09-10T21:10:31.165773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:43.846568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168511+0000","last_change":"2024-09-10T21:10:30.180534+0000","last_active":"2024-09-10T21:10:31.168511+0000","last_peered":"2024-09-10T21:10:31.168511+0000","last_clean":"2024-09-10T21:10:31.168511+0000","last_became_active":"2024-09-10T21:10:30.180196+0000","last_became_peered":"2024-09-10T21:10:30.180196+0000","last_unstale":"2024-09-10T21:10:31.168511+0000","last_undegraded":"2024-09-10T21:10:31.168511+0000","last_fullsized":"2024-09-10T21:10:31.168511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:04.038591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179548+0000","last_change":"2024-09-10T21:10:30.191472+0000","last_active":"2024-09-10T21:10:31.179548+0000","last_peered":"2024-09-10T21:10:31.179548+0000","last_clean":"2024-09-10T21:10:31.179548+0000","last_became_active":"2024-09-10T21:10:30.182657+0000","last_became_peered":"2024-09-10T21:10:30.182657+0000","last_unstale":"2024-09-10T21:10:31.179548+0000","last_undegraded":"2024-09-10T21:10:31.179548+0000","last_fullsized":"2024-09-10T21:10:31.179548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:42.385122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179302+0000","last_change":"2024-09-10T21:10:30.196473+0000","last_active":"2024-09-10T21:10:31.179302+0000","last_peered":"2024-09-10T21:10:31.179302+0000","last_clean":"2024-09-10T21:10:31.179302+0000","last_became_active":"2024-09-10T21:10:30.195712+0000","last_became_peered":"2024-09-10T21:10:30.195712+0000","last_unstale":"2024-09-10T21:10:31.179302+0000","last_undegraded":"2024-09-10T21:10:31.179302+0000","last_fullsized":"2024-09-10T21:10:31.179302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:14.592947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179252+0000","last_change":"2024-09-10T21:10:30.187870+0000","last_active":"2024-09-10T21:10:31.179252+0000","last_peered":"2024-09-10T21:10:31.179252+0000","last_clean":"2024-09-10T21:10:31.179252+0000","last_became_active":"2024-09-10T21:10:30.186791+0000","last_became_peered":"2024-09-10T21:10:30.186791+0000","last_unstale":"2024-09-10T21:10:31.179252+0000","last_undegraded":"2024-09-10T21:10:31.179252+0000","last_fullsized":"2024-09-10T21:10:31.179252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:49.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707773+0000","last_change":"2024-09-10T21:10:30.174269+0000","last_active":"2024-09-10T21:10:31.707773+0000","last_peered":"2024-09-10T21:10:31.707773+0000","last_clean":"2024-09-10T21:10:31.707773+0000","last_became_active":"2024-09-10T21:10:30.173714+0000","last_became_peered":"2024-09-10T21:10:30.173714+0000","last_unstale":"2024-09-10T21:10:31.707773+0000","last_undegraded":"2024-09-10T21:10:31.707773+0000","last_fullsized":"2024-09-10T21:10:31.707773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:30.207193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707685+0000","last_change":"2024-09-10T21:10:30.172867+0000","last_active":"2024-09-10T21:10:31.707685+0000","last_peered":"2024-09-10T21:10:31.707685+0000","last_clean":"2024-09-10T21:10:31.707685+0000","last_became_active":"2024-09-10T21:10:30.172301+0000","last_became_peered":"2024-09-10T21:10:30.172301+0000","last_unstale":"2024-09-10T21:10:31.707685+0000","last_undegraded":"2024-09-10T21:10:31.707685+0000","last_fullsized":"2024-09-10T21:10:31.707685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:32.205639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.165740+0000","last_change":"2024-09-10T21:10:30.181460+0000","last_active":"2024-09-10T21:10:31.165740+0000","last_peered":"2024-09-10T21:10:31.165740+0000","last_clean":"2024-09-10T21:10:31.165740+0000","last_became_active":"2024-09-10T21:10:30.175816+0000","last_became_peered":"2024-09-10T21:10:30.175816+0000","last_unstale":"2024-09-10T21:10:31.165740+0000","last_undegraded":"2024-09-10T21:10:31.165740+0000","last_fullsized":"2024-09-10T21:10:31.165740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:26.624987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708880+0000","last_change":"2024-09-10T21:10:30.185809+0000","last_active":"2024-09-10T21:10:31.708880+0000","last_peered":"2024-09-10T21:10:31.708880+0000","last_clean":"2024-09-10T21:10:31.708880+0000","last_became_active":"2024-09-10T21:10:30.185485+0000","last_became_peered":"2024-09-10T21:10:30.185485+0000","last_unstale":"2024-09-10T21:10:31.708880+0000","last_undegraded":"2024-09-10T21:10:31.708880+0000","last_fullsized":"2024-09-10T21:10:31.708880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:16.358565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709494+0000","last_change":"2024-09-10T21:10:30.196400+0000","last_active":"2024-09-10T21:10:31.709494+0000","last_peered":"2024-09-10T21:10:31.709494+0000","last_clean":"2024-09-10T21:10:31.709494+0000","last_became_active":"2024-09-10T21:10:30.194199+0000","last_became_peered":"2024-09-10T21:10:30.194199+0000","last_unstale":"2024-09-10T21:10:31.709494+0000","last_undegraded":"2024-09-10T21:10:31.709494+0000","last_fullsized":"2024-09-10T21:10:31.709494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:21.510129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446465+0000","last_change":"2024-09-10T21:10:30.169268+0000","last_active":"2024-09-10T21:10:31.446465+0000","last_peered":"2024-09-10T21:10:31.446465+0000","last_clean":"2024-09-10T21:10:31.446465+0000","last_became_active":"2024-09-10T21:10:30.168869+0000","last_became_peered":"2024-09-10T21:10:30.168869+0000","last_unstale":"2024-09-10T21:10:31.446465+0000","last_undegraded":"2024-09-10T21:10:31.446465+0000","last_fullsized":"2024-09-10T21:10:31.446465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:06.070406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709259+0000","last_change":"2024-09-10T21:10:30.180496+0000","last_active":"2024-09-10T21:10:31.709259+0000","last_peered":"2024-09-10T21:10:31.709259+0000","last_clean":"2024-09-10T21:10:31.709259+0000","last_became_active":"2024-09-10T21:10:30.180199+0000","last_became_peered":"2024-09-10T21:10:30.180199+0000","last_unstale":"2024-09-10T21:10:31.709259+0000","last_undegraded":"2024-09-10T21:10:31.709259+0000","last_fullsized":"2024-09-10T21:10:31.709259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:36.737204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179263+0000","last_change":"2024-09-10T21:10:30.187710+0000","last_active":"2024-09-10T21:10:31.179263+0000","last_peered":"2024-09-10T21:10:31.179263+0000","last_clean":"2024-09-10T21:10:31.179263+0000","last_became_active":"2024-09-10T21:10:30.186422+0000","last_became_peered":"2024-09-10T21:10:30.186422+0000","last_unstale":"2024-09-10T21:10:31.179263+0000","last_undegraded":"2024-09-10T21:10:31.179263+0000","last_fullsized":"2024-09-10T21:10:31.179263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:37.171374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173731+0000","last_change":"2024-09-10T21:10:30.157845+0000","last_active":"2024-09-10T21:10:31.173731+0000","last_peered":"2024-09-10T21:10:31.173731+0000","last_clean":"2024-09-10T21:10:31.173731+0000","last_became_active":"2024-09-10T21:10:30.155515+0000","last_became_peered":"2024-09-10T21:10:30.155515+0000","last_unstale":"2024-09-10T21:10:31.173731+0000","last_undegraded":"2024-09-10T21:10:31.173731+0000","last_fullsized":"2024-09-10T21:10:31.173731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:36.354773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167488+0000","last_change":"2024-09-10T21:10:30.168344+0000","last_active":"2024-09-10T21:10:31.167488+0000","last_peered":"2024-09-10T21:10:31.167488+0000","last_clean":"2024-09-10T21:10:31.167488+0000","last_became_active":"2024-09-10T21:10:30.167966+0000","last_became_peered":"2024-09-10T21:10:30.167966+0000","last_unstale":"2024-09-10T21:10:31.167488+0000","last_undegraded":"2024-09-10T21:10:31.167488+0000","last_fullsized":"2024-09-10T21:10:31.167488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:38.100207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166637+0000","last_change":"2024-09-10T21:10:30.159521+0000","last_active":"2024-09-10T21:10:31.166637+0000","last_peered":"2024-09-10T21:10:31.166637+0000","last_clean":"2024-09-10T21:10:31.166637+0000","last_became_active":"2024-09-10T21:10:30.159269+0000","last_became_peered":"2024-09-10T21:10:30.159269+0000","last_unstale":"2024-09-10T21:10:31.166637+0000","last_undegraded":"2024-09-10T21:10:31.166637+0000","last_fullsized":"2024-09-10T21:10:31.166637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:21.433157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708679+0000","last_change":"2024-09-10T21:10:30.182227+0000","last_active":"2024-09-10T21:10:31.708679+0000","last_peered":"2024-09-10T21:10:31.708679+0000","last_clean":"2024-09-10T21:10:31.708679+0000","last_became_active":"2024-09-10T21:10:30.178359+0000","last_became_peered":"2024-09-10T21:10:30.178359+0000","last_unstale":"2024-09-10T21:10:31.708679+0000","last_undegraded":"2024-09-10T21:10:31.708679+0000","last_fullsized":"2024-09-10T21:10:31.708679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:50.308163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166494+0000","last_change":"2024-09-10T21:10:30.170127+0000","last_active":"2024-09-10T21:10:31.166494+0000","last_peered":"2024-09-10T21:10:31.166494+0000","last_clean":"2024-09-10T21:10:31.166494+0000","last_became_active":"2024-09-10T21:10:30.169781+0000","last_became_peered":"2024-09-10T21:10:30.169781+0000","last_unstale":"2024-09-10T21:10:31.166494+0000","last_undegraded":"2024-09-10T21:10:31.166494+0000","last_fullsized":"2024-09-10T21:10:31.166494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:16.729840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167717+0000","last_change":"2024-09-10T21:10:30.161842+0000","last_active":"2024-09-10T21:10:31.167717+0000","last_peered":"2024-09-10T21:10:31.167717+0000","last_clean":"2024-09-10T21:10:31.167717+0000","last_became_active":"2024-09-10T21:10:30.160146+0000","last_became_peered":"2024-09-10T21:10:30.160146+0000","last_unstale":"2024-09-10T21:10:31.167717+0000","last_undegraded":"2024-09-10T21:10:31.167717+0000","last_fullsized":"2024-09-10T21:10:31.167717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:06.600402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707965+0000","last_change":"2024-09-10T21:10:30.176930+0000","last_active":"2024-09-10T21:10:31.707965+0000","last_peered":"2024-09-10T21:10:31.707965+0000","last_clean":"2024-09-10T21:10:31.707965+0000","last_became_active":"2024-09-10T21:10:30.175522+0000","last_became_peered":"2024-09-10T21:10:30.175522+0000","last_unstale":"2024-09-10T21:10:31.707965+0000","last_undegraded":"2024-09-10T21:10:31.707965+0000","last_fullsized":"2024-09-10T21:10:31.707965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:36.354773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707870+0000","last_change":"2024-09-10T21:10:30.179884+0000","last_active":"2024-09-10T21:10:31.707870+0000","last_peered":"2024-09-10T21:10:31.707870+0000","last_clean":"2024-09-10T21:10:31.707870+0000","last_became_active":"2024-09-10T21:10:30.177866+0000","last_became_peered":"2024-09-10T21:10:30.177866+0000","last_unstale":"2024-09-10T21:10:31.707870+0000","last_undegraded":"2024-09-10T21:10:31.707870+0000","last_fullsized":"2024-09-10T21:10:31.707870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:57.957611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166067+0000","last_change":"2024-09-10T21:10:30.169322+0000","last_active":"2024-09-10T21:10:31.166067+0000","last_peered":"2024-09-10T21:10:31.166067+0000","last_clean":"2024-09-10T21:10:31.166067+0000","last_became_active":"2024-09-10T21:10:30.169030+0000","last_became_peered":"2024-09-10T21:10:30.169030+0000","last_unstale":"2024-09-10T21:10:31.166067+0000","last_undegraded":"2024-09-10T21:10:31.166067+0000","last_fullsized":"2024-09-10T21:10:31.166067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:41.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708760+0000","last_change":"2024-09-10T21:10:30.194536+0000","last_active":"2024-09-10T21:10:31.708760+0000","last_peered":"2024-09-10T21:10:31.708760+0000","last_clean":"2024-09-10T21:10:31.708760+0000","last_became_active":"2024-09-10T21:10:30.188848+0000","last_became_peered":"2024-09-10T21:10:30.188848+0000","last_unstale":"2024-09-10T21:10:31.708760+0000","last_undegraded":"2024-09-10T21:10:31.708760+0000","last_fullsized":"2024-09-10T21:10:31.708760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:31.264063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446249+0000","last_change":"2024-09-10T21:10:30.171833+0000","last_active":"2024-09-10T21:10:31.446249+0000","last_peered":"2024-09-10T21:10:31.446249+0000","last_clean":"2024-09-10T21:10:31.446249+0000","last_became_active":"2024-09-10T21:10:30.171461+0000","last_became_peered":"2024-09-10T21:10:30.171461+0000","last_unstale":"2024-09-10T21:10:31.446249+0000","last_undegraded":"2024-09-10T21:10:31.446249+0000","last_fullsized":"2024-09-10T21:10:31.446249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:32.018476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166344+0000","last_change":"2024-09-10T21:10:30.183415+0000","last_active":"2024-09-10T21:10:31.166344+0000","last_peered":"2024-09-10T21:10:31.166344+0000","last_clean":"2024-09-10T21:10:31.166344+0000","last_became_active":"2024-09-10T21:10:30.183093+0000","last_became_peered":"2024-09-10T21:10:30.183093+0000","last_unstale":"2024-09-10T21:10:31.166344+0000","last_undegraded":"2024-09-10T21:10:31.166344+0000","last_fullsized":"2024-09-10T21:10:31.166344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:30.558298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167929+0000","last_change":"2024-09-10T21:10:30.171980+0000","last_active":"2024-09-10T21:10:31.167929+0000","last_peered":"2024-09-10T21:10:31.167929+0000","last_clean":"2024-09-10T21:10:31.167929+0000","last_became_active":"2024-09-10T21:10:30.167136+0000","last_became_peered":"2024-09-10T21:10:30.167136+0000","last_unstale":"2024-09-10T21:10:31.167929+0000","last_undegraded":"2024-09-10T21:10:31.167929+0000","last_fullsized":"2024-09-10T21:10:31.167929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:33.142099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166187+0000","last_change":"2024-09-10T21:10:29.171062+0000","last_active":"2024-09-10T21:10:31.166187+0000","last_peered":"2024-09-10T21:10:31.166187+0000","last_clean":"2024-09-10T21:10:31.166187+0000","last_became_active":"2024-09-10T21:10:29.170765+0000","last_became_peered":"2024-09-10T21:10:29.170765+0000","last_unstale":"2024-09-10T21:10:31.166187+0000","last_undegraded":"2024-09-10T21:10:31.166187+0000","last_fullsized":"2024-09-10T21:10:31.166187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.425174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173821+0000","last_change":"2024-09-10T21:10:30.174660+0000","last_active":"2024-09-10T21:10:31.173821+0000","last_peered":"2024-09-10T21:10:31.173821+0000","last_clean":"2024-09-10T21:10:31.173821+0000","last_became_active":"2024-09-10T21:10:30.174372+0000","last_became_peered":"2024-09-10T21:10:30.174372+0000","last_unstale":"2024-09-10T21:10:31.173821+0000","last_undegraded":"2024-09-10T21:10:31.173821+0000","last_fullsized":"2024-09-10T21:10:31.173821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:59.722000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708981+0000","last_change":"2024-09-10T21:10:29.171541+0000","last_active":"2024-09-10T21:10:31.708981+0000","last_peered":"2024-09-10T21:10:31.708981+0000","last_clean":"2024-09-10T21:10:31.708981+0000","last_became_active":"2024-09-10T21:10:29.171269+0000","last_became_peered":"2024-09-10T21:10:29.171269+0000","last_unstale":"2024-09-10T21:10:31.708981+0000","last_undegraded":"2024-09-10T21:10:31.708981+0000","last_fullsized":"2024-09-10T21:10:31.708981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:10.046262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707383+0000","last_change":"2024-09-10T21:10:29.166482+0000","last_active":"2024-09-10T21:10:31.707383+0000","last_peered":"2024-09-10T21:10:31.707383+0000","last_clean":"2024-09-10T21:10:31.707383+0000","last_became_active":"2024-09-10T21:10:29.166091+0000","last_became_peered":"2024-09-10T21:10:29.166091+0000","last_unstale":"2024-09-10T21:10:31.707383+0000","last_undegraded":"2024-09-10T21:10:31.707383+0000","last_fullsized":"2024-09-10T21:10:31.707383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.425174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166297+0000","last_change":"2024-09-10T21:10:30.183180+0000","last_active":"2024-09-10T21:10:31.166297+0000","last_peered":"2024-09-10T21:10:31.166297+0000","last_clean":"2024-09-10T21:10:31.166297+0000","last_became_active":"2024-09-10T21:10:30.182908+0000","last_became_peered":"2024-09-10T21:10:30.182908+0000","last_unstale":"2024-09-10T21:10:31.166297+0000","last_undegraded":"2024-09-10T21:10:31.166297+0000","last_fullsized":"2024-09-10T21:10:31.166297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:37.697114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708748+0000","last_change":"2024-09-10T21:10:29.167898+0000","last_active":"2024-09-10T21:10:31.708748+0000","last_peered":"2024-09-10T21:10:31.708748+0000","last_clean":"2024-09-10T21:10:31.708748+0000","last_became_active":"2024-09-10T21:10:29.167498+0000","last_became_peered":"2024-09-10T21:10:29.167498+0000","last_unstale":"2024-09-10T21:10:31.708748+0000","last_undegraded":"2024-09-10T21:10:31.708748+0000","last_fullsized":"2024-09-10T21:10:31.708748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:50.337610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173782+0000","last_change":"2024-09-10T21:10:30.161329+0000","last_active":"2024-09-10T21:10:31.173782+0000","last_peered":"2024-09-10T21:10:31.173782+0000","last_clean":"2024-09-10T21:10:31.173782+0000","last_became_active":"2024-09-10T21:10:30.160309+0000","last_became_peered":"2024-09-10T21:10:30.160309+0000","last_unstale":"2024-09-10T21:10:31.173782+0000","last_undegraded":"2024-09-10T21:10:31.173782+0000","last_fullsized":"2024-09-10T21:10:31.173782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:14.235339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168376+0000","last_change":"2024-09-10T21:10:29.170975+0000","last_active":"2024-09-10T21:10:31.168376+0000","last_peered":"2024-09-10T21:10:31.168376+0000","last_clean":"2024-09-10T21:10:31.168376+0000","last_became_active":"2024-09-10T21:10:29.170623+0000","last_became_peered":"2024-09-10T21:10:29.170623+0000","last_unstale":"2024-09-10T21:10:31.168376+0000","last_undegraded":"2024-09-10T21:10:31.168376+0000","last_fullsized":"2024-09-10T21:10:31.168376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167790+0000","last_change":"2024-09-10T21:10:30.160525+0000","last_active":"2024-09-10T21:10:31.167790+0000","last_peered":"2024-09-10T21:10:31.167790+0000","last_clean":"2024-09-10T21:10:31.167790+0000","last_became_active":"2024-09-10T21:10:30.159639+0000","last_became_peered":"2024-09-10T21:10:30.159639+0000","last_unstale":"2024-09-10T21:10:31.167790+0000","last_undegraded":"2024-09-10T21:10:31.167790+0000","last_fullsized":"2024-09-10T21:10:31.167790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:46.116954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179609+0000","last_change":"2024-09-10T21:10:29.175134+0000","last_active":"2024-09-10T21:10:31.179609+0000","last_peered":"2024-09-10T21:10:31.179609+0000","last_clean":"2024-09-10T21:10:31.179609+0000","last_became_active":"2024-09-10T21:10:29.174808+0000","last_became_peered":"2024-09-10T21:10:29.174808+0000","last_unstale":"2024-09-10T21:10:31.179609+0000","last_undegraded":"2024-09-10T21:10:31.179609+0000","last_fullsized":"2024-09-10T21:10:31.179609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:52.586551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166756+0000","last_change":"2024-09-10T21:10:30.167445+0000","last_active":"2024-09-10T21:10:31.166756+0000","last_peered":"2024-09-10T21:10:31.166756+0000","last_clean":"2024-09-10T21:10:31.166756+0000","last_became_active":"2024-09-10T21:10:30.167163+0000","last_became_peered":"2024-09-10T21:10:30.167163+0000","last_unstale":"2024-09-10T21:10:31.166756+0000","last_undegraded":"2024-09-10T21:10:31.166756+0000","last_fullsized":"2024-09-10T21:10:31.166756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:59.291659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446376+0000","last_change":"2024-09-10T21:10:29.171850+0000","last_active":"2024-09-10T21:10:31.446376+0000","last_peered":"2024-09-10T21:10:31.446376+0000","last_clean":"2024-09-10T21:10:31.446376+0000","last_became_active":"2024-09-10T21:10:29.171589+0000","last_became_peered":"2024-09-10T21:10:29.171589+0000","last_unstale":"2024-09-10T21:10:31.446376+0000","last_undegraded":"2024-09-10T21:10:31.446376+0000","last_fullsized":"2024-09-10T21:10:31.446376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:43.143812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708938+0000","last_change":"2024-09-10T21:10:30.162228+0000","last_active":"2024-09-10T21:10:31.708938+0000","last_peered":"2024-09-10T21:10:31.708938+0000","last_clean":"2024-09-10T21:10:31.708938+0000","last_became_active":"2024-09-10T21:10:30.161901+0000","last_became_peered":"2024-09-10T21:10:30.161901+0000","last_unstale":"2024-09-10T21:10:31.708938+0000","last_undegraded":"2024-09-10T21:10:31.708938+0000","last_fullsized":"2024-09-10T21:10:31.708938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:12.489819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167129+0000","last_change":"2024-09-10T21:10:29.164653+0000","last_active":"2024-09-10T21:10:31.167129+0000","last_peered":"2024-09-10T21:10:31.167129+0000","last_clean":"2024-09-10T21:10:31.167129+0000","last_became_active":"2024-09-10T21:10:29.164338+0000","last_became_peered":"2024-09-10T21:10:29.164338+0000","last_unstale":"2024-09-10T21:10:31.167129+0000","last_undegraded":"2024-09-10T21:10:31.167129+0000","last_fullsized":"2024-09-10T21:10:31.167129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:24.324180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179343+0000","last_change":"2024-09-10T21:10:30.191023+0000","last_active":"2024-09-10T21:10:31.179343+0000","last_peered":"2024-09-10T21:10:31.179343+0000","last_clean":"2024-09-10T21:10:31.179343+0000","last_became_active":"2024-09-10T21:10:30.190264+0000","last_became_peered":"2024-09-10T21:10:30.190264+0000","last_unstale":"2024-09-10T21:10:31.179343+0000","last_undegraded":"2024-09-10T21:10:31.179343+0000","last_fullsized":"2024-09-10T21:10:31.179343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:08.381083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446251+0000","last_change":"2024-09-10T21:10:29.178312+0000","last_active":"2024-09-10T21:10:31.446251+0000","last_peered":"2024-09-10T21:10:31.446251+0000","last_clean":"2024-09-10T21:10:31.446251+0000","last_became_active":"2024-09-10T21:10:29.178044+0000","last_became_peered":"2024-09-10T21:10:29.178044+0000","last_unstale":"2024-09-10T21:10:31.446251+0000","last_undegraded":"2024-09-10T21:10:31.446251+0000","last_fullsized":"2024-09-10T21:10:31.446251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:41.871122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707682+0000","last_change":"2024-09-10T21:10:30.181507+0000","last_active":"2024-09-10T21:10:31.707682+0000","last_peered":"2024-09-10T21:10:31.707682+0000","last_clean":"2024-09-10T21:10:31.707682+0000","last_became_active":"2024-09-10T21:10:30.177520+0000","last_became_peered":"2024-09-10T21:10:30.177520+0000","last_unstale":"2024-09-10T21:10:31.707682+0000","last_undegraded":"2024-09-10T21:10:31.707682+0000","last_fullsized":"2024-09-10T21:10:31.707682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:56.567997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709035+0000","last_change":"2024-09-10T21:10:29.166737+0000","last_active":"2024-09-10T21:10:31.709035+0000","last_peered":"2024-09-10T21:10:31.709035+0000","last_clean":"2024-09-10T21:10:31.709035+0000","last_became_active":"2024-09-10T21:10:29.166301+0000","last_became_peered":"2024-09-10T21:10:29.166301+0000","last_unstale":"2024-09-10T21:10:31.709035+0000","last_undegraded":"2024-09-10T21:10:31.709035+0000","last_fullsized":"2024-09-10T21:10:31.709035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:52.586551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709252+0000","last_change":"2024-09-10T21:10:30.180470+0000","last_active":"2024-09-10T21:10:31.709252+0000","last_peered":"2024-09-10T21:10:31.709252+0000","last_clean":"2024-09-10T21:10:31.709252+0000","last_became_active":"2024-09-10T21:10:30.179447+0000","last_became_peered":"2024-09-10T21:10:30.179447+0000","last_unstale":"2024-09-10T21:10:31.709252+0000","last_undegraded":"2024-09-10T21:10:31.709252+0000","last_fullsized":"2024-09-10T21:10:31.709252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:36.701958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178781+0000","last_change":"2024-09-10T21:10:29.169914+0000","last_active":"2024-09-10T21:10:31.178781+0000","last_peered":"2024-09-10T21:10:31.178781+0000","last_clean":"2024-09-10T21:10:31.178781+0000","last_became_active":"2024-09-10T21:10:29.169617+0000","last_became_peered":"2024-09-10T21:10:29.169617+0000","last_unstale":"2024-09-10T21:10:31.178781+0000","last_undegraded":"2024-09-10T21:10:31.178781+0000","last_fullsized":"2024-09-10T21:10:31.178781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:48.286592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167103+0000","last_change":"2024-09-10T21:10:30.169299+0000","last_active":"2024-09-10T21:10:31.167103+0000","last_peered":"2024-09-10T21:10:31.167103+0000","last_clean":"2024-09-10T21:10:31.167103+0000","last_became_active":"2024-09-10T21:10:30.169013+0000","last_became_peered":"2024-09-10T21:10:30.169013+0000","last_unstale":"2024-09-10T21:10:31.167103+0000","last_undegraded":"2024-09-10T21:10:31.167103+0000","last_fullsized":"2024-09-10T21:10:31.167103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:40.546116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173457+0000","last_change":"2024-09-10T21:10:29.162746+0000","last_active":"2024-09-10T21:10:31.173457+0000","last_peered":"2024-09-10T21:10:31.173457+0000","last_clean":"2024-09-10T21:10:31.173457+0000","last_became_active":"2024-09-10T21:10:29.162466+0000","last_became_peered":"2024-09-10T21:10:29.162466+0000","last_unstale":"2024-09-10T21:10:31.173457+0000","last_undegraded":"2024-09-10T21:10:31.173457+0000","last_fullsized":"2024-09-10T21:10:31.173457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.425174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708924+0000","last_change":"2024-09-10T21:10:30.162565+0000","last_active":"2024-09-10T21:10:31.708924+0000","last_peered":"2024-09-10T21:10:31.708924+0000","last_clean":"2024-09-10T21:10:31.708924+0000","last_became_active":"2024-09-10T21:10:30.161927+0000","last_became_peered":"2024-09-10T21:10:30.161927+0000","last_unstale":"2024-09-10T21:10:31.708924+0000","last_undegraded":"2024-09-10T21:10:31.708924+0000","last_fullsized":"2024-09-10T21:10:31.708924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:43.633279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.180274+0000","last_change":"2024-09-10T21:10:29.170318+0000","last_active":"2024-09-10T21:10:31.180274+0000","last_peered":"2024-09-10T21:10:31.180274+0000","last_clean":"2024-09-10T21:10:31.180274+0000","last_became_active":"2024-09-10T21:10:29.169096+0000","last_became_peered":"2024-09-10T21:10:29.169096+0000","last_unstale":"2024-09-10T21:10:31.180274+0000","last_undegraded":"2024-09-10T21:10:31.180274+0000","last_fullsized":"2024-09-10T21:10:31.180274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:01.835985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167829+0000","last_change":"2024-09-10T21:10:30.176716+0000","last_active":"2024-09-10T21:10:31.167829+0000","last_peered":"2024-09-10T21:10:31.167829+0000","last_clean":"2024-09-10T21:10:31.167829+0000","last_became_active":"2024-09-10T21:10:30.175255+0000","last_became_peered":"2024-09-10T21:10:30.175255+0000","last_unstale":"2024-09-10T21:10:31.167829+0000","last_undegraded":"2024-09-10T21:10:31.167829+0000","last_fullsized":"2024-09-10T21:10:31.167829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.859164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173973+0000","last_change":"2024-09-10T21:10:29.163718+0000","last_active":"2024-09-10T21:10:31.173973+0000","last_peered":"2024-09-10T21:10:31.173973+0000","last_clean":"2024-09-10T21:10:31.173973+0000","last_became_active":"2024-09-10T21:10:29.163328+0000","last_became_peered":"2024-09-10T21:10:29.163328+0000","last_unstale":"2024-09-10T21:10:31.173973+0000","last_undegraded":"2024-09-10T21:10:31.173973+0000","last_fullsized":"2024-09-10T21:10:31.173973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:57.741500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.180052+0000","last_change":"2024-09-10T21:10:30.183206+0000","last_active":"2024-09-10T21:10:31.180052+0000","last_peered":"2024-09-10T21:10:31.180052+0000","last_clean":"2024-09-10T21:10:31.180052+0000","last_became_active":"2024-09-10T21:10:30.182950+0000","last_became_peered":"2024-09-10T21:10:30.182950+0000","last_unstale":"2024-09-10T21:10:31.180052+0000","last_undegraded":"2024-09-10T21:10:31.180052+0000","last_fullsized":"2024-09-10T21:10:31.180052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:45.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166843+0000","last_change":"2024-09-10T21:10:29.165292+0000","last_active":"2024-09-10T21:10:31.166843+0000","last_peered":"2024-09-10T21:10:31.166843+0000","last_clean":"2024-09-10T21:10:31.166843+0000","last_became_active":"2024-09-10T21:10:29.164991+0000","last_became_peered":"2024-09-10T21:10:29.164991+0000","last_unstale":"2024-09-10T21:10:31.166843+0000","last_undegraded":"2024-09-10T21:10:31.166843+0000","last_fullsized":"2024-09-10T21:10:31.166843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:57.741500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179462+0000","last_change":"2024-09-10T21:10:30.188067+0000","last_active":"2024-09-10T21:10:31.179462+0000","last_peered":"2024-09-10T21:10:31.179462+0000","last_clean":"2024-09-10T21:10:31.179462+0000","last_became_active":"2024-09-10T21:10:30.187079+0000","last_became_peered":"2024-09-10T21:10:30.187079+0000","last_unstale":"2024-09-10T21:10:31.179462+0000","last_undegraded":"2024-09-10T21:10:31.179462+0000","last_fullsized":"2024-09-10T21:10:31.179462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:16.358565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707972+0000","last_change":"2024-09-10T21:10:29.168326+0000","last_active":"2024-09-10T21:10:31.707972+0000","last_peered":"2024-09-10T21:10:31.707972+0000","last_clean":"2024-09-10T21:10:31.707972+0000","last_became_active":"2024-09-10T21:10:29.168048+0000","last_became_peered":"2024-09-10T21:10:29.168048+0000","last_unstale":"2024-09-10T21:10:31.707972+0000","last_undegraded":"2024-09-10T21:10:31.707972+0000","last_fullsized":"2024-09-10T21:10:31.707972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:56.871372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709184+0000","last_change":"2024-09-10T21:10:30.179705+0000","last_active":"2024-09-10T21:10:31.709184+0000","last_peered":"2024-09-10T21:10:31.709184+0000","last_clean":"2024-09-10T21:10:31.709184+0000","last_became_active":"2024-09-10T21:10:30.175366+0000","last_became_peered":"2024-09-10T21:10:30.175366+0000","last_unstale":"2024-09-10T21:10:31.709184+0000","last_undegraded":"2024-09-10T21:10:31.709184+0000","last_fullsized":"2024-09-10T21:10:31.709184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:44.960366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167202+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:10:31.167202+0000","last_peered":"2024-09-10T21:10:31.167202+0000","last_clean":"2024-09-10T21:10:31.167202+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:10:31.167202+0000","last_undegraded":"2024-09-10T21:10:31.167202+0000","last_fullsized":"2024-09-10T21:10:31.167202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708999+0000","last_change":"2024-09-10T21:10:29.172283+0000","last_active":"2024-09-10T21:10:31.708999+0000","last_peered":"2024-09-10T21:10:31.708999+0000","last_clean":"2024-09-10T21:10:31.708999+0000","last_became_active":"2024-09-10T21:10:29.170550+0000","last_became_peered":"2024-09-10T21:10:29.170550+0000","last_unstale":"2024-09-10T21:10:31.708999+0000","last_undegraded":"2024-09-10T21:10:31.708999+0000","last_fullsized":"2024-09-10T21:10:31.708999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:09.439008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173734+0000","last_change":"2024-09-10T21:10:30.160275+0000","last_active":"2024-09-10T21:10:31.173734+0000","last_peered":"2024-09-10T21:10:31.173734+0000","last_clean":"2024-09-10T21:10:31.173734+0000","last_became_active":"2024-09-10T21:10:30.159817+0000","last_became_peered":"2024-09-10T21:10:30.159817+0000","last_unstale":"2024-09-10T21:10:31.173734+0000","last_undegraded":"2024-09-10T21:10:31.173734+0000","last_fullsized":"2024-09-10T21:10:31.173734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.698693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708902+0000","last_change":"2024-09-10T21:10:29.164034+0000","last_active":"2024-09-10T21:10:31.708902+0000","last_peered":"2024-09-10T21:10:31.708902+0000","last_clean":"2024-09-10T21:10:31.708902+0000","last_became_active":"2024-09-10T21:10:29.163755+0000","last_became_peered":"2024-09-10T21:10:29.163755+0000","last_unstale":"2024-09-10T21:10:31.708902+0000","last_undegraded":"2024-09-10T21:10:31.708902+0000","last_fullsized":"2024-09-10T21:10:31.708902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:10.200363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707745+0000","last_change":"2024-09-10T21:10:30.171490+0000","last_active":"2024-09-10T21:10:31.707745+0000","last_peered":"2024-09-10T21:10:31.707745+0000","last_clean":"2024-09-10T21:10:31.707745+0000","last_became_active":"2024-09-10T21:10:30.170892+0000","last_became_peered":"2024-09-10T21:10:30.170892+0000","last_unstale":"2024-09-10T21:10:31.707745+0000","last_undegraded":"2024-09-10T21:10:31.707745+0000","last_fullsized":"2024-09-10T21:10:31.707745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:36.425140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446253+0000","last_change":"2024-09-10T21:10:29.170636+0000","last_active":"2024-09-10T21:10:31.446253+0000","last_peered":"2024-09-10T21:10:31.446253+0000","last_clean":"2024-09-10T21:10:31.446253+0000","last_became_active":"2024-09-10T21:10:29.170342+0000","last_became_peered":"2024-09-10T21:10:29.170342+0000","last_unstale":"2024-09-10T21:10:31.446253+0000","last_undegraded":"2024-09-10T21:10:31.446253+0000","last_fullsized":"2024-09-10T21:10:31.446253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166047+0000","last_change":"2024-09-10T21:10:30.164807+0000","last_active":"2024-09-10T21:10:31.166047+0000","last_peered":"2024-09-10T21:10:31.166047+0000","last_clean":"2024-09-10T21:10:31.166047+0000","last_became_active":"2024-09-10T21:10:30.164545+0000","last_became_peered":"2024-09-10T21:10:30.164545+0000","last_unstale":"2024-09-10T21:10:31.166047+0000","last_undegraded":"2024-09-10T21:10:31.166047+0000","last_fullsized":"2024-09-10T21:10:31.166047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:35.647295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709113+0000","last_change":"2024-09-10T21:10:29.173240+0000","last_active":"2024-09-10T21:10:31.709113+0000","last_peered":"2024-09-10T21:10:31.709113+0000","last_clean":"2024-09-10T21:10:31.709113+0000","last_became_active":"2024-09-10T21:10:29.172991+0000","last_became_peered":"2024-09-10T21:10:29.172991+0000","last_unstale":"2024-09-10T21:10:31.709113+0000","last_undegraded":"2024-09-10T21:10:31.709113+0000","last_fullsized":"2024-09-10T21:10:31.709113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:00.278081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178965+0000","last_change":"2024-09-10T21:10:30.189010+0000","last_active":"2024-09-10T21:10:31.178965+0000","last_peered":"2024-09-10T21:10:31.178965+0000","last_clean":"2024-09-10T21:10:31.178965+0000","last_became_active":"2024-09-10T21:10:30.188712+0000","last_became_peered":"2024-09-10T21:10:30.188712+0000","last_unstale":"2024-09-10T21:10:31.178965+0000","last_undegraded":"2024-09-10T21:10:31.178965+0000","last_fullsized":"2024-09-10T21:10:31.178965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:04.853111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.165941+0000","last_change":"2024-09-10T21:10:29.173419+0000","last_active":"2024-09-10T21:10:31.165941+0000","last_peered":"2024-09-10T21:10:31.165941+0000","last_clean":"2024-09-10T21:10:31.165941+0000","last_became_active":"2024-09-10T21:10:29.173104+0000","last_became_peered":"2024-09-10T21:10:29.173104+0000","last_unstale":"2024-09-10T21:10:31.165941+0000","last_undegraded":"2024-09-10T21:10:31.165941+0000","last_fullsized":"2024-09-10T21:10:31.165941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:25.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167265+0000","last_change":"2024-09-10T21:10:30.169171+0000","last_active":"2024-09-10T21:10:31.167265+0000","last_peered":"2024-09-10T21:10:31.167265+0000","last_clean":"2024-09-10T21:10:31.167265+0000","last_became_active":"2024-09-10T21:10:30.168866+0000","last_became_peered":"2024-09-10T21:10:30.168866+0000","last_unstale":"2024-09-10T21:10:31.167265+0000","last_undegraded":"2024-09-10T21:10:31.167265+0000","last_fullsized":"2024-09-10T21:10:31.167265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:45.633966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709233+0000","last_change":"2024-09-10T21:10:29.170135+0000","last_active":"2024-09-10T21:10:31.709233+0000","last_peered":"2024-09-10T21:10:31.709233+0000","last_clean":"2024-09-10T21:10:31.709233+0000","last_became_active":"2024-09-10T21:10:29.169894+0000","last_became_peered":"2024-09-10T21:10:29.169894+0000","last_unstale":"2024-09-10T21:10:31.709233+0000","last_undegraded":"2024-09-10T21:10:31.709233+0000","last_fullsized":"2024-09-10T21:10:31.709233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:10.057351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707993+0000","last_change":"2024-09-10T21:10:30.171973+0000","last_active":"2024-09-10T21:10:31.707993+0000","last_peered":"2024-09-10T21:10:31.707993+0000","last_clean":"2024-09-10T21:10:31.707993+0000","last_became_active":"2024-09-10T21:10:30.171622+0000","last_became_peered":"2024-09-10T21:10:30.171622+0000","last_unstale":"2024-09-10T21:10:31.707993+0000","last_undegraded":"2024-09-10T21:10:31.707993+0000","last_fullsized":"2024-09-10T21:10:31.707993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:06.330104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174157+0000","last_change":"2024-09-10T21:10:29.165107+0000","last_active":"2024-09-10T21:10:31.174157+0000","last_peered":"2024-09-10T21:10:31.174157+0000","last_clean":"2024-09-10T21:10:31.174157+0000","last_became_active":"2024-09-10T21:10:29.164798+0000","last_became_peered":"2024-09-10T21:10:29.164798+0000","last_unstale":"2024-09-10T21:10:31.174157+0000","last_undegraded":"2024-09-10T21:10:31.174157+0000","last_fullsized":"2024-09-10T21:10:31.174157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:04.055586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174286+0000","last_change":"2024-09-10T21:10:30.173086+0000","last_active":"2024-09-10T21:10:31.174286+0000","last_peered":"2024-09-10T21:10:31.174286+0000","last_clean":"2024-09-10T21:10:31.174286+0000","last_became_active":"2024-09-10T21:10:30.172730+0000","last_became_peered":"2024-09-10T21:10:30.172730+0000","last_unstale":"2024-09-10T21:10:31.174286+0000","last_undegraded":"2024-09-10T21:10:31.174286+0000","last_fullsized":"2024-09-10T21:10:31.174286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:50.512074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709420+0000","last_change":"2024-09-10T21:10:29.179231+0000","last_active":"2024-09-10T21:10:31.709420+0000","last_peered":"2024-09-10T21:10:31.709420+0000","last_clean":"2024-09-10T21:10:31.709420+0000","last_became_active":"2024-09-10T21:10:29.178937+0000","last_became_peered":"2024-09-10T21:10:29.178937+0000","last_unstale":"2024-09-10T21:10:31.709420+0000","last_undegraded":"2024-09-10T21:10:31.709420+0000","last_fullsized":"2024-09-10T21:10:31.709420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:41.528182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174168+0000","last_change":"2024-09-10T21:10:30.167226+0000","last_active":"2024-09-10T21:10:31.174168+0000","last_peered":"2024-09-10T21:10:31.174168+0000","last_clean":"2024-09-10T21:10:31.174168+0000","last_became_active":"2024-09-10T21:10:30.166959+0000","last_became_peered":"2024-09-10T21:10:30.166959+0000","last_unstale":"2024-09-10T21:10:31.174168+0000","last_undegraded":"2024-09-10T21:10:31.174168+0000","last_fullsized":"2024-09-10T21:10:31.174168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:43.225225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178611+0000","last_change":"2024-09-10T21:10:29.175872+0000","last_active":"2024-09-10T21:10:31.178611+0000","last_peered":"2024-09-10T21:10:31.178611+0000","last_clean":"2024-09-10T21:10:31.178611+0000","last_became_active":"2024-09-10T21:10:29.175659+0000","last_became_peered":"2024-09-10T21:10:29.175659+0000","last_unstale":"2024-09-10T21:10:31.178611+0000","last_undegraded":"2024-09-10T21:10:31.178611+0000","last_fullsized":"2024-09-10T21:10:31.178611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:21.990689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708884+0000","last_change":"2024-09-10T21:10:30.179720+0000","last_active":"2024-09-10T21:10:31.708884+0000","last_peered":"2024-09-10T21:10:31.708884+0000","last_clean":"2024-09-10T21:10:31.708884+0000","last_became_active":"2024-09-10T21:10:30.179435+0000","last_became_peered":"2024-09-10T21:10:30.179435+0000","last_unstale":"2024-09-10T21:10:31.708884+0000","last_undegraded":"2024-09-10T21:10:31.708884+0000","last_fullsized":"2024-09-10T21:10:31.708884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:45.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173726+0000","last_change":"2024-09-10T21:10:29.165142+0000","last_active":"2024-09-10T21:10:31.173726+0000","last_peered":"2024-09-10T21:10:31.173726+0000","last_clean":"2024-09-10T21:10:31.173726+0000","last_became_active":"2024-09-10T21:10:29.164732+0000","last_became_peered":"2024-09-10T21:10:29.164732+0000","last_unstale":"2024-09-10T21:10:31.173726+0000","last_undegraded":"2024-09-10T21:10:31.173726+0000","last_fullsized":"2024-09-10T21:10:31.173726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:26.261468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173644+0000","last_change":"2024-09-10T21:10:30.164096+0000","last_active":"2024-09-10T21:10:31.173644+0000","last_peered":"2024-09-10T21:10:31.173644+0000","last_clean":"2024-09-10T21:10:31.173644+0000","last_became_active":"2024-09-10T21:10:30.163474+0000","last_became_peered":"2024-09-10T21:10:30.163474+0000","last_unstale":"2024-09-10T21:10:31.173644+0000","last_undegraded":"2024-09-10T21:10:31.173644+0000","last_fullsized":"2024-09-10T21:10:31.173644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:57.957611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707812+0000","last_change":"2024-09-10T21:10:29.166991+0000","last_active":"2024-09-10T21:10:31.707812+0000","last_peered":"2024-09-10T21:10:31.707812+0000","last_clean":"2024-09-10T21:10:31.707812+0000","last_became_active":"2024-09-10T21:10:29.166675+0000","last_became_peered":"2024-09-10T21:10:29.166675+0000","last_unstale":"2024-09-10T21:10:31.707812+0000","last_undegraded":"2024-09-10T21:10:31.707812+0000","last_fullsized":"2024-09-10T21:10:31.707812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:25.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446431+0000","last_change":"2024-09-10T21:10:30.176046+0000","last_active":"2024-09-10T21:10:31.446431+0000","last_peered":"2024-09-10T21:10:31.446431+0000","last_clean":"2024-09-10T21:10:31.446431+0000","last_became_active":"2024-09-10T21:10:30.175784+0000","last_became_peered":"2024-09-10T21:10:30.175784+0000","last_unstale":"2024-09-10T21:10:31.446431+0000","last_undegraded":"2024-09-10T21:10:31.446431+0000","last_fullsized":"2024-09-10T21:10:31.446431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:37.972763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708858+0000","last_change":"2024-09-10T21:10:29.172106+0000","last_active":"2024-09-10T21:10:31.708858+0000","last_peered":"2024-09-10T21:10:31.708858+0000","last_clean":"2024-09-10T21:10:31.708858+0000","last_became_active":"2024-09-10T21:10:29.167132+0000","last_became_peered":"2024-09-10T21:10:29.167132+0000","last_unstale":"2024-09-10T21:10:31.708858+0000","last_undegraded":"2024-09-10T21:10:31.708858+0000","last_fullsized":"2024-09-10T21:10:31.708858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:48.493501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708898+0000","last_change":"2024-09-10T21:10:30.176689+0000","last_active":"2024-09-10T21:10:31.708898+0000","last_peered":"2024-09-10T21:10:31.708898+0000","last_clean":"2024-09-10T21:10:31.708898+0000","last_became_active":"2024-09-10T21:10:30.176361+0000","last_became_peered":"2024-09-10T21:10:30.176361+0000","last_unstale":"2024-09-10T21:10:31.708898+0000","last_undegraded":"2024-09-10T21:10:31.708898+0000","last_fullsized":"2024-09-10T21:10:31.708898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:55.720343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709088+0000","last_change":"2024-09-10T21:10:30.180387+0000","last_active":"2024-09-10T21:10:31.709088+0000","last_peered":"2024-09-10T21:10:31.709088+0000","last_clean":"2024-09-10T21:10:31.709088+0000","last_became_active":"2024-09-10T21:10:30.180100+0000","last_became_peered":"2024-09-10T21:10:30.180100+0000","last_unstale":"2024-09-10T21:10:31.709088+0000","last_undegraded":"2024-09-10T21:10:31.709088+0000","last_fullsized":"2024-09-10T21:10:31.709088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:05.939523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179323+0000","last_change":"2024-09-10T21:10:29.178074+0000","last_active":"2024-09-10T21:10:31.179323+0000","last_peered":"2024-09-10T21:10:31.179323+0000","last_clean":"2024-09-10T21:10:31.179323+0000","last_became_active":"2024-09-10T21:10:29.177794+0000","last_became_peered":"2024-09-10T21:10:29.177794+0000","last_unstale":"2024-09-10T21:10:31.179323+0000","last_undegraded":"2024-09-10T21:10:31.179323+0000","last_fullsized":"2024-09-10T21:10:31.179323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:18.267264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167690+0000","last_change":"2024-09-10T21:10:30.179424+0000","last_active":"2024-09-10T21:10:31.167690+0000","last_peered":"2024-09-10T21:10:31.167690+0000","last_clean":"2024-09-10T21:10:31.167690+0000","last_became_active":"2024-09-10T21:10:30.178557+0000","last_became_peered":"2024-09-10T21:10:30.178557+0000","last_unstale":"2024-09-10T21:10:31.167690+0000","last_undegraded":"2024-09-10T21:10:31.167690+0000","last_fullsized":"2024-09-10T21:10:31.167690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:14.118820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179366+0000","last_change":"2024-09-10T21:10:29.169287+0000","last_active":"2024-09-10T21:10:31.179366+0000","last_peered":"2024-09-10T21:10:31.179366+0000","last_clean":"2024-09-10T21:10:31.179366+0000","last_became_active":"2024-09-10T21:10:29.169010+0000","last_became_peered":"2024-09-10T21:10:29.169010+0000","last_unstale":"2024-09-10T21:10:31.179366+0000","last_undegraded":"2024-09-10T21:10:31.179366+0000","last_fullsized":"2024-09-10T21:10:31.179366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:41.528182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166555+0000","last_change":"2024-09-10T21:10:30.166099+0000","last_active":"2024-09-10T21:10:31.166555+0000","last_peered":"2024-09-10T21:10:31.166555+0000","last_clean":"2024-09-10T21:10:31.166555+0000","last_became_active":"2024-09-10T21:10:30.165762+0000","last_became_peered":"2024-09-10T21:10:30.165762+0000","last_unstale":"2024-09-10T21:10:31.166555+0000","last_undegraded":"2024-09-10T21:10:31.166555+0000","last_fullsized":"2024-09-10T21:10:31.166555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:54.002262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166762+0000","last_change":"2024-09-10T21:10:29.167583+0000","last_active":"2024-09-10T21:10:31.166762+0000","last_peered":"2024-09-10T21:10:31.166762+0000","last_clean":"2024-09-10T21:10:31.166762+0000","last_became_active":"2024-09-10T21:10:29.162012+0000","last_became_peered":"2024-09-10T21:10:29.162012+0000","last_unstale":"2024-09-10T21:10:31.166762+0000","last_undegraded":"2024-09-10T21:10:31.166762+0000","last_fullsized":"2024-09-10T21:10:31.166762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:04.055586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166801+0000","last_change":"2024-09-10T21:10:30.162898+0000","last_active":"2024-09-10T21:10:31.166801+0000","last_peered":"2024-09-10T21:10:31.166801+0000","last_clean":"2024-09-10T21:10:31.166801+0000","last_became_active":"2024-09-10T21:10:30.162536+0000","last_became_peered":"2024-09-10T21:10:30.162536+0000","last_unstale":"2024-09-10T21:10:31.166801+0000","last_undegraded":"2024-09-10T21:10:31.166801+0000","last_fullsized":"2024-09-10T21:10:31.166801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:09.399612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174191+0000","last_change":"2024-09-10T21:10:29.166448+0000","last_active":"2024-09-10T21:10:31.174191+0000","last_peered":"2024-09-10T21:10:31.174191+0000","last_clean":"2024-09-10T21:10:31.174191+0000","last_became_active":"2024-09-10T21:10:29.166153+0000","last_became_peered":"2024-09-10T21:10:29.166153+0000","last_unstale":"2024-09-10T21:10:31.174191+0000","last_undegraded":"2024-09-10T21:10:31.174191+0000","last_fullsized":"2024-09-10T21:10:31.174191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:08.960569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":811008,"data_stored":662647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":74471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":72769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":217088,"data_stored":72243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":806912,"data_stored":662625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:10:42.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:10:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.518+0000 7f3880800640 1 -- 172.21.15.16:0/4117654627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3860004da0 msgr2=0x7f3860005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.518+0000 7f3880800640 1 --2- 172.21.15.16:0/4117654627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3860004da0 0x7f3860005200 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3878074ce0 tx=0x7f387401d590 comp rx=0 tx=0).stop 2024-09-10T21:10:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.518+0000 7f387e800640 1 -- 172.21.15.16:0/4117654627 reap_dead start 2024-09-10T21:10:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.518+0000 7f3880800640 1 -- 172.21.15.16:0/4117654627 shutdown_connections 2024-09-10T21:10:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.518+0000 7f3880800640 1 -- 172.21.15.16:0/4117654627 >> 172.21.15.16:0/4117654627 conn(0x7f387809b130 msgr2=0x7f38780d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.518+0000 7f3880800640 1 -- 172.21.15.16:0/4117654627 shutdown_connections 2024-09-10T21:10:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.518+0000 7f3880800640 1 -- 172.21.15.16:0/4117654627 wait complete. 2024-09-10T21:10:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3880800640 1 Processor -- start 2024-09-10T21:10:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3880800640 1 -- start start 2024-09-10T21:10:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3880800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3860004da0 0x7f3878158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3880800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f38780b56d0 0x7f38781fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3880800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3878070b50 0x7f3878070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3880800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3878167bd0 con 0x7f3860004da0 2024-09-10T21:10:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3880800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f38780b0760 con 0x7f3878070b50 2024-09-10T21:10:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3880800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3878167d50 con 0x7f38780b56d0 2024-09-10T21:10:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f387de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3860004da0 0x7f3878158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f387de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3860004da0 0x7f3878158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36334/0 (socket says 172.21.15.16:36334) 2024-09-10T21:10:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f387de00640 1 -- 172.21.15.16:0/2265467474 learned_addr learned my addr 172.21.15.16:0/2265467474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f387e800640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3878070b50 0x7f3878070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f387d400640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f38780b56d0 0x7f38781fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f387de00640 1 -- 172.21.15.16:0/2265467474 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f38780b56d0 msgr2=0x7f38781fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f387de00640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f38780b56d0 0x7f38781fb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f387de00640 1 -- 172.21.15.16:0/2265467474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3878070b50 msgr2=0x7f3878070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f387de00640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3878070b50 0x7f3878070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f387de00640 1 -- 172.21.15.16:0/2265467474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38780b5bb0 con 0x7f3860004da0 2024-09-10T21:10:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f387de00640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3860004da0 0x7f3878158ed0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f386c0063d0 tx=0x7f386c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3872000640 1 -- 172.21.15.16:0/2265467474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c015070 con 0x7f3860004da0 2024-09-10T21:10:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3872000640 1 -- 172.21.15.16:0/2265467474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f386c011900 con 0x7f3860004da0 2024-09-10T21:10:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3880800640 1 -- 172.21.15.16:0/2265467474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38780b5e90 con 0x7f3860004da0 2024-09-10T21:10:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3872000640 1 -- 172.21.15.16:0/2265467474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c011d50 con 0x7f3860004da0 2024-09-10T21:10:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3880800640 1 -- 172.21.15.16:0/2265467474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3878156310 con 0x7f3860004da0 2024-09-10T21:10:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.522+0000 7f3880800640 1 -- 172.21.15.16:0/2265467474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38780b63b0 con 0x7f3860004da0 2024-09-10T21:10:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.526+0000 7f3872000640 1 -- 172.21.15.16:0/2265467474 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f386c0103c0 con 0x7f3860004da0 2024-09-10T21:10:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.526+0000 7f3872000640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f385c072610 0x7f385c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.526+0000 7f3872000640 1 -- 172.21.15.16:0/2265467474 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f386c094270 con 0x7f3860004da0 2024-09-10T21:10:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.526+0000 7f387d400640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f385c072610 0x7f385c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:42.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.526+0000 7f387d400640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f385c072610 0x7f385c074ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f38780c5670 tx=0x7f3868006b80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:42.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.530+0000 7f3872000640 1 -- 172.21.15.16:0/2265467474 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f38780b63b0 con 0x7f3860004da0 2024-09-10T21:10:42.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.702+0000 7f3880800640 1 -- 172.21.15.16:0/2265467474 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f38780b54c0 con 0x7f385c072610 2024-09-10T21:10:42.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.706+0000 7f3872000640 1 -- 172.21.15.16:0/2265467474 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f3868016070 con 0x7f385c072610 2024-09-10T21:10:42.713 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:42.714 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:10:42.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.710+0000 7f3880800640 1 -- 172.21.15.16:0/2265467474 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f385c072610 msgr2=0x7f385c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:42.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.710+0000 7f3880800640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f385c072610 0x7f385c074ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f38780c5670 tx=0x7f3868006b80 comp rx=0 tx=0).stop 2024-09-10T21:10:42.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.710+0000 7f3880800640 1 -- 172.21.15.16:0/2265467474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3860004da0 msgr2=0x7f3878158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.710+0000 7f3880800640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3860004da0 0x7f3878158ed0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f386c0063d0 tx=0x7f386c0039d0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.710+0000 7f3880800640 1 -- 172.21.15.16:0/2265467474 shutdown_connections 2024-09-10T21:10:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.710+0000 7f3880800640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f385c072610 0x7f385c074ad0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.710+0000 7f3880800640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3878070b50 0x7f3878070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.710+0000 7f3880800640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f38780b56d0 0x7f38781fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.710+0000 7f3880800640 1 --2- 172.21.15.16:0/2265467474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3860004da0 0x7f3878158ed0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.710+0000 7f3880800640 1 -- 172.21.15.16:0/2265467474 >> 172.21.15.16:0/2265467474 conn(0x7f387809b130 msgr2=0x7f38780b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.710+0000 7f3880800640 1 -- 172.21.15.16:0/2265467474 shutdown_connections 2024-09-10T21:10:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.710+0000 7f3880800640 1 -- 172.21.15.16:0/2265467474 wait complete. 2024-09-10T21:10:42.733 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-10T21:10:41.459095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252400,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754722320,"statfs":{"total":773094113280,"available":772835655680,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.293839"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179857+0000","last_change":"2024-09-10T21:10:30.197837+0000","last_active":"2024-09-10T21:10:31.179857+0000","last_peered":"2024-09-10T21:10:31.179857+0000","last_clean":"2024-09-10T21:10:31.179857+0000","last_became_active":"2024-09-10T21:10:30.197476+0000","last_became_peered":"2024-09-10T21:10:30.197476+0000","last_unstale":"2024-09-10T21:10:31.179857+0000","last_undegraded":"2024-09-10T21:10:31.179857+0000","last_fullsized":"2024-09-10T21:10:31.179857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:31.264063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166547+0000","last_change":"2024-09-10T21:10:30.162505+0000","last_active":"2024-09-10T21:10:31.166547+0000","last_peered":"2024-09-10T21:10:31.166547+0000","last_clean":"2024-09-10T21:10:31.166547+0000","last_became_active":"2024-09-10T21:10:30.162220+0000","last_became_peered":"2024-09-10T21:10:30.162220+0000","last_unstale":"2024-09-10T21:10:31.166547+0000","last_undegraded":"2024-09-10T21:10:31.166547+0000","last_fullsized":"2024-09-10T21:10:31.166547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:28.836761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166183+0000","last_change":"2024-09-10T21:10:30.166240+0000","last_active":"2024-09-10T21:10:31.166183+0000","last_peered":"2024-09-10T21:10:31.166183+0000","last_clean":"2024-09-10T21:10:31.166183+0000","last_became_active":"2024-09-10T21:10:30.165827+0000","last_became_peered":"2024-09-10T21:10:30.165827+0000","last_unstale":"2024-09-10T21:10:31.166183+0000","last_undegraded":"2024-09-10T21:10:31.166183+0000","last_fullsized":"2024-09-10T21:10:31.166183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:31.342353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166786+0000","last_change":"2024-09-10T21:10:30.163365+0000","last_active":"2024-09-10T21:10:31.166786+0000","last_peered":"2024-09-10T21:10:31.166786+0000","last_clean":"2024-09-10T21:10:31.166786+0000","last_became_active":"2024-09-10T21:10:30.163034+0000","last_became_peered":"2024-09-10T21:10:30.163034+0000","last_unstale":"2024-09-10T21:10:31.166786+0000","last_undegraded":"2024-09-10T21:10:31.166786+0000","last_fullsized":"2024-09-10T21:10:31.166786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:36.337660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179172+0000","last_change":"2024-09-10T21:10:30.174443+0000","last_active":"2024-09-10T21:10:31.179172+0000","last_peered":"2024-09-10T21:10:31.179172+0000","last_clean":"2024-09-10T21:10:31.179172+0000","last_became_active":"2024-09-10T21:10:30.173975+0000","last_became_peered":"2024-09-10T21:10:30.173975+0000","last_unstale":"2024-09-10T21:10:31.179172+0000","last_undegraded":"2024-09-10T21:10:31.179172+0000","last_fullsized":"2024-09-10T21:10:31.179172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:37.732853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166683+0000","last_change":"2024-09-10T21:10:30.166122+0000","last_active":"2024-09-10T21:10:31.166683+0000","last_peered":"2024-09-10T21:10:31.166683+0000","last_clean":"2024-09-10T21:10:31.166683+0000","last_became_active":"2024-09-10T21:10:30.165627+0000","last_became_peered":"2024-09-10T21:10:30.165627+0000","last_unstale":"2024-09-10T21:10:31.166683+0000","last_undegraded":"2024-09-10T21:10:31.166683+0000","last_fullsized":"2024-09-10T21:10:31.166683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:39.070890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178907+0000","last_change":"2024-09-10T21:10:30.173891+0000","last_active":"2024-09-10T21:10:31.178907+0000","last_peered":"2024-09-10T21:10:31.178907+0000","last_clean":"2024-09-10T21:10:31.178907+0000","last_became_active":"2024-09-10T21:10:30.173513+0000","last_became_peered":"2024-09-10T21:10:30.173513+0000","last_unstale":"2024-09-10T21:10:31.178907+0000","last_undegraded":"2024-09-10T21:10:31.178907+0000","last_fullsized":"2024-09-10T21:10:31.178907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:49.396588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.165773+0000","last_change":"2024-09-10T21:10:30.173739+0000","last_active":"2024-09-10T21:10:31.165773+0000","last_peered":"2024-09-10T21:10:31.165773+0000","last_clean":"2024-09-10T21:10:31.165773+0000","last_became_active":"2024-09-10T21:10:30.173401+0000","last_became_peered":"2024-09-10T21:10:30.173401+0000","last_unstale":"2024-09-10T21:10:31.165773+0000","last_undegraded":"2024-09-10T21:10:31.165773+0000","last_fullsized":"2024-09-10T21:10:31.165773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:43.846568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168511+0000","last_change":"2024-09-10T21:10:30.180534+0000","last_active":"2024-09-10T21:10:31.168511+0000","last_peered":"2024-09-10T21:10:31.168511+0000","last_clean":"2024-09-10T21:10:31.168511+0000","last_became_active":"2024-09-10T21:10:30.180196+0000","last_became_peered":"2024-09-10T21:10:30.180196+0000","last_unstale":"2024-09-10T21:10:31.168511+0000","last_undegraded":"2024-09-10T21:10:31.168511+0000","last_fullsized":"2024-09-10T21:10:31.168511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:04.038591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179548+0000","last_change":"2024-09-10T21:10:30.191472+0000","last_active":"2024-09-10T21:10:31.179548+0000","last_peered":"2024-09-10T21:10:31.179548+0000","last_clean":"2024-09-10T21:10:31.179548+0000","last_became_active":"2024-09-10T21:10:30.182657+0000","last_became_peered":"2024-09-10T21:10:30.182657+0000","last_unstale":"2024-09-10T21:10:31.179548+0000","last_undegraded":"2024-09-10T21:10:31.179548+0000","last_fullsized":"2024-09-10T21:10:31.179548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:42.385122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179302+0000","last_change":"2024-09-10T21:10:30.196473+0000","last_active":"2024-09-10T21:10:31.179302+0000","last_peered":"2024-09-10T21:10:31.179302+0000","last_clean":"2024-09-10T21:10:31.179302+0000","last_became_active":"2024-09-10T21:10:30.195712+0000","last_became_peered":"2024-09-10T21:10:30.195712+0000","last_unstale":"2024-09-10T21:10:31.179302+0000","last_undegraded":"2024-09-10T21:10:31.179302+0000","last_fullsized":"2024-09-10T21:10:31.179302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:14.592947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179252+0000","last_change":"2024-09-10T21:10:30.187870+0000","last_active":"2024-09-10T21:10:31.179252+0000","last_peered":"2024-09-10T21:10:31.179252+0000","last_clean":"2024-09-10T21:10:31.179252+0000","last_became_active":"2024-09-10T21:10:30.186791+0000","last_became_peered":"2024-09-10T21:10:30.186791+0000","last_unstale":"2024-09-10T21:10:31.179252+0000","last_undegraded":"2024-09-10T21:10:31.179252+0000","last_fullsized":"2024-09-10T21:10:31.179252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:49.609046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707773+0000","last_change":"2024-09-10T21:10:30.174269+0000","last_active":"2024-09-10T21:10:31.707773+0000","last_peered":"2024-09-10T21:10:31.707773+0000","last_clean":"2024-09-10T21:10:31.707773+0000","last_became_active":"2024-09-10T21:10:30.173714+0000","last_became_peered":"2024-09-10T21:10:30.173714+0000","last_unstale":"2024-09-10T21:10:31.707773+0000","last_undegraded":"2024-09-10T21:10:31.707773+0000","last_fullsized":"2024-09-10T21:10:31.707773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:30.207193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707685+0000","last_change":"2024-09-10T21:10:30.172867+0000","last_active":"2024-09-10T21:10:31.707685+0000","last_peered":"2024-09-10T21:10:31.707685+0000","last_clean":"2024-09-10T21:10:31.707685+0000","last_became_active":"2024-09-10T21:10:30.172301+0000","last_became_peered":"2024-09-10T21:10:30.172301+0000","last_unstale":"2024-09-10T21:10:31.707685+0000","last_undegraded":"2024-09-10T21:10:31.707685+0000","last_fullsized":"2024-09-10T21:10:31.707685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:32.205639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.165740+0000","last_change":"2024-09-10T21:10:30.181460+0000","last_active":"2024-09-10T21:10:31.165740+0000","last_peered":"2024-09-10T21:10:31.165740+0000","last_clean":"2024-09-10T21:10:31.165740+0000","last_became_active":"2024-09-10T21:10:30.175816+0000","last_became_peered":"2024-09-10T21:10:30.175816+0000","last_unstale":"2024-09-10T21:10:31.165740+0000","last_undegraded":"2024-09-10T21:10:31.165740+0000","last_fullsized":"2024-09-10T21:10:31.165740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:26.624987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708880+0000","last_change":"2024-09-10T21:10:30.185809+0000","last_active":"2024-09-10T21:10:31.708880+0000","last_peered":"2024-09-10T21:10:31.708880+0000","last_clean":"2024-09-10T21:10:31.708880+0000","last_became_active":"2024-09-10T21:10:30.185485+0000","last_became_peered":"2024-09-10T21:10:30.185485+0000","last_unstale":"2024-09-10T21:10:31.708880+0000","last_undegraded":"2024-09-10T21:10:31.708880+0000","last_fullsized":"2024-09-10T21:10:31.708880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:16.358565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709494+0000","last_change":"2024-09-10T21:10:30.196400+0000","last_active":"2024-09-10T21:10:31.709494+0000","last_peered":"2024-09-10T21:10:31.709494+0000","last_clean":"2024-09-10T21:10:31.709494+0000","last_became_active":"2024-09-10T21:10:30.194199+0000","last_became_peered":"2024-09-10T21:10:30.194199+0000","last_unstale":"2024-09-10T21:10:31.709494+0000","last_undegraded":"2024-09-10T21:10:31.709494+0000","last_fullsized":"2024-09-10T21:10:31.709494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:21.510129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446465+0000","last_change":"2024-09-10T21:10:30.169268+0000","last_active":"2024-09-10T21:10:31.446465+0000","last_peered":"2024-09-10T21:10:31.446465+0000","last_clean":"2024-09-10T21:10:31.446465+0000","last_became_active":"2024-09-10T21:10:30.168869+0000","last_became_peered":"2024-09-10T21:10:30.168869+0000","last_unstale":"2024-09-10T21:10:31.446465+0000","last_undegraded":"2024-09-10T21:10:31.446465+0000","last_fullsized":"2024-09-10T21:10:31.446465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:06.070406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709259+0000","last_change":"2024-09-10T21:10:30.180496+0000","last_active":"2024-09-10T21:10:31.709259+0000","last_peered":"2024-09-10T21:10:31.709259+0000","last_clean":"2024-09-10T21:10:31.709259+0000","last_became_active":"2024-09-10T21:10:30.180199+0000","last_became_peered":"2024-09-10T21:10:30.180199+0000","last_unstale":"2024-09-10T21:10:31.709259+0000","last_undegraded":"2024-09-10T21:10:31.709259+0000","last_fullsized":"2024-09-10T21:10:31.709259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:36.737204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179263+0000","last_change":"2024-09-10T21:10:30.187710+0000","last_active":"2024-09-10T21:10:31.179263+0000","last_peered":"2024-09-10T21:10:31.179263+0000","last_clean":"2024-09-10T21:10:31.179263+0000","last_became_active":"2024-09-10T21:10:30.186422+0000","last_became_peered":"2024-09-10T21:10:30.186422+0000","last_unstale":"2024-09-10T21:10:31.179263+0000","last_undegraded":"2024-09-10T21:10:31.179263+0000","last_fullsized":"2024-09-10T21:10:31.179263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:37.171374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173731+0000","last_change":"2024-09-10T21:10:30.157845+0000","last_active":"2024-09-10T21:10:31.173731+0000","last_peered":"2024-09-10T21:10:31.173731+0000","last_clean":"2024-09-10T21:10:31.173731+0000","last_became_active":"2024-09-10T21:10:30.155515+0000","last_became_peered":"2024-09-10T21:10:30.155515+0000","last_unstale":"2024-09-10T21:10:31.173731+0000","last_undegraded":"2024-09-10T21:10:31.173731+0000","last_fullsized":"2024-09-10T21:10:31.173731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:36.354773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167488+0000","last_change":"2024-09-10T21:10:30.168344+0000","last_active":"2024-09-10T21:10:31.167488+0000","last_peered":"2024-09-10T21:10:31.167488+0000","last_clean":"2024-09-10T21:10:31.167488+0000","last_became_active":"2024-09-10T21:10:30.167966+0000","last_became_peered":"2024-09-10T21:10:30.167966+0000","last_unstale":"2024-09-10T21:10:31.167488+0000","last_undegraded":"2024-09-10T21:10:31.167488+0000","last_fullsized":"2024-09-10T21:10:31.167488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:38.100207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166637+0000","last_change":"2024-09-10T21:10:30.159521+0000","last_active":"2024-09-10T21:10:31.166637+0000","last_peered":"2024-09-10T21:10:31.166637+0000","last_clean":"2024-09-10T21:10:31.166637+0000","last_became_active":"2024-09-10T21:10:30.159269+0000","last_became_peered":"2024-09-10T21:10:30.159269+0000","last_unstale":"2024-09-10T21:10:31.166637+0000","last_undegraded":"2024-09-10T21:10:31.166637+0000","last_fullsized":"2024-09-10T21:10:31.166637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:21.433157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708679+0000","last_change":"2024-09-10T21:10:30.182227+0000","last_active":"2024-09-10T21:10:31.708679+0000","last_peered":"2024-09-10T21:10:31.708679+0000","last_clean":"2024-09-10T21:10:31.708679+0000","last_became_active":"2024-09-10T21:10:30.178359+0000","last_became_peered":"2024-09-10T21:10:30.178359+0000","last_unstale":"2024-09-10T21:10:31.708679+0000","last_undegraded":"2024-09-10T21:10:31.708679+0000","last_fullsized":"2024-09-10T21:10:31.708679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:50.308163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166494+0000","last_change":"2024-09-10T21:10:30.170127+0000","last_active":"2024-09-10T21:10:31.166494+0000","last_peered":"2024-09-10T21:10:31.166494+0000","last_clean":"2024-09-10T21:10:31.166494+0000","last_became_active":"2024-09-10T21:10:30.169781+0000","last_became_peered":"2024-09-10T21:10:30.169781+0000","last_unstale":"2024-09-10T21:10:31.166494+0000","last_undegraded":"2024-09-10T21:10:31.166494+0000","last_fullsized":"2024-09-10T21:10:31.166494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:16.729840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167717+0000","last_change":"2024-09-10T21:10:30.161842+0000","last_active":"2024-09-10T21:10:31.167717+0000","last_peered":"2024-09-10T21:10:31.167717+0000","last_clean":"2024-09-10T21:10:31.167717+0000","last_became_active":"2024-09-10T21:10:30.160146+0000","last_became_peered":"2024-09-10T21:10:30.160146+0000","last_unstale":"2024-09-10T21:10:31.167717+0000","last_undegraded":"2024-09-10T21:10:31.167717+0000","last_fullsized":"2024-09-10T21:10:31.167717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:06.600402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707965+0000","last_change":"2024-09-10T21:10:30.176930+0000","last_active":"2024-09-10T21:10:31.707965+0000","last_peered":"2024-09-10T21:10:31.707965+0000","last_clean":"2024-09-10T21:10:31.707965+0000","last_became_active":"2024-09-10T21:10:30.175522+0000","last_became_peered":"2024-09-10T21:10:30.175522+0000","last_unstale":"2024-09-10T21:10:31.707965+0000","last_undegraded":"2024-09-10T21:10:31.707965+0000","last_fullsized":"2024-09-10T21:10:31.707965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:36.354773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707870+0000","last_change":"2024-09-10T21:10:30.179884+0000","last_active":"2024-09-10T21:10:31.707870+0000","last_peered":"2024-09-10T21:10:31.707870+0000","last_clean":"2024-09-10T21:10:31.707870+0000","last_became_active":"2024-09-10T21:10:30.177866+0000","last_became_peered":"2024-09-10T21:10:30.177866+0000","last_unstale":"2024-09-10T21:10:31.707870+0000","last_undegraded":"2024-09-10T21:10:31.707870+0000","last_fullsized":"2024-09-10T21:10:31.707870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:57.957611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166067+0000","last_change":"2024-09-10T21:10:30.169322+0000","last_active":"2024-09-10T21:10:31.166067+0000","last_peered":"2024-09-10T21:10:31.166067+0000","last_clean":"2024-09-10T21:10:31.166067+0000","last_became_active":"2024-09-10T21:10:30.169030+0000","last_became_peered":"2024-09-10T21:10:30.169030+0000","last_unstale":"2024-09-10T21:10:31.166067+0000","last_undegraded":"2024-09-10T21:10:31.166067+0000","last_fullsized":"2024-09-10T21:10:31.166067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:41.356402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708760+0000","last_change":"2024-09-10T21:10:30.194536+0000","last_active":"2024-09-10T21:10:31.708760+0000","last_peered":"2024-09-10T21:10:31.708760+0000","last_clean":"2024-09-10T21:10:31.708760+0000","last_became_active":"2024-09-10T21:10:30.188848+0000","last_became_peered":"2024-09-10T21:10:30.188848+0000","last_unstale":"2024-09-10T21:10:31.708760+0000","last_undegraded":"2024-09-10T21:10:31.708760+0000","last_fullsized":"2024-09-10T21:10:31.708760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:31.264063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446249+0000","last_change":"2024-09-10T21:10:30.171833+0000","last_active":"2024-09-10T21:10:31.446249+0000","last_peered":"2024-09-10T21:10:31.446249+0000","last_clean":"2024-09-10T21:10:31.446249+0000","last_became_active":"2024-09-10T21:10:30.171461+0000","last_became_peered":"2024-09-10T21:10:30.171461+0000","last_unstale":"2024-09-10T21:10:31.446249+0000","last_undegraded":"2024-09-10T21:10:31.446249+0000","last_fullsized":"2024-09-10T21:10:31.446249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:32.018476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166344+0000","last_change":"2024-09-10T21:10:30.183415+0000","last_active":"2024-09-10T21:10:31.166344+0000","last_peered":"2024-09-10T21:10:31.166344+0000","last_clean":"2024-09-10T21:10:31.166344+0000","last_became_active":"2024-09-10T21:10:30.183093+0000","last_became_peered":"2024-09-10T21:10:30.183093+0000","last_unstale":"2024-09-10T21:10:31.166344+0000","last_undegraded":"2024-09-10T21:10:31.166344+0000","last_fullsized":"2024-09-10T21:10:31.166344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:30.558298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167929+0000","last_change":"2024-09-10T21:10:30.171980+0000","last_active":"2024-09-10T21:10:31.167929+0000","last_peered":"2024-09-10T21:10:31.167929+0000","last_clean":"2024-09-10T21:10:31.167929+0000","last_became_active":"2024-09-10T21:10:30.167136+0000","last_became_peered":"2024-09-10T21:10:30.167136+0000","last_unstale":"2024-09-10T21:10:31.167929+0000","last_undegraded":"2024-09-10T21:10:31.167929+0000","last_fullsized":"2024-09-10T21:10:31.167929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:33.142099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166187+0000","last_change":"2024-09-10T21:10:29.171062+0000","last_active":"2024-09-10T21:10:31.166187+0000","last_peered":"2024-09-10T21:10:31.166187+0000","last_clean":"2024-09-10T21:10:31.166187+0000","last_became_active":"2024-09-10T21:10:29.170765+0000","last_became_peered":"2024-09-10T21:10:29.170765+0000","last_unstale":"2024-09-10T21:10:31.166187+0000","last_undegraded":"2024-09-10T21:10:31.166187+0000","last_fullsized":"2024-09-10T21:10:31.166187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.425174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173821+0000","last_change":"2024-09-10T21:10:30.174660+0000","last_active":"2024-09-10T21:10:31.173821+0000","last_peered":"2024-09-10T21:10:31.173821+0000","last_clean":"2024-09-10T21:10:31.173821+0000","last_became_active":"2024-09-10T21:10:30.174372+0000","last_became_peered":"2024-09-10T21:10:30.174372+0000","last_unstale":"2024-09-10T21:10:31.173821+0000","last_undegraded":"2024-09-10T21:10:31.173821+0000","last_fullsized":"2024-09-10T21:10:31.173821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:59.722000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708981+0000","last_change":"2024-09-10T21:10:29.171541+0000","last_active":"2024-09-10T21:10:31.708981+0000","last_peered":"2024-09-10T21:10:31.708981+0000","last_clean":"2024-09-10T21:10:31.708981+0000","last_became_active":"2024-09-10T21:10:29.171269+0000","last_became_peered":"2024-09-10T21:10:29.171269+0000","last_unstale":"2024-09-10T21:10:31.708981+0000","last_undegraded":"2024-09-10T21:10:31.708981+0000","last_fullsized":"2024-09-10T21:10:31.708981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:10.046262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707383+0000","last_change":"2024-09-10T21:10:29.166482+0000","last_active":"2024-09-10T21:10:31.707383+0000","last_peered":"2024-09-10T21:10:31.707383+0000","last_clean":"2024-09-10T21:10:31.707383+0000","last_became_active":"2024-09-10T21:10:29.166091+0000","last_became_peered":"2024-09-10T21:10:29.166091+0000","last_unstale":"2024-09-10T21:10:31.707383+0000","last_undegraded":"2024-09-10T21:10:31.707383+0000","last_fullsized":"2024-09-10T21:10:31.707383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.425174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166297+0000","last_change":"2024-09-10T21:10:30.183180+0000","last_active":"2024-09-10T21:10:31.166297+0000","last_peered":"2024-09-10T21:10:31.166297+0000","last_clean":"2024-09-10T21:10:31.166297+0000","last_became_active":"2024-09-10T21:10:30.182908+0000","last_became_peered":"2024-09-10T21:10:30.182908+0000","last_unstale":"2024-09-10T21:10:31.166297+0000","last_undegraded":"2024-09-10T21:10:31.166297+0000","last_fullsized":"2024-09-10T21:10:31.166297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:37.697114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708748+0000","last_change":"2024-09-10T21:10:29.167898+0000","last_active":"2024-09-10T21:10:31.708748+0000","last_peered":"2024-09-10T21:10:31.708748+0000","last_clean":"2024-09-10T21:10:31.708748+0000","last_became_active":"2024-09-10T21:10:29.167498+0000","last_became_peered":"2024-09-10T21:10:29.167498+0000","last_unstale":"2024-09-10T21:10:31.708748+0000","last_undegraded":"2024-09-10T21:10:31.708748+0000","last_fullsized":"2024-09-10T21:10:31.708748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:50.337610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173782+0000","last_change":"2024-09-10T21:10:30.161329+0000","last_active":"2024-09-10T21:10:31.173782+0000","last_peered":"2024-09-10T21:10:31.173782+0000","last_clean":"2024-09-10T21:10:31.173782+0000","last_became_active":"2024-09-10T21:10:30.160309+0000","last_became_peered":"2024-09-10T21:10:30.160309+0000","last_unstale":"2024-09-10T21:10:31.173782+0000","last_undegraded":"2024-09-10T21:10:31.173782+0000","last_fullsized":"2024-09-10T21:10:31.173782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:14.235339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.168376+0000","last_change":"2024-09-10T21:10:29.170975+0000","last_active":"2024-09-10T21:10:31.168376+0000","last_peered":"2024-09-10T21:10:31.168376+0000","last_clean":"2024-09-10T21:10:31.168376+0000","last_became_active":"2024-09-10T21:10:29.170623+0000","last_became_peered":"2024-09-10T21:10:29.170623+0000","last_unstale":"2024-09-10T21:10:31.168376+0000","last_undegraded":"2024-09-10T21:10:31.168376+0000","last_fullsized":"2024-09-10T21:10:31.168376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167790+0000","last_change":"2024-09-10T21:10:30.160525+0000","last_active":"2024-09-10T21:10:31.167790+0000","last_peered":"2024-09-10T21:10:31.167790+0000","last_clean":"2024-09-10T21:10:31.167790+0000","last_became_active":"2024-09-10T21:10:30.159639+0000","last_became_peered":"2024-09-10T21:10:30.159639+0000","last_unstale":"2024-09-10T21:10:31.167790+0000","last_undegraded":"2024-09-10T21:10:31.167790+0000","last_fullsized":"2024-09-10T21:10:31.167790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:46.116954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179609+0000","last_change":"2024-09-10T21:10:29.175134+0000","last_active":"2024-09-10T21:10:31.179609+0000","last_peered":"2024-09-10T21:10:31.179609+0000","last_clean":"2024-09-10T21:10:31.179609+0000","last_became_active":"2024-09-10T21:10:29.174808+0000","last_became_peered":"2024-09-10T21:10:29.174808+0000","last_unstale":"2024-09-10T21:10:31.179609+0000","last_undegraded":"2024-09-10T21:10:31.179609+0000","last_fullsized":"2024-09-10T21:10:31.179609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:52.586551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166756+0000","last_change":"2024-09-10T21:10:30.167445+0000","last_active":"2024-09-10T21:10:31.166756+0000","last_peered":"2024-09-10T21:10:31.166756+0000","last_clean":"2024-09-10T21:10:31.166756+0000","last_became_active":"2024-09-10T21:10:30.167163+0000","last_became_peered":"2024-09-10T21:10:30.167163+0000","last_unstale":"2024-09-10T21:10:31.166756+0000","last_undegraded":"2024-09-10T21:10:31.166756+0000","last_fullsized":"2024-09-10T21:10:31.166756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:59.291659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446376+0000","last_change":"2024-09-10T21:10:29.171850+0000","last_active":"2024-09-10T21:10:31.446376+0000","last_peered":"2024-09-10T21:10:31.446376+0000","last_clean":"2024-09-10T21:10:31.446376+0000","last_became_active":"2024-09-10T21:10:29.171589+0000","last_became_peered":"2024-09-10T21:10:29.171589+0000","last_unstale":"2024-09-10T21:10:31.446376+0000","last_undegraded":"2024-09-10T21:10:31.446376+0000","last_fullsized":"2024-09-10T21:10:31.446376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:43.143812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708938+0000","last_change":"2024-09-10T21:10:30.162228+0000","last_active":"2024-09-10T21:10:31.708938+0000","last_peered":"2024-09-10T21:10:31.708938+0000","last_clean":"2024-09-10T21:10:31.708938+0000","last_became_active":"2024-09-10T21:10:30.161901+0000","last_became_peered":"2024-09-10T21:10:30.161901+0000","last_unstale":"2024-09-10T21:10:31.708938+0000","last_undegraded":"2024-09-10T21:10:31.708938+0000","last_fullsized":"2024-09-10T21:10:31.708938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:12.489819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167129+0000","last_change":"2024-09-10T21:10:29.164653+0000","last_active":"2024-09-10T21:10:31.167129+0000","last_peered":"2024-09-10T21:10:31.167129+0000","last_clean":"2024-09-10T21:10:31.167129+0000","last_became_active":"2024-09-10T21:10:29.164338+0000","last_became_peered":"2024-09-10T21:10:29.164338+0000","last_unstale":"2024-09-10T21:10:31.167129+0000","last_undegraded":"2024-09-10T21:10:31.167129+0000","last_fullsized":"2024-09-10T21:10:31.167129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:24.324180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179343+0000","last_change":"2024-09-10T21:10:30.191023+0000","last_active":"2024-09-10T21:10:31.179343+0000","last_peered":"2024-09-10T21:10:31.179343+0000","last_clean":"2024-09-10T21:10:31.179343+0000","last_became_active":"2024-09-10T21:10:30.190264+0000","last_became_peered":"2024-09-10T21:10:30.190264+0000","last_unstale":"2024-09-10T21:10:31.179343+0000","last_undegraded":"2024-09-10T21:10:31.179343+0000","last_fullsized":"2024-09-10T21:10:31.179343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:08.381083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446251+0000","last_change":"2024-09-10T21:10:29.178312+0000","last_active":"2024-09-10T21:10:31.446251+0000","last_peered":"2024-09-10T21:10:31.446251+0000","last_clean":"2024-09-10T21:10:31.446251+0000","last_became_active":"2024-09-10T21:10:29.178044+0000","last_became_peered":"2024-09-10T21:10:29.178044+0000","last_unstale":"2024-09-10T21:10:31.446251+0000","last_undegraded":"2024-09-10T21:10:31.446251+0000","last_fullsized":"2024-09-10T21:10:31.446251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:41.871122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707682+0000","last_change":"2024-09-10T21:10:30.181507+0000","last_active":"2024-09-10T21:10:31.707682+0000","last_peered":"2024-09-10T21:10:31.707682+0000","last_clean":"2024-09-10T21:10:31.707682+0000","last_became_active":"2024-09-10T21:10:30.177520+0000","last_became_peered":"2024-09-10T21:10:30.177520+0000","last_unstale":"2024-09-10T21:10:31.707682+0000","last_undegraded":"2024-09-10T21:10:31.707682+0000","last_fullsized":"2024-09-10T21:10:31.707682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:56.567997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709035+0000","last_change":"2024-09-10T21:10:29.166737+0000","last_active":"2024-09-10T21:10:31.709035+0000","last_peered":"2024-09-10T21:10:31.709035+0000","last_clean":"2024-09-10T21:10:31.709035+0000","last_became_active":"2024-09-10T21:10:29.166301+0000","last_became_peered":"2024-09-10T21:10:29.166301+0000","last_unstale":"2024-09-10T21:10:31.709035+0000","last_undegraded":"2024-09-10T21:10:31.709035+0000","last_fullsized":"2024-09-10T21:10:31.709035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:52.586551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709252+0000","last_change":"2024-09-10T21:10:30.180470+0000","last_active":"2024-09-10T21:10:31.709252+0000","last_peered":"2024-09-10T21:10:31.709252+0000","last_clean":"2024-09-10T21:10:31.709252+0000","last_became_active":"2024-09-10T21:10:30.179447+0000","last_became_peered":"2024-09-10T21:10:30.179447+0000","last_unstale":"2024-09-10T21:10:31.709252+0000","last_undegraded":"2024-09-10T21:10:31.709252+0000","last_fullsized":"2024-09-10T21:10:31.709252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:36.701958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178781+0000","last_change":"2024-09-10T21:10:29.169914+0000","last_active":"2024-09-10T21:10:31.178781+0000","last_peered":"2024-09-10T21:10:31.178781+0000","last_clean":"2024-09-10T21:10:31.178781+0000","last_became_active":"2024-09-10T21:10:29.169617+0000","last_became_peered":"2024-09-10T21:10:29.169617+0000","last_unstale":"2024-09-10T21:10:31.178781+0000","last_undegraded":"2024-09-10T21:10:31.178781+0000","last_fullsized":"2024-09-10T21:10:31.178781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:48.286592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167103+0000","last_change":"2024-09-10T21:10:30.169299+0000","last_active":"2024-09-10T21:10:31.167103+0000","last_peered":"2024-09-10T21:10:31.167103+0000","last_clean":"2024-09-10T21:10:31.167103+0000","last_became_active":"2024-09-10T21:10:30.169013+0000","last_became_peered":"2024-09-10T21:10:30.169013+0000","last_unstale":"2024-09-10T21:10:31.167103+0000","last_undegraded":"2024-09-10T21:10:31.167103+0000","last_fullsized":"2024-09-10T21:10:31.167103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:40.546116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173457+0000","last_change":"2024-09-10T21:10:29.162746+0000","last_active":"2024-09-10T21:10:31.173457+0000","last_peered":"2024-09-10T21:10:31.173457+0000","last_clean":"2024-09-10T21:10:31.173457+0000","last_became_active":"2024-09-10T21:10:29.162466+0000","last_became_peered":"2024-09-10T21:10:29.162466+0000","last_unstale":"2024-09-10T21:10:31.173457+0000","last_undegraded":"2024-09-10T21:10:31.173457+0000","last_fullsized":"2024-09-10T21:10:31.173457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.425174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708924+0000","last_change":"2024-09-10T21:10:30.162565+0000","last_active":"2024-09-10T21:10:31.708924+0000","last_peered":"2024-09-10T21:10:31.708924+0000","last_clean":"2024-09-10T21:10:31.708924+0000","last_became_active":"2024-09-10T21:10:30.161927+0000","last_became_peered":"2024-09-10T21:10:30.161927+0000","last_unstale":"2024-09-10T21:10:31.708924+0000","last_undegraded":"2024-09-10T21:10:31.708924+0000","last_fullsized":"2024-09-10T21:10:31.708924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:43.633279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.180274+0000","last_change":"2024-09-10T21:10:29.170318+0000","last_active":"2024-09-10T21:10:31.180274+0000","last_peered":"2024-09-10T21:10:31.180274+0000","last_clean":"2024-09-10T21:10:31.180274+0000","last_became_active":"2024-09-10T21:10:29.169096+0000","last_became_peered":"2024-09-10T21:10:29.169096+0000","last_unstale":"2024-09-10T21:10:31.180274+0000","last_undegraded":"2024-09-10T21:10:31.180274+0000","last_fullsized":"2024-09-10T21:10:31.180274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:01.835985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167829+0000","last_change":"2024-09-10T21:10:30.176716+0000","last_active":"2024-09-10T21:10:31.167829+0000","last_peered":"2024-09-10T21:10:31.167829+0000","last_clean":"2024-09-10T21:10:31.167829+0000","last_became_active":"2024-09-10T21:10:30.175255+0000","last_became_peered":"2024-09-10T21:10:30.175255+0000","last_unstale":"2024-09-10T21:10:31.167829+0000","last_undegraded":"2024-09-10T21:10:31.167829+0000","last_fullsized":"2024-09-10T21:10:31.167829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.859164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173973+0000","last_change":"2024-09-10T21:10:29.163718+0000","last_active":"2024-09-10T21:10:31.173973+0000","last_peered":"2024-09-10T21:10:31.173973+0000","last_clean":"2024-09-10T21:10:31.173973+0000","last_became_active":"2024-09-10T21:10:29.163328+0000","last_became_peered":"2024-09-10T21:10:29.163328+0000","last_unstale":"2024-09-10T21:10:31.173973+0000","last_undegraded":"2024-09-10T21:10:31.173973+0000","last_fullsized":"2024-09-10T21:10:31.173973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:57.741500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.180052+0000","last_change":"2024-09-10T21:10:30.183206+0000","last_active":"2024-09-10T21:10:31.180052+0000","last_peered":"2024-09-10T21:10:31.180052+0000","last_clean":"2024-09-10T21:10:31.180052+0000","last_became_active":"2024-09-10T21:10:30.182950+0000","last_became_peered":"2024-09-10T21:10:30.182950+0000","last_unstale":"2024-09-10T21:10:31.180052+0000","last_undegraded":"2024-09-10T21:10:31.180052+0000","last_fullsized":"2024-09-10T21:10:31.180052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:45.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166843+0000","last_change":"2024-09-10T21:10:29.165292+0000","last_active":"2024-09-10T21:10:31.166843+0000","last_peered":"2024-09-10T21:10:31.166843+0000","last_clean":"2024-09-10T21:10:31.166843+0000","last_became_active":"2024-09-10T21:10:29.164991+0000","last_became_peered":"2024-09-10T21:10:29.164991+0000","last_unstale":"2024-09-10T21:10:31.166843+0000","last_undegraded":"2024-09-10T21:10:31.166843+0000","last_fullsized":"2024-09-10T21:10:31.166843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:57.741500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179462+0000","last_change":"2024-09-10T21:10:30.188067+0000","last_active":"2024-09-10T21:10:31.179462+0000","last_peered":"2024-09-10T21:10:31.179462+0000","last_clean":"2024-09-10T21:10:31.179462+0000","last_became_active":"2024-09-10T21:10:30.187079+0000","last_became_peered":"2024-09-10T21:10:30.187079+0000","last_unstale":"2024-09-10T21:10:31.179462+0000","last_undegraded":"2024-09-10T21:10:31.179462+0000","last_fullsized":"2024-09-10T21:10:31.179462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:16.358565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707972+0000","last_change":"2024-09-10T21:10:29.168326+0000","last_active":"2024-09-10T21:10:31.707972+0000","last_peered":"2024-09-10T21:10:31.707972+0000","last_clean":"2024-09-10T21:10:31.707972+0000","last_became_active":"2024-09-10T21:10:29.168048+0000","last_became_peered":"2024-09-10T21:10:29.168048+0000","last_unstale":"2024-09-10T21:10:31.707972+0000","last_undegraded":"2024-09-10T21:10:31.707972+0000","last_fullsized":"2024-09-10T21:10:31.707972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:56.871372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709184+0000","last_change":"2024-09-10T21:10:30.179705+0000","last_active":"2024-09-10T21:10:31.709184+0000","last_peered":"2024-09-10T21:10:31.709184+0000","last_clean":"2024-09-10T21:10:31.709184+0000","last_became_active":"2024-09-10T21:10:30.175366+0000","last_became_peered":"2024-09-10T21:10:30.175366+0000","last_unstale":"2024-09-10T21:10:31.709184+0000","last_undegraded":"2024-09-10T21:10:31.709184+0000","last_fullsized":"2024-09-10T21:10:31.709184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:44.960366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167202+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:10:31.167202+0000","last_peered":"2024-09-10T21:10:31.167202+0000","last_clean":"2024-09-10T21:10:31.167202+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:10:31.167202+0000","last_undegraded":"2024-09-10T21:10:31.167202+0000","last_fullsized":"2024-09-10T21:10:31.167202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708999+0000","last_change":"2024-09-10T21:10:29.172283+0000","last_active":"2024-09-10T21:10:31.708999+0000","last_peered":"2024-09-10T21:10:31.708999+0000","last_clean":"2024-09-10T21:10:31.708999+0000","last_became_active":"2024-09-10T21:10:29.170550+0000","last_became_peered":"2024-09-10T21:10:29.170550+0000","last_unstale":"2024-09-10T21:10:31.708999+0000","last_undegraded":"2024-09-10T21:10:31.708999+0000","last_fullsized":"2024-09-10T21:10:31.708999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:09.439008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173734+0000","last_change":"2024-09-10T21:10:30.160275+0000","last_active":"2024-09-10T21:10:31.173734+0000","last_peered":"2024-09-10T21:10:31.173734+0000","last_clean":"2024-09-10T21:10:31.173734+0000","last_became_active":"2024-09-10T21:10:30.159817+0000","last_became_peered":"2024-09-10T21:10:30.159817+0000","last_unstale":"2024-09-10T21:10:31.173734+0000","last_undegraded":"2024-09-10T21:10:31.173734+0000","last_fullsized":"2024-09-10T21:10:31.173734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:26.698693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708902+0000","last_change":"2024-09-10T21:10:29.164034+0000","last_active":"2024-09-10T21:10:31.708902+0000","last_peered":"2024-09-10T21:10:31.708902+0000","last_clean":"2024-09-10T21:10:31.708902+0000","last_became_active":"2024-09-10T21:10:29.163755+0000","last_became_peered":"2024-09-10T21:10:29.163755+0000","last_unstale":"2024-09-10T21:10:31.708902+0000","last_undegraded":"2024-09-10T21:10:31.708902+0000","last_fullsized":"2024-09-10T21:10:31.708902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:10.200363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707745+0000","last_change":"2024-09-10T21:10:30.171490+0000","last_active":"2024-09-10T21:10:31.707745+0000","last_peered":"2024-09-10T21:10:31.707745+0000","last_clean":"2024-09-10T21:10:31.707745+0000","last_became_active":"2024-09-10T21:10:30.170892+0000","last_became_peered":"2024-09-10T21:10:30.170892+0000","last_unstale":"2024-09-10T21:10:31.707745+0000","last_undegraded":"2024-09-10T21:10:31.707745+0000","last_fullsized":"2024-09-10T21:10:31.707745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:36.425140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446253+0000","last_change":"2024-09-10T21:10:29.170636+0000","last_active":"2024-09-10T21:10:31.446253+0000","last_peered":"2024-09-10T21:10:31.446253+0000","last_clean":"2024-09-10T21:10:31.446253+0000","last_became_active":"2024-09-10T21:10:29.170342+0000","last_became_peered":"2024-09-10T21:10:29.170342+0000","last_unstale":"2024-09-10T21:10:31.446253+0000","last_undegraded":"2024-09-10T21:10:31.446253+0000","last_fullsized":"2024-09-10T21:10:31.446253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.186850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166047+0000","last_change":"2024-09-10T21:10:30.164807+0000","last_active":"2024-09-10T21:10:31.166047+0000","last_peered":"2024-09-10T21:10:31.166047+0000","last_clean":"2024-09-10T21:10:31.166047+0000","last_became_active":"2024-09-10T21:10:30.164545+0000","last_became_peered":"2024-09-10T21:10:30.164545+0000","last_unstale":"2024-09-10T21:10:31.166047+0000","last_undegraded":"2024-09-10T21:10:31.166047+0000","last_fullsized":"2024-09-10T21:10:31.166047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:35.647295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709113+0000","last_change":"2024-09-10T21:10:29.173240+0000","last_active":"2024-09-10T21:10:31.709113+0000","last_peered":"2024-09-10T21:10:31.709113+0000","last_clean":"2024-09-10T21:10:31.709113+0000","last_became_active":"2024-09-10T21:10:29.172991+0000","last_became_peered":"2024-09-10T21:10:29.172991+0000","last_unstale":"2024-09-10T21:10:31.709113+0000","last_undegraded":"2024-09-10T21:10:31.709113+0000","last_fullsized":"2024-09-10T21:10:31.709113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:00.278081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178965+0000","last_change":"2024-09-10T21:10:30.189010+0000","last_active":"2024-09-10T21:10:31.178965+0000","last_peered":"2024-09-10T21:10:31.178965+0000","last_clean":"2024-09-10T21:10:31.178965+0000","last_became_active":"2024-09-10T21:10:30.188712+0000","last_became_peered":"2024-09-10T21:10:30.188712+0000","last_unstale":"2024-09-10T21:10:31.178965+0000","last_undegraded":"2024-09-10T21:10:31.178965+0000","last_fullsized":"2024-09-10T21:10:31.178965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:04.853111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.165941+0000","last_change":"2024-09-10T21:10:29.173419+0000","last_active":"2024-09-10T21:10:31.165941+0000","last_peered":"2024-09-10T21:10:31.165941+0000","last_clean":"2024-09-10T21:10:31.165941+0000","last_became_active":"2024-09-10T21:10:29.173104+0000","last_became_peered":"2024-09-10T21:10:29.173104+0000","last_unstale":"2024-09-10T21:10:31.165941+0000","last_undegraded":"2024-09-10T21:10:31.165941+0000","last_fullsized":"2024-09-10T21:10:31.165941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:25.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167265+0000","last_change":"2024-09-10T21:10:30.169171+0000","last_active":"2024-09-10T21:10:31.167265+0000","last_peered":"2024-09-10T21:10:31.167265+0000","last_clean":"2024-09-10T21:10:31.167265+0000","last_became_active":"2024-09-10T21:10:30.168866+0000","last_became_peered":"2024-09-10T21:10:30.168866+0000","last_unstale":"2024-09-10T21:10:31.167265+0000","last_undegraded":"2024-09-10T21:10:31.167265+0000","last_fullsized":"2024-09-10T21:10:31.167265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:45.633966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709233+0000","last_change":"2024-09-10T21:10:29.170135+0000","last_active":"2024-09-10T21:10:31.709233+0000","last_peered":"2024-09-10T21:10:31.709233+0000","last_clean":"2024-09-10T21:10:31.709233+0000","last_became_active":"2024-09-10T21:10:29.169894+0000","last_became_peered":"2024-09-10T21:10:29.169894+0000","last_unstale":"2024-09-10T21:10:31.709233+0000","last_undegraded":"2024-09-10T21:10:31.709233+0000","last_fullsized":"2024-09-10T21:10:31.709233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:10.057351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707993+0000","last_change":"2024-09-10T21:10:30.171973+0000","last_active":"2024-09-10T21:10:31.707993+0000","last_peered":"2024-09-10T21:10:31.707993+0000","last_clean":"2024-09-10T21:10:31.707993+0000","last_became_active":"2024-09-10T21:10:30.171622+0000","last_became_peered":"2024-09-10T21:10:30.171622+0000","last_unstale":"2024-09-10T21:10:31.707993+0000","last_undegraded":"2024-09-10T21:10:31.707993+0000","last_fullsized":"2024-09-10T21:10:31.707993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:06.330104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174157+0000","last_change":"2024-09-10T21:10:29.165107+0000","last_active":"2024-09-10T21:10:31.174157+0000","last_peered":"2024-09-10T21:10:31.174157+0000","last_clean":"2024-09-10T21:10:31.174157+0000","last_became_active":"2024-09-10T21:10:29.164798+0000","last_became_peered":"2024-09-10T21:10:29.164798+0000","last_unstale":"2024-09-10T21:10:31.174157+0000","last_undegraded":"2024-09-10T21:10:31.174157+0000","last_fullsized":"2024-09-10T21:10:31.174157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:04.055586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174286+0000","last_change":"2024-09-10T21:10:30.173086+0000","last_active":"2024-09-10T21:10:31.174286+0000","last_peered":"2024-09-10T21:10:31.174286+0000","last_clean":"2024-09-10T21:10:31.174286+0000","last_became_active":"2024-09-10T21:10:30.172730+0000","last_became_peered":"2024-09-10T21:10:30.172730+0000","last_unstale":"2024-09-10T21:10:31.174286+0000","last_undegraded":"2024-09-10T21:10:31.174286+0000","last_fullsized":"2024-09-10T21:10:31.174286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:50.512074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709420+0000","last_change":"2024-09-10T21:10:29.179231+0000","last_active":"2024-09-10T21:10:31.709420+0000","last_peered":"2024-09-10T21:10:31.709420+0000","last_clean":"2024-09-10T21:10:31.709420+0000","last_became_active":"2024-09-10T21:10:29.178937+0000","last_became_peered":"2024-09-10T21:10:29.178937+0000","last_unstale":"2024-09-10T21:10:31.709420+0000","last_undegraded":"2024-09-10T21:10:31.709420+0000","last_fullsized":"2024-09-10T21:10:31.709420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:41.528182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174168+0000","last_change":"2024-09-10T21:10:30.167226+0000","last_active":"2024-09-10T21:10:31.174168+0000","last_peered":"2024-09-10T21:10:31.174168+0000","last_clean":"2024-09-10T21:10:31.174168+0000","last_became_active":"2024-09-10T21:10:30.166959+0000","last_became_peered":"2024-09-10T21:10:30.166959+0000","last_unstale":"2024-09-10T21:10:31.174168+0000","last_undegraded":"2024-09-10T21:10:31.174168+0000","last_fullsized":"2024-09-10T21:10:31.174168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:43.225225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.178611+0000","last_change":"2024-09-10T21:10:29.175872+0000","last_active":"2024-09-10T21:10:31.178611+0000","last_peered":"2024-09-10T21:10:31.178611+0000","last_clean":"2024-09-10T21:10:31.178611+0000","last_became_active":"2024-09-10T21:10:29.175659+0000","last_became_peered":"2024-09-10T21:10:29.175659+0000","last_unstale":"2024-09-10T21:10:31.178611+0000","last_undegraded":"2024-09-10T21:10:31.178611+0000","last_fullsized":"2024-09-10T21:10:31.178611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:21.990689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708884+0000","last_change":"2024-09-10T21:10:30.179720+0000","last_active":"2024-09-10T21:10:31.708884+0000","last_peered":"2024-09-10T21:10:31.708884+0000","last_clean":"2024-09-10T21:10:31.708884+0000","last_became_active":"2024-09-10T21:10:30.179435+0000","last_became_peered":"2024-09-10T21:10:30.179435+0000","last_unstale":"2024-09-10T21:10:31.708884+0000","last_undegraded":"2024-09-10T21:10:31.708884+0000","last_fullsized":"2024-09-10T21:10:31.708884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:45.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173726+0000","last_change":"2024-09-10T21:10:29.165142+0000","last_active":"2024-09-10T21:10:31.173726+0000","last_peered":"2024-09-10T21:10:31.173726+0000","last_clean":"2024-09-10T21:10:31.173726+0000","last_became_active":"2024-09-10T21:10:29.164732+0000","last_became_peered":"2024-09-10T21:10:29.164732+0000","last_unstale":"2024-09-10T21:10:31.173726+0000","last_undegraded":"2024-09-10T21:10:31.173726+0000","last_fullsized":"2024-09-10T21:10:31.173726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:26.261468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.173644+0000","last_change":"2024-09-10T21:10:30.164096+0000","last_active":"2024-09-10T21:10:31.173644+0000","last_peered":"2024-09-10T21:10:31.173644+0000","last_clean":"2024-09-10T21:10:31.173644+0000","last_became_active":"2024-09-10T21:10:30.163474+0000","last_became_peered":"2024-09-10T21:10:30.163474+0000","last_unstale":"2024-09-10T21:10:31.173644+0000","last_undegraded":"2024-09-10T21:10:31.173644+0000","last_fullsized":"2024-09-10T21:10:31.173644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:57.957611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.707812+0000","last_change":"2024-09-10T21:10:29.166991+0000","last_active":"2024-09-10T21:10:31.707812+0000","last_peered":"2024-09-10T21:10:31.707812+0000","last_clean":"2024-09-10T21:10:31.707812+0000","last_became_active":"2024-09-10T21:10:29.166675+0000","last_became_peered":"2024-09-10T21:10:29.166675+0000","last_unstale":"2024-09-10T21:10:31.707812+0000","last_undegraded":"2024-09-10T21:10:31.707812+0000","last_fullsized":"2024-09-10T21:10:31.707812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:25.986804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.446431+0000","last_change":"2024-09-10T21:10:30.176046+0000","last_active":"2024-09-10T21:10:31.446431+0000","last_peered":"2024-09-10T21:10:31.446431+0000","last_clean":"2024-09-10T21:10:31.446431+0000","last_became_active":"2024-09-10T21:10:30.175784+0000","last_became_peered":"2024-09-10T21:10:30.175784+0000","last_unstale":"2024-09-10T21:10:31.446431+0000","last_undegraded":"2024-09-10T21:10:31.446431+0000","last_fullsized":"2024-09-10T21:10:31.446431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:37.972763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708858+0000","last_change":"2024-09-10T21:10:29.172106+0000","last_active":"2024-09-10T21:10:31.708858+0000","last_peered":"2024-09-10T21:10:31.708858+0000","last_clean":"2024-09-10T21:10:31.708858+0000","last_became_active":"2024-09-10T21:10:29.167132+0000","last_became_peered":"2024-09-10T21:10:29.167132+0000","last_unstale":"2024-09-10T21:10:31.708858+0000","last_undegraded":"2024-09-10T21:10:31.708858+0000","last_fullsized":"2024-09-10T21:10:31.708858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:48.493501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.708898+0000","last_change":"2024-09-10T21:10:30.176689+0000","last_active":"2024-09-10T21:10:31.708898+0000","last_peered":"2024-09-10T21:10:31.708898+0000","last_clean":"2024-09-10T21:10:31.708898+0000","last_became_active":"2024-09-10T21:10:30.176361+0000","last_became_peered":"2024-09-10T21:10:30.176361+0000","last_unstale":"2024-09-10T21:10:31.708898+0000","last_undegraded":"2024-09-10T21:10:31.708898+0000","last_fullsized":"2024-09-10T21:10:31.708898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:55.720343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.709088+0000","last_change":"2024-09-10T21:10:30.180387+0000","last_active":"2024-09-10T21:10:31.709088+0000","last_peered":"2024-09-10T21:10:31.709088+0000","last_clean":"2024-09-10T21:10:31.709088+0000","last_became_active":"2024-09-10T21:10:30.180100+0000","last_became_peered":"2024-09-10T21:10:30.180100+0000","last_unstale":"2024-09-10T21:10:31.709088+0000","last_undegraded":"2024-09-10T21:10:31.709088+0000","last_fullsized":"2024-09-10T21:10:31.709088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:05.939523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179323+0000","last_change":"2024-09-10T21:10:29.178074+0000","last_active":"2024-09-10T21:10:31.179323+0000","last_peered":"2024-09-10T21:10:31.179323+0000","last_clean":"2024-09-10T21:10:31.179323+0000","last_became_active":"2024-09-10T21:10:29.177794+0000","last_became_peered":"2024-09-10T21:10:29.177794+0000","last_unstale":"2024-09-10T21:10:31.179323+0000","last_undegraded":"2024-09-10T21:10:31.179323+0000","last_fullsized":"2024-09-10T21:10:31.179323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:18.267264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.167690+0000","last_change":"2024-09-10T21:10:30.179424+0000","last_active":"2024-09-10T21:10:31.167690+0000","last_peered":"2024-09-10T21:10:31.167690+0000","last_clean":"2024-09-10T21:10:31.167690+0000","last_became_active":"2024-09-10T21:10:30.178557+0000","last_became_peered":"2024-09-10T21:10:30.178557+0000","last_unstale":"2024-09-10T21:10:31.167690+0000","last_undegraded":"2024-09-10T21:10:31.167690+0000","last_fullsized":"2024-09-10T21:10:31.167690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:14.118820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.179366+0000","last_change":"2024-09-10T21:10:29.169287+0000","last_active":"2024-09-10T21:10:31.179366+0000","last_peered":"2024-09-10T21:10:31.179366+0000","last_clean":"2024-09-10T21:10:31.179366+0000","last_became_active":"2024-09-10T21:10:29.169010+0000","last_became_peered":"2024-09-10T21:10:29.169010+0000","last_unstale":"2024-09-10T21:10:31.179366+0000","last_undegraded":"2024-09-10T21:10:31.179366+0000","last_fullsized":"2024-09-10T21:10:31.179366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:41.528182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166555+0000","last_change":"2024-09-10T21:10:30.166099+0000","last_active":"2024-09-10T21:10:31.166555+0000","last_peered":"2024-09-10T21:10:31.166555+0000","last_clean":"2024-09-10T21:10:31.166555+0000","last_became_active":"2024-09-10T21:10:30.165762+0000","last_became_peered":"2024-09-10T21:10:30.165762+0000","last_unstale":"2024-09-10T21:10:31.166555+0000","last_undegraded":"2024-09-10T21:10:31.166555+0000","last_fullsized":"2024-09-10T21:10:31.166555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:54.002262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166762+0000","last_change":"2024-09-10T21:10:29.167583+0000","last_active":"2024-09-10T21:10:31.166762+0000","last_peered":"2024-09-10T21:10:31.166762+0000","last_clean":"2024-09-10T21:10:31.166762+0000","last_became_active":"2024-09-10T21:10:29.162012+0000","last_became_peered":"2024-09-10T21:10:29.162012+0000","last_unstale":"2024-09-10T21:10:31.166762+0000","last_undegraded":"2024-09-10T21:10:31.166762+0000","last_fullsized":"2024-09-10T21:10:31.166762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:04.055586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.166801+0000","last_change":"2024-09-10T21:10:30.162898+0000","last_active":"2024-09-10T21:10:31.166801+0000","last_peered":"2024-09-10T21:10:31.166801+0000","last_clean":"2024-09-10T21:10:31.166801+0000","last_became_active":"2024-09-10T21:10:30.162536+0000","last_became_peered":"2024-09-10T21:10:30.162536+0000","last_unstale":"2024-09-10T21:10:31.166801+0000","last_undegraded":"2024-09-10T21:10:31.166801+0000","last_fullsized":"2024-09-10T21:10:31.166801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:29.137082+0000","last_clean_scrub_stamp":"2024-09-10T21:10:29.137082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:09.399612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:10:31.174191+0000","last_change":"2024-09-10T21:10:29.166448+0000","last_active":"2024-09-10T21:10:31.174191+0000","last_peered":"2024-09-10T21:10:31.174191+0000","last_clean":"2024-09-10T21:10:31.174191+0000","last_became_active":"2024-09-10T21:10:29.166153+0000","last_became_peered":"2024-09-10T21:10:29.166153+0000","last_unstale":"2024-09-10T21:10:31.174191+0000","last_undegraded":"2024-09-10T21:10:31.174191+0000","last_fullsized":"2024-09-10T21:10:31.174191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:28.130271+0000","last_clean_scrub_stamp":"2024-09-10T21:10:28.130271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:08.960569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":811008,"data_stored":662647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":74471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":72769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":217088,"data_stored":72243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":806912,"data_stored":662625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:10:42.737 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:10:42.737 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T21:10:42.737 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T21:10:42.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:10:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 -- 172.21.15.16:0/1127261563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe64c0b13c0 msgr2=0x7fe64c0afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 --2- 172.21.15.16:0/1127261563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe64c0b13c0 0x7fe64c0afac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe644009c80 tx=0x7fe64401cc80 comp rx=0 tx=0).stop 2024-09-10T21:10:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 -- 172.21.15.16:0/1127261563 shutdown_connections 2024-09-10T21:10:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 --2- 172.21.15.16:0/1127261563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe64c0b13c0 0x7fe64c0afac0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 --2- 172.21.15.16:0/1127261563 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe64c0716f0 0x7fe64c0af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 -- 172.21.15.16:0/1127261563 >> 172.21.15.16:0/1127261563 conn(0x7fe64c09b130 msgr2=0x7fe64c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 -- 172.21.15.16:0/1127261563 shutdown_connections 2024-09-10T21:10:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 -- 172.21.15.16:0/1127261563 wait complete. 2024-09-10T21:10:42.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 Processor -- start 2024-09-10T21:10:42.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 -- start start 2024-09-10T21:10:42.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0716f0 0x7fe64c1fb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:42.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe64c0b13c0 0x7fe64c14e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe64c15f400 0x7fe64c164f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c168fc0 con 0x7fe64c0b13c0 2024-09-10T21:10:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c168e40 con 0x7fe64c0716f0 2024-09-10T21:10:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe651e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe64c169140 con 0x7fe64c15f400 2024-09-10T21:10:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe642000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe64c0b13c0 0x7fe64c14e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe642000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe64c0b13c0 0x7fe64c14e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36348/0 (socket says 172.21.15.16:36348) 2024-09-10T21:10:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe642000640 1 -- 172.21.15.16:0/4193259308 learned_addr learned my addr 172.21.15.16:0/4193259308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe64aa00640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0716f0 0x7fe64c1fb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe64b400640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe64c15f400 0x7fe64c164f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe642000640 1 -- 172.21.15.16:0/4193259308 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe64c15f400 msgr2=0x7fe64c164f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe642000640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe64c15f400 0x7fe64c164f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe642000640 1 -- 172.21.15.16:0/4193259308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0716f0 msgr2=0x7fe64c1fb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe642000640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0716f0 0x7fe64c1fb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.922+0000 7fe642000640 1 -- 172.21.15.16:0/4193259308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c1fbd20 con 0x7fe64c0b13c0 2024-09-10T21:10:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.926+0000 7fe642000640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe64c0b13c0 0x7fe64c14e560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe63400e9c0 tx=0x7fe63400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.926+0000 7fe643e00640 1 -- 172.21.15.16:0/4193259308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63400cd80 con 0x7fe64c0b13c0 2024-09-10T21:10:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.926+0000 7fe643e00640 1 -- 172.21.15.16:0/4193259308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe634005020 con 0x7fe64c0b13c0 2024-09-10T21:10:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.926+0000 7fe643e00640 1 -- 172.21.15.16:0/4193259308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe634005450 con 0x7fe64c0b13c0 2024-09-10T21:10:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.926+0000 7fe651e00640 1 -- 172.21.15.16:0/4193259308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64c0b6080 con 0x7fe64c0b13c0 2024-09-10T21:10:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.926+0000 7fe651e00640 1 -- 172.21.15.16:0/4193259308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64c06e3b0 con 0x7fe64c0b13c0 2024-09-10T21:10:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.926+0000 7fe651e00640 1 -- 172.21.15.16:0/4193259308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64c070fe0 con 0x7fe64c0b13c0 2024-09-10T21:10:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.926+0000 7fe643e00640 1 -- 172.21.15.16:0/4193259308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe6340047b0 con 0x7fe64c0b13c0 2024-09-10T21:10:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.926+0000 7fe643e00640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6280725e0 0x7fe628074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.926+0000 7fe643e00640 1 -- 172.21.15.16:0/4193259308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe634093c40 con 0x7fe64c0b13c0 2024-09-10T21:10:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.926+0000 7fe64aa00640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6280725e0 0x7fe628074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.930+0000 7fe64aa00640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6280725e0 0x7fe628074aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe63c001780 tx=0x7fe63c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:42.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:42.930+0000 7fe643e00640 1 -- 172.21.15.16:0/4193259308 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe63405ecc0 con 0x7fe64c0b13c0 2024-09-10T21:10:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe651e00640 1 -- 172.21.15.16:0/4193259308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe64c0d0aa0 con 0x7fe64c0b13c0 2024-09-10T21:10:43.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe643e00640 1 -- 172.21.15.16:0/4193259308 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe6340636f0 con 0x7fe64c0b13c0 2024-09-10T21:10:43.146 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:43.146 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T21:10:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe651e00640 1 -- 172.21.15.16:0/4193259308 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6280725e0 msgr2=0x7fe628074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe651e00640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6280725e0 0x7fe628074aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe63c001780 tx=0x7fe63c009040 comp rx=0 tx=0).stop 2024-09-10T21:10:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe651e00640 1 -- 172.21.15.16:0/4193259308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe64c0b13c0 msgr2=0x7fe64c14e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe651e00640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe64c0b13c0 0x7fe64c14e560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe63400e9c0 tx=0x7fe63400ee80 comp rx=0 tx=0).stop 2024-09-10T21:10:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe651e00640 1 -- 172.21.15.16:0/4193259308 shutdown_connections 2024-09-10T21:10:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe651e00640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6280725e0 0x7fe628074aa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe651e00640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe64c15f400 0x7fe64c164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe651e00640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe64c0b13c0 0x7fe64c14e560 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe651e00640 1 --2- 172.21.15.16:0/4193259308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0716f0 0x7fe64c1fb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe651e00640 1 -- 172.21.15.16:0/4193259308 >> 172.21.15.16:0/4193259308 conn(0x7fe64c09b130 msgr2=0x7fe64c1661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe651e00640 1 -- 172.21.15.16:0/4193259308 shutdown_connections 2024-09-10T21:10:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.142+0000 7fe651e00640 1 -- 172.21.15.16:0/4193259308 wait complete. 2024-09-10T21:10:43.167 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T21:10:43.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:10:43.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.306+0000 7f3277800640 1 -- 172.21.15.16:0/3466011979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32701599d0 msgr2=0x7f327015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.306+0000 7f3277800640 1 --2- 172.21.15.16:0/3466011979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32701599d0 0x7f327015f830 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3260009900 tx=0x7f326001c850 comp rx=0 tx=0).stop 2024-09-10T21:10:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.306+0000 7f3277800640 1 -- 172.21.15.16:0/3466011979 shutdown_connections 2024-09-10T21:10:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.306+0000 7f3277800640 1 --2- 172.21.15.16:0/3466011979 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f327006e640 0x7f32700ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.306+0000 7f3277800640 1 --2- 172.21.15.16:0/3466011979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32701599d0 0x7f327015f830 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.306+0000 7f3277800640 1 -- 172.21.15.16:0/3466011979 >> 172.21.15.16:0/3466011979 conn(0x7f32700cfb20 msgr2=0x7f32700cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.306+0000 7f3277800640 1 -- 172.21.15.16:0/3466011979 shutdown_connections 2024-09-10T21:10:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3277800640 1 -- 172.21.15.16:0/3466011979 wait complete. 2024-09-10T21:10:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3277800640 1 Processor -- start 2024-09-10T21:10:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3277800640 1 -- start start 2024-09-10T21:10:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3277800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327006e640 0x7f32701f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3277800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f32700ca6b0 0x7f3270155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3277800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32701599d0 0x7f32700c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3277800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3270168bd0 con 0x7f327006e640 2024-09-10T21:10:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3277800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3270073040 con 0x7f32701599d0 2024-09-10T21:10:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3277800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3270168d50 con 0x7f32700ca6b0 2024-09-10T21:10:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3274e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327006e640 0x7f32701f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3274e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327006e640 0x7f32701f85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36360/0 (socket says 172.21.15.16:36360) 2024-09-10T21:10:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3274e00640 1 -- 172.21.15.16:0/1696640740 learned_addr learned my addr 172.21.15.16:0/1696640740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3267e00640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f32700ca6b0 0x7f3270155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3275800640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32701599d0 0x7f32700c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3274e00640 1 -- 172.21.15.16:0/1696640740 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f32700ca6b0 msgr2=0x7f3270155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3274e00640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f32700ca6b0 0x7f3270155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3274e00640 1 -- 172.21.15.16:0/1696640740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32701599d0 msgr2=0x7f32700c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3274e00640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32701599d0 0x7f32700c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3274e00640 1 -- 172.21.15.16:0/1696640740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32700b1000 con 0x7f327006e640 2024-09-10T21:10:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3274e00640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327006e640 0x7f32701f85c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3260009850 tx=0x7f32600a56a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f326d600640 1 -- 172.21.15.16:0/1696640740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32600bc070 con 0x7f327006e640 2024-09-10T21:10:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f326d600640 1 -- 172.21.15.16:0/1696640740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32600043b0 con 0x7f327006e640 2024-09-10T21:10:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f326d600640 1 -- 172.21.15.16:0/1696640740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32600b2040 con 0x7f327006e640 2024-09-10T21:10:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3277800640 1 -- 172.21.15.16:0/1696640740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32700b12e0 con 0x7f327006e640 2024-09-10T21:10:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.310+0000 7f3277800640 1 -- 172.21.15.16:0/1696640740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3270156420 con 0x7f327006e640 2024-09-10T21:10:43.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.314+0000 7f3277800640 1 -- 172.21.15.16:0/1696640740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32700ca170 con 0x7f327006e640 2024-09-10T21:10:43.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.314+0000 7f326d600640 1 -- 172.21.15.16:0/1696640740 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3260005280 con 0x7f327006e640 2024-09-10T21:10:43.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.314+0000 7f326d600640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f324c072610 0x7f324c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:43.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.314+0000 7f326d600640 1 -- 172.21.15.16:0/1696640740 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f326012d230 con 0x7f327006e640 2024-09-10T21:10:43.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.314+0000 7f3267e00640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f324c072610 0x7f324c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:43.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.314+0000 7f3267e00640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f324c072610 0x7f324c074ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32580058d0 tx=0x7f3258005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:43.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.318+0000 7f326d600640 1 -- 172.21.15.16:0/1696640740 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f32600164d0 con 0x7f327006e640 2024-09-10T21:10:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.526+0000 7f3277800640 1 -- 172.21.15.16:0/1696640740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32700cd4d0 con 0x7f327006e640 2024-09-10T21:10:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.526+0000 7f326d600640 1 -- 172.21.15.16:0/1696640740 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f32600f8880 con 0x7f327006e640 2024-09-10T21:10:43.533 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:43.534 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-10T21:10:33:173828+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2107270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2107270748},{"type":"v1","addr":"172.21.15.16:6836","nonce":2107270748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/591195258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":591195258},{"type":"v1","addr":"172.21.15.16:6837","nonce":591195258}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2924498152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2924498152},{"type":"v1","addr":"172.21.15.52:6834","nonce":2924498152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:10:29.560192+0000","modified":"2024-09-10T21:10:32.575127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4356":{"gid":4356,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/3067177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3067177622},{"type":"v1","addr":"172.21.15.52:6835","nonce":3067177622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4356,"qdb_cluster":[4356]},"id":1}]} 2024-09-10T21:10:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-10T21:10:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.530+0000 7f3277800640 1 -- 172.21.15.16:0/1696640740 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f324c072610 msgr2=0x7f324c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.530+0000 7f3277800640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f324c072610 0x7f324c074ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32580058d0 tx=0x7f3258005860 comp rx=0 tx=0).stop 2024-09-10T21:10:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.530+0000 7f3277800640 1 -- 172.21.15.16:0/1696640740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327006e640 msgr2=0x7f32701f85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.530+0000 7f3277800640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327006e640 0x7f32701f85c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3260009850 tx=0x7f32600a56a0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.530+0000 7f3277800640 1 -- 172.21.15.16:0/1696640740 shutdown_connections 2024-09-10T21:10:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.530+0000 7f3277800640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f324c072610 0x7f324c074ad0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.530+0000 7f3277800640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32701599d0 0x7f32700c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.530+0000 7f3277800640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f32700ca6b0 0x7f3270155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.530+0000 7f3277800640 1 --2- 172.21.15.16:0/1696640740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327006e640 0x7f32701f85c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.530+0000 7f3277800640 1 -- 172.21.15.16:0/1696640740 >> 172.21.15.16:0/1696640740 conn(0x7f32700cfb20 msgr2=0x7f32700ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.530+0000 7f3277800640 1 -- 172.21.15.16:0/1696640740 shutdown_connections 2024-09-10T21:10:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.530+0000 7f3277800640 1 -- 172.21.15.16:0/1696640740 wait complete. 2024-09-10T21:10:43.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:10:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.694+0000 7f0975600640 1 -- 172.21.15.16:0/1039196471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0970160540 msgr2=0x7f0970073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.694+0000 7f0975600640 1 --2- 172.21.15.16:0/1039196471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0970160540 0x7f0970073d90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0970075400 tx=0x7f096801cd50 comp rx=0 tx=0).stop 2024-09-10T21:10:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.694+0000 7f0975600640 1 -- 172.21.15.16:0/1039196471 shutdown_connections 2024-09-10T21:10:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.694+0000 7f0975600640 1 --2- 172.21.15.16:0/1039196471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0970160540 0x7f0970073d90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.694+0000 7f0975600640 1 --2- 172.21.15.16:0/1039196471 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f097006cbe0 0x7f0970160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.694+0000 7f0975600640 1 --2- 172.21.15.16:0/1039196471 >> v2:172.21.15.52:3300/0 conn(0x7f09700cc480 0x7f09700cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.694+0000 7f0975600640 1 -- 172.21.15.16:0/1039196471 >> 172.21.15.16:0/1039196471 conn(0x7f09700d0c10 msgr2=0x7f09700d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.694+0000 7f0975600640 1 -- 172.21.15.16:0/1039196471 shutdown_connections 2024-09-10T21:10:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.694+0000 7f0975600640 1 -- 172.21.15.16:0/1039196471 wait complete. 2024-09-10T21:10:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0975600640 1 Processor -- start 2024-09-10T21:10:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0975600640 1 -- start start 2024-09-10T21:10:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0975600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f097006cbe0 0x7f0970149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0975600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09700cc480 0x7f09700ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0975600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0970160540 0x7f09700c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0975600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09700cb670 con 0x7f09700cc480 2024-09-10T21:10:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0975600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09700cb4f0 con 0x7f0970160540 2024-09-10T21:10:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0975600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f09700cb7f0 con 0x7f097006cbe0 2024-09-10T21:10:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0967e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09700cc480 0x7f09700ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0967e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09700cc480 0x7f09700ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36380/0 (socket says 172.21.15.16:36380) 2024-09-10T21:10:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0967e00640 1 -- 172.21.15.16:0/3731990035 learned_addr learned my addr 172.21.15.16:0/3731990035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f096ea00640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f097006cbe0 0x7f0970149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f096f400640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0970160540 0x7f09700c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0967e00640 1 -- 172.21.15.16:0/3731990035 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f097006cbe0 msgr2=0x7f0970149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0967e00640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f097006cbe0 0x7f0970149400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0967e00640 1 -- 172.21.15.16:0/3731990035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0970160540 msgr2=0x7f09700c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0967e00640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0970160540 0x7f09700c6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0967e00640 1 -- 172.21.15.16:0/3731990035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09700c7450 con 0x7f09700cc480 2024-09-10T21:10:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0967e00640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09700cc480 0x7f09700ca7d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f095800c970 tx=0x7f095800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0966a00640 1 -- 172.21.15.16:0/3731990035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958007c70 con 0x7f09700cc480 2024-09-10T21:10:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0966a00640 1 -- 172.21.15.16:0/3731990035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f095800f450 con 0x7f09700cc480 2024-09-10T21:10:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0966a00640 1 -- 172.21.15.16:0/3731990035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095800f8a0 con 0x7f09700cc480 2024-09-10T21:10:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.698+0000 7f0975600640 1 -- 172.21.15.16:0/3731990035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09700c7730 con 0x7f09700cc480 2024-09-10T21:10:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.702+0000 7f0975600640 1 -- 172.21.15.16:0/3731990035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09700c6ab0 con 0x7f09700cc480 2024-09-10T21:10:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.702+0000 7f0975600640 1 -- 172.21.15.16:0/3731990035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09700cbd80 con 0x7f09700cc480 2024-09-10T21:10:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.702+0000 7f0966a00640 1 -- 172.21.15.16:0/3731990035 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f095800fa40 con 0x7f09700cc480 2024-09-10T21:10:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.702+0000 7f0966a00640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f094c072610 0x7f094c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.702+0000 7f0966a00640 1 -- 172.21.15.16:0/3731990035 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f09580938d0 con 0x7f09700cc480 2024-09-10T21:10:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.702+0000 7f096ea00640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f094c072610 0x7f094c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:43.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.702+0000 7f096ea00640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f094c072610 0x7f094c074ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f096000bb80 tx=0x7f096000b390 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.706+0000 7f0966a00640 1 -- 172.21.15.16:0/3731990035 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0958014030 con 0x7f09700cc480 2024-09-10T21:10:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0975600640 1 -- 172.21.15.16:0/3731990035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0970074760 con 0x7f09700cc480 2024-09-10T21:10:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0966a00640 1 -- 172.21.15.16:0/3731990035 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f095805e950 con 0x7f09700cc480 2024-09-10T21:10:43.923 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:43.923 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-10T21:10:33:173828+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2107270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2107270748},{"type":"v1","addr":"172.21.15.16:6836","nonce":2107270748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/591195258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":591195258},{"type":"v1","addr":"172.21.15.16:6837","nonce":591195258}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2924498152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2924498152},{"type":"v1","addr":"172.21.15.52:6834","nonce":2924498152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:10:29.560192+0000","modified":"2024-09-10T21:10:32.575127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4356":{"gid":4356,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/3067177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3067177622},{"type":"v1","addr":"172.21.15.52:6835","nonce":3067177622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4356,"qdb_cluster":[4356]},"id":1}]} 2024-09-10T21:10:43.923 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-10T21:10:43.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0975600640 1 -- 172.21.15.16:0/3731990035 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f094c072610 msgr2=0x7f094c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:43.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0975600640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f094c072610 0x7f094c074ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f096000bb80 tx=0x7f096000b390 comp rx=0 tx=0).stop 2024-09-10T21:10:43.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0975600640 1 -- 172.21.15.16:0/3731990035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09700cc480 msgr2=0x7f09700ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:43.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0975600640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09700cc480 0x7f09700ca7d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f095800c970 tx=0x7f095800ce30 comp rx=0 tx=0).stop 2024-09-10T21:10:43.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0975600640 1 -- 172.21.15.16:0/3731990035 shutdown_connections 2024-09-10T21:10:43.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0975600640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f094c072610 0x7f094c074ad0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0975600640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0970160540 0x7f09700c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0975600640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09700cc480 0x7f09700ca7d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0975600640 1 --2- 172.21.15.16:0/3731990035 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f097006cbe0 0x7f0970149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:43.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0975600640 1 -- 172.21.15.16:0/3731990035 >> 172.21.15.16:0/3731990035 conn(0x7f09700d0c10 msgr2=0x7f09701663e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:43.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0975600640 1 -- 172.21.15.16:0/3731990035 shutdown_connections 2024-09-10T21:10:43.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:43.918+0000 7f0975600640 1 -- 172.21.15.16:0/3731990035 wait complete. 2024-09-10T21:10:43.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:10:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.086+0000 7f1dbd000640 1 -- 172.21.15.16:0/2580192281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1db8161620 msgr2=0x7f1db8070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.086+0000 7f1dbd000640 1 --2- 172.21.15.16:0/2580192281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1db8161620 0x7f1db8070cf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1da80098f0 tx=0x7f1da801c8b0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.086+0000 7f1dbd000640 1 -- 172.21.15.16:0/2580192281 shutdown_connections 2024-09-10T21:10:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.086+0000 7f1dbd000640 1 --2- 172.21.15.16:0/2580192281 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1db8071230 0x7f1db80d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.086+0000 7f1dbd000640 1 --2- 172.21.15.16:0/2580192281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1db8161620 0x7f1db8070cf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.086+0000 7f1dbd000640 1 -- 172.21.15.16:0/2580192281 >> 172.21.15.16:0/2580192281 conn(0x7f1db809b130 msgr2=0x7f1db80d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.086+0000 7f1dbd000640 1 -- 172.21.15.16:0/2580192281 shutdown_connections 2024-09-10T21:10:44.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.086+0000 7f1dbd000640 1 -- 172.21.15.16:0/2580192281 wait complete. 2024-09-10T21:10:44.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dbd000640 1 Processor -- start 2024-09-10T21:10:44.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dbd000640 1 -- start start 2024-09-10T21:10:44.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dbd000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1db8071230 0x7f1db80b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:44.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dbd000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1db80b0b40 0x7f1db8157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:44.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dbd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db8161620 0x7f1db8157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:44.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dbd000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1db816b2c0 con 0x7f1db8161620 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dbd000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1db816b140 con 0x7f1db80b0b40 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dbd000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f1db816b440 con 0x7f1db8071230 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1db6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db8161620 0x7f1db8157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1db6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db8161620 0x7f1db8157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36382/0 (socket says 172.21.15.16:36382) 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1db6800640 1 -- 172.21.15.16:0/3608059925 learned_addr learned my addr 172.21.15.16:0/3608059925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1db5e00640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1db8071230 0x7f1db80b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1db5400640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1db80b0b40 0x7f1db8157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1db6800640 1 -- 172.21.15.16:0/3608059925 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1db8071230 msgr2=0x7f1db80b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1db6800640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1db8071230 0x7f1db80b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1db6800640 1 -- 172.21.15.16:0/3608059925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1db80b0b40 msgr2=0x7f1db8157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1db6800640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1db80b0b40 0x7f1db8157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1db6800640 1 -- 172.21.15.16:0/3608059925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db815a970 con 0x7f1db8161620 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1db6800640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db8161620 0x7f1db8157ac0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1db000c780 tx=0x7f1db000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dae000640 1 -- 172.21.15.16:0/3608059925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0003b80 con 0x7f1db8161620 2024-09-10T21:10:44.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dae000640 1 -- 172.21.15.16:0/3608059925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1db0003d20 con 0x7f1db8161620 2024-09-10T21:10:44.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dae000640 1 -- 172.21.15.16:0/3608059925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db000aab0 con 0x7f1db8161620 2024-09-10T21:10:44.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dbd000640 1 -- 172.21.15.16:0/3608059925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db80b4650 con 0x7f1db8161620 2024-09-10T21:10:44.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dbd000640 1 -- 172.21.15.16:0/3608059925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db8159e90 con 0x7f1db8161620 2024-09-10T21:10:44.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.090+0000 7f1dbd000640 1 -- 172.21.15.16:0/3608059925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db80d0590 con 0x7f1db8161620 2024-09-10T21:10:44.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.094+0000 7f1dae000640 1 -- 172.21.15.16:0/3608059925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1db000ac50 con 0x7f1db8161620 2024-09-10T21:10:44.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.094+0000 7f1dae000640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1d980725e0 0x7f1d98074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:44.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.094+0000 7f1dae000640 1 -- 172.21.15.16:0/3608059925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1db0093ae0 con 0x7f1db8161620 2024-09-10T21:10:44.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.094+0000 7f1db5e00640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1d980725e0 0x7f1d98074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:44.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.094+0000 7f1db5e00640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1d980725e0 0x7f1d98074aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1da80af000 tx=0x7f1da801b0f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:44.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.098+0000 7f1dae000640 1 -- 172.21.15.16:0/3608059925 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1db005eb50 con 0x7f1db8161620 2024-09-10T21:10:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.274+0000 7f1dbd000640 1 -- 172.21.15.16:0/3608059925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1db815b4c0 con 0x7f1db8161620 2024-09-10T21:10:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.274+0000 7f1dae000640 1 -- 172.21.15.16:0/3608059925 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f1db815b4c0 con 0x7f1db8161620 2024-09-10T21:10:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:44.279 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:10:31.158120+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:10:27.523741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:10:28.533379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:10:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.274+0000 7f1dbd000640 1 -- 172.21.15.16:0/3608059925 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1d980725e0 msgr2=0x7f1d98074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.274+0000 7f1dbd000640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1d980725e0 0x7f1d98074aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1da80af000 tx=0x7f1da801b0f0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.274+0000 7f1dbd000640 1 -- 172.21.15.16:0/3608059925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db8161620 msgr2=0x7f1db8157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.274+0000 7f1dbd000640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db8161620 0x7f1db8157ac0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1db000c780 tx=0x7f1db000cc40 comp rx=0 tx=0).stop 2024-09-10T21:10:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.274+0000 7f1dbd000640 1 -- 172.21.15.16:0/3608059925 shutdown_connections 2024-09-10T21:10:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.274+0000 7f1dbd000640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1d980725e0 0x7f1d98074aa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.274+0000 7f1dbd000640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db8161620 0x7f1db8157ac0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.274+0000 7f1dbd000640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1db80b0b40 0x7f1db8157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.274+0000 7f1dbd000640 1 --2- 172.21.15.16:0/3608059925 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1db8071230 0x7f1db80b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.274+0000 7f1dbd000640 1 -- 172.21.15.16:0/3608059925 >> 172.21.15.16:0/3608059925 conn(0x7f1db809b130 msgr2=0x7f1db8158320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.278+0000 7f1dbd000640 1 -- 172.21.15.16:0/3608059925 shutdown_connections 2024-09-10T21:10:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.278+0000 7f1dbd000640 1 -- 172.21.15.16:0/3608059925 wait complete. 2024-09-10T21:10:44.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:10:44.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.442+0000 7fb8eba00640 1 -- 172.21.15.16:0/1110866574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc003a60 msgr2=0x7fb8cc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:44.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.442+0000 7fb8eba00640 1 --2- 172.21.15.16:0/1110866574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8cc003a60 0x7fb8cc003e80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009b60 tx=0x7fb8d801d2b0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.442+0000 7fb8e9a00640 1 -- 172.21.15.16:0/1110866574 reap_dead start 2024-09-10T21:10:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.442+0000 7fb8eba00640 1 -- 172.21.15.16:0/1110866574 shutdown_connections 2024-09-10T21:10:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.442+0000 7fb8eba00640 1 -- 172.21.15.16:0/1110866574 >> 172.21.15.16:0/1110866574 conn(0x7fb8e40d0c30 msgr2=0x7fb8e40d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.442+0000 7fb8eba00640 1 -- 172.21.15.16:0/1110866574 shutdown_connections 2024-09-10T21:10:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.442+0000 7fb8eba00640 1 -- 172.21.15.16:0/1110866574 wait complete. 2024-09-10T21:10:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.442+0000 7fb8eba00640 1 Processor -- start 2024-09-10T21:10:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.442+0000 7fb8eba00640 1 -- start start 2024-09-10T21:10:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8eba00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8cc003a60 0x7fb8e40c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8eba00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb8e4161a20 0x7fb8e41fb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8eba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e41fba20 0x7fb8e41561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8eba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e4168df0 con 0x7fb8e41fba20 2024-09-10T21:10:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8eba00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e4168c70 con 0x7fb8cc003a60 2024-09-10T21:10:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8eba00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e4168f70 con 0x7fb8e4161a20 2024-09-10T21:10:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e41fba20 0x7fb8e41561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e41fba20 0x7fb8e41561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36400/0 (socket says 172.21.15.16:36400) 2024-09-10T21:10:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e9a00640 1 -- 172.21.15.16:0/111136748 learned_addr learned my addr 172.21.15.16:0/111136748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e3e00640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb8e4161a20 0x7fb8e41fb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e9000640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8cc003a60 0x7fb8e40c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e9a00640 1 -- 172.21.15.16:0/111136748 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb8e4161a20 msgr2=0x7fb8e41fb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e9a00640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb8e4161a20 0x7fb8e41fb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e9a00640 1 -- 172.21.15.16:0/111136748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8cc003a60 msgr2=0x7fb8e40c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e9a00640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8cc003a60 0x7fb8e40c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e9a00640 1 -- 172.21.15.16:0/111136748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e4163f10 con 0x7fb8e41fba20 2024-09-10T21:10:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e9a00640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e41fba20 0x7fb8e41561c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40cbab0 tx=0x7fb8dc006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e0c00640 1 -- 172.21.15.16:0/111136748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00fa80 con 0x7fb8e41fba20 2024-09-10T21:10:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e0c00640 1 -- 172.21.15.16:0/111136748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8dc017070 con 0x7fb8e41fba20 2024-09-10T21:10:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e0c00640 1 -- 172.21.15.16:0/111136748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc011430 con 0x7fb8e41fba20 2024-09-10T21:10:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8eba00640 1 -- 172.21.15.16:0/111136748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e40c7d00 con 0x7fb8e41fba20 2024-09-10T21:10:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8eba00640 1 -- 172.21.15.16:0/111136748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e4161ea0 con 0x7fb8e41fba20 2024-09-10T21:10:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8eba00640 1 -- 172.21.15.16:0/111136748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e40c44b0 con 0x7fb8e41fba20 2024-09-10T21:10:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.446+0000 7fb8e0c00640 1 -- 172.21.15.16:0/111136748 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb8dc0115d0 con 0x7fb8e41fba20 2024-09-10T21:10:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.450+0000 7fb8e0c00640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb8c4072610 0x7fb8c4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.450+0000 7fb8e0c00640 1 -- 172.21.15.16:0/111136748 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb8dc093920 con 0x7fb8e41fba20 2024-09-10T21:10:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.450+0000 7fb8e9000640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb8c4072610 0x7fb8c4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.450+0000 7fb8e9000640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb8c4072610 0x7fb8c4074ad0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80ae000 tx=0x7fb8d801a050 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:44.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.454+0000 7fb8e0c00640 1 -- 172.21.15.16:0/111136748 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb8dc05e910 con 0x7fb8e41fba20 2024-09-10T21:10:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.662+0000 7fb8eba00640 1 -- 172.21.15.16:0/111136748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8e40ccd20 con 0x7fb8e41fba20 2024-09-10T21:10:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.662+0000 7fb8e0c00640 1 -- 172.21.15.16:0/111136748 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7fb8dc063340 con 0x7fb8e41fba20 2024-09-10T21:10:44.669 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:44.669 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-10T21:10:33:173828+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2107270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2107270748},{"type":"v1","addr":"172.21.15.16:6836","nonce":2107270748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/591195258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":591195258},{"type":"v1","addr":"172.21.15.16:6837","nonce":591195258}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2924498152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2924498152},{"type":"v1","addr":"172.21.15.52:6834","nonce":2924498152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:10:29.560192+0000","modified":"2024-09-10T21:10:32.575127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4356":{"gid":4356,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/3067177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3067177622},{"type":"v1","addr":"172.21.15.52:6835","nonce":3067177622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4356,"qdb_cluster":[4356]},"id":1}]} 2024-09-10T21:10:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-10T21:10:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.666+0000 7fb8eba00640 1 -- 172.21.15.16:0/111136748 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb8c4072610 msgr2=0x7fb8c4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.666+0000 7fb8eba00640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb8c4072610 0x7fb8c4074ad0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80ae000 tx=0x7fb8d801a050 comp rx=0 tx=0).stop 2024-09-10T21:10:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.666+0000 7fb8eba00640 1 -- 172.21.15.16:0/111136748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e41fba20 msgr2=0x7fb8e41561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.666+0000 7fb8eba00640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e41fba20 0x7fb8e41561c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40cbab0 tx=0x7fb8dc006720 comp rx=0 tx=0).stop 2024-09-10T21:10:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.666+0000 7fb8eba00640 1 -- 172.21.15.16:0/111136748 shutdown_connections 2024-09-10T21:10:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.666+0000 7fb8eba00640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb8c4072610 0x7fb8c4074ad0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.666+0000 7fb8eba00640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e41fba20 0x7fb8e41561c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.666+0000 7fb8eba00640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb8e4161a20 0x7fb8e41fb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.666+0000 7fb8eba00640 1 --2- 172.21.15.16:0/111136748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8cc003a60 0x7fb8e40c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.666+0000 7fb8eba00640 1 -- 172.21.15.16:0/111136748 >> 172.21.15.16:0/111136748 conn(0x7fb8e40d0c30 msgr2=0x7fb8e4157260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.666+0000 7fb8eba00640 1 -- 172.21.15.16:0/111136748 shutdown_connections 2024-09-10T21:10:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:44.666+0000 7fb8eba00640 1 -- 172.21.15.16:0/111136748 wait complete. 2024-09-10T21:10:45.094 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:10:29.560192+0000', 'modified': '2024-09-10T21:10:32.575127+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4356}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4356': {'gid': 4356, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.52:6835/3067177622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6833', 'nonce': 3067177622}, {'type': 'v1', 'addr': '172.21.15.52:6835', 'nonce': 3067177622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4356, 'qdb_cluster': [4356]} 2024-09-10T21:10:45.094 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:10:45.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T21:10:45.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 -- 172.21.15.16:0/3553670456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e2015f350 msgr2=0x7f3e200cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:45.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 --2- 172.21.15.16:0/3553670456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e2015f350 0x7f3e200cd130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009840 tx=0x7f3e1001c760 comp rx=0 tx=0).stop 2024-09-10T21:10:45.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 -- 172.21.15.16:0/3553670456 shutdown_connections 2024-09-10T21:10:45.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 --2- 172.21.15.16:0/3553670456 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3e200d0700 0x7f3e200aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 --2- 172.21.15.16:0/3553670456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e2015f350 0x7f3e200cd130 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 -- 172.21.15.16:0/3553670456 >> 172.21.15.16:0/3553670456 conn(0x7f3e200d1d80 msgr2=0x7f3e200d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:45.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 -- 172.21.15.16:0/3553670456 shutdown_connections 2024-09-10T21:10:45.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 -- 172.21.15.16:0/3553670456 wait complete. 2024-09-10T21:10:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 Processor -- start 2024-09-10T21:10:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 -- start start 2024-09-10T21:10:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3e200cfb60 0x7f3e20143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e200d0700 0x7f3e20154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e2015f350 0x7f3e200c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e20168c50 con 0x7f3e2015f350 2024-09-10T21:10:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3e20168ad0 con 0x7f3e200d0700 2024-09-10T21:10:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e25e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3e20168dd0 con 0x7f3e200cfb60 2024-09-10T21:10:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e1f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e2015f350 0x7f3e200c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e1f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e2015f350 0x7f3e200c5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36422/0 (socket says 172.21.15.16:36422) 2024-09-10T21:10:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e1f400640 1 -- 172.21.15.16:0/1270480102 learned_addr learned my addr 172.21.15.16:0/1270480102 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e1ea00640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3e200cfb60 0x7f3e20143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.242+0000 7f3e17e00640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e200d0700 0x7f3e20154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.246+0000 7f3e1f400640 1 -- 172.21.15.16:0/1270480102 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3e200cfb60 msgr2=0x7f3e20143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.246+0000 7f3e1f400640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3e200cfb60 0x7f3e20143970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.246+0000 7f3e1f400640 1 -- 172.21.15.16:0/1270480102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e200d0700 msgr2=0x7f3e20154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.246+0000 7f3e1f400640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e200d0700 0x7f3e20154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.246+0000 7f3e1f400640 1 -- 172.21.15.16:0/1270480102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e200c7f40 con 0x7f3e2015f350 2024-09-10T21:10:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.246+0000 7f3e1f400640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e2015f350 0x7f3e200c5890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800ea20 tx=0x7f3e1800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.246+0000 7f3e17400640 1 -- 172.21.15.16:0/1270480102 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1800cde0 con 0x7f3e2015f350 2024-09-10T21:10:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.246+0000 7f3e17400640 1 -- 172.21.15.16:0/1270480102 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e18004c40 con 0x7f3e2015f350 2024-09-10T21:10:45.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.246+0000 7f3e25e00640 1 -- 172.21.15.16:0/1270480102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e200c8220 con 0x7f3e2015f350 2024-09-10T21:10:45.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.246+0000 7f3e17400640 1 -- 172.21.15.16:0/1270480102 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18005280 con 0x7f3e2015f350 2024-09-10T21:10:45.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.246+0000 7f3e25e00640 1 -- 172.21.15.16:0/1270480102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e200c87e0 con 0x7f3e2015f350 2024-09-10T21:10:45.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.246+0000 7f3e17400640 1 -- 172.21.15.16:0/1270480102 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3e1800f040 con 0x7f3e2015f350 2024-09-10T21:10:45.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e17400640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3dfc0725e0 0x7f3dfc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e17400640 1 -- 172.21.15.16:0/1270480102 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3e18093980 con 0x7f3e2015f350 2024-09-10T21:10:45.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e1ea00640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3dfc0725e0 0x7f3dfc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 Processor -- start 2024-09-10T21:10:45.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 -- start start 2024-09-10T21:10:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e1ea00640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3dfc0725e0 0x7f3dfc074aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3e100af000 tx=0x7f3e1001afa0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e19d0 0x55d5d90e1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90958d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d9095e10 0x55d5d90961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d5d8fb63f0 con 0x55d5d90e19d0 2024-09-10T21:10:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d5d8ef3b20 con 0x55d5d90e22f0 2024-09-10T21:10:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55d5d8ee2f50 con 0x55d5d9095e10 2024-09-10T21:10:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e1ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e19d0 0x55d5d90e1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e1ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e19d0 0x55d5d90e1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36436/0 (socket says 172.21.15.16:36436) 2024-09-10T21:10:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e1ea00640 1 -- 172.21.15.16:0/2171825098 learned_addr learned my addr 172.21.15.16:0/2171825098 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e1f400640 1 --2- 172.21.15.16:0/2171825098 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d9095e10 0x55d5d90961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e17e00640 1 --2- 172.21.15.16:0/2171825098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e1ea00640 1 -- 172.21.15.16:0/2171825098 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d9095e10 msgr2=0x55d5d90961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e1ea00640 1 --2- 172.21.15.16:0/2171825098 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d9095e10 0x55d5d90961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e1ea00640 1 -- 172.21.15.16:0/2171825098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 msgr2=0x55d5d90958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:45.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e1ea00640 1 --2- 172.21.15.16:0/2171825098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e1ea00640 1 -- 172.21.15.16:0/2171825098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5d8eaa590 con 0x55d5d90e19d0 2024-09-10T21:10:45.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e17e00640 1 --2- 172.21.15.16:0/2171825098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:45.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e1ea00640 1 --2- 172.21.15.16:0/2171825098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e19d0 0x55d5d90e1db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3e1001ae10 tx=0x7f3e100b5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd75446fc5ede7e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e0fe00640 1 -- 172.21.15.16:0/2171825098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100c8070 con 0x55d5d90e19d0 2024-09-10T21:10:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e0fe00640 1 -- 172.21.15.16:0/2171825098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e100bf040 con 0x55d5d90e19d0 2024-09-10T21:10:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e0fe00640 1 -- 172.21.15.16:0/2171825098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100b2040 con 0x55d5d90e19d0 2024-09-10T21:10:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 -- 172.21.15.16:0/2171825098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e19d0 msgr2=0x55d5d90e1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 --2- 172.21.15.16:0/2171825098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e19d0 0x55d5d90e1db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3e1001ae10 tx=0x7f3e100b5000 comp rx=0 tx=0).stop 2024-09-10T21:10:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 -- 172.21.15.16:0/2171825098 shutdown_connections 2024-09-10T21:10:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 --2- 172.21.15.16:0/2171825098 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d9095e10 0x55d5d90961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 --2- 172.21.15.16:0/2171825098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 --2- 172.21.15.16:0/2171825098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e19d0 0x55d5d90e1db0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 -- 172.21.15.16:0/2171825098 >> 172.21.15.16:0/2171825098 conn(0x55d5d8f7f130 msgr2=0x55d5d90c4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 -- 172.21.15.16:0/2171825098 shutdown_connections 2024-09-10T21:10:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.250+0000 7f3e292a2000 1 -- 172.21.15.16:0/2171825098 wait complete. 2024-09-10T21:10:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 1 -- start start 2024-09-10T21:10:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 1 Processor -- start 2024-09-10T21:10:45.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 0x55d5d90c2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d90e19d0 0x55d5d9097f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90984d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d5d8e7d5d0 con 0x55d5d9095e10 2024-09-10T21:10:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d5d8ee6d60 con 0x55d5d90e22f0 2024-09-10T21:10:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55d5d8e89500 con 0x55d5d90e19d0 2024-09-10T21:10:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 10 client.0 _ll_register_callbacks cb 0x55d5d8e8d810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e1ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 0x55d5d90c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e1ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 0x55d5d90c2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36450/0 (socket says 172.21.15.16:36450) 2024-09-10T21:10:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e1ea00640 1 -- 172.21.15.16:0/593137803 learned_addr learned my addr 172.21.15.16:0/593137803 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e17e00640 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d90e19d0 0x55d5d9097f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e1f400640 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e1ea00640 1 -- 172.21.15.16:0/593137803 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d90e19d0 msgr2=0x55d5d9097f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e1ea00640 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d90e19d0 0x55d5d9097f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e1ea00640 1 -- 172.21.15.16:0/593137803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 msgr2=0x55d5d90984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e1ea00640 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e1ea00640 1 -- 172.21.15.16:0/593137803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5d8e81170 con 0x55d5d9095e10 2024-09-10T21:10:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e1f400640 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e1ea00640 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 0x55d5d90c2280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3e1001cc40 tx=0x7f3e10004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bd312456d6f19d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:10:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e0cc00640 1 -- 172.21.15.16:0/593137803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100b2070 con 0x55d5d9095e10 2024-09-10T21:10:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e0cc00640 1 -- 172.21.15.16:0/593137803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3e10004cb0 con 0x55d5d9095e10 2024-09-10T21:10:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 1 -- 172.21.15.16:0/593137803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d5d8e7b0f0 con 0x55d5d9095e10 2024-09-10T21:10:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e0cc00640 1 -- 172.21.15.16:0/593137803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f3e100d5050 con 0x55d5d9095e10 2024-09-10T21:10:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 10 client.4629 fetch_fsmap learned FSMap version 8 2024-09-10T21:10:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 1 -- 172.21.15.16:0/593137803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55d5d8f67ef0 con 0x55d5d9095e10 2024-09-10T21:10:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e0cc00640 1 -- 172.21.15.16:0/593137803 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d5d8e7b0f0 con 0x55d5d9095e10 2024-09-10T21:10:45.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e0cc00640 1 -- 172.21.15.16:0/593137803 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f3e100d8020 con 0x55d5d9095e10 2024-09-10T21:10:45.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 10 client.4629 fetch_fsmap finished waiting for FSMap version 8 2024-09-10T21:10:45.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 10 client.4629 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:10:45.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 20 client.4629 populate_metadata read hostname 'smithi016' 2024-09-10T21:10:45.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.52:6833/3067177622,v1:172.21.15.52:6835/3067177622] conn(0x55d5d9098ee0 0x55d5d90992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 4 client.4629 mds_command: new command op to 4356 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:45.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e292a2000 1 -- 172.21.15.16:0/593137803 --> [v2:172.21.15.52:6833/3067177622,v1:172.21.15.52:6835/3067177622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5d8ef5a70 con 0x55d5d9098ee0 2024-09-10T21:10:45.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.254+0000 7f3e17e00640 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.52:6833/3067177622,v1:172.21.15.52:6835/3067177622] conn(0x55d5d9098ee0 0x55d5d90992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e17e00640 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.52:6833/3067177622,v1:172.21.15.52:6835/3067177622] conn(0x55d5d9098ee0 0x55d5d90992c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7413d0a7b184da1 server_cookie=5563bb9937d5250f in_seq=0 out_seq=0 2024-09-10T21:10:45.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e0cc00640 10 client.4629 ms_handle_connect on v2:172.21.15.52:6833/3067177622 2024-09-10T21:10:45.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e0cc00640 1 -- 172.21.15.16:0/593137803 <== mds.0 v2:172.21.15.52:6833/3067177622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d5d8ef5a70 con 0x55d5d9098ee0 2024-09-10T21:10:45.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e0cc00640 10 client.4629 handle_command_reply: tid=0 2024-09-10T21:10:45.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e292a2000 1 client.4629 shutdown 2024-09-10T21:10:45.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e292a2000 1 -- 172.21.15.16:0/593137803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 msgr2=0x55d5d90c2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:45.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e292a2000 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 0x55d5d90c2280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3e1001cc40 tx=0x7f3e10004ac0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e292a2000 1 -- 172.21.15.16:0/593137803 shutdown_connections 2024-09-10T21:10:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e292a2000 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.52:6833/3067177622,v1:172.21.15.52:6835/3067177622] conn(0x55d5d9098ee0 0x55d5d90992c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e292a2000 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e292a2000 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d90e19d0 0x55d5d9097f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e292a2000 1 --2- 172.21.15.16:0/593137803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 0x55d5d90c2280 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e0cc00640 0 client.4629 ms_handle_reset on v2:172.21.15.52:6833/3067177622 2024-09-10T21:10:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e292a2000 1 -- 172.21.15.16:0/593137803 >> 172.21.15.16:0/593137803 conn(0x55d5d8f7f130 msgr2=0x55d5d9094fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e292a2000 1 -- 172.21.15.16:0/593137803 shutdown_connections 2024-09-10T21:10:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e292a2000 1 -- 172.21.15.16:0/593137803 wait complete. 2024-09-10T21:10:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.258+0000 7f3e292a2000 20 client.4629 trim_cache size 0 max 16384 2024-09-10T21:10:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 Processor -- start 2024-09-10T21:10:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 -- start start 2024-09-10T21:10:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d9095e10 0x55d5d90f3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e19d0 0x55d5d90f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e22f0 0x55d5d90c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e201ff910 con 0x55d5d90e22f0 2024-09-10T21:10:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2006ef50 con 0x55d5d90e19d0 2024-09-10T21:10:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3e200b6980 con 0x55d5d9095e10 2024-09-10T21:10:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e1f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e22f0 0x55d5d90c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e1f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e22f0 0x55d5d90c0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36466/0 (socket says 172.21.15.16:36466) 2024-09-10T21:10:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e1f400640 1 -- 172.21.15.16:0/3119503236 learned_addr learned my addr 172.21.15.16:0/3119503236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e1ea00640 1 --2- 172.21.15.16:0/3119503236 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d9095e10 0x55d5d90f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e17e00640 1 --2- 172.21.15.16:0/3119503236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e19d0 0x55d5d90f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e1f400640 1 -- 172.21.15.16:0/3119503236 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d9095e10 msgr2=0x55d5d90f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:45.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e1f400640 1 --2- 172.21.15.16:0/3119503236 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d9095e10 0x55d5d90f3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e1f400640 1 -- 172.21.15.16:0/3119503236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e19d0 msgr2=0x55d5d90f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:45.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e1f400640 1 --2- 172.21.15.16:0/3119503236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e19d0 0x55d5d90f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e1f400640 1 -- 172.21.15.16:0/3119503236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5d8f3ed90 con 0x55d5d90e22f0 2024-09-10T21:10:45.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e1f400640 1 --2- 172.21.15.16:0/3119503236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e22f0 0x55d5d90c0b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800c700 tx=0x7f3e1805e400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800d6a45e144977b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e0f400640 1 -- 172.21.15.16:0/3119503236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1801f050 con 0x55d5d90e22f0 2024-09-10T21:10:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e0f400640 1 -- 172.21.15.16:0/3119503236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e18067d40 con 0x55d5d90e22f0 2024-09-10T21:10:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e0f400640 1 -- 172.21.15.16:0/3119503236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18004e30 con 0x55d5d90e22f0 2024-09-10T21:10:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 -- 172.21.15.16:0/3119503236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e22f0 msgr2=0x55d5d90c0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 --2- 172.21.15.16:0/3119503236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e22f0 0x55d5d90c0b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800c700 tx=0x7f3e1805e400 comp rx=0 tx=0).stop 2024-09-10T21:10:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 -- 172.21.15.16:0/3119503236 shutdown_connections 2024-09-10T21:10:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 --2- 172.21.15.16:0/3119503236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d90e22f0 0x55d5d90c0b50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 --2- 172.21.15.16:0/3119503236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e19d0 0x55d5d90f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 --2- 172.21.15.16:0/3119503236 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d9095e10 0x55d5d90f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 -- 172.21.15.16:0/3119503236 >> 172.21.15.16:0/3119503236 conn(0x55d5d8f7f130 msgr2=0x55d5d90940a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 -- 172.21.15.16:0/3119503236 shutdown_connections 2024-09-10T21:10:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.278+0000 7f3e292a2000 1 -- 172.21.15.16:0/3119503236 wait complete. 2024-09-10T21:10:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 1 -- start start 2024-09-10T21:10:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 1 Processor -- start 2024-09-10T21:10:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 0x55d5d90f0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d90e19d0 0x55d5d90f05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90f0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d5d8eaeec0 con 0x55d5d9095e10 2024-09-10T21:10:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d5d8eeb390 con 0x55d5d90e22f0 2024-09-10T21:10:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55d5d8f67a60 con 0x55d5d90e19d0 2024-09-10T21:10:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 10 client.0 _ll_register_callbacks cb 0x7f3e100c6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e1ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 0x55d5d90f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e1ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 0x55d5d90f0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36482/0 (socket says 172.21.15.16:36482) 2024-09-10T21:10:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e1ea00640 1 -- 172.21.15.16:0/413598570 learned_addr learned my addr 172.21.15.16:0/413598570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e17e00640 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d90e19d0 0x55d5d90f05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e1f400640 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90f0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e1ea00640 1 -- 172.21.15.16:0/413598570 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d90e19d0 msgr2=0x55d5d90f05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e1ea00640 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d90e19d0 0x55d5d90f05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e1ea00640 1 -- 172.21.15.16:0/413598570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 msgr2=0x55d5d90f0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e1ea00640 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90f0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e1ea00640 1 -- 172.21.15.16:0/413598570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5d8ea9a90 con 0x55d5d9095e10 2024-09-10T21:10:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e17e00640 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d90e19d0 0x55d5d90f05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e1ea00640 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 0x55d5d90f0090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3e10098340 tx=0x7f3e10005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ad029b7a58b69e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:10:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e0cc00640 1 -- 172.21.15.16:0/413598570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100b2040 con 0x55d5d9095e10 2024-09-10T21:10:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e0cc00640 1 -- 172.21.15.16:0/413598570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3e100c4ae0 con 0x55d5d9095e10 2024-09-10T21:10:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 1 -- 172.21.15.16:0/413598570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e18067d40 con 0x55d5d9095e10 2024-09-10T21:10:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e0cc00640 1 -- 172.21.15.16:0/413598570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f3e100c8040 con 0x55d5d9095e10 2024-09-10T21:10:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 10 client.4635 fetch_fsmap learned FSMap version 8 2024-09-10T21:10:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 1 -- 172.21.15.16:0/413598570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55d5d8e88050 con 0x55d5d9095e10 2024-09-10T21:10:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e0cc00640 1 -- 172.21.15.16:0/413598570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e18067d40 con 0x55d5d9095e10 2024-09-10T21:10:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e0cc00640 1 -- 172.21.15.16:0/413598570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f3e100d8070 con 0x55d5d9095e10 2024-09-10T21:10:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 10 client.4635 fetch_fsmap finished waiting for FSMap version 8 2024-09-10T21:10:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 10 client.4635 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:10:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 20 client.4635 populate_metadata read hostname 'smithi016' 2024-09-10T21:10:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.52:6833/3067177622,v1:172.21.15.52:6835/3067177622] conn(0x55d5d90f1440 0x55d5d91647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 4 client.4635 mds_command: new command op to 4356 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:10:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e292a2000 1 -- 172.21.15.16:0/413598570 --> [v2:172.21.15.52:6833/3067177622,v1:172.21.15.52:6835/3067177622] -- command(tid 0: {"prefix": "status"}) -- 0x55d5d8ef1520 con 0x55d5d90f1440 2024-09-10T21:10:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.282+0000 7f3e17e00640 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.52:6833/3067177622,v1:172.21.15.52:6835/3067177622] conn(0x55d5d90f1440 0x55d5d91647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e17e00640 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.52:6833/3067177622,v1:172.21.15.52:6835/3067177622] conn(0x55d5d90f1440 0x55d5d91647c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dccbb76091bd1803 server_cookie=504455672b75bdb4 in_seq=0 out_seq=0 2024-09-10T21:10:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e0cc00640 10 client.4635 ms_handle_connect on v2:172.21.15.52:6833/3067177622 2024-09-10T21:10:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e0cc00640 1 -- 172.21.15.16:0/413598570 <== mds.0 v2:172.21.15.52:6833/3067177622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55d5d8ef1520 con 0x55d5d90f1440 2024-09-10T21:10:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e0cc00640 10 client.4635 handle_command_reply: tid=0 2024-09-10T21:10:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e292a2000 1 client.4635 shutdown 2024-09-10T21:10:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e292a2000 1 -- 172.21.15.16:0/413598570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 msgr2=0x55d5d90f0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e292a2000 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 0x55d5d90f0090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3e10098340 tx=0x7f3e10005ca0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e292a2000 1 -- 172.21.15.16:0/413598570 shutdown_connections 2024-09-10T21:10:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e292a2000 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.52:6833/3067177622,v1:172.21.15.52:6835/3067177622] conn(0x55d5d90f1440 0x55d5d91647c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e292a2000 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5d90e22f0 0x55d5d90f0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e292a2000 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55d5d90e19d0 0x55d5d90f05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e292a2000 1 --2- 172.21.15.16:0/413598570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5d9095e10 0x55d5d90f0090 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e0cc00640 0 client.4635 ms_handle_reset on v2:172.21.15.52:6833/3067177622 2024-09-10T21:10:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e292a2000 1 -- 172.21.15.16:0/413598570 >> 172.21.15.16:0/413598570 conn(0x55d5d8f7f130 msgr2=0x55d5d90948c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e292a2000 1 -- 172.21.15.16:0/413598570 shutdown_connections 2024-09-10T21:10:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e292a2000 1 -- 172.21.15.16:0/413598570 wait complete. 2024-09-10T21:10:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e292a2000 20 client.4635 trim_cache size 0 max 16384 2024-09-10T21:10:45.295 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:10:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "eb19be60-6d8f-4816-b778-3d7d6698304d", 2024-09-10T21:10:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-10T21:10:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4356, 2024-09-10T21:10:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-10T21:10:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-10T21:10:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-10T21:10:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 15.127656387, 2024-09-10T21:10:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 8, 2024-09-10T21:10:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 21, 2024-09-10T21:10:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 21, 2024-09-10T21:10:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 21.157471162 2024-09-10T21:10:45.296 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:10:45.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e25e00640 1 -- 172.21.15.16:0/1270480102 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3dfc0725e0 msgr2=0x7f3dfc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:45.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e25e00640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3dfc0725e0 0x7f3dfc074aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3e100af000 tx=0x7f3e1001afa0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e25e00640 1 -- 172.21.15.16:0/1270480102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e2015f350 msgr2=0x7f3e200c5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:45.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e25e00640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e2015f350 0x7f3e200c5890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800ea20 tx=0x7f3e1800eee0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e25e00640 1 -- 172.21.15.16:0/1270480102 shutdown_connections 2024-09-10T21:10:45.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e25e00640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3dfc0725e0 0x7f3dfc074aa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e25e00640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e2015f350 0x7f3e200c5890 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e25e00640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e200d0700 0x7f3e20154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e25e00640 1 --2- 172.21.15.16:0/1270480102 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3e200cfb60 0x7f3e20143970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e25e00640 1 -- 172.21.15.16:0/1270480102 >> 172.21.15.16:0/1270480102 conn(0x7f3e200d1d80 msgr2=0x7f3e200ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:45.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e25e00640 1 -- 172.21.15.16:0/1270480102 shutdown_connections 2024-09-10T21:10:45.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.286+0000 7f3e25e00640 1 -- 172.21.15.16:0/1270480102 wait complete. 2024-09-10T21:10:45.311 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T21:10:45.337 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T21:10:45.337 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-10T21:10:45.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:45.338 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:10:45.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:45.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-10T21:10:45.410 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-09-10T21:10:45.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:10:45.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:45.413 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:10:45.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:45.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-10T21:10:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-09-10T21:10:45.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:10:45.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:45.537 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:10:45.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:45.548 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-09-10T21:10:45.614 INFO:teuthology.orchestra.run.smithi052.stderr:Cannot find device "ceph-brx" 2024-09-10T21:10:45.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:10:45.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:45.616 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:10:45.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:45.671 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-09-10T21:10:45.741 INFO:teuthology.orchestra.run.smithi052.stderr:Cannot find device "ceph-brx" 2024-09-10T21:10:45.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:10:45.744 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-09-10T21:10:45.744 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:10:45.744 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:10:45.744 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:10:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.886+0000 7fbc4a400640 1 -- 172.21.15.16:0/2713989758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc28004d50 msgr2=0x7fbc280051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.886+0000 7fbc4a400640 1 --2- 172.21.15.16:0/2713989758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc28004d50 0x7fbc280051d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009ee0 tx=0x7fbc3c01d630 comp rx=0 tx=0).stop 2024-09-10T21:10:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.886+0000 7fbc43e00640 1 -- 172.21.15.16:0/2713989758 reap_dead start 2024-09-10T21:10:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.886+0000 7fbc4a400640 1 -- 172.21.15.16:0/2713989758 shutdown_connections 2024-09-10T21:10:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.886+0000 7fbc4a400640 1 -- 172.21.15.16:0/2713989758 >> 172.21.15.16:0/2713989758 conn(0x7fbc440d1d80 msgr2=0x7fbc440d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.886+0000 7fbc4a400640 1 -- 172.21.15.16:0/2713989758 shutdown_connections 2024-09-10T21:10:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.886+0000 7fbc4a400640 1 -- 172.21.15.16:0/2713989758 wait complete. 2024-09-10T21:10:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc4a400640 1 Processor -- start 2024-09-10T21:10:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc4a400640 1 -- start start 2024-09-10T21:10:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc4a400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc28004d50 0x7fbc440b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc4a400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbc440cbdc0 0x7fbc441f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc4a400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc441f83e0 0x7fbc441f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc4a400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc440d0600 con 0x7fbc28004d50 2024-09-10T21:10:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc4a400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc440d0480 con 0x7fbc441f83e0 2024-09-10T21:10:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc4a400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fbc440d0780 con 0x7fbc440cbdc0 2024-09-10T21:10:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc43400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc28004d50 0x7fbc440b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc43400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc28004d50 0x7fbc440b55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36488/0 (socket says 172.21.15.16:36488) 2024-09-10T21:10:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc43400640 1 -- 172.21.15.16:0/2182701553 learned_addr learned my addr 172.21.15.16:0/2182701553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc3be00640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbc440cbdc0 0x7fbc441f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc43e00640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc441f83e0 0x7fbc441f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc43400640 1 -- 172.21.15.16:0/2182701553 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbc440cbdc0 msgr2=0x7fbc441f7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc43400640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbc440cbdc0 0x7fbc441f7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc43400640 1 -- 172.21.15.16:0/2182701553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc441f83e0 msgr2=0x7fbc441f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc43400640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc441f83e0 0x7fbc441f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc43400640 1 -- 172.21.15.16:0/2182701553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc440b5cc0 con 0x7fbc28004d50 2024-09-10T21:10:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc43e00640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc441f83e0 0x7fbc441f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc43400640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc28004d50 0x7fbc440b55c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc34012670 tx=0x7fbc34012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc3b400640 1 -- 172.21.15.16:0/2182701553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34010ae0 con 0x7fbc28004d50 2024-09-10T21:10:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc3b400640 1 -- 172.21.15.16:0/2182701553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc34017030 con 0x7fbc28004d50 2024-09-10T21:10:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc3b400640 1 -- 172.21.15.16:0/2182701553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34013450 con 0x7fbc28004d50 2024-09-10T21:10:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc4a400640 1 -- 172.21.15.16:0/2182701553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc440b5fa0 con 0x7fbc28004d50 2024-09-10T21:10:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc4a400640 1 -- 172.21.15.16:0/2182701553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc440b6560 con 0x7fbc28004d50 2024-09-10T21:10:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.890+0000 7fbc4a400640 1 -- 172.21.15.16:0/2182701553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc440af1e0 con 0x7fbc28004d50 2024-09-10T21:10:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.894+0000 7fbc3b400640 1 -- 172.21.15.16:0/2182701553 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbc340135f0 con 0x7fbc28004d50 2024-09-10T21:10:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.894+0000 7fbc3b400640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbc20072610 0x7fbc20074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.894+0000 7fbc3b400640 1 -- 172.21.15.16:0/2182701553 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbc34096fa0 con 0x7fbc28004d50 2024-09-10T21:10:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.894+0000 7fbc3be00640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbc20072610 0x7fbc20074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.894+0000 7fbc3be00640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbc20072610 0x7fbc20074ad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc440b6a00 tx=0x7fbc2c0091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:45.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:45.898+0000 7fbc3b400640 1 -- 172.21.15.16:0/2182701553 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbc340625e0 con 0x7fbc28004d50 2024-09-10T21:10:46.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.110+0000 7fbc4a400640 1 -- 172.21.15.16:0/2182701553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbc440cc1a0 con 0x7fbc28004d50 2024-09-10T21:10:46.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.110+0000 7fbc3b400640 1 -- 172.21.15.16:0/2182701553 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fbc34061ee0 con 0x7fbc28004d50 2024-09-10T21:10:46.116 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:10:46.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.114+0000 7fbc4a400640 1 -- 172.21.15.16:0/2182701553 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbc20072610 msgr2=0x7fbc20074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:46.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.114+0000 7fbc4a400640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbc20072610 0x7fbc20074ad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc440b6a00 tx=0x7fbc2c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:10:46.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.114+0000 7fbc4a400640 1 -- 172.21.15.16:0/2182701553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc28004d50 msgr2=0x7fbc440b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:46.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.114+0000 7fbc4a400640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc28004d50 0x7fbc440b55c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc34012670 tx=0x7fbc34012b30 comp rx=0 tx=0).stop 2024-09-10T21:10:46.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.114+0000 7fbc4a400640 1 -- 172.21.15.16:0/2182701553 shutdown_connections 2024-09-10T21:10:46.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.114+0000 7fbc4a400640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbc20072610 0x7fbc20074ad0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:46.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.114+0000 7fbc4a400640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc441f83e0 0x7fbc441f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:46.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.114+0000 7fbc4a400640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbc440cbdc0 0x7fbc441f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:46.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.114+0000 7fbc4a400640 1 --2- 172.21.15.16:0/2182701553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc28004d50 0x7fbc440b55c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:46.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.114+0000 7fbc4a400640 1 -- 172.21.15.16:0/2182701553 >> 172.21.15.16:0/2182701553 conn(0x7fbc440d1d80 msgr2=0x7fbc440cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:46.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.114+0000 7fbc4a400640 1 -- 172.21.15.16:0/2182701553 shutdown_connections 2024-09-10T21:10:46.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46.114+0000 7fbc4a400640 1 -- 172.21.15.16:0/2182701553 wait complete. 2024-09-10T21:10:46.137 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:10:46.137 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:10:46.137 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-10T21:10:46.137 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:10:46.137 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:10:46.137 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:10:46.138 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:10:46.138 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:10:46.138 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:10:46.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:46.138 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-10T21:10:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:10:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:10:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:10:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:10:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:10:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-10T21:10:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:10:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-10T21:10:46.148 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:10:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-10T21:10:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:10:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42403sec preferred_lft 42403sec 2024-09-10T21:10:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-10T21:10:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:46.148 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:10:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-10T21:10:46.148 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:10:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:30:fb:74:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:10:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:46.150 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T21:10:46.150 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:46.150 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:10:46.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-10T21:10:46.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-10T21:10:46.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-10T21:10:46.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T21:10:46.150 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:10:46.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:46.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:46.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:46.307 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:10:46.326 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-10T21:10:46.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:46.328 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-10T21:10:46.398 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-10T21:10:46.399 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:10:46.399 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:10:46.399 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:10:46.399 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:10:46.399 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:10:46.399 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:46.400 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:10:46.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:10:46.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:10:46.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:10:46.400 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:10:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:46.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:46.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:46.540 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:10:46.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:46.550 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-10T21:10:46.606 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:46.606 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:10:46.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:10:46.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T21:10:46.606 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:10:46.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:46.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:46.738 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T21:10:46.739 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:46.739 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:10:46.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T21:10:46.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T21:10:46.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T21:10:46.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T21:10:46.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T21:10:46.739 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:10:46.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:46.909 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:46.909 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:10:46.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-10T21:10:46.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T21:10:46.909 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:10:46.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:46.995 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:10:46.995 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:10:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:10:47.050 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:10:47.050 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:10:47.100 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:10:47.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:47.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:10:47.190 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:10:47.190 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:10:47.875 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:10:47.876 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:10:47.876 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:10:47.876 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:10:47.876 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:10:47.880 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:10:47.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:47.881 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:10:47.904 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:10:48.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:10:48.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:48.028 INFO:teuthology.orchestra.run.smithi016.stdout:client4636 2024-09-10T21:10:48.029 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:10:48.029 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:48.038 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:10:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:10:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:48.150 INFO:teuthology.orchestra.run.smithi016.stdout:client4636 2024-09-10T21:10:48.151 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:10:48.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:10:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:48.218 INFO:teuthology.orchestra.run.smithi016.stdout:client4636 2024-09-10T21:10:48.219 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb19be60-6d8f-4816-b778-3d7d6698304d.client4636/status') 2024-09-10T21:10:48.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:48.287 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.4636 (3)172.21.15.16:0/3118031991 2024-09-10T21:10:48.287 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-09-10T21:10:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-09-10T21:10:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-09-10T21:10:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:70 bytes copied, 5.2079e-05 s, 1.3 MB/s 2024-09-10T21:10:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:48.295 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-09-10T21:10:48.295 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:10:48.295 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T21:10:48.296 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:10:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 -- 172.21.15.16:0/2381922374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a00b3200 msgr2=0x7fc3a00d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 --2- 172.21.15.16:0/2381922374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a00b3200 0x7fc3a00d01c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00b3810 tx=0x7fc38c01f090 comp rx=0 tx=0).stop 2024-09-10T21:10:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 -- 172.21.15.16:0/2381922374 shutdown_connections 2024-09-10T21:10:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 --2- 172.21.15.16:0/2381922374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a00b3200 0x7fc3a00d01c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 --2- 172.21.15.16:0/2381922374 >> v2:172.21.15.52:3300/0 conn(0x7fc3a015f450 0x7fc3a00cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 -- 172.21.15.16:0/2381922374 >> 172.21.15.16:0/2381922374 conn(0x7fc3a00d1d80 msgr2=0x7fc3a00d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 -- 172.21.15.16:0/2381922374 shutdown_connections 2024-09-10T21:10:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 -- 172.21.15.16:0/2381922374 wait complete. 2024-09-10T21:10:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 Processor -- start 2024-09-10T21:10:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 -- start start 2024-09-10T21:10:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a00d0890 0x7fc3a00ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3a015f450 0x7fc3a01597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3a00cf0f0 0x7fc3a01f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a00b2e30 con 0x7fc3a00d0890 2024-09-10T21:10:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a00b2cb0 con 0x7fc3a015f450 2024-09-10T21:10:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a7600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a00b2fb0 con 0x7fc3a00cf0f0 2024-09-10T21:10:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a00d0890 0x7fc3a00ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a00d0890 0x7fc3a00ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36510/0 (socket says 172.21.15.16:36510) 2024-09-10T21:10:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a4c00640 1 -- 172.21.15.16:0/1155711050 learned_addr learned my addr 172.21.15.16:0/1155711050 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a4c00640 1 -- 172.21.15.16:0/1155711050 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3a00cf0f0 msgr2=0x7fc3a01f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc39fe00640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3a015f450 0x7fc3a01597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a4c00640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3a00cf0f0 0x7fc3a01f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a4c00640 1 -- 172.21.15.16:0/1155711050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3a015f450 msgr2=0x7fc3a01597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a4c00640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3a015f450 0x7fc3a01597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc3a4c00640 1 -- 172.21.15.16:0/1155711050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a0159d30 con 0x7fc3a00d0890 2024-09-10T21:10:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.437+0000 7fc39fe00640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3a015f450 0x7fc3a01597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.441+0000 7fc3a4c00640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a00d0890 0x7fc3a00ca770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc39400b410 tx=0x7fc39400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.441+0000 7fc39cc00640 1 -- 172.21.15.16:0/1155711050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394015020 con 0x7fc3a00d0890 2024-09-10T21:10:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.441+0000 7fc39cc00640 1 -- 172.21.15.16:0/1155711050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc394011920 con 0x7fc3a00d0890 2024-09-10T21:10:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.441+0000 7fc3a7600640 1 -- 172.21.15.16:0/1155711050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a00cad50 con 0x7fc3a00d0890 2024-09-10T21:10:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.441+0000 7fc39cc00640 1 -- 172.21.15.16:0/1155711050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394011d70 con 0x7fc3a00d0890 2024-09-10T21:10:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.441+0000 7fc3a7600640 1 -- 172.21.15.16:0/1155711050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a00cb310 con 0x7fc3a00d0890 2024-09-10T21:10:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.441+0000 7fc3a7600640 1 -- 172.21.15.16:0/1155711050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a00b1e50 con 0x7fc3a00d0890 2024-09-10T21:10:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.441+0000 7fc39cc00640 1 -- 172.21.15.16:0/1155711050 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc3940193b0 con 0x7fc3a00d0890 2024-09-10T21:10:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.441+0000 7fc39cc00640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc3800725e0 0x7fc380074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.441+0000 7fc39cc00640 1 -- 172.21.15.16:0/1155711050 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc394093230 con 0x7fc3a00d0890 2024-09-10T21:10:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.441+0000 7fc39fe00640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc3800725e0 0x7fc380074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.445+0000 7fc39fe00640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc3800725e0 0x7fc380074aa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0b2000 tx=0x7fc38c01d8d0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.445+0000 7fc39cc00640 1 -- 172.21.15.16:0/1155711050 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc39401f030 con 0x7fc3a00d0890 2024-09-10T21:10:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc3a7600640 1 -- 172.21.15.16:0/1155711050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc3a015e680 con 0x7fc3a00d0890 2024-09-10T21:10:48.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc39cc00640 1 -- 172.21.15.16:0/1155711050 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fc39405e870 con 0x7fc3a00d0890 2024-09-10T21:10:48.666 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:10:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc3a7600640 1 -- 172.21.15.16:0/1155711050 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc3800725e0 msgr2=0x7fc380074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc3a7600640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc3800725e0 0x7fc380074aa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0b2000 tx=0x7fc38c01d8d0 comp rx=0 tx=0).stop 2024-09-10T21:10:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc3a7600640 1 -- 172.21.15.16:0/1155711050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a00d0890 msgr2=0x7fc3a00ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:48.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc3a7600640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a00d0890 0x7fc3a00ca770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc39400b410 tx=0x7fc39400b8d0 comp rx=0 tx=0).stop 2024-09-10T21:10:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc3a7600640 1 -- 172.21.15.16:0/1155711050 shutdown_connections 2024-09-10T21:10:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc3a7600640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc3800725e0 0x7fc380074aa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc3a7600640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3a00cf0f0 0x7fc3a01f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc3a7600640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3a015f450 0x7fc3a01597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc3a7600640 1 --2- 172.21.15.16:0/1155711050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a00d0890 0x7fc3a00ca770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc3a7600640 1 -- 172.21.15.16:0/1155711050 >> 172.21.15.16:0/1155711050 conn(0x7fc3a00d1d80 msgr2=0x7fc3a0070960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc3a7600640 1 -- 172.21.15.16:0/1155711050 shutdown_connections 2024-09-10T21:10:48.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48.661+0000 7fc3a7600640 1 -- 172.21.15.16:0/1155711050 wait complete. 2024-09-10T21:10:48.688 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:10:48.688 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:10:48.689 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-10T21:10:48.689 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-10T21:10:48.689 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T21:10:48.689 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:10:48.689 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:10:48.689 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:10:48.689 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-10T21:10:48.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:48.689 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-10T21:10:48.698 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:10:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:10:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:10:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:10:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:48.698 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:10:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-10T21:10:48.699 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:10:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-10T21:10:48.699 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:10:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-10T21:10:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:10:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42401sec preferred_lft 42401sec 2024-09-10T21:10:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-10T21:10:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:48.699 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:10:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:30:fb:74:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:10:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:48.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:48.701 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:10:48.766 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-10T21:10:48.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:48.770 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-10T21:10:48.822 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-10T21:10:48.822 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:10:48.822 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:10:48.822 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:10:48.822 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:10:48.822 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:10:48.823 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:48.823 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:10:48.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:10:48.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:10:48.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:10:48.823 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:10:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:48.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:48.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:48.961 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:10:48.970 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:10:48.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:48.971 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-10T21:10:49.025 INFO:teuthology.orchestra.run.smithi016.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-10T21:10:49.026 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:49.026 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:10:49.026 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:10:49.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-10T21:10:49.027 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:10:49.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:49.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:49.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T21:10:49.174 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:10:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:10:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:10:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:10:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:49.175 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:10:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 76:c5:cb:e4:77:4d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T21:10:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T21:10:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::74c5:cbff:fee4:774d/64 scope link 2024-09-10T21:10:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:49.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:49.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T21:10:49.241 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:10:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:10:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:10:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:10:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:49.241 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:10:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 76:c5:cb:e4:77:4d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T21:10:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T21:10:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::74c5:cbff:fee4:774d/64 scope link 2024-09-10T21:10:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:49.243 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-10T21:10:49.243 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:49.243 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:10:49.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-10T21:10:49.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T21:10:49.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-10T21:10:49.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-10T21:10:49.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-10T21:10:49.244 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:10:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:49.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:49.424 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:49.424 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:10:49.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.1 up 2024-09-10T21:10:49.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T21:10:49.425 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:10:49.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:49.554 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:10:49.554 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-10T21:10:49.563 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-10T21:10:49.563 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:10:49.563 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-10T21:10:49.612 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-09-10T21:10:49.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:49.613 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:10:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:10:49.702 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:10:49.715 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:10:49.715 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:10:49.715 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:10:49.715 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 2@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:10:49.716 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:10:49.719 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:10:49.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:49.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-10T21:10:49.739 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:10:49.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:10:49.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:49.844 INFO:teuthology.orchestra.run.smithi016.stdout:client4639 2024-09-10T21:10:49.844 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:10:49.844 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:49.853 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:10:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:10:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:49.954 INFO:teuthology.orchestra.run.smithi016.stdout:client4639 2024-09-10T21:10:49.954 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:10:50.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:10:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:50.016 INFO:teuthology.orchestra.run.smithi016.stdout:client4639 2024-09-10T21:10:50.017 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb19be60-6d8f-4816-b778-3d7d6698304d.client4639/status') 2024-09-10T21:10:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:50.077 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.4639 (3)172.21.15.16:0/3298223107 2024-09-10T21:10:50.077 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-09-10T21:10:50.077 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-09-10T21:10:50.077 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-09-10T21:10:50.077 INFO:teuthology.orchestra.run.smithi016.stderr:70 bytes copied, 7.0189e-05 s, 997 kB/s 2024-09-10T21:10:50.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:50.085 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-09-10T21:10:50.085 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:10:50.085 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:10:50.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:10:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.225+0000 7f9a51e00640 1 -- 172.21.15.16:0/3107798022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a4c15ac50 msgr2=0x7f9a4c073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.225+0000 7f9a51e00640 1 --2- 172.21.15.16:0/3107798022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a4c15ac50 0x7f9a4c073750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009900 tx=0x7f9a3c01c850 comp rx=0 tx=0).stop 2024-09-10T21:10:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.225+0000 7f9a51e00640 1 -- 172.21.15.16:0/3107798022 shutdown_connections 2024-09-10T21:10:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.225+0000 7f9a51e00640 1 --2- 172.21.15.16:0/3107798022 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a4c0d06e0 0x7f9a4c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.225+0000 7f9a51e00640 1 --2- 172.21.15.16:0/3107798022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a4c15ac50 0x7f9a4c073750 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.225+0000 7f9a51e00640 1 -- 172.21.15.16:0/3107798022 >> 172.21.15.16:0/3107798022 conn(0x7f9a4c0d0c10 msgr2=0x7f9a4c0d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.225+0000 7f9a51e00640 1 -- 172.21.15.16:0/3107798022 shutdown_connections 2024-09-10T21:10:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.225+0000 7f9a51e00640 1 -- 172.21.15.16:0/3107798022 wait complete. 2024-09-10T21:10:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a51e00640 1 Processor -- start 2024-09-10T21:10:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a51e00640 1 -- start start 2024-09-10T21:10:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a51e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a4c0cff40 0x7f9a4c0c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a51e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a4c0d06e0 0x7f9a4c159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a51e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a4c15ac50 0x7f9a4c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a51e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c167fa0 con 0x7f9a4c0cff40 2024-09-10T21:10:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a51e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c167e20 con 0x7f9a4c0d06e0 2024-09-10T21:10:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a51e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4c168120 con 0x7f9a4c15ac50 2024-09-10T21:10:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a4aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a4c0cff40 0x7f9a4c0c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a4aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a4c0cff40 0x7f9a4c0c46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36528/0 (socket says 172.21.15.16:36528) 2024-09-10T21:10:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a4aa00640 1 -- 172.21.15.16:0/2485073629 learned_addr learned my addr 172.21.15.16:0/2485073629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a42000640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a4c0d06e0 0x7f9a4c159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a4b400640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a4c15ac50 0x7f9a4c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a4aa00640 1 -- 172.21.15.16:0/2485073629 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a4c15ac50 msgr2=0x7f9a4c159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a4aa00640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a4c15ac50 0x7f9a4c159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a4aa00640 1 -- 172.21.15.16:0/2485073629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a4c0d06e0 msgr2=0x7f9a4c159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a4aa00640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a4c0d06e0 0x7f9a4c159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a4aa00640 1 -- 172.21.15.16:0/2485073629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c0b4c10 con 0x7f9a4c0cff40 2024-09-10T21:10:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a42000640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a4c0d06e0 0x7f9a4c159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a4b400640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a4c15ac50 0x7f9a4c159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a4aa00640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a4c0cff40 0x7f9a4c0c46d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0b0040 tx=0x7f9a3c0a5520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a43e00640 1 -- 172.21.15.16:0/2485073629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0bb070 con 0x7f9a4c0cff40 2024-09-10T21:10:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a43e00640 1 -- 172.21.15.16:0/2485073629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a3c0043b0 con 0x7f9a4c0cff40 2024-09-10T21:10:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a43e00640 1 -- 172.21.15.16:0/2485073629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0b2040 con 0x7f9a4c0cff40 2024-09-10T21:10:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a51e00640 1 -- 172.21.15.16:0/2485073629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c0b4ed0 con 0x7f9a4c0cff40 2024-09-10T21:10:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.229+0000 7f9a51e00640 1 -- 172.21.15.16:0/2485073629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c1658d0 con 0x7f9a4c0cff40 2024-09-10T21:10:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.233+0000 7f9a51e00640 1 -- 172.21.15.16:0/2485073629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4c0d18e0 con 0x7f9a4c0cff40 2024-09-10T21:10:50.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.233+0000 7f9a43e00640 1 -- 172.21.15.16:0/2485073629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9a3c005270 con 0x7f9a4c0cff40 2024-09-10T21:10:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.233+0000 7f9a43e00640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9a28072610 0x7f9a28074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.233+0000 7f9a43e00640 1 -- 172.21.15.16:0/2485073629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9a3c12ca40 con 0x7f9a4c0cff40 2024-09-10T21:10:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.233+0000 7f9a42000640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9a28072610 0x7f9a28074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:50.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.237+0000 7f9a42000640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9a28072610 0x7f9a28074ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9a34005fd0 tx=0x7f9a34005e80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:50.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.237+0000 7f9a43e00640 1 -- 172.21.15.16:0/2485073629 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9a3c0164d0 con 0x7f9a4c0cff40 2024-09-10T21:10:50.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.449+0000 7f9a51e00640 1 -- 172.21.15.16:0/2485073629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9a4c164430 con 0x7f9a4c0cff40 2024-09-10T21:10:50.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.449+0000 7f9a43e00640 1 -- 172.21.15.16:0/2485073629 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f9a3c0f8110 con 0x7f9a4c0cff40 2024-09-10T21:10:50.456 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.453+0000 7f9a51e00640 1 -- 172.21.15.16:0/2485073629 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9a28072610 msgr2=0x7f9a28074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.453+0000 7f9a51e00640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9a28072610 0x7f9a28074ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9a34005fd0 tx=0x7f9a34005e80 comp rx=0 tx=0).stop 2024-09-10T21:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.453+0000 7f9a51e00640 1 -- 172.21.15.16:0/2485073629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a4c0cff40 msgr2=0x7f9a4c0c46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.453+0000 7f9a51e00640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a4c0cff40 0x7f9a4c0c46d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0b0040 tx=0x7f9a3c0a5520 comp rx=0 tx=0).stop 2024-09-10T21:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.453+0000 7f9a51e00640 1 -- 172.21.15.16:0/2485073629 shutdown_connections 2024-09-10T21:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.453+0000 7f9a51e00640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9a28072610 0x7f9a28074ad0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.453+0000 7f9a51e00640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a4c15ac50 0x7f9a4c159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.453+0000 7f9a51e00640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a4c0d06e0 0x7f9a4c159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.453+0000 7f9a51e00640 1 --2- 172.21.15.16:0/2485073629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a4c0cff40 0x7f9a4c0c46d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.453+0000 7f9a51e00640 1 -- 172.21.15.16:0/2485073629 >> 172.21.15.16:0/2485073629 conn(0x7f9a4c0d0c10 msgr2=0x7f9a4c0cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.453+0000 7f9a51e00640 1 -- 172.21.15.16:0/2485073629 shutdown_connections 2024-09-10T21:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:50.453+0000 7f9a51e00640 1 -- 172.21.15.16:0/2485073629 wait complete. 2024-09-10T21:10:50.476 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:10:50.477 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:10:50.477 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-09-10T21:10:50.477 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T21:10:50.477 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:10:50.477 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:10:50.477 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:10:50.477 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:10:50.477 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:10:50.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:50.477 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-09-10T21:10:50.487 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:10:50.488 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:10:50.488 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:10:50.488 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:50.488 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:10:50.488 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:50.488 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:10:50.488 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:50.488 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-09-10T21:10:50.488 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:10:50.488 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:50.488 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-09-10T21:10:50.489 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:10:50.489 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:50.489 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-09-10T21:10:50.489 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:10:50.489 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42400sec preferred_lft 42400sec 2024-09-10T21:10:50.489 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-09-10T21:10:50.489 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:50.489 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:10:50.489 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:50.489 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-09-10T21:10:50.489 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:10:50.489 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:7e:b4:71:a2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:50.489 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:10:50.490 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:50.490 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T21:10:50.491 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:50.491 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:10:50.491 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add name ceph-brx type bridge 2024-09-10T21:10:50.491 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr flush dev ceph-brx 2024-09-10T21:10:50.491 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set ceph-brx up 2024-09-10T21:10:50.491 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T21:10:50.491 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:10:50.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:50.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:50.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:50.664 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:10:50.683 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-09-10T21:10:50.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:50.686 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-09-10T21:10:50.770 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-09-10T21:10:50.770 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:10:50.771 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:10:50.771 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:10:50.771 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:10:50.771 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:10:50.771 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:50.771 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:10:50.771 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:10:50.771 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:10:50.771 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:10:50.771 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:10:50.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:50.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:50.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:50.910 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:10:50.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:50.920 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-09-10T21:10:50.970 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:50.970 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:10:50.970 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:10:50.970 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-10T21:10:50.970 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:10:51.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:51.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:51.104 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-10T21:10:51.104 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:51.104 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:10:51.104 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-10T21:10:51.104 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T21:10:51.104 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-10T21:10:51.105 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-10T21:10:51.105 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-10T21:10:51.105 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:10:51.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:51.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:51.273 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:51.273 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:10:51.273 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-09-10T21:10:51.273 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T21:10:51.273 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:10:51.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:51.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:51.360 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:10:51.360 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T21:10:51.409 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:10:51.410 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:10:51.410 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:10:51.461 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T21:10:51.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:51.461 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:10:51.550 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:10:51.550 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:10:52.307 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:10:52.307 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:10:52.307 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:10:52.307 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 1@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:10:52.307 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:10:52.311 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:10:52.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:52.311 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:10:52.331 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:10:52.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:10:52.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:52.444 INFO:teuthology.orchestra.run.smithi052.stdout:client134396 2024-09-10T21:10:52.444 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:10:52.444 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:52.453 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:10:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:10:52.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:52.554 INFO:teuthology.orchestra.run.smithi052.stdout:client134396 2024-09-10T21:10:52.555 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:10:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:10:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:52.654 INFO:teuthology.orchestra.run.smithi052.stdout:client134396 2024-09-10T21:10:52.654 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb19be60-6d8f-4816-b778-3d7d6698304d.client134396/status') 2024-09-10T21:10:52.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:52.714 INFO:teuthology.orchestra.run.smithi052.stdout:instance: client.134396 (3)192.168.0.1:0/1848242217 2024-09-10T21:10:52.714 INFO:teuthology.orchestra.run.smithi052.stdout:blocklisted: false 2024-09-10T21:10:52.715 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records in 2024-09-10T21:10:52.715 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records out 2024-09-10T21:10:52.715 INFO:teuthology.orchestra.run.smithi052.stderr:71 bytes copied, 6.3983e-05 s, 1.1 MB/s 2024-09-10T21:10:52.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:52.722 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-09-10T21:10:52.723 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:10:52.723 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T21:10:52.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:10:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 -- 172.21.15.16:0/3052190043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6178004810 msgr2=0x7f6178004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 --2- 172.21.15.16:0/3052190043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6178004810 0x7f6178004c90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f617c00bbb0 tx=0x7f617c01cda0 comp rx=0 tx=0).stop 2024-09-10T21:10:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 -- 172.21.15.16:0/3052190043 shutdown_connections 2024-09-10T21:10:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 --2- 172.21.15.16:0/3052190043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6178004810 0x7f6178004c90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 -- 172.21.15.16:0/3052190043 >> 172.21.15.16:0/3052190043 conn(0x7f619409b130 msgr2=0x7f61940d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 -- 172.21.15.16:0/3052190043 shutdown_connections 2024-09-10T21:10:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 -- 172.21.15.16:0/3052190043 wait complete. 2024-09-10T21:10:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 Processor -- start 2024-09-10T21:10:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 -- start start 2024-09-10T21:10:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6178004810 0x7f6194158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61940b69b0 0x7f61941550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61940b6e30 0x7f6194155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61940712b0 con 0x7f61940b69b0 2024-09-10T21:10:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6194071130 con 0x7f61940b6e30 2024-09-10T21:10:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6199200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f61940d26a0 con 0x7f6178004810 2024-09-10T21:10:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f618be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61940b69b0 0x7f61941550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f618be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61940b69b0 0x7f61941550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59772/0 (socket says 172.21.15.16:59772) 2024-09-10T21:10:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f618be00640 1 -- 172.21.15.16:0/661114419 learned_addr learned my addr 172.21.15.16:0/661114419 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6192800640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61940b6e30 0x7f6194155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f6191e00640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6178004810 0x7f6194158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f618be00640 1 -- 172.21.15.16:0/661114419 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6178004810 msgr2=0x7f6194158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f618be00640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6178004810 0x7f6194158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f618be00640 1 -- 172.21.15.16:0/661114419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61940b6e30 msgr2=0x7f6194155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f618be00640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61940b6e30 0x7f6194155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.877+0000 7f618be00640 1 -- 172.21.15.16:0/661114419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61940b36b0 con 0x7f61940b69b0 2024-09-10T21:10:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.881+0000 7f618be00640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61940b69b0 0x7f61941550e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f617c0b1040 tx=0x7f617c005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.881+0000 7f618a000640 1 -- 172.21.15.16:0/661114419 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c0be070 con 0x7f61940b69b0 2024-09-10T21:10:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.881+0000 7f618a000640 1 -- 172.21.15.16:0/661114419 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f617c0049d0 con 0x7f61940b69b0 2024-09-10T21:10:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.881+0000 7f618a000640 1 -- 172.21.15.16:0/661114419 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c0ab050 con 0x7f61940b69b0 2024-09-10T21:10:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.881+0000 7f6199200640 1 -- 172.21.15.16:0/661114419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61940b3950 con 0x7f61940b69b0 2024-09-10T21:10:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.881+0000 7f6199200640 1 -- 172.21.15.16:0/661114419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61940b3f10 con 0x7f61940b69b0 2024-09-10T21:10:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.881+0000 7f6199200640 1 -- 172.21.15.16:0/661114419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61940d2b70 con 0x7f61940b69b0 2024-09-10T21:10:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.881+0000 7f618a000640 1 -- 172.21.15.16:0/661114419 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f617c0af400 con 0x7f61940b69b0 2024-09-10T21:10:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.881+0000 7f618a000640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f61700725e0 0x7f6170074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.881+0000 7f618a000640 1 -- 172.21.15.16:0/661114419 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f617c12d9b0 con 0x7f61940b69b0 2024-09-10T21:10:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.885+0000 7f6191e00640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f61700725e0 0x7f6170074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.885+0000 7f6191e00640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f61700725e0 0x7f6170074aa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6194158910 tx=0x7f6184006b80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:52.889+0000 7f618a000640 1 -- 172.21.15.16:0/661114419 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f617c016af0 con 0x7f61940b69b0 2024-09-10T21:10:53.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f6199200640 1 -- 172.21.15.16:0/661114419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f619409b5d0 con 0x7f61940b69b0 2024-09-10T21:10:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f618a000640 1 -- 172.21.15.16:0/661114419 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f617c0f8a30 con 0x7f61940b69b0 2024-09-10T21:10:53.107 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:10:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f6199200640 1 -- 172.21.15.16:0/661114419 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f61700725e0 msgr2=0x7f6170074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f6199200640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f61700725e0 0x7f6170074aa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6194158910 tx=0x7f6184006b80 comp rx=0 tx=0).stop 2024-09-10T21:10:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f6199200640 1 -- 172.21.15.16:0/661114419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61940b69b0 msgr2=0x7f61941550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f6199200640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61940b69b0 0x7f61941550e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f617c0b1040 tx=0x7f617c005630 comp rx=0 tx=0).stop 2024-09-10T21:10:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f6199200640 1 -- 172.21.15.16:0/661114419 shutdown_connections 2024-09-10T21:10:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f6199200640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f61700725e0 0x7f6170074aa0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f6199200640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61940b6e30 0x7f6194155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f6199200640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61940b69b0 0x7f61941550e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f6199200640 1 --2- 172.21.15.16:0/661114419 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6178004810 0x7f6194158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f6199200640 1 -- 172.21.15.16:0/661114419 >> 172.21.15.16:0/661114419 conn(0x7f619409b130 msgr2=0x7f61940c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f6199200640 1 -- 172.21.15.16:0/661114419 shutdown_connections 2024-09-10T21:10:53.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:53.101+0000 7f6199200640 1 -- 172.21.15.16:0/661114419 wait complete. 2024-09-10T21:10:53.129 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:10:53.129 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:10:53.129 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-09-10T21:10:53.129 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-10T21:10:53.129 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T21:10:53.129 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:10:53.129 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:10:53.129 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:10:53.129 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-10T21:10:53.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:53.130 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-09-10T21:10:53.139 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:10:53.139 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:10:53.139 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:10:53.139 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.139 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:10:53.139 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.139 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:10:53.139 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:53.139 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-09-10T21:10:53.139 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:10:53.140 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:53.140 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-09-10T21:10:53.140 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:10:53.140 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:53.140 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-09-10T21:10:53.140 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:10:53.140 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42397sec preferred_lft 42397sec 2024-09-10T21:10:53.140 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-09-10T21:10:53.140 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.140 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:10:53.140 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:53.140 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-09-10T21:10:53.140 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:10:53.141 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:7e:b4:71:a2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:53.141 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:10:53.141 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.141 INFO:teuthology.orchestra.run.smithi052.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:10:53.141 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:10:53.141 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:10:53.141 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.141 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:10:53.141 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.141 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:10:53.141 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:10:53.141 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:10:53.141 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:53.142 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:10:53.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:10:53.150 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:10:53.208 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-09-10T21:10:53.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:53.210 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-09-10T21:10:53.294 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-09-10T21:10:53.294 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:10:53.294 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:10:53.294 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:10:53.294 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:10:53.295 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:10:53.295 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:53.295 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:10:53.295 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:10:53.295 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:10:53.295 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:10:53.295 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:10:53.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:53.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:53.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:53.435 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:10:53.445 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:10:53.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:53.445 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-09-10T21:10:53.497 INFO:teuthology.orchestra.run.smithi052.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-10T21:10:53.499 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:53.499 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:10:53.499 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:10:53.499 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-10T21:10:53.499 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:10:53.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:53.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:53.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:53.626 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T21:10:53.690 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:10:53.690 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:10:53.690 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:10:53.690 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.690 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:10:53.690 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.690 INFO:teuthology.orchestra.run.smithi052.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:10:53.690 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether ae:7b:2c:da:5a:29 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T21:10:53.690 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T21:10:53.690 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.691 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ac7b:2cff:feda:5a29/64 scope link 2024-09-10T21:10:53.691 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:53.692 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T21:10:53.758 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:10:53.758 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:10:53.758 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:10:53.758 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.758 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:10:53.759 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.759 INFO:teuthology.orchestra.run.smithi052.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:10:53.759 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether ae:7b:2c:da:5a:29 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T21:10:53.759 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T21:10:53.759 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.760 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ac7b:2cff:feda:5a29/64 scope link 2024-09-10T21:10:53.760 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:10:53.761 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-10T21:10:53.762 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:53.762 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:10:53.762 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-10T21:10:53.763 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T21:10:53.763 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-10T21:10:53.763 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-10T21:10:53.763 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-10T21:10:53.763 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:10:53.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:53.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:53.944 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:10:53.944 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:10:53.944 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.1 up 2024-09-10T21:10:53.944 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T21:10:53.944 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:10:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:54.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:54.030 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:10:54.031 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-10T21:10:54.039 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-10T21:10:54.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:10:54.039 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-10T21:10:54.092 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-09-10T21:10:54.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:54.093 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:10:54.185 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:10:54.185 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:10:54.198 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:10:54.198 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:10:54.198 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:10:54.198 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 3@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:10:54.199 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:10:54.202 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:10:54.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:54.202 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-10T21:10:54.222 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:10:54.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:10:54.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:54.322 INFO:teuthology.orchestra.run.smithi052.stdout:client4665 2024-09-10T21:10:54.322 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:10:54.323 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:54.331 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:10:54.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:10:54.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:54.429 INFO:teuthology.orchestra.run.smithi052.stdout:client4665 2024-09-10T21:10:54.430 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:10:54.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:10:54.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:54.487 INFO:teuthology.orchestra.run.smithi052.stdout:client4665 2024-09-10T21:10:54.487 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb19be60-6d8f-4816-b778-3d7d6698304d.client4665/status') 2024-09-10T21:10:54.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:10:54.547 INFO:teuthology.orchestra.run.smithi052.stdout:instance: client.4665 (3)172.21.15.52:0/981016284 2024-09-10T21:10:54.548 INFO:teuthology.orchestra.run.smithi052.stdout:blocklisted: false 2024-09-10T21:10:54.548 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records in 2024-09-10T21:10:54.548 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records out 2024-09-10T21:10:54.548 INFO:teuthology.orchestra.run.smithi052.stderr:69 bytes copied, 6.448e-05 s, 1.1 MB/s 2024-09-10T21:10:54.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:10:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:10:54.556 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-10T21:10:54.603 INFO:tasks.cephfs_test_runner:Starting test: test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) Mostly readdir acquires caps faster than the mds recalls, so the cap 2024-09-10T21:10:54.603 INFO:tasks.cephfs_test_runner:test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-10T21:10:54.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:10:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.753+0000 7fe185000640 1 -- 172.21.15.16:0/747314414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800726b0 msgr2=0x7fe180072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.753+0000 7fe185000640 1 --2- 172.21.15.16:0/747314414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800726b0 0x7fe180072b40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe178009bc0 tx=0x7fe17801cba0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.753+0000 7fe185000640 1 -- 172.21.15.16:0/747314414 shutdown_connections 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.753+0000 7fe185000640 1 --2- 172.21.15.16:0/747314414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1800726b0 0x7fe180072b40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.753+0000 7fe185000640 1 -- 172.21.15.16:0/747314414 >> 172.21.15.16:0/747314414 conn(0x7fe1800cfb20 msgr2=0x7fe1800cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.753+0000 7fe185000640 1 -- 172.21.15.16:0/747314414 shutdown_connections 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.753+0000 7fe185000640 1 -- 172.21.15.16:0/747314414 wait complete. 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.753+0000 7fe185000640 1 Processor -- start 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.753+0000 7fe185000640 1 -- start start 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe185000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1800726b0 0x7fe180153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe185000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1801599d0 0x7fe1800c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe185000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18015f830 0x7fe1800ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe185000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe18016a0e0 con 0x7fe18015f830 2024-09-10T21:10:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe185000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe180169f60 con 0x7fe1801599d0 2024-09-10T21:10:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe185000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe18016a260 con 0x7fe1800726b0 2024-09-10T21:10:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe17e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18015f830 0x7fe1800ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe17e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18015f830 0x7fe1800ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59786/0 (socket says 172.21.15.16:59786) 2024-09-10T21:10:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe17e800640 1 -- 172.21.15.16:0/1703890499 learned_addr learned my addr 172.21.15.16:0/1703890499 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe17de00640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1800726b0 0x7fe180153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe177e00640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1801599d0 0x7fe1800c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe17e800640 1 -- 172.21.15.16:0/1703890499 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1800726b0 msgr2=0x7fe180153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe17e800640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1800726b0 0x7fe180153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe17e800640 1 -- 172.21.15.16:0/1703890499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1801599d0 msgr2=0x7fe1800c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe17e800640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1801599d0 0x7fe1800c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe17e800640 1 -- 172.21.15.16:0/1703890499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe180158300 con 0x7fe18015f830 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe17e800640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18015f830 0x7fe1800ce850 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe1780b0040 tx=0x7fe1780a5870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe176000640 1 -- 172.21.15.16:0/1703890499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1780ba020 con 0x7fe18015f830 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe176000640 1 -- 172.21.15.16:0/1703890499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe178004470 con 0x7fe18015f830 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe176000640 1 -- 172.21.15.16:0/1703890499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1780b2040 con 0x7fe18015f830 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe185000640 1 -- 172.21.15.16:0/1703890499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1801585e0 con 0x7fe18015f830 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe185000640 1 -- 172.21.15.16:0/1703890499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1800b3150 con 0x7fe18015f830 2024-09-10T21:10:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe185000640 1 -- 172.21.15.16:0/1703890499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe180072b40 con 0x7fe18015f830 2024-09-10T21:10:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.757+0000 7fe176000640 1 -- 172.21.15.16:0/1703890499 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe1780b5030 con 0x7fe18015f830 2024-09-10T21:10:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.761+0000 7fe176000640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe15c072610 0x7fe15c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.761+0000 7fe176000640 1 -- 172.21.15.16:0/1703890499 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe17812db00 con 0x7fe18015f830 2024-09-10T21:10:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.761+0000 7fe17de00640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe15c072610 0x7fe15c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.761+0000 7fe17de00640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe15c072610 0x7fe15c074ad0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe170007fd0 tx=0x7fe170009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:54.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.765+0000 7fe176000640 1 -- 172.21.15.16:0/1703890499 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe1780c4030 con 0x7fe18015f830 2024-09-10T21:10:54.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.941+0000 7fe185000640 1 -- 172.21.15.16:0/1703890499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1800ca2f0 con 0x7fe18015f830 2024-09-10T21:10:54.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.941+0000 7fe176000640 1 -- 172.21.15.16:0/1703890499 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7fe178016790 con 0x7fe18015f830 2024-09-10T21:10:54.948 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:10:54.948 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:10:31.158120+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:10:27.523741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:10:28.533379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:10:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.945+0000 7fe185000640 1 -- 172.21.15.16:0/1703890499 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe15c072610 msgr2=0x7fe15c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.945+0000 7fe185000640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe15c072610 0x7fe15c074ad0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe170007fd0 tx=0x7fe170009040 comp rx=0 tx=0).stop 2024-09-10T21:10:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.945+0000 7fe185000640 1 -- 172.21.15.16:0/1703890499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18015f830 msgr2=0x7fe1800ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.945+0000 7fe185000640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18015f830 0x7fe1800ce850 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe1780b0040 tx=0x7fe1780a5870 comp rx=0 tx=0).stop 2024-09-10T21:10:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.945+0000 7fe185000640 1 -- 172.21.15.16:0/1703890499 shutdown_connections 2024-09-10T21:10:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.945+0000 7fe185000640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe15c072610 0x7fe15c074ad0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.945+0000 7fe185000640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18015f830 0x7fe1800ce850 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.945+0000 7fe185000640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1801599d0 0x7fe1800c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.945+0000 7fe185000640 1 --2- 172.21.15.16:0/1703890499 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1800726b0 0x7fe180153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.945+0000 7fe185000640 1 -- 172.21.15.16:0/1703890499 >> 172.21.15.16:0/1703890499 conn(0x7fe1800cfb20 msgr2=0x7fe18015e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.945+0000 7fe185000640 1 -- 172.21.15.16:0/1703890499 shutdown_connections 2024-09-10T21:10:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:54.945+0000 7fe185000640 1 -- 172.21.15.16:0/1703890499 wait complete. 2024-09-10T21:10:54.970 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:10:17.493770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T21:10:27.523741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T21:10:28.533379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:10:54.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:10:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.113+0000 7fda11200640 1 -- 172.21.15.16:0/2303436207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0c15bdc0 msgr2=0x7fda0c0d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.113+0000 7fda11200640 1 --2- 172.21.15.16:0/2303436207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0c15bdc0 0x7fda0c0d2800 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009840 tx=0x7fd9fc01c700 comp rx=0 tx=0).stop 2024-09-10T21:10:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.113+0000 7fda11200640 1 -- 172.21.15.16:0/2303436207 shutdown_connections 2024-09-10T21:10:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.113+0000 7fda11200640 1 --2- 172.21.15.16:0/2303436207 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fda0c0cd300 0x7fda0c0b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.113+0000 7fda11200640 1 --2- 172.21.15.16:0/2303436207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0c15bdc0 0x7fda0c0d2800 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.113+0000 7fda11200640 1 -- 172.21.15.16:0/2303436207 >> 172.21.15.16:0/2303436207 conn(0x7fda0c0d1ce0 msgr2=0x7fda0c0d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.113+0000 7fda11200640 1 -- 172.21.15.16:0/2303436207 shutdown_connections 2024-09-10T21:10:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.113+0000 7fda11200640 1 -- 172.21.15.16:0/2303436207 wait complete. 2024-09-10T21:10:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.113+0000 7fda11200640 1 Processor -- start 2024-09-10T21:10:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda11200640 1 -- start start 2024-09-10T21:10:55.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda11200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0c0b5050 0x7fda0c06f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:55.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda11200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fda0c0cd300 0x7fda0c1f8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda11200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda0c15bdc0 0x7fda0c15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda11200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c168e90 con 0x7fda0c0b5050 2024-09-10T21:10:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda11200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1602c0 con 0x7fda0c15bdc0 2024-09-10T21:10:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda11200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fda0c169010 con 0x7fda0c0cd300 2024-09-10T21:10:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda09e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0c0b5050 0x7fda0c06f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda09e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0c0b5050 0x7fda0c06f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59796/0 (socket says 172.21.15.16:59796) 2024-09-10T21:10:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda09e00640 1 -- 172.21.15.16:0/2643301540 learned_addr learned my addr 172.21.15.16:0/2643301540 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda0a800640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda0c15bdc0 0x7fda0c15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda03e00640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fda0c0cd300 0x7fda0c1f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda09e00640 1 -- 172.21.15.16:0/2643301540 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fda0c0cd300 msgr2=0x7fda0c1f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda09e00640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fda0c0cd300 0x7fda0c1f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda09e00640 1 -- 172.21.15.16:0/2643301540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda0c15bdc0 msgr2=0x7fda0c15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda09e00640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda0c15bdc0 0x7fda0c15ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda09e00640 1 -- 172.21.15.16:0/2643301540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c1f7ab0 con 0x7fda0c0b5050 2024-09-10T21:10:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda03e00640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fda0c0cd300 0x7fda0c1f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda0a800640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda0c15bdc0 0x7fda0c15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda09e00640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0c0b5050 0x7fda0c06f920 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc000c10 tx=0x7fd9fc0a8590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda02000640 1 -- 172.21.15.16:0/2643301540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc0a8a70 con 0x7fda0c0b5050 2024-09-10T21:10:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda02000640 1 -- 172.21.15.16:0/2643301540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9fc0a8c10 con 0x7fda0c0b5050 2024-09-10T21:10:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda02000640 1 -- 172.21.15.16:0/2643301540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc005030 con 0x7fda0c0b5050 2024-09-10T21:10:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda11200640 1 -- 172.21.15.16:0/2643301540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c0c82b0 con 0x7fda0c0b5050 2024-09-10T21:10:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda11200640 1 -- 172.21.15.16:0/2643301540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c1f7d70 con 0x7fda0c0b5050 2024-09-10T21:10:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.117+0000 7fda11200640 1 -- 172.21.15.16:0/2643301540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0c0b4b10 con 0x7fda0c0b5050 2024-09-10T21:10:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.121+0000 7fda02000640 1 -- 172.21.15.16:0/2643301540 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd9fc0051d0 con 0x7fda0c0b5050 2024-09-10T21:10:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.121+0000 7fda02000640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9e80725e0 0x7fd9e8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.121+0000 7fda02000640 1 -- 172.21.15.16:0/2643301540 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd9fc12d7f0 con 0x7fda0c0b5050 2024-09-10T21:10:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.121+0000 7fda03e00640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9e80725e0 0x7fd9e8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.121+0000 7fda03e00640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9e80725e0 0x7fd9e8074aa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda0c152cd0 tx=0x7fd9f40091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:55.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.125+0000 7fda02000640 1 -- 172.21.15.16:0/2643301540 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd9fc0ac030 con 0x7fda0c0b5050 2024-09-10T21:10:55.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.297+0000 7fda11200640 1 -- 172.21.15.16:0/2643301540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda0c0704a0 con 0x7fda0c0b5050 2024-09-10T21:10:55.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.297+0000 7fda02000640 1 -- 172.21.15.16:0/2643301540 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fd9fc016410 con 0x7fda0c0b5050 2024-09-10T21:10:55.305 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-10T21:10:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.301+0000 7fda11200640 1 -- 172.21.15.16:0/2643301540 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9e80725e0 msgr2=0x7fd9e8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.301+0000 7fda11200640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9e80725e0 0x7fd9e8074aa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda0c152cd0 tx=0x7fd9f40091e0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.301+0000 7fda11200640 1 -- 172.21.15.16:0/2643301540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0c0b5050 msgr2=0x7fda0c06f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.301+0000 7fda11200640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0c0b5050 0x7fda0c06f920 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc000c10 tx=0x7fd9fc0a8590 comp rx=0 tx=0).stop 2024-09-10T21:10:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.301+0000 7fda11200640 1 -- 172.21.15.16:0/2643301540 shutdown_connections 2024-09-10T21:10:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.301+0000 7fda11200640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9e80725e0 0x7fd9e8074aa0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.301+0000 7fda11200640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda0c15bdc0 0x7fda0c15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.301+0000 7fda11200640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fda0c0cd300 0x7fda0c1f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.301+0000 7fda11200640 1 --2- 172.21.15.16:0/2643301540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0c0b5050 0x7fda0c06f920 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.301+0000 7fda11200640 1 -- 172.21.15.16:0/2643301540 >> 172.21.15.16:0/2643301540 conn(0x7fda0c0d1ce0 msgr2=0x7fda0c1552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.301+0000 7fda11200640 1 -- 172.21.15.16:0/2643301540 shutdown_connections 2024-09-10T21:10:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.301+0000 7fda11200640 1 -- 172.21.15.16:0/2643301540 wait complete. 2024-09-10T21:10:55.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T21:10:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 -- 172.21.15.16:0/4226610721 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4e540d1750 msgr2=0x7f4e5406d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 --2- 172.21.15.16:0/4226610721 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4e540d1750 0x7f4e5406d5f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009870 tx=0x7f4e4001c7d0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 -- 172.21.15.16:0/4226610721 shutdown_connections 2024-09-10T21:10:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 --2- 172.21.15.16:0/4226610721 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4e540d1750 0x7f4e5406d5f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 -- 172.21.15.16:0/4226610721 >> 172.21.15.16:0/4226610721 conn(0x7f4e540d0c30 msgr2=0x7f4e540d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 -- 172.21.15.16:0/4226610721 shutdown_connections 2024-09-10T21:10:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 -- 172.21.15.16:0/4226610721 wait complete. 2024-09-10T21:10:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 Processor -- start 2024-09-10T21:10:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 -- start start 2024-09-10T21:10:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4e54073ec0 0x7f4e54158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e540d1750 0x7f4e54157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5415f280 0x7f4e54151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5416a6a0 con 0x7f4e5415f280 2024-09-10T21:10:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5416a520 con 0x7f4e540d1750 2024-09-10T21:10:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e59200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4e5416a820 con 0x7f4e54073ec0 2024-09-10T21:10:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e52800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5415f280 0x7f4e54151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e52800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5415f280 0x7f4e54151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59806/0 (socket says 172.21.15.16:59806) 2024-09-10T21:10:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e51400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e540d1750 0x7f4e54157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e52800640 1 -- 172.21.15.16:0/692623923 learned_addr learned my addr 172.21.15.16:0/692623923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e51e00640 1 --2- 172.21.15.16:0/692623923 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4e54073ec0 0x7f4e54158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e52800640 1 -- 172.21.15.16:0/692623923 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4e54073ec0 msgr2=0x7f4e54158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e52800640 1 --2- 172.21.15.16:0/692623923 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4e54073ec0 0x7f4e54158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e52800640 1 -- 172.21.15.16:0/692623923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e540d1750 msgr2=0x7f4e54157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e52800640 1 --2- 172.21.15.16:0/692623923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e540d1750 0x7f4e54157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.473+0000 7f4e52800640 1 -- 172.21.15.16:0/692623923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e540c4bb0 con 0x7f4e5415f280 2024-09-10T21:10:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.477+0000 7f4e52800640 1 --2- 172.21.15.16:0/692623923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5415f280 0x7f4e54151de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00cbd0 tx=0x7f4e4c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.477+0000 7f4e4a000640 1 -- 172.21.15.16:0/692623923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c016020 con 0x7f4e5415f280 2024-09-10T21:10:55.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.477+0000 7f4e4a000640 1 -- 172.21.15.16:0/692623923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e4c00aa50 con 0x7f4e5415f280 2024-09-10T21:10:55.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.477+0000 7f4e59200640 1 -- 172.21.15.16:0/692623923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e540c4e90 con 0x7f4e5415f280 2024-09-10T21:10:55.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.477+0000 7f4e4a000640 1 -- 172.21.15.16:0/692623923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c012420 con 0x7f4e5415f280 2024-09-10T21:10:55.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.477+0000 7f4e59200640 1 -- 172.21.15.16:0/692623923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e540c5450 con 0x7f4e5415f280 2024-09-10T21:10:55.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.477+0000 7f4e59200640 1 -- 172.21.15.16:0/692623923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e5406d5f0 con 0x7f4e5415f280 2024-09-10T21:10:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.477+0000 7f4e4a000640 1 -- 172.21.15.16:0/692623923 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4e4c0125c0 con 0x7f4e5415f280 2024-09-10T21:10:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.477+0000 7f4e4a000640 1 --2- 172.21.15.16:0/692623923 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4e340725e0 0x7f4e34074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.477+0000 7f4e4a000640 1 -- 172.21.15.16:0/692623923 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4e4c093720 con 0x7f4e5415f280 2024-09-10T21:10:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.477+0000 7f4e51e00640 1 --2- 172.21.15.16:0/692623923 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4e340725e0 0x7f4e34074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.481+0000 7f4e51e00640 1 --2- 172.21.15.16:0/692623923 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4e340725e0 0x7f4e34074aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e44007fd0 tx=0x7f4e44009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.481+0000 7f4e4a000640 1 -- 172.21.15.16:0/692623923 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4e4c020020 con 0x7f4e5415f280 2024-09-10T21:10:55.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.657+0000 7f4e59200640 1 -- 172.21.15.16:0/692623923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4e54166ed0 con 0x7f4e5415f280 2024-09-10T21:10:55.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.657+0000 7f4e4a000640 1 -- 172.21.15.16:0/692623923 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f4e4c05e7a0 con 0x7f4e5415f280 2024-09-10T21:10:55.663 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-10T21:10:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.661+0000 7f4e59200640 1 -- 172.21.15.16:0/692623923 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4e340725e0 msgr2=0x7f4e34074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.661+0000 7f4e59200640 1 --2- 172.21.15.16:0/692623923 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4e340725e0 0x7f4e34074aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e44007fd0 tx=0x7f4e44009040 comp rx=0 tx=0).stop 2024-09-10T21:10:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.661+0000 7f4e59200640 1 -- 172.21.15.16:0/692623923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5415f280 msgr2=0x7f4e54151de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.661+0000 7f4e59200640 1 --2- 172.21.15.16:0/692623923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5415f280 0x7f4e54151de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00cbd0 tx=0x7f4e4c00ccc0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.661+0000 7f4e59200640 1 -- 172.21.15.16:0/692623923 shutdown_connections 2024-09-10T21:10:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.661+0000 7f4e59200640 1 --2- 172.21.15.16:0/692623923 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4e340725e0 0x7f4e34074aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.661+0000 7f4e59200640 1 --2- 172.21.15.16:0/692623923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e5415f280 0x7f4e54151de0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.661+0000 7f4e59200640 1 --2- 172.21.15.16:0/692623923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e540d1750 0x7f4e54157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.661+0000 7f4e59200640 1 --2- 172.21.15.16:0/692623923 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4e54073ec0 0x7f4e54158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.661+0000 7f4e59200640 1 -- 172.21.15.16:0/692623923 >> 172.21.15.16:0/692623923 conn(0x7f4e540d0c30 msgr2=0x7f4e540b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.661+0000 7f4e59200640 1 -- 172.21.15.16:0/692623923 shutdown_connections 2024-09-10T21:10:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.661+0000 7f4e59200640 1 -- 172.21.15.16:0/692623923 wait complete. 2024-09-10T21:10:55.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T21:10:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 -- 172.21.15.16:0/3095510348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ac415ad80 msgr2=0x7f0ac40b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 --2- 172.21.15.16:0/3095510348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ac415ad80 0x7f0ac40b3f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009900 tx=0x7f0ab001c850 comp rx=0 tx=0).stop 2024-09-10T21:10:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 -- 172.21.15.16:0/3095510348 shutdown_connections 2024-09-10T21:10:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 --2- 172.21.15.16:0/3095510348 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0ac40b33b0 0x7f0ac4070070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 --2- 172.21.15.16:0/3095510348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ac415ad80 0x7f0ac40b3f80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 -- 172.21.15.16:0/3095510348 >> 172.21.15.16:0/3095510348 conn(0x7f0ac40b16a0 msgr2=0x7f0ac40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 -- 172.21.15.16:0/3095510348 shutdown_connections 2024-09-10T21:10:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 -- 172.21.15.16:0/3095510348 wait complete. 2024-09-10T21:10:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 Processor -- start 2024-09-10T21:10:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 -- start start 2024-09-10T21:10:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac40b33b0 0x7f0ac4159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0ac415a9a0 0x7f0ac41fafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ac415ad80 0x7f0ac4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4167f20 con 0x7f0ac40b33b0 2024-09-10T21:10:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4167da0 con 0x7f0ac415ad80 2024-09-10T21:10:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.829+0000 7f0acb600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac41680a0 con 0x7f0ac415a9a0 2024-09-10T21:10:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac40b33b0 0x7f0ac4159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac40b33b0 0x7f0ac4159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59816/0 (socket says 172.21.15.16:59816) 2024-09-10T21:10:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac8c00640 1 -- 172.21.15.16:0/2893495396 learned_addr learned my addr 172.21.15.16:0/2893495396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac8c00640 1 -- 172.21.15.16:0/2893495396 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0ac415a9a0 msgr2=0x7f0ac41fafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac9600640 1 --2- 172.21.15.16:0/2893495396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ac415ad80 0x7f0ac4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac8c00640 1 --2- 172.21.15.16:0/2893495396 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0ac415a9a0 0x7f0ac41fafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac8c00640 1 -- 172.21.15.16:0/2893495396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ac415ad80 msgr2=0x7f0ac4073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac8c00640 1 --2- 172.21.15.16:0/2893495396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ac415ad80 0x7f0ac4073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac8c00640 1 -- 172.21.15.16:0/2893495396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac41fb4f0 con 0x7f0ac40b33b0 2024-09-10T21:10:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac8c00640 1 --2- 172.21.15.16:0/2893495396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac40b33b0 0x7f0ac4159d30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c500 tx=0x7f0ab800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac0c00640 1 -- 172.21.15.16:0/2893495396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8017070 con 0x7f0ac40b33b0 2024-09-10T21:10:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac0c00640 1 -- 172.21.15.16:0/2893495396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ab80053c0 con 0x7f0ac40b33b0 2024-09-10T21:10:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac0c00640 1 -- 172.21.15.16:0/2893495396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8012890 con 0x7f0ac40b33b0 2024-09-10T21:10:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0acb600640 1 -- 172.21.15.16:0/2893495396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac41fb770 con 0x7f0ac40b33b0 2024-09-10T21:10:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0acb600640 1 -- 172.21.15.16:0/2893495396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac41fbd30 con 0x7f0ac40b33b0 2024-09-10T21:10:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0acb600640 1 -- 172.21.15.16:0/2893495396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac415d770 con 0x7f0ac40b33b0 2024-09-10T21:10:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.833+0000 7f0ac0c00640 1 -- 172.21.15.16:0/2893495396 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0ab8003600 con 0x7f0ac40b33b0 2024-09-10T21:10:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.837+0000 7f0ac0c00640 1 --2- 172.21.15.16:0/2893495396 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0aa4072610 0x7f0aa4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.837+0000 7f0ac0c00640 1 -- 172.21.15.16:0/2893495396 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0ab8092830 con 0x7f0ac40b33b0 2024-09-10T21:10:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.837+0000 7f0ac3e00640 1 --2- 172.21.15.16:0/2893495396 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0aa4072610 0x7f0aa4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.837+0000 7f0ac3e00640 1 --2- 172.21.15.16:0/2893495396 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0aa4072610 0x7f0aa4074ad0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00af000 tx=0x7f0ab00195f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:55.841+0000 7f0ac0c00640 1 -- 172.21.15.16:0/2893495396 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0ac41fb4f0 con 0x7f0ac40b33b0 2024-09-10T21:10:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0acb600640 1 -- 172.21.15.16:0/2893495396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0ac4070070 con 0x7f0ac40b33b0 2024-09-10T21:10:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0ac0c00640 1 -- 172.21.15.16:0/2893495396 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f0ab805df20 con 0x7f0ac40b33b0 2024-09-10T21:10:56.018 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-10T21:10:56.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0acb600640 1 -- 172.21.15.16:0/2893495396 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0aa4072610 msgr2=0x7f0aa4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0acb600640 1 --2- 172.21.15.16:0/2893495396 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0aa4072610 0x7f0aa4074ad0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00af000 tx=0x7f0ab00195f0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0acb600640 1 -- 172.21.15.16:0/2893495396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac40b33b0 msgr2=0x7f0ac4159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0acb600640 1 --2- 172.21.15.16:0/2893495396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac40b33b0 0x7f0ac4159d30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c500 tx=0x7f0ab800c9c0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0acb600640 1 -- 172.21.15.16:0/2893495396 shutdown_connections 2024-09-10T21:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0acb600640 1 --2- 172.21.15.16:0/2893495396 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0aa4072610 0x7f0aa4074ad0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0acb600640 1 --2- 172.21.15.16:0/2893495396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ac415ad80 0x7f0ac4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0acb600640 1 --2- 172.21.15.16:0/2893495396 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0ac415a9a0 0x7f0ac41fafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0acb600640 1 --2- 172.21.15.16:0/2893495396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac40b33b0 0x7f0ac4159d30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0acb600640 1 -- 172.21.15.16:0/2893495396 >> 172.21.15.16:0/2893495396 conn(0x7f0ac40b16a0 msgr2=0x7f0ac4155b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0acb600640 1 -- 172.21.15.16:0/2893495396 shutdown_connections 2024-09-10T21:10:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.013+0000 7f0acb600640 1 -- 172.21.15.16:0/2893495396 wait complete. 2024-09-10T21:10:56.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2024-09-10T21:10:56.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.181+0000 7f9d72000640 1 -- 172.21.15.16:0/244478637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d54004da0 msgr2=0x7f9d54005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.181+0000 7f9d72000640 1 --2- 172.21.15.16:0/244478637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d54004da0 0x7f9d54005220 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9d64001dc0 tx=0x7f9d6401cb30 comp rx=0 tx=0).stop 2024-09-10T21:10:56.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.181+0000 7f9d71600640 1 -- 172.21.15.16:0/244478637 reap_dead start 2024-09-10T21:10:56.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.181+0000 7f9d72000640 1 -- 172.21.15.16:0/244478637 shutdown_connections 2024-09-10T21:10:56.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.181+0000 7f9d72000640 1 -- 172.21.15.16:0/244478637 >> 172.21.15.16:0/244478637 conn(0x7f9d6c0d1d00 msgr2=0x7f9d6c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:56.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.181+0000 7f9d72000640 1 -- 172.21.15.16:0/244478637 shutdown_connections 2024-09-10T21:10:56.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.181+0000 7f9d72000640 1 -- 172.21.15.16:0/244478637 wait complete. 2024-09-10T21:10:56.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d72000640 1 Processor -- start 2024-09-10T21:10:56.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d72000640 1 -- start start 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d72000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d54004da0 0x7f9d6c0b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d72000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d6c15f280 0x7f9d6c1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d72000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d6c0afe60 0x7f9d6c0ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d72000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c0b2b10 con 0x7f9d54004da0 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d72000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c0b2990 con 0x7f9d6c15f280 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d72000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9d6c0b2c90 con 0x7f9d6c0afe60 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d70c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d54004da0 0x7f9d6c0b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d70c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d54004da0 0x7f9d6c0b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59830/0 (socket says 172.21.15.16:59830) 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d70c00640 1 -- 172.21.15.16:0/1126531816 learned_addr learned my addr 172.21.15.16:0/1126531816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d6be00640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d6c15f280 0x7f9d6c1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d71600640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d6c0afe60 0x7f9d6c0ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d70c00640 1 -- 172.21.15.16:0/1126531816 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d6c0afe60 msgr2=0x7f9d6c0ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d70c00640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d6c0afe60 0x7f9d6c0ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d70c00640 1 -- 172.21.15.16:0/1126531816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d6c15f280 msgr2=0x7f9d6c1571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d70c00640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d6c15f280 0x7f9d6c1571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d70c00640 1 -- 172.21.15.16:0/1126531816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6c157700 con 0x7f9d54004da0 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d6be00640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d6c15f280 0x7f9d6c1571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d70c00640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d54004da0 0x7f9d6c0b0830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9d60011b40 tx=0x7f9d60009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d68c00640 1 -- 172.21.15.16:0/1126531816 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6001e070 con 0x7f9d54004da0 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d68c00640 1 -- 172.21.15.16:0/1126531816 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d60004050 con 0x7f9d54004da0 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d68c00640 1 -- 172.21.15.16:0/1126531816 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000fea0 con 0x7f9d54004da0 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d72000640 1 -- 172.21.15.16:0/1126531816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6c0c6200 con 0x7f9d54004da0 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.185+0000 7f9d72000640 1 -- 172.21.15.16:0/1126531816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d6c0c67c0 con 0x7f9d54004da0 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.189+0000 7f9d72000640 1 -- 172.21.15.16:0/1126531816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d6c0ce2f0 con 0x7f9d54004da0 2024-09-10T21:10:56.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.189+0000 7f9d68c00640 1 -- 172.21.15.16:0/1126531816 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9d60003600 con 0x7f9d54004da0 2024-09-10T21:10:56.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.189+0000 7f9d68c00640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d4c0725e0 0x7f9d4c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.189+0000 7f9d68c00640 1 -- 172.21.15.16:0/1126531816 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9d60093f90 con 0x7f9d54004da0 2024-09-10T21:10:56.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.189+0000 7f9d6be00640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d4c0725e0 0x7f9d4c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.189+0000 7f9d6be00640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d4c0725e0 0x7f9d4c074aa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9d58000fc0 tx=0x7f9d58006730 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.193+0000 7f9d68c00640 1 -- 172.21.15.16:0/1126531816 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9d6005f0d0 con 0x7f9d54004da0 2024-09-10T21:10:56.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.429+0000 7f9d72000640 1 -- 172.21.15.16:0/1126531816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) -- 0x7f9d6c0cf2a0 con 0x7f9d54004da0 2024-09-10T21:10:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.589+0000 7f9d68c00640 1 -- 172.21.15.16:0/1126531816 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v48) ==== 168+0+0 (secure 0 0 0) 0x7f9d60063b00 con 0x7f9d54004da0 2024-09-10T21:10:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.593+0000 7f9d72000640 1 -- 172.21.15.16:0/1126531816 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d4c0725e0 msgr2=0x7f9d4c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.593+0000 7f9d72000640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d4c0725e0 0x7f9d4c074aa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9d58000fc0 tx=0x7f9d58006730 comp rx=0 tx=0).stop 2024-09-10T21:10:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.593+0000 7f9d72000640 1 -- 172.21.15.16:0/1126531816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d54004da0 msgr2=0x7f9d6c0b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.593+0000 7f9d72000640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d54004da0 0x7f9d6c0b0830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9d60011b40 tx=0x7f9d60009ea0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.593+0000 7f9d72000640 1 -- 172.21.15.16:0/1126531816 shutdown_connections 2024-09-10T21:10:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.593+0000 7f9d72000640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d4c0725e0 0x7f9d4c074aa0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.593+0000 7f9d72000640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d6c0afe60 0x7f9d6c0ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.593+0000 7f9d72000640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d6c15f280 0x7f9d6c1571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.593+0000 7f9d72000640 1 --2- 172.21.15.16:0/1126531816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d54004da0 0x7f9d6c0b0830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.593+0000 7f9d72000640 1 -- 172.21.15.16:0/1126531816 >> 172.21.15.16:0/1126531816 conn(0x7f9d6c0d1d00 msgr2=0x7f9d6c0cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.593+0000 7f9d72000640 1 -- 172.21.15.16:0/1126531816 shutdown_connections 2024-09-10T21:10:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.593+0000 7f9d72000640 1 -- 172.21.15.16:0/1126531816 wait complete. 2024-09-10T21:10:56.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:10:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 -- 172.21.15.16:0/241116621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8071780 msgr2=0x7fbaa80afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 --2- 172.21.15.16:0/241116621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8071780 0x7fbaa80afec0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009bf0 tx=0x7fbaa001ca50 comp rx=0 tx=0).stop 2024-09-10T21:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 -- 172.21.15.16:0/241116621 shutdown_connections 2024-09-10T21:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 --2- 172.21.15.16:0/241116621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8071780 0x7fbaa80afec0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 --2- 172.21.15.16:0/241116621 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbaa80d16a0 0x7fbaa80af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 -- 172.21.15.16:0/241116621 >> 172.21.15.16:0/241116621 conn(0x7fbaa809b130 msgr2=0x7fbaa80d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 -- 172.21.15.16:0/241116621 shutdown_connections 2024-09-10T21:10:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 -- 172.21.15.16:0/241116621 wait complete. 2024-09-10T21:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 Processor -- start 2024-09-10T21:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 -- start start 2024-09-10T21:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8071780 0x7fbaa8157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbaa815f2e0 0x7fbaa81f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaa8150460 0x7fbaa80b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa81694e0 con 0x7fbaa8071780 2024-09-10T21:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa8169360 con 0x7fbaa8150460 2024-09-10T21:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaaec00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa8169660 con 0x7fbaa815f2e0 2024-09-10T21:10:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.761+0000 7fbaa7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8071780 0x7fbaa8157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaa7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8071780 0x7fbaa8157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59850/0 (socket says 172.21.15.16:59850) 2024-09-10T21:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaa7e00640 1 -- 172.21.15.16:0/1258580193 learned_addr learned my addr 172.21.15.16:0/1258580193 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fba9fe00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbaa815f2e0 0x7fbaa81f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaacc00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaa8150460 0x7fbaa80b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaa7e00640 1 -- 172.21.15.16:0/1258580193 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbaa815f2e0 msgr2=0x7fbaa81f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaa7e00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbaa815f2e0 0x7fbaa81f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaa7e00640 1 -- 172.21.15.16:0/1258580193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaa8150460 msgr2=0x7fbaa80b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaa7e00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaa8150460 0x7fbaa80b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaa7e00640 1 -- 172.21.15.16:0/1258580193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa8150bd0 con 0x7fbaa8071780 2024-09-10T21:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaacc00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaa8150460 0x7fbaa80b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fba9fe00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbaa815f2e0 0x7fbaa81f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaa7e00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8071780 0x7fbaa8157900 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba9800c400 tx=0x7fba9800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaa4c00640 1 -- 172.21.15.16:0/1258580193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98017070 con 0x7fbaa8071780 2024-09-10T21:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaa4c00640 1 -- 172.21.15.16:0/1258580193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba98003600 con 0x7fbaa8071780 2024-09-10T21:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaaec00640 1 -- 172.21.15.16:0/1258580193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa81508f0 con 0x7fbaa8071780 2024-09-10T21:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaa4c00640 1 -- 172.21.15.16:0/1258580193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98003a50 con 0x7fbaa8071780 2024-09-10T21:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaaec00640 1 -- 172.21.15.16:0/1258580193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa80b5840 con 0x7fbaa8071780 2024-09-10T21:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaaec00640 1 -- 172.21.15.16:0/1258580193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa8071070 con 0x7fbaa8071780 2024-09-10T21:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.765+0000 7fbaa4c00640 1 -- 172.21.15.16:0/1258580193 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fba98003bf0 con 0x7fbaa8071780 2024-09-10T21:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.769+0000 7fbaa4c00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba840725e0 0x7fba84074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.769+0000 7fbaa4c00640 1 -- 172.21.15.16:0/1258580193 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fba98092840 con 0x7fbaa8071780 2024-09-10T21:10:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.769+0000 7fba9fe00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba840725e0 0x7fba84074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.769+0000 7fba9fe00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba840725e0 0x7fba84074aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fba900058d0 tx=0x7fba90005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.773+0000 7fbaa4c00640 1 -- 172.21.15.16:0/1258580193 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fba9805df10 con 0x7fbaa8071780 2024-09-10T21:10:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.949+0000 7fbaaec00640 1 -- 172.21.15.16:0/1258580193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fbaa80d0990 con 0x7fbaa8071780 2024-09-10T21:10:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.949+0000 7fbaa4c00640 1 -- 172.21.15.16:0/1258580193 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7fba9805d810 con 0x7fbaa8071780 2024-09-10T21:10:56.955 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:10:56.955 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:10:56.955 INFO:teuthology.orchestra.run.smithi016.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:10:56.955 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:10:56.955 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4226.427518 2024-09-10T21:10:56.955 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5426.620519 2024-09-10T21:10:56.956 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4255.210203 2024-09-10T21:10:56.956 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4255.734871 2024-09-10T21:10:56.956 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4192.833008 2024-09-10T21:10:56.956 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4136.940759 2024-09-10T21:10:56.956 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4217.384701 2024-09-10T21:10:56.956 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5220.337972 2024-09-10T21:10:56.956 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-10T21:10:56.956 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:10:56.956 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-10T21:10:56.956 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-10T21:10:56.956 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:10:56.956 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:10:56.956 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-10T21:10:56.957 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:10:56.957 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:10:56.957 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:10:56.957 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-10T21:10:56.957 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-10T21:10:56.957 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-10T21:10:56.957 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:10:56.957 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:10:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.949+0000 7fbaaec00640 1 -- 172.21.15.16:0/1258580193 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba840725e0 msgr2=0x7fba84074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.949+0000 7fbaaec00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba840725e0 0x7fba84074aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fba900058d0 tx=0x7fba90005860 comp rx=0 tx=0).stop 2024-09-10T21:10:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.949+0000 7fbaaec00640 1 -- 172.21.15.16:0/1258580193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8071780 msgr2=0x7fbaa8157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.949+0000 7fbaaec00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8071780 0x7fbaa8157900 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba9800c400 tx=0x7fba9800c8c0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.949+0000 7fbaaec00640 1 -- 172.21.15.16:0/1258580193 shutdown_connections 2024-09-10T21:10:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.949+0000 7fbaaec00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba840725e0 0x7fba84074aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.949+0000 7fbaaec00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaa8150460 0x7fbaa80b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.949+0000 7fbaaec00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbaa815f2e0 0x7fbaa81f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.949+0000 7fbaaec00640 1 --2- 172.21.15.16:0/1258580193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa8071780 0x7fbaa8157900 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.949+0000 7fbaaec00640 1 -- 172.21.15.16:0/1258580193 >> 172.21.15.16:0/1258580193 conn(0x7fbaa809b130 msgr2=0x7fbaa80b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.953+0000 7fbaaec00640 1 -- 172.21.15.16:0/1258580193 shutdown_connections 2024-09-10T21:10:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:56.953+0000 7fbaaec00640 1 -- 172.21.15.16:0/1258580193 wait complete. 2024-09-10T21:10:56.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:10:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.121+0000 7f7f06600640 1 -- 172.21.15.16:0/322836979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f000b33b0 msgr2=0x7f7f00070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.121+0000 7f7f06600640 1 --2- 172.21.15.16:0/322836979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f000b33b0 0x7f7f00070070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009bc0 tx=0x7f7ef801cb10 comp rx=0 tx=0).stop 2024-09-10T21:10:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.121+0000 7f7f06600640 1 -- 172.21.15.16:0/322836979 shutdown_connections 2024-09-10T21:10:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.121+0000 7f7f06600640 1 --2- 172.21.15.16:0/322836979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f000b33b0 0x7f7f00070070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.121+0000 7f7f06600640 1 --2- 172.21.15.16:0/322836979 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7f0015ad80 0x7f7f000b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.121+0000 7f7f06600640 1 -- 172.21.15.16:0/322836979 >> 172.21.15.16:0/322836979 conn(0x7f7f000b16a0 msgr2=0x7f7f000b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.121+0000 7f7f06600640 1 -- 172.21.15.16:0/322836979 shutdown_connections 2024-09-10T21:10:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.121+0000 7f7f06600640 1 -- 172.21.15.16:0/322836979 wait complete. 2024-09-10T21:10:57.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.121+0000 7f7f06600640 1 Processor -- start 2024-09-10T21:10:57.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.121+0000 7f7f06600640 1 -- start start 2024-09-10T21:10:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7f06600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f000b33b0 0x7f7f00157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7f06600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0015a9a0 0x7f7f0014fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7f06600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7f0015ad80 0x7f7f001640a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7f06600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0006f540 con 0x7f7f0015a9a0 2024-09-10T21:10:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7f06600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0006f3c0 con 0x7f7f000b33b0 2024-09-10T21:10:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7f06600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0006f6c0 con 0x7f7f0015ad80 2024-09-10T21:10:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7efea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0015a9a0 0x7f7f0014fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7efea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0015a9a0 0x7f7f0014fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59872/0 (socket says 172.21.15.16:59872) 2024-09-10T21:10:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7efea00640 1 -- 172.21.15.16:0/2051473236 learned_addr learned my addr 172.21.15.16:0/2051473236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7eff400640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f000b33b0 0x7f7f00157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7effe00640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7f0015ad80 0x7f7f001640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7efea00640 1 -- 172.21.15.16:0/2051473236 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7f0015ad80 msgr2=0x7f7f001640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7efea00640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7f0015ad80 0x7f7f001640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7efea00640 1 -- 172.21.15.16:0/2051473236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f000b33b0 msgr2=0x7f7f00157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7efea00640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f000b33b0 0x7f7f00157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7efea00640 1 -- 172.21.15.16:0/2051473236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f00150130 con 0x7f7f0015a9a0 2024-09-10T21:10:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7efea00640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0015a9a0 0x7f7f0014fa60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00c9f0 tx=0x7f7eec00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7ef7400640 1 -- 172.21.15.16:0/2051473236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec007b50 con 0x7f7f0015a9a0 2024-09-10T21:10:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7ef7400640 1 -- 172.21.15.16:0/2051473236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7eec007de0 con 0x7f7f0015a9a0 2024-09-10T21:10:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7ef7400640 1 -- 172.21.15.16:0/2051473236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec00f6b0 con 0x7f7f0015a9a0 2024-09-10T21:10:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7f06600640 1 -- 172.21.15.16:0/2051473236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f000af770 con 0x7f7f0015a9a0 2024-09-10T21:10:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7f06600640 1 -- 172.21.15.16:0/2051473236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f001505e0 con 0x7f7f0015a9a0 2024-09-10T21:10:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.125+0000 7f7f06600640 1 -- 172.21.15.16:0/2051473236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f0015e560 con 0x7f7f0015a9a0 2024-09-10T21:10:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.129+0000 7f7ef7400640 1 -- 172.21.15.16:0/2051473236 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7eec00f850 con 0x7f7f0015a9a0 2024-09-10T21:10:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.129+0000 7f7ef7400640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7ee0072610 0x7f7ee0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.129+0000 7f7ef7400640 1 -- 172.21.15.16:0/2051473236 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7eec0937f0 con 0x7f7f0015a9a0 2024-09-10T21:10:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.129+0000 7f7eff400640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7ee0072610 0x7f7ee0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.129+0000 7f7eff400640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7ee0072610 0x7f7ee0074ad0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0001910 tx=0x7f7ef0001420 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.133+0000 7f7ef7400640 1 -- 172.21.15.16:0/2051473236 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7eec014030 con 0x7f7f0015a9a0 2024-09-10T21:10:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.309+0000 7f7f06600640 1 -- 172.21.15.16:0/2051473236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7f0015f310 con 0x7f7f0015a9a0 2024-09-10T21:10:57.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.313+0000 7f7ef7400640 1 -- 172.21.15.16:0/2051473236 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f7eec05e870 con 0x7f7f0015a9a0 2024-09-10T21:10:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.313+0000 7f7f06600640 1 -- 172.21.15.16:0/2051473236 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7ee0072610 msgr2=0x7f7ee0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.313+0000 7f7f06600640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7ee0072610 0x7f7ee0074ad0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0001910 tx=0x7f7ef0001420 comp rx=0 tx=0).stop 2024-09-10T21:10:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.313+0000 7f7f06600640 1 -- 172.21.15.16:0/2051473236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0015a9a0 msgr2=0x7f7f0014fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.317+0000 7f7f06600640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0015a9a0 0x7f7f0014fa60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00c9f0 tx=0x7f7eec00ceb0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.317+0000 7f7f06600640 1 -- 172.21.15.16:0/2051473236 shutdown_connections 2024-09-10T21:10:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.317+0000 7f7f06600640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7ee0072610 0x7f7ee0074ad0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.317+0000 7f7f06600640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7f0015ad80 0x7f7f001640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.317+0000 7f7f06600640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0015a9a0 0x7f7f0014fa60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.317+0000 7f7f06600640 1 --2- 172.21.15.16:0/2051473236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f000b33b0 0x7f7f00157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.317+0000 7f7f06600640 1 -- 172.21.15.16:0/2051473236 >> 172.21.15.16:0/2051473236 conn(0x7f7f000b16a0 msgr2=0x7f7f000cff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:57.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.317+0000 7f7f06600640 1 -- 172.21.15.16:0/2051473236 shutdown_connections 2024-09-10T21:10:57.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.317+0000 7f7f06600640 1 -- 172.21.15.16:0/2051473236 wait complete. 2024-09-10T21:10:57.341 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:10:57.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:10:57.427 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:10:57.529 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:10:57.529 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:10:57.529 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:57.538 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:10:57.538 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:57.589 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:10:57.590 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:57.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:10:57.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.829+0000 7fe2c3e00640 1 -- 172.21.15.16:0/3764958717 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2bc0b13c0 msgr2=0x7fe2bc0afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.829+0000 7fe2c3e00640 1 --2- 172.21.15.16:0/3764958717 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2bc0b13c0 0x7fe2bc0afac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009c80 tx=0x7fe2b401cc80 comp rx=0 tx=0).stop 2024-09-10T21:10:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.829+0000 7fe2c3e00640 1 -- 172.21.15.16:0/3764958717 shutdown_connections 2024-09-10T21:10:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.829+0000 7fe2c3e00640 1 --2- 172.21.15.16:0/3764958717 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2bc0b13c0 0x7fe2bc0afac0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.829+0000 7fe2c3e00640 1 --2- 172.21.15.16:0/3764958717 >> v2:172.21.15.52:3300/0 conn(0x7fe2bc0716f0 0x7fe2bc0af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.829+0000 7fe2c3e00640 1 -- 172.21.15.16:0/3764958717 >> 172.21.15.16:0/3764958717 conn(0x7fe2bc09b130 msgr2=0x7fe2bc0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.829+0000 7fe2c3e00640 1 -- 172.21.15.16:0/3764958717 shutdown_connections 2024-09-10T21:10:57.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.829+0000 7fe2c3e00640 1 -- 172.21.15.16:0/3764958717 wait complete. 2024-09-10T21:10:57.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c3e00640 1 Processor -- start 2024-09-10T21:10:57.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c3e00640 1 -- start start 2024-09-10T21:10:57.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c3e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2bc0716f0 0x7fe2bc14a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc0b13c0 0x7fe2bc0b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc15f400 0x7fe2bc0b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c3e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc168fc0 con 0x7fe2bc15f400 2024-09-10T21:10:57.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c3e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc168e40 con 0x7fe2bc0b13c0 2024-09-10T21:10:57.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c3e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe2bc169140 con 0x7fe2bc0716f0 2024-09-10T21:10:57.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc15f400 0x7fe2bc0b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc15f400 0x7fe2bc0b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59876/0 (socket says 172.21.15.16:59876) 2024-09-10T21:10:57.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c1e00640 1 -- 172.21.15.16:0/1205641466 learned_addr learned my addr 172.21.15.16:0/1205641466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c1400640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2bc0716f0 0x7fe2bc14a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c0a00640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc0b13c0 0x7fe2bc0b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c1e00640 1 -- 172.21.15.16:0/1205641466 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2bc0716f0 msgr2=0x7fe2bc14a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c1e00640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2bc0716f0 0x7fe2bc14a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c1e00640 1 -- 172.21.15.16:0/1205641466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc0b13c0 msgr2=0x7fe2bc0b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c1e00640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc0b13c0 0x7fe2bc0b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c1e00640 1 -- 172.21.15.16:0/1205641466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2bc0c5910 con 0x7fe2bc15f400 2024-09-10T21:10:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c1400640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2bc0716f0 0x7fe2bc14a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:57.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c0a00640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc0b13c0 0x7fe2bc0b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c1e00640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc15f400 0x7fe2bc0b2d30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009bd0 tx=0x7fe2b40092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2b9600640 1 -- 172.21.15.16:0/1205641466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40bb070 con 0x7fe2bc15f400 2024-09-10T21:10:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2b9600640 1 -- 172.21.15.16:0/1205641466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2b4004d40 con 0x7fe2bc15f400 2024-09-10T21:10:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c3e00640 1 -- 172.21.15.16:0/1205641466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2bc0c5bf0 con 0x7fe2bc15f400 2024-09-10T21:10:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2b9600640 1 -- 172.21.15.16:0/1205641466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40b2040 con 0x7fe2bc15f400 2024-09-10T21:10:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c3e00640 1 -- 172.21.15.16:0/1205641466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2bc0c61b0 con 0x7fe2bc15f400 2024-09-10T21:10:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.833+0000 7fe2c3e00640 1 -- 172.21.15.16:0/1205641466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2bc070fe0 con 0x7fe2bc15f400 2024-09-10T21:10:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.837+0000 7fe2b9600640 1 -- 172.21.15.16:0/1205641466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe2b40b5030 con 0x7fe2bc15f400 2024-09-10T21:10:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.837+0000 7fe2b9600640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe29c0725e0 0x7fe29c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.837+0000 7fe2b9600640 1 -- 172.21.15.16:0/1205641466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe2b412cd80 con 0x7fe2bc15f400 2024-09-10T21:10:57.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.837+0000 7fe2c1400640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe29c0725e0 0x7fe29c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.837+0000 7fe2c1400640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe29c0725e0 0x7fe29c074aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0007fd0 tx=0x7fe2b0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:57.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:57.841+0000 7fe2b9600640 1 -- 172.21.15.16:0/1205641466 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe2b4016990 con 0x7fe2bc15f400 2024-09-10T21:10:58.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.025+0000 7fe2c3e00640 1 -- 172.21.15.16:0/1205641466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe2bc0b7350 con 0x7fe2bc15f400 2024-09-10T21:10:58.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.025+0000 7fe2b9600640 1 -- 172.21.15.16:0/1205641466 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fe2b40f83d0 con 0x7fe2bc15f400 2024-09-10T21:10:58.033 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:10:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.029+0000 7fe2c3e00640 1 -- 172.21.15.16:0/1205641466 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe29c0725e0 msgr2=0x7fe29c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.029+0000 7fe2c3e00640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe29c0725e0 0x7fe29c074aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0007fd0 tx=0x7fe2b0009040 comp rx=0 tx=0).stop 2024-09-10T21:10:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.029+0000 7fe2c3e00640 1 -- 172.21.15.16:0/1205641466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc15f400 msgr2=0x7fe2bc0b2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.029+0000 7fe2c3e00640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc15f400 0x7fe2bc0b2d30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009bd0 tx=0x7fe2b40092e0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.029+0000 7fe2c3e00640 1 -- 172.21.15.16:0/1205641466 shutdown_connections 2024-09-10T21:10:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.029+0000 7fe2c3e00640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe29c0725e0 0x7fe29c074aa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.029+0000 7fe2c3e00640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc15f400 0x7fe2bc0b2d30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.029+0000 7fe2c3e00640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc0b13c0 0x7fe2bc0b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.029+0000 7fe2c3e00640 1 --2- 172.21.15.16:0/1205641466 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2bc0716f0 0x7fe2bc14a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.029+0000 7fe2c3e00640 1 -- 172.21.15.16:0/1205641466 >> 172.21.15.16:0/1205641466 conn(0x7fe2bc09b130 msgr2=0x7fe2bc0cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.029+0000 7fe2c3e00640 1 -- 172.21.15.16:0/1205641466 shutdown_connections 2024-09-10T21:10:58.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.029+0000 7fe2c3e00640 1 -- 172.21.15.16:0/1205641466 wait complete. 2024-09-10T21:10:58.054 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:10:58.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:58.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T21:10:58.086 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:10:58.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:58.087 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:10:58.146 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:10:58.146 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:58.157 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:10:58.158 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:58.206 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:10:58.207 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:58.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:10:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.401+0000 7f9c91a00640 1 -- 172.21.15.16:0/2947493841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c155000 msgr2=0x7f9c8c15bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.401+0000 7f9c91a00640 1 --2- 172.21.15.16:0/2947493841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c155000 0x7f9c8c15bbb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009840 tx=0x7f9c7c01c7b0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.401+0000 7f9c91a00640 1 -- 172.21.15.16:0/2947493841 shutdown_connections 2024-09-10T21:10:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.401+0000 7f9c91a00640 1 --2- 172.21.15.16:0/2947493841 >> v2:172.21.15.52:3300/0 conn(0x7f9c8c0706f0 0x7f9c8c0ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.401+0000 7f9c91a00640 1 --2- 172.21.15.16:0/2947493841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c155000 0x7f9c8c15bbb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.401+0000 7f9c91a00640 1 -- 172.21.15.16:0/2947493841 >> 172.21.15.16:0/2947493841 conn(0x7f9c8c0cfb20 msgr2=0x7f9c8c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.401+0000 7f9c91a00640 1 -- 172.21.15.16:0/2947493841 shutdown_connections 2024-09-10T21:10:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.401+0000 7f9c91a00640 1 -- 172.21.15.16:0/2947493841 wait complete. 2024-09-10T21:10:58.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c91a00640 1 Processor -- start 2024-09-10T21:10:58.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c91a00640 1 -- start start 2024-09-10T21:10:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c91a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c8c0706f0 0x7f9c8c1556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c91a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c072d40 0x7f9c8c148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c91a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9c8c155000 0x7f9c8c0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c91a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c167910 con 0x7f9c8c072d40 2024-09-10T21:10:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c91a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c167790 con 0x7f9c8c0706f0 2024-09-10T21:10:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c91a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8c167a90 con 0x7f9c8c155000 2024-09-10T21:10:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c072d40 0x7f9c8c148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c072d40 0x7f9c8c148fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59896/0 (socket says 172.21.15.16:59896) 2024-09-10T21:10:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c83e00640 1 -- 172.21.15.16:0/2378217419 learned_addr learned my addr 172.21.15.16:0/2378217419 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c8aa00640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c8c0706f0 0x7f9c8c1556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c8b400640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9c8c155000 0x7f9c8c0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c83e00640 1 -- 172.21.15.16:0/2378217419 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9c8c155000 msgr2=0x7f9c8c0b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c83e00640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9c8c155000 0x7f9c8c0b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c83e00640 1 -- 172.21.15.16:0/2378217419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c8c0706f0 msgr2=0x7f9c8c1556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c83e00640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c8c0706f0 0x7f9c8c1556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c83e00640 1 -- 172.21.15.16:0/2378217419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8c1494f0 con 0x7f9c8c072d40 2024-09-10T21:10:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c83e00640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c072d40 0x7f9c8c148fb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400c530 tx=0x7f9c7400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c82a00640 1 -- 172.21.15.16:0/2378217419 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74017070 con 0x7f9c8c072d40 2024-09-10T21:10:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c82a00640 1 -- 172.21.15.16:0/2378217419 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c74003600 con 0x7f9c8c072d40 2024-09-10T21:10:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c82a00640 1 -- 172.21.15.16:0/2378217419 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74003a50 con 0x7f9c8c072d40 2024-09-10T21:10:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c91a00640 1 -- 172.21.15.16:0/2378217419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8c158cd0 con 0x7f9c8c072d40 2024-09-10T21:10:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.405+0000 7f9c91a00640 1 -- 172.21.15.16:0/2378217419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8c159290 con 0x7f9c8c072d40 2024-09-10T21:10:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.409+0000 7f9c91a00640 1 -- 172.21.15.16:0/2378217419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8c0ca4c0 con 0x7f9c8c072d40 2024-09-10T21:10:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.409+0000 7f9c82a00640 1 -- 172.21.15.16:0/2378217419 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9c74003bf0 con 0x7f9c8c072d40 2024-09-10T21:10:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.409+0000 7f9c82a00640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9c680725e0 0x7f9c68074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.409+0000 7f9c82a00640 1 -- 172.21.15.16:0/2378217419 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9c740928b0 con 0x7f9c8c072d40 2024-09-10T21:10:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.409+0000 7f9c8aa00640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9c680725e0 0x7f9c68074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.409+0000 7f9c8aa00640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9c680725e0 0x7f9c68074aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0af000 tx=0x7f9c7c01aff0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:58.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.413+0000 7f9c82a00640 1 -- 172.21.15.16:0/2378217419 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9c7405d9f0 con 0x7f9c8c072d40 2024-09-10T21:10:58.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.605+0000 7f9c91a00640 1 -- 172.21.15.16:0/2378217419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9c8c153cf0 con 0x7f9c8c072d40 2024-09-10T21:10:58.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.605+0000 7f9c82a00640 1 -- 172.21.15.16:0/2378217419 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f9c8c153cf0 con 0x7f9c8c072d40 2024-09-10T21:10:58.612 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:10:58.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.609+0000 7f9c91a00640 1 -- 172.21.15.16:0/2378217419 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9c680725e0 msgr2=0x7f9c68074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:58.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.609+0000 7f9c91a00640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9c680725e0 0x7f9c68074aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0af000 tx=0x7f9c7c01aff0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.609+0000 7f9c91a00640 1 -- 172.21.15.16:0/2378217419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c072d40 msgr2=0x7f9c8c148fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:58.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.609+0000 7f9c91a00640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c072d40 0x7f9c8c148fb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400c530 tx=0x7f9c7400c9f0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.609+0000 7f9c91a00640 1 -- 172.21.15.16:0/2378217419 shutdown_connections 2024-09-10T21:10:58.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.609+0000 7f9c91a00640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9c680725e0 0x7f9c68074aa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.609+0000 7f9c91a00640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9c8c155000 0x7f9c8c0b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.609+0000 7f9c91a00640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c072d40 0x7f9c8c148fb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.609+0000 7f9c91a00640 1 --2- 172.21.15.16:0/2378217419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c8c0706f0 0x7f9c8c1556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.609+0000 7f9c91a00640 1 -- 172.21.15.16:0/2378217419 >> 172.21.15.16:0/2378217419 conn(0x7f9c8c0cfb20 msgr2=0x7f9c8c0b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.609+0000 7f9c91a00640 1 -- 172.21.15.16:0/2378217419 shutdown_connections 2024-09-10T21:10:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:58.609+0000 7f9c91a00640 1 -- 172.21.15.16:0/2378217419 wait complete. 2024-09-10T21:10:58.632 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T21:10:58.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:58.633 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-10T21:10:58.676 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:10:58.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:58.676 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T21:10:58.732 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:10:58.733 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:58.742 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:10:58.742 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:58.793 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:10:58.793 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:58.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:10:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.033+0000 7fcddb000640 1 -- 172.21.15.16:0/882887460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd40b13c0 msgr2=0x7fcdd4071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.033+0000 7fcddb000640 1 --2- 172.21.15.16:0/882887460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd40b13c0 0x7fcdd4071520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009900 tx=0x7fcdbc01c7c0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.033+0000 7fcddb000640 1 -- 172.21.15.16:0/882887460 shutdown_connections 2024-09-10T21:10:59.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.033+0000 7fcddb000640 1 --2- 172.21.15.16:0/882887460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd40b13c0 0x7fcdd4071520 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.033+0000 7fcddb000640 1 --2- 172.21.15.16:0/882887460 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcdd415f400 0x7fcdd4070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.033+0000 7fcddb000640 1 -- 172.21.15.16:0/882887460 >> 172.21.15.16:0/882887460 conn(0x7fcdd409b130 msgr2=0x7fcdd40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:59.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.033+0000 7fcddb000640 1 -- 172.21.15.16:0/882887460 shutdown_connections 2024-09-10T21:10:59.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.033+0000 7fcddb000640 1 -- 172.21.15.16:0/882887460 wait complete. 2024-09-10T21:10:59.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcddb000640 1 Processor -- start 2024-09-10T21:10:59.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcddb000640 1 -- start start 2024-09-10T21:10:59.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcddb000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd40b13c0 0x7fcdd415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcddb000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd40cfc80 0x7fcdd4153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcddb000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcdd415f400 0x7fcdd4158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcddb000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd4169770 con 0x7fcdd40cfc80 2024-09-10T21:10:59.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcddb000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd41695f0 con 0x7fcdd40b13c0 2024-09-10T21:10:59.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcddb000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd41698f0 con 0x7fcdd415f400 2024-09-10T21:10:59.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdcbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd40cfc80 0x7fcdd4153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdcbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd40cfc80 0x7fcdd4153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59914/0 (socket says 172.21.15.16:59914) 2024-09-10T21:10:59.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdcbe00640 1 -- 172.21.15.16:0/2928300042 learned_addr learned my addr 172.21.15.16:0/2928300042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:59.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdd3e00640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd40b13c0 0x7fcdd415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdd9000640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcdd415f400 0x7fcdd4158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdcbe00640 1 -- 172.21.15.16:0/2928300042 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcdd415f400 msgr2=0x7fcdd4158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdcbe00640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcdd415f400 0x7fcdd4158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdcbe00640 1 -- 172.21.15.16:0/2928300042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd40b13c0 msgr2=0x7fcdd415b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdcbe00640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd40b13c0 0x7fcdd415b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdcbe00640 1 -- 172.21.15.16:0/2928300042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd41fba50 con 0x7fcdd40cfc80 2024-09-10T21:10:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdd9000640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcdd415f400 0x7fcdd4158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdd3e00640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd40b13c0 0x7fcdd415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdcbe00640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd40cfc80 0x7fcdd4153de0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009850 tx=0x7fcdbc0a5590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdd0c00640 1 -- 172.21.15.16:0/2928300042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0b2040 con 0x7fcdd40cfc80 2024-09-10T21:10:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdd0c00640 1 -- 172.21.15.16:0/2928300042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdbc004380 con 0x7fcdd40cfc80 2024-09-10T21:10:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcddb000640 1 -- 172.21.15.16:0/2928300042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd41fbd30 con 0x7fcdd40cfc80 2024-09-10T21:10:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcdd0c00640 1 -- 172.21.15.16:0/2928300042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0052d0 con 0x7fcdd40cfc80 2024-09-10T21:10:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.037+0000 7fcddb000640 1 -- 172.21.15.16:0/2928300042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd41fc2f0 con 0x7fcdd40cfc80 2024-09-10T21:10:59.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.041+0000 7fcddb000640 1 -- 172.21.15.16:0/2928300042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd40d00d0 con 0x7fcdd40cfc80 2024-09-10T21:10:59.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.041+0000 7fcdd0c00640 1 -- 172.21.15.16:0/2928300042 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcdbc005470 con 0x7fcdd40cfc80 2024-09-10T21:10:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.041+0000 7fcdd0c00640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcdb00725e0 0x7fcdb0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.041+0000 7fcdd0c00640 1 -- 172.21.15.16:0/2928300042 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcdbc12cce0 con 0x7fcdd40cfc80 2024-09-10T21:10:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.041+0000 7fcdd3e00640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcdb00725e0 0x7fcdb0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.041+0000 7fcdd3e00640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcdb00725e0 0x7fcdb0074aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400bbc0 tx=0x7fcdc4005ef0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.045+0000 7fcdd0c00640 1 -- 172.21.15.16:0/2928300042 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcdbc0164d0 con 0x7fcdd40cfc80 2024-09-10T21:10:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcddb000640 1 -- 172.21.15.16:0/2928300042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcdd40ce6d0 con 0x7fcdd40cfc80 2024-09-10T21:10:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcdd0c00640 1 -- 172.21.15.16:0/2928300042 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fcdbc0f8330 con 0x7fcdd40cfc80 2024-09-10T21:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcddb000640 1 -- 172.21.15.16:0/2928300042 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcdb00725e0 msgr2=0x7fcdb0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcddb000640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcdb00725e0 0x7fcdb0074aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400bbc0 tx=0x7fcdc4005ef0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcddb000640 1 -- 172.21.15.16:0/2928300042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd40cfc80 msgr2=0x7fcdd4153de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcddb000640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd40cfc80 0x7fcdd4153de0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009850 tx=0x7fcdbc0a5590 comp rx=0 tx=0).stop 2024-09-10T21:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcddb000640 1 -- 172.21.15.16:0/2928300042 shutdown_connections 2024-09-10T21:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcddb000640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcdb00725e0 0x7fcdb0074aa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcddb000640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcdd415f400 0x7fcdd4158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcddb000640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdd40cfc80 0x7fcdd4153de0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcddb000640 1 --2- 172.21.15.16:0/2928300042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdd40b13c0 0x7fcdd415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcddb000640 1 -- 172.21.15.16:0/2928300042 >> 172.21.15.16:0/2928300042 conn(0x7fcdd409b130 msgr2=0x7fcdd415df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcddb000640 1 -- 172.21.15.16:0/2928300042 shutdown_connections 2024-09-10T21:10:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.233+0000 7fcddb000640 1 -- 172.21.15.16:0/2928300042 wait complete. 2024-09-10T21:10:59.331 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-10T21:10:59.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:59.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-10T21:10:59.355 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:10:59.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:59.355 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T21:10:59.407 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:10:59.407 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:59.419 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:10:59.419 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:59.471 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:10:59.471 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:10:59.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:10:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.669+0000 7f8c91c00640 1 -- 172.21.15.16:0/711057213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c8c161620 msgr2=0x7f8c8c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.669+0000 7f8c91c00640 1 --2- 172.21.15.16:0/711057213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c8c161620 0x7f8c8c070cf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0098f0 tx=0x7f8c7c01c8b0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.669+0000 7f8c91c00640 1 -- 172.21.15.16:0/711057213 shutdown_connections 2024-09-10T21:10:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.669+0000 7f8c91c00640 1 --2- 172.21.15.16:0/711057213 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c8c0b0b40 0x7f8c8c06e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.669+0000 7f8c91c00640 1 --2- 172.21.15.16:0/711057213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c8c161620 0x7f8c8c070cf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.669+0000 7f8c91c00640 1 -- 172.21.15.16:0/711057213 >> 172.21.15.16:0/711057213 conn(0x7f8c8c09b130 msgr2=0x7f8c8c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.669+0000 7f8c91c00640 1 -- 172.21.15.16:0/711057213 shutdown_connections 2024-09-10T21:10:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.669+0000 7f8c91c00640 1 -- 172.21.15.16:0/711057213 wait complete. 2024-09-10T21:10:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c91c00640 1 Processor -- start 2024-09-10T21:10:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c91c00640 1 -- start start 2024-09-10T21:10:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c91c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c8c071230 0x7f8c8c14e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c91c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c8c0b0b40 0x7f8c8c14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c91c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c8c161620 0x7f8c8c0c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c91c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c16b640 con 0x7f8c8c0b0b40 2024-09-10T21:10:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c91c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c16b4c0 con 0x7f8c8c161620 2024-09-10T21:10:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c91c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8c8c16b7c0 con 0x7f8c8c071230 2024-09-10T21:10:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c82000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c8c0b0b40 0x7f8c8c14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c82000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c8c0b0b40 0x7f8c8c14cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59926/0 (socket says 172.21.15.16:59926) 2024-09-10T21:10:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c82000640 1 -- 172.21.15.16:0/35984802 learned_addr learned my addr 172.21.15.16:0/35984802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:10:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c8aa00640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c8c071230 0x7f8c8c14e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c8b400640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c8c161620 0x7f8c8c0c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c82000640 1 -- 172.21.15.16:0/35984802 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c8c071230 msgr2=0x7f8c8c14e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c82000640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c8c071230 0x7f8c8c14e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c82000640 1 -- 172.21.15.16:0/35984802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c8c161620 msgr2=0x7f8c8c0c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c82000640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c8c161620 0x7f8c8c0c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c82000640 1 -- 172.21.15.16:0/35984802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8c1f91b0 con 0x7f8c8c0b0b40 2024-09-10T21:10:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c82000640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c8c0b0b40 0x7f8c8c14cc50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c410 tx=0x7f8c7400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c83400640 1 -- 172.21.15.16:0/35984802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74017070 con 0x7f8c8c0b0b40 2024-09-10T21:10:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c83400640 1 -- 172.21.15.16:0/35984802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c74003600 con 0x7f8c8c0b0b40 2024-09-10T21:10:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c91c00640 1 -- 172.21.15.16:0/35984802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8c14d780 con 0x7f8c8c0b0b40 2024-09-10T21:10:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c83400640 1 -- 172.21.15.16:0/35984802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74003a50 con 0x7f8c8c0b0b40 2024-09-10T21:10:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c91c00640 1 -- 172.21.15.16:0/35984802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c8c14dd40 con 0x7f8c8c0b0b40 2024-09-10T21:10:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.673+0000 7f8c91c00640 1 -- 172.21.15.16:0/35984802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c8c06e020 con 0x7f8c8c0b0b40 2024-09-10T21:10:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.677+0000 7f8c83400640 1 -- 172.21.15.16:0/35984802 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8c74003bf0 con 0x7f8c8c0b0b40 2024-09-10T21:10:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.677+0000 7f8c83400640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8c680725e0 0x7f8c68074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.677+0000 7f8c83400640 1 -- 172.21.15.16:0/35984802 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8c740927a0 con 0x7f8c8c0b0b40 2024-09-10T21:10:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.677+0000 7f8c8aa00640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8c680725e0 0x7f8c68074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.677+0000 7f8c8aa00640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8c680725e0 0x7f8c68074aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0af000 tx=0x7f8c7c01b0f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.681+0000 7f8c83400640 1 -- 172.21.15.16:0/35984802 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8c7405d820 con 0x7f8c8c0b0b40 2024-09-10T21:10:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.865+0000 7f8c91c00640 1 -- 172.21.15.16:0/35984802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8c8c1591e0 con 0x7f8c8c0b0b40 2024-09-10T21:10:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.869+0000 7f8c83400640 1 -- 172.21.15.16:0/35984802 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f8c74062250 con 0x7f8c8c0b0b40 2024-09-10T21:10:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:10:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.869+0000 7f8c91c00640 1 -- 172.21.15.16:0/35984802 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8c680725e0 msgr2=0x7f8c68074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.869+0000 7f8c91c00640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8c680725e0 0x7f8c68074aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0af000 tx=0x7f8c7c01b0f0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.869+0000 7f8c91c00640 1 -- 172.21.15.16:0/35984802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c8c0b0b40 msgr2=0x7f8c8c14cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.869+0000 7f8c91c00640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c8c0b0b40 0x7f8c8c14cc50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c410 tx=0x7f8c7400c8d0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.869+0000 7f8c91c00640 1 -- 172.21.15.16:0/35984802 shutdown_connections 2024-09-10T21:10:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.869+0000 7f8c91c00640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8c680725e0 0x7f8c68074aa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.869+0000 7f8c91c00640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c8c161620 0x7f8c8c0c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.869+0000 7f8c91c00640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c8c0b0b40 0x7f8c8c14cc50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.869+0000 7f8c91c00640 1 --2- 172.21.15.16:0/35984802 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c8c071230 0x7f8c8c14e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.869+0000 7f8c91c00640 1 -- 172.21.15.16:0/35984802 >> 172.21.15.16:0/35984802 conn(0x7f8c8c09b130 msgr2=0x7f8c8c14e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.869+0000 7f8c91c00640 1 -- 172.21.15.16:0/35984802 shutdown_connections 2024-09-10T21:10:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:10:59.869+0000 7f8c91c00640 1 -- 172.21.15.16:0/35984802 wait complete. 2024-09-10T21:10:59.895 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-10T21:10:59.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:59.896 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-10T21:10:59.923 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:10:59.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:10:59.924 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T21:10:59.975 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:10:59.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:11:00.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.117+0000 7f502a000640 1 -- 172.21.15.16:0/1956820520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f502415f450 msgr2=0x7f50240cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.117+0000 7f502a000640 1 --2- 172.21.15.16:0/1956820520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f502415f450 0x7f50240cfc80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5014009900 tx=0x7f501401f070 comp rx=0 tx=0).stop 2024-09-10T21:11:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.117+0000 7f502a000640 1 -- 172.21.15.16:0/1956820520 shutdown_connections 2024-09-10T21:11:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.117+0000 7f502a000640 1 --2- 172.21.15.16:0/1956820520 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50240d0890 0x7f50240d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.117+0000 7f502a000640 1 --2- 172.21.15.16:0/1956820520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f502415f450 0x7f50240cfc80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.117+0000 7f502a000640 1 -- 172.21.15.16:0/1956820520 >> 172.21.15.16:0/1956820520 conn(0x7f50240d1d80 msgr2=0x7f50240d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.117+0000 7f502a000640 1 -- 172.21.15.16:0/1956820520 shutdown_connections 2024-09-10T21:11:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.117+0000 7f502a000640 1 -- 172.21.15.16:0/1956820520 wait complete. 2024-09-10T21:11:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.117+0000 7f502a000640 1 Processor -- start 2024-09-10T21:11:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.117+0000 7f502a000640 1 -- start start 2024-09-10T21:11:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f502a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50240d0890 0x7f50241f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f502a000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f502415f450 0x7f50240c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:00.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f502a000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50240c7fc0 0x7f50240ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f502a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50240b2e30 con 0x7f50240d0890 2024-09-10T21:11:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f502a000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f50240b2cb0 con 0x7f50240c7fc0 2024-09-10T21:11:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f502a000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f50240b2fb0 con 0x7f502415f450 2024-09-10T21:11:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f5023400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50240d0890 0x7f50241f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f5023400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50240d0890 0x7f50241f3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59928/0 (socket says 172.21.15.16:59928) 2024-09-10T21:11:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f5023400640 1 -- 172.21.15.16:0/4206535062 learned_addr learned my addr 172.21.15.16:0/4206535062 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f5023e00640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50240c7fc0 0x7f50240ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f501aa00640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f502415f450 0x7f50240c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f5023400640 1 -- 172.21.15.16:0/4206535062 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f502415f450 msgr2=0x7f50240c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f5023400640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f502415f450 0x7f50240c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f5023400640 1 -- 172.21.15.16:0/4206535062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50240c7fc0 msgr2=0x7f50240ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f5023400640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50240c7fc0 0x7f50240ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f5023400640 1 -- 172.21.15.16:0/4206535062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50241f6b90 con 0x7f50240d0890 2024-09-10T21:11:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f5023e00640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50240c7fc0 0x7f50240ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f5023400640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50240d0890 0x7f50241f3ad0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f501401bce0 tx=0x7f5014005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f501be00640 1 -- 172.21.15.16:0/4206535062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50140be070 con 0x7f50240d0890 2024-09-10T21:11:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f501be00640 1 -- 172.21.15.16:0/4206535062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50140b08e0 con 0x7f50240d0890 2024-09-10T21:11:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f502a000640 1 -- 172.21.15.16:0/4206535062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50240cdea0 con 0x7f50240d0890 2024-09-10T21:11:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f501be00640 1 -- 172.21.15.16:0/4206535062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50140b5040 con 0x7f50240d0890 2024-09-10T21:11:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f502a000640 1 -- 172.21.15.16:0/4206535062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50240ceec0 con 0x7f50240d0890 2024-09-10T21:11:00.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f502a000640 1 -- 172.21.15.16:0/4206535062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50240c83d0 con 0x7f50240d0890 2024-09-10T21:11:00.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.121+0000 7f501be00640 1 -- 172.21.15.16:0/4206535062 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f50140b8070 con 0x7f50240d0890 2024-09-10T21:11:00.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.125+0000 7f501be00640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5000072610 0x7f5000074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:00.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.125+0000 7f501be00640 1 -- 172.21.15.16:0/4206535062 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f501412fd10 con 0x7f50240d0890 2024-09-10T21:11:00.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.125+0000 7f501aa00640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5000072610 0x7f5000074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:00.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.125+0000 7f501aa00640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5000072610 0x7f5000074ad0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5024154000 tx=0x7f500c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:00.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.129+0000 7f501be00640 1 -- 172.21.15.16:0/4206535062 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f50240c83d0 con 0x7f50240d0890 2024-09-10T21:11:00.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f502a000640 1 -- 172.21.15.16:0/4206535062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50240b1e90 con 0x7f50240d0890 2024-09-10T21:11:00.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f501be00640 1 -- 172.21.15.16:0/4206535062 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f5014135020 con 0x7f50240d0890 2024-09-10T21:11:00.343 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:00.343 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-10T21:10:33:173828+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2107270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2107270748},{"type":"v1","addr":"172.21.15.16:6836","nonce":2107270748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/591195258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":591195258},{"type":"v1","addr":"172.21.15.16:6837","nonce":591195258}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2924498152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2924498152},{"type":"v1","addr":"172.21.15.52:6834","nonce":2924498152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:10:29.560192+0000","modified":"2024-09-10T21:10:32.575127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4356":{"gid":4356,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/3067177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3067177622},{"type":"v1","addr":"172.21.15.52:6835","nonce":3067177622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4356,"qdb_cluster":[4356]},"id":1}]} 2024-09-10T21:11:00.343 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-10T21:11:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f502a000640 1 -- 172.21.15.16:0/4206535062 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5000072610 msgr2=0x7f5000074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f502a000640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5000072610 0x7f5000074ad0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5024154000 tx=0x7f500c009040 comp rx=0 tx=0).stop 2024-09-10T21:11:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f502a000640 1 -- 172.21.15.16:0/4206535062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50240d0890 msgr2=0x7f50241f3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f502a000640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50240d0890 0x7f50241f3ad0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f501401bce0 tx=0x7f5014005790 comp rx=0 tx=0).stop 2024-09-10T21:11:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f502a000640 1 -- 172.21.15.16:0/4206535062 shutdown_connections 2024-09-10T21:11:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f502a000640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5000072610 0x7f5000074ad0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f502a000640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50240c7fc0 0x7f50240ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f502a000640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f502415f450 0x7f50240c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f502a000640 1 --2- 172.21.15.16:0/4206535062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50240d0890 0x7f50241f3ad0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f502a000640 1 -- 172.21.15.16:0/4206535062 >> 172.21.15.16:0/4206535062 conn(0x7f50240d1d80 msgr2=0x7f502415d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:00.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f502a000640 1 -- 172.21.15.16:0/4206535062 shutdown_connections 2024-09-10T21:11:00.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.337+0000 7f502a000640 1 -- 172.21.15.16:0/4206535062 wait complete. 2024-09-10T21:11:00.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:11:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.505+0000 7f306a000640 1 -- 172.21.15.16:0/1163960018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f304c003ad0 msgr2=0x7f304c003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:00.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.505+0000 7f306a000640 1 --2- 172.21.15.16:0/1163960018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f304c003ad0 0x7f304c003ef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f30580016f0 tx=0x7f305801c690 comp rx=0 tx=0).stop 2024-09-10T21:11:00.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.505+0000 7f306a000640 1 -- 172.21.15.16:0/1163960018 shutdown_connections 2024-09-10T21:11:00.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.505+0000 7f306a000640 1 --2- 172.21.15.16:0/1163960018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f304c003ad0 0x7f304c003ef0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.505+0000 7f306a000640 1 -- 172.21.15.16:0/1163960018 >> 172.21.15.16:0/1163960018 conn(0x7f30640d1d00 msgr2=0x7f30640d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:00.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.505+0000 7f306a000640 1 -- 172.21.15.16:0/1163960018 shutdown_connections 2024-09-10T21:11:00.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.505+0000 7f306a000640 1 -- 172.21.15.16:0/1163960018 wait complete. 2024-09-10T21:11:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f306a000640 1 Processor -- start 2024-09-10T21:11:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f306a000640 1 -- start start 2024-09-10T21:11:00.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f306a000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f304c003ad0 0x7f30640c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:00.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f306a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30641521b0 0x7f30640c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:00.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f306a000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3064151b60 0x7f3064159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:00.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f306a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30640ce890 con 0x7f30641521b0 2024-09-10T21:11:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f306a000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f30640ce710 con 0x7f304c003ad0 2024-09-10T21:11:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f306a000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f30640cea10 con 0x7f3064151b60 2024-09-10T21:11:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3063e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30641521b0 0x7f30640c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3063e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30641521b0 0x7f30640c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59946/0 (socket says 172.21.15.16:59946) 2024-09-10T21:11:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3063e00640 1 -- 172.21.15.16:0/4124041311 learned_addr learned my addr 172.21.15.16:0/4124041311 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3068c00640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f304c003ad0 0x7f30640c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3069600640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3064151b60 0x7f3064159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3063e00640 1 -- 172.21.15.16:0/4124041311 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3064151b60 msgr2=0x7f3064159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3063e00640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3064151b60 0x7f3064159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3063e00640 1 -- 172.21.15.16:0/4124041311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f304c003ad0 msgr2=0x7f30640c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3063e00640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f304c003ad0 0x7f30640c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3063e00640 1 -- 172.21.15.16:0/4124041311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30640cba40 con 0x7f30641521b0 2024-09-10T21:11:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3069600640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3064151b60 0x7f3064159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3063e00640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30641521b0 0x7f30640c7f40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3050012640 tx=0x7f3050012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3060c00640 1 -- 172.21.15.16:0/4124041311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500139b0 con 0x7f30641521b0 2024-09-10T21:11:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3060c00640 1 -- 172.21.15.16:0/4124041311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3050013b50 con 0x7f30641521b0 2024-09-10T21:11:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f3060c00640 1 -- 172.21.15.16:0/4124041311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050010480 con 0x7f30641521b0 2024-09-10T21:11:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f306a000640 1 -- 172.21.15.16:0/4124041311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30640cbd20 con 0x7f30641521b0 2024-09-10T21:11:00.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f306a000640 1 -- 172.21.15.16:0/4124041311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30640c7a20 con 0x7f30641521b0 2024-09-10T21:11:00.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.509+0000 7f306a000640 1 -- 172.21.15.16:0/4124041311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30640b2a70 con 0x7f30641521b0 2024-09-10T21:11:00.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.513+0000 7f3060c00640 1 -- 172.21.15.16:0/4124041311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3050010620 con 0x7f30641521b0 2024-09-10T21:11:00.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.513+0000 7f3060c00640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3044072610 0x7f3044074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:00.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.513+0000 7f3060c00640 1 -- 172.21.15.16:0/4124041311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f30500970f0 con 0x7f30641521b0 2024-09-10T21:11:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.513+0000 7f3068c00640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3044072610 0x7f3044074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.513+0000 7f3068c00640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3044072610 0x7f3044074ad0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f30580016c0 tx=0x7f30580a8040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:00.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.517+0000 7f3060c00640 1 -- 172.21.15.16:0/4124041311 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3050062170 con 0x7f30641521b0 2024-09-10T21:11:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f306a000640 1 -- 172.21.15.16:0/4124041311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30640c7bf0 con 0x7f30641521b0 2024-09-10T21:11:00.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f3060c00640 1 -- 172.21.15.16:0/4124041311 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f3050066ba0 con 0x7f30641521b0 2024-09-10T21:11:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:00.734 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-10T21:10:33:173828+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2107270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2107270748},{"type":"v1","addr":"172.21.15.16:6836","nonce":2107270748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/591195258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":591195258},{"type":"v1","addr":"172.21.15.16:6837","nonce":591195258}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2924498152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2924498152},{"type":"v1","addr":"172.21.15.52:6834","nonce":2924498152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:10:29.560192+0000","modified":"2024-09-10T21:10:32.575127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4356":{"gid":4356,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/3067177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3067177622},{"type":"v1","addr":"172.21.15.52:6835","nonce":3067177622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4356,"qdb_cluster":[4356]},"id":1}]} 2024-09-10T21:11:00.734 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-10T21:11:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f306a000640 1 -- 172.21.15.16:0/4124041311 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3044072610 msgr2=0x7f3044074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f306a000640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3044072610 0x7f3044074ad0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f30580016c0 tx=0x7f30580a8040 comp rx=0 tx=0).stop 2024-09-10T21:11:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f306a000640 1 -- 172.21.15.16:0/4124041311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30641521b0 msgr2=0x7f30640c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f306a000640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30641521b0 0x7f30640c7f40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3050012640 tx=0x7f3050012b00 comp rx=0 tx=0).stop 2024-09-10T21:11:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f306a000640 1 -- 172.21.15.16:0/4124041311 shutdown_connections 2024-09-10T21:11:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f306a000640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3044072610 0x7f3044074ad0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f306a000640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3064151b60 0x7f3064159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f306a000640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30641521b0 0x7f30640c7f40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f306a000640 1 --2- 172.21.15.16:0/4124041311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f304c003ad0 0x7f30640c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f306a000640 1 -- 172.21.15.16:0/4124041311 >> 172.21.15.16:0/4124041311 conn(0x7f30640d1d00 msgr2=0x7f306415f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f306a000640 1 -- 172.21.15.16:0/4124041311 shutdown_connections 2024-09-10T21:11:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.729+0000 7f306a000640 1 -- 172.21.15.16:0/4124041311 wait complete. 2024-09-10T21:11:00.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:11:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 -- 172.21.15.16:0/2007314209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444154f90 msgr2=0x7f8444072d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 --2- 172.21.15.16:0/2007314209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444154f90 0x7f8444072d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f84440c3350 tx=0x7f843801c890 comp rx=0 tx=0).stop 2024-09-10T21:11:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 -- 172.21.15.16:0/2007314209 shutdown_connections 2024-09-10T21:11:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 --2- 172.21.15.16:0/2007314209 >> v2:172.21.15.52:3300/0 conn(0x7f84440706f0 0x7f84440ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 --2- 172.21.15.16:0/2007314209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444154f90 0x7f8444072d40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 -- 172.21.15.16:0/2007314209 >> 172.21.15.16:0/2007314209 conn(0x7f84440cfb20 msgr2=0x7f84440cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 -- 172.21.15.16:0/2007314209 shutdown_connections 2024-09-10T21:11:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 -- 172.21.15.16:0/2007314209 wait complete. 2024-09-10T21:11:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 Processor -- start 2024-09-10T21:11:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 -- start start 2024-09-10T21:11:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84440706f0 0x7f844415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444154f90 0x7f844414a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f844415bc70 0x7f84441539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84441678f0 con 0x7f8444154f90 2024-09-10T21:11:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8444167770 con 0x7f84440706f0 2024-09-10T21:11:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f844be00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8444167a70 con 0x7f844415bc70 2024-09-10T21:11:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f8448a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444154f90 0x7f844414a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f8448a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444154f90 0x7f844414a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59962/0 (socket says 172.21.15.16:59962) 2024-09-10T21:11:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f8448a00640 1 -- 172.21.15.16:0/224371964 learned_addr learned my addr 172.21.15.16:0/224371964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.897+0000 7f8449400640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84440706f0 0x7f844415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f8449e00640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f844415bc70 0x7f84441539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f8448a00640 1 -- 172.21.15.16:0/224371964 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f844415bc70 msgr2=0x7f84441539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f8448a00640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f844415bc70 0x7f84441539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f8448a00640 1 -- 172.21.15.16:0/224371964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84440706f0 msgr2=0x7f844415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f8448a00640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84440706f0 0x7f844415a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f8448a00640 1 -- 172.21.15.16:0/224371964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844414a6b0 con 0x7f8444154f90 2024-09-10T21:11:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f8449400640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84440706f0 0x7f844415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f8448a00640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444154f90 0x7f844414a170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8430001ec0 tx=0x7f843000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:00.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f8441600640 1 -- 172.21.15.16:0/224371964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430017070 con 0x7f8444154f90 2024-09-10T21:11:00.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f8441600640 1 -- 172.21.15.16:0/224371964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8430003600 con 0x7f8444154f90 2024-09-10T21:11:00.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f844be00640 1 -- 172.21.15.16:0/224371964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84440b36e0 con 0x7f8444154f90 2024-09-10T21:11:00.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f8441600640 1 -- 172.21.15.16:0/224371964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430003a50 con 0x7f8444154f90 2024-09-10T21:11:00.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f844be00640 1 -- 172.21.15.16:0/224371964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844414e6f0 con 0x7f8444154f90 2024-09-10T21:11:00.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f844be00640 1 -- 172.21.15.16:0/224371964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84440ca510 con 0x7f8444154f90 2024-09-10T21:11:00.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f8441600640 1 -- 172.21.15.16:0/224371964 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8430003bf0 con 0x7f8444154f90 2024-09-10T21:11:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.901+0000 7f8441600640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f84240725e0 0x7f8424074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.905+0000 7f8441600640 1 -- 172.21.15.16:0/224371964 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8430092800 con 0x7f8444154f90 2024-09-10T21:11:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.905+0000 7f8449400640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f84240725e0 0x7f8424074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.905+0000 7f8449400640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f84240725e0 0x7f8424074aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f84380af000 tx=0x7f843801b0d0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:00.905+0000 7f8441600640 1 -- 172.21.15.16:0/224371964 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f843005d880 con 0x7f8444154f90 2024-09-10T21:11:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.081+0000 7f844be00640 1 -- 172.21.15.16:0/224371964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84441546e0 con 0x7f8444154f90 2024-09-10T21:11:01.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.081+0000 7f8441600640 1 -- 172.21.15.16:0/224371964 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7f84300622b0 con 0x7f8444154f90 2024-09-10T21:11:01.088 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:01.089 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:10:31.158120+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:10:27.523741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:10:28.533379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:11:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.085+0000 7f844be00640 1 -- 172.21.15.16:0/224371964 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f84240725e0 msgr2=0x7f8424074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.085+0000 7f844be00640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f84240725e0 0x7f8424074aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f84380af000 tx=0x7f843801b0d0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.085+0000 7f844be00640 1 -- 172.21.15.16:0/224371964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444154f90 msgr2=0x7f844414a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.085+0000 7f844be00640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444154f90 0x7f844414a170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8430001ec0 tx=0x7f843000c890 comp rx=0 tx=0).stop 2024-09-10T21:11:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.085+0000 7f844be00640 1 -- 172.21.15.16:0/224371964 shutdown_connections 2024-09-10T21:11:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.085+0000 7f844be00640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f84240725e0 0x7f8424074aa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.085+0000 7f844be00640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f844415bc70 0x7f84441539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.085+0000 7f844be00640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444154f90 0x7f844414a170 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.085+0000 7f844be00640 1 --2- 172.21.15.16:0/224371964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84440706f0 0x7f844415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.085+0000 7f844be00640 1 -- 172.21.15.16:0/224371964 >> 172.21.15.16:0/224371964 conn(0x7f84440cfb20 msgr2=0x7f844414fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:01.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.085+0000 7f844be00640 1 -- 172.21.15.16:0/224371964 shutdown_connections 2024-09-10T21:11:01.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.085+0000 7f844be00640 1 -- 172.21.15.16:0/224371964 wait complete. 2024-09-10T21:11:01.111 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:11:01.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:11:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.253+0000 7ff81ae00640 1 -- 172.21.15.16:0/316939437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8003a60 msgr2=0x7ff7f8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.253+0000 7ff81ae00640 1 --2- 172.21.15.16:0/316939437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8003a60 0x7ff7f8003e80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff804009b50 tx=0x7ff80401d290 comp rx=0 tx=0).stop 2024-09-10T21:11:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.253+0000 7ff818e00640 1 -- 172.21.15.16:0/316939437 reap_dead start 2024-09-10T21:11:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.253+0000 7ff81ae00640 1 -- 172.21.15.16:0/316939437 shutdown_connections 2024-09-10T21:11:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.253+0000 7ff81ae00640 1 -- 172.21.15.16:0/316939437 >> 172.21.15.16:0/316939437 conn(0x7ff81409b130 msgr2=0x7ff8140d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.253+0000 7ff81ae00640 1 -- 172.21.15.16:0/316939437 shutdown_connections 2024-09-10T21:11:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.253+0000 7ff81ae00640 1 -- 172.21.15.16:0/316939437 wait complete. 2024-09-10T21:11:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff81ae00640 1 Processor -- start 2024-09-10T21:11:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff81ae00640 1 -- start start 2024-09-10T21:11:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff81ae00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f8003a60 0x7ff81414f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff81ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff814165d30 0x7ff814166170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff81ae00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff814161cd0 0x7ff814162150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff81ae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8140718b0 con 0x7ff814165d30 2024-09-10T21:11:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff81ae00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff814071730 con 0x7ff7f8003a60 2024-09-10T21:11:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff81ae00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff8141673d0 con 0x7ff814161cd0 2024-09-10T21:11:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff80be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff814165d30 0x7ff814166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff80be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff814165d30 0x7ff814166170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59978/0 (socket says 172.21.15.16:59978) 2024-09-10T21:11:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff80be00640 1 -- 172.21.15.16:0/1406799605 learned_addr learned my addr 172.21.15.16:0/1406799605 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff813e00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f8003a60 0x7ff81414f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff818e00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff814161cd0 0x7ff814162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff80be00640 1 -- 172.21.15.16:0/1406799605 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff814161cd0 msgr2=0x7ff814162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff80be00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff814161cd0 0x7ff814162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff80be00640 1 -- 172.21.15.16:0/1406799605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f8003a60 msgr2=0x7ff81414f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff80be00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f8003a60 0x7ff81414f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff80be00640 1 -- 172.21.15.16:0/1406799605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8140b2ea0 con 0x7ff814165d30 2024-09-10T21:11:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff80be00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff814165d30 0x7ff814166170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc011880 tx=0x7ff7fc011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff810c00640 1 -- 172.21.15.16:0/1406799605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc00fcf0 con 0x7ff814165d30 2024-09-10T21:11:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff810c00640 1 -- 172.21.15.16:0/1406799605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7fc005020 con 0x7ff814165d30 2024-09-10T21:11:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff810c00640 1 -- 172.21.15.16:0/1406799605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc005430 con 0x7ff814165d30 2024-09-10T21:11:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff81ae00640 1 -- 172.21.15.16:0/1406799605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8140b3180 con 0x7ff814165d30 2024-09-10T21:11:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.257+0000 7ff81ae00640 1 -- 172.21.15.16:0/1406799605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8140b3740 con 0x7ff814165d30 2024-09-10T21:11:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.261+0000 7ff81ae00640 1 -- 172.21.15.16:0/1406799605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff814070fe0 con 0x7ff814165d30 2024-09-10T21:11:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.261+0000 7ff810c00640 1 -- 172.21.15.16:0/1406799605 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff7fc0055d0 con 0x7ff814165d30 2024-09-10T21:11:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.261+0000 7ff810c00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff7f00725e0 0x7ff7f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.261+0000 7ff810c00640 1 -- 172.21.15.16:0/1406799605 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff7fc093f10 con 0x7ff814165d30 2024-09-10T21:11:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.261+0000 7ff813e00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff7f00725e0 0x7ff7f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.261+0000 7ff813e00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff7f00725e0 0x7ff7f0074aa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff8040ad000 tx=0x7ff8040af040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.265+0000 7ff810c00640 1 -- 172.21.15.16:0/1406799605 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff7fc05f550 con 0x7ff814165d30 2024-09-10T21:11:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff81ae00640 1 -- 172.21.15.16:0/1406799605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8141666b0 con 0x7ff814165d30 2024-09-10T21:11:01.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff810c00640 1 -- 172.21.15.16:0/1406799605 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7ff7fc05ee50 con 0x7ff814165d30 2024-09-10T21:11:01.483 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:01.483 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-10T21:10:33:173828+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2107270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2107270748},{"type":"v1","addr":"172.21.15.16:6836","nonce":2107270748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/591195258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":591195258},{"type":"v1","addr":"172.21.15.16:6837","nonce":591195258}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2924498152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2924498152},{"type":"v1","addr":"172.21.15.52:6834","nonce":2924498152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:10:29.560192+0000","modified":"2024-09-10T21:10:32.575127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4356":{"gid":4356,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/3067177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3067177622},{"type":"v1","addr":"172.21.15.52:6835","nonce":3067177622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4356,"qdb_cluster":[4356]},"id":1}]} 2024-09-10T21:11:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-10T21:11:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff81ae00640 1 -- 172.21.15.16:0/1406799605 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff7f00725e0 msgr2=0x7ff7f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff81ae00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff7f00725e0 0x7ff7f0074aa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff8040ad000 tx=0x7ff8040af040 comp rx=0 tx=0).stop 2024-09-10T21:11:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff81ae00640 1 -- 172.21.15.16:0/1406799605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff814165d30 msgr2=0x7ff814166170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff81ae00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff814165d30 0x7ff814166170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc011880 tx=0x7ff7fc011d40 comp rx=0 tx=0).stop 2024-09-10T21:11:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff81ae00640 1 -- 172.21.15.16:0/1406799605 shutdown_connections 2024-09-10T21:11:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff81ae00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff7f00725e0 0x7ff7f0074aa0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff81ae00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff814161cd0 0x7ff814162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff81ae00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff814165d30 0x7ff814166170 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff81ae00640 1 --2- 172.21.15.16:0/1406799605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f8003a60 0x7ff81414f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff81ae00640 1 -- 172.21.15.16:0/1406799605 >> 172.21.15.16:0/1406799605 conn(0x7ff81409b130 msgr2=0x7ff81406e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff81ae00640 1 -- 172.21.15.16:0/1406799605 shutdown_connections 2024-09-10T21:11:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.477+0000 7ff81ae00640 1 -- 172.21.15.16:0/1406799605 wait complete. 2024-09-10T21:11:01.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:11:01.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.645+0000 7ff3d7a00640 1 -- 172.21.15.16:0/4230996489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d00b0b40 msgr2=0x7ff3d00d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.645+0000 7ff3d7a00640 1 --2- 172.21.15.16:0/4230996489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d00b0b40 0x7ff3d00d0590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009900 tx=0x7ff3b801c8b0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.645+0000 7ff3d7a00640 1 -- 172.21.15.16:0/4230996489 shutdown_connections 2024-09-10T21:11:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.645+0000 7ff3d7a00640 1 --2- 172.21.15.16:0/4230996489 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3d00d0ad0 0x7ff3d006e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.645+0000 7ff3d7a00640 1 --2- 172.21.15.16:0/4230996489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d00b0b40 0x7ff3d00d0590 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.645+0000 7ff3d7a00640 1 -- 172.21.15.16:0/4230996489 >> 172.21.15.16:0/4230996489 conn(0x7ff3d009b130 msgr2=0x7ff3d00d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.645+0000 7ff3d7a00640 1 -- 172.21.15.16:0/4230996489 shutdown_connections 2024-09-10T21:11:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.645+0000 7ff3d7a00640 1 -- 172.21.15.16:0/4230996489 wait complete. 2024-09-10T21:11:01.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3d7a00640 1 Processor -- start 2024-09-10T21:11:01.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3d7a00640 1 -- start start 2024-09-10T21:11:01.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3d7a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3d00b0b40 0x7ff3d01523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3d7a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d00d0ad0 0x7ff3d00b6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3d7a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3d0161620 0x7ff3d00b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3d7a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d016b100 con 0x7ff3d00d0ad0 2024-09-10T21:11:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3d7a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d016af80 con 0x7ff3d0161620 2024-09-10T21:11:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3d7a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d016b280 con 0x7ff3d00b0b40 2024-09-10T21:11:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3c7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d00d0ad0 0x7ff3d00b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3c7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d00d0ad0 0x7ff3d00b6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60002/0 (socket says 172.21.15.16:60002) 2024-09-10T21:11:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3c7e00640 1 -- 172.21.15.16:0/450188790 learned_addr learned my addr 172.21.15.16:0/450188790 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3d5a00640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3d0161620 0x7ff3d00b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3d5000640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3d00b0b40 0x7ff3d01523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3c7e00640 1 -- 172.21.15.16:0/450188790 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3d00b0b40 msgr2=0x7ff3d01523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3c7e00640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3d00b0b40 0x7ff3d01523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3c7e00640 1 -- 172.21.15.16:0/450188790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3d0161620 msgr2=0x7ff3d00b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3c7e00640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3d0161620 0x7ff3d00b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3c7e00640 1 -- 172.21.15.16:0/450188790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d00c5ee0 con 0x7ff3d00d0ad0 2024-09-10T21:11:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3c7e00640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d00d0ad0 0x7ff3d00b6970 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009850 tx=0x7ff3b8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3cd600640 1 -- 172.21.15.16:0/450188790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80b2040 con 0x7ff3d00d0ad0 2024-09-10T21:11:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3cd600640 1 -- 172.21.15.16:0/450188790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3b8005070 con 0x7ff3d00d0ad0 2024-09-10T21:11:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3cd600640 1 -- 172.21.15.16:0/450188790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80044f0 con 0x7ff3d00d0ad0 2024-09-10T21:11:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3d7a00640 1 -- 172.21.15.16:0/450188790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d00b73c0 con 0x7ff3d00d0ad0 2024-09-10T21:11:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.649+0000 7ff3d7a00640 1 -- 172.21.15.16:0/450188790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d00b7980 con 0x7ff3d00d0ad0 2024-09-10T21:11:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.653+0000 7ff3d7a00640 1 -- 172.21.15.16:0/450188790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d00d0590 con 0x7ff3d00d0ad0 2024-09-10T21:11:01.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.653+0000 7ff3cd600640 1 -- 172.21.15.16:0/450188790 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff3b80054a0 con 0x7ff3d00d0ad0 2024-09-10T21:11:01.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.653+0000 7ff3cd600640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff3ac0725e0 0x7ff3ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.653+0000 7ff3cd600640 1 -- 172.21.15.16:0/450188790 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff3b812cf30 con 0x7ff3d00d0ad0 2024-09-10T21:11:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.653+0000 7ff3d5000640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff3ac0725e0 0x7ff3ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.653+0000 7ff3d5000640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff3ac0725e0 0x7ff3ac074aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0006fd0 tx=0x7ff3c0008040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.657+0000 7ff3cd600640 1 -- 172.21.15.16:0/450188790 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff3d00c5ee0 con 0x7ff3d00d0ad0 2024-09-10T21:11:01.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.865+0000 7ff3d7a00640 1 -- 172.21.15.16:0/450188790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3d00c5030 con 0x7ff3d00d0ad0 2024-09-10T21:11:01.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.865+0000 7ff3cd600640 1 -- 172.21.15.16:0/450188790 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7ff3b80164d0 con 0x7ff3d00d0ad0 2024-09-10T21:11:01.872 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:01.872 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-10T21:10:33:173828+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2107270748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2107270748},{"type":"v1","addr":"172.21.15.16:6836","nonce":2107270748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/591195258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":591195258},{"type":"v1","addr":"172.21.15.16:6837","nonce":591195258}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2924498152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2924498152},{"type":"v1","addr":"172.21.15.52:6834","nonce":2924498152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:10:29.560192+0000","modified":"2024-09-10T21:10:32.575127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4356":{"gid":4356,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/3067177622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3067177622},{"type":"v1","addr":"172.21.15.52:6835","nonce":3067177622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4356,"qdb_cluster":[4356]},"id":1}]} 2024-09-10T21:11:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-10T21:11:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.865+0000 7ff3d7a00640 1 -- 172.21.15.16:0/450188790 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff3ac0725e0 msgr2=0x7ff3ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.865+0000 7ff3d7a00640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff3ac0725e0 0x7ff3ac074aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0006fd0 tx=0x7ff3c0008040 comp rx=0 tx=0).stop 2024-09-10T21:11:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.865+0000 7ff3d7a00640 1 -- 172.21.15.16:0/450188790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d00d0ad0 msgr2=0x7ff3d00b6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.865+0000 7ff3d7a00640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d00d0ad0 0x7ff3d00b6970 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009850 tx=0x7ff3b8004040 comp rx=0 tx=0).stop 2024-09-10T21:11:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.869+0000 7ff3d7a00640 1 -- 172.21.15.16:0/450188790 shutdown_connections 2024-09-10T21:11:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.869+0000 7ff3d7a00640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff3ac0725e0 0x7ff3ac074aa0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.869+0000 7ff3d7a00640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3d0161620 0x7ff3d00b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.869+0000 7ff3d7a00640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d00d0ad0 0x7ff3d00b6970 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.869+0000 7ff3d7a00640 1 --2- 172.21.15.16:0/450188790 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3d00b0b40 0x7ff3d01523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.869+0000 7ff3d7a00640 1 -- 172.21.15.16:0/450188790 >> 172.21.15.16:0/450188790 conn(0x7ff3d009b130 msgr2=0x7ff3d00ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.869+0000 7ff3d7a00640 1 -- 172.21.15.16:0/450188790 shutdown_connections 2024-09-10T21:11:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:01.869+0000 7ff3d7a00640 1 -- 172.21.15.16:0/450188790 wait complete. 2024-09-10T21:11:01.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:11:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.033+0000 7fbac2400640 1 -- 172.21.15.16:0/4097480025 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbabc0b3200 msgr2=0x7fbabc0d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.033+0000 7fbac2400640 1 --2- 172.21.15.16:0/4097480025 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbabc0b3200 0x7fbabc0d01c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009900 tx=0x7fbaa401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.033+0000 7fbac2400640 1 -- 172.21.15.16:0/4097480025 shutdown_connections 2024-09-10T21:11:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.033+0000 7fbac2400640 1 --2- 172.21.15.16:0/4097480025 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbabc0b3200 0x7fbabc0d01c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.033+0000 7fbac2400640 1 --2- 172.21.15.16:0/4097480025 >> v2:172.21.15.52:3300/0 conn(0x7fbabc15f450 0x7fbabc0cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.033+0000 7fbac2400640 1 -- 172.21.15.16:0/4097480025 >> 172.21.15.16:0/4097480025 conn(0x7fbabc0d1d80 msgr2=0x7fbabc0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.033+0000 7fbac2400640 1 -- 172.21.15.16:0/4097480025 shutdown_connections 2024-09-10T21:11:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.033+0000 7fbac2400640 1 -- 172.21.15.16:0/4097480025 wait complete. 2024-09-10T21:11:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.033+0000 7fbac2400640 1 Processor -- start 2024-09-10T21:11:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.033+0000 7fbac2400640 1 -- start start 2024-09-10T21:11:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbac2400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbabc0d0890 0x7fbabc0c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbac2400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbabc15f450 0x7fbabc0cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbac2400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbabc0b6fe0 0x7fbabc154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbac2400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc0b2e30 con 0x7fbabc15f450 2024-09-10T21:11:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbac2400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc0b2cb0 con 0x7fbabc0d0890 2024-09-10T21:11:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbac2400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fbabc0b2fb0 con 0x7fbabc0b6fe0 2024-09-10T21:11:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbab2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbabc15f450 0x7fbabc0cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbab2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbabc15f450 0x7fbabc0cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60018/0 (socket says 172.21.15.16:60018) 2024-09-10T21:11:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbab2a00640 1 -- 172.21.15.16:0/833551478 learned_addr learned my addr 172.21.15.16:0/833551478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbabb400640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbabc0d0890 0x7fbabc0c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbabbe00640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbabc0b6fe0 0x7fbabc154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbab2a00640 1 -- 172.21.15.16:0/833551478 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbabc0b6fe0 msgr2=0x7fbabc154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbab2a00640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbabc0b6fe0 0x7fbabc154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbab2a00640 1 -- 172.21.15.16:0/833551478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbabc0d0890 msgr2=0x7fbabc0c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbab2a00640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbabc0d0890 0x7fbabc0c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbab2a00640 1 -- 172.21.15.16:0/833551478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbabc0c5df0 con 0x7fbabc15f450 2024-09-10T21:11:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbabbe00640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbabc0b6fe0 0x7fbabc154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbab2a00640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbabc15f450 0x7fbabc0cefc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40b3040 tx=0x7fbaa4005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbab3e00640 1 -- 172.21.15.16:0/833551478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa40be070 con 0x7fbabc15f450 2024-09-10T21:11:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbab3e00640 1 -- 172.21.15.16:0/833551478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaa40053e0 con 0x7fbabc15f450 2024-09-10T21:11:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbac2400640 1 -- 172.21.15.16:0/833551478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbabc0cadd0 con 0x7fbabc15f450 2024-09-10T21:11:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbab3e00640 1 -- 172.21.15.16:0/833551478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa40b5040 con 0x7fbabc15f450 2024-09-10T21:11:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbac2400640 1 -- 172.21.15.16:0/833551478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbabc09f2f0 con 0x7fbabc15f450 2024-09-10T21:11:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.037+0000 7fbac2400640 1 -- 172.21.15.16:0/833551478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbabc0cb220 con 0x7fbabc15f450 2024-09-10T21:11:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.041+0000 7fbab3e00640 1 -- 172.21.15.16:0/833551478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbaa40b8030 con 0x7fbabc15f450 2024-09-10T21:11:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.041+0000 7fbab3e00640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba980725d0 0x7fba98074a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.041+0000 7fbab3e00640 1 -- 172.21.15.16:0/833551478 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbaa412ff20 con 0x7fbabc15f450 2024-09-10T21:11:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.041+0000 7fbabb400640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba980725d0 0x7fba98074a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.041+0000 7fbabb400640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba980725d0 0x7fba98074a90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009b40 tx=0x7fbaac0091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.045+0000 7fbab3e00640 1 -- 172.21.15.16:0/833551478 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbabc0cadd0 con 0x7fbabc15f450 2024-09-10T21:11:02.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.225+0000 7fbac2400640 1 -- 172.21.15.16:0/833551478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbabc074ba0 con 0x7fbabc15f450 2024-09-10T21:11:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.225+0000 7fbab3e00640 1 -- 172.21.15.16:0/833551478 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16621 (secure 0 0 0) 0x7fbaa4018c30 con 0x7fbabc15f450 2024-09-10T21:11:02.231 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:02.232 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:10:31.158120+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:10:27.523741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:10:28.533379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:11:02.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.225+0000 7fbac2400640 1 -- 172.21.15.16:0/833551478 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba980725d0 msgr2=0x7fba98074a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:02.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.225+0000 7fbac2400640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba980725d0 0x7fba98074a90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009b40 tx=0x7fbaac0091e0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.225+0000 7fbac2400640 1 -- 172.21.15.16:0/833551478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbabc15f450 msgr2=0x7fbabc0cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.225+0000 7fbac2400640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbabc15f450 0x7fbabc0cefc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40b3040 tx=0x7fbaa4005e70 comp rx=0 tx=0).stop 2024-09-10T21:11:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.225+0000 7fbac2400640 1 -- 172.21.15.16:0/833551478 shutdown_connections 2024-09-10T21:11:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.225+0000 7fbac2400640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba980725d0 0x7fba98074a90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.225+0000 7fbac2400640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbabc0b6fe0 0x7fbabc154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.225+0000 7fbac2400640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbabc15f450 0x7fbabc0cefc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.225+0000 7fbac2400640 1 --2- 172.21.15.16:0/833551478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbabc0d0890 0x7fbabc0c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.225+0000 7fbac2400640 1 -- 172.21.15.16:0/833551478 >> 172.21.15.16:0/833551478 conn(0x7fbabc0d1d80 msgr2=0x7fbabc0b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:02.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.225+0000 7fbac2400640 1 -- 172.21.15.16:0/833551478 shutdown_connections 2024-09-10T21:11:02.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.229+0000 7fbac2400640 1 -- 172.21.15.16:0/833551478 wait complete. 2024-09-10T21:11:02.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:11:02.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.393+0000 7f2938c00640 1 -- 172.21.15.16:0/625350294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340cfb60 msgr2=0x7f29340cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:02.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.393+0000 7f2938c00640 1 --2- 172.21.15.16:0/625350294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340cfb60 0x7f29340cffc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2934074410 tx=0x7f291c01c800 comp rx=0 tx=0).stop 2024-09-10T21:11:02.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.393+0000 7f2938c00640 1 -- 172.21.15.16:0/625350294 shutdown_connections 2024-09-10T21:11:02.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.393+0000 7f2938c00640 1 --2- 172.21.15.16:0/625350294 >> v2:172.21.15.52:3300/0 conn(0x7f29340d0500 0x7f29340ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.393+0000 7f2938c00640 1 --2- 172.21.15.16:0/625350294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340cfb60 0x7f29340cffc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.393+0000 7f2938c00640 1 --2- 172.21.15.16:0/625350294 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f293415f350 0x7f29340cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.393+0000 7f2938c00640 1 -- 172.21.15.16:0/625350294 >> 172.21.15.16:0/625350294 conn(0x7f29340d1d80 msgr2=0x7f29340d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:02.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.393+0000 7f2938c00640 1 -- 172.21.15.16:0/625350294 shutdown_connections 2024-09-10T21:11:02.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.393+0000 7f2938c00640 1 -- 172.21.15.16:0/625350294 wait complete. 2024-09-10T21:11:02.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2938c00640 1 Processor -- start 2024-09-10T21:11:02.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2938c00640 1 -- start start 2024-09-10T21:11:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2938c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340cfb60 0x7f29340c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2938c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29340d0500 0x7f293406e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2938c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293415f350 0x7f293406ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2938c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29340af2f0 con 0x7f29340cfb60 2024-09-10T21:11:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2938c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f29340af170 con 0x7f293415f350 2024-09-10T21:11:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2938c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f29340af470 con 0x7f29340d0500 2024-09-10T21:11:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2931e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340cfb60 0x7f29340c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2931e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340cfb60 0x7f29340c7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60868/0 (socket says 172.21.15.16:60868) 2024-09-10T21:11:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2931e00640 1 -- 172.21.15.16:0/4018839802 learned_addr learned my addr 172.21.15.16:0/4018839802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2932800640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293415f350 0x7f293406ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f292be00640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29340d0500 0x7f293406e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2931e00640 1 -- 172.21.15.16:0/4018839802 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29340d0500 msgr2=0x7f293406e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2931e00640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29340d0500 0x7f293406e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2931e00640 1 -- 172.21.15.16:0/4018839802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293415f350 msgr2=0x7f293406ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2931e00640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293415f350 0x7f293406ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2931e00640 1 -- 172.21.15.16:0/4018839802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29340cba20 con 0x7f29340cfb60 2024-09-10T21:11:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2931e00640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340cfb60 0x7f29340c7a90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f292400c9f0 tx=0x7f292400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f292a000640 1 -- 172.21.15.16:0/4018839802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924007bb0 con 0x7f29340cfb60 2024-09-10T21:11:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f292a000640 1 -- 172.21.15.16:0/4018839802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2924007e40 con 0x7f29340cfb60 2024-09-10T21:11:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2938c00640 1 -- 172.21.15.16:0/4018839802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29340cb180 con 0x7f29340cfb60 2024-09-10T21:11:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f292a000640 1 -- 172.21.15.16:0/4018839802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292400f690 con 0x7f29340cfb60 2024-09-10T21:11:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.397+0000 7f2938c00640 1 -- 172.21.15.16:0/4018839802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29340cb740 con 0x7f29340cfb60 2024-09-10T21:11:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.401+0000 7f2938c00640 1 -- 172.21.15.16:0/4018839802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29340ce500 con 0x7f29340cfb60 2024-09-10T21:11:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.401+0000 7f292a000640 1 -- 172.21.15.16:0/4018839802 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2924020030 con 0x7f29340cfb60 2024-09-10T21:11:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.401+0000 7f292a000640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f29100725e0 0x7f2910074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.401+0000 7f292a000640 1 -- 172.21.15.16:0/4018839802 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f29240937e0 con 0x7f29340cfb60 2024-09-10T21:11:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.401+0000 7f292be00640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f29100725e0 0x7f2910074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.401+0000 7f292be00640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f29100725e0 0x7f2910074aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f291c0af000 tx=0x7f291c01b040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.405+0000 7f292a000640 1 -- 172.21.15.16:0/4018839802 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2924014030 con 0x7f29340cfb60 2024-09-10T21:11:02.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.613+0000 7f2938c00640 1 -- 172.21.15.16:0/4018839802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f29340b2530 con 0x7f29340cfb60 2024-09-10T21:11:02.621 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-10T21:11:02.617+0000 7ff429400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:11:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.617+0000 7f292a000640 1 -- 172.21.15.16:0/4018839802 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f292405e7d0 con 0x7f29340cfb60 2024-09-10T21:11:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:11:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.621+0000 7f2938c00640 1 -- 172.21.15.16:0/4018839802 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f29100725e0 msgr2=0x7f2910074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.621+0000 7f2938c00640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f29100725e0 0x7f2910074aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f291c0af000 tx=0x7f291c01b040 comp rx=0 tx=0).stop 2024-09-10T21:11:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.621+0000 7f2938c00640 1 -- 172.21.15.16:0/4018839802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340cfb60 msgr2=0x7f29340c7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.621+0000 7f2938c00640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340cfb60 0x7f29340c7a90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f292400c9f0 tx=0x7f292400ceb0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.621+0000 7f2938c00640 1 -- 172.21.15.16:0/4018839802 shutdown_connections 2024-09-10T21:11:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.621+0000 7f2938c00640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f29100725e0 0x7f2910074aa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.621+0000 7f2938c00640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293415f350 0x7f293406ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.621+0000 7f2938c00640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f29340d0500 0x7f293406e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.621+0000 7f2938c00640 1 --2- 172.21.15.16:0/4018839802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340cfb60 0x7f29340c7a90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.621+0000 7f2938c00640 1 -- 172.21.15.16:0/4018839802 >> 172.21.15.16:0/4018839802 conn(0x7f29340d1d80 msgr2=0x7f29340b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.621+0000 7f2938c00640 1 -- 172.21.15.16:0/4018839802 shutdown_connections 2024-09-10T21:11:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.621+0000 7f2938c00640 1 -- 172.21.15.16:0/4018839802 wait complete. 2024-09-10T21:11:02.636 INFO:tasks.ceph.mds.d.smithi052.stderr: -8286> 2024-09-10T21:10:24.120+0000 7fa2b7600640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-10T21:11:02.639 INFO:tasks.ceph.mds.d.smithi052.stderr: -6956> 2024-09-10T21:10:35.128+0000 7fa2afe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:11:02.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:11:02.693 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:11:02.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.789+0000 7f5b8e600640 1 -- 172.21.15.16:0/948824065 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b880d0210 msgr2=0x7f5b880ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.789+0000 7f5b8e600640 1 --2- 172.21.15.16:0/948824065 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b880d0210 0x7f5b880ceca0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009870 tx=0x7f5b7001f050 comp rx=0 tx=0).stop 2024-09-10T21:11:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.789+0000 7f5b8e600640 1 -- 172.21.15.16:0/948824065 shutdown_connections 2024-09-10T21:11:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.789+0000 7f5b8e600640 1 --2- 172.21.15.16:0/948824065 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b880d0210 0x7f5b880ceca0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.789+0000 7f5b8e600640 1 -- 172.21.15.16:0/948824065 >> 172.21.15.16:0/948824065 conn(0x7f5b880d1df0 msgr2=0x7f5b880d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.789+0000 7f5b8e600640 1 -- 172.21.15.16:0/948824065 shutdown_connections 2024-09-10T21:11:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.789+0000 7f5b8e600640 1 -- 172.21.15.16:0/948824065 wait complete. 2024-09-10T21:11:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b8e600640 1 Processor -- start 2024-09-10T21:11:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b8e600640 1 -- start start 2024-09-10T21:11:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b8e600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b880ce040 0x7f5b8814afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b8e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b880cfe30 0x7f5b88148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b8e600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880d0210 0x7f5b88149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b8e600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b880cf1e0 con 0x7f5b880cfe30 2024-09-10T21:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b8e600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b880b14f0 con 0x7f5b880d0210 2024-09-10T21:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b8e600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5b880cf360 con 0x7f5b880ce040 2024-09-10T21:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b7fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b880cfe30 0x7f5b88148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b7fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b880cfe30 0x7f5b88148b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60874/0 (socket says 172.21.15.16:60874) 2024-09-10T21:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b7fe00640 1 -- 172.21.15.16:0/3658519848 learned_addr learned my addr 172.21.15.16:0/3658519848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b87400640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b880ce040 0x7f5b8814afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b87e00640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880d0210 0x7f5b88149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b7fe00640 1 -- 172.21.15.16:0/3658519848 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b880ce040 msgr2=0x7f5b8814afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b7fe00640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b880ce040 0x7f5b8814afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b7fe00640 1 -- 172.21.15.16:0/3658519848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880d0210 msgr2=0x7f5b88149060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b7fe00640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880d0210 0x7f5b88149060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b7fe00640 1 -- 172.21.15.16:0/3658519848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8814b500 con 0x7f5b880cfe30 2024-09-10T21:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b87e00640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880d0210 0x7f5b88149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b7fe00640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b880cfe30 0x7f5b88148b20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5b7001bc60 tx=0x7f5b70004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b7f400640 1 -- 172.21.15.16:0/3658519848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700b5040 con 0x7f5b880cfe30 2024-09-10T21:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b7f400640 1 -- 172.21.15.16:0/3658519848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b700be070 con 0x7f5b880cfe30 2024-09-10T21:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b7f400640 1 -- 172.21.15.16:0/3658519848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700038f0 con 0x7f5b880cfe30 2024-09-10T21:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b8e600640 1 -- 172.21.15.16:0/3658519848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8815d000 con 0x7f5b880cfe30 2024-09-10T21:11:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.793+0000 7f5b8e600640 1 -- 172.21.15.16:0/3658519848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b880b6ab0 con 0x7f5b880cfe30 2024-09-10T21:11:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.797+0000 7f5b8e600640 1 -- 172.21.15.16:0/3658519848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b880b1710 con 0x7f5b880cfe30 2024-09-10T21:11:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.797+0000 7f5b7f400640 1 -- 172.21.15.16:0/3658519848 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5b700b8070 con 0x7f5b880cfe30 2024-09-10T21:11:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.797+0000 7f5b7f400640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b640725e0 0x7f5b64074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.797+0000 7f5b7f400640 1 -- 172.21.15.16:0/3658519848 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f5b70131360 con 0x7f5b880cfe30 2024-09-10T21:11:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.797+0000 7f5b87400640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b640725e0 0x7f5b64074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.797+0000 7f5b87400640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b640725e0 0x7f5b64074aa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5b78007a80 tx=0x7f5b78009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:02.801+0000 7f5b7f400640 1 -- 172.21.15.16:0/3658519848 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5b70018bb0 con 0x7f5b880cfe30 2024-09-10T21:11:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.009+0000 7f5b8e600640 1 -- 172.21.15.16:0/3658519848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5b880b2430 con 0x7f5b880cfe30 2024-09-10T21:11:03.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.621+0000 7f5b7f400640 1 -- 172.21.15.16:0/3658519848 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f5b700fca50 con 0x7f5b880cfe30 2024-09-10T21:11:03.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.625+0000 7f5b8e600640 1 -- 172.21.15.16:0/3658519848 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b640725e0 msgr2=0x7f5b64074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.625+0000 7f5b8e600640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b640725e0 0x7f5b64074aa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5b78007a80 tx=0x7f5b78009040 comp rx=0 tx=0).stop 2024-09-10T21:11:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.625+0000 7f5b8e600640 1 -- 172.21.15.16:0/3658519848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b880cfe30 msgr2=0x7f5b88148b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.625+0000 7f5b8e600640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b880cfe30 0x7f5b88148b20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5b7001bc60 tx=0x7f5b70004620 comp rx=0 tx=0).stop 2024-09-10T21:11:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.625+0000 7f5b8e600640 1 -- 172.21.15.16:0/3658519848 shutdown_connections 2024-09-10T21:11:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.625+0000 7f5b8e600640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b640725e0 0x7f5b64074aa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.625+0000 7f5b8e600640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b880d0210 0x7f5b88149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.625+0000 7f5b8e600640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b880cfe30 0x7f5b88148b20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.625+0000 7f5b8e600640 1 --2- 172.21.15.16:0/3658519848 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b880ce040 0x7f5b8814afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.625+0000 7f5b8e600640 1 -- 172.21.15.16:0/3658519848 >> 172.21.15.16:0/3658519848 conn(0x7f5b880d1df0 msgr2=0x7f5b880aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.625+0000 7f5b8e600640 1 -- 172.21.15.16:0/3658519848 shutdown_connections 2024-09-10T21:11:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.625+0000 7f5b8e600640 1 -- 172.21.15.16:0/3658519848 wait complete. 2024-09-10T21:11:03.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:11:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.801+0000 7f6bb9a00640 1 -- 172.21.15.16:0/2479091170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb415bcc0 msgr2=0x7f6bb415b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.801+0000 7f6bb9a00640 1 --2- 172.21.15.16:0/2479091170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb415bcc0 0x7f6bb415b150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009900 tx=0x7f6ba401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.801+0000 7f6bb9a00640 1 -- 172.21.15.16:0/2479091170 shutdown_connections 2024-09-10T21:11:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.801+0000 7f6bb9a00640 1 --2- 172.21.15.16:0/2479091170 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6bb40b3550 0x7f6bb40cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.801+0000 7f6bb9a00640 1 --2- 172.21.15.16:0/2479091170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb415bcc0 0x7f6bb415b150 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.801+0000 7f6bb9a00640 1 -- 172.21.15.16:0/2479091170 >> 172.21.15.16:0/2479091170 conn(0x7f6bb406d1e0 msgr2=0x7f6bb40b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.801+0000 7f6bb9a00640 1 -- 172.21.15.16:0/2479091170 shutdown_connections 2024-09-10T21:11:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.801+0000 7f6bb9a00640 1 -- 172.21.15.16:0/2479091170 wait complete. 2024-09-10T21:11:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb9a00640 1 Processor -- start 2024-09-10T21:11:03.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb9a00640 1 -- start start 2024-09-10T21:11:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb4072710 0x7f6bb4154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb9a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6bb40b3550 0x7f6bb41355b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb9a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bb415bcc0 0x7f6bb414f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb9a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4167ae0 con 0x7f6bb4072710 2024-09-10T21:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb9a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb40cec30 con 0x7f6bb415bcc0 2024-09-10T21:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb9a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb4167c60 con 0x7f6bb40b3550 2024-09-10T21:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb4072710 0x7f6bb4154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb4072710 0x7f6bb4154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60892/0 (socket says 172.21.15.16:60892) 2024-09-10T21:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb2a00640 1 -- 172.21.15.16:0/3608282314 learned_addr learned my addr 172.21.15.16:0/3608282314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6babe00640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6bb40b3550 0x7f6bb41355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb3400640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bb415bcc0 0x7f6bb414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb2a00640 1 -- 172.21.15.16:0/3608282314 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6bb40b3550 msgr2=0x7f6bb41355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb2a00640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6bb40b3550 0x7f6bb41355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb2a00640 1 -- 172.21.15.16:0/3608282314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bb415bcc0 msgr2=0x7f6bb414f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb2a00640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bb415bcc0 0x7f6bb414f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb2a00640 1 -- 172.21.15.16:0/3608282314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb4050ef0 con 0x7f6bb4072710 2024-09-10T21:11:03.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb2a00640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb4072710 0x7f6bb4154ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009850 tx=0x7f6ba40a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:03.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6baaa00640 1 -- 172.21.15.16:0/3608282314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40b2040 con 0x7f6bb4072710 2024-09-10T21:11:03.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6baaa00640 1 -- 172.21.15.16:0/3608282314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ba40044b0 con 0x7f6bb4072710 2024-09-10T21:11:03.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6baaa00640 1 -- 172.21.15.16:0/3608282314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4005260 con 0x7f6bb4072710 2024-09-10T21:11:03.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb9a00640 1 -- 172.21.15.16:0/3608282314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb414dd80 con 0x7f6bb4072710 2024-09-10T21:11:03.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.805+0000 7f6bb9a00640 1 -- 172.21.15.16:0/3608282314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb414e340 con 0x7f6bb4072710 2024-09-10T21:11:03.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.809+0000 7f6bb9a00640 1 -- 172.21.15.16:0/3608282314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb414e510 con 0x7f6bb4072710 2024-09-10T21:11:03.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.809+0000 7f6baaa00640 1 -- 172.21.15.16:0/3608282314 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6ba4005400 con 0x7f6bb4072710 2024-09-10T21:11:03.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.809+0000 7f6baaa00640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b900725e0 0x7f6b90074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.809+0000 7f6baaa00640 1 -- 172.21.15.16:0/3608282314 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f6ba412cee0 con 0x7f6bb4072710 2024-09-10T21:11:03.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.809+0000 7f6babe00640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b900725e0 0x7f6b90074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.809+0000 7f6babe00640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b900725e0 0x7f6b90074aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6bb41557c0 tx=0x7f6b9c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:03.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.813+0000 7f6baaa00640 1 -- 172.21.15.16:0/3608282314 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6ba40164d0 con 0x7f6bb4072710 2024-09-10T21:11:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:03.993+0000 7f6bb9a00640 1 -- 172.21.15.16:0/3608282314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6bb40afe50 con 0x7f6bb4072710 2024-09-10T21:11:04.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.621+0000 7f6baaa00640 1 -- 172.21.15.16:0/3608282314 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7f6ba40f84c0 con 0x7f6bb4072710 2024-09-10T21:11:04.629 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:11:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.625+0000 7f6bb9a00640 1 -- 172.21.15.16:0/3608282314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b900725e0 msgr2=0x7f6b90074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.625+0000 7f6bb9a00640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b900725e0 0x7f6b90074aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6bb41557c0 tx=0x7f6b9c009040 comp rx=0 tx=0).stop 2024-09-10T21:11:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.625+0000 7f6bb9a00640 1 -- 172.21.15.16:0/3608282314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb4072710 msgr2=0x7f6bb4154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.625+0000 7f6bb9a00640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb4072710 0x7f6bb4154ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009850 tx=0x7f6ba40a57f0 comp rx=0 tx=0).stop 2024-09-10T21:11:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.625+0000 7f6bb9a00640 1 -- 172.21.15.16:0/3608282314 shutdown_connections 2024-09-10T21:11:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.625+0000 7f6bb9a00640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b900725e0 0x7f6b90074aa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.625+0000 7f6bb9a00640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bb415bcc0 0x7f6bb414f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.625+0000 7f6bb9a00640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6bb40b3550 0x7f6bb41355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.625+0000 7f6bb9a00640 1 --2- 172.21.15.16:0/3608282314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb4072710 0x7f6bb4154ef0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.625+0000 7f6bb9a00640 1 -- 172.21.15.16:0/3608282314 >> 172.21.15.16:0/3608282314 conn(0x7f6bb406d1e0 msgr2=0x7f6bb415d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:04.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.625+0000 7f6bb9a00640 1 -- 172.21.15.16:0/3608282314 shutdown_connections 2024-09-10T21:11:04.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.625+0000 7f6bb9a00640 1 -- 172.21.15.16:0/3608282314 wait complete. 2024-09-10T21:11:04.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:11:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.801+0000 7f0f8ee00640 1 -- 172.21.15.16:0/237609006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f880d2960 msgr2=0x7f0f880ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.801+0000 7f0f8ee00640 1 --2- 172.21.15.16:0/237609006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f880d2960 0x7f0f880ced90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009900 tx=0x7f0f7001c850 comp rx=0 tx=0).stop 2024-09-10T21:11:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.801+0000 7f0f8ee00640 1 -- 172.21.15.16:0/237609006 shutdown_connections 2024-09-10T21:11:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.801+0000 7f0f8ee00640 1 --2- 172.21.15.16:0/237609006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f880d2960 0x7f0f880ced90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.801+0000 7f0f8ee00640 1 --2- 172.21.15.16:0/237609006 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0f8815bcf0 0x7f0f880ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.801+0000 7f0f8ee00640 1 -- 172.21.15.16:0/237609006 >> 172.21.15.16:0/237609006 conn(0x7f0f880d1e40 msgr2=0x7f0f880d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.801+0000 7f0f8ee00640 1 -- 172.21.15.16:0/237609006 shutdown_connections 2024-09-10T21:11:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f8ee00640 1 -- 172.21.15.16:0/237609006 wait complete. 2024-09-10T21:11:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f8ee00640 1 Processor -- start 2024-09-10T21:11:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f8ee00640 1 -- start start 2024-09-10T21:11:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f8ee00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0f880b1a60 0x7f0f8814d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f8ee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f880d2960 0x7f0f8806ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f8ee00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f8815bcf0 0x7f0f881542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f8ee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881691b0 con 0x7f0f880d2960 2024-09-10T21:11:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f8ee00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f88169030 con 0x7f0f8815bcf0 2024-09-10T21:11:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f8ee00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0f88169330 con 0x7f0f880b1a60 2024-09-10T21:11:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f7f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f880d2960 0x7f0f8806ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f7f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f880d2960 0x7f0f8806ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60906/0 (socket says 172.21.15.16:60906) 2024-09-10T21:11:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f7f400640 1 -- 172.21.15.16:0/2150575274 learned_addr learned my addr 172.21.15.16:0/2150575274 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f87e00640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0f880b1a60 0x7f0f8814d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f8ce00640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f8815bcf0 0x7f0f881542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f7f400640 1 -- 172.21.15.16:0/2150575274 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0f880b1a60 msgr2=0x7f0f8814d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f7f400640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0f880b1a60 0x7f0f8814d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f7f400640 1 -- 172.21.15.16:0/2150575274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f8815bcf0 msgr2=0x7f0f881542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f7f400640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f8815bcf0 0x7f0f881542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f7f400640 1 -- 172.21.15.16:0/2150575274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8806f650 con 0x7f0f880d2960 2024-09-10T21:11:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.805+0000 7f0f7f400640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f880d2960 0x7f0f8806ebf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0f700b0040 tx=0x7f0f700a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.809+0000 7f0f84c00640 1 -- 172.21.15.16:0/2150575274 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f700bb070 con 0x7f0f880d2960 2024-09-10T21:11:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.809+0000 7f0f84c00640 1 -- 172.21.15.16:0/2150575274 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f70005070 con 0x7f0f880d2960 2024-09-10T21:11:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.809+0000 7f0f84c00640 1 -- 172.21.15.16:0/2150575274 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f700b2040 con 0x7f0f880d2960 2024-09-10T21:11:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.809+0000 7f0f8ee00640 1 -- 172.21.15.16:0/2150575274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8806f910 con 0x7f0f880d2960 2024-09-10T21:11:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.809+0000 7f0f8ee00640 1 -- 172.21.15.16:0/2150575274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f88159ad0 con 0x7f0f880d2960 2024-09-10T21:11:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.809+0000 7f0f8ee00640 1 -- 172.21.15.16:0/2150575274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f880b1ef0 con 0x7f0f880d2960 2024-09-10T21:11:04.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.809+0000 7f0f84c00640 1 -- 172.21.15.16:0/2150575274 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0f700054a0 con 0x7f0f880d2960 2024-09-10T21:11:04.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.809+0000 7f0f84c00640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0f640725e0 0x7f0f64074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:04.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.809+0000 7f0f84c00640 1 -- 172.21.15.16:0/2150575274 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5357+0+0 (secure 0 0 0) 0x7f0f7012cac0 con 0x7f0f880d2960 2024-09-10T21:11:04.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.809+0000 7f0f87e00640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0f640725e0 0x7f0f64074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:04.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.813+0000 7f0f87e00640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0f640725e0 0x7f0f64074aa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f78004630 tx=0x7f0f7800b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.813+0000 7f0f84c00640 1 -- 172.21.15.16:0/2150575274 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0f700164d0 con 0x7f0f880d2960 2024-09-10T21:11:04.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:04.989+0000 7f0f8ee00640 1 -- 172.21.15.16:0/2150575274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0f881590d0 con 0x7f0f880d2960 2024-09-10T21:11:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.629+0000 7f0f84c00640 1 -- 172.21.15.16:0/2150575274 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7f0f700f7d30 con 0x7f0f880d2960 2024-09-10T21:11:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-10T21:11:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.633+0000 7f0f8ee00640 1 -- 172.21.15.16:0/2150575274 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0f640725e0 msgr2=0x7f0f64074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.633+0000 7f0f8ee00640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0f640725e0 0x7f0f64074aa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f78004630 tx=0x7f0f7800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.633+0000 7f0f8ee00640 1 -- 172.21.15.16:0/2150575274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f880d2960 msgr2=0x7f0f8806ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.633+0000 7f0f8ee00640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f880d2960 0x7f0f8806ebf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0f700b0040 tx=0x7f0f700a56b0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.633+0000 7f0f8ee00640 1 -- 172.21.15.16:0/2150575274 shutdown_connections 2024-09-10T21:11:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.633+0000 7f0f8ee00640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0f640725e0 0x7f0f64074aa0 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f78004630 tx=0x7f0f7800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.633+0000 7f0f8ee00640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f8815bcf0 0x7f0f881542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.633+0000 7f0f8ee00640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f880d2960 0x7f0f8806ebf0 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0f700b0040 tx=0x7f0f700a56b0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.633+0000 7f0f8ee00640 1 --2- 172.21.15.16:0/2150575274 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0f880b1a60 0x7f0f8814d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.633+0000 7f0f8ee00640 1 -- 172.21.15.16:0/2150575274 >> 172.21.15.16:0/2150575274 conn(0x7f0f880d1e40 msgr2=0x7f0f880aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:05.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.633+0000 7f0f8ee00640 1 -- 172.21.15.16:0/2150575274 shutdown_connections 2024-09-10T21:11:05.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.633+0000 7f0f8ee00640 1 -- 172.21.15.16:0/2150575274 wait complete. 2024-09-10T21:11:05.662 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:11:05.662 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:11:05.662 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:11:05.662 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:11:05.663 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:11:05.663 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:11:05.663 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:11:05.663 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:11:05.663 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:11:05.663 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:11:05.663 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:11:05.663 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:11:05.664 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:11:05.664 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:11:05.664 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:11:05.664 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:11:05.677 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:11:05.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:11:05.678 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:11:05.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:11:05.679 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:11:05.680 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:11:05.680 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:11:05.680 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:11:05.725 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:11:05.725 DEBUG:teuthology.parallel:result is None 2024-09-10T21:11:05.726 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:11:05.726 DEBUG:teuthology.parallel:result is None 2024-09-10T21:11:05.728 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:11:05.728 DEBUG:teuthology.parallel:result is None 2024-09-10T21:11:05.730 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:11:05.730 DEBUG:teuthology.parallel:result is None 2024-09-10T21:11:05.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:11:05.817 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-10T21:11:05.818 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-10T21:11:05.823 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:11:05.824 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:11:05.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 -- 172.21.15.16:0/3347955458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd4815f400 msgr2=0x7efd48070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 --2- 172.21.15.16:0/3347955458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd4815f400 0x7efd48070fe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efd480cc360 tx=0x7efd3801c6c0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 -- 172.21.15.16:0/3347955458 shutdown_connections 2024-09-10T21:11:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 --2- 172.21.15.16:0/3347955458 >> v2:172.21.15.52:3300/0 conn(0x7efd480b0590 0x7efd48071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 --2- 172.21.15.16:0/3347955458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd4815f400 0x7efd48070fe0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 -- 172.21.15.16:0/3347955458 >> 172.21.15.16:0/3347955458 conn(0x7efd4809b130 msgr2=0x7efd480d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 -- 172.21.15.16:0/3347955458 shutdown_connections 2024-09-10T21:11:05.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 -- 172.21.15.16:0/3347955458 wait complete. 2024-09-10T21:11:05.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 Processor -- start 2024-09-10T21:11:05.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 -- start start 2024-09-10T21:11:05.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd480b0590 0x7efd480cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd480b13c0 0x7efd48156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd4815f400 0x7efd480c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd48168f60 con 0x7efd480b0590 2024-09-10T21:11:05.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efd48168de0 con 0x7efd480b13c0 2024-09-10T21:11:05.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd4ce00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7efd481690e0 con 0x7efd4815f400 2024-09-10T21:11:05.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd480b0590 0x7efd480cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd480b0590 0x7efd480cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60952/0 (socket says 172.21.15.16:60952) 2024-09-10T21:11:05.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd45e00640 1 -- 172.21.15.16:0/2782047484 learned_addr learned my addr 172.21.15.16:0/2782047484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:05.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd3d400640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd480b13c0 0x7efd48156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd46800640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd4815f400 0x7efd480c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd45e00640 1 -- 172.21.15.16:0/2782047484 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd4815f400 msgr2=0x7efd480c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.885+0000 7efd45e00640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd4815f400 0x7efd480c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.889+0000 7efd45e00640 1 -- 172.21.15.16:0/2782047484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd480b13c0 msgr2=0x7efd48156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.889+0000 7efd45e00640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd480b13c0 0x7efd48156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.889+0000 7efd45e00640 1 -- 172.21.15.16:0/2782047484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd481577c0 con 0x7efd480b0590 2024-09-10T21:11:05.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.889+0000 7efd45e00640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd480b0590 0x7efd480cbac0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efd380192d0 tx=0x7efd38003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:05.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.889+0000 7efd3ea00640 1 -- 172.21.15.16:0/2782047484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38004030 con 0x7efd480b0590 2024-09-10T21:11:05.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.889+0000 7efd3ea00640 1 -- 172.21.15.16:0/2782047484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd380041d0 con 0x7efd480b0590 2024-09-10T21:11:05.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.889+0000 7efd3ea00640 1 -- 172.21.15.16:0/2782047484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38005250 con 0x7efd480b0590 2024-09-10T21:11:05.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.889+0000 7efd4ce00640 1 -- 172.21.15.16:0/2782047484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd48157b00 con 0x7efd480b0590 2024-09-10T21:11:05.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.889+0000 7efd4ce00640 1 -- 172.21.15.16:0/2782047484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd48159880 con 0x7efd480b0590 2024-09-10T21:11:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.889+0000 7efd4ce00640 1 -- 172.21.15.16:0/2782047484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd48157ce0 con 0x7efd480b0590 2024-09-10T21:11:05.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.889+0000 7efd3ea00640 1 -- 172.21.15.16:0/2782047484 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efd380053f0 con 0x7efd480b0590 2024-09-10T21:11:05.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.893+0000 7efd3ea00640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd24072610 0x7efd24074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.893+0000 7efd3ea00640 1 -- 172.21.15.16:0/2782047484 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7efd3812d680 con 0x7efd480b0590 2024-09-10T21:11:05.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.893+0000 7efd3d400640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd24072610 0x7efd24074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.893+0000 7efd3d400640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd24072610 0x7efd24074ad0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efd480c8a80 tx=0x7efd30009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:05.897+0000 7efd3ea00640 1 -- 172.21.15.16:0/2782047484 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efd48157b00 con 0x7efd480b0590 2024-09-10T21:11:06.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.081+0000 7efd4ce00640 1 -- 172.21.15.16:0/2782047484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7efd480cfc80 con 0x7efd480b0590 2024-09-10T21:11:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.641+0000 7efd3ea00640 1 -- 172.21.15.16:0/2782047484 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7efd380ac070 con 0x7efd480b0590 2024-09-10T21:11:06.647 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-10T21:11:06.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.641+0000 7efd4ce00640 1 -- 172.21.15.16:0/2782047484 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd24072610 msgr2=0x7efd24074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:06.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.641+0000 7efd4ce00640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd24072610 0x7efd24074ad0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efd480c8a80 tx=0x7efd30009040 comp rx=0 tx=0).stop 2024-09-10T21:11:06.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.641+0000 7efd4ce00640 1 -- 172.21.15.16:0/2782047484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd480b0590 msgr2=0x7efd480cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:06.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.641+0000 7efd4ce00640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd480b0590 0x7efd480cbac0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efd380192d0 tx=0x7efd38003df0 comp rx=0 tx=0).stop 2024-09-10T21:11:06.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.641+0000 7efd4ce00640 1 -- 172.21.15.16:0/2782047484 shutdown_connections 2024-09-10T21:11:06.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.641+0000 7efd4ce00640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd24072610 0x7efd24074ad0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:06.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.641+0000 7efd4ce00640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd4815f400 0x7efd480c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:06.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.641+0000 7efd4ce00640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd480b13c0 0x7efd48156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:06.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.641+0000 7efd4ce00640 1 --2- 172.21.15.16:0/2782047484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd480b0590 0x7efd480cbac0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:06.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.641+0000 7efd4ce00640 1 -- 172.21.15.16:0/2782047484 >> 172.21.15.16:0/2782047484 conn(0x7efd4809b130 msgr2=0x7efd48158440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:06.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.645+0000 7efd4ce00640 1 -- 172.21.15.16:0/2782047484 shutdown_connections 2024-09-10T21:11:06.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.645+0000 7efd4ce00640 1 -- 172.21.15.16:0/2782047484 wait complete. 2024-09-10T21:11:06.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:11:06.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 -- 172.21.15.16:0/1782039292 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f94a815f350 msgr2=0x7f94a80cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:06.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 --2- 172.21.15.16:0/1782039292 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f94a815f350 0x7f94a80cd130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f949c009900 tx=0x7f949c01c820 comp rx=0 tx=0).stop 2024-09-10T21:11:06.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 -- 172.21.15.16:0/1782039292 shutdown_connections 2024-09-10T21:11:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 --2- 172.21.15.16:0/1782039292 >> v2:172.21.15.52:3300/0 conn(0x7f94a80cfb60 0x7f94a80cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 --2- 172.21.15.16:0/1782039292 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f94a815f350 0x7f94a80cd130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 -- 172.21.15.16:0/1782039292 >> 172.21.15.16:0/1782039292 conn(0x7f94a80d1d80 msgr2=0x7f94a80d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 -- 172.21.15.16:0/1782039292 shutdown_connections 2024-09-10T21:11:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 -- 172.21.15.16:0/1782039292 wait complete. 2024-09-10T21:11:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 Processor -- start 2024-09-10T21:11:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 -- start start 2024-09-10T21:11:06.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94a80cfb60 0x7f94a81544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:06.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f94a80d0500 0x7f94a806e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a815f350 0x7f94a80c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8168fc0 con 0x7f94a815f350 2024-09-10T21:11:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8168e40 con 0x7f94a80cfb60 2024-09-10T21:11:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94afe00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f94a8169140 con 0x7f94a80d0500 2024-09-10T21:11:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94ade00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a815f350 0x7f94a80c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94ad400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94a80cfb60 0x7f94a81544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94ade00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a815f350 0x7f94a80c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60968/0 (socket says 172.21.15.16:60968) 2024-09-10T21:11:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94ade00640 1 -- 172.21.15.16:0/4047272784 learned_addr learned my addr 172.21.15.16:0/4047272784 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94aca00640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f94a80d0500 0x7f94a806e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94ade00640 1 -- 172.21.15.16:0/4047272784 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f94a80d0500 msgr2=0x7f94a806e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94ade00640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f94a80d0500 0x7f94a806e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94ade00640 1 -- 172.21.15.16:0/4047272784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94a80cfb60 msgr2=0x7f94a81544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94ade00640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94a80cfb60 0x7f94a81544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94ade00640 1 -- 172.21.15.16:0/4047272784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a81fb6c0 con 0x7f94a815f350 2024-09-10T21:11:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94aca00640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f94a80d0500 0x7f94a806e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.817+0000 7f94ad400640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94a80cfb60 0x7f94a81544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:06.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.821+0000 7f94ade00640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a815f350 0x7f94a80c92d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f94a000c7d0 tx=0x7f94a000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:06.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.821+0000 7f94a5600640 1 -- 172.21.15.16:0/4047272784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0003b80 con 0x7f94a815f350 2024-09-10T21:11:06.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.821+0000 7f94a5600640 1 -- 172.21.15.16:0/4047272784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94a0003d40 con 0x7f94a815f350 2024-09-10T21:11:06.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.821+0000 7f94a5600640 1 -- 172.21.15.16:0/4047272784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a000aa50 con 0x7f94a815f350 2024-09-10T21:11:06.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.821+0000 7f94afe00640 1 -- 172.21.15.16:0/4047272784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a81fb9a0 con 0x7f94a815f350 2024-09-10T21:11:06.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.821+0000 7f94afe00640 1 -- 172.21.15.16:0/4047272784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a81fbf60 con 0x7f94a815f350 2024-09-10T21:11:06.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.821+0000 7f94afe00640 1 -- 172.21.15.16:0/4047272784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a8159870 con 0x7f94a815f350 2024-09-10T21:11:06.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.821+0000 7f94a5600640 1 -- 172.21.15.16:0/4047272784 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f94a000abf0 con 0x7f94a815f350 2024-09-10T21:11:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.821+0000 7f94a5600640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9488072610 0x7f9488074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.821+0000 7f94a5600640 1 -- 172.21.15.16:0/4047272784 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f94a0092b00 con 0x7f94a815f350 2024-09-10T21:11:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.821+0000 7f94ad400640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9488072610 0x7f9488074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:06.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.825+0000 7f94ad400640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9488072610 0x7f9488074ad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f949c0af000 tx=0x7f949c01b060 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:06.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:06.825+0000 7f94a5600640 1 -- 172.21.15.16:0/4047272784 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f94a005dee0 con 0x7f94a815f350 2024-09-10T21:11:07.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.065+0000 7f94afe00640 1 -- 172.21.15.16:0/4047272784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f94a81fc130 con 0x7f94a815f350 2024-09-10T21:11:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.065+0000 7f94a5600640 1 -- 172.21.15.16:0/4047272784 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+6645 (secure 0 0 0) 0x7f94a81fc130 con 0x7f94a815f350 2024-09-10T21:11:07.073 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:07.073 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:11:07.073 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-10T21:11:07.073 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.074 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-10T21:11:07.074 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmhg+lJBAAWjuYW75BGo2dmyAUi7s9+A==", 2024-09-10T21:11:07.074 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.074 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:11:07.074 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:11:07.074 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:11:07.074 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.074 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.074 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.074 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-10T21:11:07.074 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmQr7+MBAAOzXhWIirBOoi/7wKnrFgmg==", 2024-09-10T21:11:07.075 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.075 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:11:07.075 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:11:07.075 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:11:07.075 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.075 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.075 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.075 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-10T21:11:07.075 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmxxoALBAAgkmw91/VP1Uv2WMZsJKAyA==", 2024-09-10T21:11:07.075 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.075 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:11:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:11:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:11:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-10T21:11:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmA4tzOBAAiS+8n+Gokz6yXapVf2ecYg==", 2024-09-10T21:11:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:11:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:11:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:11:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.077 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.077 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.077 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-10T21:11:07.077 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHteBmRoahLRAA32PsUkqtGkeI7tMFRwas5Q==", 2024-09-10T21:11:07.077 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.077 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:11:07.077 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:11:07.077 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.077 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-10T21:11:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCLteBmGmEMBhAAr1yQEf/NkiN4+ezAQ9VR2A==", 2024-09-10T21:11:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:11:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:11:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-10T21:11:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCOteBmmMuAHBAAt+G/KApsZymjTLRSmyRbdw==", 2024-09-10T21:11:07.078 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:11:07.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:11:07.079 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.079 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.079 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.079 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-10T21:11:07.079 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCRteBmoQfNMBAA0vnIqlYoI13KNAAfNmJg3w==", 2024-09-10T21:11:07.079 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:11:07.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:11:07.079 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.079 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.080 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.080 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-10T21:11:07.080 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCWteBmc/7lDxAADlSi4UpcFJVNdp6YTkpdSw==", 2024-09-10T21:11:07.080 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.080 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:11:07.080 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:11:07.080 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.080 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.080 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.080 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-10T21:11:07.080 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZteBmBZ1LJhAAPcbZa1LQu4kNbSsJ/8Z81A==", 2024-09-10T21:11:07.081 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.081 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:11:07.081 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:11:07.081 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.081 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.081 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.081 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-10T21:11:07.081 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCcteBm7/czOxAAqREnY84+LF1N9dRXOhzz6g==", 2024-09-10T21:11:07.081 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.081 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:11:07.081 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:11:07.081 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-10T21:11:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgteBmG2J8FRAAB+cMFCnf9wJg1R3RD5SUJw==", 2024-09-10T21:11:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:11:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:11:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-10T21:11:07.083 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA==", 2024-09-10T21:11:07.083 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:11:07.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:11:07.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:11:07.083 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:11:07.083 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.083 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.083 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-10T21:11:07.083 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmuEm4ChAAe08QvhwIcN8ZHa3AoVcYww==", 2024-09-10T21:11:07.083 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:11:07.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:11:07.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:11:07.084 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:11:07.084 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.084 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.084 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-10T21:11:07.084 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBm50o1BxAAIFbVxVqU4rU8eKYpuZjTHA==", 2024-09-10T21:11:07.084 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:11:07.085 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:11:07.085 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:11:07.085 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:11:07.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.085 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-10T21:11:07.085 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmxtu0EBAAmN7+07lDiwjsIC/pNYUlSw==", 2024-09-10T21:11:07.085 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.085 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:11:07.085 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:11:07.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:11:07.086 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:11:07.086 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.086 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.086 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-10T21:11:07.086 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmfqgYCBAAm00tBe45phGieCcp1q46cw==", 2024-09-10T21:11:07.086 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:11:07.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:11:07.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:11:07.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.087 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.087 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.087 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mds", 2024-09-10T21:11:07.087 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCrteBmSVSLKBAAApoOlj1O+vsa6BeC0htwyw==", 2024-09-10T21:11:07.087 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mds" 2024-09-10T21:11:07.087 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.088 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.088 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mgr", 2024-09-10T21:11:07.088 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCrteBmql6LKBAAUdwg07ojhUE+AqFRIXrMnw==", 2024-09-10T21:11:07.088 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-10T21:11:07.088 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.088 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.088 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-osd", 2024-09-10T21:11:07.088 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCrteBmr26LKBAAtDFQMO1Ysit//7/DLrgfow==", 2024-09-10T21:11:07.088 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-osd" 2024-09-10T21:11:07.088 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd", 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCrteBmbHeLKBAAzrQzFcZkE1PtwqfHz+LBAQ==", 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCrteBm+YCLKBAAev3uc/PIiqqRExbomCG2lg==", 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-10T21:11:07.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.090 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.090 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rgw", 2024-09-10T21:11:07.090 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCrteBmSoqLKBAA/y6kWwVns0s63F2R8wWMGA==", 2024-09-10T21:11:07.090 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.090 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-10T21:11:07.090 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.090 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.090 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-10T21:11:07.090 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmMVABIBAAoB72mPyXyUO6Y6k+fMqQQg==", 2024-09-10T21:11:07.090 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.090 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:11:07.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:11:07.091 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:11:07.091 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:07.091 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-10T21:11:07.091 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmeyYxHBAAvBkpTzjPQKAOlHzI1jQ46g==", 2024-09-10T21:11:07.091 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:11:07.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:11:07.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:11:07.091 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:11:07.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:07.092 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-10T21:11:07.092 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:11:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.069+0000 7f94afe00640 1 -- 172.21.15.16:0/4047272784 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9488072610 msgr2=0x7f9488074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.069+0000 7f94afe00640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9488072610 0x7f9488074ad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f949c0af000 tx=0x7f949c01b060 comp rx=0 tx=0).stop 2024-09-10T21:11:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.069+0000 7f94afe00640 1 -- 172.21.15.16:0/4047272784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a815f350 msgr2=0x7f94a80c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.069+0000 7f94afe00640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a815f350 0x7f94a80c92d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f94a000c7d0 tx=0x7f94a000cc90 comp rx=0 tx=0).stop 2024-09-10T21:11:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.069+0000 7f94afe00640 1 -- 172.21.15.16:0/4047272784 shutdown_connections 2024-09-10T21:11:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.069+0000 7f94afe00640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9488072610 0x7f9488074ad0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.069+0000 7f94afe00640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a815f350 0x7f94a80c92d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.069+0000 7f94afe00640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f94a80d0500 0x7f94a806e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.069+0000 7f94afe00640 1 --2- 172.21.15.16:0/4047272784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94a80cfb60 0x7f94a81544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.069+0000 7f94afe00640 1 -- 172.21.15.16:0/4047272784 >> 172.21.15.16:0/4047272784 conn(0x7f94a80d1d80 msgr2=0x7f94a8070960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:07.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.069+0000 7f94afe00640 1 -- 172.21.15.16:0/4047272784 shutdown_connections 2024-09-10T21:11:07.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.069+0000 7f94afe00640 1 -- 172.21.15.16:0/4047272784 wait complete. 2024-09-10T21:11:07.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-10T21:11:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 -- 172.21.15.16:0/3016659920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de40d03c0 msgr2=0x7f4de40ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 --2- 172.21.15.16:0/3016659920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de40d03c0 0x7f4de40ce2b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4de40b4930 tx=0x7f4dcc01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 -- 172.21.15.16:0/3016659920 shutdown_connections 2024-09-10T21:11:07.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 --2- 172.21.15.16:0/3016659920 >> v2:172.21.15.52:3300/0 conn(0x7f4de40b2860 0x7f4de4160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 --2- 172.21.15.16:0/3016659920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de40d03c0 0x7f4de40ce2b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 -- 172.21.15.16:0/3016659920 >> 172.21.15.16:0/3016659920 conn(0x7f4de40d1d80 msgr2=0x7f4de40d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:07.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 -- 172.21.15.16:0/3016659920 shutdown_connections 2024-09-10T21:11:07.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 -- 172.21.15.16:0/3016659920 wait complete. 2024-09-10T21:11:07.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 Processor -- start 2024-09-10T21:11:07.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 -- start start 2024-09-10T21:11:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4de40b2860 0x7f4de4151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de40cffe0 0x7f4de40afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4de40d03c0 0x7f4de415e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4de40cd930 con 0x7f4de40cffe0 2024-09-10T21:11:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4de40cd7b0 con 0x7f4de40b2860 2024-09-10T21:11:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4dea000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4de40cdab0 con 0x7f4de40d03c0 2024-09-10T21:11:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4ddbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de40cffe0 0x7f4de40afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4ddbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de40cffe0 0x7f4de40afde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60986/0 (socket says 172.21.15.16:60986) 2024-09-10T21:11:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4ddbe00640 1 -- 172.21.15.16:0/1058246917 learned_addr learned my addr 172.21.15.16:0/1058246917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4de3400640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4de40b2860 0x7f4de4151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4ddbe00640 1 -- 172.21.15.16:0/1058246917 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4de40d03c0 msgr2=0x7f4de415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4de3e00640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4de40d03c0 0x7f4de415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.241+0000 7f4ddbe00640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4de40d03c0 0x7f4de415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4ddbe00640 1 -- 172.21.15.16:0/1058246917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4de40b2860 msgr2=0x7f4de4151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4ddbe00640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4de40b2860 0x7f4de4151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4ddbe00640 1 -- 172.21.15.16:0/1058246917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de40b0b80 con 0x7f4de40cffe0 2024-09-10T21:11:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4de3400640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4de40b2860 0x7f4de4151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4de3e00640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4de40d03c0 0x7f4de415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4ddbe00640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de40cffe0 0x7f4de40afde0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0b0040 tx=0x7f4dcc005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4ddb400640 1 -- 172.21.15.16:0/1058246917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0b2040 con 0x7f4de40cffe0 2024-09-10T21:11:07.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4ddb400640 1 -- 172.21.15.16:0/1058246917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4dcc0a6d60 con 0x7f4de40cffe0 2024-09-10T21:11:07.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4ddb400640 1 -- 172.21.15.16:0/1058246917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc005290 con 0x7f4de40cffe0 2024-09-10T21:11:07.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4dea000640 1 -- 172.21.15.16:0/1058246917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de4152150 con 0x7f4de40cffe0 2024-09-10T21:11:07.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4dea000640 1 -- 172.21.15.16:0/1058246917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de40b08a0 con 0x7f4de40cffe0 2024-09-10T21:11:07.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4dea000640 1 -- 172.21.15.16:0/1058246917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de40ce2b0 con 0x7f4de40cffe0 2024-09-10T21:11:07.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4ddb400640 1 -- 172.21.15.16:0/1058246917 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4dcc005430 con 0x7f4de40cffe0 2024-09-10T21:11:07.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4ddb400640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4dc00725e0 0x7f4dc0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:07.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.245+0000 7f4ddb400640 1 -- 172.21.15.16:0/1058246917 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f4dcc12cd80 con 0x7f4de40cffe0 2024-09-10T21:11:07.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.249+0000 7f4de3400640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4dc00725e0 0x7f4dc0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:07.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.249+0000 7f4de3400640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4dc00725e0 0x7f4dc0074aa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4003ec0 tx=0x7f4dd4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:07.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.249+0000 7f4ddb400640 1 -- 172.21.15.16:0/1058246917 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4dcc0164b0 con 0x7f4de40cffe0 2024-09-10T21:11:07.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.489+0000 7f4dea000640 1 -- 172.21.15.16:0/1058246917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f4de40cf950 con 0x7f4de40cffe0 2024-09-10T21:11:07.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.489+0000 7f4ddb400640 1 -- 172.21.15.16:0/1058246917 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v16) ==== 90+0+0 (secure 0 0 0) 0x7f4dcc0f87a0 con 0x7f4de40cffe0 2024-09-10T21:11:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.493+0000 7f4dea000640 1 -- 172.21.15.16:0/1058246917 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4dc00725e0 msgr2=0x7f4dc0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.493+0000 7f4dea000640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4dc00725e0 0x7f4dc0074aa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4003ec0 tx=0x7f4dd4009040 comp rx=0 tx=0).stop 2024-09-10T21:11:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.493+0000 7f4dea000640 1 -- 172.21.15.16:0/1058246917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de40cffe0 msgr2=0x7f4de40afde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.493+0000 7f4dea000640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de40cffe0 0x7f4de40afde0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0b0040 tx=0x7f4dcc005760 comp rx=0 tx=0).stop 2024-09-10T21:11:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.493+0000 7f4dea000640 1 -- 172.21.15.16:0/1058246917 shutdown_connections 2024-09-10T21:11:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.493+0000 7f4dea000640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4dc00725e0 0x7f4dc0074aa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.493+0000 7f4dea000640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4de40d03c0 0x7f4de415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.493+0000 7f4dea000640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de40cffe0 0x7f4de40afde0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.493+0000 7f4dea000640 1 --2- 172.21.15.16:0/1058246917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4de40b2860 0x7f4de4151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.493+0000 7f4dea000640 1 -- 172.21.15.16:0/1058246917 >> 172.21.15.16:0/1058246917 conn(0x7f4de40d1d80 msgr2=0x7f4de4070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.493+0000 7f4dea000640 1 -- 172.21.15.16:0/1058246917 shutdown_connections 2024-09-10T21:11:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.493+0000 7f4dea000640 1 -- 172.21.15.16:0/1058246917 wait complete. 2024-09-10T21:11:07.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-10T21:11:07.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.661+0000 7fb3ef200640 1 -- 172.21.15.16:0/3562457989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3d4004da0 msgr2=0x7fb3d4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.661+0000 7fb3ef200640 1 --2- 172.21.15.16:0/3562457989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3d4004da0 0x7fb3d4005200 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009ee0 tx=0x7fb3e801d5f0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.661+0000 7fb3ee800640 1 -- 172.21.15.16:0/3562457989 reap_dead start 2024-09-10T21:11:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.661+0000 7fb3ef200640 1 -- 172.21.15.16:0/3562457989 shutdown_connections 2024-09-10T21:11:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.661+0000 7fb3ef200640 1 -- 172.21.15.16:0/3562457989 >> 172.21.15.16:0/3562457989 conn(0x7fb3f00d1c10 msgr2=0x7fb3f00d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.661+0000 7fb3ef200640 1 -- 172.21.15.16:0/3562457989 shutdown_connections 2024-09-10T21:11:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.661+0000 7fb3ef200640 1 -- 172.21.15.16:0/3562457989 wait complete. 2024-09-10T21:11:07.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ef200640 1 Processor -- start 2024-09-10T21:11:07.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ef200640 1 -- start start 2024-09-10T21:11:07.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ef200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3d4004da0 0x7fb3f00b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:07.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ef200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00c9a30 0x7fb3f0153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:07.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ef200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3f00b5f10 0x7fb3f00b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:07.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ef200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f00b1d90 con 0x7fb3f00c9a30 2024-09-10T21:11:07.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ef200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f00b1c10 con 0x7fb3f00b5f10 2024-09-10T21:11:07.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ef200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f00b1f10 con 0x7fb3d4004da0 2024-09-10T21:11:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ed400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00c9a30 0x7fb3f0153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ed400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00c9a30 0x7fb3f0153c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32774/0 (socket says 172.21.15.16:32774) 2024-09-10T21:11:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ed400640 1 -- 172.21.15.16:0/2829952169 learned_addr learned my addr 172.21.15.16:0/2829952169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ede00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3d4004da0 0x7fb3f00b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ee800640 1 --2- 172.21.15.16:0/2829952169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3f00b5f10 0x7fb3f00b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ed400640 1 -- 172.21.15.16:0/2829952169 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3d4004da0 msgr2=0x7fb3f00b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ed400640 1 --2- 172.21.15.16:0/2829952169 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3d4004da0 0x7fb3f00b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ed400640 1 -- 172.21.15.16:0/2829952169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3f00b5f10 msgr2=0x7fb3f00b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ed400640 1 --2- 172.21.15.16:0/2829952169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3f00b5f10 0x7fb3f00b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ed400640 1 -- 172.21.15.16:0/2829952169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f00b6bd0 con 0x7fb3f00c9a30 2024-09-10T21:11:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ed400640 1 --2- 172.21.15.16:0/2829952169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00c9a30 0x7fb3f0153c70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc001130 tx=0x7fb3dc00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3e6000640 1 -- 172.21.15.16:0/2829952169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc015020 con 0x7fb3f00c9a30 2024-09-10T21:11:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3e6000640 1 -- 172.21.15.16:0/2829952169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3dc010980 con 0x7fb3f00c9a30 2024-09-10T21:11:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3e6000640 1 -- 172.21.15.16:0/2829952169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc010dd0 con 0x7fb3f00c9a30 2024-09-10T21:11:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ef200640 1 -- 172.21.15.16:0/2829952169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f00b68f0 con 0x7fb3f00c9a30 2024-09-10T21:11:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ef200640 1 -- 172.21.15.16:0/2829952169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f00b4ec0 con 0x7fb3f00c9a30 2024-09-10T21:11:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.665+0000 7fb3ef200640 1 -- 172.21.15.16:0/2829952169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f00cd800 con 0x7fb3f00c9a30 2024-09-10T21:11:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.669+0000 7fb3e6000640 1 -- 172.21.15.16:0/2829952169 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb3dc011460 con 0x7fb3f00c9a30 2024-09-10T21:11:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.669+0000 7fb3e6000640 1 --2- 172.21.15.16:0/2829952169 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3d00725e0 0x7fb3d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.669+0000 7fb3e6000640 1 -- 172.21.15.16:0/2829952169 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fb3dc093860 con 0x7fb3f00c9a30 2024-09-10T21:11:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.669+0000 7fb3ede00640 1 --2- 172.21.15.16:0/2829952169 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3d00725e0 0x7fb3d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:07.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.669+0000 7fb3ede00640 1 --2- 172.21.15.16:0/2829952169 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3d00725e0 0x7fb3d0074aa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0000c10 tx=0x7fb3e0006b20 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:07.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.673+0000 7fb3e6000640 1 -- 172.21.15.16:0/2829952169 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb3dc01f020 con 0x7fb3f00c9a30 2024-09-10T21:11:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.913+0000 7fb3ef200640 1 -- 172.21.15.16:0/2829952169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fb3f01541b0 con 0x7fb3f00c9a30 2024-09-10T21:11:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.917+0000 7fb3e6000640 1 -- 172.21.15.16:0/2829952169 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v17) ==== 90+0+0 (secure 0 0 0) 0x7fb3f00b6bd0 con 0x7fb3f00c9a30 2024-09-10T21:11:07.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.917+0000 7fb3ef200640 1 -- 172.21.15.16:0/2829952169 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3d00725e0 msgr2=0x7fb3d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:07.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.917+0000 7fb3ef200640 1 --2- 172.21.15.16:0/2829952169 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3d00725e0 0x7fb3d0074aa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0000c10 tx=0x7fb3e0006b20 comp rx=0 tx=0).stop 2024-09-10T21:11:07.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.917+0000 7fb3ef200640 1 -- 172.21.15.16:0/2829952169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00c9a30 msgr2=0x7fb3f0153c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.917+0000 7fb3ef200640 1 --2- 172.21.15.16:0/2829952169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00c9a30 0x7fb3f0153c70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc001130 tx=0x7fb3dc00ea20 comp rx=0 tx=0).stop 2024-09-10T21:11:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.917+0000 7fb3ef200640 1 -- 172.21.15.16:0/2829952169 shutdown_connections 2024-09-10T21:11:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.917+0000 7fb3ef200640 1 --2- 172.21.15.16:0/2829952169 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3d00725e0 0x7fb3d0074aa0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.917+0000 7fb3ef200640 1 --2- 172.21.15.16:0/2829952169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3f00b5f10 0x7fb3f00b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.917+0000 7fb3ef200640 1 --2- 172.21.15.16:0/2829952169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f00c9a30 0x7fb3f0153c70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.917+0000 7fb3ef200640 1 --2- 172.21.15.16:0/2829952169 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3d4004da0 0x7fb3f00b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.917+0000 7fb3ef200640 1 -- 172.21.15.16:0/2829952169 >> 172.21.15.16:0/2829952169 conn(0x7fb3f00d1c10 msgr2=0x7fb3f0155e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.917+0000 7fb3ef200640 1 -- 172.21.15.16:0/2829952169 shutdown_connections 2024-09-10T21:11:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:07.917+0000 7fb3ef200640 1 -- 172.21.15.16:0/2829952169 wait complete. 2024-09-10T21:11:07.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-10T21:11:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.085+0000 7f918bc00640 1 -- 172.21.15.16:0/1348016079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f918415ad00 msgr2=0x7f91840cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.085+0000 7f918bc00640 1 --2- 172.21.15.16:0/1348016079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f918415ad00 0x7f91840cb280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9178009840 tx=0x7f917801c6a0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.085+0000 7f918bc00640 1 -- 172.21.15.16:0/1348016079 shutdown_connections 2024-09-10T21:11:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.085+0000 7f918bc00640 1 --2- 172.21.15.16:0/1348016079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f918415ad00 0x7f91840cb280 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.085+0000 7f918bc00640 1 -- 172.21.15.16:0/1348016079 >> 172.21.15.16:0/1348016079 conn(0x7f91840d0c30 msgr2=0x7f91840d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.085+0000 7f918bc00640 1 -- 172.21.15.16:0/1348016079 shutdown_connections 2024-09-10T21:11:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.085+0000 7f918bc00640 1 -- 172.21.15.16:0/1348016079 wait complete. 2024-09-10T21:11:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.085+0000 7f918bc00640 1 Processor -- start 2024-09-10T21:11:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.085+0000 7f918bc00640 1 -- start start 2024-09-10T21:11:08.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.085+0000 7f918bc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91840d07b0 0x7f91840c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:08.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.085+0000 7f918bc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f918415ad00 0x7f91840af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:08.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.085+0000 7f918bc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9184156580 0x7f91840c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:08.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.085+0000 7f918bc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9184169300 con 0x7f91840d07b0 2024-09-10T21:11:08.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f918bc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9184169180 con 0x7f9184156580 2024-09-10T21:11:08.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f918bc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9184169480 con 0x7f918415ad00 2024-09-10T21:11:08.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9189200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91840d07b0 0x7f91840c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9189200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91840d07b0 0x7f91840c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32776/0 (socket says 172.21.15.16:32776) 2024-09-10T21:11:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9189200640 1 -- 172.21.15.16:0/2401758292 learned_addr learned my addr 172.21.15.16:0/2401758292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9183e00640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f918415ad00 0x7f91840af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9189c00640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9184156580 0x7f91840c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9189200640 1 -- 172.21.15.16:0/2401758292 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f918415ad00 msgr2=0x7f91840af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9189200640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f918415ad00 0x7f91840af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9189200640 1 -- 172.21.15.16:0/2401758292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9184156580 msgr2=0x7f91840c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9189200640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9184156580 0x7f91840c5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9189200640 1 -- 172.21.15.16:0/2401758292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91841f8c00 con 0x7f91840d07b0 2024-09-10T21:11:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9189200640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91840d07b0 0x7f91840c7360 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9178000c10 tx=0x7f91780a6580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:08.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9180c00640 1 -- 172.21.15.16:0/2401758292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178005030 con 0x7f91840d07b0 2024-09-10T21:11:08.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9180c00640 1 -- 172.21.15.16:0/2401758292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91780051d0 con 0x7f91840d07b0 2024-09-10T21:11:08.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f918bc00640 1 -- 172.21.15.16:0/2401758292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91841f8ee0 con 0x7f91840d07b0 2024-09-10T21:11:08.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9180c00640 1 -- 172.21.15.16:0/2401758292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91780055c0 con 0x7f91840d07b0 2024-09-10T21:11:08.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f918bc00640 1 -- 172.21.15.16:0/2401758292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91840c4e60 con 0x7f91840d07b0 2024-09-10T21:11:08.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f918bc00640 1 -- 172.21.15.16:0/2401758292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91840737c0 con 0x7f91840d07b0 2024-09-10T21:11:08.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.089+0000 7f9180c00640 1 -- 172.21.15.16:0/2401758292 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f91780a6040 con 0x7f91840d07b0 2024-09-10T21:11:08.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.093+0000 7f9180c00640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91640725e0 0x7f9164074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:08.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.093+0000 7f9180c00640 1 -- 172.21.15.16:0/2401758292 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f917812c810 con 0x7f91840d07b0 2024-09-10T21:11:08.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.093+0000 7f9183e00640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91640725e0 0x7f9164074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:08.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.093+0000 7f9183e00640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91640725e0 0x7f9164074aa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9184070bb0 tx=0x7f9170009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:08.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.097+0000 7f9180c00640 1 -- 172.21.15.16:0/2401758292 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f91780bf070 con 0x7f91840d07b0 2024-09-10T21:11:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.329+0000 7f918bc00640 1 -- 172.21.15.16:0/2401758292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f918415e640 con 0x7f91840d07b0 2024-09-10T21:11:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.333+0000 7f9180c00640 1 -- 172.21.15.16:0/2401758292 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v18) ==== 90+0+0 (secure 0 0 0) 0x7f91780ac030 con 0x7f91840d07b0 2024-09-10T21:11:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.337+0000 7f918bc00640 1 -- 172.21.15.16:0/2401758292 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91640725e0 msgr2=0x7f9164074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.337+0000 7f918bc00640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91640725e0 0x7f9164074aa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9184070bb0 tx=0x7f9170009040 comp rx=0 tx=0).stop 2024-09-10T21:11:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.337+0000 7f918bc00640 1 -- 172.21.15.16:0/2401758292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91840d07b0 msgr2=0x7f91840c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.337+0000 7f918bc00640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91840d07b0 0x7f91840c7360 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9178000c10 tx=0x7f91780a6580 comp rx=0 tx=0).stop 2024-09-10T21:11:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.337+0000 7f918bc00640 1 -- 172.21.15.16:0/2401758292 shutdown_connections 2024-09-10T21:11:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.337+0000 7f918bc00640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91640725e0 0x7f9164074aa0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.337+0000 7f918bc00640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9184156580 0x7f91840c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.337+0000 7f918bc00640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f918415ad00 0x7f91840af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.337+0000 7f918bc00640 1 --2- 172.21.15.16:0/2401758292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91840d07b0 0x7f91840c7360 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.337+0000 7f918bc00640 1 -- 172.21.15.16:0/2401758292 >> 172.21.15.16:0/2401758292 conn(0x7f91840d0c30 msgr2=0x7f91840b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.337+0000 7f918bc00640 1 -- 172.21.15.16:0/2401758292 shutdown_connections 2024-09-10T21:11:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.337+0000 7f918bc00640 1 -- 172.21.15.16:0/2401758292 wait complete. 2024-09-10T21:11:08.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-10T21:11:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 -- 172.21.15.16:0/3130343155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c440040c0 msgr2=0x7f1c44004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 --2- 172.21.15.16:0/3130343155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c440040c0 0x7f1c44004810 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800c2e0 tx=0x7f1c4801f700 comp rx=0 tx=0).stop 2024-09-10T21:11:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c61800640 1 -- 172.21.15.16:0/3130343155 reap_dead start 2024-09-10T21:11:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 -- 172.21.15.16:0/3130343155 shutdown_connections 2024-09-10T21:11:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 -- 172.21.15.16:0/3130343155 >> 172.21.15.16:0/3130343155 conn(0x7f1c5c0d1d80 msgr2=0x7f1c5c0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 -- 172.21.15.16:0/3130343155 shutdown_connections 2024-09-10T21:11:08.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 -- 172.21.15.16:0/3130343155 wait complete. 2024-09-10T21:11:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 Processor -- start 2024-09-10T21:11:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 -- start start 2024-09-10T21:11:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c440040c0 0x7f1c5c158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c1f6b90 0x7f1c5c1f5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1c5c1f5f80 0x7f1c5c1f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c0ce2b0 con 0x7f1c5c1f6b90 2024-09-10T21:11:08.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c0cdbc0 con 0x7f1c440040c0 2024-09-10T21:11:08.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c63800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5c0ce430 con 0x7f1c5c1f5f80 2024-09-10T21:11:08.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c5be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c1f6b90 0x7f1c5c1f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:08.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c5be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c1f6b90 0x7f1c5c1f5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32790/0 (socket says 172.21.15.16:32790) 2024-09-10T21:11:08.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.505+0000 7f1c5be00640 1 -- 172.21.15.16:0/3562114060 learned_addr learned my addr 172.21.15.16:0/3562114060 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:08.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c61800640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1c5c1f5f80 0x7f1c5c1f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:08.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c60e00640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c440040c0 0x7f1c5c158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:08.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c5be00640 1 -- 172.21.15.16:0/3562114060 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1c5c1f5f80 msgr2=0x7f1c5c1f63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:08.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c5be00640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1c5c1f5f80 0x7f1c5c1f63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c5be00640 1 -- 172.21.15.16:0/3562114060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c440040c0 msgr2=0x7f1c5c158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:08.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c5be00640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c440040c0 0x7f1c5c158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c5be00640 1 -- 172.21.15.16:0/3562114060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c0cbbc0 con 0x7f1c5c1f6b90 2024-09-10T21:11:08.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c60e00640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c440040c0 0x7f1c5c158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:08.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c5be00640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c1f6b90 0x7f1c5c1f5680 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1c480b1040 tx=0x7f1c48006cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:08.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c58c00640 1 -- 172.21.15.16:0/3562114060 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c480bf070 con 0x7f1c5c1f6b90 2024-09-10T21:11:08.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c58c00640 1 -- 172.21.15.16:0/3562114060 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c48003600 con 0x7f1c5c1f6b90 2024-09-10T21:11:08.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c63800640 1 -- 172.21.15.16:0/3562114060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5c1e7020 con 0x7f1c5c1f6b90 2024-09-10T21:11:08.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c58c00640 1 -- 172.21.15.16:0/3562114060 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c480b3070 con 0x7f1c5c1f6b90 2024-09-10T21:11:08.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c63800640 1 -- 172.21.15.16:0/3562114060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5c0cb8c0 con 0x7f1c5c1f6b90 2024-09-10T21:11:08.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c63800640 1 -- 172.21.15.16:0/3562114060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c5c15fe00 con 0x7f1c5c1f6b90 2024-09-10T21:11:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c58c00640 1 -- 172.21.15.16:0/3562114060 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1c480b9070 con 0x7f1c5c1f6b90 2024-09-10T21:11:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c58c00640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1c3c0725e0 0x7f1c3c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c58c00640 1 -- 172.21.15.16:0/3562114060 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f1c48130660 con 0x7f1c5c1f6b90 2024-09-10T21:11:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.509+0000 7f1c60e00640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1c3c0725e0 0x7f1c3c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.513+0000 7f1c60e00640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1c3c0725e0 0x7f1c3c074aa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1c50005fd0 tx=0x7f1c5000b260 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.513+0000 7f1c58c00640 1 -- 172.21.15.16:0/3562114060 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1c480194b0 con 0x7f1c5c1f6b90 2024-09-10T21:11:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.765+0000 7f1c63800640 1 -- 172.21.15.16:0/3562114060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f1c5c0cf950 con 0x7f1c5c1f6b90 2024-09-10T21:11:08.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.769+0000 7f1c58c00640 1 -- 172.21.15.16:0/3562114060 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v19) ==== 90+0+0 (secure 0 0 0) 0x7f1c480fb9a0 con 0x7f1c5c1f6b90 2024-09-10T21:11:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.769+0000 7f1c63800640 1 -- 172.21.15.16:0/3562114060 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1c3c0725e0 msgr2=0x7f1c3c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.769+0000 7f1c63800640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1c3c0725e0 0x7f1c3c074aa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1c50005fd0 tx=0x7f1c5000b260 comp rx=0 tx=0).stop 2024-09-10T21:11:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.769+0000 7f1c63800640 1 -- 172.21.15.16:0/3562114060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c1f6b90 msgr2=0x7f1c5c1f5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.769+0000 7f1c63800640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c1f6b90 0x7f1c5c1f5680 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1c480b1040 tx=0x7f1c48006cf0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.769+0000 7f1c63800640 1 -- 172.21.15.16:0/3562114060 shutdown_connections 2024-09-10T21:11:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.769+0000 7f1c63800640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1c3c0725e0 0x7f1c3c074aa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.769+0000 7f1c63800640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1c5c1f5f80 0x7f1c5c1f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.769+0000 7f1c63800640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c1f6b90 0x7f1c5c1f5680 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.769+0000 7f1c63800640 1 --2- 172.21.15.16:0/3562114060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c440040c0 0x7f1c5c158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.769+0000 7f1c63800640 1 -- 172.21.15.16:0/3562114060 >> 172.21.15.16:0/3562114060 conn(0x7f1c5c0d1d80 msgr2=0x7f1c5c15e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.769+0000 7f1c63800640 1 -- 172.21.15.16:0/3562114060 shutdown_connections 2024-09-10T21:11:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.769+0000 7f1c63800640 1 -- 172.21.15.16:0/3562114060 wait complete. 2024-09-10T21:11:08.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-10T21:11:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 -- 172.21.15.16:0/1182960535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180ce1d0 msgr2=0x7fc3180ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 --2- 172.21.15.16:0/1182960535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180ce1d0 0x7fc3180ce5b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc308009930 tx=0x7fc30801c850 comp rx=0 tx=0).stop 2024-09-10T21:11:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 -- 172.21.15.16:0/1182960535 shutdown_connections 2024-09-10T21:11:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 --2- 172.21.15.16:0/1182960535 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc31815f850 0x7fc3180cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 --2- 172.21.15.16:0/1182960535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180ce1d0 0x7fc3180ce5b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 -- 172.21.15.16:0/1182960535 >> 172.21.15.16:0/1182960535 conn(0x7fc3180d03a0 msgr2=0x7fc3180d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 -- 172.21.15.16:0/1182960535 shutdown_connections 2024-09-10T21:11:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 -- 172.21.15.16:0/1182960535 wait complete. 2024-09-10T21:11:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 Processor -- start 2024-09-10T21:11:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 -- start start 2024-09-10T21:11:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180ce1d0 0x7fc3180b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3180d0e90 0x7fc3181faf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc31815f850 0x7fc3181fb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.937+0000 7fc31f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3180cc470 con 0x7fc3180ce1d0 2024-09-10T21:11:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc31f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3180cc2f0 con 0x7fc3180d0e90 2024-09-10T21:11:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc31f200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc318167f10 con 0x7fc31815f850 2024-09-10T21:11:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc317e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180ce1d0 0x7fc3180b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc317e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180ce1d0 0x7fc3180b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32806/0 (socket says 172.21.15.16:32806) 2024-09-10T21:11:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc317e00640 1 -- 172.21.15.16:0/3991321807 learned_addr learned my addr 172.21.15.16:0/3991321807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc31d200640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc31815f850 0x7fc3181fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc30fe00640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3180d0e90 0x7fc3181faf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc317e00640 1 -- 172.21.15.16:0/3991321807 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc31815f850 msgr2=0x7fc3181fb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc317e00640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc31815f850 0x7fc3181fb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc317e00640 1 -- 172.21.15.16:0/3991321807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3180d0e90 msgr2=0x7fc3181faf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc317e00640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3180d0e90 0x7fc3181faf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc317e00640 1 -- 172.21.15.16:0/3991321807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3180ca440 con 0x7fc3180ce1d0 2024-09-10T21:11:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc30fe00640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3180d0e90 0x7fc3181faf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc317e00640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180ce1d0 0x7fc3180b3cb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc308009850 tx=0x7fc3080a55e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc314c00640 1 -- 172.21.15.16:0/3991321807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3080b2040 con 0x7fc3180ce1d0 2024-09-10T21:11:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc314c00640 1 -- 172.21.15.16:0/3991321807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc308004470 con 0x7fc3180ce1d0 2024-09-10T21:11:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc31f200640 1 -- 172.21.15.16:0/3991321807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31815b6c0 con 0x7fc3180ce1d0 2024-09-10T21:11:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc314c00640 1 -- 172.21.15.16:0/3991321807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308005290 con 0x7fc3180ce1d0 2024-09-10T21:11:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc31f200640 1 -- 172.21.15.16:0/3991321807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3181fbd10 con 0x7fc3180ce1d0 2024-09-10T21:11:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc31f200640 1 -- 172.21.15.16:0/3991321807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3180ce5b0 con 0x7fc3180ce1d0 2024-09-10T21:11:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.941+0000 7fc314c00640 1 -- 172.21.15.16:0/3991321807 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc308005430 con 0x7fc3180ce1d0 2024-09-10T21:11:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.945+0000 7fc314c00640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc2f40725e0 0x7fc2f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.945+0000 7fc314c00640 1 -- 172.21.15.16:0/3991321807 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fc30812c9e0 con 0x7fc3180ce1d0 2024-09-10T21:11:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.945+0000 7fc30fe00640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc2f40725e0 0x7fc2f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.945+0000 7fc30fe00640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc2f40725e0 0x7fc2f4074aa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc31809e420 tx=0x7fc300009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:08.949+0000 7fc314c00640 1 -- 172.21.15.16:0/3991321807 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc3080164d0 con 0x7fc3180ce1d0 2024-09-10T21:11:09.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.185+0000 7fc31f200640 1 -- 172.21.15.16:0/3991321807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fc318071790 con 0x7fc3180ce1d0 2024-09-10T21:11:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.185+0000 7fc314c00640 1 -- 172.21.15.16:0/3991321807 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v20) ==== 97+0+0 (secure 0 0 0) 0x7fc3080f8380 con 0x7fc3180ce1d0 2024-09-10T21:11:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.189+0000 7fc31f200640 1 -- 172.21.15.16:0/3991321807 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc2f40725e0 msgr2=0x7fc2f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.189+0000 7fc31f200640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc2f40725e0 0x7fc2f4074aa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc31809e420 tx=0x7fc300009040 comp rx=0 tx=0).stop 2024-09-10T21:11:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.189+0000 7fc31f200640 1 -- 172.21.15.16:0/3991321807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180ce1d0 msgr2=0x7fc3180b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.189+0000 7fc31f200640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180ce1d0 0x7fc3180b3cb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc308009850 tx=0x7fc3080a55e0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.189+0000 7fc31f200640 1 -- 172.21.15.16:0/3991321807 shutdown_connections 2024-09-10T21:11:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.189+0000 7fc31f200640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc2f40725e0 0x7fc2f4074aa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.189+0000 7fc31f200640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc31815f850 0x7fc3181fb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.189+0000 7fc31f200640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3180d0e90 0x7fc3181faf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.189+0000 7fc31f200640 1 --2- 172.21.15.16:0/3991321807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180ce1d0 0x7fc3180b3cb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.189+0000 7fc31f200640 1 -- 172.21.15.16:0/3991321807 >> 172.21.15.16:0/3991321807 conn(0x7fc3180d03a0 msgr2=0x7fc3181656f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.189+0000 7fc31f200640 1 -- 172.21.15.16:0/3991321807 shutdown_connections 2024-09-10T21:11:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.189+0000 7fc31f200640 1 -- 172.21.15.16:0/3991321807 wait complete. 2024-09-10T21:11:09.213 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-10T21:11:09.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.353+0000 7f3b4a000640 1 -- 172.21.15.16:0/2974749419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b440d0fc0 msgr2=0x7f3b440cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:09.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.353+0000 7f3b4a000640 1 --2- 172.21.15.16:0/2974749419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b440d0fc0 0x7f3b440cee80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009c80 tx=0x7f3b3c01cce0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.353+0000 7f3b4a000640 1 -- 172.21.15.16:0/2974749419 shutdown_connections 2024-09-10T21:11:09.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.353+0000 7f3b4a000640 1 --2- 172.21.15.16:0/2974749419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b440d0fc0 0x7f3b440cee80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.353+0000 7f3b4a000640 1 --2- 172.21.15.16:0/2974749419 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3b440d15a0 0x7f3b440ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.353+0000 7f3b4a000640 1 -- 172.21.15.16:0/2974749419 >> 172.21.15.16:0/2974749419 conn(0x7f3b440d3ea0 msgr2=0x7f3b440d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:09.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.353+0000 7f3b4a000640 1 -- 172.21.15.16:0/2974749419 shutdown_connections 2024-09-10T21:11:09.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.353+0000 7f3b4a000640 1 -- 172.21.15.16:0/2974749419 wait complete. 2024-09-10T21:11:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b4a000640 1 Processor -- start 2024-09-10T21:11:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b4a000640 1 -- start start 2024-09-10T21:11:09.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b4a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b440d0fc0 0x7f3b440c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:09.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b4a000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b440d15a0 0x7f3b440ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:09.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b4a000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3b4415dd20 0x7f3b4415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:09.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b4a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4416b490 con 0x7f3b440d0fc0 2024-09-10T21:11:09.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b4a000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4416b310 con 0x7f3b440d15a0 2024-09-10T21:11:09.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b4a000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4416b610 con 0x7f3b4415dd20 2024-09-10T21:11:09.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b48c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b440d0fc0 0x7f3b440c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:09.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b48c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b440d0fc0 0x7f3b440c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32830/0 (socket says 172.21.15.16:32830) 2024-09-10T21:11:09.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b48c00640 1 -- 172.21.15.16:0/4088262649 learned_addr learned my addr 172.21.15.16:0/4088262649 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:09.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b49600640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3b4415dd20 0x7f3b4415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:09.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b43e00640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b440d15a0 0x7f3b440ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:09.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b48c00640 1 -- 172.21.15.16:0/4088262649 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3b4415dd20 msgr2=0x7f3b4415b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:09.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b48c00640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3b4415dd20 0x7f3b4415b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b48c00640 1 -- 172.21.15.16:0/4088262649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b440d15a0 msgr2=0x7f3b440ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:09.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b48c00640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b440d15a0 0x7f3b440ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b48c00640 1 -- 172.21.15.16:0/4088262649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b440cb110 con 0x7f3b440d0fc0 2024-09-10T21:11:09.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b43e00640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b440d15a0 0x7f3b440ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:09.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b48c00640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b440d0fc0 0x7f3b440c8be0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800c490 tx=0x7f3b38001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:09.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b40c00640 1 -- 172.21.15.16:0/4088262649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38017070 con 0x7f3b440d0fc0 2024-09-10T21:11:09.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b40c00640 1 -- 172.21.15.16:0/4088262649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b38003600 con 0x7f3b440d0fc0 2024-09-10T21:11:09.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b40c00640 1 -- 172.21.15.16:0/4088262649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38003a90 con 0x7f3b440d0fc0 2024-09-10T21:11:09.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b4a000640 1 -- 172.21.15.16:0/4088262649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b440c9270 con 0x7f3b440d0fc0 2024-09-10T21:11:09.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b4a000640 1 -- 172.21.15.16:0/4088262649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b440cb690 con 0x7f3b440d0fc0 2024-09-10T21:11:09.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.357+0000 7f3b4a000640 1 -- 172.21.15.16:0/4088262649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b440b4a90 con 0x7f3b440d0fc0 2024-09-10T21:11:09.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.361+0000 7f3b40c00640 1 -- 172.21.15.16:0/4088262649 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3b38003c30 con 0x7f3b440d0fc0 2024-09-10T21:11:09.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.361+0000 7f3b40c00640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3b240725e0 0x7f3b24074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:09.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.361+0000 7f3b40c00640 1 -- 172.21.15.16:0/4088262649 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f3b380926b0 con 0x7f3b440d0fc0 2024-09-10T21:11:09.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.361+0000 7f3b43e00640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3b240725e0 0x7f3b24074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:09.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.361+0000 7f3b43e00640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3b240725e0 0x7f3b24074aa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3b440c7a40 tx=0x7f3b30008040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:09.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.365+0000 7f3b40c00640 1 -- 172.21.15.16:0/4088262649 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3b3805e040 con 0x7f3b440d0fc0 2024-09-10T21:11:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.601+0000 7f3b4a000640 1 -- 172.21.15.16:0/4088262649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f3b440d0830 con 0x7f3b440d0fc0 2024-09-10T21:11:09.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.605+0000 7f3b40c00640 1 -- 172.21.15.16:0/4088262649 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v21) ==== 90+0+0 (secure 0 0 0) 0x7f3b3805d940 con 0x7f3b440d0fc0 2024-09-10T21:11:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.605+0000 7f3b4a000640 1 -- 172.21.15.16:0/4088262649 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3b240725e0 msgr2=0x7f3b24074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.605+0000 7f3b4a000640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3b240725e0 0x7f3b24074aa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3b440c7a40 tx=0x7f3b30008040 comp rx=0 tx=0).stop 2024-09-10T21:11:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.605+0000 7f3b4a000640 1 -- 172.21.15.16:0/4088262649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b440d0fc0 msgr2=0x7f3b440c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.605+0000 7f3b4a000640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b440d0fc0 0x7f3b440c8be0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800c490 tx=0x7f3b38001e80 comp rx=0 tx=0).stop 2024-09-10T21:11:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.605+0000 7f3b4a000640 1 -- 172.21.15.16:0/4088262649 shutdown_connections 2024-09-10T21:11:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.605+0000 7f3b4a000640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3b240725e0 0x7f3b24074aa0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.605+0000 7f3b4a000640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3b4415dd20 0x7f3b4415b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.605+0000 7f3b4a000640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b440d15a0 0x7f3b440ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.605+0000 7f3b4a000640 1 --2- 172.21.15.16:0/4088262649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b440d0fc0 0x7f3b440c8be0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.605+0000 7f3b4a000640 1 -- 172.21.15.16:0/4088262649 >> 172.21.15.16:0/4088262649 conn(0x7f3b440d3ea0 msgr2=0x7f3b440d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.609+0000 7f3b4a000640 1 -- 172.21.15.16:0/4088262649 shutdown_connections 2024-09-10T21:11:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.609+0000 7f3b4a000640 1 -- 172.21.15.16:0/4088262649 wait complete. 2024-09-10T21:11:09.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:11:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.773+0000 7f1467a00640 1 -- 172.21.15.16:0/3792777019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14600705e0 msgr2=0x7f14600ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.773+0000 7f1467a00640 1 --2- 172.21.15.16:0/3792777019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14600705e0 0x7f14600ca170 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f146006f4f0 tx=0x7f145801dbb0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.773+0000 7f1467a00640 1 -- 172.21.15.16:0/3792777019 shutdown_connections 2024-09-10T21:11:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.773+0000 7f1467a00640 1 --2- 172.21.15.16:0/3792777019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14600705e0 0x7f14600ca170 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.773+0000 7f1467a00640 1 --2- 172.21.15.16:0/3792777019 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14600d06b0 0x7f14600700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.773+0000 7f1467a00640 1 --2- 172.21.15.16:0/3792777019 >> v2:172.21.15.52:3300/0 conn(0x7f146015a270 0x7f14600ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.773+0000 7f1467a00640 1 -- 172.21.15.16:0/3792777019 >> 172.21.15.16:0/3792777019 conn(0x7f14600cfb20 msgr2=0x7f14600cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.773+0000 7f1467a00640 1 -- 172.21.15.16:0/3792777019 shutdown_connections 2024-09-10T21:11:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.773+0000 7f1467a00640 1 -- 172.21.15.16:0/3792777019 wait complete. 2024-09-10T21:11:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1467a00640 1 Processor -- start 2024-09-10T21:11:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1467a00640 1 -- start start 2024-09-10T21:11:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1467a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14600705e0 0x7f14600c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1467a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14600d06b0 0x7f1460145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1467a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f146015a270 0x7f1460151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1467a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1460072f00 con 0x7f14600705e0 2024-09-10T21:11:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1467a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1460072d80 con 0x7f14600d06b0 2024-09-10T21:11:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1467a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f1460146610 con 0x7f146015a270 2024-09-10T21:11:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1465000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14600705e0 0x7f14600c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1465000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14600705e0 0x7f14600c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32846/0 (socket says 172.21.15.16:32846) 2024-09-10T21:11:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1465000640 1 -- 172.21.15.16:0/405008318 learned_addr learned my addr 172.21.15.16:0/405008318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f145fe00640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14600d06b0 0x7f1460145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1465000640 1 -- 172.21.15.16:0/405008318 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f146015a270 msgr2=0x7f1460151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1465a00640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f146015a270 0x7f1460151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1465000640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f146015a270 0x7f1460151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1465000640 1 -- 172.21.15.16:0/405008318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14600d06b0 msgr2=0x7f1460145c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1465000640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14600d06b0 0x7f1460145c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1465000640 1 -- 172.21.15.16:0/405008318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1460157910 con 0x7f14600705e0 2024-09-10T21:11:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f145fe00640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14600d06b0 0x7f1460145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1465000640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14600705e0 0x7f14600c96b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f145400e7a0 tx=0x7f145400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f145cc00640 1 -- 172.21.15.16:0/405008318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145400ccc0 con 0x7f14600705e0 2024-09-10T21:11:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f145cc00640 1 -- 172.21.15.16:0/405008318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1454010430 con 0x7f14600705e0 2024-09-10T21:11:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1467a00640 1 -- 172.21.15.16:0/405008318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1460157bf0 con 0x7f14600705e0 2024-09-10T21:11:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f145cc00640 1 -- 172.21.15.16:0/405008318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1454010860 con 0x7f14600705e0 2024-09-10T21:11:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1467a00640 1 -- 172.21.15.16:0/405008318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14601581b0 con 0x7f14600705e0 2024-09-10T21:11:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.777+0000 7f1467a00640 1 -- 172.21.15.16:0/405008318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14600700a0 con 0x7f14600705e0 2024-09-10T21:11:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.781+0000 7f145cc00640 1 -- 172.21.15.16:0/405008318 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1454010a00 con 0x7f14600705e0 2024-09-10T21:11:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.781+0000 7f145cc00640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14400725e0 0x7f1440074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.781+0000 7f145cc00640 1 -- 172.21.15.16:0/405008318 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f14540939c0 con 0x7f14600705e0 2024-09-10T21:11:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.781+0000 7f145fe00640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14400725e0 0x7f1440074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.781+0000 7f145fe00640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14400725e0 0x7f1440074aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f14600ca2b0 tx=0x7f144c005f90 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.785+0000 7f145cc00640 1 -- 172.21.15.16:0/405008318 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f145405f3e0 con 0x7f14600705e0 2024-09-10T21:11:09.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.965+0000 7f1467a00640 1 -- 172.21.15.16:0/405008318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f14600b5aa0 con 0x7f14600705e0 2024-09-10T21:11:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.965+0000 7f145cc00640 1 -- 172.21.15.16:0/405008318 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) ==== 84+0+65 (secure 0 0 0) 0x7f145405ece0 con 0x7f14600705e0 2024-09-10T21:11:09.973 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:09.973 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-10T21:11:09.973 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:11:09.973 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-10T21:11:09.974 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-10T21:11:09.974 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:11:09.974 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-10T21:11:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.969+0000 7f1467a00640 1 -- 172.21.15.16:0/405008318 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14400725e0 msgr2=0x7f1440074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.969+0000 7f1467a00640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14400725e0 0x7f1440074aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f14600ca2b0 tx=0x7f144c005f90 comp rx=0 tx=0).stop 2024-09-10T21:11:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.969+0000 7f1467a00640 1 -- 172.21.15.16:0/405008318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14600705e0 msgr2=0x7f14600c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.969+0000 7f1467a00640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14600705e0 0x7f14600c96b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f145400e7a0 tx=0x7f145400ec60 comp rx=0 tx=0).stop 2024-09-10T21:11:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.969+0000 7f1467a00640 1 -- 172.21.15.16:0/405008318 shutdown_connections 2024-09-10T21:11:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.969+0000 7f1467a00640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14400725e0 0x7f1440074aa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.969+0000 7f1467a00640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f146015a270 0x7f1460151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.969+0000 7f1467a00640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14600d06b0 0x7f1460145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.969+0000 7f1467a00640 1 --2- 172.21.15.16:0/405008318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14600705e0 0x7f14600c96b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.969+0000 7f1467a00640 1 -- 172.21.15.16:0/405008318 >> 172.21.15.16:0/405008318 conn(0x7f14600cfb20 msgr2=0x7f1460156d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.969+0000 7f1467a00640 1 -- 172.21.15.16:0/405008318 shutdown_connections 2024-09-10T21:11:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:09.969+0000 7f1467a00640 1 -- 172.21.15.16:0/405008318 wait complete. 2024-09-10T21:11:09.994 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:11:09.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:11:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.137+0000 7f1513600640 1 -- 172.21.15.16:0/638411104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f150c0b3550 msgr2=0x7f150c15b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.137+0000 7f1513600640 1 --2- 172.21.15.16:0/638411104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f150c0b3550 0x7f150c15b150 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f150c0b4240 tx=0x7f14f801c830 comp rx=0 tx=0).stop 2024-09-10T21:11:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.137+0000 7f1513600640 1 -- 172.21.15.16:0/638411104 shutdown_connections 2024-09-10T21:11:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.137+0000 7f1513600640 1 --2- 172.21.15.16:0/638411104 >> v2:172.21.15.52:3300/0 conn(0x7f150c072710 0x7f150c0cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.137+0000 7f1513600640 1 --2- 172.21.15.16:0/638411104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f150c0b3550 0x7f150c15b150 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.137+0000 7f1513600640 1 -- 172.21.15.16:0/638411104 >> 172.21.15.16:0/638411104 conn(0x7f150c06d1e0 msgr2=0x7f150c0b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.137+0000 7f1513600640 1 -- 172.21.15.16:0/638411104 shutdown_connections 2024-09-10T21:11:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.137+0000 7f1513600640 1 -- 172.21.15.16:0/638411104 wait complete. 2024-09-10T21:11:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1513600640 1 Processor -- start 2024-09-10T21:11:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1513600640 1 -- start start 2024-09-10T21:11:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1513600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f150c072710 0x7f150c13a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1513600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f150c0b3550 0x7f150c164f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1513600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f150c15bcc0 0x7f150c1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1513600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f150c167ae0 con 0x7f150c0b3550 2024-09-10T21:11:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1513600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f150c0cec30 con 0x7f150c072710 2024-09-10T21:11:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1513600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f150c167c60 con 0x7f150c15bcc0 2024-09-10T21:11:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f150be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f150c0b3550 0x7f150c164f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f150be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f150c0b3550 0x7f150c164f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32872/0 (socket says 172.21.15.16:32872) 2024-09-10T21:11:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f150be00640 1 -- 172.21.15.16:0/3604692658 learned_addr learned my addr 172.21.15.16:0/3604692658 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1511600640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f150c15bcc0 0x7f150c1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1510c00640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f150c072710 0x7f150c13a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f150be00640 1 -- 172.21.15.16:0/3604692658 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f150c15bcc0 msgr2=0x7f150c1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f150be00640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f150c15bcc0 0x7f150c1549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f150be00640 1 -- 172.21.15.16:0/3604692658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f150c072710 msgr2=0x7f150c13a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f150be00640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f150c072710 0x7f150c13a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f150be00640 1 -- 172.21.15.16:0/3604692658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c13a630 con 0x7f150c0b3550 2024-09-10T21:11:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f150be00640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f150c0b3550 0x7f150c164f10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f14f80b0040 tx=0x7f14f80a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1508c00640 1 -- 172.21.15.16:0/3604692658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f80bb070 con 0x7f150c0b3550 2024-09-10T21:11:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1508c00640 1 -- 172.21.15.16:0/3604692658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14f8005030 con 0x7f150c0b3550 2024-09-10T21:11:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1508c00640 1 -- 172.21.15.16:0/3604692658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f80b2040 con 0x7f150c0b3550 2024-09-10T21:11:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1513600640 1 -- 172.21.15.16:0/3604692658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c153940 con 0x7f150c0b3550 2024-09-10T21:11:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.141+0000 7f1513600640 1 -- 172.21.15.16:0/3604692658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c153f00 con 0x7f150c0b3550 2024-09-10T21:11:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.145+0000 7f1513600640 1 -- 172.21.15.16:0/3604692658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150c0cda80 con 0x7f150c0b3550 2024-09-10T21:11:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.145+0000 7f1508c00640 1 -- 172.21.15.16:0/3604692658 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f14f8005480 con 0x7f150c0b3550 2024-09-10T21:11:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.145+0000 7f1508c00640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14ec0725e0 0x7f14ec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.145+0000 7f1508c00640 1 -- 172.21.15.16:0/3604692658 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f14f812c780 con 0x7f150c0b3550 2024-09-10T21:11:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.145+0000 7f1510c00640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14ec0725e0 0x7f14ec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.145+0000 7f1510c00640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14ec0725e0 0x7f14ec074aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1500001650 tx=0x7f1500009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:10.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.149+0000 7f1508c00640 1 -- 172.21.15.16:0/3604692658 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f14f80164b0 con 0x7f150c0b3550 2024-09-10T21:11:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:10.325+0000 7f1513600640 1 -- 172.21.15.16:0/3604692658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f150c15ea40 con 0x7f150c0b3550 2024-09-10T21:11:11.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.189+0000 7f1508c00640 1 -- 172.21.15.16:0/3604692658 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) ==== 138+0+0 (secure 0 0 0) 0x7f14f80f8110 con 0x7f150c0b3550 2024-09-10T21:11:11.197 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-10T21:11:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.193+0000 7f1513600640 1 -- 172.21.15.16:0/3604692658 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14ec0725e0 msgr2=0x7f14ec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.193+0000 7f1513600640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14ec0725e0 0x7f14ec074aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1500001650 tx=0x7f1500009040 comp rx=0 tx=0).stop 2024-09-10T21:11:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.193+0000 7f1513600640 1 -- 172.21.15.16:0/3604692658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f150c0b3550 msgr2=0x7f150c164f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.193+0000 7f1513600640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f150c0b3550 0x7f150c164f10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f14f80b0040 tx=0x7f14f80a56d0 comp rx=0 tx=0).stop 2024-09-10T21:11:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.193+0000 7f1513600640 1 -- 172.21.15.16:0/3604692658 shutdown_connections 2024-09-10T21:11:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.193+0000 7f1513600640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14ec0725e0 0x7f14ec074aa0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.193+0000 7f1513600640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f150c15bcc0 0x7f150c1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.193+0000 7f1513600640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f150c0b3550 0x7f150c164f10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:11.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.193+0000 7f1513600640 1 --2- 172.21.15.16:0/3604692658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f150c072710 0x7f150c13a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:11.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.193+0000 7f1513600640 1 -- 172.21.15.16:0/3604692658 >> 172.21.15.16:0/3604692658 conn(0x7f150c06d1e0 msgr2=0x7f150c15ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:11.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.197+0000 7f1513600640 1 -- 172.21.15.16:0/3604692658 shutdown_connections 2024-09-10T21:11:11.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.197+0000 7f1513600640 1 -- 172.21.15.16:0/3604692658 wait complete. 2024-09-10T21:11:11.230 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:11:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.381+0000 7f170ac00640 1 -- 172.21.15.16:0/562536453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17040cd300 msgr2=0x7f17040d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.381+0000 7f170ac00640 1 --2- 172.21.15.16:0/562536453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17040cd300 0x7f17040d2800 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1704073000 tx=0x7f16f001c7a0 comp rx=0 tx=0).stop 2024-09-10T21:11:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.381+0000 7f170ac00640 1 -- 172.21.15.16:0/562536453 shutdown_connections 2024-09-10T21:11:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.381+0000 7f170ac00640 1 --2- 172.21.15.16:0/562536453 >> v2:172.21.15.52:3300/0 conn(0x7f170415f940 0x7f170415fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.381+0000 7f170ac00640 1 --2- 172.21.15.16:0/562536453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17040cd300 0x7f17040d2800 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.381+0000 7f170ac00640 1 -- 172.21.15.16:0/562536453 >> 172.21.15.16:0/562536453 conn(0x7f17040d1ce0 msgr2=0x7f17040d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.381+0000 7f170ac00640 1 -- 172.21.15.16:0/562536453 shutdown_connections 2024-09-10T21:11:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.381+0000 7f170ac00640 1 -- 172.21.15.16:0/562536453 wait complete. 2024-09-10T21:11:11.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.381+0000 7f170ac00640 1 Processor -- start 2024-09-10T21:11:11.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.381+0000 7f170ac00640 1 -- start start 2024-09-10T21:11:11.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f170ac00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17040cd300 0x7f17040c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:11.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f170ac00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f170415bdc0 0x7f17041f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:11.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f170ac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170415f940 0x7f17041f7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:11.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f170ac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1704169010 con 0x7f170415f940 2024-09-10T21:11:11.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f170ac00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1704168e90 con 0x7f17040cd300 2024-09-10T21:11:11.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f170ac00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f1704169190 con 0x7f170415bdc0 2024-09-10T21:11:11.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f1708c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170415f940 0x7f17041f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:11.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f1708c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170415f940 0x7f17041f7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32888/0 (socket says 172.21.15.16:32888) 2024-09-10T21:11:11.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f1708c00640 1 -- 172.21.15.16:0/32358865 learned_addr learned my addr 172.21.15.16:0/32358865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:11.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f1703e00640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17040cd300 0x7f17040c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:11.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f1703400640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f170415bdc0 0x7f17041f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:11.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f1708c00640 1 -- 172.21.15.16:0/32358865 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f170415bdc0 msgr2=0x7f17041f8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:11.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f1708c00640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f170415bdc0 0x7f17041f8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:11.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f1708c00640 1 -- 172.21.15.16:0/32358865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17040cd300 msgr2=0x7f17040c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:11.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f1708c00640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17040cd300 0x7f17040c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:11.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f1708c00640 1 -- 172.21.15.16:0/32358865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17041f9060 con 0x7f170415f940 2024-09-10T21:11:11.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f1708c00640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170415f940 0x7f17041f7980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00b9a0 tx=0x7f16fc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:11.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f16fbe00640 1 -- 172.21.15.16:0/32358865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc019070 con 0x7f170415f940 2024-09-10T21:11:11.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f16fbe00640 1 -- 172.21.15.16:0/32358865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16fc00ccb0 con 0x7f170415f940 2024-09-10T21:11:11.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f16fbe00640 1 -- 172.21.15.16:0/32358865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0146a0 con 0x7f170415f940 2024-09-10T21:11:11.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f170ac00640 1 -- 172.21.15.16:0/32358865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17040c6cc0 con 0x7f170415f940 2024-09-10T21:11:11.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f170ac00640 1 -- 172.21.15.16:0/32358865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17040c7280 con 0x7f170415f940 2024-09-10T21:11:11.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.385+0000 7f170ac00640 1 -- 172.21.15.16:0/32358865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17040d2800 con 0x7f170415f940 2024-09-10T21:11:11.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.389+0000 7f16fbe00640 1 -- 172.21.15.16:0/32358865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f16fc01e440 con 0x7f170415f940 2024-09-10T21:11:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.389+0000 7f16fbe00640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f16e40725e0 0x7f16e4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.389+0000 7f16fbe00640 1 -- 172.21.15.16:0/32358865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5227+0+0 (secure 0 0 0) 0x7f16fc093480 con 0x7f170415f940 2024-09-10T21:11:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.389+0000 7f1703e00640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f16e40725e0 0x7f16e4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:11.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.389+0000 7f1703e00640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f16e40725e0 0x7f16e4074aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f16f4007fd0 tx=0x7f16f4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.393+0000 7f16fbe00640 1 -- 172.21.15.16:0/32358865 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f16fc023020 con 0x7f170415f940 2024-09-10T21:11:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:11.565+0000 7f170ac00640 1 -- 172.21.15.16:0/32358865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f17040704a0 con 0x7f170415f940 2024-09-10T21:11:12.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.197+0000 7f16fbe00640 1 -- 172.21.15.16:0/32358865 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) ==== 170+0+0 (secure 0 0 0) 0x7f17041f9060 con 0x7f170415f940 2024-09-10T21:11:12.204 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-10T21:11:12.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.201+0000 7f170ac00640 1 -- 172.21.15.16:0/32358865 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f16e40725e0 msgr2=0x7f16e4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:12.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.201+0000 7f170ac00640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f16e40725e0 0x7f16e4074aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f16f4007fd0 tx=0x7f16f4009040 comp rx=0 tx=0).stop 2024-09-10T21:11:12.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.201+0000 7f170ac00640 1 -- 172.21.15.16:0/32358865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170415f940 msgr2=0x7f17041f7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:12.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.201+0000 7f170ac00640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170415f940 0x7f17041f7980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00b9a0 tx=0x7f16fc00be60 comp rx=0 tx=0).stop 2024-09-10T21:11:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.205+0000 7f170ac00640 1 -- 172.21.15.16:0/32358865 shutdown_connections 2024-09-10T21:11:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.205+0000 7f170ac00640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f16e40725e0 0x7f16e4074aa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.205+0000 7f170ac00640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170415f940 0x7f17041f7980 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.205+0000 7f170ac00640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f170415bdc0 0x7f17041f8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.205+0000 7f170ac00640 1 --2- 172.21.15.16:0/32358865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17040cd300 0x7f17040c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.205+0000 7f170ac00640 1 -- 172.21.15.16:0/32358865 >> 172.21.15.16:0/32358865 conn(0x7f17040d1ce0 msgr2=0x7f17040d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.205+0000 7f170ac00640 1 -- 172.21.15.16:0/32358865 shutdown_connections 2024-09-10T21:11:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.205+0000 7f170ac00640 1 -- 172.21.15.16:0/32358865 wait complete. 2024-09-10T21:11:12.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:11:12.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 -- 172.21.15.16:0/2312470370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea00ce8c0 msgr2=0x7f9ea00ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:12.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 --2- 172.21.15.16:0/2312470370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea00ce8c0 0x7f9ea00ceca0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009900 tx=0x7f9e9401c870 comp rx=0 tx=0).stop 2024-09-10T21:11:12.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 -- 172.21.15.16:0/2312470370 shutdown_connections 2024-09-10T21:11:12.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 --2- 172.21.15.16:0/2312470370 >> v2:172.21.15.52:3300/0 conn(0x7f9ea00b0980 0x7f9ea00ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:12.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 --2- 172.21.15.16:0/2312470370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea00ce8c0 0x7f9ea00ceca0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:12.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 -- 172.21.15.16:0/2312470370 >> 172.21.15.16:0/2312470370 conn(0x7f9ea00d1c70 msgr2=0x7f9ea00d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:12.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 -- 172.21.15.16:0/2312470370 shutdown_connections 2024-09-10T21:11:12.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 -- 172.21.15.16:0/2312470370 wait complete. 2024-09-10T21:11:12.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 Processor -- start 2024-09-10T21:11:12.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 -- start start 2024-09-10T21:11:12.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea00b0980 0x7f9ea01544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:12.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9ea00ce8c0 0x7f9ea00cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:12.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ea00cf1e0 0x7f9ea0154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:12.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.397+0000 7f9ea6400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea0169310 con 0x7f9ea00b0980 2024-09-10T21:11:12.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea6400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea0169190 con 0x7f9ea00cf1e0 2024-09-10T21:11:12.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea6400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea0169490 con 0x7f9ea00ce8c0 2024-09-10T21:11:12.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea00b0980 0x7f9ea01544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea00b0980 0x7f9ea01544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46840/0 (socket says 172.21.15.16:46840) 2024-09-10T21:11:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea5000640 1 -- 172.21.15.16:0/3080059521 learned_addr learned my addr 172.21.15.16:0/3080059521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea5000640 1 -- 172.21.15.16:0/3080059521 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9ea00ce8c0 msgr2=0x7f9ea00cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea5000640 1 --2- 172.21.15.16:0/3080059521 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9ea00ce8c0 0x7f9ea00cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea5000640 1 -- 172.21.15.16:0/3080059521 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ea00cf1e0 msgr2=0x7f9ea0154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea5000640 1 --2- 172.21.15.16:0/3080059521 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ea00cf1e0 0x7f9ea0154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea5000640 1 -- 172.21.15.16:0/3080059521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea0153720 con 0x7f9ea00b0980 2024-09-10T21:11:12.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea5000640 1 --2- 172.21.15.16:0/3080059521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea00b0980 0x7f9ea01544e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9e940b0040 tx=0x7f9e940a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9e9cc00640 1 -- 172.21.15.16:0/3080059521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e940b2040 con 0x7f9ea00b0980 2024-09-10T21:11:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9e9cc00640 1 -- 172.21.15.16:0/3080059521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e940052f0 con 0x7f9ea00b0980 2024-09-10T21:11:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea6400640 1 -- 172.21.15.16:0/3080059521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea0153a00 con 0x7f9ea00b0980 2024-09-10T21:11:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9e9cc00640 1 -- 172.21.15.16:0/3080059521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e940046d0 con 0x7f9ea00b0980 2024-09-10T21:11:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea6400640 1 -- 172.21.15.16:0/3080059521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea00cb980 con 0x7f9ea00b0980 2024-09-10T21:11:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9ea6400640 1 -- 172.21.15.16:0/3080059521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea00ceca0 con 0x7f9ea00b0980 2024-09-10T21:11:12.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.401+0000 7f9e9cc00640 1 -- 172.21.15.16:0/3080059521 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9e94004870 con 0x7f9ea00b0980 2024-09-10T21:11:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.405+0000 7f9e9cc00640 1 --2- 172.21.15.16:0/3080059521 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e800725e0 0x7f9e80074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.405+0000 7f9e9cc00640 1 -- 172.21.15.16:0/3080059521 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5609+0+0 (secure 0 0 0) 0x7f9e9412dbb0 con 0x7f9ea00b0980 2024-09-10T21:11:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.405+0000 7f9e9fe00640 1 --2- 172.21.15.16:0/3080059521 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e800725e0 0x7f9e80074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.405+0000 7f9e9fe00640 1 --2- 172.21.15.16:0/3080059521 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e800725e0 0x7f9e80074aa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0058d0 tx=0x7f9e8c005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.409+0000 7f9e9cc00640 1 -- 172.21.15.16:0/3080059521 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9e940c4020 con 0x7f9ea00b0980 2024-09-10T21:11:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:12.617+0000 7f9ea6400640 1 -- 172.21.15.16:0/3080059521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9ea0050ef0 con 0x7f9ea00b0980 2024-09-10T21:11:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.201+0000 7f9e9cc00640 1 -- 172.21.15.16:0/3080059521 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f9e94016490 con 0x7f9ea00b0980 2024-09-10T21:11:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 4 and data pool 5 2024-09-10T21:11:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.205+0000 7f9ea6400640 1 -- 172.21.15.16:0/3080059521 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e800725e0 msgr2=0x7f9e80074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.205+0000 7f9ea6400640 1 --2- 172.21.15.16:0/3080059521 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e800725e0 0x7f9e80074aa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0058d0 tx=0x7f9e8c005860 comp rx=0 tx=0).stop 2024-09-10T21:11:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.205+0000 7f9ea6400640 1 -- 172.21.15.16:0/3080059521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea00b0980 msgr2=0x7f9ea01544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.205+0000 7f9ea6400640 1 --2- 172.21.15.16:0/3080059521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea00b0980 0x7f9ea01544e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9e940b0040 tx=0x7f9e940a5750 comp rx=0 tx=0).stop 2024-09-10T21:11:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.205+0000 7f9ea6400640 1 -- 172.21.15.16:0/3080059521 shutdown_connections 2024-09-10T21:11:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.205+0000 7f9ea6400640 1 --2- 172.21.15.16:0/3080059521 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e800725e0 0x7f9e80074aa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.205+0000 7f9ea6400640 1 --2- 172.21.15.16:0/3080059521 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ea00cf1e0 0x7f9ea0154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.205+0000 7f9ea6400640 1 --2- 172.21.15.16:0/3080059521 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9ea00ce8c0 0x7f9ea00cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.205+0000 7f9ea6400640 1 --2- 172.21.15.16:0/3080059521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ea00b0980 0x7f9ea01544e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.205+0000 7f9ea6400640 1 -- 172.21.15.16:0/3080059521 >> 172.21.15.16:0/3080059521 conn(0x7f9ea00d1c70 msgr2=0x7f9ea00cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.205+0000 7f9ea6400640 1 -- 172.21.15.16:0/3080059521 shutdown_connections 2024-09-10T21:11:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.205+0000 7f9ea6400640 1 -- 172.21.15.16:0/3080059521 wait complete. 2024-09-10T21:11:13.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:11:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.405+0000 7efeda000640 1 -- 172.21.15.16:0/495933889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efed40d0890 msgr2=0x7efed40b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.405+0000 7efeda000640 1 --2- 172.21.15.16:0/495933889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efed40d0890 0x7efed40b0020 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efed40b3810 tx=0x7efecc01cd60 comp rx=0 tx=0).stop 2024-09-10T21:11:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.405+0000 7efeda000640 1 -- 172.21.15.16:0/495933889 shutdown_connections 2024-09-10T21:11:13.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.405+0000 7efeda000640 1 --2- 172.21.15.16:0/495933889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efed40d0890 0x7efed40b0020 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.405+0000 7efeda000640 1 --2- 172.21.15.16:0/495933889 >> v2:172.21.15.52:3300/0 conn(0x7efed40b3200 0x7efed40afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.405+0000 7efeda000640 1 -- 172.21.15.16:0/495933889 >> 172.21.15.16:0/495933889 conn(0x7efed40d1d80 msgr2=0x7efed40d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:13.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efeda000640 1 -- 172.21.15.16:0/495933889 shutdown_connections 2024-09-10T21:11:13.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efeda000640 1 -- 172.21.15.16:0/495933889 wait complete. 2024-09-10T21:11:13.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efeda000640 1 Processor -- start 2024-09-10T21:11:13.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efeda000640 1 -- start start 2024-09-10T21:11:13.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efeda000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efed40d0890 0x7efed40ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:13.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efeda000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efed415f450 0x7efed41529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efeda000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed414f350 0x7efed4152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efeda000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efed4168d50 con 0x7efed40d0890 2024-09-10T21:11:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efeda000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efed4168bd0 con 0x7efed414f350 2024-09-10T21:11:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efeda000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7efed4168ed0 con 0x7efed415f450 2024-09-10T21:11:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efed3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efed40d0890 0x7efed40ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efed3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efed40d0890 0x7efed40ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46864/0 (socket says 172.21.15.16:46864) 2024-09-10T21:11:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efed3400640 1 -- 172.21.15.16:0/1018058013 learned_addr learned my addr 172.21.15.16:0/1018058013 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efecbe00640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efed415f450 0x7efed41529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efed3e00640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed414f350 0x7efed4152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efed3400640 1 -- 172.21.15.16:0/1018058013 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efed415f450 msgr2=0x7efed41529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efed3400640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efed415f450 0x7efed41529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efed3400640 1 -- 172.21.15.16:0/1018058013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed414f350 msgr2=0x7efed4152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efed3400640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed414f350 0x7efed4152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efed3400640 1 -- 172.21.15.16:0/1018058013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed40cebc0 con 0x7efed40d0890 2024-09-10T21:11:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efecbe00640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efed415f450 0x7efed41529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efed3e00640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed414f350 0x7efed4152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efed3400640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efed40d0890 0x7efed40ce680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efec400c3f0 tx=0x7efec400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efecb400640 1 -- 172.21.15.16:0/1018058013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec40052d0 con 0x7efed40d0890 2024-09-10T21:11:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efecb400640 1 -- 172.21.15.16:0/1018058013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efec4007020 con 0x7efed40d0890 2024-09-10T21:11:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efecb400640 1 -- 172.21.15.16:0/1018058013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4007470 con 0x7efed40d0890 2024-09-10T21:11:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.409+0000 7efeda000640 1 -- 172.21.15.16:0/1018058013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed40ccbe0 con 0x7efed40d0890 2024-09-10T21:11:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.413+0000 7efeda000640 1 -- 172.21.15.16:0/1018058013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed40cd1a0 con 0x7efed40d0890 2024-09-10T21:11:13.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.413+0000 7efeda000640 1 -- 172.21.15.16:0/1018058013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed40cfc80 con 0x7efed40d0890 2024-09-10T21:11:13.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.413+0000 7efecb400640 1 -- 172.21.15.16:0/1018058013 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efec40067e0 con 0x7efed40d0890 2024-09-10T21:11:13.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.413+0000 7efecb400640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efeb0072610 0x7efeb0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:13.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.413+0000 7efecb400640 1 -- 172.21.15.16:0/1018058013 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7efec4092680 con 0x7efed40d0890 2024-09-10T21:11:13.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.413+0000 7efecbe00640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efeb0072610 0x7efeb0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:13.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.417+0000 7efecbe00640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efeb0072610 0x7efeb0074ad0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efebc00bc70 tx=0x7efebc00b2c0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.417+0000 7efecb400640 1 -- 172.21.15.16:0/1018058013 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efed40cebc0 con 0x7efed40d0890 2024-09-10T21:11:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.597+0000 7efeda000640 1 -- 172.21.15.16:0/1018058013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efed41562a0 con 0x7efed40d0890 2024-09-10T21:11:13.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.597+0000 7efecb400640 1 -- 172.21.15.16:0/1018058013 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16634 (secure 0 0 0) 0x7efec405dcc0 con 0x7efed40d0890 2024-09-10T21:11:13.605 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:13.605 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:11:12.621883+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:11:10.338788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T21:11:11.578459+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:11:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.601+0000 7efeda000640 1 -- 172.21.15.16:0/1018058013 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efeb0072610 msgr2=0x7efeb0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.601+0000 7efeda000640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efeb0072610 0x7efeb0074ad0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efebc00bc70 tx=0x7efebc00b2c0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.601+0000 7efeda000640 1 -- 172.21.15.16:0/1018058013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efed40d0890 msgr2=0x7efed40ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.601+0000 7efeda000640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efed40d0890 0x7efed40ce680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efec400c3f0 tx=0x7efec400c8b0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.601+0000 7efeda000640 1 -- 172.21.15.16:0/1018058013 shutdown_connections 2024-09-10T21:11:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.601+0000 7efeda000640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efeb0072610 0x7efeb0074ad0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.601+0000 7efeda000640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed414f350 0x7efed4152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.601+0000 7efeda000640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efed415f450 0x7efed41529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.601+0000 7efeda000640 1 --2- 172.21.15.16:0/1018058013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efed40d0890 0x7efed40ce680 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.601+0000 7efeda000640 1 -- 172.21.15.16:0/1018058013 >> 172.21.15.16:0/1018058013 conn(0x7efed40d1d80 msgr2=0x7efed41567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.601+0000 7efeda000640 1 -- 172.21.15.16:0/1018058013 shutdown_connections 2024-09-10T21:11:13.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.601+0000 7efeda000640 1 -- 172.21.15.16:0/1018058013 wait complete. 2024-09-10T21:11:13.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:11:13.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 -- 172.21.15.16:0/556102635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f40039f0 msgr2=0x7fb7f4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:13.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 --2- 172.21.15.16:0/556102635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f40039f0 0x7fb7f4003dd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb800001770 tx=0x7fb80001c880 comp rx=0 tx=0).stop 2024-09-10T21:11:13.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb80fe00640 1 -- 172.21.15.16:0/556102635 reap_dead start 2024-09-10T21:11:13.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 -- 172.21.15.16:0/556102635 shutdown_connections 2024-09-10T21:11:13.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 -- 172.21.15.16:0/556102635 >> 172.21.15.16:0/556102635 conn(0x7fb8100cfb20 msgr2=0x7fb8100cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:13.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 -- 172.21.15.16:0/556102635 shutdown_connections 2024-09-10T21:11:13.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 -- 172.21.15.16:0/556102635 wait complete. 2024-09-10T21:11:13.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 Processor -- start 2024-09-10T21:11:13.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 -- start start 2024-09-10T21:11:13.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7f40039f0 0x7fb8100c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:13.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb81015ab00 0x7fb81015af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:13.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb810151290 0x7fb810151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:13.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb810073030 con 0x7fb810151290 2024-09-10T21:11:13.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb810072eb0 con 0x7fb7f40039f0 2024-09-10T21:11:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb816800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb81015f1b0 con 0x7fb81015ab00 2024-09-10T21:11:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb80fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb810151290 0x7fb810151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb80fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb810151290 0x7fb810151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46880/0 (socket says 172.21.15.16:46880) 2024-09-10T21:11:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb80fe00640 1 -- 172.21.15.16:0/3013874568 learned_addr learned my addr 172.21.15.16:0/3013874568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb806a00640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb81015ab00 0x7fb81015af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.769+0000 7fb80f400640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7f40039f0 0x7fb8100c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb80fe00640 1 -- 172.21.15.16:0/3013874568 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb81015ab00 msgr2=0x7fb81015af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb80fe00640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb81015ab00 0x7fb81015af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb80fe00640 1 -- 172.21.15.16:0/3013874568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7f40039f0 msgr2=0x7fb8100c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb80fe00640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7f40039f0 0x7fb8100c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb80fe00640 1 -- 172.21.15.16:0/3013874568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81015cb30 con 0x7fb810151290 2024-09-10T21:11:13.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb80fe00640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb810151290 0x7fb810151710 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb808009fd0 tx=0x7fb808010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb807e00640 1 -- 172.21.15.16:0/3013874568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80800ee20 con 0x7fb810151290 2024-09-10T21:11:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb807e00640 1 -- 172.21.15.16:0/3013874568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb808015070 con 0x7fb810151290 2024-09-10T21:11:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb807e00640 1 -- 172.21.15.16:0/3013874568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808003d10 con 0x7fb810151290 2024-09-10T21:11:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb816800640 1 -- 172.21.15.16:0/3013874568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb810152050 con 0x7fb810151290 2024-09-10T21:11:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb816800640 1 -- 172.21.15.16:0/3013874568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81015cdd0 con 0x7fb810151290 2024-09-10T21:11:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb816800640 1 -- 172.21.15.16:0/3013874568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8100ce610 con 0x7fb810151290 2024-09-10T21:11:13.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb807e00640 1 -- 172.21.15.16:0/3013874568 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb808021030 con 0x7fb810151290 2024-09-10T21:11:13.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb807e00640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7ec0725e0 0x7fb7ec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:13.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.773+0000 7fb807e00640 1 -- 172.21.15.16:0/3013874568 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fb8080934f0 con 0x7fb810151290 2024-09-10T21:11:13.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.777+0000 7fb80f400640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7ec0725e0 0x7fb7ec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:13.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.777+0000 7fb80f400640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7ec0725e0 0x7fb7ec074aa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb800005000 tx=0x7fb800019620 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:13.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.777+0000 7fb807e00640 1 -- 172.21.15.16:0/3013874568 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb80805e4e0 con 0x7fb810151290 2024-09-10T21:11:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb816800640 1 -- 172.21.15.16:0/3013874568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8100b5aa0 con 0x7fb810151290 2024-09-10T21:11:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb807e00640 1 -- 172.21.15.16:0/3013874568 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16634 (secure 0 0 0) 0x7fb808062f10 con 0x7fb810151290 2024-09-10T21:11:13.959 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:13.960 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:11:12.621883+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:11:10.338788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T21:11:11.578459+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:11:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb816800640 1 -- 172.21.15.16:0/3013874568 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7ec0725e0 msgr2=0x7fb7ec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb816800640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7ec0725e0 0x7fb7ec074aa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb800005000 tx=0x7fb800019620 comp rx=0 tx=0).stop 2024-09-10T21:11:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb816800640 1 -- 172.21.15.16:0/3013874568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb810151290 msgr2=0x7fb810151710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb816800640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb810151290 0x7fb810151710 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb808009fd0 tx=0x7fb808010f40 comp rx=0 tx=0).stop 2024-09-10T21:11:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb816800640 1 -- 172.21.15.16:0/3013874568 shutdown_connections 2024-09-10T21:11:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb816800640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7ec0725e0 0x7fb7ec074aa0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb816800640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb810151290 0x7fb810151710 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb816800640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb81015ab00 0x7fb81015af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb816800640 1 --2- 172.21.15.16:0/3013874568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7f40039f0 0x7fb8100c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb816800640 1 -- 172.21.15.16:0/3013874568 >> 172.21.15.16:0/3013874568 conn(0x7fb8100cfb20 msgr2=0x7fb810156fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb816800640 1 -- 172.21.15.16:0/3013874568 shutdown_connections 2024-09-10T21:11:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:13.953+0000 7fb816800640 1 -- 172.21.15.16:0/3013874568 wait complete. 2024-09-10T21:11:13.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:11:14.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.125+0000 7f8093a00640 1 -- 172.21.15.16:0/3621880337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808c151210 msgr2=0x7f808c06e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:14.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.125+0000 7f8093a00640 1 --2- 172.21.15.16:0/3621880337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808c151210 0x7f808c06e6d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8080009900 tx=0x7f808001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:11:14.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.125+0000 7f8093a00640 1 -- 172.21.15.16:0/3621880337 shutdown_connections 2024-09-10T21:11:14.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.125+0000 7f8093a00640 1 --2- 172.21.15.16:0/3621880337 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f808c15e490 0x7f808c072ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.125+0000 7f8093a00640 1 --2- 172.21.15.16:0/3621880337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808c151210 0x7f808c06e6d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.125+0000 7f8093a00640 1 -- 172.21.15.16:0/3621880337 >> 172.21.15.16:0/3621880337 conn(0x7f808c0b16a0 msgr2=0x7f808c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.125+0000 7f8093a00640 1 -- 172.21.15.16:0/3621880337 shutdown_connections 2024-09-10T21:11:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.125+0000 7f8093a00640 1 -- 172.21.15.16:0/3621880337 wait complete. 2024-09-10T21:11:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8093a00640 1 Processor -- start 2024-09-10T21:11:14.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8093a00640 1 -- start start 2024-09-10T21:11:14.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8093a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f808c0726b0 0x7f808c15d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:14.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8093a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808c151210 0x7f808c1fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:14.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8093a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f808c15e490 0x7f808c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:14.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8093a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f808c167d70 con 0x7f808c151210 2024-09-10T21:11:14.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8093a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f808c167bf0 con 0x7f808c15e490 2024-09-10T21:11:14.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8093a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f808c167ef0 con 0x7f808c0726b0 2024-09-10T21:11:14.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f808be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808c151210 0x7f808c1fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:14.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f808be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808c151210 0x7f808c1fbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46906/0 (socket says 172.21.15.16:46906) 2024-09-10T21:11:14.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f808be00640 1 -- 172.21.15.16:0/2079449738 learned_addr learned my addr 172.21.15.16:0/2079449738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8091000640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f808c0726b0 0x7f808c15d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8091a00640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f808c15e490 0x7f808c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f808be00640 1 -- 172.21.15.16:0/2079449738 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f808c0726b0 msgr2=0x7f808c15d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f808be00640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f808c0726b0 0x7f808c15d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f808be00640 1 -- 172.21.15.16:0/2079449738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f808c15e490 msgr2=0x7f808c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f808be00640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f808c15e490 0x7f808c15a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f808be00640 1 -- 172.21.15.16:0/2079449738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c145280 con 0x7f808c151210 2024-09-10T21:11:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8091000640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f808c0726b0 0x7f808c15d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f808be00640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808c151210 0x7f808c1fbcb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f807800c490 tx=0x7f8078001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:14.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8088c00640 1 -- 172.21.15.16:0/2079449738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078017070 con 0x7f808c151210 2024-09-10T21:11:14.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8088c00640 1 -- 172.21.15.16:0/2079449738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8078003600 con 0x7f808c151210 2024-09-10T21:11:14.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8093a00640 1 -- 172.21.15.16:0/2079449738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c0ce210 con 0x7f808c151210 2024-09-10T21:11:14.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8088c00640 1 -- 172.21.15.16:0/2079449738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078003a90 con 0x7f808c151210 2024-09-10T21:11:14.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.129+0000 7f8093a00640 1 -- 172.21.15.16:0/2079449738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c0cc9c0 con 0x7f808c151210 2024-09-10T21:11:14.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.133+0000 7f8093a00640 1 -- 172.21.15.16:0/2079449738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c06e6d0 con 0x7f808c151210 2024-09-10T21:11:14.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.133+0000 7f8088c00640 1 -- 172.21.15.16:0/2079449738 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8078003c30 con 0x7f808c151210 2024-09-10T21:11:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.133+0000 7f8088c00640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f806c0725e0 0x7f806c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.133+0000 7f8088c00640 1 -- 172.21.15.16:0/2079449738 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f8078092840 con 0x7f808c151210 2024-09-10T21:11:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.133+0000 7f8091000640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f806c0725e0 0x7f806c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.133+0000 7f8091000640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f806c0725e0 0x7f806c074aa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f80800af000 tx=0x7f808001b120 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:14.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.137+0000 7f8088c00640 1 -- 172.21.15.16:0/2079449738 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f807805df90 con 0x7f808c151210 2024-09-10T21:11:14.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:14.353+0000 7f8093a00640 1 -- 172.21.15.16:0/2079449738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f808c0b2c50 con 0x7f808c151210 2024-09-10T21:11:15.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.229+0000 7f8088c00640 1 -- 172.21.15.16:0/2079449738 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f807805d890 con 0x7f808c151210 2024-09-10T21:11:15.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.229+0000 7f8093a00640 1 -- 172.21.15.16:0/2079449738 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f806c0725e0 msgr2=0x7f806c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:15.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.229+0000 7f8093a00640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f806c0725e0 0x7f806c074aa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f80800af000 tx=0x7f808001b120 comp rx=0 tx=0).stop 2024-09-10T21:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.229+0000 7f8093a00640 1 -- 172.21.15.16:0/2079449738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808c151210 msgr2=0x7f808c1fbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.229+0000 7f8093a00640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808c151210 0x7f808c1fbcb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f807800c490 tx=0x7f8078001e80 comp rx=0 tx=0).stop 2024-09-10T21:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.233+0000 7f8093a00640 1 -- 172.21.15.16:0/2079449738 shutdown_connections 2024-09-10T21:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.233+0000 7f8093a00640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f806c0725e0 0x7f806c074aa0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.233+0000 7f8093a00640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f808c15e490 0x7f808c15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.233+0000 7f8093a00640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808c151210 0x7f808c1fbcb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.233+0000 7f8093a00640 1 --2- 172.21.15.16:0/2079449738 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f808c0726b0 0x7f808c15d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.233+0000 7f8093a00640 1 -- 172.21.15.16:0/2079449738 >> 172.21.15.16:0/2079449738 conn(0x7f808c0b16a0 msgr2=0x7f808c1505c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.233+0000 7f8093a00640 1 -- 172.21.15.16:0/2079449738 shutdown_connections 2024-09-10T21:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.233+0000 7f8093a00640 1 -- 172.21.15.16:0/2079449738 wait complete. 2024-09-10T21:11:15.264 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:11:15.264 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:11:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 -- 172.21.15.16:0/212246110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed80cf1e0 msgr2=0x7fbed80af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 --2- 172.21.15.16:0/212246110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed80cf1e0 0x7fbed80af050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009c80 tx=0x7fbed001cd30 comp rx=0 tx=0).stop 2024-09-10T21:11:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 -- 172.21.15.16:0/212246110 shutdown_connections 2024-09-10T21:11:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 --2- 172.21.15.16:0/212246110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed80cf1e0 0x7fbed80af050 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 --2- 172.21.15.16:0/212246110 >> v2:172.21.15.52:3300/0 conn(0x7fbed80ce8c0 0x7fbed80ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 -- 172.21.15.16:0/212246110 >> 172.21.15.16:0/212246110 conn(0x7fbed80d1c70 msgr2=0x7fbed80d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 -- 172.21.15.16:0/212246110 shutdown_connections 2024-09-10T21:11:15.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 -- 172.21.15.16:0/212246110 wait complete. 2024-09-10T21:11:15.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 Processor -- start 2024-09-10T21:11:15.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 -- start start 2024-09-10T21:11:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed80b0980 0x7fbed815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbed80ce8c0 0x7fbed8159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbed80cf1e0 0x7fbed8156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbed8169340 con 0x7fbed80b0980 2024-09-10T21:11:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81691c0 con 0x7fbed80cf1e0 2024-09-10T21:11:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.413+0000 7fbedec00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fbed81694c0 con 0x7fbed80ce8c0 2024-09-10T21:11:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed80b0980 0x7fbed815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed80b0980 0x7fbed815ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46932/0 (socket says 172.21.15.16:46932) 2024-09-10T21:11:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedd800640 1 -- 172.21.15.16:0/3239099578 learned_addr learned my addr 172.21.15.16:0/3239099578 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbede200640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbed80cf1e0 0x7fbed8156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedce00640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbed80ce8c0 0x7fbed8159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedd800640 1 -- 172.21.15.16:0/3239099578 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbed80ce8c0 msgr2=0x7fbed8159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedd800640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbed80ce8c0 0x7fbed8159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedd800640 1 -- 172.21.15.16:0/3239099578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbed80cf1e0 msgr2=0x7fbed8156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedd800640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbed80cf1e0 0x7fbed8156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedd800640 1 -- 172.21.15.16:0/3239099578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed80ca480 con 0x7fbed80b0980 2024-09-10T21:11:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedce00640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbed80ce8c0 0x7fbed8159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedd800640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed80b0980 0x7fbed815ada0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00b550 tx=0x7fbecc00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbed5600640 1 -- 172.21.15.16:0/3239099578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc014070 con 0x7fbed80b0980 2024-09-10T21:11:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbed5600640 1 -- 172.21.15.16:0/3239099578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbecc010980 con 0x7fbed80b0980 2024-09-10T21:11:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbed5600640 1 -- 172.21.15.16:0/3239099578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc010dd0 con 0x7fbed80b0980 2024-09-10T21:11:15.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedec00640 1 -- 172.21.15.16:0/3239099578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed8159b80 con 0x7fbed80b0980 2024-09-10T21:11:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedec00640 1 -- 172.21.15.16:0/3239099578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed815a140 con 0x7fbed80b0980 2024-09-10T21:11:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbedec00640 1 -- 172.21.15.16:0/3239099578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed80ceca0 con 0x7fbed80b0980 2024-09-10T21:11:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.417+0000 7fbed5600640 1 -- 172.21.15.16:0/3239099578 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbecc00f3c0 con 0x7fbed80b0980 2024-09-10T21:11:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.421+0000 7fbed5600640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbeb80725e0 0x7fbeb8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.421+0000 7fbed5600640 1 -- 172.21.15.16:0/3239099578 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fbecc093310 con 0x7fbed80b0980 2024-09-10T21:11:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.421+0000 7fbedce00640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbeb80725e0 0x7fbeb8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.421+0000 7fbedce00640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbeb80725e0 0x7fbeb8074aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbed815fb20 tx=0x7fbec4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.425+0000 7fbed5600640 1 -- 172.21.15.16:0/3239099578 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbecc05e9e0 con 0x7fbed80b0980 2024-09-10T21:11:15.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:15.633+0000 7fbedec00640 1 -- 172.21.15.16:0/3239099578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbed815c040 con 0x7fbed80b0980 2024-09-10T21:11:16.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.237+0000 7fbed5600640 1 -- 172.21.15.16:0/3239099578 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fbecc05e2e0 con 0x7fbed80b0980 2024-09-10T21:11:16.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.237+0000 7fbedec00640 1 -- 172.21.15.16:0/3239099578 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbeb80725e0 msgr2=0x7fbeb8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:16.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.237+0000 7fbedec00640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbeb80725e0 0x7fbeb8074aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbed815fb20 tx=0x7fbec4009040 comp rx=0 tx=0).stop 2024-09-10T21:11:16.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.237+0000 7fbedec00640 1 -- 172.21.15.16:0/3239099578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed80b0980 msgr2=0x7fbed815ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:16.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.237+0000 7fbedec00640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed80b0980 0x7fbed815ada0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00b550 tx=0x7fbecc00ba10 comp rx=0 tx=0).stop 2024-09-10T21:11:16.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.237+0000 7fbedec00640 1 -- 172.21.15.16:0/3239099578 shutdown_connections 2024-09-10T21:11:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.237+0000 7fbedec00640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbeb80725e0 0x7fbeb8074aa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.237+0000 7fbedec00640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbed80cf1e0 0x7fbed8156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.237+0000 7fbedec00640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbed80ce8c0 0x7fbed8159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.237+0000 7fbedec00640 1 --2- 172.21.15.16:0/3239099578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed80b0980 0x7fbed815ada0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.237+0000 7fbedec00640 1 -- 172.21.15.16:0/3239099578 >> 172.21.15.16:0/3239099578 conn(0x7fbed80d1c70 msgr2=0x7fbed8071d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.237+0000 7fbedec00640 1 -- 172.21.15.16:0/3239099578 shutdown_connections 2024-09-10T21:11:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.237+0000 7fbedec00640 1 -- 172.21.15.16:0/3239099578 wait complete. 2024-09-10T21:11:16.270 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:11:16.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:11:16.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 -- 172.21.15.16:0/3514632022 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f248c0ca170 msgr2=0x7f248c0ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:16.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 --2- 172.21.15.16:0/3514632022 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f248c0ca170 0x7f248c0ca5b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f248400ad90 tx=0x7f248401dbf0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 -- 172.21.15.16:0/3514632022 shutdown_connections 2024-09-10T21:11:16.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 --2- 172.21.15.16:0/3514632022 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f248c0ca170 0x7f248c0ca5b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 --2- 172.21.15.16:0/3514632022 >> v2:172.21.15.52:3300/0 conn(0x7f248c15a270 0x7f248c0ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 -- 172.21.15.16:0/3514632022 >> 172.21.15.16:0/3514632022 conn(0x7f248c0cfb20 msgr2=0x7f248c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:16.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 -- 172.21.15.16:0/3514632022 shutdown_connections 2024-09-10T21:11:16.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 -- 172.21.15.16:0/3514632022 wait complete. 2024-09-10T21:11:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 Processor -- start 2024-09-10T21:11:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 -- start start 2024-09-10T21:11:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0ca170 0x7f248c145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f248c0d06b0 0x7f248c1459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f248c15a270 0x7f248c155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f248c072bb0 con 0x7f248c0ca170 2024-09-10T21:11:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f248c072a30 con 0x7f248c15a270 2024-09-10T21:11:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2494200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f248c072d30 con 0x7f248c0d06b0 2024-09-10T21:11:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2491800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0ca170 0x7f248c145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2491800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0ca170 0x7f248c145000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46940/0 (socket says 172.21.15.16:46940) 2024-09-10T21:11:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2491800640 1 -- 172.21.15.16:0/3965670389 learned_addr learned my addr 172.21.15.16:0/3965670389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2490e00640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f248c0d06b0 0x7f248c1459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.417+0000 7f2492200640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f248c15a270 0x7f248c155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2491800640 1 -- 172.21.15.16:0/3965670389 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f248c0d06b0 msgr2=0x7f248c1459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2491800640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f248c0d06b0 0x7f248c1459f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2491800640 1 -- 172.21.15.16:0/3965670389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f248c15a270 msgr2=0x7f248c155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2491800640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f248c15a270 0x7f248c155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2491800640 1 -- 172.21.15.16:0/3965670389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c0c6e40 con 0x7f248c0ca170 2024-09-10T21:11:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2490e00640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f248c0d06b0 0x7f248c1459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2492200640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f248c15a270 0x7f248c155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2491800640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0ca170 0x7f248c145000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f248000e970 tx=0x7f248000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2489600640 1 -- 172.21.15.16:0/3965670389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248000cd80 con 0x7f248c0ca170 2024-09-10T21:11:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2489600640 1 -- 172.21.15.16:0/3965670389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2480004020 con 0x7f248c0ca170 2024-09-10T21:11:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2489600640 1 -- 172.21.15.16:0/3965670389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480004470 con 0x7f248c0ca170 2024-09-10T21:11:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2494200640 1 -- 172.21.15.16:0/3965670389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c0c7120 con 0x7f248c0ca170 2024-09-10T21:11:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2494200640 1 -- 172.21.15.16:0/3965670389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c0c76e0 con 0x7f248c0ca170 2024-09-10T21:11:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2494200640 1 -- 172.21.15.16:0/3965670389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248c0c6bd0 con 0x7f248c0ca170 2024-09-10T21:11:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2489600640 1 -- 172.21.15.16:0/3965670389 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2480003600 con 0x7f248c0ca170 2024-09-10T21:11:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2489600640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f246c072610 0x7f246c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.421+0000 7f2489600640 1 -- 172.21.15.16:0/3965670389 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2480093c50 con 0x7f248c0ca170 2024-09-10T21:11:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.425+0000 7f2490e00640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f246c072610 0x7f246c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.425+0000 7f2490e00640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f246c072610 0x7f246c074ad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f248c0ca120 tx=0x7f2478009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:16.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.429+0000 7f2489600640 1 -- 172.21.15.16:0/3965670389 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f248005f290 con 0x7f248c0ca170 2024-09-10T21:11:16.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.633+0000 7f2494200640 1 -- 172.21.15.16:0/3965670389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f248c0ca5b0 con 0x7f248c0ca170 2024-09-10T21:11:16.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.637+0000 7f2489600640 1 -- 172.21.15.16:0/3965670389 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5232 (secure 0 0 0) 0x7f248005eb90 con 0x7f248c0ca170 2024-09-10T21:11:16.642 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:16.642 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-10T21:11:16:232531+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/788814920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":788814920},{"type":"v1","addr":"172.21.15.16:6837","nonce":788814920}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2275087718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2275087718},{"type":"v1","addr":"172.21.15.16:6836","nonce":2275087718}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":134516,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3457135235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3457135235},{"type":"v1","addr":"172.21.15.52:6834","nonce":3457135235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:11:12.621848+0000","modified":"2024-09-10T21:11:15.636822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134519":{"gid":134519,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/4218881085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4218881085},{"type":"v1","addr":"172.21.15.52:6835","nonce":4218881085}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134519,"qdb_cluster":[134519]},"id":2}]} 2024-09-10T21:11:16.642 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-10T21:11:16.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.637+0000 7f2494200640 1 -- 172.21.15.16:0/3965670389 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f246c072610 msgr2=0x7f246c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:16.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.637+0000 7f2494200640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f246c072610 0x7f246c074ad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f248c0ca120 tx=0x7f2478009040 comp rx=0 tx=0).stop 2024-09-10T21:11:16.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.637+0000 7f2494200640 1 -- 172.21.15.16:0/3965670389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0ca170 msgr2=0x7f248c145000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:16.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.637+0000 7f2494200640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0ca170 0x7f248c145000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f248000e970 tx=0x7f248000ee30 comp rx=0 tx=0).stop 2024-09-10T21:11:16.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.637+0000 7f2494200640 1 -- 172.21.15.16:0/3965670389 shutdown_connections 2024-09-10T21:11:16.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.637+0000 7f2494200640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f246c072610 0x7f246c074ad0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.637+0000 7f2494200640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f248c15a270 0x7f248c155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.637+0000 7f2494200640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f248c0d06b0 0x7f248c1459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.637+0000 7f2494200640 1 --2- 172.21.15.16:0/3965670389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0ca170 0x7f248c145000 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.637+0000 7f2494200640 1 -- 172.21.15.16:0/3965670389 >> 172.21.15.16:0/3965670389 conn(0x7f248c0cfb20 msgr2=0x7f248c159350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.637+0000 7f2494200640 1 -- 172.21.15.16:0/3965670389 shutdown_connections 2024-09-10T21:11:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.637+0000 7f2494200640 1 -- 172.21.15.16:0/3965670389 wait complete. 2024-09-10T21:11:16.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:11:16.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 -- 172.21.15.16:0/3949720599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a40b13c0 msgr2=0x7f50a40afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 --2- 172.21.15.16:0/3949720599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a40b13c0 0x7f50a40afac0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f50a40698f0 tx=0x7f509c01cc60 comp rx=0 tx=0).stop 2024-09-10T21:11:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 -- 172.21.15.16:0/3949720599 shutdown_connections 2024-09-10T21:11:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 --2- 172.21.15.16:0/3949720599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a40b13c0 0x7f50a40afac0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 --2- 172.21.15.16:0/3949720599 >> v2:172.21.15.52:3300/0 conn(0x7f50a40716f0 0x7f50a40af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 -- 172.21.15.16:0/3949720599 >> 172.21.15.16:0/3949720599 conn(0x7f50a409b130 msgr2=0x7f50a40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 -- 172.21.15.16:0/3949720599 shutdown_connections 2024-09-10T21:11:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 -- 172.21.15.16:0/3949720599 wait complete. 2024-09-10T21:11:16.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 Processor -- start 2024-09-10T21:11:16.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 -- start start 2024-09-10T21:11:16.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50a40716f0 0x7f50a415dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:16.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40b13c0 0x7f50a4149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:16.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a415f400 0x7f50a414a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50a4168f60 con 0x7f50a415f400 2024-09-10T21:11:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f50a4168de0 con 0x7f50a40b13c0 2024-09-10T21:11:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a9200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f50a41690e0 con 0x7f50a40716f0 2024-09-10T21:11:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a415f400 0x7f50a414a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a415f400 0x7f50a414a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46962/0 (socket says 172.21.15.16:46962) 2024-09-10T21:11:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a2800640 1 -- 172.21.15.16:0/545724355 learned_addr learned my addr 172.21.15.16:0/545724355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f509be00640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40b13c0 0x7f50a4149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.805+0000 7f50a1e00640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50a40716f0 0x7f50a415dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f50a2800640 1 -- 172.21.15.16:0/545724355 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50a40716f0 msgr2=0x7f50a415dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f50a2800640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50a40716f0 0x7f50a415dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f50a2800640 1 -- 172.21.15.16:0/545724355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40b13c0 msgr2=0x7f50a4149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f50a2800640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40b13c0 0x7f50a4149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:16.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f50a2800640 1 -- 172.21.15.16:0/545724355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a41f8c10 con 0x7f50a415f400 2024-09-10T21:11:16.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f50a1e00640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50a40716f0 0x7f50a415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:16.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f509be00640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40b13c0 0x7f50a4149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:16.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f50a2800640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a415f400 0x7f50a414a1d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f509c019870 tx=0x7f509c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f509a000640 1 -- 172.21.15.16:0/545724355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c0bb070 con 0x7f50a415f400 2024-09-10T21:11:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f509a000640 1 -- 172.21.15.16:0/545724355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f509c004d40 con 0x7f50a415f400 2024-09-10T21:11:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f50a9200640 1 -- 172.21.15.16:0/545724355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a41f8f10 con 0x7f50a415f400 2024-09-10T21:11:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f509a000640 1 -- 172.21.15.16:0/545724355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c0b2040 con 0x7f50a415f400 2024-09-10T21:11:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f50a9200640 1 -- 172.21.15.16:0/545724355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a41f9750 con 0x7f50a415f400 2024-09-10T21:11:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f50a9200640 1 -- 172.21.15.16:0/545724355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a4070fe0 con 0x7f50a415f400 2024-09-10T21:11:16.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f509a000640 1 -- 172.21.15.16:0/545724355 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f509c0b5030 con 0x7f50a415f400 2024-09-10T21:11:16.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f509a000640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50800725e0 0x7f5080074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:16.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.809+0000 7f509a000640 1 -- 172.21.15.16:0/545724355 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f509c12ce70 con 0x7f50a415f400 2024-09-10T21:11:16.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.813+0000 7f50a1e00640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50800725e0 0x7f5080074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:16.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.813+0000 7f50a1e00640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50800725e0 0x7f5080074aa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f50940016a0 tx=0x7f5094009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.813+0000 7f509a000640 1 -- 172.21.15.16:0/545724355 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f50a41f8f10 con 0x7f50a415f400 2024-09-10T21:11:16.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.989+0000 7f50a9200640 1 -- 172.21.15.16:0/545724355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50a40cc340 con 0x7f50a415f400 2024-09-10T21:11:16.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.993+0000 7f509a000640 1 -- 172.21.15.16:0/545724355 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16621 (secure 0 0 0) 0x7f509c016970 con 0x7f50a415f400 2024-09-10T21:11:16.998 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:16.999 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:11:14.204029+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:11:10.338788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T21:11:11.578459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:11:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.993+0000 7f50a9200640 1 -- 172.21.15.16:0/545724355 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50800725e0 msgr2=0x7f5080074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.993+0000 7f50a9200640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50800725e0 0x7f5080074aa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f50940016a0 tx=0x7f5094009040 comp rx=0 tx=0).stop 2024-09-10T21:11:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.993+0000 7f50a9200640 1 -- 172.21.15.16:0/545724355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a415f400 msgr2=0x7f50a414a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.993+0000 7f50a9200640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a415f400 0x7f50a414a1d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f509c019870 tx=0x7f509c009300 comp rx=0 tx=0).stop 2024-09-10T21:11:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.993+0000 7f50a9200640 1 -- 172.21.15.16:0/545724355 shutdown_connections 2024-09-10T21:11:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.993+0000 7f50a9200640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50800725e0 0x7f5080074aa0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.993+0000 7f50a9200640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a415f400 0x7f50a414a1d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.993+0000 7f50a9200640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50a40b13c0 0x7f50a4149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.993+0000 7f50a9200640 1 --2- 172.21.15.16:0/545724355 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50a40716f0 0x7f50a415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.993+0000 7f50a9200640 1 -- 172.21.15.16:0/545724355 >> 172.21.15.16:0/545724355 conn(0x7f50a409b130 msgr2=0x7f50a40d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.993+0000 7f50a9200640 1 -- 172.21.15.16:0/545724355 shutdown_connections 2024-09-10T21:11:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:16.993+0000 7f50a9200640 1 -- 172.21.15.16:0/545724355 wait complete. 2024-09-10T21:11:17.020 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:11:17.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:11:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 -- 172.21.15.16:0/39943540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb415bc70 msgr2=0x7f5eb406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 --2- 172.21.15.16:0/39943540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb415bc70 0x7f5eb406ffe0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009900 tx=0x7f5e9c01c880 comp rx=0 tx=0).stop 2024-09-10T21:11:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 -- 172.21.15.16:0/39943540 shutdown_connections 2024-09-10T21:11:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 --2- 172.21.15.16:0/39943540 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5eb4070520 0x7f5eb40ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 --2- 172.21.15.16:0/39943540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb415bc70 0x7f5eb406ffe0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 -- 172.21.15.16:0/39943540 >> 172.21.15.16:0/39943540 conn(0x7f5eb40cfb20 msgr2=0x7f5eb40cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 -- 172.21.15.16:0/39943540 shutdown_connections 2024-09-10T21:11:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 -- 172.21.15.16:0/39943540 wait complete. 2024-09-10T21:11:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 Processor -- start 2024-09-10T21:11:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 -- start start 2024-09-10T21:11:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5eb4070520 0x7f5eb4153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eb4154f90 0x7f5eb4148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb415bc70 0x7f5eb41f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb4167980 con 0x7f5eb415bc70 2024-09-10T21:11:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb40cb690 con 0x7f5eb4154f90 2024-09-10T21:11:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb9a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb4167b00 con 0x7f5eb4070520 2024-09-10T21:11:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb415bc70 0x7f5eb41f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb415bc70 0x7f5eb41f8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46982/0 (socket says 172.21.15.16:46982) 2024-09-10T21:11:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb3400640 1 -- 172.21.15.16:0/2990860625 learned_addr learned my addr 172.21.15.16:0/2990860625 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eb2a00640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5eb4070520 0x7f5eb4153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.161+0000 7f5eaa000640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eb4154f90 0x7f5eb4148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eb3400640 1 -- 172.21.15.16:0/2990860625 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5eb4070520 msgr2=0x7f5eb4153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eb3400640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5eb4070520 0x7f5eb4153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eb3400640 1 -- 172.21.15.16:0/2990860625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eb4154f90 msgr2=0x7f5eb4148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eb3400640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eb4154f90 0x7f5eb4148c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eb3400640 1 -- 172.21.15.16:0/2990860625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb41519f0 con 0x7f5eb415bc70 2024-09-10T21:11:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eaa000640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eb4154f90 0x7f5eb4148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eb3400640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb415bc70 0x7f5eb41f8120 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00eb20 tx=0x7f5eac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eab400640 1 -- 172.21.15.16:0/2990860625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac004b30 con 0x7f5eb415bc70 2024-09-10T21:11:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eab400640 1 -- 172.21.15.16:0/2990860625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5eac004cf0 con 0x7f5eb415bc70 2024-09-10T21:11:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eab400640 1 -- 172.21.15.16:0/2990860625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac005280 con 0x7f5eb415bc70 2024-09-10T21:11:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eb9a00640 1 -- 172.21.15.16:0/2990860625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb4151cd0 con 0x7f5eb415bc70 2024-09-10T21:11:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eb9a00640 1 -- 172.21.15.16:0/2990860625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb4152290 con 0x7f5eb415bc70 2024-09-10T21:11:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eb9a00640 1 -- 172.21.15.16:0/2990860625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb40ca530 con 0x7f5eb415bc70 2024-09-10T21:11:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eab400640 1 -- 172.21.15.16:0/2990860625 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5eac005420 con 0x7f5eb415bc70 2024-09-10T21:11:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eab400640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5e900725e0 0x7f5e90074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.165+0000 7f5eab400640 1 -- 172.21.15.16:0/2990860625 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5eac093fe0 con 0x7f5eb415bc70 2024-09-10T21:11:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.169+0000 7f5eb2a00640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5e900725e0 0x7f5e90074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.169+0000 7f5eb2a00640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5e900725e0 0x7f5e90074aa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4007fd0 tx=0x7f5ea4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.173+0000 7f5eab400640 1 -- 172.21.15.16:0/2990860625 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5eac05f620 con 0x7f5eb415bc70 2024-09-10T21:11:17.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.341+0000 7f5eb9a00640 1 -- 172.21.15.16:0/2990860625 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5eb4149150 con 0x7f5e900725e0 2024-09-10T21:11:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.345+0000 7f5eab400640 1 -- 172.21.15.16:0/2990860625 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250866 (secure 0 0 0) 0x7f5eb4149150 con 0x7f5e900725e0 2024-09-10T21:11:17.355 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:17.357 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:11:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.349+0000 7f5eb9a00640 1 -- 172.21.15.16:0/2990860625 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5e900725e0 msgr2=0x7f5e90074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.349+0000 7f5eb9a00640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5e900725e0 0x7f5e90074aa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4007fd0 tx=0x7f5ea4009040 comp rx=0 tx=0).stop 2024-09-10T21:11:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.349+0000 7f5eb9a00640 1 -- 172.21.15.16:0/2990860625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb415bc70 msgr2=0x7f5eb41f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.349+0000 7f5eb9a00640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb415bc70 0x7f5eb41f8120 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00eb20 tx=0x7f5eac00c2d0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.349+0000 7f5eb9a00640 1 -- 172.21.15.16:0/2990860625 shutdown_connections 2024-09-10T21:11:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.349+0000 7f5eb9a00640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5e900725e0 0x7f5e90074aa0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.349+0000 7f5eb9a00640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb415bc70 0x7f5eb41f8120 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.349+0000 7f5eb9a00640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eb4154f90 0x7f5eb4148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.349+0000 7f5eb9a00640 1 --2- 172.21.15.16:0/2990860625 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5eb4070520 0x7f5eb4153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.349+0000 7f5eb9a00640 1 -- 172.21.15.16:0/2990860625 >> 172.21.15.16:0/2990860625 conn(0x7f5eb40cfb20 msgr2=0x7f5eb406fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.349+0000 7f5eb9a00640 1 -- 172.21.15.16:0/2990860625 shutdown_connections 2024-09-10T21:11:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.349+0000 7f5eb9a00640 1 -- 172.21.15.16:0/2990860625 wait complete. 2024-09-10T21:11:17.377 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-09-10T21:11:15.465713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":129,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253316,"kb_used_data":3780,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721404,"statfs":{"total":773094113280,"available":772834717696,"internally_reserved":0,"allocated":3870720,"data_stored":2176274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.270513"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245694+0000","last_change":"2024-09-10T21:11:13.244223+0000","last_active":"2024-09-10T21:11:13.245694+0000","last_peered":"2024-09-10T21:11:13.245694+0000","last_clean":"2024-09-10T21:11:13.245694+0000","last_became_active":"2024-09-10T21:11:13.243954+0000","last_became_peered":"2024-09-10T21:11:13.243954+0000","last_unstale":"2024-09-10T21:11:13.245694+0000","last_undegraded":"2024-09-10T21:11:13.245694+0000","last_fullsized":"2024-09-10T21:11:13.245694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:23.544274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.249545+0000","last_change":"2024-09-10T21:11:13.247771+0000","last_active":"2024-09-10T21:11:13.249545+0000","last_peered":"2024-09-10T21:11:13.249545+0000","last_clean":"2024-09-10T21:11:13.249545+0000","last_became_active":"2024-09-10T21:11:13.247293+0000","last_became_peered":"2024-09-10T21:11:13.247293+0000","last_unstale":"2024-09-10T21:11:13.249545+0000","last_undegraded":"2024-09-10T21:11:13.249545+0000","last_fullsized":"2024-09-10T21:11:13.249545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:31.442572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.206691+0000","last_change":"2024-09-10T21:11:12.206691+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.206691+0000","last_undegraded":"2024-09-10T21:11:12.206691+0000","last_fullsized":"2024-09-10T21:11:12.206691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248003+0000","last_change":"2024-09-10T21:11:13.233899+0000","last_active":"2024-09-10T21:11:13.248003+0000","last_peered":"2024-09-10T21:11:13.248003+0000","last_clean":"2024-09-10T21:11:13.248003+0000","last_became_active":"2024-09-10T21:11:13.219250+0000","last_became_peered":"2024-09-10T21:11:13.219250+0000","last_unstale":"2024-09-10T21:11:13.248003+0000","last_undegraded":"2024-09-10T21:11:13.248003+0000","last_fullsized":"2024-09-10T21:11:13.248003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:51.841630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245745+0000","last_change":"2024-09-10T21:11:13.235979+0000","last_active":"2024-09-10T21:11:13.245745+0000","last_peered":"2024-09-10T21:11:13.245745+0000","last_clean":"2024-09-10T21:11:13.245745+0000","last_became_active":"2024-09-10T21:11:13.233126+0000","last_became_peered":"2024-09-10T21:11:13.233126+0000","last_unstale":"2024-09-10T21:11:13.245745+0000","last_undegraded":"2024-09-10T21:11:13.245745+0000","last_fullsized":"2024-09-10T21:11:13.245745+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:15.822007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.209212+0000","last_change":"2024-09-10T21:11:12.209212+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.209212+0000","last_undegraded":"2024-09-10T21:11:12.209212+0000","last_fullsized":"2024-09-10T21:11:12.209212+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.234785+0000","last_change":"2024-09-10T21:11:13.232062+0000","last_active":"2024-09-10T21:11:13.234785+0000","last_peered":"2024-09-10T21:11:13.234785+0000","last_clean":"2024-09-10T21:11:13.234785+0000","last_became_active":"2024-09-10T21:11:13.231580+0000","last_became_peered":"2024-09-10T21:11:13.231580+0000","last_unstale":"2024-09-10T21:11:13.234785+0000","last_undegraded":"2024-09-10T21:11:13.234785+0000","last_fullsized":"2024-09-10T21:11:13.234785+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:12.602121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245385+0000","last_change":"2024-09-10T21:11:13.240369+0000","last_active":"2024-09-10T21:11:13.245385+0000","last_peered":"2024-09-10T21:11:13.245385+0000","last_clean":"2024-09-10T21:11:13.245385+0000","last_became_active":"2024-09-10T21:11:13.239717+0000","last_became_peered":"2024-09-10T21:11:13.239717+0000","last_unstale":"2024-09-10T21:11:13.245385+0000","last_undegraded":"2024-09-10T21:11:13.245385+0000","last_fullsized":"2024-09-10T21:11:13.245385+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:21.291610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.207224+0000","last_change":"2024-09-10T21:11:12.207224+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.207224+0000","last_undegraded":"2024-09-10T21:11:12.207224+0000","last_fullsized":"2024-09-10T21:11:12.207224+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247504+0000","last_change":"2024-09-10T21:11:13.233269+0000","last_active":"2024-09-10T21:11:13.247504+0000","last_peered":"2024-09-10T21:11:13.247504+0000","last_clean":"2024-09-10T21:11:13.247504+0000","last_became_active":"2024-09-10T21:11:13.228592+0000","last_became_peered":"2024-09-10T21:11:13.228592+0000","last_unstale":"2024-09-10T21:11:13.247504+0000","last_undegraded":"2024-09-10T21:11:13.247504+0000","last_fullsized":"2024-09-10T21:11:13.247504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:10.238782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.251364+0000","last_change":"2024-09-10T21:11:13.233758+0000","last_active":"2024-09-10T21:11:13.251364+0000","last_peered":"2024-09-10T21:11:13.251364+0000","last_clean":"2024-09-10T21:11:13.251364+0000","last_became_active":"2024-09-10T21:11:13.233244+0000","last_became_peered":"2024-09-10T21:11:13.233244+0000","last_unstale":"2024-09-10T21:11:13.251364+0000","last_undegraded":"2024-09-10T21:11:13.251364+0000","last_fullsized":"2024-09-10T21:11:13.251364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:53.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.207305+0000","last_change":"2024-09-10T21:11:12.207305+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.207305+0000","last_undegraded":"2024-09-10T21:11:12.207305+0000","last_fullsized":"2024-09-10T21:11:12.207305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.253665+0000","last_change":"2024-09-10T21:11:13.250940+0000","last_active":"2024-09-10T21:11:13.253665+0000","last_peered":"2024-09-10T21:11:13.253665+0000","last_clean":"2024-09-10T21:11:13.253665+0000","last_became_active":"2024-09-10T21:11:13.250694+0000","last_became_peered":"2024-09-10T21:11:13.250694+0000","last_unstale":"2024-09-10T21:11:13.253665+0000","last_undegraded":"2024-09-10T21:11:13.253665+0000","last_fullsized":"2024-09-10T21:11:13.253665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:33.344003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.228510+0000","last_change":"2024-09-10T21:11:13.216431+0000","last_active":"2024-09-10T21:11:13.228510+0000","last_peered":"2024-09-10T21:11:13.228510+0000","last_clean":"2024-09-10T21:11:13.228510+0000","last_became_active":"2024-09-10T21:11:13.216126+0000","last_became_peered":"2024-09-10T21:11:13.216126+0000","last_unstale":"2024-09-10T21:11:13.228510+0000","last_undegraded":"2024-09-10T21:11:13.228510+0000","last_fullsized":"2024-09-10T21:11:13.228510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:27.677406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.229218+0000","last_change":"2024-09-10T21:11:13.227625+0000","last_active":"2024-09-10T21:11:13.229218+0000","last_peered":"2024-09-10T21:11:13.229218+0000","last_clean":"2024-09-10T21:11:13.229218+0000","last_became_active":"2024-09-10T21:11:13.227357+0000","last_became_peered":"2024-09-10T21:11:13.227357+0000","last_unstale":"2024-09-10T21:11:13.229218+0000","last_undegraded":"2024-09-10T21:11:13.229218+0000","last_fullsized":"2024-09-10T21:11:13.229218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:04.567850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239511+0000","last_change":"2024-09-10T21:11:13.233228+0000","last_active":"2024-09-10T21:11:13.239511+0000","last_peered":"2024-09-10T21:11:13.239511+0000","last_clean":"2024-09-10T21:11:13.239511+0000","last_became_active":"2024-09-10T21:11:13.232879+0000","last_became_peered":"2024-09-10T21:11:13.232879+0000","last_unstale":"2024-09-10T21:11:13.239511+0000","last_undegraded":"2024-09-10T21:11:13.239511+0000","last_fullsized":"2024-09-10T21:11:13.239511+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:42.047609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.238484+0000","last_change":"2024-09-10T21:11:13.228537+0000","last_active":"2024-09-10T21:11:13.238484+0000","last_peered":"2024-09-10T21:11:13.238484+0000","last_clean":"2024-09-10T21:11:13.238484+0000","last_became_active":"2024-09-10T21:11:13.225545+0000","last_became_peered":"2024-09-10T21:11:13.225545+0000","last_unstale":"2024-09-10T21:11:13.238484+0000","last_undegraded":"2024-09-10T21:11:13.238484+0000","last_fullsized":"2024-09-10T21:11:13.238484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:07.902864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.211200+0000","last_change":"2024-09-10T21:11:12.211200+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.211200+0000","last_undegraded":"2024-09-10T21:11:12.211200+0000","last_fullsized":"2024-09-10T21:11:12.211200+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.209879+0000","last_change":"2024-09-10T21:11:12.209879+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.209879+0000","last_undegraded":"2024-09-10T21:11:12.209879+0000","last_fullsized":"2024-09-10T21:11:12.209879+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247720+0000","last_change":"2024-09-10T21:11:13.241250+0000","last_active":"2024-09-10T21:11:13.247720+0000","last_peered":"2024-09-10T21:11:13.247720+0000","last_clean":"2024-09-10T21:11:13.247720+0000","last_became_active":"2024-09-10T21:11:13.240973+0000","last_became_peered":"2024-09-10T21:11:13.240973+0000","last_unstale":"2024-09-10T21:11:13.247720+0000","last_undegraded":"2024-09-10T21:11:13.247720+0000","last_fullsized":"2024-09-10T21:11:13.247720+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:22.307668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.240111+0000","last_change":"2024-09-10T21:11:13.236702+0000","last_active":"2024-09-10T21:11:13.240111+0000","last_peered":"2024-09-10T21:11:13.240111+0000","last_clean":"2024-09-10T21:11:13.240111+0000","last_became_active":"2024-09-10T21:11:13.236376+0000","last_became_peered":"2024-09-10T21:11:13.236376+0000","last_unstale":"2024-09-10T21:11:13.240111+0000","last_undegraded":"2024-09-10T21:11:13.240111+0000","last_fullsized":"2024-09-10T21:11:13.240111+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:42.462690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225606+0000","last_change":"2024-09-10T21:11:13.219569+0000","last_active":"2024-09-10T21:11:13.225606+0000","last_peered":"2024-09-10T21:11:13.225606+0000","last_clean":"2024-09-10T21:11:13.225606+0000","last_became_active":"2024-09-10T21:11:13.218463+0000","last_became_peered":"2024-09-10T21:11:13.218463+0000","last_unstale":"2024-09-10T21:11:13.225606+0000","last_undegraded":"2024-09-10T21:11:13.225606+0000","last_fullsized":"2024-09-10T21:11:13.225606+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:12.585123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247420+0000","last_change":"2024-09-10T21:11:13.242608+0000","last_active":"2024-09-10T21:11:13.247420+0000","last_peered":"2024-09-10T21:11:13.247420+0000","last_clean":"2024-09-10T21:11:13.247420+0000","last_became_active":"2024-09-10T21:11:13.241793+0000","last_became_peered":"2024-09-10T21:11:13.241793+0000","last_unstale":"2024-09-10T21:11:13.247420+0000","last_undegraded":"2024-09-10T21:11:13.247420+0000","last_fullsized":"2024-09-10T21:11:13.247420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:01.678771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245420+0000","last_change":"2024-09-10T21:11:13.240633+0000","last_active":"2024-09-10T21:11:13.245420+0000","last_peered":"2024-09-10T21:11:13.245420+0000","last_clean":"2024-09-10T21:11:13.245420+0000","last_became_active":"2024-09-10T21:11:13.240046+0000","last_became_peered":"2024-09-10T21:11:13.240046+0000","last_unstale":"2024-09-10T21:11:13.245420+0000","last_undegraded":"2024-09-10T21:11:13.245420+0000","last_fullsized":"2024-09-10T21:11:13.245420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:30.065234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.237948+0000","last_change":"2024-09-10T21:11:13.219704+0000","last_active":"2024-09-10T21:11:13.237948+0000","last_peered":"2024-09-10T21:11:13.237948+0000","last_clean":"2024-09-10T21:11:13.237948+0000","last_became_active":"2024-09-10T21:11:13.218621+0000","last_became_peered":"2024-09-10T21:11:13.218621+0000","last_unstale":"2024-09-10T21:11:13.237948+0000","last_undegraded":"2024-09-10T21:11:13.237948+0000","last_fullsized":"2024-09-10T21:11:13.237948+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:54.179687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245811+0000","last_change":"2024-09-10T21:11:13.232802+0000","last_active":"2024-09-10T21:11:13.245811+0000","last_peered":"2024-09-10T21:11:13.245811+0000","last_clean":"2024-09-10T21:11:13.245811+0000","last_became_active":"2024-09-10T21:11:13.227485+0000","last_became_peered":"2024-09-10T21:11:13.227485+0000","last_unstale":"2024-09-10T21:11:13.245811+0000","last_undegraded":"2024-09-10T21:11:13.245811+0000","last_fullsized":"2024-09-10T21:11:13.245811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:51.065370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.246021+0000","last_change":"2024-09-10T21:11:13.239036+0000","last_active":"2024-09-10T21:11:13.246021+0000","last_peered":"2024-09-10T21:11:13.246021+0000","last_clean":"2024-09-10T21:11:13.246021+0000","last_became_active":"2024-09-10T21:11:13.238795+0000","last_became_peered":"2024-09-10T21:11:13.238795+0000","last_unstale":"2024-09-10T21:11:13.246021+0000","last_undegraded":"2024-09-10T21:11:13.246021+0000","last_fullsized":"2024-09-10T21:11:13.246021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:32.087701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.243288+0000","last_change":"2024-09-10T21:11:13.235338+0000","last_active":"2024-09-10T21:11:13.243288+0000","last_peered":"2024-09-10T21:11:13.243288+0000","last_clean":"2024-09-10T21:11:13.243288+0000","last_became_active":"2024-09-10T21:11:13.231675+0000","last_became_peered":"2024-09-10T21:11:13.231675+0000","last_unstale":"2024-09-10T21:11:13.243288+0000","last_undegraded":"2024-09-10T21:11:13.243288+0000","last_fullsized":"2024-09-10T21:11:13.243288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:42.770316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.252762+0000","last_change":"2024-09-10T21:11:13.249311+0000","last_active":"2024-09-10T21:11:13.252762+0000","last_peered":"2024-09-10T21:11:13.252762+0000","last_clean":"2024-09-10T21:11:13.252762+0000","last_became_active":"2024-09-10T21:11:13.248813+0000","last_became_peered":"2024-09-10T21:11:13.248813+0000","last_unstale":"2024-09-10T21:11:13.252762+0000","last_undegraded":"2024-09-10T21:11:13.252762+0000","last_fullsized":"2024-09-10T21:11:13.252762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:18.708305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.249039+0000","last_change":"2024-09-10T21:11:13.248118+0000","last_active":"2024-09-10T21:11:13.249039+0000","last_peered":"2024-09-10T21:11:13.249039+0000","last_clean":"2024-09-10T21:11:13.249039+0000","last_became_active":"2024-09-10T21:11:13.247828+0000","last_became_peered":"2024-09-10T21:11:13.247828+0000","last_unstale":"2024-09-10T21:11:13.249039+0000","last_undegraded":"2024-09-10T21:11:13.249039+0000","last_fullsized":"2024-09-10T21:11:13.249039+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:30.684628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.234753+0000","last_change":"2024-09-10T21:11:13.231198+0000","last_active":"2024-09-10T21:11:13.234753+0000","last_peered":"2024-09-10T21:11:13.234753+0000","last_clean":"2024-09-10T21:11:13.234753+0000","last_became_active":"2024-09-10T21:11:13.229553+0000","last_became_peered":"2024-09-10T21:11:13.229553+0000","last_unstale":"2024-09-10T21:11:13.234753+0000","last_undegraded":"2024-09-10T21:11:13.234753+0000","last_fullsized":"2024-09-10T21:11:13.234753+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:40.266888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.222617+0000","last_change":"2024-09-10T21:11:13.220392+0000","last_active":"2024-09-10T21:11:13.222617+0000","last_peered":"2024-09-10T21:11:13.222617+0000","last_clean":"2024-09-10T21:11:13.222617+0000","last_became_active":"2024-09-10T21:11:13.220117+0000","last_became_peered":"2024-09-10T21:11:13.220117+0000","last_unstale":"2024-09-10T21:11:13.222617+0000","last_undegraded":"2024-09-10T21:11:13.222617+0000","last_fullsized":"2024-09-10T21:11:13.222617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:31.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.234865+0000","last_change":"2024-09-10T21:11:13.224627+0000","last_active":"2024-09-10T21:11:13.234865+0000","last_peered":"2024-09-10T21:11:13.234865+0000","last_clean":"2024-09-10T21:11:13.234865+0000","last_became_active":"2024-09-10T21:11:13.224165+0000","last_became_peered":"2024-09-10T21:11:13.224165+0000","last_unstale":"2024-09-10T21:11:13.234865+0000","last_undegraded":"2024-09-10T21:11:13.234865+0000","last_fullsized":"2024-09-10T21:11:13.234865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:49.105483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.256451+0000","last_change":"2024-09-10T21:11:12.225366+0000","last_active":"2024-09-10T21:11:13.256451+0000","last_peered":"2024-09-10T21:11:13.256451+0000","last_clean":"2024-09-10T21:11:13.256451+0000","last_became_active":"2024-09-10T21:11:12.223302+0000","last_became_peered":"2024-09-10T21:11:12.223302+0000","last_unstale":"2024-09-10T21:11:13.256451+0000","last_undegraded":"2024-09-10T21:11:13.256451+0000","last_fullsized":"2024-09-10T21:11:13.256451+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:27.847102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225535+0000","last_change":"2024-09-10T21:11:13.221051+0000","last_active":"2024-09-10T21:11:13.225535+0000","last_peered":"2024-09-10T21:11:13.225535+0000","last_clean":"2024-09-10T21:11:13.225535+0000","last_became_active":"2024-09-10T21:11:13.220762+0000","last_became_peered":"2024-09-10T21:11:13.220762+0000","last_unstale":"2024-09-10T21:11:13.225535+0000","last_undegraded":"2024-09-10T21:11:13.225535+0000","last_fullsized":"2024-09-10T21:11:13.225535+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:09.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248834+0000","last_change":"2024-09-10T21:11:12.222288+0000","last_active":"2024-09-10T21:11:13.248834+0000","last_peered":"2024-09-10T21:11:13.248834+0000","last_clean":"2024-09-10T21:11:13.248834+0000","last_became_active":"2024-09-10T21:11:12.221928+0000","last_became_peered":"2024-09-10T21:11:12.221928+0000","last_unstale":"2024-09-10T21:11:13.248834+0000","last_undegraded":"2024-09-10T21:11:13.248834+0000","last_fullsized":"2024-09-10T21:11:13.248834+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:35.884414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225276+0000","last_change":"2024-09-10T21:11:13.219137+0000","last_active":"2024-09-10T21:11:13.225276+0000","last_peered":"2024-09-10T21:11:13.225276+0000","last_clean":"2024-09-10T21:11:13.225276+0000","last_became_active":"2024-09-10T21:11:13.218827+0000","last_became_peered":"2024-09-10T21:11:13.218827+0000","last_unstale":"2024-09-10T21:11:13.225276+0000","last_undegraded":"2024-09-10T21:11:13.225276+0000","last_fullsized":"2024-09-10T21:11:13.225276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:47.276393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.228590+0000","last_change":"2024-09-10T21:11:12.223630+0000","last_active":"2024-09-10T21:11:12.228590+0000","last_peered":"2024-09-10T21:11:12.228590+0000","last_clean":"2024-09-10T21:11:12.228590+0000","last_became_active":"2024-09-10T21:11:12.223352+0000","last_became_peered":"2024-09-10T21:11:12.223352+0000","last_unstale":"2024-09-10T21:11:12.228590+0000","last_undegraded":"2024-09-10T21:11:12.228590+0000","last_fullsized":"2024-09-10T21:11:12.228590+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:20.065424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.249650+0000","last_change":"2024-09-10T21:11:13.236750+0000","last_active":"2024-09-10T21:11:13.249650+0000","last_peered":"2024-09-10T21:11:13.249650+0000","last_clean":"2024-09-10T21:11:13.249650+0000","last_became_active":"2024-09-10T21:11:13.236417+0000","last_became_peered":"2024-09-10T21:11:13.236417+0000","last_unstale":"2024-09-10T21:11:13.249650+0000","last_undegraded":"2024-09-10T21:11:13.249650+0000","last_fullsized":"2024-09-10T21:11:13.249650+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:30.284239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.250861+0000","last_change":"2024-09-10T21:11:12.223783+0000","last_active":"2024-09-10T21:11:13.250861+0000","last_peered":"2024-09-10T21:11:13.250861+0000","last_clean":"2024-09-10T21:11:13.250861+0000","last_became_active":"2024-09-10T21:11:12.223509+0000","last_became_peered":"2024-09-10T21:11:12.223509+0000","last_unstale":"2024-09-10T21:11:13.250861+0000","last_undegraded":"2024-09-10T21:11:13.250861+0000","last_fullsized":"2024-09-10T21:11:13.250861+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:14.818515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.205623+0000","last_change":"2024-09-10T21:11:12.205623+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.205623+0000","last_undegraded":"2024-09-10T21:11:12.205623+0000","last_fullsized":"2024-09-10T21:11:12.205623+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247537+0000","last_change":"2024-09-10T21:11:12.212357+0000","last_active":"2024-09-10T21:11:13.247537+0000","last_peered":"2024-09-10T21:11:13.247537+0000","last_clean":"2024-09-10T21:11:13.247537+0000","last_became_active":"2024-09-10T21:11:12.209827+0000","last_became_peered":"2024-09-10T21:11:12.209827+0000","last_unstale":"2024-09-10T21:11:13.247537+0000","last_undegraded":"2024-09-10T21:11:13.247537+0000","last_fullsized":"2024-09-10T21:11:13.247537+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:38.717269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225721+0000","last_change":"2024-09-10T21:11:13.222468+0000","last_active":"2024-09-10T21:11:13.225721+0000","last_peered":"2024-09-10T21:11:13.225721+0000","last_clean":"2024-09-10T21:11:13.225721+0000","last_became_active":"2024-09-10T21:11:13.221546+0000","last_became_peered":"2024-09-10T21:11:13.221546+0000","last_unstale":"2024-09-10T21:11:13.225721+0000","last_undegraded":"2024-09-10T21:11:13.225721+0000","last_fullsized":"2024-09-10T21:11:13.225721+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:03.734116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.251808+0000","last_change":"2024-09-10T21:11:12.225033+0000","last_active":"2024-09-10T21:11:13.251808+0000","last_peered":"2024-09-10T21:11:13.251808+0000","last_clean":"2024-09-10T21:11:13.251808+0000","last_became_active":"2024-09-10T21:11:12.224007+0000","last_became_peered":"2024-09-10T21:11:12.224007+0000","last_unstale":"2024-09-10T21:11:13.251808+0000","last_undegraded":"2024-09-10T21:11:13.251808+0000","last_fullsized":"2024-09-10T21:11:13.251808+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:37.774571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.205574+0000","last_change":"2024-09-10T21:11:12.205574+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.205574+0000","last_undegraded":"2024-09-10T21:11:12.205574+0000","last_fullsized":"2024-09-10T21:11:12.205574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.227968+0000","last_change":"2024-09-10T21:11:12.221685+0000","last_active":"2024-09-10T21:11:12.227968+0000","last_peered":"2024-09-10T21:11:12.227968+0000","last_clean":"2024-09-10T21:11:12.227968+0000","last_became_active":"2024-09-10T21:11:12.221393+0000","last_became_peered":"2024-09-10T21:11:12.221393+0000","last_unstale":"2024-09-10T21:11:12.227968+0000","last_undegraded":"2024-09-10T21:11:12.227968+0000","last_fullsized":"2024-09-10T21:11:12.227968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:09.242206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248043+0000","last_change":"2024-09-10T21:11:13.234906+0000","last_active":"2024-09-10T21:11:13.248043+0000","last_peered":"2024-09-10T21:11:13.248043+0000","last_clean":"2024-09-10T21:11:13.248043+0000","last_became_active":"2024-09-10T21:11:13.228823+0000","last_became_peered":"2024-09-10T21:11:13.228823+0000","last_unstale":"2024-09-10T21:11:13.248043+0000","last_undegraded":"2024-09-10T21:11:13.248043+0000","last_fullsized":"2024-09-10T21:11:13.248043+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:36.486425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.209235+0000","last_change":"2024-09-10T21:11:12.224827+0000","last_active":"2024-09-10T21:11:13.209235+0000","last_peered":"2024-09-10T21:11:13.209235+0000","last_clean":"2024-09-10T21:11:13.209235+0000","last_became_active":"2024-09-10T21:11:12.224356+0000","last_became_peered":"2024-09-10T21:11:12.224356+0000","last_unstale":"2024-09-10T21:11:13.209235+0000","last_undegraded":"2024-09-10T21:11:13.209235+0000","last_fullsized":"2024-09-10T21:11:13.209235+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:00.054106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.252576+0000","last_change":"2024-09-10T21:11:13.233922+0000","last_active":"2024-09-10T21:11:13.252576+0000","last_peered":"2024-09-10T21:11:13.252576+0000","last_clean":"2024-09-10T21:11:13.252576+0000","last_became_active":"2024-09-10T21:11:13.230732+0000","last_became_peered":"2024-09-10T21:11:13.230732+0000","last_unstale":"2024-09-10T21:11:13.252576+0000","last_undegraded":"2024-09-10T21:11:13.252576+0000","last_fullsized":"2024-09-10T21:11:13.252576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:07.175259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.213442+0000","last_change":"2024-09-10T21:11:12.221271+0000","last_active":"2024-09-10T21:11:13.213442+0000","last_peered":"2024-09-10T21:11:13.213442+0000","last_clean":"2024-09-10T21:11:13.213442+0000","last_became_active":"2024-09-10T21:11:12.220975+0000","last_became_peered":"2024-09-10T21:11:12.220975+0000","last_unstale":"2024-09-10T21:11:13.213442+0000","last_undegraded":"2024-09-10T21:11:13.213442+0000","last_fullsized":"2024-09-10T21:11:13.213442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:29.280746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247971+0000","last_change":"2024-09-10T21:11:13.234418+0000","last_active":"2024-09-10T21:11:13.247971+0000","last_peered":"2024-09-10T21:11:13.247971+0000","last_clean":"2024-09-10T21:11:13.247971+0000","last_became_active":"2024-09-10T21:11:13.234159+0000","last_became_peered":"2024-09-10T21:11:13.234159+0000","last_unstale":"2024-09-10T21:11:13.247971+0000","last_undegraded":"2024-09-10T21:11:13.247971+0000","last_fullsized":"2024-09-10T21:11:13.247971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:33.043662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.211318+0000","last_change":"2024-09-10T21:11:12.222257+0000","last_active":"2024-09-10T21:11:13.211318+0000","last_peered":"2024-09-10T21:11:13.211318+0000","last_clean":"2024-09-10T21:11:13.211318+0000","last_became_active":"2024-09-10T21:11:12.221959+0000","last_became_peered":"2024-09-10T21:11:12.221959+0000","last_unstale":"2024-09-10T21:11:13.211318+0000","last_undegraded":"2024-09-10T21:11:13.211318+0000","last_fullsized":"2024-09-10T21:11:13.211318+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:35.482932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.240430+0000","last_change":"2024-09-10T21:11:13.220496+0000","last_active":"2024-09-10T21:11:13.240430+0000","last_peered":"2024-09-10T21:11:13.240430+0000","last_clean":"2024-09-10T21:11:13.240430+0000","last_became_active":"2024-09-10T21:11:13.218857+0000","last_became_peered":"2024-09-10T21:11:13.218857+0000","last_unstale":"2024-09-10T21:11:13.240430+0000","last_undegraded":"2024-09-10T21:11:13.240430+0000","last_fullsized":"2024-09-10T21:11:13.240430+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:08.219546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254368+0000","last_change":"2024-09-10T21:11:12.229925+0000","last_active":"2024-09-10T21:11:13.254368+0000","last_peered":"2024-09-10T21:11:13.254368+0000","last_clean":"2024-09-10T21:11:13.254368+0000","last_became_active":"2024-09-10T21:11:12.229037+0000","last_became_peered":"2024-09-10T21:11:12.229037+0000","last_unstale":"2024-09-10T21:11:13.254368+0000","last_undegraded":"2024-09-10T21:11:13.254368+0000","last_fullsized":"2024-09-10T21:11:13.254368+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:43.936262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.251337+0000","last_change":"2024-09-10T21:11:13.233489+0000","last_active":"2024-09-10T21:11:13.251337+0000","last_peered":"2024-09-10T21:11:13.251337+0000","last_clean":"2024-09-10T21:11:13.251337+0000","last_became_active":"2024-09-10T21:11:13.232909+0000","last_became_peered":"2024-09-10T21:11:13.232909+0000","last_unstale":"2024-09-10T21:11:13.251337+0000","last_undegraded":"2024-09-10T21:11:13.251337+0000","last_fullsized":"2024-09-10T21:11:13.251337+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:15.390510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.256092+0000","last_change":"2024-09-10T21:11:12.230626+0000","last_active":"2024-09-10T21:11:13.256092+0000","last_peered":"2024-09-10T21:11:13.256092+0000","last_clean":"2024-09-10T21:11:13.256092+0000","last_became_active":"2024-09-10T21:11:12.230308+0000","last_became_peered":"2024-09-10T21:11:12.230308+0000","last_unstale":"2024-09-10T21:11:13.256092+0000","last_undegraded":"2024-09-10T21:11:13.256092+0000","last_fullsized":"2024-09-10T21:11:13.256092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:09.728148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.250370+0000","last_change":"2024-09-10T21:11:13.242994+0000","last_active":"2024-09-10T21:11:13.250370+0000","last_peered":"2024-09-10T21:11:13.250370+0000","last_clean":"2024-09-10T21:11:13.250370+0000","last_became_active":"2024-09-10T21:11:13.242683+0000","last_became_peered":"2024-09-10T21:11:13.242683+0000","last_unstale":"2024-09-10T21:11:13.250370+0000","last_undegraded":"2024-09-10T21:11:13.250370+0000","last_fullsized":"2024-09-10T21:11:13.250370+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:04.248893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254320+0000","last_change":"2024-09-10T21:11:12.224674+0000","last_active":"2024-09-10T21:11:13.254320+0000","last_peered":"2024-09-10T21:11:13.254320+0000","last_clean":"2024-09-10T21:11:13.254320+0000","last_became_active":"2024-09-10T21:11:12.224325+0000","last_became_peered":"2024-09-10T21:11:12.224325+0000","last_unstale":"2024-09-10T21:11:13.254320+0000","last_undegraded":"2024-09-10T21:11:13.254320+0000","last_fullsized":"2024-09-10T21:11:13.254320+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:06.355342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248957+0000","last_change":"2024-09-10T21:11:13.236816+0000","last_active":"2024-09-10T21:11:13.248957+0000","last_peered":"2024-09-10T21:11:13.248957+0000","last_clean":"2024-09-10T21:11:13.248957+0000","last_became_active":"2024-09-10T21:11:13.236502+0000","last_became_peered":"2024-09-10T21:11:13.236502+0000","last_unstale":"2024-09-10T21:11:13.248957+0000","last_undegraded":"2024-09-10T21:11:13.248957+0000","last_fullsized":"2024-09-10T21:11:13.248957+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:55.941374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.237205+0000","last_change":"2024-09-10T21:11:12.226932+0000","last_active":"2024-09-10T21:11:13.237205+0000","last_peered":"2024-09-10T21:11:13.237205+0000","last_clean":"2024-09-10T21:11:13.237205+0000","last_became_active":"2024-09-10T21:11:12.226622+0000","last_became_peered":"2024-09-10T21:11:12.226622+0000","last_unstale":"2024-09-10T21:11:13.237205+0000","last_undegraded":"2024-09-10T21:11:13.237205+0000","last_fullsized":"2024-09-10T21:11:13.237205+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:15.020787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239479+0000","last_change":"2024-09-10T21:11:13.226536+0000","last_active":"2024-09-10T21:11:13.239479+0000","last_peered":"2024-09-10T21:11:13.239479+0000","last_clean":"2024-09-10T21:11:13.239479+0000","last_became_active":"2024-09-10T21:11:13.226204+0000","last_became_peered":"2024-09-10T21:11:13.226204+0000","last_unstale":"2024-09-10T21:11:13.239479+0000","last_undegraded":"2024-09-10T21:11:13.239479+0000","last_fullsized":"2024-09-10T21:11:13.239479+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:28.355449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.221557+0000","last_change":"2024-09-10T21:11:12.222663+0000","last_active":"2024-09-10T21:11:13.221557+0000","last_peered":"2024-09-10T21:11:13.221557+0000","last_clean":"2024-09-10T21:11:13.221557+0000","last_became_active":"2024-09-10T21:11:12.222491+0000","last_became_peered":"2024-09-10T21:11:12.222491+0000","last_unstale":"2024-09-10T21:11:13.221557+0000","last_undegraded":"2024-09-10T21:11:13.221557+0000","last_fullsized":"2024-09-10T21:11:13.221557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:26.481636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.221362+0000","last_change":"2024-09-10T21:11:13.218025+0000","last_active":"2024-09-10T21:11:13.221362+0000","last_peered":"2024-09-10T21:11:13.221362+0000","last_clean":"2024-09-10T21:11:13.221362+0000","last_became_active":"2024-09-10T21:11:13.217119+0000","last_became_peered":"2024-09-10T21:11:13.217119+0000","last_unstale":"2024-09-10T21:11:13.221362+0000","last_undegraded":"2024-09-10T21:11:13.221362+0000","last_fullsized":"2024-09-10T21:11:13.221362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:42.864605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.229482+0000","last_change":"2024-09-10T21:11:12.225066+0000","last_active":"2024-09-10T21:11:12.229482+0000","last_peered":"2024-09-10T21:11:12.229482+0000","last_clean":"2024-09-10T21:11:12.229482+0000","last_became_active":"2024-09-10T21:11:12.224555+0000","last_became_peered":"2024-09-10T21:11:12.224555+0000","last_unstale":"2024-09-10T21:11:12.229482+0000","last_undegraded":"2024-09-10T21:11:12.229482+0000","last_fullsized":"2024-09-10T21:11:12.229482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:51.343919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.252602+0000","last_change":"2024-09-10T21:11:13.238225+0000","last_active":"2024-09-10T21:11:13.252602+0000","last_peered":"2024-09-10T21:11:13.252602+0000","last_clean":"2024-09-10T21:11:13.252602+0000","last_became_active":"2024-09-10T21:11:13.236412+0000","last_became_peered":"2024-09-10T21:11:13.236412+0000","last_unstale":"2024-09-10T21:11:13.252602+0000","last_undegraded":"2024-09-10T21:11:13.252602+0000","last_fullsized":"2024-09-10T21:11:13.252602+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:48.715087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.229803+0000","last_change":"2024-09-10T21:11:12.228489+0000","last_active":"2024-09-10T21:11:12.229803+0000","last_peered":"2024-09-10T21:11:12.229803+0000","last_clean":"2024-09-10T21:11:12.229803+0000","last_became_active":"2024-09-10T21:11:12.228136+0000","last_became_peered":"2024-09-10T21:11:12.228136+0000","last_unstale":"2024-09-10T21:11:12.229803+0000","last_undegraded":"2024-09-10T21:11:12.229803+0000","last_fullsized":"2024-09-10T21:11:12.229803+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:12.504083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.206728+0000","last_change":"2024-09-10T21:11:12.206728+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.206728+0000","last_undegraded":"2024-09-10T21:11:12.206728+0000","last_fullsized":"2024-09-10T21:11:12.206728+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254662+0000","last_change":"2024-09-10T21:11:12.222819+0000","last_active":"2024-09-10T21:11:13.254662+0000","last_peered":"2024-09-10T21:11:13.254662+0000","last_clean":"2024-09-10T21:11:13.254662+0000","last_became_active":"2024-09-10T21:11:12.222499+0000","last_became_peered":"2024-09-10T21:11:12.222499+0000","last_unstale":"2024-09-10T21:11:13.254662+0000","last_undegraded":"2024-09-10T21:11:13.254662+0000","last_fullsized":"2024-09-10T21:11:13.254662+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:11.810704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.229207+0000","last_change":"2024-09-10T21:11:13.227904+0000","last_active":"2024-09-10T21:11:13.229207+0000","last_peered":"2024-09-10T21:11:13.229207+0000","last_clean":"2024-09-10T21:11:13.229207+0000","last_became_active":"2024-09-10T21:11:13.227548+0000","last_became_peered":"2024-09-10T21:11:13.227548+0000","last_unstale":"2024-09-10T21:11:13.229207+0000","last_undegraded":"2024-09-10T21:11:13.229207+0000","last_fullsized":"2024-09-10T21:11:13.229207+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:36.887907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254448+0000","last_change":"2024-09-10T21:11:12.230259+0000","last_active":"2024-09-10T21:11:13.254448+0000","last_peered":"2024-09-10T21:11:13.254448+0000","last_clean":"2024-09-10T21:11:13.254448+0000","last_became_active":"2024-09-10T21:11:12.229526+0000","last_became_peered":"2024-09-10T21:11:12.229526+0000","last_unstale":"2024-09-10T21:11:13.254448+0000","last_undegraded":"2024-09-10T21:11:13.254448+0000","last_fullsized":"2024-09-10T21:11:13.254448+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:26.566827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.209289+0000","last_change":"2024-09-10T21:11:12.209289+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.209289+0000","last_undegraded":"2024-09-10T21:11:12.209289+0000","last_fullsized":"2024-09-10T21:11:12.209289+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.236111+0000","last_change":"2024-09-10T21:11:12.230126+0000","last_active":"2024-09-10T21:11:13.236111+0000","last_peered":"2024-09-10T21:11:13.236111+0000","last_clean":"2024-09-10T21:11:13.236111+0000","last_became_active":"2024-09-10T21:11:12.229860+0000","last_became_peered":"2024-09-10T21:11:12.229860+0000","last_unstale":"2024-09-10T21:11:13.236111+0000","last_undegraded":"2024-09-10T21:11:13.236111+0000","last_fullsized":"2024-09-10T21:11:13.236111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:09.292762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.250343+0000","last_change":"2024-09-10T21:11:13.241293+0000","last_active":"2024-09-10T21:11:13.250343+0000","last_peered":"2024-09-10T21:11:13.250343+0000","last_clean":"2024-09-10T21:11:13.250343+0000","last_became_active":"2024-09-10T21:11:13.240911+0000","last_became_peered":"2024-09-10T21:11:13.240911+0000","last_unstale":"2024-09-10T21:11:13.250343+0000","last_undegraded":"2024-09-10T21:11:13.250343+0000","last_fullsized":"2024-09-10T21:11:13.250343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:48.860100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248317+0000","last_change":"2024-09-10T21:11:12.221176+0000","last_active":"2024-09-10T21:11:13.248317+0000","last_peered":"2024-09-10T21:11:13.248317+0000","last_clean":"2024-09-10T21:11:13.248317+0000","last_became_active":"2024-09-10T21:11:12.220902+0000","last_became_peered":"2024-09-10T21:11:12.220902+0000","last_unstale":"2024-09-10T21:11:13.248317+0000","last_undegraded":"2024-09-10T21:11:13.248317+0000","last_fullsized":"2024-09-10T21:11:13.248317+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:54.937881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.227103+0000","last_change":"2024-09-10T21:11:13.224634+0000","last_active":"2024-09-10T21:11:13.227103+0000","last_peered":"2024-09-10T21:11:13.227103+0000","last_clean":"2024-09-10T21:11:13.227103+0000","last_became_active":"2024-09-10T21:11:13.224333+0000","last_became_peered":"2024-09-10T21:11:13.224333+0000","last_unstale":"2024-09-10T21:11:13.227103+0000","last_undegraded":"2024-09-10T21:11:13.227103+0000","last_fullsized":"2024-09-10T21:11:13.227103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:16.024280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.209741+0000","last_change":"2024-09-10T21:11:12.226970+0000","last_active":"2024-09-10T21:11:13.209741+0000","last_peered":"2024-09-10T21:11:13.209741+0000","last_clean":"2024-09-10T21:11:13.209741+0000","last_became_active":"2024-09-10T21:11:12.226548+0000","last_became_peered":"2024-09-10T21:11:12.226548+0000","last_unstale":"2024-09-10T21:11:13.209741+0000","last_undegraded":"2024-09-10T21:11:13.209741+0000","last_fullsized":"2024-09-10T21:11:13.209741+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:20.427638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.255041+0000","last_change":"2024-09-10T21:11:13.253121+0000","last_active":"2024-09-10T21:11:13.255041+0000","last_peered":"2024-09-10T21:11:13.255041+0000","last_clean":"2024-09-10T21:11:13.255041+0000","last_became_active":"2024-09-10T21:11:13.252833+0000","last_became_peered":"2024-09-10T21:11:13.252833+0000","last_unstale":"2024-09-10T21:11:13.255041+0000","last_undegraded":"2024-09-10T21:11:13.255041+0000","last_fullsized":"2024-09-10T21:11:13.255041+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:19.718594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.220466+0000","last_change":"2024-09-10T21:11:12.215786+0000","last_active":"2024-09-10T21:11:12.220466+0000","last_peered":"2024-09-10T21:11:12.220466+0000","last_clean":"2024-09-10T21:11:12.220466+0000","last_became_active":"2024-09-10T21:11:12.215497+0000","last_became_peered":"2024-09-10T21:11:12.215497+0000","last_unstale":"2024-09-10T21:11:12.220466+0000","last_undegraded":"2024-09-10T21:11:12.220466+0000","last_fullsized":"2024-09-10T21:11:12.220466+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:08.359968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.205996+0000","last_change":"2024-09-10T21:11:12.205996+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.205996+0000","last_undegraded":"2024-09-10T21:11:12.205996+0000","last_fullsized":"2024-09-10T21:11:12.205996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.212985+0000","last_change":"2024-09-10T21:11:12.234069+0000","last_active":"2024-09-10T21:11:13.212985+0000","last_peered":"2024-09-10T21:11:13.212985+0000","last_clean":"2024-09-10T21:11:13.212985+0000","last_became_active":"2024-09-10T21:11:12.233714+0000","last_became_peered":"2024-09-10T21:11:12.233714+0000","last_unstale":"2024-09-10T21:11:13.212985+0000","last_undegraded":"2024-09-10T21:11:13.212985+0000","last_fullsized":"2024-09-10T21:11:13.212985+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.570509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.210725+0000","last_change":"2024-09-10T21:11:12.210725+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.210725+0000","last_undegraded":"2024-09-10T21:11:12.210725+0000","last_fullsized":"2024-09-10T21:11:12.210725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.241476+0000","last_change":"2024-09-10T21:11:12.223676+0000","last_active":"2024-09-10T21:11:13.241476+0000","last_peered":"2024-09-10T21:11:13.241476+0000","last_clean":"2024-09-10T21:11:13.241476+0000","last_became_active":"2024-09-10T21:11:12.222053+0000","last_became_peered":"2024-09-10T21:11:12.222053+0000","last_unstale":"2024-09-10T21:11:13.241476+0000","last_undegraded":"2024-09-10T21:11:13.241476+0000","last_fullsized":"2024-09-10T21:11:13.241476+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:51.453841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.228362+0000","last_change":"2024-09-10T21:11:13.223973+0000","last_active":"2024-09-10T21:11:13.228362+0000","last_peered":"2024-09-10T21:11:13.228362+0000","last_clean":"2024-09-10T21:11:13.228362+0000","last_became_active":"2024-09-10T21:11:13.223701+0000","last_became_peered":"2024-09-10T21:11:13.223701+0000","last_unstale":"2024-09-10T21:11:13.228362+0000","last_undegraded":"2024-09-10T21:11:13.228362+0000","last_fullsized":"2024-09-10T21:11:13.228362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:57.527538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.211786+0000","last_change":"2024-09-10T21:11:12.222106+0000","last_active":"2024-09-10T21:11:13.211786+0000","last_peered":"2024-09-10T21:11:13.211786+0000","last_clean":"2024-09-10T21:11:13.211786+0000","last_became_active":"2024-09-10T21:11:12.221770+0000","last_became_peered":"2024-09-10T21:11:12.221770+0000","last_unstale":"2024-09-10T21:11:13.211786+0000","last_undegraded":"2024-09-10T21:11:13.211786+0000","last_fullsized":"2024-09-10T21:11:13.211786+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:07.999372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.208795+0000","last_change":"2024-09-10T21:11:12.208795+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.208795+0000","last_undegraded":"2024-09-10T21:11:12.208795+0000","last_fullsized":"2024-09-10T21:11:12.208795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.227006+0000","last_change":"2024-09-10T21:11:12.229500+0000","last_active":"2024-09-10T21:11:13.227006+0000","last_peered":"2024-09-10T21:11:13.227006+0000","last_clean":"2024-09-10T21:11:13.227006+0000","last_became_active":"2024-09-10T21:11:12.229152+0000","last_became_peered":"2024-09-10T21:11:12.229152+0000","last_unstale":"2024-09-10T21:11:13.227006+0000","last_undegraded":"2024-09-10T21:11:13.227006+0000","last_fullsized":"2024-09-10T21:11:13.227006+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:11.772150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239465+0000","last_change":"2024-09-10T21:11:13.232669+0000","last_active":"2024-09-10T21:11:13.239465+0000","last_peered":"2024-09-10T21:11:13.239465+0000","last_clean":"2024-09-10T21:11:13.239465+0000","last_became_active":"2024-09-10T21:11:13.232049+0000","last_became_peered":"2024-09-10T21:11:13.232049+0000","last_unstale":"2024-09-10T21:11:13.239465+0000","last_undegraded":"2024-09-10T21:11:13.239465+0000","last_fullsized":"2024-09-10T21:11:13.239465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:57.314451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.231085+0000","last_change":"2024-09-10T21:11:12.224196+0000","last_active":"2024-09-10T21:11:13.231085+0000","last_peered":"2024-09-10T21:11:13.231085+0000","last_clean":"2024-09-10T21:11:13.231085+0000","last_became_active":"2024-09-10T21:11:12.223853+0000","last_became_peered":"2024-09-10T21:11:12.223853+0000","last_unstale":"2024-09-10T21:11:13.231085+0000","last_undegraded":"2024-09-10T21:11:13.231085+0000","last_fullsized":"2024-09-10T21:11:13.231085+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:23.761112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248934+0000","last_change":"2024-09-10T21:11:13.236581+0000","last_active":"2024-09-10T21:11:13.248934+0000","last_peered":"2024-09-10T21:11:13.248934+0000","last_clean":"2024-09-10T21:11:13.248934+0000","last_became_active":"2024-09-10T21:11:13.236310+0000","last_became_peered":"2024-09-10T21:11:13.236310+0000","last_unstale":"2024-09-10T21:11:13.248934+0000","last_undegraded":"2024-09-10T21:11:13.248934+0000","last_fullsized":"2024-09-10T21:11:13.248934+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:09.002865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.220680+0000","last_change":"2024-09-10T21:11:12.224815+0000","last_active":"2024-09-10T21:11:13.220680+0000","last_peered":"2024-09-10T21:11:13.220680+0000","last_clean":"2024-09-10T21:11:13.220680+0000","last_became_active":"2024-09-10T21:11:12.224504+0000","last_became_peered":"2024-09-10T21:11:12.224504+0000","last_unstale":"2024-09-10T21:11:13.220680+0000","last_undegraded":"2024-09-10T21:11:13.220680+0000","last_fullsized":"2024-09-10T21:11:13.220680+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:29.112283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.246966+0000","last_change":"2024-09-10T21:11:13.226608+0000","last_active":"2024-09-10T21:11:13.246966+0000","last_peered":"2024-09-10T21:11:13.246966+0000","last_clean":"2024-09-10T21:11:13.246966+0000","last_became_active":"2024-09-10T21:11:13.226297+0000","last_became_peered":"2024-09-10T21:11:13.226297+0000","last_unstale":"2024-09-10T21:11:13.246966+0000","last_undegraded":"2024-09-10T21:11:13.246966+0000","last_fullsized":"2024-09-10T21:11:13.246966+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:34.526987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.208044+0000","last_change":"2024-09-10T21:11:12.222940+0000","last_active":"2024-09-10T21:11:13.208044+0000","last_peered":"2024-09-10T21:11:13.208044+0000","last_clean":"2024-09-10T21:11:13.208044+0000","last_became_active":"2024-09-10T21:11:12.222474+0000","last_became_peered":"2024-09-10T21:11:12.222474+0000","last_unstale":"2024-09-10T21:11:13.208044+0000","last_undegraded":"2024-09-10T21:11:13.208044+0000","last_fullsized":"2024-09-10T21:11:13.208044+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:32.362391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.237409+0000","last_change":"2024-09-10T21:11:13.228335+0000","last_active":"2024-09-10T21:11:13.237409+0000","last_peered":"2024-09-10T21:11:13.237409+0000","last_clean":"2024-09-10T21:11:13.237409+0000","last_became_active":"2024-09-10T21:11:13.228013+0000","last_became_peered":"2024-09-10T21:11:13.228013+0000","last_unstale":"2024-09-10T21:11:13.237409+0000","last_undegraded":"2024-09-10T21:11:13.237409+0000","last_fullsized":"2024-09-10T21:11:13.237409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:22.229299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.223563+0000","last_change":"2024-09-10T21:11:12.220700+0000","last_active":"2024-09-10T21:11:13.223563+0000","last_peered":"2024-09-10T21:11:13.223563+0000","last_clean":"2024-09-10T21:11:13.223563+0000","last_became_active":"2024-09-10T21:11:12.220364+0000","last_became_peered":"2024-09-10T21:11:12.220364+0000","last_unstale":"2024-09-10T21:11:13.223563+0000","last_undegraded":"2024-09-10T21:11:13.223563+0000","last_fullsized":"2024-09-10T21:11:13.223563+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:30.271305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239565+0000","last_change":"2024-09-10T21:11:13.234881+0000","last_active":"2024-09-10T21:11:13.239565+0000","last_peered":"2024-09-10T21:11:13.239565+0000","last_clean":"2024-09-10T21:11:13.239565+0000","last_became_active":"2024-09-10T21:11:13.234666+0000","last_became_peered":"2024-09-10T21:11:13.234666+0000","last_unstale":"2024-09-10T21:11:13.239565+0000","last_undegraded":"2024-09-10T21:11:13.239565+0000","last_fullsized":"2024-09-10T21:11:13.239565+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:42.843772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.214147+0000","last_change":"2024-09-10T21:11:12.230047+0000","last_active":"2024-09-10T21:11:13.214147+0000","last_peered":"2024-09-10T21:11:13.214147+0000","last_clean":"2024-09-10T21:11:13.214147+0000","last_became_active":"2024-09-10T21:11:12.229249+0000","last_became_peered":"2024-09-10T21:11:12.229249+0000","last_unstale":"2024-09-10T21:11:13.214147+0000","last_undegraded":"2024-09-10T21:11:13.214147+0000","last_fullsized":"2024-09-10T21:11:13.214147+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:59.482848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":127,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.209277+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:11:12.209277+0000","last_peered":"2024-09-10T21:11:12.209277+0000","last_clean":"2024-09-10T21:11:12.209277+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:11:12.209277+0000","last_undegraded":"2024-09-10T21:11:12.209277+0000","last_fullsized":"2024-09-10T21:11:12.209277+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":937984,"data_stored":720037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":356352,"data_stored":130721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":356352,"data_stored":129863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542157,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":909312,"data_stored":706405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:11:17.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:11:17.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.561+0000 7fbf34000640 1 -- 172.21.15.16:0/2841382778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf2c0706f0 msgr2=0x7fbf2c0ca530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.561+0000 7fbf34000640 1 --2- 172.21.15.16:0/2841382778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf2c0706f0 0x7fbf2c0ca530 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009c80 tx=0x7fbf2401cd70 comp rx=0 tx=0).stop 2024-09-10T21:11:17.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.561+0000 7fbf34000640 1 -- 172.21.15.16:0/2841382778 shutdown_connections 2024-09-10T21:11:17.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.561+0000 7fbf34000640 1 --2- 172.21.15.16:0/2841382778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf2c0706f0 0x7fbf2c0ca530 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.561+0000 7fbf34000640 1 --2- 172.21.15.16:0/2841382778 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf2c154f90 0x7fbf2c072d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.561+0000 7fbf34000640 1 -- 172.21.15.16:0/2841382778 >> 172.21.15.16:0/2841382778 conn(0x7fbf2c0cfb20 msgr2=0x7fbf2c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:17.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.561+0000 7fbf34000640 1 -- 172.21.15.16:0/2841382778 shutdown_connections 2024-09-10T21:11:17.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.561+0000 7fbf34000640 1 -- 172.21.15.16:0/2841382778 wait complete. 2024-09-10T21:11:17.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.561+0000 7fbf34000640 1 Processor -- start 2024-09-10T21:11:17.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.561+0000 7fbf34000640 1 -- start start 2024-09-10T21:11:17.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf34000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf2c0706f0 0x7fbf2c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:17.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf34000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf2c154f90 0x7fbf2c0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:17.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf34000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf2c15bc70 0x7fbf2c159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:17.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf34000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c167b00 con 0x7fbf2c154f90 2024-09-10T21:11:17.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf34000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c167980 con 0x7fbf2c15bc70 2024-09-10T21:11:17.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf34000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c167c80 con 0x7fbf2c0706f0 2024-09-10T21:11:17.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf30c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf2c154f90 0x7fbf2c0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:17.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf30c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf2c154f90 0x7fbf2c0c2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46990/0 (socket says 172.21.15.16:46990) 2024-09-10T21:11:17.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf30c00640 1 -- 172.21.15.16:0/2869476785 learned_addr learned my addr 172.21.15.16:0/2869476785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:17.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf31600640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf2c0706f0 0x7fbf2c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:17.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf32000640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf2c15bc70 0x7fbf2c159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:17.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf30c00640 1 -- 172.21.15.16:0/2869476785 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf2c0706f0 msgr2=0x7fbf2c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf30c00640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf2c0706f0 0x7fbf2c0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf30c00640 1 -- 172.21.15.16:0/2869476785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf2c15bc70 msgr2=0x7fbf2c159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf30c00640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf2c15bc70 0x7fbf2c159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf30c00640 1 -- 172.21.15.16:0/2869476785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c14f790 con 0x7fbf2c154f90 2024-09-10T21:11:17.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf30c00640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf2c154f90 0x7fbf2c0c2950 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800b3e0 tx=0x7fbf1800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:17.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf29600640 1 -- 172.21.15.16:0/2869476785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18015020 con 0x7fbf2c154f90 2024-09-10T21:11:17.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf29600640 1 -- 172.21.15.16:0/2869476785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf18011920 con 0x7fbf2c154f90 2024-09-10T21:11:17.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf34000640 1 -- 172.21.15.16:0/2869476785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c14fa70 con 0x7fbf2c154f90 2024-09-10T21:11:17.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf29600640 1 -- 172.21.15.16:0/2869476785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18011db0 con 0x7fbf2c154f90 2024-09-10T21:11:17.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf34000640 1 -- 172.21.15.16:0/2869476785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c0b5550 con 0x7fbf2c154f90 2024-09-10T21:11:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf34000640 1 -- 172.21.15.16:0/2869476785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2c14fcb0 con 0x7fbf2c154f90 2024-09-10T21:11:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.565+0000 7fbf29600640 1 -- 172.21.15.16:0/2869476785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbf1801a400 con 0x7fbf2c154f90 2024-09-10T21:11:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.569+0000 7fbf29600640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbf0c0725e0 0x7fbf0c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:17.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.569+0000 7fbf29600640 1 -- 172.21.15.16:0/2869476785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fbf18093170 con 0x7fbf2c154f90 2024-09-10T21:11:17.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.569+0000 7fbf31600640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbf0c0725e0 0x7fbf0c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:17.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.569+0000 7fbf31600640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbf0c0725e0 0x7fbf0c074aa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000abe0 tx=0x7fbf2000a2d0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.573+0000 7fbf29600640 1 -- 172.21.15.16:0/2869476785 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbf2c14fcb0 con 0x7fbf2c154f90 2024-09-10T21:11:17.657 INFO:tasks.ceph.mds.b.smithi052.stderr:2024-09-10T21:11:17.651+0000 7fb71c600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:11:17.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.741+0000 7fbf34000640 1 -- 172.21.15.16:0/2869476785 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf2c076c50 con 0x7fbf0c0725e0 2024-09-10T21:11:17.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.745+0000 7fbf29600640 1 -- 172.21.15.16:0/2869476785 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250866 (secure 0 0 0) 0x7fbf2c076c50 con 0x7fbf0c0725e0 2024-09-10T21:11:17.755 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:17.757 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:11:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.749+0000 7fbf34000640 1 -- 172.21.15.16:0/2869476785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbf0c0725e0 msgr2=0x7fbf0c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.749+0000 7fbf34000640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbf0c0725e0 0x7fbf0c074aa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000abe0 tx=0x7fbf2000a2d0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.749+0000 7fbf34000640 1 -- 172.21.15.16:0/2869476785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf2c154f90 msgr2=0x7fbf2c0c2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.749+0000 7fbf34000640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf2c154f90 0x7fbf2c0c2950 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800b3e0 tx=0x7fbf1800b8a0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.749+0000 7fbf34000640 1 -- 172.21.15.16:0/2869476785 shutdown_connections 2024-09-10T21:11:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.749+0000 7fbf34000640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbf0c0725e0 0x7fbf0c074aa0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.749+0000 7fbf34000640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf2c15bc70 0x7fbf2c159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.749+0000 7fbf34000640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf2c154f90 0x7fbf2c0c2950 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.749+0000 7fbf34000640 1 --2- 172.21.15.16:0/2869476785 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf2c0706f0 0x7fbf2c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.749+0000 7fbf34000640 1 -- 172.21.15.16:0/2869476785 >> 172.21.15.16:0/2869476785 conn(0x7fbf2c0cfb20 msgr2=0x7fbf2c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:17.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.749+0000 7fbf34000640 1 -- 172.21.15.16:0/2869476785 shutdown_connections 2024-09-10T21:11:17.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.749+0000 7fbf34000640 1 -- 172.21.15.16:0/2869476785 wait complete. 2024-09-10T21:11:17.777 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-10T21:11:17.466029+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":129,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253316,"kb_used_data":3780,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721404,"statfs":{"total":773094113280,"available":772834717696,"internally_reserved":0,"allocated":3870720,"data_stored":2176274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000978"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245694+0000","last_change":"2024-09-10T21:11:13.244223+0000","last_active":"2024-09-10T21:11:13.245694+0000","last_peered":"2024-09-10T21:11:13.245694+0000","last_clean":"2024-09-10T21:11:13.245694+0000","last_became_active":"2024-09-10T21:11:13.243954+0000","last_became_peered":"2024-09-10T21:11:13.243954+0000","last_unstale":"2024-09-10T21:11:13.245694+0000","last_undegraded":"2024-09-10T21:11:13.245694+0000","last_fullsized":"2024-09-10T21:11:13.245694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:23.544274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.249545+0000","last_change":"2024-09-10T21:11:13.247771+0000","last_active":"2024-09-10T21:11:13.249545+0000","last_peered":"2024-09-10T21:11:13.249545+0000","last_clean":"2024-09-10T21:11:13.249545+0000","last_became_active":"2024-09-10T21:11:13.247293+0000","last_became_peered":"2024-09-10T21:11:13.247293+0000","last_unstale":"2024-09-10T21:11:13.249545+0000","last_undegraded":"2024-09-10T21:11:13.249545+0000","last_fullsized":"2024-09-10T21:11:13.249545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:31.442572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.206691+0000","last_change":"2024-09-10T21:11:12.206691+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.206691+0000","last_undegraded":"2024-09-10T21:11:12.206691+0000","last_fullsized":"2024-09-10T21:11:12.206691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248003+0000","last_change":"2024-09-10T21:11:13.233899+0000","last_active":"2024-09-10T21:11:13.248003+0000","last_peered":"2024-09-10T21:11:13.248003+0000","last_clean":"2024-09-10T21:11:13.248003+0000","last_became_active":"2024-09-10T21:11:13.219250+0000","last_became_peered":"2024-09-10T21:11:13.219250+0000","last_unstale":"2024-09-10T21:11:13.248003+0000","last_undegraded":"2024-09-10T21:11:13.248003+0000","last_fullsized":"2024-09-10T21:11:13.248003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:51.841630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245745+0000","last_change":"2024-09-10T21:11:13.235979+0000","last_active":"2024-09-10T21:11:13.245745+0000","last_peered":"2024-09-10T21:11:13.245745+0000","last_clean":"2024-09-10T21:11:13.245745+0000","last_became_active":"2024-09-10T21:11:13.233126+0000","last_became_peered":"2024-09-10T21:11:13.233126+0000","last_unstale":"2024-09-10T21:11:13.245745+0000","last_undegraded":"2024-09-10T21:11:13.245745+0000","last_fullsized":"2024-09-10T21:11:13.245745+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:15.822007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.209212+0000","last_change":"2024-09-10T21:11:12.209212+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.209212+0000","last_undegraded":"2024-09-10T21:11:12.209212+0000","last_fullsized":"2024-09-10T21:11:12.209212+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.234785+0000","last_change":"2024-09-10T21:11:13.232062+0000","last_active":"2024-09-10T21:11:13.234785+0000","last_peered":"2024-09-10T21:11:13.234785+0000","last_clean":"2024-09-10T21:11:13.234785+0000","last_became_active":"2024-09-10T21:11:13.231580+0000","last_became_peered":"2024-09-10T21:11:13.231580+0000","last_unstale":"2024-09-10T21:11:13.234785+0000","last_undegraded":"2024-09-10T21:11:13.234785+0000","last_fullsized":"2024-09-10T21:11:13.234785+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:12.602121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245385+0000","last_change":"2024-09-10T21:11:13.240369+0000","last_active":"2024-09-10T21:11:13.245385+0000","last_peered":"2024-09-10T21:11:13.245385+0000","last_clean":"2024-09-10T21:11:13.245385+0000","last_became_active":"2024-09-10T21:11:13.239717+0000","last_became_peered":"2024-09-10T21:11:13.239717+0000","last_unstale":"2024-09-10T21:11:13.245385+0000","last_undegraded":"2024-09-10T21:11:13.245385+0000","last_fullsized":"2024-09-10T21:11:13.245385+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:21.291610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.207224+0000","last_change":"2024-09-10T21:11:12.207224+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.207224+0000","last_undegraded":"2024-09-10T21:11:12.207224+0000","last_fullsized":"2024-09-10T21:11:12.207224+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247504+0000","last_change":"2024-09-10T21:11:13.233269+0000","last_active":"2024-09-10T21:11:13.247504+0000","last_peered":"2024-09-10T21:11:13.247504+0000","last_clean":"2024-09-10T21:11:13.247504+0000","last_became_active":"2024-09-10T21:11:13.228592+0000","last_became_peered":"2024-09-10T21:11:13.228592+0000","last_unstale":"2024-09-10T21:11:13.247504+0000","last_undegraded":"2024-09-10T21:11:13.247504+0000","last_fullsized":"2024-09-10T21:11:13.247504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:10.238782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.251364+0000","last_change":"2024-09-10T21:11:13.233758+0000","last_active":"2024-09-10T21:11:13.251364+0000","last_peered":"2024-09-10T21:11:13.251364+0000","last_clean":"2024-09-10T21:11:13.251364+0000","last_became_active":"2024-09-10T21:11:13.233244+0000","last_became_peered":"2024-09-10T21:11:13.233244+0000","last_unstale":"2024-09-10T21:11:13.251364+0000","last_undegraded":"2024-09-10T21:11:13.251364+0000","last_fullsized":"2024-09-10T21:11:13.251364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:53.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.207305+0000","last_change":"2024-09-10T21:11:12.207305+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.207305+0000","last_undegraded":"2024-09-10T21:11:12.207305+0000","last_fullsized":"2024-09-10T21:11:12.207305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.253665+0000","last_change":"2024-09-10T21:11:13.250940+0000","last_active":"2024-09-10T21:11:13.253665+0000","last_peered":"2024-09-10T21:11:13.253665+0000","last_clean":"2024-09-10T21:11:13.253665+0000","last_became_active":"2024-09-10T21:11:13.250694+0000","last_became_peered":"2024-09-10T21:11:13.250694+0000","last_unstale":"2024-09-10T21:11:13.253665+0000","last_undegraded":"2024-09-10T21:11:13.253665+0000","last_fullsized":"2024-09-10T21:11:13.253665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:33.344003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.228510+0000","last_change":"2024-09-10T21:11:13.216431+0000","last_active":"2024-09-10T21:11:13.228510+0000","last_peered":"2024-09-10T21:11:13.228510+0000","last_clean":"2024-09-10T21:11:13.228510+0000","last_became_active":"2024-09-10T21:11:13.216126+0000","last_became_peered":"2024-09-10T21:11:13.216126+0000","last_unstale":"2024-09-10T21:11:13.228510+0000","last_undegraded":"2024-09-10T21:11:13.228510+0000","last_fullsized":"2024-09-10T21:11:13.228510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:27.677406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.229218+0000","last_change":"2024-09-10T21:11:13.227625+0000","last_active":"2024-09-10T21:11:13.229218+0000","last_peered":"2024-09-10T21:11:13.229218+0000","last_clean":"2024-09-10T21:11:13.229218+0000","last_became_active":"2024-09-10T21:11:13.227357+0000","last_became_peered":"2024-09-10T21:11:13.227357+0000","last_unstale":"2024-09-10T21:11:13.229218+0000","last_undegraded":"2024-09-10T21:11:13.229218+0000","last_fullsized":"2024-09-10T21:11:13.229218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:04.567850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239511+0000","last_change":"2024-09-10T21:11:13.233228+0000","last_active":"2024-09-10T21:11:13.239511+0000","last_peered":"2024-09-10T21:11:13.239511+0000","last_clean":"2024-09-10T21:11:13.239511+0000","last_became_active":"2024-09-10T21:11:13.232879+0000","last_became_peered":"2024-09-10T21:11:13.232879+0000","last_unstale":"2024-09-10T21:11:13.239511+0000","last_undegraded":"2024-09-10T21:11:13.239511+0000","last_fullsized":"2024-09-10T21:11:13.239511+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:42.047609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.238484+0000","last_change":"2024-09-10T21:11:13.228537+0000","last_active":"2024-09-10T21:11:13.238484+0000","last_peered":"2024-09-10T21:11:13.238484+0000","last_clean":"2024-09-10T21:11:13.238484+0000","last_became_active":"2024-09-10T21:11:13.225545+0000","last_became_peered":"2024-09-10T21:11:13.225545+0000","last_unstale":"2024-09-10T21:11:13.238484+0000","last_undegraded":"2024-09-10T21:11:13.238484+0000","last_fullsized":"2024-09-10T21:11:13.238484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:07.902864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.211200+0000","last_change":"2024-09-10T21:11:12.211200+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.211200+0000","last_undegraded":"2024-09-10T21:11:12.211200+0000","last_fullsized":"2024-09-10T21:11:12.211200+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.209879+0000","last_change":"2024-09-10T21:11:12.209879+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.209879+0000","last_undegraded":"2024-09-10T21:11:12.209879+0000","last_fullsized":"2024-09-10T21:11:12.209879+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247720+0000","last_change":"2024-09-10T21:11:13.241250+0000","last_active":"2024-09-10T21:11:13.247720+0000","last_peered":"2024-09-10T21:11:13.247720+0000","last_clean":"2024-09-10T21:11:13.247720+0000","last_became_active":"2024-09-10T21:11:13.240973+0000","last_became_peered":"2024-09-10T21:11:13.240973+0000","last_unstale":"2024-09-10T21:11:13.247720+0000","last_undegraded":"2024-09-10T21:11:13.247720+0000","last_fullsized":"2024-09-10T21:11:13.247720+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:22.307668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.240111+0000","last_change":"2024-09-10T21:11:13.236702+0000","last_active":"2024-09-10T21:11:13.240111+0000","last_peered":"2024-09-10T21:11:13.240111+0000","last_clean":"2024-09-10T21:11:13.240111+0000","last_became_active":"2024-09-10T21:11:13.236376+0000","last_became_peered":"2024-09-10T21:11:13.236376+0000","last_unstale":"2024-09-10T21:11:13.240111+0000","last_undegraded":"2024-09-10T21:11:13.240111+0000","last_fullsized":"2024-09-10T21:11:13.240111+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:42.462690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225606+0000","last_change":"2024-09-10T21:11:13.219569+0000","last_active":"2024-09-10T21:11:13.225606+0000","last_peered":"2024-09-10T21:11:13.225606+0000","last_clean":"2024-09-10T21:11:13.225606+0000","last_became_active":"2024-09-10T21:11:13.218463+0000","last_became_peered":"2024-09-10T21:11:13.218463+0000","last_unstale":"2024-09-10T21:11:13.225606+0000","last_undegraded":"2024-09-10T21:11:13.225606+0000","last_fullsized":"2024-09-10T21:11:13.225606+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:12.585123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247420+0000","last_change":"2024-09-10T21:11:13.242608+0000","last_active":"2024-09-10T21:11:13.247420+0000","last_peered":"2024-09-10T21:11:13.247420+0000","last_clean":"2024-09-10T21:11:13.247420+0000","last_became_active":"2024-09-10T21:11:13.241793+0000","last_became_peered":"2024-09-10T21:11:13.241793+0000","last_unstale":"2024-09-10T21:11:13.247420+0000","last_undegraded":"2024-09-10T21:11:13.247420+0000","last_fullsized":"2024-09-10T21:11:13.247420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:01.678771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245420+0000","last_change":"2024-09-10T21:11:13.240633+0000","last_active":"2024-09-10T21:11:13.245420+0000","last_peered":"2024-09-10T21:11:13.245420+0000","last_clean":"2024-09-10T21:11:13.245420+0000","last_became_active":"2024-09-10T21:11:13.240046+0000","last_became_peered":"2024-09-10T21:11:13.240046+0000","last_unstale":"2024-09-10T21:11:13.245420+0000","last_undegraded":"2024-09-10T21:11:13.245420+0000","last_fullsized":"2024-09-10T21:11:13.245420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:30.065234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.237948+0000","last_change":"2024-09-10T21:11:13.219704+0000","last_active":"2024-09-10T21:11:13.237948+0000","last_peered":"2024-09-10T21:11:13.237948+0000","last_clean":"2024-09-10T21:11:13.237948+0000","last_became_active":"2024-09-10T21:11:13.218621+0000","last_became_peered":"2024-09-10T21:11:13.218621+0000","last_unstale":"2024-09-10T21:11:13.237948+0000","last_undegraded":"2024-09-10T21:11:13.237948+0000","last_fullsized":"2024-09-10T21:11:13.237948+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:54.179687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245811+0000","last_change":"2024-09-10T21:11:13.232802+0000","last_active":"2024-09-10T21:11:13.245811+0000","last_peered":"2024-09-10T21:11:13.245811+0000","last_clean":"2024-09-10T21:11:13.245811+0000","last_became_active":"2024-09-10T21:11:13.227485+0000","last_became_peered":"2024-09-10T21:11:13.227485+0000","last_unstale":"2024-09-10T21:11:13.245811+0000","last_undegraded":"2024-09-10T21:11:13.245811+0000","last_fullsized":"2024-09-10T21:11:13.245811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:51.065370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.246021+0000","last_change":"2024-09-10T21:11:13.239036+0000","last_active":"2024-09-10T21:11:13.246021+0000","last_peered":"2024-09-10T21:11:13.246021+0000","last_clean":"2024-09-10T21:11:13.246021+0000","last_became_active":"2024-09-10T21:11:13.238795+0000","last_became_peered":"2024-09-10T21:11:13.238795+0000","last_unstale":"2024-09-10T21:11:13.246021+0000","last_undegraded":"2024-09-10T21:11:13.246021+0000","last_fullsized":"2024-09-10T21:11:13.246021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:32.087701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.243288+0000","last_change":"2024-09-10T21:11:13.235338+0000","last_active":"2024-09-10T21:11:13.243288+0000","last_peered":"2024-09-10T21:11:13.243288+0000","last_clean":"2024-09-10T21:11:13.243288+0000","last_became_active":"2024-09-10T21:11:13.231675+0000","last_became_peered":"2024-09-10T21:11:13.231675+0000","last_unstale":"2024-09-10T21:11:13.243288+0000","last_undegraded":"2024-09-10T21:11:13.243288+0000","last_fullsized":"2024-09-10T21:11:13.243288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:42.770316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.252762+0000","last_change":"2024-09-10T21:11:13.249311+0000","last_active":"2024-09-10T21:11:13.252762+0000","last_peered":"2024-09-10T21:11:13.252762+0000","last_clean":"2024-09-10T21:11:13.252762+0000","last_became_active":"2024-09-10T21:11:13.248813+0000","last_became_peered":"2024-09-10T21:11:13.248813+0000","last_unstale":"2024-09-10T21:11:13.252762+0000","last_undegraded":"2024-09-10T21:11:13.252762+0000","last_fullsized":"2024-09-10T21:11:13.252762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:18.708305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.249039+0000","last_change":"2024-09-10T21:11:13.248118+0000","last_active":"2024-09-10T21:11:13.249039+0000","last_peered":"2024-09-10T21:11:13.249039+0000","last_clean":"2024-09-10T21:11:13.249039+0000","last_became_active":"2024-09-10T21:11:13.247828+0000","last_became_peered":"2024-09-10T21:11:13.247828+0000","last_unstale":"2024-09-10T21:11:13.249039+0000","last_undegraded":"2024-09-10T21:11:13.249039+0000","last_fullsized":"2024-09-10T21:11:13.249039+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:30.684628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.234753+0000","last_change":"2024-09-10T21:11:13.231198+0000","last_active":"2024-09-10T21:11:13.234753+0000","last_peered":"2024-09-10T21:11:13.234753+0000","last_clean":"2024-09-10T21:11:13.234753+0000","last_became_active":"2024-09-10T21:11:13.229553+0000","last_became_peered":"2024-09-10T21:11:13.229553+0000","last_unstale":"2024-09-10T21:11:13.234753+0000","last_undegraded":"2024-09-10T21:11:13.234753+0000","last_fullsized":"2024-09-10T21:11:13.234753+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:40.266888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.222617+0000","last_change":"2024-09-10T21:11:13.220392+0000","last_active":"2024-09-10T21:11:13.222617+0000","last_peered":"2024-09-10T21:11:13.222617+0000","last_clean":"2024-09-10T21:11:13.222617+0000","last_became_active":"2024-09-10T21:11:13.220117+0000","last_became_peered":"2024-09-10T21:11:13.220117+0000","last_unstale":"2024-09-10T21:11:13.222617+0000","last_undegraded":"2024-09-10T21:11:13.222617+0000","last_fullsized":"2024-09-10T21:11:13.222617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:31.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.234865+0000","last_change":"2024-09-10T21:11:13.224627+0000","last_active":"2024-09-10T21:11:13.234865+0000","last_peered":"2024-09-10T21:11:13.234865+0000","last_clean":"2024-09-10T21:11:13.234865+0000","last_became_active":"2024-09-10T21:11:13.224165+0000","last_became_peered":"2024-09-10T21:11:13.224165+0000","last_unstale":"2024-09-10T21:11:13.234865+0000","last_undegraded":"2024-09-10T21:11:13.234865+0000","last_fullsized":"2024-09-10T21:11:13.234865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:49.105483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.256451+0000","last_change":"2024-09-10T21:11:12.225366+0000","last_active":"2024-09-10T21:11:13.256451+0000","last_peered":"2024-09-10T21:11:13.256451+0000","last_clean":"2024-09-10T21:11:13.256451+0000","last_became_active":"2024-09-10T21:11:12.223302+0000","last_became_peered":"2024-09-10T21:11:12.223302+0000","last_unstale":"2024-09-10T21:11:13.256451+0000","last_undegraded":"2024-09-10T21:11:13.256451+0000","last_fullsized":"2024-09-10T21:11:13.256451+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:27.847102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225535+0000","last_change":"2024-09-10T21:11:13.221051+0000","last_active":"2024-09-10T21:11:13.225535+0000","last_peered":"2024-09-10T21:11:13.225535+0000","last_clean":"2024-09-10T21:11:13.225535+0000","last_became_active":"2024-09-10T21:11:13.220762+0000","last_became_peered":"2024-09-10T21:11:13.220762+0000","last_unstale":"2024-09-10T21:11:13.225535+0000","last_undegraded":"2024-09-10T21:11:13.225535+0000","last_fullsized":"2024-09-10T21:11:13.225535+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:09.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248834+0000","last_change":"2024-09-10T21:11:12.222288+0000","last_active":"2024-09-10T21:11:13.248834+0000","last_peered":"2024-09-10T21:11:13.248834+0000","last_clean":"2024-09-10T21:11:13.248834+0000","last_became_active":"2024-09-10T21:11:12.221928+0000","last_became_peered":"2024-09-10T21:11:12.221928+0000","last_unstale":"2024-09-10T21:11:13.248834+0000","last_undegraded":"2024-09-10T21:11:13.248834+0000","last_fullsized":"2024-09-10T21:11:13.248834+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:35.884414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225276+0000","last_change":"2024-09-10T21:11:13.219137+0000","last_active":"2024-09-10T21:11:13.225276+0000","last_peered":"2024-09-10T21:11:13.225276+0000","last_clean":"2024-09-10T21:11:13.225276+0000","last_became_active":"2024-09-10T21:11:13.218827+0000","last_became_peered":"2024-09-10T21:11:13.218827+0000","last_unstale":"2024-09-10T21:11:13.225276+0000","last_undegraded":"2024-09-10T21:11:13.225276+0000","last_fullsized":"2024-09-10T21:11:13.225276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:47.276393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.228590+0000","last_change":"2024-09-10T21:11:12.223630+0000","last_active":"2024-09-10T21:11:12.228590+0000","last_peered":"2024-09-10T21:11:12.228590+0000","last_clean":"2024-09-10T21:11:12.228590+0000","last_became_active":"2024-09-10T21:11:12.223352+0000","last_became_peered":"2024-09-10T21:11:12.223352+0000","last_unstale":"2024-09-10T21:11:12.228590+0000","last_undegraded":"2024-09-10T21:11:12.228590+0000","last_fullsized":"2024-09-10T21:11:12.228590+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:20.065424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.249650+0000","last_change":"2024-09-10T21:11:13.236750+0000","last_active":"2024-09-10T21:11:13.249650+0000","last_peered":"2024-09-10T21:11:13.249650+0000","last_clean":"2024-09-10T21:11:13.249650+0000","last_became_active":"2024-09-10T21:11:13.236417+0000","last_became_peered":"2024-09-10T21:11:13.236417+0000","last_unstale":"2024-09-10T21:11:13.249650+0000","last_undegraded":"2024-09-10T21:11:13.249650+0000","last_fullsized":"2024-09-10T21:11:13.249650+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:30.284239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.250861+0000","last_change":"2024-09-10T21:11:12.223783+0000","last_active":"2024-09-10T21:11:13.250861+0000","last_peered":"2024-09-10T21:11:13.250861+0000","last_clean":"2024-09-10T21:11:13.250861+0000","last_became_active":"2024-09-10T21:11:12.223509+0000","last_became_peered":"2024-09-10T21:11:12.223509+0000","last_unstale":"2024-09-10T21:11:13.250861+0000","last_undegraded":"2024-09-10T21:11:13.250861+0000","last_fullsized":"2024-09-10T21:11:13.250861+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:14.818515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.205623+0000","last_change":"2024-09-10T21:11:12.205623+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.205623+0000","last_undegraded":"2024-09-10T21:11:12.205623+0000","last_fullsized":"2024-09-10T21:11:12.205623+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247537+0000","last_change":"2024-09-10T21:11:12.212357+0000","last_active":"2024-09-10T21:11:13.247537+0000","last_peered":"2024-09-10T21:11:13.247537+0000","last_clean":"2024-09-10T21:11:13.247537+0000","last_became_active":"2024-09-10T21:11:12.209827+0000","last_became_peered":"2024-09-10T21:11:12.209827+0000","last_unstale":"2024-09-10T21:11:13.247537+0000","last_undegraded":"2024-09-10T21:11:13.247537+0000","last_fullsized":"2024-09-10T21:11:13.247537+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:38.717269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225721+0000","last_change":"2024-09-10T21:11:13.222468+0000","last_active":"2024-09-10T21:11:13.225721+0000","last_peered":"2024-09-10T21:11:13.225721+0000","last_clean":"2024-09-10T21:11:13.225721+0000","last_became_active":"2024-09-10T21:11:13.221546+0000","last_became_peered":"2024-09-10T21:11:13.221546+0000","last_unstale":"2024-09-10T21:11:13.225721+0000","last_undegraded":"2024-09-10T21:11:13.225721+0000","last_fullsized":"2024-09-10T21:11:13.225721+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:03.734116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.251808+0000","last_change":"2024-09-10T21:11:12.225033+0000","last_active":"2024-09-10T21:11:13.251808+0000","last_peered":"2024-09-10T21:11:13.251808+0000","last_clean":"2024-09-10T21:11:13.251808+0000","last_became_active":"2024-09-10T21:11:12.224007+0000","last_became_peered":"2024-09-10T21:11:12.224007+0000","last_unstale":"2024-09-10T21:11:13.251808+0000","last_undegraded":"2024-09-10T21:11:13.251808+0000","last_fullsized":"2024-09-10T21:11:13.251808+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:37.774571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.205574+0000","last_change":"2024-09-10T21:11:12.205574+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.205574+0000","last_undegraded":"2024-09-10T21:11:12.205574+0000","last_fullsized":"2024-09-10T21:11:12.205574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.227968+0000","last_change":"2024-09-10T21:11:12.221685+0000","last_active":"2024-09-10T21:11:12.227968+0000","last_peered":"2024-09-10T21:11:12.227968+0000","last_clean":"2024-09-10T21:11:12.227968+0000","last_became_active":"2024-09-10T21:11:12.221393+0000","last_became_peered":"2024-09-10T21:11:12.221393+0000","last_unstale":"2024-09-10T21:11:12.227968+0000","last_undegraded":"2024-09-10T21:11:12.227968+0000","last_fullsized":"2024-09-10T21:11:12.227968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:09.242206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248043+0000","last_change":"2024-09-10T21:11:13.234906+0000","last_active":"2024-09-10T21:11:13.248043+0000","last_peered":"2024-09-10T21:11:13.248043+0000","last_clean":"2024-09-10T21:11:13.248043+0000","last_became_active":"2024-09-10T21:11:13.228823+0000","last_became_peered":"2024-09-10T21:11:13.228823+0000","last_unstale":"2024-09-10T21:11:13.248043+0000","last_undegraded":"2024-09-10T21:11:13.248043+0000","last_fullsized":"2024-09-10T21:11:13.248043+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:36.486425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.209235+0000","last_change":"2024-09-10T21:11:12.224827+0000","last_active":"2024-09-10T21:11:13.209235+0000","last_peered":"2024-09-10T21:11:13.209235+0000","last_clean":"2024-09-10T21:11:13.209235+0000","last_became_active":"2024-09-10T21:11:12.224356+0000","last_became_peered":"2024-09-10T21:11:12.224356+0000","last_unstale":"2024-09-10T21:11:13.209235+0000","last_undegraded":"2024-09-10T21:11:13.209235+0000","last_fullsized":"2024-09-10T21:11:13.209235+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:00.054106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.252576+0000","last_change":"2024-09-10T21:11:13.233922+0000","last_active":"2024-09-10T21:11:13.252576+0000","last_peered":"2024-09-10T21:11:13.252576+0000","last_clean":"2024-09-10T21:11:13.252576+0000","last_became_active":"2024-09-10T21:11:13.230732+0000","last_became_peered":"2024-09-10T21:11:13.230732+0000","last_unstale":"2024-09-10T21:11:13.252576+0000","last_undegraded":"2024-09-10T21:11:13.252576+0000","last_fullsized":"2024-09-10T21:11:13.252576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:07.175259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.213442+0000","last_change":"2024-09-10T21:11:12.221271+0000","last_active":"2024-09-10T21:11:13.213442+0000","last_peered":"2024-09-10T21:11:13.213442+0000","last_clean":"2024-09-10T21:11:13.213442+0000","last_became_active":"2024-09-10T21:11:12.220975+0000","last_became_peered":"2024-09-10T21:11:12.220975+0000","last_unstale":"2024-09-10T21:11:13.213442+0000","last_undegraded":"2024-09-10T21:11:13.213442+0000","last_fullsized":"2024-09-10T21:11:13.213442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:29.280746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247971+0000","last_change":"2024-09-10T21:11:13.234418+0000","last_active":"2024-09-10T21:11:13.247971+0000","last_peered":"2024-09-10T21:11:13.247971+0000","last_clean":"2024-09-10T21:11:13.247971+0000","last_became_active":"2024-09-10T21:11:13.234159+0000","last_became_peered":"2024-09-10T21:11:13.234159+0000","last_unstale":"2024-09-10T21:11:13.247971+0000","last_undegraded":"2024-09-10T21:11:13.247971+0000","last_fullsized":"2024-09-10T21:11:13.247971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:33.043662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.211318+0000","last_change":"2024-09-10T21:11:12.222257+0000","last_active":"2024-09-10T21:11:13.211318+0000","last_peered":"2024-09-10T21:11:13.211318+0000","last_clean":"2024-09-10T21:11:13.211318+0000","last_became_active":"2024-09-10T21:11:12.221959+0000","last_became_peered":"2024-09-10T21:11:12.221959+0000","last_unstale":"2024-09-10T21:11:13.211318+0000","last_undegraded":"2024-09-10T21:11:13.211318+0000","last_fullsized":"2024-09-10T21:11:13.211318+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:35.482932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.240430+0000","last_change":"2024-09-10T21:11:13.220496+0000","last_active":"2024-09-10T21:11:13.240430+0000","last_peered":"2024-09-10T21:11:13.240430+0000","last_clean":"2024-09-10T21:11:13.240430+0000","last_became_active":"2024-09-10T21:11:13.218857+0000","last_became_peered":"2024-09-10T21:11:13.218857+0000","last_unstale":"2024-09-10T21:11:13.240430+0000","last_undegraded":"2024-09-10T21:11:13.240430+0000","last_fullsized":"2024-09-10T21:11:13.240430+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:08.219546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254368+0000","last_change":"2024-09-10T21:11:12.229925+0000","last_active":"2024-09-10T21:11:13.254368+0000","last_peered":"2024-09-10T21:11:13.254368+0000","last_clean":"2024-09-10T21:11:13.254368+0000","last_became_active":"2024-09-10T21:11:12.229037+0000","last_became_peered":"2024-09-10T21:11:12.229037+0000","last_unstale":"2024-09-10T21:11:13.254368+0000","last_undegraded":"2024-09-10T21:11:13.254368+0000","last_fullsized":"2024-09-10T21:11:13.254368+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:43.936262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.251337+0000","last_change":"2024-09-10T21:11:13.233489+0000","last_active":"2024-09-10T21:11:13.251337+0000","last_peered":"2024-09-10T21:11:13.251337+0000","last_clean":"2024-09-10T21:11:13.251337+0000","last_became_active":"2024-09-10T21:11:13.232909+0000","last_became_peered":"2024-09-10T21:11:13.232909+0000","last_unstale":"2024-09-10T21:11:13.251337+0000","last_undegraded":"2024-09-10T21:11:13.251337+0000","last_fullsized":"2024-09-10T21:11:13.251337+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:15.390510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.256092+0000","last_change":"2024-09-10T21:11:12.230626+0000","last_active":"2024-09-10T21:11:13.256092+0000","last_peered":"2024-09-10T21:11:13.256092+0000","last_clean":"2024-09-10T21:11:13.256092+0000","last_became_active":"2024-09-10T21:11:12.230308+0000","last_became_peered":"2024-09-10T21:11:12.230308+0000","last_unstale":"2024-09-10T21:11:13.256092+0000","last_undegraded":"2024-09-10T21:11:13.256092+0000","last_fullsized":"2024-09-10T21:11:13.256092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:09.728148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.250370+0000","last_change":"2024-09-10T21:11:13.242994+0000","last_active":"2024-09-10T21:11:13.250370+0000","last_peered":"2024-09-10T21:11:13.250370+0000","last_clean":"2024-09-10T21:11:13.250370+0000","last_became_active":"2024-09-10T21:11:13.242683+0000","last_became_peered":"2024-09-10T21:11:13.242683+0000","last_unstale":"2024-09-10T21:11:13.250370+0000","last_undegraded":"2024-09-10T21:11:13.250370+0000","last_fullsized":"2024-09-10T21:11:13.250370+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:04.248893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254320+0000","last_change":"2024-09-10T21:11:12.224674+0000","last_active":"2024-09-10T21:11:13.254320+0000","last_peered":"2024-09-10T21:11:13.254320+0000","last_clean":"2024-09-10T21:11:13.254320+0000","last_became_active":"2024-09-10T21:11:12.224325+0000","last_became_peered":"2024-09-10T21:11:12.224325+0000","last_unstale":"2024-09-10T21:11:13.254320+0000","last_undegraded":"2024-09-10T21:11:13.254320+0000","last_fullsized":"2024-09-10T21:11:13.254320+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:06.355342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248957+0000","last_change":"2024-09-10T21:11:13.236816+0000","last_active":"2024-09-10T21:11:13.248957+0000","last_peered":"2024-09-10T21:11:13.248957+0000","last_clean":"2024-09-10T21:11:13.248957+0000","last_became_active":"2024-09-10T21:11:13.236502+0000","last_became_peered":"2024-09-10T21:11:13.236502+0000","last_unstale":"2024-09-10T21:11:13.248957+0000","last_undegraded":"2024-09-10T21:11:13.248957+0000","last_fullsized":"2024-09-10T21:11:13.248957+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:55.941374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.237205+0000","last_change":"2024-09-10T21:11:12.226932+0000","last_active":"2024-09-10T21:11:13.237205+0000","last_peered":"2024-09-10T21:11:13.237205+0000","last_clean":"2024-09-10T21:11:13.237205+0000","last_became_active":"2024-09-10T21:11:12.226622+0000","last_became_peered":"2024-09-10T21:11:12.226622+0000","last_unstale":"2024-09-10T21:11:13.237205+0000","last_undegraded":"2024-09-10T21:11:13.237205+0000","last_fullsized":"2024-09-10T21:11:13.237205+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:15.020787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239479+0000","last_change":"2024-09-10T21:11:13.226536+0000","last_active":"2024-09-10T21:11:13.239479+0000","last_peered":"2024-09-10T21:11:13.239479+0000","last_clean":"2024-09-10T21:11:13.239479+0000","last_became_active":"2024-09-10T21:11:13.226204+0000","last_became_peered":"2024-09-10T21:11:13.226204+0000","last_unstale":"2024-09-10T21:11:13.239479+0000","last_undegraded":"2024-09-10T21:11:13.239479+0000","last_fullsized":"2024-09-10T21:11:13.239479+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:28.355449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.221557+0000","last_change":"2024-09-10T21:11:12.222663+0000","last_active":"2024-09-10T21:11:13.221557+0000","last_peered":"2024-09-10T21:11:13.221557+0000","last_clean":"2024-09-10T21:11:13.221557+0000","last_became_active":"2024-09-10T21:11:12.222491+0000","last_became_peered":"2024-09-10T21:11:12.222491+0000","last_unstale":"2024-09-10T21:11:13.221557+0000","last_undegraded":"2024-09-10T21:11:13.221557+0000","last_fullsized":"2024-09-10T21:11:13.221557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:26.481636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.221362+0000","last_change":"2024-09-10T21:11:13.218025+0000","last_active":"2024-09-10T21:11:13.221362+0000","last_peered":"2024-09-10T21:11:13.221362+0000","last_clean":"2024-09-10T21:11:13.221362+0000","last_became_active":"2024-09-10T21:11:13.217119+0000","last_became_peered":"2024-09-10T21:11:13.217119+0000","last_unstale":"2024-09-10T21:11:13.221362+0000","last_undegraded":"2024-09-10T21:11:13.221362+0000","last_fullsized":"2024-09-10T21:11:13.221362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:42.864605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.229482+0000","last_change":"2024-09-10T21:11:12.225066+0000","last_active":"2024-09-10T21:11:12.229482+0000","last_peered":"2024-09-10T21:11:12.229482+0000","last_clean":"2024-09-10T21:11:12.229482+0000","last_became_active":"2024-09-10T21:11:12.224555+0000","last_became_peered":"2024-09-10T21:11:12.224555+0000","last_unstale":"2024-09-10T21:11:12.229482+0000","last_undegraded":"2024-09-10T21:11:12.229482+0000","last_fullsized":"2024-09-10T21:11:12.229482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:51.343919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.252602+0000","last_change":"2024-09-10T21:11:13.238225+0000","last_active":"2024-09-10T21:11:13.252602+0000","last_peered":"2024-09-10T21:11:13.252602+0000","last_clean":"2024-09-10T21:11:13.252602+0000","last_became_active":"2024-09-10T21:11:13.236412+0000","last_became_peered":"2024-09-10T21:11:13.236412+0000","last_unstale":"2024-09-10T21:11:13.252602+0000","last_undegraded":"2024-09-10T21:11:13.252602+0000","last_fullsized":"2024-09-10T21:11:13.252602+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:48.715087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.229803+0000","last_change":"2024-09-10T21:11:12.228489+0000","last_active":"2024-09-10T21:11:12.229803+0000","last_peered":"2024-09-10T21:11:12.229803+0000","last_clean":"2024-09-10T21:11:12.229803+0000","last_became_active":"2024-09-10T21:11:12.228136+0000","last_became_peered":"2024-09-10T21:11:12.228136+0000","last_unstale":"2024-09-10T21:11:12.229803+0000","last_undegraded":"2024-09-10T21:11:12.229803+0000","last_fullsized":"2024-09-10T21:11:12.229803+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:12.504083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.206728+0000","last_change":"2024-09-10T21:11:12.206728+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.206728+0000","last_undegraded":"2024-09-10T21:11:12.206728+0000","last_fullsized":"2024-09-10T21:11:12.206728+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254662+0000","last_change":"2024-09-10T21:11:12.222819+0000","last_active":"2024-09-10T21:11:13.254662+0000","last_peered":"2024-09-10T21:11:13.254662+0000","last_clean":"2024-09-10T21:11:13.254662+0000","last_became_active":"2024-09-10T21:11:12.222499+0000","last_became_peered":"2024-09-10T21:11:12.222499+0000","last_unstale":"2024-09-10T21:11:13.254662+0000","last_undegraded":"2024-09-10T21:11:13.254662+0000","last_fullsized":"2024-09-10T21:11:13.254662+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:11.810704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.229207+0000","last_change":"2024-09-10T21:11:13.227904+0000","last_active":"2024-09-10T21:11:13.229207+0000","last_peered":"2024-09-10T21:11:13.229207+0000","last_clean":"2024-09-10T21:11:13.229207+0000","last_became_active":"2024-09-10T21:11:13.227548+0000","last_became_peered":"2024-09-10T21:11:13.227548+0000","last_unstale":"2024-09-10T21:11:13.229207+0000","last_undegraded":"2024-09-10T21:11:13.229207+0000","last_fullsized":"2024-09-10T21:11:13.229207+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:36.887907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254448+0000","last_change":"2024-09-10T21:11:12.230259+0000","last_active":"2024-09-10T21:11:13.254448+0000","last_peered":"2024-09-10T21:11:13.254448+0000","last_clean":"2024-09-10T21:11:13.254448+0000","last_became_active":"2024-09-10T21:11:12.229526+0000","last_became_peered":"2024-09-10T21:11:12.229526+0000","last_unstale":"2024-09-10T21:11:13.254448+0000","last_undegraded":"2024-09-10T21:11:13.254448+0000","last_fullsized":"2024-09-10T21:11:13.254448+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:26.566827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.209289+0000","last_change":"2024-09-10T21:11:12.209289+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.209289+0000","last_undegraded":"2024-09-10T21:11:12.209289+0000","last_fullsized":"2024-09-10T21:11:12.209289+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.236111+0000","last_change":"2024-09-10T21:11:12.230126+0000","last_active":"2024-09-10T21:11:13.236111+0000","last_peered":"2024-09-10T21:11:13.236111+0000","last_clean":"2024-09-10T21:11:13.236111+0000","last_became_active":"2024-09-10T21:11:12.229860+0000","last_became_peered":"2024-09-10T21:11:12.229860+0000","last_unstale":"2024-09-10T21:11:13.236111+0000","last_undegraded":"2024-09-10T21:11:13.236111+0000","last_fullsized":"2024-09-10T21:11:13.236111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:09.292762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.250343+0000","last_change":"2024-09-10T21:11:13.241293+0000","last_active":"2024-09-10T21:11:13.250343+0000","last_peered":"2024-09-10T21:11:13.250343+0000","last_clean":"2024-09-10T21:11:13.250343+0000","last_became_active":"2024-09-10T21:11:13.240911+0000","last_became_peered":"2024-09-10T21:11:13.240911+0000","last_unstale":"2024-09-10T21:11:13.250343+0000","last_undegraded":"2024-09-10T21:11:13.250343+0000","last_fullsized":"2024-09-10T21:11:13.250343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:48.860100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248317+0000","last_change":"2024-09-10T21:11:12.221176+0000","last_active":"2024-09-10T21:11:13.248317+0000","last_peered":"2024-09-10T21:11:13.248317+0000","last_clean":"2024-09-10T21:11:13.248317+0000","last_became_active":"2024-09-10T21:11:12.220902+0000","last_became_peered":"2024-09-10T21:11:12.220902+0000","last_unstale":"2024-09-10T21:11:13.248317+0000","last_undegraded":"2024-09-10T21:11:13.248317+0000","last_fullsized":"2024-09-10T21:11:13.248317+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:54.937881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.227103+0000","last_change":"2024-09-10T21:11:13.224634+0000","last_active":"2024-09-10T21:11:13.227103+0000","last_peered":"2024-09-10T21:11:13.227103+0000","last_clean":"2024-09-10T21:11:13.227103+0000","last_became_active":"2024-09-10T21:11:13.224333+0000","last_became_peered":"2024-09-10T21:11:13.224333+0000","last_unstale":"2024-09-10T21:11:13.227103+0000","last_undegraded":"2024-09-10T21:11:13.227103+0000","last_fullsized":"2024-09-10T21:11:13.227103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:16.024280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.209741+0000","last_change":"2024-09-10T21:11:12.226970+0000","last_active":"2024-09-10T21:11:13.209741+0000","last_peered":"2024-09-10T21:11:13.209741+0000","last_clean":"2024-09-10T21:11:13.209741+0000","last_became_active":"2024-09-10T21:11:12.226548+0000","last_became_peered":"2024-09-10T21:11:12.226548+0000","last_unstale":"2024-09-10T21:11:13.209741+0000","last_undegraded":"2024-09-10T21:11:13.209741+0000","last_fullsized":"2024-09-10T21:11:13.209741+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:20.427638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.255041+0000","last_change":"2024-09-10T21:11:13.253121+0000","last_active":"2024-09-10T21:11:13.255041+0000","last_peered":"2024-09-10T21:11:13.255041+0000","last_clean":"2024-09-10T21:11:13.255041+0000","last_became_active":"2024-09-10T21:11:13.252833+0000","last_became_peered":"2024-09-10T21:11:13.252833+0000","last_unstale":"2024-09-10T21:11:13.255041+0000","last_undegraded":"2024-09-10T21:11:13.255041+0000","last_fullsized":"2024-09-10T21:11:13.255041+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:19.718594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.220466+0000","last_change":"2024-09-10T21:11:12.215786+0000","last_active":"2024-09-10T21:11:12.220466+0000","last_peered":"2024-09-10T21:11:12.220466+0000","last_clean":"2024-09-10T21:11:12.220466+0000","last_became_active":"2024-09-10T21:11:12.215497+0000","last_became_peered":"2024-09-10T21:11:12.215497+0000","last_unstale":"2024-09-10T21:11:12.220466+0000","last_undegraded":"2024-09-10T21:11:12.220466+0000","last_fullsized":"2024-09-10T21:11:12.220466+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:08.359968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.205996+0000","last_change":"2024-09-10T21:11:12.205996+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.205996+0000","last_undegraded":"2024-09-10T21:11:12.205996+0000","last_fullsized":"2024-09-10T21:11:12.205996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.212985+0000","last_change":"2024-09-10T21:11:12.234069+0000","last_active":"2024-09-10T21:11:13.212985+0000","last_peered":"2024-09-10T21:11:13.212985+0000","last_clean":"2024-09-10T21:11:13.212985+0000","last_became_active":"2024-09-10T21:11:12.233714+0000","last_became_peered":"2024-09-10T21:11:12.233714+0000","last_unstale":"2024-09-10T21:11:13.212985+0000","last_undegraded":"2024-09-10T21:11:13.212985+0000","last_fullsized":"2024-09-10T21:11:13.212985+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.570509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.210725+0000","last_change":"2024-09-10T21:11:12.210725+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.210725+0000","last_undegraded":"2024-09-10T21:11:12.210725+0000","last_fullsized":"2024-09-10T21:11:12.210725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.241476+0000","last_change":"2024-09-10T21:11:12.223676+0000","last_active":"2024-09-10T21:11:13.241476+0000","last_peered":"2024-09-10T21:11:13.241476+0000","last_clean":"2024-09-10T21:11:13.241476+0000","last_became_active":"2024-09-10T21:11:12.222053+0000","last_became_peered":"2024-09-10T21:11:12.222053+0000","last_unstale":"2024-09-10T21:11:13.241476+0000","last_undegraded":"2024-09-10T21:11:13.241476+0000","last_fullsized":"2024-09-10T21:11:13.241476+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:51.453841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.228362+0000","last_change":"2024-09-10T21:11:13.223973+0000","last_active":"2024-09-10T21:11:13.228362+0000","last_peered":"2024-09-10T21:11:13.228362+0000","last_clean":"2024-09-10T21:11:13.228362+0000","last_became_active":"2024-09-10T21:11:13.223701+0000","last_became_peered":"2024-09-10T21:11:13.223701+0000","last_unstale":"2024-09-10T21:11:13.228362+0000","last_undegraded":"2024-09-10T21:11:13.228362+0000","last_fullsized":"2024-09-10T21:11:13.228362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:57.527538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.211786+0000","last_change":"2024-09-10T21:11:12.222106+0000","last_active":"2024-09-10T21:11:13.211786+0000","last_peered":"2024-09-10T21:11:13.211786+0000","last_clean":"2024-09-10T21:11:13.211786+0000","last_became_active":"2024-09-10T21:11:12.221770+0000","last_became_peered":"2024-09-10T21:11:12.221770+0000","last_unstale":"2024-09-10T21:11:13.211786+0000","last_undegraded":"2024-09-10T21:11:13.211786+0000","last_fullsized":"2024-09-10T21:11:13.211786+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:07.999372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.208795+0000","last_change":"2024-09-10T21:11:12.208795+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.208795+0000","last_undegraded":"2024-09-10T21:11:12.208795+0000","last_fullsized":"2024-09-10T21:11:12.208795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.227006+0000","last_change":"2024-09-10T21:11:12.229500+0000","last_active":"2024-09-10T21:11:13.227006+0000","last_peered":"2024-09-10T21:11:13.227006+0000","last_clean":"2024-09-10T21:11:13.227006+0000","last_became_active":"2024-09-10T21:11:12.229152+0000","last_became_peered":"2024-09-10T21:11:12.229152+0000","last_unstale":"2024-09-10T21:11:13.227006+0000","last_undegraded":"2024-09-10T21:11:13.227006+0000","last_fullsized":"2024-09-10T21:11:13.227006+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:11.772150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239465+0000","last_change":"2024-09-10T21:11:13.232669+0000","last_active":"2024-09-10T21:11:13.239465+0000","last_peered":"2024-09-10T21:11:13.239465+0000","last_clean":"2024-09-10T21:11:13.239465+0000","last_became_active":"2024-09-10T21:11:13.232049+0000","last_became_peered":"2024-09-10T21:11:13.232049+0000","last_unstale":"2024-09-10T21:11:13.239465+0000","last_undegraded":"2024-09-10T21:11:13.239465+0000","last_fullsized":"2024-09-10T21:11:13.239465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:57.314451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.231085+0000","last_change":"2024-09-10T21:11:12.224196+0000","last_active":"2024-09-10T21:11:13.231085+0000","last_peered":"2024-09-10T21:11:13.231085+0000","last_clean":"2024-09-10T21:11:13.231085+0000","last_became_active":"2024-09-10T21:11:12.223853+0000","last_became_peered":"2024-09-10T21:11:12.223853+0000","last_unstale":"2024-09-10T21:11:13.231085+0000","last_undegraded":"2024-09-10T21:11:13.231085+0000","last_fullsized":"2024-09-10T21:11:13.231085+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:23.761112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248934+0000","last_change":"2024-09-10T21:11:13.236581+0000","last_active":"2024-09-10T21:11:13.248934+0000","last_peered":"2024-09-10T21:11:13.248934+0000","last_clean":"2024-09-10T21:11:13.248934+0000","last_became_active":"2024-09-10T21:11:13.236310+0000","last_became_peered":"2024-09-10T21:11:13.236310+0000","last_unstale":"2024-09-10T21:11:13.248934+0000","last_undegraded":"2024-09-10T21:11:13.248934+0000","last_fullsized":"2024-09-10T21:11:13.248934+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:09.002865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.220680+0000","last_change":"2024-09-10T21:11:12.224815+0000","last_active":"2024-09-10T21:11:13.220680+0000","last_peered":"2024-09-10T21:11:13.220680+0000","last_clean":"2024-09-10T21:11:13.220680+0000","last_became_active":"2024-09-10T21:11:12.224504+0000","last_became_peered":"2024-09-10T21:11:12.224504+0000","last_unstale":"2024-09-10T21:11:13.220680+0000","last_undegraded":"2024-09-10T21:11:13.220680+0000","last_fullsized":"2024-09-10T21:11:13.220680+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:29.112283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.246966+0000","last_change":"2024-09-10T21:11:13.226608+0000","last_active":"2024-09-10T21:11:13.246966+0000","last_peered":"2024-09-10T21:11:13.246966+0000","last_clean":"2024-09-10T21:11:13.246966+0000","last_became_active":"2024-09-10T21:11:13.226297+0000","last_became_peered":"2024-09-10T21:11:13.226297+0000","last_unstale":"2024-09-10T21:11:13.246966+0000","last_undegraded":"2024-09-10T21:11:13.246966+0000","last_fullsized":"2024-09-10T21:11:13.246966+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:34.526987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.208044+0000","last_change":"2024-09-10T21:11:12.222940+0000","last_active":"2024-09-10T21:11:13.208044+0000","last_peered":"2024-09-10T21:11:13.208044+0000","last_clean":"2024-09-10T21:11:13.208044+0000","last_became_active":"2024-09-10T21:11:12.222474+0000","last_became_peered":"2024-09-10T21:11:12.222474+0000","last_unstale":"2024-09-10T21:11:13.208044+0000","last_undegraded":"2024-09-10T21:11:13.208044+0000","last_fullsized":"2024-09-10T21:11:13.208044+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:32.362391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.237409+0000","last_change":"2024-09-10T21:11:13.228335+0000","last_active":"2024-09-10T21:11:13.237409+0000","last_peered":"2024-09-10T21:11:13.237409+0000","last_clean":"2024-09-10T21:11:13.237409+0000","last_became_active":"2024-09-10T21:11:13.228013+0000","last_became_peered":"2024-09-10T21:11:13.228013+0000","last_unstale":"2024-09-10T21:11:13.237409+0000","last_undegraded":"2024-09-10T21:11:13.237409+0000","last_fullsized":"2024-09-10T21:11:13.237409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:22.229299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.223563+0000","last_change":"2024-09-10T21:11:12.220700+0000","last_active":"2024-09-10T21:11:13.223563+0000","last_peered":"2024-09-10T21:11:13.223563+0000","last_clean":"2024-09-10T21:11:13.223563+0000","last_became_active":"2024-09-10T21:11:12.220364+0000","last_became_peered":"2024-09-10T21:11:12.220364+0000","last_unstale":"2024-09-10T21:11:13.223563+0000","last_undegraded":"2024-09-10T21:11:13.223563+0000","last_fullsized":"2024-09-10T21:11:13.223563+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:30.271305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239565+0000","last_change":"2024-09-10T21:11:13.234881+0000","last_active":"2024-09-10T21:11:13.239565+0000","last_peered":"2024-09-10T21:11:13.239565+0000","last_clean":"2024-09-10T21:11:13.239565+0000","last_became_active":"2024-09-10T21:11:13.234666+0000","last_became_peered":"2024-09-10T21:11:13.234666+0000","last_unstale":"2024-09-10T21:11:13.239565+0000","last_undegraded":"2024-09-10T21:11:13.239565+0000","last_fullsized":"2024-09-10T21:11:13.239565+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:42.843772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.214147+0000","last_change":"2024-09-10T21:11:12.230047+0000","last_active":"2024-09-10T21:11:13.214147+0000","last_peered":"2024-09-10T21:11:13.214147+0000","last_clean":"2024-09-10T21:11:13.214147+0000","last_became_active":"2024-09-10T21:11:12.229249+0000","last_became_peered":"2024-09-10T21:11:12.229249+0000","last_unstale":"2024-09-10T21:11:13.214147+0000","last_undegraded":"2024-09-10T21:11:13.214147+0000","last_fullsized":"2024-09-10T21:11:13.214147+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:59.482848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":127,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.209277+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:11:12.209277+0000","last_peered":"2024-09-10T21:11:12.209277+0000","last_clean":"2024-09-10T21:11:12.209277+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:11:12.209277+0000","last_undegraded":"2024-09-10T21:11:12.209277+0000","last_fullsized":"2024-09-10T21:11:12.209277+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":937984,"data_stored":720037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":356352,"data_stored":130721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":356352,"data_stored":129863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542157,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":909312,"data_stored":706405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:11:17.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:11:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.957+0000 7fdd53000640 1 -- 172.21.15.16:0/2342683171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4c0b0590 msgr2=0x7fdd4c071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.957+0000 7fdd53000640 1 --2- 172.21.15.16:0/2342683171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4c0b0590 0x7fdd4c071520 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c073290 tx=0x7fdd4401cc60 comp rx=0 tx=0).stop 2024-09-10T21:11:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.957+0000 7fdd53000640 1 -- 172.21.15.16:0/2342683171 shutdown_connections 2024-09-10T21:11:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.957+0000 7fdd53000640 1 --2- 172.21.15.16:0/2342683171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4c0b0590 0x7fdd4c071520 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.957+0000 7fdd53000640 1 --2- 172.21.15.16:0/2342683171 >> v2:172.21.15.52:3300/0 conn(0x7fdd4c15f400 0x7fdd4c070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.957+0000 7fdd53000640 1 -- 172.21.15.16:0/2342683171 >> 172.21.15.16:0/2342683171 conn(0x7fdd4c09b130 msgr2=0x7fdd4c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.957+0000 7fdd53000640 1 -- 172.21.15.16:0/2342683171 shutdown_connections 2024-09-10T21:11:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.957+0000 7fdd53000640 1 -- 172.21.15.16:0/2342683171 wait complete. 2024-09-10T21:11:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd53000640 1 Processor -- start 2024-09-10T21:11:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd53000640 1 -- start start 2024-09-10T21:11:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd53000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4c0b0590 0x7fdd4c152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd53000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdd4c0b13c0 0x7fdd4c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd53000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd4c15f400 0x7fdd4c06f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd53000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c168f60 con 0x7fdd4c0b0590 2024-09-10T21:11:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd53000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c168de0 con 0x7fdd4c15f400 2024-09-10T21:11:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd53000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fdd4c1690e0 con 0x7fdd4c0b13c0 2024-09-10T21:11:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd4be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4c0b0590 0x7fdd4c152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd4be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4c0b0590 0x7fdd4c152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47008/0 (socket says 172.21.15.16:47008) 2024-09-10T21:11:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd4be00640 1 -- 172.21.15.16:0/4203744676 learned_addr learned my addr 172.21.15.16:0/4203744676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd51000640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd4c15f400 0x7fdd4c06f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd43e00640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdd4c0b13c0 0x7fdd4c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd4be00640 1 -- 172.21.15.16:0/4203744676 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdd4c0b13c0 msgr2=0x7fdd4c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd4be00640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdd4c0b13c0 0x7fdd4c06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd4be00640 1 -- 172.21.15.16:0/4203744676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd4c15f400 msgr2=0x7fdd4c06f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd4be00640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd4c15f400 0x7fdd4c06f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd4be00640 1 -- 172.21.15.16:0/4203744676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4c06e3f0 con 0x7fdd4c0b0590 2024-09-10T21:11:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd4be00640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4c0b0590 0x7fdd4c152430 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00d770 tx=0x7fdd3c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd48c00640 1 -- 172.21.15.16:0/4203744676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c009c30 con 0x7fdd4c0b0590 2024-09-10T21:11:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd48c00640 1 -- 172.21.15.16:0/4203744676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd3c00de80 con 0x7fdd4c0b0590 2024-09-10T21:11:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd48c00640 1 -- 172.21.15.16:0/4203744676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c004280 con 0x7fdd4c0b0590 2024-09-10T21:11:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd53000640 1 -- 172.21.15.16:0/4203744676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4c06e6d0 con 0x7fdd4c0b0590 2024-09-10T21:11:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.961+0000 7fdd53000640 1 -- 172.21.15.16:0/4203744676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4c156990 con 0x7fdd4c0b0590 2024-09-10T21:11:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.965+0000 7fdd53000640 1 -- 172.21.15.16:0/4203744676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd4c0cfc80 con 0x7fdd4c0b0590 2024-09-10T21:11:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.965+0000 7fdd48c00640 1 -- 172.21.15.16:0/4203744676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdd3c019070 con 0x7fdd4c0b0590 2024-09-10T21:11:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.965+0000 7fdd48c00640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd280725e0 0x7fdd28074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.965+0000 7fdd48c00640 1 -- 172.21.15.16:0/4203744676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fdd3c093460 con 0x7fdd4c0b0590 2024-09-10T21:11:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.965+0000 7fdd43e00640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd280725e0 0x7fdd28074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:17.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.965+0000 7fdd43e00640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd280725e0 0x7fdd28074aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0b46b0 tx=0x7fdd34007e40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:17.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:17.969+0000 7fdd48c00640 1 -- 172.21.15.16:0/4203744676 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdd3c05e4e0 con 0x7fdd4c0b0590 2024-09-10T21:11:18.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.141+0000 7fdd53000640 1 -- 172.21.15.16:0/4203744676 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdd4c0b0010 con 0x7fdd280725e0 2024-09-10T21:11:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.145+0000 7fdd48c00640 1 -- 172.21.15.16:0/4203744676 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250866 (secure 0 0 0) 0x7fdd4c0b0010 con 0x7fdd280725e0 2024-09-10T21:11:18.154 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:11:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.149+0000 7fdd53000640 1 -- 172.21.15.16:0/4203744676 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd280725e0 msgr2=0x7fdd28074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.149+0000 7fdd53000640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd280725e0 0x7fdd28074aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0b46b0 tx=0x7fdd34007e40 comp rx=0 tx=0).stop 2024-09-10T21:11:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.149+0000 7fdd53000640 1 -- 172.21.15.16:0/4203744676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4c0b0590 msgr2=0x7fdd4c152430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.149+0000 7fdd53000640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4c0b0590 0x7fdd4c152430 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00d770 tx=0x7fdd3c00dc30 comp rx=0 tx=0).stop 2024-09-10T21:11:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.149+0000 7fdd53000640 1 -- 172.21.15.16:0/4203744676 shutdown_connections 2024-09-10T21:11:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.149+0000 7fdd53000640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd280725e0 0x7fdd28074aa0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.149+0000 7fdd53000640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd4c15f400 0x7fdd4c06f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.149+0000 7fdd53000640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdd4c0b13c0 0x7fdd4c06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.149+0000 7fdd53000640 1 --2- 172.21.15.16:0/4203744676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4c0b0590 0x7fdd4c152430 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.149+0000 7fdd53000640 1 -- 172.21.15.16:0/4203744676 >> 172.21.15.16:0/4203744676 conn(0x7fdd4c09b130 msgr2=0x7fdd4c0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.149+0000 7fdd53000640 1 -- 172.21.15.16:0/4203744676 shutdown_connections 2024-09-10T21:11:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.149+0000 7fdd53000640 1 -- 172.21.15.16:0/4203744676 wait complete. 2024-09-10T21:11:18.174 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-10T21:11:17.466029+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":129,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253316,"kb_used_data":3780,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721404,"statfs":{"total":773094113280,"available":772834717696,"internally_reserved":0,"allocated":3870720,"data_stored":2176274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000978"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245694+0000","last_change":"2024-09-10T21:11:13.244223+0000","last_active":"2024-09-10T21:11:13.245694+0000","last_peered":"2024-09-10T21:11:13.245694+0000","last_clean":"2024-09-10T21:11:13.245694+0000","last_became_active":"2024-09-10T21:11:13.243954+0000","last_became_peered":"2024-09-10T21:11:13.243954+0000","last_unstale":"2024-09-10T21:11:13.245694+0000","last_undegraded":"2024-09-10T21:11:13.245694+0000","last_fullsized":"2024-09-10T21:11:13.245694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:23.544274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.249545+0000","last_change":"2024-09-10T21:11:13.247771+0000","last_active":"2024-09-10T21:11:13.249545+0000","last_peered":"2024-09-10T21:11:13.249545+0000","last_clean":"2024-09-10T21:11:13.249545+0000","last_became_active":"2024-09-10T21:11:13.247293+0000","last_became_peered":"2024-09-10T21:11:13.247293+0000","last_unstale":"2024-09-10T21:11:13.249545+0000","last_undegraded":"2024-09-10T21:11:13.249545+0000","last_fullsized":"2024-09-10T21:11:13.249545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:31.442572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.206691+0000","last_change":"2024-09-10T21:11:12.206691+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.206691+0000","last_undegraded":"2024-09-10T21:11:12.206691+0000","last_fullsized":"2024-09-10T21:11:12.206691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248003+0000","last_change":"2024-09-10T21:11:13.233899+0000","last_active":"2024-09-10T21:11:13.248003+0000","last_peered":"2024-09-10T21:11:13.248003+0000","last_clean":"2024-09-10T21:11:13.248003+0000","last_became_active":"2024-09-10T21:11:13.219250+0000","last_became_peered":"2024-09-10T21:11:13.219250+0000","last_unstale":"2024-09-10T21:11:13.248003+0000","last_undegraded":"2024-09-10T21:11:13.248003+0000","last_fullsized":"2024-09-10T21:11:13.248003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:51.841630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245745+0000","last_change":"2024-09-10T21:11:13.235979+0000","last_active":"2024-09-10T21:11:13.245745+0000","last_peered":"2024-09-10T21:11:13.245745+0000","last_clean":"2024-09-10T21:11:13.245745+0000","last_became_active":"2024-09-10T21:11:13.233126+0000","last_became_peered":"2024-09-10T21:11:13.233126+0000","last_unstale":"2024-09-10T21:11:13.245745+0000","last_undegraded":"2024-09-10T21:11:13.245745+0000","last_fullsized":"2024-09-10T21:11:13.245745+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:15.822007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.209212+0000","last_change":"2024-09-10T21:11:12.209212+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.209212+0000","last_undegraded":"2024-09-10T21:11:12.209212+0000","last_fullsized":"2024-09-10T21:11:12.209212+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.234785+0000","last_change":"2024-09-10T21:11:13.232062+0000","last_active":"2024-09-10T21:11:13.234785+0000","last_peered":"2024-09-10T21:11:13.234785+0000","last_clean":"2024-09-10T21:11:13.234785+0000","last_became_active":"2024-09-10T21:11:13.231580+0000","last_became_peered":"2024-09-10T21:11:13.231580+0000","last_unstale":"2024-09-10T21:11:13.234785+0000","last_undegraded":"2024-09-10T21:11:13.234785+0000","last_fullsized":"2024-09-10T21:11:13.234785+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:12.602121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245385+0000","last_change":"2024-09-10T21:11:13.240369+0000","last_active":"2024-09-10T21:11:13.245385+0000","last_peered":"2024-09-10T21:11:13.245385+0000","last_clean":"2024-09-10T21:11:13.245385+0000","last_became_active":"2024-09-10T21:11:13.239717+0000","last_became_peered":"2024-09-10T21:11:13.239717+0000","last_unstale":"2024-09-10T21:11:13.245385+0000","last_undegraded":"2024-09-10T21:11:13.245385+0000","last_fullsized":"2024-09-10T21:11:13.245385+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:21.291610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.207224+0000","last_change":"2024-09-10T21:11:12.207224+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.207224+0000","last_undegraded":"2024-09-10T21:11:12.207224+0000","last_fullsized":"2024-09-10T21:11:12.207224+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247504+0000","last_change":"2024-09-10T21:11:13.233269+0000","last_active":"2024-09-10T21:11:13.247504+0000","last_peered":"2024-09-10T21:11:13.247504+0000","last_clean":"2024-09-10T21:11:13.247504+0000","last_became_active":"2024-09-10T21:11:13.228592+0000","last_became_peered":"2024-09-10T21:11:13.228592+0000","last_unstale":"2024-09-10T21:11:13.247504+0000","last_undegraded":"2024-09-10T21:11:13.247504+0000","last_fullsized":"2024-09-10T21:11:13.247504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:10.238782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.251364+0000","last_change":"2024-09-10T21:11:13.233758+0000","last_active":"2024-09-10T21:11:13.251364+0000","last_peered":"2024-09-10T21:11:13.251364+0000","last_clean":"2024-09-10T21:11:13.251364+0000","last_became_active":"2024-09-10T21:11:13.233244+0000","last_became_peered":"2024-09-10T21:11:13.233244+0000","last_unstale":"2024-09-10T21:11:13.251364+0000","last_undegraded":"2024-09-10T21:11:13.251364+0000","last_fullsized":"2024-09-10T21:11:13.251364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:53.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.207305+0000","last_change":"2024-09-10T21:11:12.207305+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.207305+0000","last_undegraded":"2024-09-10T21:11:12.207305+0000","last_fullsized":"2024-09-10T21:11:12.207305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.253665+0000","last_change":"2024-09-10T21:11:13.250940+0000","last_active":"2024-09-10T21:11:13.253665+0000","last_peered":"2024-09-10T21:11:13.253665+0000","last_clean":"2024-09-10T21:11:13.253665+0000","last_became_active":"2024-09-10T21:11:13.250694+0000","last_became_peered":"2024-09-10T21:11:13.250694+0000","last_unstale":"2024-09-10T21:11:13.253665+0000","last_undegraded":"2024-09-10T21:11:13.253665+0000","last_fullsized":"2024-09-10T21:11:13.253665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:33.344003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.228510+0000","last_change":"2024-09-10T21:11:13.216431+0000","last_active":"2024-09-10T21:11:13.228510+0000","last_peered":"2024-09-10T21:11:13.228510+0000","last_clean":"2024-09-10T21:11:13.228510+0000","last_became_active":"2024-09-10T21:11:13.216126+0000","last_became_peered":"2024-09-10T21:11:13.216126+0000","last_unstale":"2024-09-10T21:11:13.228510+0000","last_undegraded":"2024-09-10T21:11:13.228510+0000","last_fullsized":"2024-09-10T21:11:13.228510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:27.677406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.229218+0000","last_change":"2024-09-10T21:11:13.227625+0000","last_active":"2024-09-10T21:11:13.229218+0000","last_peered":"2024-09-10T21:11:13.229218+0000","last_clean":"2024-09-10T21:11:13.229218+0000","last_became_active":"2024-09-10T21:11:13.227357+0000","last_became_peered":"2024-09-10T21:11:13.227357+0000","last_unstale":"2024-09-10T21:11:13.229218+0000","last_undegraded":"2024-09-10T21:11:13.229218+0000","last_fullsized":"2024-09-10T21:11:13.229218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:04.567850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239511+0000","last_change":"2024-09-10T21:11:13.233228+0000","last_active":"2024-09-10T21:11:13.239511+0000","last_peered":"2024-09-10T21:11:13.239511+0000","last_clean":"2024-09-10T21:11:13.239511+0000","last_became_active":"2024-09-10T21:11:13.232879+0000","last_became_peered":"2024-09-10T21:11:13.232879+0000","last_unstale":"2024-09-10T21:11:13.239511+0000","last_undegraded":"2024-09-10T21:11:13.239511+0000","last_fullsized":"2024-09-10T21:11:13.239511+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:42.047609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.238484+0000","last_change":"2024-09-10T21:11:13.228537+0000","last_active":"2024-09-10T21:11:13.238484+0000","last_peered":"2024-09-10T21:11:13.238484+0000","last_clean":"2024-09-10T21:11:13.238484+0000","last_became_active":"2024-09-10T21:11:13.225545+0000","last_became_peered":"2024-09-10T21:11:13.225545+0000","last_unstale":"2024-09-10T21:11:13.238484+0000","last_undegraded":"2024-09-10T21:11:13.238484+0000","last_fullsized":"2024-09-10T21:11:13.238484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:07.902864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.211200+0000","last_change":"2024-09-10T21:11:12.211200+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.211200+0000","last_undegraded":"2024-09-10T21:11:12.211200+0000","last_fullsized":"2024-09-10T21:11:12.211200+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.209879+0000","last_change":"2024-09-10T21:11:12.209879+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.209879+0000","last_undegraded":"2024-09-10T21:11:12.209879+0000","last_fullsized":"2024-09-10T21:11:12.209879+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247720+0000","last_change":"2024-09-10T21:11:13.241250+0000","last_active":"2024-09-10T21:11:13.247720+0000","last_peered":"2024-09-10T21:11:13.247720+0000","last_clean":"2024-09-10T21:11:13.247720+0000","last_became_active":"2024-09-10T21:11:13.240973+0000","last_became_peered":"2024-09-10T21:11:13.240973+0000","last_unstale":"2024-09-10T21:11:13.247720+0000","last_undegraded":"2024-09-10T21:11:13.247720+0000","last_fullsized":"2024-09-10T21:11:13.247720+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:22.307668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.240111+0000","last_change":"2024-09-10T21:11:13.236702+0000","last_active":"2024-09-10T21:11:13.240111+0000","last_peered":"2024-09-10T21:11:13.240111+0000","last_clean":"2024-09-10T21:11:13.240111+0000","last_became_active":"2024-09-10T21:11:13.236376+0000","last_became_peered":"2024-09-10T21:11:13.236376+0000","last_unstale":"2024-09-10T21:11:13.240111+0000","last_undegraded":"2024-09-10T21:11:13.240111+0000","last_fullsized":"2024-09-10T21:11:13.240111+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:42.462690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225606+0000","last_change":"2024-09-10T21:11:13.219569+0000","last_active":"2024-09-10T21:11:13.225606+0000","last_peered":"2024-09-10T21:11:13.225606+0000","last_clean":"2024-09-10T21:11:13.225606+0000","last_became_active":"2024-09-10T21:11:13.218463+0000","last_became_peered":"2024-09-10T21:11:13.218463+0000","last_unstale":"2024-09-10T21:11:13.225606+0000","last_undegraded":"2024-09-10T21:11:13.225606+0000","last_fullsized":"2024-09-10T21:11:13.225606+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:12.585123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247420+0000","last_change":"2024-09-10T21:11:13.242608+0000","last_active":"2024-09-10T21:11:13.247420+0000","last_peered":"2024-09-10T21:11:13.247420+0000","last_clean":"2024-09-10T21:11:13.247420+0000","last_became_active":"2024-09-10T21:11:13.241793+0000","last_became_peered":"2024-09-10T21:11:13.241793+0000","last_unstale":"2024-09-10T21:11:13.247420+0000","last_undegraded":"2024-09-10T21:11:13.247420+0000","last_fullsized":"2024-09-10T21:11:13.247420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:01.678771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245420+0000","last_change":"2024-09-10T21:11:13.240633+0000","last_active":"2024-09-10T21:11:13.245420+0000","last_peered":"2024-09-10T21:11:13.245420+0000","last_clean":"2024-09-10T21:11:13.245420+0000","last_became_active":"2024-09-10T21:11:13.240046+0000","last_became_peered":"2024-09-10T21:11:13.240046+0000","last_unstale":"2024-09-10T21:11:13.245420+0000","last_undegraded":"2024-09-10T21:11:13.245420+0000","last_fullsized":"2024-09-10T21:11:13.245420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:30.065234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.237948+0000","last_change":"2024-09-10T21:11:13.219704+0000","last_active":"2024-09-10T21:11:13.237948+0000","last_peered":"2024-09-10T21:11:13.237948+0000","last_clean":"2024-09-10T21:11:13.237948+0000","last_became_active":"2024-09-10T21:11:13.218621+0000","last_became_peered":"2024-09-10T21:11:13.218621+0000","last_unstale":"2024-09-10T21:11:13.237948+0000","last_undegraded":"2024-09-10T21:11:13.237948+0000","last_fullsized":"2024-09-10T21:11:13.237948+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:54.179687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245811+0000","last_change":"2024-09-10T21:11:13.232802+0000","last_active":"2024-09-10T21:11:13.245811+0000","last_peered":"2024-09-10T21:11:13.245811+0000","last_clean":"2024-09-10T21:11:13.245811+0000","last_became_active":"2024-09-10T21:11:13.227485+0000","last_became_peered":"2024-09-10T21:11:13.227485+0000","last_unstale":"2024-09-10T21:11:13.245811+0000","last_undegraded":"2024-09-10T21:11:13.245811+0000","last_fullsized":"2024-09-10T21:11:13.245811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:51.065370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.246021+0000","last_change":"2024-09-10T21:11:13.239036+0000","last_active":"2024-09-10T21:11:13.246021+0000","last_peered":"2024-09-10T21:11:13.246021+0000","last_clean":"2024-09-10T21:11:13.246021+0000","last_became_active":"2024-09-10T21:11:13.238795+0000","last_became_peered":"2024-09-10T21:11:13.238795+0000","last_unstale":"2024-09-10T21:11:13.246021+0000","last_undegraded":"2024-09-10T21:11:13.246021+0000","last_fullsized":"2024-09-10T21:11:13.246021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:32.087701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.243288+0000","last_change":"2024-09-10T21:11:13.235338+0000","last_active":"2024-09-10T21:11:13.243288+0000","last_peered":"2024-09-10T21:11:13.243288+0000","last_clean":"2024-09-10T21:11:13.243288+0000","last_became_active":"2024-09-10T21:11:13.231675+0000","last_became_peered":"2024-09-10T21:11:13.231675+0000","last_unstale":"2024-09-10T21:11:13.243288+0000","last_undegraded":"2024-09-10T21:11:13.243288+0000","last_fullsized":"2024-09-10T21:11:13.243288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:42.770316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.252762+0000","last_change":"2024-09-10T21:11:13.249311+0000","last_active":"2024-09-10T21:11:13.252762+0000","last_peered":"2024-09-10T21:11:13.252762+0000","last_clean":"2024-09-10T21:11:13.252762+0000","last_became_active":"2024-09-10T21:11:13.248813+0000","last_became_peered":"2024-09-10T21:11:13.248813+0000","last_unstale":"2024-09-10T21:11:13.252762+0000","last_undegraded":"2024-09-10T21:11:13.252762+0000","last_fullsized":"2024-09-10T21:11:13.252762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:18.708305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.249039+0000","last_change":"2024-09-10T21:11:13.248118+0000","last_active":"2024-09-10T21:11:13.249039+0000","last_peered":"2024-09-10T21:11:13.249039+0000","last_clean":"2024-09-10T21:11:13.249039+0000","last_became_active":"2024-09-10T21:11:13.247828+0000","last_became_peered":"2024-09-10T21:11:13.247828+0000","last_unstale":"2024-09-10T21:11:13.249039+0000","last_undegraded":"2024-09-10T21:11:13.249039+0000","last_fullsized":"2024-09-10T21:11:13.249039+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:30.684628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.234753+0000","last_change":"2024-09-10T21:11:13.231198+0000","last_active":"2024-09-10T21:11:13.234753+0000","last_peered":"2024-09-10T21:11:13.234753+0000","last_clean":"2024-09-10T21:11:13.234753+0000","last_became_active":"2024-09-10T21:11:13.229553+0000","last_became_peered":"2024-09-10T21:11:13.229553+0000","last_unstale":"2024-09-10T21:11:13.234753+0000","last_undegraded":"2024-09-10T21:11:13.234753+0000","last_fullsized":"2024-09-10T21:11:13.234753+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:40.266888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.222617+0000","last_change":"2024-09-10T21:11:13.220392+0000","last_active":"2024-09-10T21:11:13.222617+0000","last_peered":"2024-09-10T21:11:13.222617+0000","last_clean":"2024-09-10T21:11:13.222617+0000","last_became_active":"2024-09-10T21:11:13.220117+0000","last_became_peered":"2024-09-10T21:11:13.220117+0000","last_unstale":"2024-09-10T21:11:13.222617+0000","last_undegraded":"2024-09-10T21:11:13.222617+0000","last_fullsized":"2024-09-10T21:11:13.222617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:31.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.234865+0000","last_change":"2024-09-10T21:11:13.224627+0000","last_active":"2024-09-10T21:11:13.234865+0000","last_peered":"2024-09-10T21:11:13.234865+0000","last_clean":"2024-09-10T21:11:13.234865+0000","last_became_active":"2024-09-10T21:11:13.224165+0000","last_became_peered":"2024-09-10T21:11:13.224165+0000","last_unstale":"2024-09-10T21:11:13.234865+0000","last_undegraded":"2024-09-10T21:11:13.234865+0000","last_fullsized":"2024-09-10T21:11:13.234865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:49.105483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.256451+0000","last_change":"2024-09-10T21:11:12.225366+0000","last_active":"2024-09-10T21:11:13.256451+0000","last_peered":"2024-09-10T21:11:13.256451+0000","last_clean":"2024-09-10T21:11:13.256451+0000","last_became_active":"2024-09-10T21:11:12.223302+0000","last_became_peered":"2024-09-10T21:11:12.223302+0000","last_unstale":"2024-09-10T21:11:13.256451+0000","last_undegraded":"2024-09-10T21:11:13.256451+0000","last_fullsized":"2024-09-10T21:11:13.256451+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:27.847102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225535+0000","last_change":"2024-09-10T21:11:13.221051+0000","last_active":"2024-09-10T21:11:13.225535+0000","last_peered":"2024-09-10T21:11:13.225535+0000","last_clean":"2024-09-10T21:11:13.225535+0000","last_became_active":"2024-09-10T21:11:13.220762+0000","last_became_peered":"2024-09-10T21:11:13.220762+0000","last_unstale":"2024-09-10T21:11:13.225535+0000","last_undegraded":"2024-09-10T21:11:13.225535+0000","last_fullsized":"2024-09-10T21:11:13.225535+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:09.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248834+0000","last_change":"2024-09-10T21:11:12.222288+0000","last_active":"2024-09-10T21:11:13.248834+0000","last_peered":"2024-09-10T21:11:13.248834+0000","last_clean":"2024-09-10T21:11:13.248834+0000","last_became_active":"2024-09-10T21:11:12.221928+0000","last_became_peered":"2024-09-10T21:11:12.221928+0000","last_unstale":"2024-09-10T21:11:13.248834+0000","last_undegraded":"2024-09-10T21:11:13.248834+0000","last_fullsized":"2024-09-10T21:11:13.248834+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:35.884414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225276+0000","last_change":"2024-09-10T21:11:13.219137+0000","last_active":"2024-09-10T21:11:13.225276+0000","last_peered":"2024-09-10T21:11:13.225276+0000","last_clean":"2024-09-10T21:11:13.225276+0000","last_became_active":"2024-09-10T21:11:13.218827+0000","last_became_peered":"2024-09-10T21:11:13.218827+0000","last_unstale":"2024-09-10T21:11:13.225276+0000","last_undegraded":"2024-09-10T21:11:13.225276+0000","last_fullsized":"2024-09-10T21:11:13.225276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:47.276393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.228590+0000","last_change":"2024-09-10T21:11:12.223630+0000","last_active":"2024-09-10T21:11:12.228590+0000","last_peered":"2024-09-10T21:11:12.228590+0000","last_clean":"2024-09-10T21:11:12.228590+0000","last_became_active":"2024-09-10T21:11:12.223352+0000","last_became_peered":"2024-09-10T21:11:12.223352+0000","last_unstale":"2024-09-10T21:11:12.228590+0000","last_undegraded":"2024-09-10T21:11:12.228590+0000","last_fullsized":"2024-09-10T21:11:12.228590+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:20.065424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.249650+0000","last_change":"2024-09-10T21:11:13.236750+0000","last_active":"2024-09-10T21:11:13.249650+0000","last_peered":"2024-09-10T21:11:13.249650+0000","last_clean":"2024-09-10T21:11:13.249650+0000","last_became_active":"2024-09-10T21:11:13.236417+0000","last_became_peered":"2024-09-10T21:11:13.236417+0000","last_unstale":"2024-09-10T21:11:13.249650+0000","last_undegraded":"2024-09-10T21:11:13.249650+0000","last_fullsized":"2024-09-10T21:11:13.249650+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:30.284239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.250861+0000","last_change":"2024-09-10T21:11:12.223783+0000","last_active":"2024-09-10T21:11:13.250861+0000","last_peered":"2024-09-10T21:11:13.250861+0000","last_clean":"2024-09-10T21:11:13.250861+0000","last_became_active":"2024-09-10T21:11:12.223509+0000","last_became_peered":"2024-09-10T21:11:12.223509+0000","last_unstale":"2024-09-10T21:11:13.250861+0000","last_undegraded":"2024-09-10T21:11:13.250861+0000","last_fullsized":"2024-09-10T21:11:13.250861+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:14.818515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.205623+0000","last_change":"2024-09-10T21:11:12.205623+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.205623+0000","last_undegraded":"2024-09-10T21:11:12.205623+0000","last_fullsized":"2024-09-10T21:11:12.205623+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247537+0000","last_change":"2024-09-10T21:11:12.212357+0000","last_active":"2024-09-10T21:11:13.247537+0000","last_peered":"2024-09-10T21:11:13.247537+0000","last_clean":"2024-09-10T21:11:13.247537+0000","last_became_active":"2024-09-10T21:11:12.209827+0000","last_became_peered":"2024-09-10T21:11:12.209827+0000","last_unstale":"2024-09-10T21:11:13.247537+0000","last_undegraded":"2024-09-10T21:11:13.247537+0000","last_fullsized":"2024-09-10T21:11:13.247537+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:38.717269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225721+0000","last_change":"2024-09-10T21:11:13.222468+0000","last_active":"2024-09-10T21:11:13.225721+0000","last_peered":"2024-09-10T21:11:13.225721+0000","last_clean":"2024-09-10T21:11:13.225721+0000","last_became_active":"2024-09-10T21:11:13.221546+0000","last_became_peered":"2024-09-10T21:11:13.221546+0000","last_unstale":"2024-09-10T21:11:13.225721+0000","last_undegraded":"2024-09-10T21:11:13.225721+0000","last_fullsized":"2024-09-10T21:11:13.225721+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:03.734116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.251808+0000","last_change":"2024-09-10T21:11:12.225033+0000","last_active":"2024-09-10T21:11:13.251808+0000","last_peered":"2024-09-10T21:11:13.251808+0000","last_clean":"2024-09-10T21:11:13.251808+0000","last_became_active":"2024-09-10T21:11:12.224007+0000","last_became_peered":"2024-09-10T21:11:12.224007+0000","last_unstale":"2024-09-10T21:11:13.251808+0000","last_undegraded":"2024-09-10T21:11:13.251808+0000","last_fullsized":"2024-09-10T21:11:13.251808+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:37.774571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.205574+0000","last_change":"2024-09-10T21:11:12.205574+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.205574+0000","last_undegraded":"2024-09-10T21:11:12.205574+0000","last_fullsized":"2024-09-10T21:11:12.205574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.227968+0000","last_change":"2024-09-10T21:11:12.221685+0000","last_active":"2024-09-10T21:11:12.227968+0000","last_peered":"2024-09-10T21:11:12.227968+0000","last_clean":"2024-09-10T21:11:12.227968+0000","last_became_active":"2024-09-10T21:11:12.221393+0000","last_became_peered":"2024-09-10T21:11:12.221393+0000","last_unstale":"2024-09-10T21:11:12.227968+0000","last_undegraded":"2024-09-10T21:11:12.227968+0000","last_fullsized":"2024-09-10T21:11:12.227968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:09.242206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248043+0000","last_change":"2024-09-10T21:11:13.234906+0000","last_active":"2024-09-10T21:11:13.248043+0000","last_peered":"2024-09-10T21:11:13.248043+0000","last_clean":"2024-09-10T21:11:13.248043+0000","last_became_active":"2024-09-10T21:11:13.228823+0000","last_became_peered":"2024-09-10T21:11:13.228823+0000","last_unstale":"2024-09-10T21:11:13.248043+0000","last_undegraded":"2024-09-10T21:11:13.248043+0000","last_fullsized":"2024-09-10T21:11:13.248043+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:36.486425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.209235+0000","last_change":"2024-09-10T21:11:12.224827+0000","last_active":"2024-09-10T21:11:13.209235+0000","last_peered":"2024-09-10T21:11:13.209235+0000","last_clean":"2024-09-10T21:11:13.209235+0000","last_became_active":"2024-09-10T21:11:12.224356+0000","last_became_peered":"2024-09-10T21:11:12.224356+0000","last_unstale":"2024-09-10T21:11:13.209235+0000","last_undegraded":"2024-09-10T21:11:13.209235+0000","last_fullsized":"2024-09-10T21:11:13.209235+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:00.054106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.252576+0000","last_change":"2024-09-10T21:11:13.233922+0000","last_active":"2024-09-10T21:11:13.252576+0000","last_peered":"2024-09-10T21:11:13.252576+0000","last_clean":"2024-09-10T21:11:13.252576+0000","last_became_active":"2024-09-10T21:11:13.230732+0000","last_became_peered":"2024-09-10T21:11:13.230732+0000","last_unstale":"2024-09-10T21:11:13.252576+0000","last_undegraded":"2024-09-10T21:11:13.252576+0000","last_fullsized":"2024-09-10T21:11:13.252576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:07.175259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.213442+0000","last_change":"2024-09-10T21:11:12.221271+0000","last_active":"2024-09-10T21:11:13.213442+0000","last_peered":"2024-09-10T21:11:13.213442+0000","last_clean":"2024-09-10T21:11:13.213442+0000","last_became_active":"2024-09-10T21:11:12.220975+0000","last_became_peered":"2024-09-10T21:11:12.220975+0000","last_unstale":"2024-09-10T21:11:13.213442+0000","last_undegraded":"2024-09-10T21:11:13.213442+0000","last_fullsized":"2024-09-10T21:11:13.213442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:29.280746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247971+0000","last_change":"2024-09-10T21:11:13.234418+0000","last_active":"2024-09-10T21:11:13.247971+0000","last_peered":"2024-09-10T21:11:13.247971+0000","last_clean":"2024-09-10T21:11:13.247971+0000","last_became_active":"2024-09-10T21:11:13.234159+0000","last_became_peered":"2024-09-10T21:11:13.234159+0000","last_unstale":"2024-09-10T21:11:13.247971+0000","last_undegraded":"2024-09-10T21:11:13.247971+0000","last_fullsized":"2024-09-10T21:11:13.247971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:33.043662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.211318+0000","last_change":"2024-09-10T21:11:12.222257+0000","last_active":"2024-09-10T21:11:13.211318+0000","last_peered":"2024-09-10T21:11:13.211318+0000","last_clean":"2024-09-10T21:11:13.211318+0000","last_became_active":"2024-09-10T21:11:12.221959+0000","last_became_peered":"2024-09-10T21:11:12.221959+0000","last_unstale":"2024-09-10T21:11:13.211318+0000","last_undegraded":"2024-09-10T21:11:13.211318+0000","last_fullsized":"2024-09-10T21:11:13.211318+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:35.482932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.240430+0000","last_change":"2024-09-10T21:11:13.220496+0000","last_active":"2024-09-10T21:11:13.240430+0000","last_peered":"2024-09-10T21:11:13.240430+0000","last_clean":"2024-09-10T21:11:13.240430+0000","last_became_active":"2024-09-10T21:11:13.218857+0000","last_became_peered":"2024-09-10T21:11:13.218857+0000","last_unstale":"2024-09-10T21:11:13.240430+0000","last_undegraded":"2024-09-10T21:11:13.240430+0000","last_fullsized":"2024-09-10T21:11:13.240430+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:08.219546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254368+0000","last_change":"2024-09-10T21:11:12.229925+0000","last_active":"2024-09-10T21:11:13.254368+0000","last_peered":"2024-09-10T21:11:13.254368+0000","last_clean":"2024-09-10T21:11:13.254368+0000","last_became_active":"2024-09-10T21:11:12.229037+0000","last_became_peered":"2024-09-10T21:11:12.229037+0000","last_unstale":"2024-09-10T21:11:13.254368+0000","last_undegraded":"2024-09-10T21:11:13.254368+0000","last_fullsized":"2024-09-10T21:11:13.254368+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:43.936262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.251337+0000","last_change":"2024-09-10T21:11:13.233489+0000","last_active":"2024-09-10T21:11:13.251337+0000","last_peered":"2024-09-10T21:11:13.251337+0000","last_clean":"2024-09-10T21:11:13.251337+0000","last_became_active":"2024-09-10T21:11:13.232909+0000","last_became_peered":"2024-09-10T21:11:13.232909+0000","last_unstale":"2024-09-10T21:11:13.251337+0000","last_undegraded":"2024-09-10T21:11:13.251337+0000","last_fullsized":"2024-09-10T21:11:13.251337+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:15.390510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.256092+0000","last_change":"2024-09-10T21:11:12.230626+0000","last_active":"2024-09-10T21:11:13.256092+0000","last_peered":"2024-09-10T21:11:13.256092+0000","last_clean":"2024-09-10T21:11:13.256092+0000","last_became_active":"2024-09-10T21:11:12.230308+0000","last_became_peered":"2024-09-10T21:11:12.230308+0000","last_unstale":"2024-09-10T21:11:13.256092+0000","last_undegraded":"2024-09-10T21:11:13.256092+0000","last_fullsized":"2024-09-10T21:11:13.256092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:09.728148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.250370+0000","last_change":"2024-09-10T21:11:13.242994+0000","last_active":"2024-09-10T21:11:13.250370+0000","last_peered":"2024-09-10T21:11:13.250370+0000","last_clean":"2024-09-10T21:11:13.250370+0000","last_became_active":"2024-09-10T21:11:13.242683+0000","last_became_peered":"2024-09-10T21:11:13.242683+0000","last_unstale":"2024-09-10T21:11:13.250370+0000","last_undegraded":"2024-09-10T21:11:13.250370+0000","last_fullsized":"2024-09-10T21:11:13.250370+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:04.248893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254320+0000","last_change":"2024-09-10T21:11:12.224674+0000","last_active":"2024-09-10T21:11:13.254320+0000","last_peered":"2024-09-10T21:11:13.254320+0000","last_clean":"2024-09-10T21:11:13.254320+0000","last_became_active":"2024-09-10T21:11:12.224325+0000","last_became_peered":"2024-09-10T21:11:12.224325+0000","last_unstale":"2024-09-10T21:11:13.254320+0000","last_undegraded":"2024-09-10T21:11:13.254320+0000","last_fullsized":"2024-09-10T21:11:13.254320+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:06.355342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248957+0000","last_change":"2024-09-10T21:11:13.236816+0000","last_active":"2024-09-10T21:11:13.248957+0000","last_peered":"2024-09-10T21:11:13.248957+0000","last_clean":"2024-09-10T21:11:13.248957+0000","last_became_active":"2024-09-10T21:11:13.236502+0000","last_became_peered":"2024-09-10T21:11:13.236502+0000","last_unstale":"2024-09-10T21:11:13.248957+0000","last_undegraded":"2024-09-10T21:11:13.248957+0000","last_fullsized":"2024-09-10T21:11:13.248957+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:55.941374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.237205+0000","last_change":"2024-09-10T21:11:12.226932+0000","last_active":"2024-09-10T21:11:13.237205+0000","last_peered":"2024-09-10T21:11:13.237205+0000","last_clean":"2024-09-10T21:11:13.237205+0000","last_became_active":"2024-09-10T21:11:12.226622+0000","last_became_peered":"2024-09-10T21:11:12.226622+0000","last_unstale":"2024-09-10T21:11:13.237205+0000","last_undegraded":"2024-09-10T21:11:13.237205+0000","last_fullsized":"2024-09-10T21:11:13.237205+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:15.020787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239479+0000","last_change":"2024-09-10T21:11:13.226536+0000","last_active":"2024-09-10T21:11:13.239479+0000","last_peered":"2024-09-10T21:11:13.239479+0000","last_clean":"2024-09-10T21:11:13.239479+0000","last_became_active":"2024-09-10T21:11:13.226204+0000","last_became_peered":"2024-09-10T21:11:13.226204+0000","last_unstale":"2024-09-10T21:11:13.239479+0000","last_undegraded":"2024-09-10T21:11:13.239479+0000","last_fullsized":"2024-09-10T21:11:13.239479+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:28.355449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.221557+0000","last_change":"2024-09-10T21:11:12.222663+0000","last_active":"2024-09-10T21:11:13.221557+0000","last_peered":"2024-09-10T21:11:13.221557+0000","last_clean":"2024-09-10T21:11:13.221557+0000","last_became_active":"2024-09-10T21:11:12.222491+0000","last_became_peered":"2024-09-10T21:11:12.222491+0000","last_unstale":"2024-09-10T21:11:13.221557+0000","last_undegraded":"2024-09-10T21:11:13.221557+0000","last_fullsized":"2024-09-10T21:11:13.221557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:26.481636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.221362+0000","last_change":"2024-09-10T21:11:13.218025+0000","last_active":"2024-09-10T21:11:13.221362+0000","last_peered":"2024-09-10T21:11:13.221362+0000","last_clean":"2024-09-10T21:11:13.221362+0000","last_became_active":"2024-09-10T21:11:13.217119+0000","last_became_peered":"2024-09-10T21:11:13.217119+0000","last_unstale":"2024-09-10T21:11:13.221362+0000","last_undegraded":"2024-09-10T21:11:13.221362+0000","last_fullsized":"2024-09-10T21:11:13.221362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:42.864605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.229482+0000","last_change":"2024-09-10T21:11:12.225066+0000","last_active":"2024-09-10T21:11:12.229482+0000","last_peered":"2024-09-10T21:11:12.229482+0000","last_clean":"2024-09-10T21:11:12.229482+0000","last_became_active":"2024-09-10T21:11:12.224555+0000","last_became_peered":"2024-09-10T21:11:12.224555+0000","last_unstale":"2024-09-10T21:11:12.229482+0000","last_undegraded":"2024-09-10T21:11:12.229482+0000","last_fullsized":"2024-09-10T21:11:12.229482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:51.343919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.252602+0000","last_change":"2024-09-10T21:11:13.238225+0000","last_active":"2024-09-10T21:11:13.252602+0000","last_peered":"2024-09-10T21:11:13.252602+0000","last_clean":"2024-09-10T21:11:13.252602+0000","last_became_active":"2024-09-10T21:11:13.236412+0000","last_became_peered":"2024-09-10T21:11:13.236412+0000","last_unstale":"2024-09-10T21:11:13.252602+0000","last_undegraded":"2024-09-10T21:11:13.252602+0000","last_fullsized":"2024-09-10T21:11:13.252602+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:48.715087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.229803+0000","last_change":"2024-09-10T21:11:12.228489+0000","last_active":"2024-09-10T21:11:12.229803+0000","last_peered":"2024-09-10T21:11:12.229803+0000","last_clean":"2024-09-10T21:11:12.229803+0000","last_became_active":"2024-09-10T21:11:12.228136+0000","last_became_peered":"2024-09-10T21:11:12.228136+0000","last_unstale":"2024-09-10T21:11:12.229803+0000","last_undegraded":"2024-09-10T21:11:12.229803+0000","last_fullsized":"2024-09-10T21:11:12.229803+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:12.504083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.206728+0000","last_change":"2024-09-10T21:11:12.206728+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.206728+0000","last_undegraded":"2024-09-10T21:11:12.206728+0000","last_fullsized":"2024-09-10T21:11:12.206728+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254662+0000","last_change":"2024-09-10T21:11:12.222819+0000","last_active":"2024-09-10T21:11:13.254662+0000","last_peered":"2024-09-10T21:11:13.254662+0000","last_clean":"2024-09-10T21:11:13.254662+0000","last_became_active":"2024-09-10T21:11:12.222499+0000","last_became_peered":"2024-09-10T21:11:12.222499+0000","last_unstale":"2024-09-10T21:11:13.254662+0000","last_undegraded":"2024-09-10T21:11:13.254662+0000","last_fullsized":"2024-09-10T21:11:13.254662+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:11.810704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.229207+0000","last_change":"2024-09-10T21:11:13.227904+0000","last_active":"2024-09-10T21:11:13.229207+0000","last_peered":"2024-09-10T21:11:13.229207+0000","last_clean":"2024-09-10T21:11:13.229207+0000","last_became_active":"2024-09-10T21:11:13.227548+0000","last_became_peered":"2024-09-10T21:11:13.227548+0000","last_unstale":"2024-09-10T21:11:13.229207+0000","last_undegraded":"2024-09-10T21:11:13.229207+0000","last_fullsized":"2024-09-10T21:11:13.229207+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:36.887907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254448+0000","last_change":"2024-09-10T21:11:12.230259+0000","last_active":"2024-09-10T21:11:13.254448+0000","last_peered":"2024-09-10T21:11:13.254448+0000","last_clean":"2024-09-10T21:11:13.254448+0000","last_became_active":"2024-09-10T21:11:12.229526+0000","last_became_peered":"2024-09-10T21:11:12.229526+0000","last_unstale":"2024-09-10T21:11:13.254448+0000","last_undegraded":"2024-09-10T21:11:13.254448+0000","last_fullsized":"2024-09-10T21:11:13.254448+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:26.566827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.209289+0000","last_change":"2024-09-10T21:11:12.209289+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.209289+0000","last_undegraded":"2024-09-10T21:11:12.209289+0000","last_fullsized":"2024-09-10T21:11:12.209289+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.236111+0000","last_change":"2024-09-10T21:11:12.230126+0000","last_active":"2024-09-10T21:11:13.236111+0000","last_peered":"2024-09-10T21:11:13.236111+0000","last_clean":"2024-09-10T21:11:13.236111+0000","last_became_active":"2024-09-10T21:11:12.229860+0000","last_became_peered":"2024-09-10T21:11:12.229860+0000","last_unstale":"2024-09-10T21:11:13.236111+0000","last_undegraded":"2024-09-10T21:11:13.236111+0000","last_fullsized":"2024-09-10T21:11:13.236111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:09.292762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.250343+0000","last_change":"2024-09-10T21:11:13.241293+0000","last_active":"2024-09-10T21:11:13.250343+0000","last_peered":"2024-09-10T21:11:13.250343+0000","last_clean":"2024-09-10T21:11:13.250343+0000","last_became_active":"2024-09-10T21:11:13.240911+0000","last_became_peered":"2024-09-10T21:11:13.240911+0000","last_unstale":"2024-09-10T21:11:13.250343+0000","last_undegraded":"2024-09-10T21:11:13.250343+0000","last_fullsized":"2024-09-10T21:11:13.250343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:48.860100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248317+0000","last_change":"2024-09-10T21:11:12.221176+0000","last_active":"2024-09-10T21:11:13.248317+0000","last_peered":"2024-09-10T21:11:13.248317+0000","last_clean":"2024-09-10T21:11:13.248317+0000","last_became_active":"2024-09-10T21:11:12.220902+0000","last_became_peered":"2024-09-10T21:11:12.220902+0000","last_unstale":"2024-09-10T21:11:13.248317+0000","last_undegraded":"2024-09-10T21:11:13.248317+0000","last_fullsized":"2024-09-10T21:11:13.248317+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:54.937881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.227103+0000","last_change":"2024-09-10T21:11:13.224634+0000","last_active":"2024-09-10T21:11:13.227103+0000","last_peered":"2024-09-10T21:11:13.227103+0000","last_clean":"2024-09-10T21:11:13.227103+0000","last_became_active":"2024-09-10T21:11:13.224333+0000","last_became_peered":"2024-09-10T21:11:13.224333+0000","last_unstale":"2024-09-10T21:11:13.227103+0000","last_undegraded":"2024-09-10T21:11:13.227103+0000","last_fullsized":"2024-09-10T21:11:13.227103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:16.024280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.209741+0000","last_change":"2024-09-10T21:11:12.226970+0000","last_active":"2024-09-10T21:11:13.209741+0000","last_peered":"2024-09-10T21:11:13.209741+0000","last_clean":"2024-09-10T21:11:13.209741+0000","last_became_active":"2024-09-10T21:11:12.226548+0000","last_became_peered":"2024-09-10T21:11:12.226548+0000","last_unstale":"2024-09-10T21:11:13.209741+0000","last_undegraded":"2024-09-10T21:11:13.209741+0000","last_fullsized":"2024-09-10T21:11:13.209741+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:20.427638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.255041+0000","last_change":"2024-09-10T21:11:13.253121+0000","last_active":"2024-09-10T21:11:13.255041+0000","last_peered":"2024-09-10T21:11:13.255041+0000","last_clean":"2024-09-10T21:11:13.255041+0000","last_became_active":"2024-09-10T21:11:13.252833+0000","last_became_peered":"2024-09-10T21:11:13.252833+0000","last_unstale":"2024-09-10T21:11:13.255041+0000","last_undegraded":"2024-09-10T21:11:13.255041+0000","last_fullsized":"2024-09-10T21:11:13.255041+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:19.718594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.220466+0000","last_change":"2024-09-10T21:11:12.215786+0000","last_active":"2024-09-10T21:11:12.220466+0000","last_peered":"2024-09-10T21:11:12.220466+0000","last_clean":"2024-09-10T21:11:12.220466+0000","last_became_active":"2024-09-10T21:11:12.215497+0000","last_became_peered":"2024-09-10T21:11:12.215497+0000","last_unstale":"2024-09-10T21:11:12.220466+0000","last_undegraded":"2024-09-10T21:11:12.220466+0000","last_fullsized":"2024-09-10T21:11:12.220466+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:08.359968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.205996+0000","last_change":"2024-09-10T21:11:12.205996+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.205996+0000","last_undegraded":"2024-09-10T21:11:12.205996+0000","last_fullsized":"2024-09-10T21:11:12.205996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.212985+0000","last_change":"2024-09-10T21:11:12.234069+0000","last_active":"2024-09-10T21:11:13.212985+0000","last_peered":"2024-09-10T21:11:13.212985+0000","last_clean":"2024-09-10T21:11:13.212985+0000","last_became_active":"2024-09-10T21:11:12.233714+0000","last_became_peered":"2024-09-10T21:11:12.233714+0000","last_unstale":"2024-09-10T21:11:13.212985+0000","last_undegraded":"2024-09-10T21:11:13.212985+0000","last_fullsized":"2024-09-10T21:11:13.212985+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.570509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.210725+0000","last_change":"2024-09-10T21:11:12.210725+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.210725+0000","last_undegraded":"2024-09-10T21:11:12.210725+0000","last_fullsized":"2024-09-10T21:11:12.210725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.241476+0000","last_change":"2024-09-10T21:11:12.223676+0000","last_active":"2024-09-10T21:11:13.241476+0000","last_peered":"2024-09-10T21:11:13.241476+0000","last_clean":"2024-09-10T21:11:13.241476+0000","last_became_active":"2024-09-10T21:11:12.222053+0000","last_became_peered":"2024-09-10T21:11:12.222053+0000","last_unstale":"2024-09-10T21:11:13.241476+0000","last_undegraded":"2024-09-10T21:11:13.241476+0000","last_fullsized":"2024-09-10T21:11:13.241476+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:51.453841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.228362+0000","last_change":"2024-09-10T21:11:13.223973+0000","last_active":"2024-09-10T21:11:13.228362+0000","last_peered":"2024-09-10T21:11:13.228362+0000","last_clean":"2024-09-10T21:11:13.228362+0000","last_became_active":"2024-09-10T21:11:13.223701+0000","last_became_peered":"2024-09-10T21:11:13.223701+0000","last_unstale":"2024-09-10T21:11:13.228362+0000","last_undegraded":"2024-09-10T21:11:13.228362+0000","last_fullsized":"2024-09-10T21:11:13.228362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:57.527538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.211786+0000","last_change":"2024-09-10T21:11:12.222106+0000","last_active":"2024-09-10T21:11:13.211786+0000","last_peered":"2024-09-10T21:11:13.211786+0000","last_clean":"2024-09-10T21:11:13.211786+0000","last_became_active":"2024-09-10T21:11:12.221770+0000","last_became_peered":"2024-09-10T21:11:12.221770+0000","last_unstale":"2024-09-10T21:11:13.211786+0000","last_undegraded":"2024-09-10T21:11:13.211786+0000","last_fullsized":"2024-09-10T21:11:13.211786+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:07.999372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.208795+0000","last_change":"2024-09-10T21:11:12.208795+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.208795+0000","last_undegraded":"2024-09-10T21:11:12.208795+0000","last_fullsized":"2024-09-10T21:11:12.208795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.227006+0000","last_change":"2024-09-10T21:11:12.229500+0000","last_active":"2024-09-10T21:11:13.227006+0000","last_peered":"2024-09-10T21:11:13.227006+0000","last_clean":"2024-09-10T21:11:13.227006+0000","last_became_active":"2024-09-10T21:11:12.229152+0000","last_became_peered":"2024-09-10T21:11:12.229152+0000","last_unstale":"2024-09-10T21:11:13.227006+0000","last_undegraded":"2024-09-10T21:11:13.227006+0000","last_fullsized":"2024-09-10T21:11:13.227006+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:11.772150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239465+0000","last_change":"2024-09-10T21:11:13.232669+0000","last_active":"2024-09-10T21:11:13.239465+0000","last_peered":"2024-09-10T21:11:13.239465+0000","last_clean":"2024-09-10T21:11:13.239465+0000","last_became_active":"2024-09-10T21:11:13.232049+0000","last_became_peered":"2024-09-10T21:11:13.232049+0000","last_unstale":"2024-09-10T21:11:13.239465+0000","last_undegraded":"2024-09-10T21:11:13.239465+0000","last_fullsized":"2024-09-10T21:11:13.239465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:57.314451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.231085+0000","last_change":"2024-09-10T21:11:12.224196+0000","last_active":"2024-09-10T21:11:13.231085+0000","last_peered":"2024-09-10T21:11:13.231085+0000","last_clean":"2024-09-10T21:11:13.231085+0000","last_became_active":"2024-09-10T21:11:12.223853+0000","last_became_peered":"2024-09-10T21:11:12.223853+0000","last_unstale":"2024-09-10T21:11:13.231085+0000","last_undegraded":"2024-09-10T21:11:13.231085+0000","last_fullsized":"2024-09-10T21:11:13.231085+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:23.761112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248934+0000","last_change":"2024-09-10T21:11:13.236581+0000","last_active":"2024-09-10T21:11:13.248934+0000","last_peered":"2024-09-10T21:11:13.248934+0000","last_clean":"2024-09-10T21:11:13.248934+0000","last_became_active":"2024-09-10T21:11:13.236310+0000","last_became_peered":"2024-09-10T21:11:13.236310+0000","last_unstale":"2024-09-10T21:11:13.248934+0000","last_undegraded":"2024-09-10T21:11:13.248934+0000","last_fullsized":"2024-09-10T21:11:13.248934+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:09.002865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.220680+0000","last_change":"2024-09-10T21:11:12.224815+0000","last_active":"2024-09-10T21:11:13.220680+0000","last_peered":"2024-09-10T21:11:13.220680+0000","last_clean":"2024-09-10T21:11:13.220680+0000","last_became_active":"2024-09-10T21:11:12.224504+0000","last_became_peered":"2024-09-10T21:11:12.224504+0000","last_unstale":"2024-09-10T21:11:13.220680+0000","last_undegraded":"2024-09-10T21:11:13.220680+0000","last_fullsized":"2024-09-10T21:11:13.220680+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:29.112283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.246966+0000","last_change":"2024-09-10T21:11:13.226608+0000","last_active":"2024-09-10T21:11:13.246966+0000","last_peered":"2024-09-10T21:11:13.246966+0000","last_clean":"2024-09-10T21:11:13.246966+0000","last_became_active":"2024-09-10T21:11:13.226297+0000","last_became_peered":"2024-09-10T21:11:13.226297+0000","last_unstale":"2024-09-10T21:11:13.246966+0000","last_undegraded":"2024-09-10T21:11:13.246966+0000","last_fullsized":"2024-09-10T21:11:13.246966+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:34.526987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.208044+0000","last_change":"2024-09-10T21:11:12.222940+0000","last_active":"2024-09-10T21:11:13.208044+0000","last_peered":"2024-09-10T21:11:13.208044+0000","last_clean":"2024-09-10T21:11:13.208044+0000","last_became_active":"2024-09-10T21:11:12.222474+0000","last_became_peered":"2024-09-10T21:11:12.222474+0000","last_unstale":"2024-09-10T21:11:13.208044+0000","last_undegraded":"2024-09-10T21:11:13.208044+0000","last_fullsized":"2024-09-10T21:11:13.208044+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:32.362391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.237409+0000","last_change":"2024-09-10T21:11:13.228335+0000","last_active":"2024-09-10T21:11:13.237409+0000","last_peered":"2024-09-10T21:11:13.237409+0000","last_clean":"2024-09-10T21:11:13.237409+0000","last_became_active":"2024-09-10T21:11:13.228013+0000","last_became_peered":"2024-09-10T21:11:13.228013+0000","last_unstale":"2024-09-10T21:11:13.237409+0000","last_undegraded":"2024-09-10T21:11:13.237409+0000","last_fullsized":"2024-09-10T21:11:13.237409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:22.229299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.223563+0000","last_change":"2024-09-10T21:11:12.220700+0000","last_active":"2024-09-10T21:11:13.223563+0000","last_peered":"2024-09-10T21:11:13.223563+0000","last_clean":"2024-09-10T21:11:13.223563+0000","last_became_active":"2024-09-10T21:11:12.220364+0000","last_became_peered":"2024-09-10T21:11:12.220364+0000","last_unstale":"2024-09-10T21:11:13.223563+0000","last_undegraded":"2024-09-10T21:11:13.223563+0000","last_fullsized":"2024-09-10T21:11:13.223563+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:30.271305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239565+0000","last_change":"2024-09-10T21:11:13.234881+0000","last_active":"2024-09-10T21:11:13.239565+0000","last_peered":"2024-09-10T21:11:13.239565+0000","last_clean":"2024-09-10T21:11:13.239565+0000","last_became_active":"2024-09-10T21:11:13.234666+0000","last_became_peered":"2024-09-10T21:11:13.234666+0000","last_unstale":"2024-09-10T21:11:13.239565+0000","last_undegraded":"2024-09-10T21:11:13.239565+0000","last_fullsized":"2024-09-10T21:11:13.239565+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:42.843772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.214147+0000","last_change":"2024-09-10T21:11:12.230047+0000","last_active":"2024-09-10T21:11:13.214147+0000","last_peered":"2024-09-10T21:11:13.214147+0000","last_clean":"2024-09-10T21:11:13.214147+0000","last_became_active":"2024-09-10T21:11:12.229249+0000","last_became_peered":"2024-09-10T21:11:12.229249+0000","last_unstale":"2024-09-10T21:11:13.214147+0000","last_undegraded":"2024-09-10T21:11:13.214147+0000","last_fullsized":"2024-09-10T21:11:13.214147+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:59.482848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":127,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.209277+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:11:12.209277+0000","last_peered":"2024-09-10T21:11:12.209277+0000","last_clean":"2024-09-10T21:11:12.209277+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:11:12.209277+0000","last_undegraded":"2024-09-10T21:11:12.209277+0000","last_fullsized":"2024-09-10T21:11:12.209277+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":937984,"data_stored":720037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":356352,"data_stored":130721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":356352,"data_stored":129863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542157,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":909312,"data_stored":706405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:11:18.178 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3d is not active+clean 2024-09-10T21:11:18.178 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:11:12.206691+0000', 'last_change': '2024-09-10T21:11:12.206691+0000', 'last_active': '2024-09-10T21:11:12.194803+0000', 'last_peered': '2024-09-10T21:11:12.194803+0000', 'last_clean': '2024-09-10T21:11:12.194803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:11:12.206691+0000', 'last_undegraded': '2024-09-10T21:11:12.206691+0000', 'last_fullsized': '2024-09-10T21:11:12.206691+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_clean_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:11:18.179 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3a is not active+clean 2024-09-10T21:11:18.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:11:12.209212+0000', 'last_change': '2024-09-10T21:11:12.209212+0000', 'last_active': '2024-09-10T21:11:12.194803+0000', 'last_peered': '2024-09-10T21:11:12.194803+0000', 'last_clean': '2024-09-10T21:11:12.194803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:11:12.209212+0000', 'last_undegraded': '2024-09-10T21:11:12.209212+0000', 'last_fullsized': '2024-09-10T21:11:12.209212+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_clean_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:11:18.179 INFO:tasks.ceph.ceph_manager.ceph:PG 5.37 is not active+clean 2024-09-10T21:11:18.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:11:12.207224+0000', 'last_change': '2024-09-10T21:11:12.207224+0000', 'last_active': '2024-09-10T21:11:12.194803+0000', 'last_peered': '2024-09-10T21:11:12.194803+0000', 'last_clean': '2024-09-10T21:11:12.194803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:11:12.207224+0000', 'last_undegraded': '2024-09-10T21:11:12.207224+0000', 'last_fullsized': '2024-09-10T21:11:12.207224+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_clean_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:11:18.179 INFO:tasks.ceph.ceph_manager.ceph:PG 5.34 is not active+clean 2024-09-10T21:11:18.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:11:12.207305+0000', 'last_change': '2024-09-10T21:11:12.207305+0000', 'last_active': '2024-09-10T21:11:12.194803+0000', 'last_peered': '2024-09-10T21:11:12.194803+0000', 'last_clean': '2024-09-10T21:11:12.194803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:11:12.207305+0000', 'last_undegraded': '2024-09-10T21:11:12.207305+0000', 'last_fullsized': '2024-09-10T21:11:12.207305+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_clean_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:11:18.179 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2e is not active+clean 2024-09-10T21:11:18.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:11:12.211200+0000', 'last_change': '2024-09-10T21:11:12.211200+0000', 'last_active': '2024-09-10T21:11:12.194803+0000', 'last_peered': '2024-09-10T21:11:12.194803+0000', 'last_clean': '2024-09-10T21:11:12.194803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:11:12.211200+0000', 'last_undegraded': '2024-09-10T21:11:12.211200+0000', 'last_fullsized': '2024-09-10T21:11:12.211200+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_clean_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:11:18.179 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2d is not active+clean 2024-09-10T21:11:18.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:11:12.209879+0000', 'last_change': '2024-09-10T21:11:12.209879+0000', 'last_active': '2024-09-10T21:11:12.194803+0000', 'last_peered': '2024-09-10T21:11:12.194803+0000', 'last_clean': '2024-09-10T21:11:12.194803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:11:12.209879+0000', 'last_undegraded': '2024-09-10T21:11:12.209879+0000', 'last_fullsized': '2024-09-10T21:11:12.209879+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_clean_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:11:18.180 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1a is not active+clean 2024-09-10T21:11:18.180 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:11:12.205623+0000', 'last_change': '2024-09-10T21:11:12.205623+0000', 'last_active': '2024-09-10T21:11:12.194803+0000', 'last_peered': '2024-09-10T21:11:12.194803+0000', 'last_clean': '2024-09-10T21:11:12.194803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:11:12.205623+0000', 'last_undegraded': '2024-09-10T21:11:12.205623+0000', 'last_fullsized': '2024-09-10T21:11:12.205623+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_clean_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:11:18.180 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-09-10T21:11:18.180 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:11:12.205574+0000', 'last_change': '2024-09-10T21:11:12.205574+0000', 'last_active': '2024-09-10T21:11:12.194803+0000', 'last_peered': '2024-09-10T21:11:12.194803+0000', 'last_clean': '2024-09-10T21:11:12.194803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:11:12.205574+0000', 'last_undegraded': '2024-09-10T21:11:12.205574+0000', 'last_fullsized': '2024-09-10T21:11:12.205574+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_clean_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:11:18.180 INFO:tasks.ceph.ceph_manager.ceph:PG 5.f is not active+clean 2024-09-10T21:11:18.180 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:11:12.206728+0000', 'last_change': '2024-09-10T21:11:12.206728+0000', 'last_active': '2024-09-10T21:11:12.194803+0000', 'last_peered': '2024-09-10T21:11:12.194803+0000', 'last_clean': '2024-09-10T21:11:12.194803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:11:12.206728+0000', 'last_undegraded': '2024-09-10T21:11:12.206728+0000', 'last_fullsized': '2024-09-10T21:11:12.206728+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_clean_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:11:18.180 INFO:tasks.ceph.ceph_manager.ceph:PG 5.d is not active+clean 2024-09-10T21:11:18.180 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:11:12.209289+0000', 'last_change': '2024-09-10T21:11:12.209289+0000', 'last_active': '2024-09-10T21:11:12.194803+0000', 'last_peered': '2024-09-10T21:11:12.194803+0000', 'last_clean': '2024-09-10T21:11:12.194803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:11:12.209289+0000', 'last_undegraded': '2024-09-10T21:11:12.209289+0000', 'last_fullsized': '2024-09-10T21:11:12.209289+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_clean_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:11:18.180 INFO:tasks.ceph.ceph_manager.ceph:PG 5.9 is not active+clean 2024-09-10T21:11:18.181 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:11:12.205996+0000', 'last_change': '2024-09-10T21:11:12.205996+0000', 'last_active': '2024-09-10T21:11:12.194803+0000', 'last_peered': '2024-09-10T21:11:12.194803+0000', 'last_clean': '2024-09-10T21:11:12.194803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:11:12.205996+0000', 'last_undegraded': '2024-09-10T21:11:12.205996+0000', 'last_fullsized': '2024-09-10T21:11:12.205996+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_clean_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:11:18.181 INFO:tasks.ceph.ceph_manager.ceph:PG 5.6 is not active+clean 2024-09-10T21:11:18.181 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:11:12.210725+0000', 'last_change': '2024-09-10T21:11:12.210725+0000', 'last_active': '2024-09-10T21:11:12.194803+0000', 'last_peered': '2024-09-10T21:11:12.194803+0000', 'last_clean': '2024-09-10T21:11:12.194803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:11:12.210725+0000', 'last_undegraded': '2024-09-10T21:11:12.210725+0000', 'last_fullsized': '2024-09-10T21:11:12.210725+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_clean_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:11:18.181 INFO:tasks.ceph.ceph_manager.ceph:PG 5.5 is not active+clean 2024-09-10T21:11:18.181 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:11:12.208795+0000', 'last_change': '2024-09-10T21:11:12.208795+0000', 'last_active': '2024-09-10T21:11:12.194803+0000', 'last_peered': '2024-09-10T21:11:12.194803+0000', 'last_clean': '2024-09-10T21:11:12.194803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:11:12.208795+0000', 'last_undegraded': '2024-09-10T21:11:12.208795+0000', 'last_fullsized': '2024-09-10T21:11:12.208795+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'last_clean_scrub_stamp': '2024-09-10T21:11:12.194803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:11:18.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T21:11:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.357+0000 7f358b200640 1 -- 172.21.15.16:0/214950607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358415f400 msgr2=0x7f3584070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.357+0000 7f358b200640 1 --2- 172.21.15.16:0/214950607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358415f400 0x7f3584070fe0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f35840cc360 tx=0x7f357401c780 comp rx=0 tx=0).stop 2024-09-10T21:11:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.357+0000 7f358b200640 1 -- 172.21.15.16:0/214950607 shutdown_connections 2024-09-10T21:11:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.357+0000 7f358b200640 1 --2- 172.21.15.16:0/214950607 >> v2:172.21.15.52:3300/0 conn(0x7f35840b0590 0x7f3584071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.357+0000 7f358b200640 1 --2- 172.21.15.16:0/214950607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358415f400 0x7f3584070fe0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.357+0000 7f358b200640 1 -- 172.21.15.16:0/214950607 >> 172.21.15.16:0/214950607 conn(0x7f358409b130 msgr2=0x7f35840d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.357+0000 7f358b200640 1 -- 172.21.15.16:0/214950607 shutdown_connections 2024-09-10T21:11:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.357+0000 7f358b200640 1 -- 172.21.15.16:0/214950607 wait complete. 2024-09-10T21:11:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.357+0000 7f358b200640 1 Processor -- start 2024-09-10T21:11:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.357+0000 7f358b200640 1 -- start start 2024-09-10T21:11:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f358b200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35840b0590 0x7f35840c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f358b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35840b13c0 0x7f35840b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f358b200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f358415f400 0x7f3584156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f358b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3584168f60 con 0x7f35840b13c0 2024-09-10T21:11:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f358b200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3584168de0 con 0x7f35840b0590 2024-09-10T21:11:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f358b200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f35841690e0 con 0x7f358415f400 2024-09-10T21:11:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f3583400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35840b13c0 0x7f35840b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f3583400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35840b13c0 0x7f35840b74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47036/0 (socket says 172.21.15.16:47036) 2024-09-10T21:11:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f3583400640 1 -- 172.21.15.16:0/2415105280 learned_addr learned my addr 172.21.15.16:0/2415105280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f3583e00640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35840b0590 0x7f35840c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f3589200640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f358415f400 0x7f3584156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f3583400640 1 -- 172.21.15.16:0/2415105280 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f358415f400 msgr2=0x7f3584156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f3583400640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f358415f400 0x7f3584156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f3583400640 1 -- 172.21.15.16:0/2415105280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35840b0590 msgr2=0x7f35840c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f3583400640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35840b0590 0x7f35840c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f3583400640 1 -- 172.21.15.16:0/2415105280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35840cb0b0 con 0x7f35840b13c0 2024-09-10T21:11:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f3589200640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f358415f400 0x7f3584156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f3583400640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35840b13c0 0x7f35840b74e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f357000c450 tx=0x7f357000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f357be00640 1 -- 172.21.15.16:0/2415105280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570012830 con 0x7f35840b13c0 2024-09-10T21:11:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f357be00640 1 -- 172.21.15.16:0/2415105280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3570017070 con 0x7f35840b13c0 2024-09-10T21:11:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f357be00640 1 -- 172.21.15.16:0/2415105280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570012d80 con 0x7f35840b13c0 2024-09-10T21:11:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f358b200640 1 -- 172.21.15.16:0/2415105280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35840c7550 con 0x7f35840b13c0 2024-09-10T21:11:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f358b200640 1 -- 172.21.15.16:0/2415105280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35840c7b10 con 0x7f35840b13c0 2024-09-10T21:11:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f358b200640 1 -- 172.21.15.16:0/2415105280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35840cfc80 con 0x7f35840b13c0 2024-09-10T21:11:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.361+0000 7f357be00640 1 -- 172.21.15.16:0/2415105280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f357001c460 con 0x7f35840b13c0 2024-09-10T21:11:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.365+0000 7f357be00640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f35640725e0 0x7f3564074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.365+0000 7f357be00640 1 -- 172.21.15.16:0/2415105280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3570093080 con 0x7f35840b13c0 2024-09-10T21:11:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.365+0000 7f3583e00640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f35640725e0 0x7f3564074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.365+0000 7f3583e00640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f35640725e0 0x7f3564074aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f35740ae000 tx=0x7f35740b3040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.369+0000 7f357be00640 1 -- 172.21.15.16:0/2415105280 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f357005e6c0 con 0x7f35840b13c0 2024-09-10T21:11:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.605+0000 7f358b200640 1 -- 172.21.15.16:0/2415105280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f35840b2bf0 con 0x7f35840b13c0 2024-09-10T21:11:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.605+0000 7f357be00640 1 -- 172.21.15.16:0/2415105280 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2754 (secure 0 0 0) 0x7f357005dfc0 con 0x7f35840b13c0 2024-09-10T21:11:18.612 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:18.613 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":73,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":32,"num_osds":8,"num_up_osds":8,"osd_up_since":1726002618,"num_in_osds":8,"osd_in_since":1726002608,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":84},{"state_name":"creating+peering","count":13}],"num_pgs":97,"num_pools":3,"num_objects":17,"data_bytes":592756,"bytes_used":259395584,"bytes_avail":772834717696,"bytes_total":773094113280,"inactive_pgs_ratio":0.1340206116437912,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":4},"fsmap":{"epoch":16,"btime":"2024-09-10T21:11:16:232531+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":134519}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-10T21:11:11.465093+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T21:11:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.609+0000 7f358b200640 1 -- 172.21.15.16:0/2415105280 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f35640725e0 msgr2=0x7f3564074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.609+0000 7f358b200640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f35640725e0 0x7f3564074aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f35740ae000 tx=0x7f35740b3040 comp rx=0 tx=0).stop 2024-09-10T21:11:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.609+0000 7f358b200640 1 -- 172.21.15.16:0/2415105280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35840b13c0 msgr2=0x7f35840b74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.609+0000 7f358b200640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35840b13c0 0x7f35840b74e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f357000c450 tx=0x7f357000c910 comp rx=0 tx=0).stop 2024-09-10T21:11:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.609+0000 7f358b200640 1 -- 172.21.15.16:0/2415105280 shutdown_connections 2024-09-10T21:11:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.609+0000 7f358b200640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f35640725e0 0x7f3564074aa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.609+0000 7f358b200640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f358415f400 0x7f3584156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.609+0000 7f358b200640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35840b13c0 0x7f35840b74e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.609+0000 7f358b200640 1 --2- 172.21.15.16:0/2415105280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35840b0590 0x7f35840c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.609+0000 7f358b200640 1 -- 172.21.15.16:0/2415105280 >> 172.21.15.16:0/2415105280 conn(0x7f358409b130 msgr2=0x7f35840b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.609+0000 7f358b200640 1 -- 172.21.15.16:0/2415105280 shutdown_connections 2024-09-10T21:11:18.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.609+0000 7f358b200640 1 -- 172.21.15.16:0/2415105280 wait complete. 2024-09-10T21:11:18.633 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T21:11:18.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:11:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.773+0000 7f58efa00640 1 -- 172.21.15.16:0/1325505979 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f58e806fb90 msgr2=0x7f58e80746a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.773+0000 7f58efa00640 1 --2- 172.21.15.16:0/1325505979 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f58e806fb90 0x7f58e80746a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009c80 tx=0x7f58e001cd10 comp rx=0 tx=0).stop 2024-09-10T21:11:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.773+0000 7f58efa00640 1 -- 172.21.15.16:0/1325505979 shutdown_connections 2024-09-10T21:11:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.773+0000 7f58efa00640 1 --2- 172.21.15.16:0/1325505979 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f58e806fb90 0x7f58e80746a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.773+0000 7f58efa00640 1 --2- 172.21.15.16:0/1325505979 >> v2:172.21.15.52:3300/0 conn(0x7f58e806f750 0x7f58e8074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.773+0000 7f58efa00640 1 -- 172.21.15.16:0/1325505979 >> 172.21.15.16:0/1325505979 conn(0x7f58e80d0c30 msgr2=0x7f58e80d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.773+0000 7f58efa00640 1 -- 172.21.15.16:0/1325505979 shutdown_connections 2024-09-10T21:11:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.773+0000 7f58efa00640 1 -- 172.21.15.16:0/1325505979 wait complete. 2024-09-10T21:11:18.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58efa00640 1 Processor -- start 2024-09-10T21:11:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58efa00640 1 -- start start 2024-09-10T21:11:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58efa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f58e806f750 0x7f58e80b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58efa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58e806fb90 0x7f58e815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58efa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e80cc940 0x7f58e80b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58efa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58e816b610 con 0x7f58e80cc940 2024-09-10T21:11:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58efa00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f58e816b490 con 0x7f58e806fb90 2024-09-10T21:11:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58efa00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f58e816b790 con 0x7f58e806f750 2024-09-10T21:11:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58eda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e80cc940 0x7f58e80b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58eda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e80cc940 0x7f58e80b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47046/0 (socket says 172.21.15.16:47046) 2024-09-10T21:11:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58eda00640 1 -- 172.21.15.16:0/3769399537 learned_addr learned my addr 172.21.15.16:0/3769399537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58ed000640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f58e806f750 0x7f58e80b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58e7e00640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58e806fb90 0x7f58e815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58eda00640 1 -- 172.21.15.16:0/3769399537 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f58e806f750 msgr2=0x7f58e80b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58eda00640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f58e806f750 0x7f58e80b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58eda00640 1 -- 172.21.15.16:0/3769399537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58e806fb90 msgr2=0x7f58e815a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58eda00640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58e806fb90 0x7f58e815a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58eda00640 1 -- 172.21.15.16:0/3769399537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e81fb600 con 0x7f58e80cc940 2024-09-10T21:11:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58eda00640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e80cc940 0x7f58e80b1360 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009bd0 tx=0x7f58e0004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58e4c00640 1 -- 172.21.15.16:0/3769399537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e00bb070 con 0x7f58e80cc940 2024-09-10T21:11:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58e4c00640 1 -- 172.21.15.16:0/3769399537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58e0004850 con 0x7f58e80cc940 2024-09-10T21:11:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58efa00640 1 -- 172.21.15.16:0/3769399537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e81fb8a0 con 0x7f58e80cc940 2024-09-10T21:11:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58e4c00640 1 -- 172.21.15.16:0/3769399537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e00b2040 con 0x7f58e80cc940 2024-09-10T21:11:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58efa00640 1 -- 172.21.15.16:0/3769399537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e81fbe60 con 0x7f58e80cc940 2024-09-10T21:11:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.777+0000 7f58efa00640 1 -- 172.21.15.16:0/3769399537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e80ccd20 con 0x7f58e80cc940 2024-09-10T21:11:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.781+0000 7f58e4c00640 1 -- 172.21.15.16:0/3769399537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f58e00b5030 con 0x7f58e80cc940 2024-09-10T21:11:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.781+0000 7f58e4c00640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f58c80725e0 0x7f58c8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.781+0000 7f58e4c00640 1 -- 172.21.15.16:0/3769399537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f58e012d300 con 0x7f58e80cc940 2024-09-10T21:11:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.781+0000 7f58ed000640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f58c80725e0 0x7f58c8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.781+0000 7f58ed000640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f58c80725e0 0x7f58c8074aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f58dc007fd0 tx=0x7f58dc009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.785+0000 7f58e4c00640 1 -- 172.21.15.16:0/3769399537 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f58e0016990 con 0x7f58e80cc940 2024-09-10T21:11:18.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.957+0000 7f58efa00640 1 -- 172.21.15.16:0/3769399537 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58e809f440 con 0x7f58c80725e0 2024-09-10T21:11:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.961+0000 7f58e4c00640 1 -- 172.21.15.16:0/3769399537 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250866 (secure 0 0 0) 0x7f58e809f440 con 0x7f58c80725e0 2024-09-10T21:11:18.967 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:11:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.965+0000 7f58efa00640 1 -- 172.21.15.16:0/3769399537 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f58c80725e0 msgr2=0x7f58c8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.965+0000 7f58efa00640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f58c80725e0 0x7f58c8074aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f58dc007fd0 tx=0x7f58dc009040 comp rx=0 tx=0).stop 2024-09-10T21:11:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.965+0000 7f58efa00640 1 -- 172.21.15.16:0/3769399537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e80cc940 msgr2=0x7f58e80b1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.965+0000 7f58efa00640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e80cc940 0x7f58e80b1360 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009bd0 tx=0x7f58e0004660 comp rx=0 tx=0).stop 2024-09-10T21:11:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.965+0000 7f58efa00640 1 -- 172.21.15.16:0/3769399537 shutdown_connections 2024-09-10T21:11:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.965+0000 7f58efa00640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f58c80725e0 0x7f58c8074aa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.965+0000 7f58efa00640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e80cc940 0x7f58e80b1360 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.965+0000 7f58efa00640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58e806fb90 0x7f58e815a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.965+0000 7f58efa00640 1 --2- 172.21.15.16:0/3769399537 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f58e806f750 0x7f58e80b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.965+0000 7f58efa00640 1 -- 172.21.15.16:0/3769399537 >> 172.21.15.16:0/3769399537 conn(0x7f58e80d0c30 msgr2=0x7f58e80a1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.965+0000 7f58efa00640 1 -- 172.21.15.16:0/3769399537 shutdown_connections 2024-09-10T21:11:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:18.965+0000 7f58efa00640 1 -- 172.21.15.16:0/3769399537 wait complete. 2024-09-10T21:11:18.990 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-10T21:11:17.466029+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":129,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253316,"kb_used_data":3780,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721404,"statfs":{"total":773094113280,"available":772834717696,"internally_reserved":0,"allocated":3870720,"data_stored":2176274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000978"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245694+0000","last_change":"2024-09-10T21:11:13.244223+0000","last_active":"2024-09-10T21:11:13.245694+0000","last_peered":"2024-09-10T21:11:13.245694+0000","last_clean":"2024-09-10T21:11:13.245694+0000","last_became_active":"2024-09-10T21:11:13.243954+0000","last_became_peered":"2024-09-10T21:11:13.243954+0000","last_unstale":"2024-09-10T21:11:13.245694+0000","last_undegraded":"2024-09-10T21:11:13.245694+0000","last_fullsized":"2024-09-10T21:11:13.245694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:23.544274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.249545+0000","last_change":"2024-09-10T21:11:13.247771+0000","last_active":"2024-09-10T21:11:13.249545+0000","last_peered":"2024-09-10T21:11:13.249545+0000","last_clean":"2024-09-10T21:11:13.249545+0000","last_became_active":"2024-09-10T21:11:13.247293+0000","last_became_peered":"2024-09-10T21:11:13.247293+0000","last_unstale":"2024-09-10T21:11:13.249545+0000","last_undegraded":"2024-09-10T21:11:13.249545+0000","last_fullsized":"2024-09-10T21:11:13.249545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:31.442572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.206691+0000","last_change":"2024-09-10T21:11:12.206691+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.206691+0000","last_undegraded":"2024-09-10T21:11:12.206691+0000","last_fullsized":"2024-09-10T21:11:12.206691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248003+0000","last_change":"2024-09-10T21:11:13.233899+0000","last_active":"2024-09-10T21:11:13.248003+0000","last_peered":"2024-09-10T21:11:13.248003+0000","last_clean":"2024-09-10T21:11:13.248003+0000","last_became_active":"2024-09-10T21:11:13.219250+0000","last_became_peered":"2024-09-10T21:11:13.219250+0000","last_unstale":"2024-09-10T21:11:13.248003+0000","last_undegraded":"2024-09-10T21:11:13.248003+0000","last_fullsized":"2024-09-10T21:11:13.248003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:51.841630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245745+0000","last_change":"2024-09-10T21:11:13.235979+0000","last_active":"2024-09-10T21:11:13.245745+0000","last_peered":"2024-09-10T21:11:13.245745+0000","last_clean":"2024-09-10T21:11:13.245745+0000","last_became_active":"2024-09-10T21:11:13.233126+0000","last_became_peered":"2024-09-10T21:11:13.233126+0000","last_unstale":"2024-09-10T21:11:13.245745+0000","last_undegraded":"2024-09-10T21:11:13.245745+0000","last_fullsized":"2024-09-10T21:11:13.245745+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:15.822007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.209212+0000","last_change":"2024-09-10T21:11:12.209212+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.209212+0000","last_undegraded":"2024-09-10T21:11:12.209212+0000","last_fullsized":"2024-09-10T21:11:12.209212+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.234785+0000","last_change":"2024-09-10T21:11:13.232062+0000","last_active":"2024-09-10T21:11:13.234785+0000","last_peered":"2024-09-10T21:11:13.234785+0000","last_clean":"2024-09-10T21:11:13.234785+0000","last_became_active":"2024-09-10T21:11:13.231580+0000","last_became_peered":"2024-09-10T21:11:13.231580+0000","last_unstale":"2024-09-10T21:11:13.234785+0000","last_undegraded":"2024-09-10T21:11:13.234785+0000","last_fullsized":"2024-09-10T21:11:13.234785+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:12.602121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245385+0000","last_change":"2024-09-10T21:11:13.240369+0000","last_active":"2024-09-10T21:11:13.245385+0000","last_peered":"2024-09-10T21:11:13.245385+0000","last_clean":"2024-09-10T21:11:13.245385+0000","last_became_active":"2024-09-10T21:11:13.239717+0000","last_became_peered":"2024-09-10T21:11:13.239717+0000","last_unstale":"2024-09-10T21:11:13.245385+0000","last_undegraded":"2024-09-10T21:11:13.245385+0000","last_fullsized":"2024-09-10T21:11:13.245385+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:21.291610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.207224+0000","last_change":"2024-09-10T21:11:12.207224+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.207224+0000","last_undegraded":"2024-09-10T21:11:12.207224+0000","last_fullsized":"2024-09-10T21:11:12.207224+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247504+0000","last_change":"2024-09-10T21:11:13.233269+0000","last_active":"2024-09-10T21:11:13.247504+0000","last_peered":"2024-09-10T21:11:13.247504+0000","last_clean":"2024-09-10T21:11:13.247504+0000","last_became_active":"2024-09-10T21:11:13.228592+0000","last_became_peered":"2024-09-10T21:11:13.228592+0000","last_unstale":"2024-09-10T21:11:13.247504+0000","last_undegraded":"2024-09-10T21:11:13.247504+0000","last_fullsized":"2024-09-10T21:11:13.247504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:10.238782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.251364+0000","last_change":"2024-09-10T21:11:13.233758+0000","last_active":"2024-09-10T21:11:13.251364+0000","last_peered":"2024-09-10T21:11:13.251364+0000","last_clean":"2024-09-10T21:11:13.251364+0000","last_became_active":"2024-09-10T21:11:13.233244+0000","last_became_peered":"2024-09-10T21:11:13.233244+0000","last_unstale":"2024-09-10T21:11:13.251364+0000","last_undegraded":"2024-09-10T21:11:13.251364+0000","last_fullsized":"2024-09-10T21:11:13.251364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:53.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.207305+0000","last_change":"2024-09-10T21:11:12.207305+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.207305+0000","last_undegraded":"2024-09-10T21:11:12.207305+0000","last_fullsized":"2024-09-10T21:11:12.207305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.253665+0000","last_change":"2024-09-10T21:11:13.250940+0000","last_active":"2024-09-10T21:11:13.253665+0000","last_peered":"2024-09-10T21:11:13.253665+0000","last_clean":"2024-09-10T21:11:13.253665+0000","last_became_active":"2024-09-10T21:11:13.250694+0000","last_became_peered":"2024-09-10T21:11:13.250694+0000","last_unstale":"2024-09-10T21:11:13.253665+0000","last_undegraded":"2024-09-10T21:11:13.253665+0000","last_fullsized":"2024-09-10T21:11:13.253665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:33.344003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.228510+0000","last_change":"2024-09-10T21:11:13.216431+0000","last_active":"2024-09-10T21:11:13.228510+0000","last_peered":"2024-09-10T21:11:13.228510+0000","last_clean":"2024-09-10T21:11:13.228510+0000","last_became_active":"2024-09-10T21:11:13.216126+0000","last_became_peered":"2024-09-10T21:11:13.216126+0000","last_unstale":"2024-09-10T21:11:13.228510+0000","last_undegraded":"2024-09-10T21:11:13.228510+0000","last_fullsized":"2024-09-10T21:11:13.228510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:27.677406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.229218+0000","last_change":"2024-09-10T21:11:13.227625+0000","last_active":"2024-09-10T21:11:13.229218+0000","last_peered":"2024-09-10T21:11:13.229218+0000","last_clean":"2024-09-10T21:11:13.229218+0000","last_became_active":"2024-09-10T21:11:13.227357+0000","last_became_peered":"2024-09-10T21:11:13.227357+0000","last_unstale":"2024-09-10T21:11:13.229218+0000","last_undegraded":"2024-09-10T21:11:13.229218+0000","last_fullsized":"2024-09-10T21:11:13.229218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:04.567850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239511+0000","last_change":"2024-09-10T21:11:13.233228+0000","last_active":"2024-09-10T21:11:13.239511+0000","last_peered":"2024-09-10T21:11:13.239511+0000","last_clean":"2024-09-10T21:11:13.239511+0000","last_became_active":"2024-09-10T21:11:13.232879+0000","last_became_peered":"2024-09-10T21:11:13.232879+0000","last_unstale":"2024-09-10T21:11:13.239511+0000","last_undegraded":"2024-09-10T21:11:13.239511+0000","last_fullsized":"2024-09-10T21:11:13.239511+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:42.047609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.238484+0000","last_change":"2024-09-10T21:11:13.228537+0000","last_active":"2024-09-10T21:11:13.238484+0000","last_peered":"2024-09-10T21:11:13.238484+0000","last_clean":"2024-09-10T21:11:13.238484+0000","last_became_active":"2024-09-10T21:11:13.225545+0000","last_became_peered":"2024-09-10T21:11:13.225545+0000","last_unstale":"2024-09-10T21:11:13.238484+0000","last_undegraded":"2024-09-10T21:11:13.238484+0000","last_fullsized":"2024-09-10T21:11:13.238484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:07.902864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.211200+0000","last_change":"2024-09-10T21:11:12.211200+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.211200+0000","last_undegraded":"2024-09-10T21:11:12.211200+0000","last_fullsized":"2024-09-10T21:11:12.211200+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.209879+0000","last_change":"2024-09-10T21:11:12.209879+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.209879+0000","last_undegraded":"2024-09-10T21:11:12.209879+0000","last_fullsized":"2024-09-10T21:11:12.209879+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247720+0000","last_change":"2024-09-10T21:11:13.241250+0000","last_active":"2024-09-10T21:11:13.247720+0000","last_peered":"2024-09-10T21:11:13.247720+0000","last_clean":"2024-09-10T21:11:13.247720+0000","last_became_active":"2024-09-10T21:11:13.240973+0000","last_became_peered":"2024-09-10T21:11:13.240973+0000","last_unstale":"2024-09-10T21:11:13.247720+0000","last_undegraded":"2024-09-10T21:11:13.247720+0000","last_fullsized":"2024-09-10T21:11:13.247720+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:22.307668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.240111+0000","last_change":"2024-09-10T21:11:13.236702+0000","last_active":"2024-09-10T21:11:13.240111+0000","last_peered":"2024-09-10T21:11:13.240111+0000","last_clean":"2024-09-10T21:11:13.240111+0000","last_became_active":"2024-09-10T21:11:13.236376+0000","last_became_peered":"2024-09-10T21:11:13.236376+0000","last_unstale":"2024-09-10T21:11:13.240111+0000","last_undegraded":"2024-09-10T21:11:13.240111+0000","last_fullsized":"2024-09-10T21:11:13.240111+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:42.462690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225606+0000","last_change":"2024-09-10T21:11:13.219569+0000","last_active":"2024-09-10T21:11:13.225606+0000","last_peered":"2024-09-10T21:11:13.225606+0000","last_clean":"2024-09-10T21:11:13.225606+0000","last_became_active":"2024-09-10T21:11:13.218463+0000","last_became_peered":"2024-09-10T21:11:13.218463+0000","last_unstale":"2024-09-10T21:11:13.225606+0000","last_undegraded":"2024-09-10T21:11:13.225606+0000","last_fullsized":"2024-09-10T21:11:13.225606+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:12.585123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247420+0000","last_change":"2024-09-10T21:11:13.242608+0000","last_active":"2024-09-10T21:11:13.247420+0000","last_peered":"2024-09-10T21:11:13.247420+0000","last_clean":"2024-09-10T21:11:13.247420+0000","last_became_active":"2024-09-10T21:11:13.241793+0000","last_became_peered":"2024-09-10T21:11:13.241793+0000","last_unstale":"2024-09-10T21:11:13.247420+0000","last_undegraded":"2024-09-10T21:11:13.247420+0000","last_fullsized":"2024-09-10T21:11:13.247420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:01.678771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245420+0000","last_change":"2024-09-10T21:11:13.240633+0000","last_active":"2024-09-10T21:11:13.245420+0000","last_peered":"2024-09-10T21:11:13.245420+0000","last_clean":"2024-09-10T21:11:13.245420+0000","last_became_active":"2024-09-10T21:11:13.240046+0000","last_became_peered":"2024-09-10T21:11:13.240046+0000","last_unstale":"2024-09-10T21:11:13.245420+0000","last_undegraded":"2024-09-10T21:11:13.245420+0000","last_fullsized":"2024-09-10T21:11:13.245420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:30.065234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.237948+0000","last_change":"2024-09-10T21:11:13.219704+0000","last_active":"2024-09-10T21:11:13.237948+0000","last_peered":"2024-09-10T21:11:13.237948+0000","last_clean":"2024-09-10T21:11:13.237948+0000","last_became_active":"2024-09-10T21:11:13.218621+0000","last_became_peered":"2024-09-10T21:11:13.218621+0000","last_unstale":"2024-09-10T21:11:13.237948+0000","last_undegraded":"2024-09-10T21:11:13.237948+0000","last_fullsized":"2024-09-10T21:11:13.237948+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:54.179687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.245811+0000","last_change":"2024-09-10T21:11:13.232802+0000","last_active":"2024-09-10T21:11:13.245811+0000","last_peered":"2024-09-10T21:11:13.245811+0000","last_clean":"2024-09-10T21:11:13.245811+0000","last_became_active":"2024-09-10T21:11:13.227485+0000","last_became_peered":"2024-09-10T21:11:13.227485+0000","last_unstale":"2024-09-10T21:11:13.245811+0000","last_undegraded":"2024-09-10T21:11:13.245811+0000","last_fullsized":"2024-09-10T21:11:13.245811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:51.065370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.246021+0000","last_change":"2024-09-10T21:11:13.239036+0000","last_active":"2024-09-10T21:11:13.246021+0000","last_peered":"2024-09-10T21:11:13.246021+0000","last_clean":"2024-09-10T21:11:13.246021+0000","last_became_active":"2024-09-10T21:11:13.238795+0000","last_became_peered":"2024-09-10T21:11:13.238795+0000","last_unstale":"2024-09-10T21:11:13.246021+0000","last_undegraded":"2024-09-10T21:11:13.246021+0000","last_fullsized":"2024-09-10T21:11:13.246021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:32.087701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.243288+0000","last_change":"2024-09-10T21:11:13.235338+0000","last_active":"2024-09-10T21:11:13.243288+0000","last_peered":"2024-09-10T21:11:13.243288+0000","last_clean":"2024-09-10T21:11:13.243288+0000","last_became_active":"2024-09-10T21:11:13.231675+0000","last_became_peered":"2024-09-10T21:11:13.231675+0000","last_unstale":"2024-09-10T21:11:13.243288+0000","last_undegraded":"2024-09-10T21:11:13.243288+0000","last_fullsized":"2024-09-10T21:11:13.243288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:42.770316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.252762+0000","last_change":"2024-09-10T21:11:13.249311+0000","last_active":"2024-09-10T21:11:13.252762+0000","last_peered":"2024-09-10T21:11:13.252762+0000","last_clean":"2024-09-10T21:11:13.252762+0000","last_became_active":"2024-09-10T21:11:13.248813+0000","last_became_peered":"2024-09-10T21:11:13.248813+0000","last_unstale":"2024-09-10T21:11:13.252762+0000","last_undegraded":"2024-09-10T21:11:13.252762+0000","last_fullsized":"2024-09-10T21:11:13.252762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:18.708305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.249039+0000","last_change":"2024-09-10T21:11:13.248118+0000","last_active":"2024-09-10T21:11:13.249039+0000","last_peered":"2024-09-10T21:11:13.249039+0000","last_clean":"2024-09-10T21:11:13.249039+0000","last_became_active":"2024-09-10T21:11:13.247828+0000","last_became_peered":"2024-09-10T21:11:13.247828+0000","last_unstale":"2024-09-10T21:11:13.249039+0000","last_undegraded":"2024-09-10T21:11:13.249039+0000","last_fullsized":"2024-09-10T21:11:13.249039+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:30.684628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.234753+0000","last_change":"2024-09-10T21:11:13.231198+0000","last_active":"2024-09-10T21:11:13.234753+0000","last_peered":"2024-09-10T21:11:13.234753+0000","last_clean":"2024-09-10T21:11:13.234753+0000","last_became_active":"2024-09-10T21:11:13.229553+0000","last_became_peered":"2024-09-10T21:11:13.229553+0000","last_unstale":"2024-09-10T21:11:13.234753+0000","last_undegraded":"2024-09-10T21:11:13.234753+0000","last_fullsized":"2024-09-10T21:11:13.234753+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:40.266888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.222617+0000","last_change":"2024-09-10T21:11:13.220392+0000","last_active":"2024-09-10T21:11:13.222617+0000","last_peered":"2024-09-10T21:11:13.222617+0000","last_clean":"2024-09-10T21:11:13.222617+0000","last_became_active":"2024-09-10T21:11:13.220117+0000","last_became_peered":"2024-09-10T21:11:13.220117+0000","last_unstale":"2024-09-10T21:11:13.222617+0000","last_undegraded":"2024-09-10T21:11:13.222617+0000","last_fullsized":"2024-09-10T21:11:13.222617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:31.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.234865+0000","last_change":"2024-09-10T21:11:13.224627+0000","last_active":"2024-09-10T21:11:13.234865+0000","last_peered":"2024-09-10T21:11:13.234865+0000","last_clean":"2024-09-10T21:11:13.234865+0000","last_became_active":"2024-09-10T21:11:13.224165+0000","last_became_peered":"2024-09-10T21:11:13.224165+0000","last_unstale":"2024-09-10T21:11:13.234865+0000","last_undegraded":"2024-09-10T21:11:13.234865+0000","last_fullsized":"2024-09-10T21:11:13.234865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:49.105483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.256451+0000","last_change":"2024-09-10T21:11:12.225366+0000","last_active":"2024-09-10T21:11:13.256451+0000","last_peered":"2024-09-10T21:11:13.256451+0000","last_clean":"2024-09-10T21:11:13.256451+0000","last_became_active":"2024-09-10T21:11:12.223302+0000","last_became_peered":"2024-09-10T21:11:12.223302+0000","last_unstale":"2024-09-10T21:11:13.256451+0000","last_undegraded":"2024-09-10T21:11:13.256451+0000","last_fullsized":"2024-09-10T21:11:13.256451+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:27.847102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225535+0000","last_change":"2024-09-10T21:11:13.221051+0000","last_active":"2024-09-10T21:11:13.225535+0000","last_peered":"2024-09-10T21:11:13.225535+0000","last_clean":"2024-09-10T21:11:13.225535+0000","last_became_active":"2024-09-10T21:11:13.220762+0000","last_became_peered":"2024-09-10T21:11:13.220762+0000","last_unstale":"2024-09-10T21:11:13.225535+0000","last_undegraded":"2024-09-10T21:11:13.225535+0000","last_fullsized":"2024-09-10T21:11:13.225535+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:09.256749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248834+0000","last_change":"2024-09-10T21:11:12.222288+0000","last_active":"2024-09-10T21:11:13.248834+0000","last_peered":"2024-09-10T21:11:13.248834+0000","last_clean":"2024-09-10T21:11:13.248834+0000","last_became_active":"2024-09-10T21:11:12.221928+0000","last_became_peered":"2024-09-10T21:11:12.221928+0000","last_unstale":"2024-09-10T21:11:13.248834+0000","last_undegraded":"2024-09-10T21:11:13.248834+0000","last_fullsized":"2024-09-10T21:11:13.248834+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:35.884414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225276+0000","last_change":"2024-09-10T21:11:13.219137+0000","last_active":"2024-09-10T21:11:13.225276+0000","last_peered":"2024-09-10T21:11:13.225276+0000","last_clean":"2024-09-10T21:11:13.225276+0000","last_became_active":"2024-09-10T21:11:13.218827+0000","last_became_peered":"2024-09-10T21:11:13.218827+0000","last_unstale":"2024-09-10T21:11:13.225276+0000","last_undegraded":"2024-09-10T21:11:13.225276+0000","last_fullsized":"2024-09-10T21:11:13.225276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:47.276393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.228590+0000","last_change":"2024-09-10T21:11:12.223630+0000","last_active":"2024-09-10T21:11:12.228590+0000","last_peered":"2024-09-10T21:11:12.228590+0000","last_clean":"2024-09-10T21:11:12.228590+0000","last_became_active":"2024-09-10T21:11:12.223352+0000","last_became_peered":"2024-09-10T21:11:12.223352+0000","last_unstale":"2024-09-10T21:11:12.228590+0000","last_undegraded":"2024-09-10T21:11:12.228590+0000","last_fullsized":"2024-09-10T21:11:12.228590+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:20.065424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.249650+0000","last_change":"2024-09-10T21:11:13.236750+0000","last_active":"2024-09-10T21:11:13.249650+0000","last_peered":"2024-09-10T21:11:13.249650+0000","last_clean":"2024-09-10T21:11:13.249650+0000","last_became_active":"2024-09-10T21:11:13.236417+0000","last_became_peered":"2024-09-10T21:11:13.236417+0000","last_unstale":"2024-09-10T21:11:13.249650+0000","last_undegraded":"2024-09-10T21:11:13.249650+0000","last_fullsized":"2024-09-10T21:11:13.249650+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:30.284239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.250861+0000","last_change":"2024-09-10T21:11:12.223783+0000","last_active":"2024-09-10T21:11:13.250861+0000","last_peered":"2024-09-10T21:11:13.250861+0000","last_clean":"2024-09-10T21:11:13.250861+0000","last_became_active":"2024-09-10T21:11:12.223509+0000","last_became_peered":"2024-09-10T21:11:12.223509+0000","last_unstale":"2024-09-10T21:11:13.250861+0000","last_undegraded":"2024-09-10T21:11:13.250861+0000","last_fullsized":"2024-09-10T21:11:13.250861+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:14.818515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.205623+0000","last_change":"2024-09-10T21:11:12.205623+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.205623+0000","last_undegraded":"2024-09-10T21:11:12.205623+0000","last_fullsized":"2024-09-10T21:11:12.205623+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247537+0000","last_change":"2024-09-10T21:11:12.212357+0000","last_active":"2024-09-10T21:11:13.247537+0000","last_peered":"2024-09-10T21:11:13.247537+0000","last_clean":"2024-09-10T21:11:13.247537+0000","last_became_active":"2024-09-10T21:11:12.209827+0000","last_became_peered":"2024-09-10T21:11:12.209827+0000","last_unstale":"2024-09-10T21:11:13.247537+0000","last_undegraded":"2024-09-10T21:11:13.247537+0000","last_fullsized":"2024-09-10T21:11:13.247537+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:38.717269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.225721+0000","last_change":"2024-09-10T21:11:13.222468+0000","last_active":"2024-09-10T21:11:13.225721+0000","last_peered":"2024-09-10T21:11:13.225721+0000","last_clean":"2024-09-10T21:11:13.225721+0000","last_became_active":"2024-09-10T21:11:13.221546+0000","last_became_peered":"2024-09-10T21:11:13.221546+0000","last_unstale":"2024-09-10T21:11:13.225721+0000","last_undegraded":"2024-09-10T21:11:13.225721+0000","last_fullsized":"2024-09-10T21:11:13.225721+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:03.734116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.251808+0000","last_change":"2024-09-10T21:11:12.225033+0000","last_active":"2024-09-10T21:11:13.251808+0000","last_peered":"2024-09-10T21:11:13.251808+0000","last_clean":"2024-09-10T21:11:13.251808+0000","last_became_active":"2024-09-10T21:11:12.224007+0000","last_became_peered":"2024-09-10T21:11:12.224007+0000","last_unstale":"2024-09-10T21:11:13.251808+0000","last_undegraded":"2024-09-10T21:11:13.251808+0000","last_fullsized":"2024-09-10T21:11:13.251808+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:37.774571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.205574+0000","last_change":"2024-09-10T21:11:12.205574+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.205574+0000","last_undegraded":"2024-09-10T21:11:12.205574+0000","last_fullsized":"2024-09-10T21:11:12.205574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.227968+0000","last_change":"2024-09-10T21:11:12.221685+0000","last_active":"2024-09-10T21:11:12.227968+0000","last_peered":"2024-09-10T21:11:12.227968+0000","last_clean":"2024-09-10T21:11:12.227968+0000","last_became_active":"2024-09-10T21:11:12.221393+0000","last_became_peered":"2024-09-10T21:11:12.221393+0000","last_unstale":"2024-09-10T21:11:12.227968+0000","last_undegraded":"2024-09-10T21:11:12.227968+0000","last_fullsized":"2024-09-10T21:11:12.227968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:09.242206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248043+0000","last_change":"2024-09-10T21:11:13.234906+0000","last_active":"2024-09-10T21:11:13.248043+0000","last_peered":"2024-09-10T21:11:13.248043+0000","last_clean":"2024-09-10T21:11:13.248043+0000","last_became_active":"2024-09-10T21:11:13.228823+0000","last_became_peered":"2024-09-10T21:11:13.228823+0000","last_unstale":"2024-09-10T21:11:13.248043+0000","last_undegraded":"2024-09-10T21:11:13.248043+0000","last_fullsized":"2024-09-10T21:11:13.248043+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:36.486425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.209235+0000","last_change":"2024-09-10T21:11:12.224827+0000","last_active":"2024-09-10T21:11:13.209235+0000","last_peered":"2024-09-10T21:11:13.209235+0000","last_clean":"2024-09-10T21:11:13.209235+0000","last_became_active":"2024-09-10T21:11:12.224356+0000","last_became_peered":"2024-09-10T21:11:12.224356+0000","last_unstale":"2024-09-10T21:11:13.209235+0000","last_undegraded":"2024-09-10T21:11:13.209235+0000","last_fullsized":"2024-09-10T21:11:13.209235+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:00.054106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.252576+0000","last_change":"2024-09-10T21:11:13.233922+0000","last_active":"2024-09-10T21:11:13.252576+0000","last_peered":"2024-09-10T21:11:13.252576+0000","last_clean":"2024-09-10T21:11:13.252576+0000","last_became_active":"2024-09-10T21:11:13.230732+0000","last_became_peered":"2024-09-10T21:11:13.230732+0000","last_unstale":"2024-09-10T21:11:13.252576+0000","last_undegraded":"2024-09-10T21:11:13.252576+0000","last_fullsized":"2024-09-10T21:11:13.252576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:07.175259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.213442+0000","last_change":"2024-09-10T21:11:12.221271+0000","last_active":"2024-09-10T21:11:13.213442+0000","last_peered":"2024-09-10T21:11:13.213442+0000","last_clean":"2024-09-10T21:11:13.213442+0000","last_became_active":"2024-09-10T21:11:12.220975+0000","last_became_peered":"2024-09-10T21:11:12.220975+0000","last_unstale":"2024-09-10T21:11:13.213442+0000","last_undegraded":"2024-09-10T21:11:13.213442+0000","last_fullsized":"2024-09-10T21:11:13.213442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:29.280746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.247971+0000","last_change":"2024-09-10T21:11:13.234418+0000","last_active":"2024-09-10T21:11:13.247971+0000","last_peered":"2024-09-10T21:11:13.247971+0000","last_clean":"2024-09-10T21:11:13.247971+0000","last_became_active":"2024-09-10T21:11:13.234159+0000","last_became_peered":"2024-09-10T21:11:13.234159+0000","last_unstale":"2024-09-10T21:11:13.247971+0000","last_undegraded":"2024-09-10T21:11:13.247971+0000","last_fullsized":"2024-09-10T21:11:13.247971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:33.043662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.211318+0000","last_change":"2024-09-10T21:11:12.222257+0000","last_active":"2024-09-10T21:11:13.211318+0000","last_peered":"2024-09-10T21:11:13.211318+0000","last_clean":"2024-09-10T21:11:13.211318+0000","last_became_active":"2024-09-10T21:11:12.221959+0000","last_became_peered":"2024-09-10T21:11:12.221959+0000","last_unstale":"2024-09-10T21:11:13.211318+0000","last_undegraded":"2024-09-10T21:11:13.211318+0000","last_fullsized":"2024-09-10T21:11:13.211318+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:35.482932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.240430+0000","last_change":"2024-09-10T21:11:13.220496+0000","last_active":"2024-09-10T21:11:13.240430+0000","last_peered":"2024-09-10T21:11:13.240430+0000","last_clean":"2024-09-10T21:11:13.240430+0000","last_became_active":"2024-09-10T21:11:13.218857+0000","last_became_peered":"2024-09-10T21:11:13.218857+0000","last_unstale":"2024-09-10T21:11:13.240430+0000","last_undegraded":"2024-09-10T21:11:13.240430+0000","last_fullsized":"2024-09-10T21:11:13.240430+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:08.219546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254368+0000","last_change":"2024-09-10T21:11:12.229925+0000","last_active":"2024-09-10T21:11:13.254368+0000","last_peered":"2024-09-10T21:11:13.254368+0000","last_clean":"2024-09-10T21:11:13.254368+0000","last_became_active":"2024-09-10T21:11:12.229037+0000","last_became_peered":"2024-09-10T21:11:12.229037+0000","last_unstale":"2024-09-10T21:11:13.254368+0000","last_undegraded":"2024-09-10T21:11:13.254368+0000","last_fullsized":"2024-09-10T21:11:13.254368+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:43.936262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.251337+0000","last_change":"2024-09-10T21:11:13.233489+0000","last_active":"2024-09-10T21:11:13.251337+0000","last_peered":"2024-09-10T21:11:13.251337+0000","last_clean":"2024-09-10T21:11:13.251337+0000","last_became_active":"2024-09-10T21:11:13.232909+0000","last_became_peered":"2024-09-10T21:11:13.232909+0000","last_unstale":"2024-09-10T21:11:13.251337+0000","last_undegraded":"2024-09-10T21:11:13.251337+0000","last_fullsized":"2024-09-10T21:11:13.251337+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:15.390510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.256092+0000","last_change":"2024-09-10T21:11:12.230626+0000","last_active":"2024-09-10T21:11:13.256092+0000","last_peered":"2024-09-10T21:11:13.256092+0000","last_clean":"2024-09-10T21:11:13.256092+0000","last_became_active":"2024-09-10T21:11:12.230308+0000","last_became_peered":"2024-09-10T21:11:12.230308+0000","last_unstale":"2024-09-10T21:11:13.256092+0000","last_undegraded":"2024-09-10T21:11:13.256092+0000","last_fullsized":"2024-09-10T21:11:13.256092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:09.728148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.250370+0000","last_change":"2024-09-10T21:11:13.242994+0000","last_active":"2024-09-10T21:11:13.250370+0000","last_peered":"2024-09-10T21:11:13.250370+0000","last_clean":"2024-09-10T21:11:13.250370+0000","last_became_active":"2024-09-10T21:11:13.242683+0000","last_became_peered":"2024-09-10T21:11:13.242683+0000","last_unstale":"2024-09-10T21:11:13.250370+0000","last_undegraded":"2024-09-10T21:11:13.250370+0000","last_fullsized":"2024-09-10T21:11:13.250370+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:04.248893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254320+0000","last_change":"2024-09-10T21:11:12.224674+0000","last_active":"2024-09-10T21:11:13.254320+0000","last_peered":"2024-09-10T21:11:13.254320+0000","last_clean":"2024-09-10T21:11:13.254320+0000","last_became_active":"2024-09-10T21:11:12.224325+0000","last_became_peered":"2024-09-10T21:11:12.224325+0000","last_unstale":"2024-09-10T21:11:13.254320+0000","last_undegraded":"2024-09-10T21:11:13.254320+0000","last_fullsized":"2024-09-10T21:11:13.254320+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:06.355342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248957+0000","last_change":"2024-09-10T21:11:13.236816+0000","last_active":"2024-09-10T21:11:13.248957+0000","last_peered":"2024-09-10T21:11:13.248957+0000","last_clean":"2024-09-10T21:11:13.248957+0000","last_became_active":"2024-09-10T21:11:13.236502+0000","last_became_peered":"2024-09-10T21:11:13.236502+0000","last_unstale":"2024-09-10T21:11:13.248957+0000","last_undegraded":"2024-09-10T21:11:13.248957+0000","last_fullsized":"2024-09-10T21:11:13.248957+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:55.941374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.237205+0000","last_change":"2024-09-10T21:11:12.226932+0000","last_active":"2024-09-10T21:11:13.237205+0000","last_peered":"2024-09-10T21:11:13.237205+0000","last_clean":"2024-09-10T21:11:13.237205+0000","last_became_active":"2024-09-10T21:11:12.226622+0000","last_became_peered":"2024-09-10T21:11:12.226622+0000","last_unstale":"2024-09-10T21:11:13.237205+0000","last_undegraded":"2024-09-10T21:11:13.237205+0000","last_fullsized":"2024-09-10T21:11:13.237205+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:15.020787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239479+0000","last_change":"2024-09-10T21:11:13.226536+0000","last_active":"2024-09-10T21:11:13.239479+0000","last_peered":"2024-09-10T21:11:13.239479+0000","last_clean":"2024-09-10T21:11:13.239479+0000","last_became_active":"2024-09-10T21:11:13.226204+0000","last_became_peered":"2024-09-10T21:11:13.226204+0000","last_unstale":"2024-09-10T21:11:13.239479+0000","last_undegraded":"2024-09-10T21:11:13.239479+0000","last_fullsized":"2024-09-10T21:11:13.239479+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:28.355449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.221557+0000","last_change":"2024-09-10T21:11:12.222663+0000","last_active":"2024-09-10T21:11:13.221557+0000","last_peered":"2024-09-10T21:11:13.221557+0000","last_clean":"2024-09-10T21:11:13.221557+0000","last_became_active":"2024-09-10T21:11:12.222491+0000","last_became_peered":"2024-09-10T21:11:12.222491+0000","last_unstale":"2024-09-10T21:11:13.221557+0000","last_undegraded":"2024-09-10T21:11:13.221557+0000","last_fullsized":"2024-09-10T21:11:13.221557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:26.481636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.221362+0000","last_change":"2024-09-10T21:11:13.218025+0000","last_active":"2024-09-10T21:11:13.221362+0000","last_peered":"2024-09-10T21:11:13.221362+0000","last_clean":"2024-09-10T21:11:13.221362+0000","last_became_active":"2024-09-10T21:11:13.217119+0000","last_became_peered":"2024-09-10T21:11:13.217119+0000","last_unstale":"2024-09-10T21:11:13.221362+0000","last_undegraded":"2024-09-10T21:11:13.221362+0000","last_fullsized":"2024-09-10T21:11:13.221362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:42.864605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.229482+0000","last_change":"2024-09-10T21:11:12.225066+0000","last_active":"2024-09-10T21:11:12.229482+0000","last_peered":"2024-09-10T21:11:12.229482+0000","last_clean":"2024-09-10T21:11:12.229482+0000","last_became_active":"2024-09-10T21:11:12.224555+0000","last_became_peered":"2024-09-10T21:11:12.224555+0000","last_unstale":"2024-09-10T21:11:12.229482+0000","last_undegraded":"2024-09-10T21:11:12.229482+0000","last_fullsized":"2024-09-10T21:11:12.229482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:51.343919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.252602+0000","last_change":"2024-09-10T21:11:13.238225+0000","last_active":"2024-09-10T21:11:13.252602+0000","last_peered":"2024-09-10T21:11:13.252602+0000","last_clean":"2024-09-10T21:11:13.252602+0000","last_became_active":"2024-09-10T21:11:13.236412+0000","last_became_peered":"2024-09-10T21:11:13.236412+0000","last_unstale":"2024-09-10T21:11:13.252602+0000","last_undegraded":"2024-09-10T21:11:13.252602+0000","last_fullsized":"2024-09-10T21:11:13.252602+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:48.715087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.229803+0000","last_change":"2024-09-10T21:11:12.228489+0000","last_active":"2024-09-10T21:11:12.229803+0000","last_peered":"2024-09-10T21:11:12.229803+0000","last_clean":"2024-09-10T21:11:12.229803+0000","last_became_active":"2024-09-10T21:11:12.228136+0000","last_became_peered":"2024-09-10T21:11:12.228136+0000","last_unstale":"2024-09-10T21:11:12.229803+0000","last_undegraded":"2024-09-10T21:11:12.229803+0000","last_fullsized":"2024-09-10T21:11:12.229803+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:12.504083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.206728+0000","last_change":"2024-09-10T21:11:12.206728+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.206728+0000","last_undegraded":"2024-09-10T21:11:12.206728+0000","last_fullsized":"2024-09-10T21:11:12.206728+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254662+0000","last_change":"2024-09-10T21:11:12.222819+0000","last_active":"2024-09-10T21:11:13.254662+0000","last_peered":"2024-09-10T21:11:13.254662+0000","last_clean":"2024-09-10T21:11:13.254662+0000","last_became_active":"2024-09-10T21:11:12.222499+0000","last_became_peered":"2024-09-10T21:11:12.222499+0000","last_unstale":"2024-09-10T21:11:13.254662+0000","last_undegraded":"2024-09-10T21:11:13.254662+0000","last_fullsized":"2024-09-10T21:11:13.254662+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:11.810704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.229207+0000","last_change":"2024-09-10T21:11:13.227904+0000","last_active":"2024-09-10T21:11:13.229207+0000","last_peered":"2024-09-10T21:11:13.229207+0000","last_clean":"2024-09-10T21:11:13.229207+0000","last_became_active":"2024-09-10T21:11:13.227548+0000","last_became_peered":"2024-09-10T21:11:13.227548+0000","last_unstale":"2024-09-10T21:11:13.229207+0000","last_undegraded":"2024-09-10T21:11:13.229207+0000","last_fullsized":"2024-09-10T21:11:13.229207+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:36.887907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.254448+0000","last_change":"2024-09-10T21:11:12.230259+0000","last_active":"2024-09-10T21:11:13.254448+0000","last_peered":"2024-09-10T21:11:13.254448+0000","last_clean":"2024-09-10T21:11:13.254448+0000","last_became_active":"2024-09-10T21:11:12.229526+0000","last_became_peered":"2024-09-10T21:11:12.229526+0000","last_unstale":"2024-09-10T21:11:13.254448+0000","last_undegraded":"2024-09-10T21:11:13.254448+0000","last_fullsized":"2024-09-10T21:11:13.254448+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:26.566827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.209289+0000","last_change":"2024-09-10T21:11:12.209289+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.209289+0000","last_undegraded":"2024-09-10T21:11:12.209289+0000","last_fullsized":"2024-09-10T21:11:12.209289+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.236111+0000","last_change":"2024-09-10T21:11:12.230126+0000","last_active":"2024-09-10T21:11:13.236111+0000","last_peered":"2024-09-10T21:11:13.236111+0000","last_clean":"2024-09-10T21:11:13.236111+0000","last_became_active":"2024-09-10T21:11:12.229860+0000","last_became_peered":"2024-09-10T21:11:12.229860+0000","last_unstale":"2024-09-10T21:11:13.236111+0000","last_undegraded":"2024-09-10T21:11:13.236111+0000","last_fullsized":"2024-09-10T21:11:13.236111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:09.292762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.250343+0000","last_change":"2024-09-10T21:11:13.241293+0000","last_active":"2024-09-10T21:11:13.250343+0000","last_peered":"2024-09-10T21:11:13.250343+0000","last_clean":"2024-09-10T21:11:13.250343+0000","last_became_active":"2024-09-10T21:11:13.240911+0000","last_became_peered":"2024-09-10T21:11:13.240911+0000","last_unstale":"2024-09-10T21:11:13.250343+0000","last_undegraded":"2024-09-10T21:11:13.250343+0000","last_fullsized":"2024-09-10T21:11:13.250343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:48.860100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248317+0000","last_change":"2024-09-10T21:11:12.221176+0000","last_active":"2024-09-10T21:11:13.248317+0000","last_peered":"2024-09-10T21:11:13.248317+0000","last_clean":"2024-09-10T21:11:13.248317+0000","last_became_active":"2024-09-10T21:11:12.220902+0000","last_became_peered":"2024-09-10T21:11:12.220902+0000","last_unstale":"2024-09-10T21:11:13.248317+0000","last_undegraded":"2024-09-10T21:11:13.248317+0000","last_fullsized":"2024-09-10T21:11:13.248317+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:54.937881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.227103+0000","last_change":"2024-09-10T21:11:13.224634+0000","last_active":"2024-09-10T21:11:13.227103+0000","last_peered":"2024-09-10T21:11:13.227103+0000","last_clean":"2024-09-10T21:11:13.227103+0000","last_became_active":"2024-09-10T21:11:13.224333+0000","last_became_peered":"2024-09-10T21:11:13.224333+0000","last_unstale":"2024-09-10T21:11:13.227103+0000","last_undegraded":"2024-09-10T21:11:13.227103+0000","last_fullsized":"2024-09-10T21:11:13.227103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:16.024280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.209741+0000","last_change":"2024-09-10T21:11:12.226970+0000","last_active":"2024-09-10T21:11:13.209741+0000","last_peered":"2024-09-10T21:11:13.209741+0000","last_clean":"2024-09-10T21:11:13.209741+0000","last_became_active":"2024-09-10T21:11:12.226548+0000","last_became_peered":"2024-09-10T21:11:12.226548+0000","last_unstale":"2024-09-10T21:11:13.209741+0000","last_undegraded":"2024-09-10T21:11:13.209741+0000","last_fullsized":"2024-09-10T21:11:13.209741+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:20.427638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.255041+0000","last_change":"2024-09-10T21:11:13.253121+0000","last_active":"2024-09-10T21:11:13.255041+0000","last_peered":"2024-09-10T21:11:13.255041+0000","last_clean":"2024-09-10T21:11:13.255041+0000","last_became_active":"2024-09-10T21:11:13.252833+0000","last_became_peered":"2024-09-10T21:11:13.252833+0000","last_unstale":"2024-09-10T21:11:13.255041+0000","last_undegraded":"2024-09-10T21:11:13.255041+0000","last_fullsized":"2024-09-10T21:11:13.255041+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:19.718594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.220466+0000","last_change":"2024-09-10T21:11:12.215786+0000","last_active":"2024-09-10T21:11:12.220466+0000","last_peered":"2024-09-10T21:11:12.220466+0000","last_clean":"2024-09-10T21:11:12.220466+0000","last_became_active":"2024-09-10T21:11:12.215497+0000","last_became_peered":"2024-09-10T21:11:12.215497+0000","last_unstale":"2024-09-10T21:11:12.220466+0000","last_undegraded":"2024-09-10T21:11:12.220466+0000","last_fullsized":"2024-09-10T21:11:12.220466+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:08.359968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.205996+0000","last_change":"2024-09-10T21:11:12.205996+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.205996+0000","last_undegraded":"2024-09-10T21:11:12.205996+0000","last_fullsized":"2024-09-10T21:11:12.205996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.212985+0000","last_change":"2024-09-10T21:11:12.234069+0000","last_active":"2024-09-10T21:11:13.212985+0000","last_peered":"2024-09-10T21:11:13.212985+0000","last_clean":"2024-09-10T21:11:13.212985+0000","last_became_active":"2024-09-10T21:11:12.233714+0000","last_became_peered":"2024-09-10T21:11:12.233714+0000","last_unstale":"2024-09-10T21:11:13.212985+0000","last_undegraded":"2024-09-10T21:11:13.212985+0000","last_fullsized":"2024-09-10T21:11:13.212985+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.570509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.210725+0000","last_change":"2024-09-10T21:11:12.210725+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.210725+0000","last_undegraded":"2024-09-10T21:11:12.210725+0000","last_fullsized":"2024-09-10T21:11:12.210725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.241476+0000","last_change":"2024-09-10T21:11:12.223676+0000","last_active":"2024-09-10T21:11:13.241476+0000","last_peered":"2024-09-10T21:11:13.241476+0000","last_clean":"2024-09-10T21:11:13.241476+0000","last_became_active":"2024-09-10T21:11:12.222053+0000","last_became_peered":"2024-09-10T21:11:12.222053+0000","last_unstale":"2024-09-10T21:11:13.241476+0000","last_undegraded":"2024-09-10T21:11:13.241476+0000","last_fullsized":"2024-09-10T21:11:13.241476+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:51.453841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.228362+0000","last_change":"2024-09-10T21:11:13.223973+0000","last_active":"2024-09-10T21:11:13.228362+0000","last_peered":"2024-09-10T21:11:13.228362+0000","last_clean":"2024-09-10T21:11:13.228362+0000","last_became_active":"2024-09-10T21:11:13.223701+0000","last_became_peered":"2024-09-10T21:11:13.223701+0000","last_unstale":"2024-09-10T21:11:13.228362+0000","last_undegraded":"2024-09-10T21:11:13.228362+0000","last_fullsized":"2024-09-10T21:11:13.228362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:57.527538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.211786+0000","last_change":"2024-09-10T21:11:12.222106+0000","last_active":"2024-09-10T21:11:13.211786+0000","last_peered":"2024-09-10T21:11:13.211786+0000","last_clean":"2024-09-10T21:11:13.211786+0000","last_became_active":"2024-09-10T21:11:12.221770+0000","last_became_peered":"2024-09-10T21:11:12.221770+0000","last_unstale":"2024-09-10T21:11:13.211786+0000","last_undegraded":"2024-09-10T21:11:13.211786+0000","last_fullsized":"2024-09-10T21:11:13.211786+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:07.999372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-10T21:11:12.208795+0000","last_change":"2024-09-10T21:11:12.208795+0000","last_active":"2024-09-10T21:11:12.194803+0000","last_peered":"2024-09-10T21:11:12.194803+0000","last_clean":"2024-09-10T21:11:12.194803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:11:12.208795+0000","last_undegraded":"2024-09-10T21:11:12.208795+0000","last_fullsized":"2024-09-10T21:11:12.208795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.227006+0000","last_change":"2024-09-10T21:11:12.229500+0000","last_active":"2024-09-10T21:11:13.227006+0000","last_peered":"2024-09-10T21:11:13.227006+0000","last_clean":"2024-09-10T21:11:13.227006+0000","last_became_active":"2024-09-10T21:11:12.229152+0000","last_became_peered":"2024-09-10T21:11:12.229152+0000","last_unstale":"2024-09-10T21:11:13.227006+0000","last_undegraded":"2024-09-10T21:11:13.227006+0000","last_fullsized":"2024-09-10T21:11:13.227006+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:11.772150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239465+0000","last_change":"2024-09-10T21:11:13.232669+0000","last_active":"2024-09-10T21:11:13.239465+0000","last_peered":"2024-09-10T21:11:13.239465+0000","last_clean":"2024-09-10T21:11:13.239465+0000","last_became_active":"2024-09-10T21:11:13.232049+0000","last_became_peered":"2024-09-10T21:11:13.232049+0000","last_unstale":"2024-09-10T21:11:13.239465+0000","last_undegraded":"2024-09-10T21:11:13.239465+0000","last_fullsized":"2024-09-10T21:11:13.239465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:57.314451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.231085+0000","last_change":"2024-09-10T21:11:12.224196+0000","last_active":"2024-09-10T21:11:13.231085+0000","last_peered":"2024-09-10T21:11:13.231085+0000","last_clean":"2024-09-10T21:11:13.231085+0000","last_became_active":"2024-09-10T21:11:12.223853+0000","last_became_peered":"2024-09-10T21:11:12.223853+0000","last_unstale":"2024-09-10T21:11:13.231085+0000","last_undegraded":"2024-09-10T21:11:13.231085+0000","last_fullsized":"2024-09-10T21:11:13.231085+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:23.761112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.248934+0000","last_change":"2024-09-10T21:11:13.236581+0000","last_active":"2024-09-10T21:11:13.248934+0000","last_peered":"2024-09-10T21:11:13.248934+0000","last_clean":"2024-09-10T21:11:13.248934+0000","last_became_active":"2024-09-10T21:11:13.236310+0000","last_became_peered":"2024-09-10T21:11:13.236310+0000","last_unstale":"2024-09-10T21:11:13.248934+0000","last_undegraded":"2024-09-10T21:11:13.248934+0000","last_fullsized":"2024-09-10T21:11:13.248934+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:09.002865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.220680+0000","last_change":"2024-09-10T21:11:12.224815+0000","last_active":"2024-09-10T21:11:13.220680+0000","last_peered":"2024-09-10T21:11:13.220680+0000","last_clean":"2024-09-10T21:11:13.220680+0000","last_became_active":"2024-09-10T21:11:12.224504+0000","last_became_peered":"2024-09-10T21:11:12.224504+0000","last_unstale":"2024-09-10T21:11:13.220680+0000","last_undegraded":"2024-09-10T21:11:13.220680+0000","last_fullsized":"2024-09-10T21:11:13.220680+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:29.112283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.246966+0000","last_change":"2024-09-10T21:11:13.226608+0000","last_active":"2024-09-10T21:11:13.246966+0000","last_peered":"2024-09-10T21:11:13.246966+0000","last_clean":"2024-09-10T21:11:13.246966+0000","last_became_active":"2024-09-10T21:11:13.226297+0000","last_became_peered":"2024-09-10T21:11:13.226297+0000","last_unstale":"2024-09-10T21:11:13.246966+0000","last_undegraded":"2024-09-10T21:11:13.246966+0000","last_fullsized":"2024-09-10T21:11:13.246966+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:34.526987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.208044+0000","last_change":"2024-09-10T21:11:12.222940+0000","last_active":"2024-09-10T21:11:13.208044+0000","last_peered":"2024-09-10T21:11:13.208044+0000","last_clean":"2024-09-10T21:11:13.208044+0000","last_became_active":"2024-09-10T21:11:12.222474+0000","last_became_peered":"2024-09-10T21:11:12.222474+0000","last_unstale":"2024-09-10T21:11:13.208044+0000","last_undegraded":"2024-09-10T21:11:13.208044+0000","last_fullsized":"2024-09-10T21:11:13.208044+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:32.362391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.237409+0000","last_change":"2024-09-10T21:11:13.228335+0000","last_active":"2024-09-10T21:11:13.237409+0000","last_peered":"2024-09-10T21:11:13.237409+0000","last_clean":"2024-09-10T21:11:13.237409+0000","last_became_active":"2024-09-10T21:11:13.228013+0000","last_became_peered":"2024-09-10T21:11:13.228013+0000","last_unstale":"2024-09-10T21:11:13.237409+0000","last_undegraded":"2024-09-10T21:11:13.237409+0000","last_fullsized":"2024-09-10T21:11:13.237409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:22.229299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.223563+0000","last_change":"2024-09-10T21:11:12.220700+0000","last_active":"2024-09-10T21:11:13.223563+0000","last_peered":"2024-09-10T21:11:13.223563+0000","last_clean":"2024-09-10T21:11:13.223563+0000","last_became_active":"2024-09-10T21:11:12.220364+0000","last_became_peered":"2024-09-10T21:11:12.220364+0000","last_unstale":"2024-09-10T21:11:13.223563+0000","last_undegraded":"2024-09-10T21:11:13.223563+0000","last_fullsized":"2024-09-10T21:11:13.223563+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:30.271305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.239565+0000","last_change":"2024-09-10T21:11:13.234881+0000","last_active":"2024-09-10T21:11:13.239565+0000","last_peered":"2024-09-10T21:11:13.239565+0000","last_clean":"2024-09-10T21:11:13.239565+0000","last_became_active":"2024-09-10T21:11:13.234666+0000","last_became_peered":"2024-09-10T21:11:13.234666+0000","last_unstale":"2024-09-10T21:11:13.239565+0000","last_undegraded":"2024-09-10T21:11:13.239565+0000","last_fullsized":"2024-09-10T21:11:13.239565+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:42.843772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:11:13.214147+0000","last_change":"2024-09-10T21:11:12.230047+0000","last_active":"2024-09-10T21:11:13.214147+0000","last_peered":"2024-09-10T21:11:13.214147+0000","last_clean":"2024-09-10T21:11:13.214147+0000","last_became_active":"2024-09-10T21:11:12.229249+0000","last_became_peered":"2024-09-10T21:11:12.229249+0000","last_unstale":"2024-09-10T21:11:13.214147+0000","last_undegraded":"2024-09-10T21:11:13.214147+0000","last_fullsized":"2024-09-10T21:11:13.214147+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:59.482848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":127,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-10T21:11:12.209277+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:11:12.209277+0000","last_peered":"2024-09-10T21:11:12.209277+0000","last_clean":"2024-09-10T21:11:12.209277+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:11:12.209277+0000","last_undegraded":"2024-09-10T21:11:12.209277+0000","last_fullsized":"2024-09-10T21:11:12.209277+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":937984,"data_stored":720037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":356352,"data_stored":130721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":356352,"data_stored":129863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":122312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542157,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":909312,"data_stored":706405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:11:21.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:11:22.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.149+0000 7f09a7e00640 1 -- 172.21.15.16:0/3838028091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a80d0c80 msgr2=0x7f09a80cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:22.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.149+0000 7f09a7e00640 1 --2- 172.21.15.16:0/3838028091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a80d0c80 0x7f09a80cea10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f09a808e590 tx=0x7f09a001cd80 comp rx=0 tx=0).stop 2024-09-10T21:11:22.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.149+0000 7f09a7e00640 1 -- 172.21.15.16:0/3838028091 shutdown_connections 2024-09-10T21:11:22.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.149+0000 7f09a7e00640 1 --2- 172.21.15.16:0/3838028091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a80d0c80 0x7f09a80cea10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.149+0000 7f09a7e00640 1 --2- 172.21.15.16:0/3838028091 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09a80b2c20 0x7f09a80d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.149+0000 7f09a7e00640 1 --2- 172.21.15.16:0/3838028091 >> v2:172.21.15.52:3300/0 conn(0x7f09a8161560 0x7f09a80d4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.149+0000 7f09a7e00640 1 -- 172.21.15.16:0/3838028091 >> 172.21.15.16:0/3838028091 conn(0x7f09a80d3f00 msgr2=0x7f09a80d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:22.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.149+0000 7f09a7e00640 1 -- 172.21.15.16:0/3838028091 shutdown_connections 2024-09-10T21:11:22.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.149+0000 7f09a7e00640 1 -- 172.21.15.16:0/3838028091 wait complete. 2024-09-10T21:11:22.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.149+0000 7f09a7e00640 1 Processor -- start 2024-09-10T21:11:22.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.149+0000 7f09a7e00640 1 -- start start 2024-09-10T21:11:22.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09a80b2c20 0x7f09a80c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:22.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09a80d0c80 0x7f09a80cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:22.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8161560 0x7f09a814c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:22.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09a80b18e0 con 0x7f09a8161560 2024-09-10T21:11:22.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09a80b1760 con 0x7f09a80d0c80 2024-09-10T21:11:22.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f09a80b1a60 con 0x7f09a80b2c20 2024-09-10T21:11:22.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8161560 0x7f09a814c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:22.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8161560 0x7f09a814c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43098/0 (socket says 172.21.15.16:43098) 2024-09-10T21:11:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7400640 1 -- 172.21.15.16:0/2633629799 learned_addr learned my addr 172.21.15.16:0/2633629799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a6a00640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09a80b2c20 0x7f09a80c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f099e000640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09a80d0c80 0x7f09a80cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7400640 1 -- 172.21.15.16:0/2633629799 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09a80b2c20 msgr2=0x7f09a80c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7400640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09a80b2c20 0x7f09a80c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7400640 1 -- 172.21.15.16:0/2633629799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09a80d0c80 msgr2=0x7f09a80cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7400640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09a80d0c80 0x7f09a80cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7400640 1 -- 172.21.15.16:0/2633629799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a80cb460 con 0x7f09a8161560 2024-09-10T21:11:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f099e000640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09a80d0c80 0x7f09a80cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7400640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8161560 0x7f09a814c310 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f09a001b430 tx=0x7f09a0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f099f400640 1 -- 172.21.15.16:0/2633629799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a00bb070 con 0x7f09a8161560 2024-09-10T21:11:22.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f099f400640 1 -- 172.21.15.16:0/2633629799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09a0004e00 con 0x7f09a8161560 2024-09-10T21:11:22.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7e00640 1 -- 172.21.15.16:0/2633629799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a80cb740 con 0x7f09a8161560 2024-09-10T21:11:22.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f099f400640 1 -- 172.21.15.16:0/2633629799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a00b2040 con 0x7f09a8161560 2024-09-10T21:11:22.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7e00640 1 -- 172.21.15.16:0/2633629799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a80cbd00 con 0x7f09a8161560 2024-09-10T21:11:22.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f09a7e00640 1 -- 172.21.15.16:0/2633629799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a80cea10 con 0x7f09a8161560 2024-09-10T21:11:22.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.153+0000 7f099f400640 1 -- 172.21.15.16:0/2633629799 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f09a00b5030 con 0x7f09a8161560 2024-09-10T21:11:22.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.157+0000 7f099f400640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09840725e0 0x7f0984074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:22.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.157+0000 7f099f400640 1 -- 172.21.15.16:0/2633629799 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f09a012ceb0 con 0x7f09a8161560 2024-09-10T21:11:22.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.157+0000 7f09a6a00640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09840725e0 0x7f0984074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:22.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.157+0000 7f09a6a00640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09840725e0 0x7f0984074aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f09a80a1f80 tx=0x7f099800b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:22.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.161+0000 7f099f400640 1 -- 172.21.15.16:0/2633629799 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f09a0016970 con 0x7f09a8161560 2024-09-10T21:11:22.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.329+0000 7f09a7e00640 1 -- 172.21.15.16:0/2633629799 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09a80d0310 con 0x7f09840725e0 2024-09-10T21:11:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.333+0000 7f099f400640 1 -- 172.21.15.16:0/2633629799 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f09a80d0310 con 0x7f09840725e0 2024-09-10T21:11:22.342 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:11:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.337+0000 7f09a7e00640 1 -- 172.21.15.16:0/2633629799 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09840725e0 msgr2=0x7f0984074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.337+0000 7f09a7e00640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09840725e0 0x7f0984074aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f09a80a1f80 tx=0x7f099800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.337+0000 7f09a7e00640 1 -- 172.21.15.16:0/2633629799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8161560 msgr2=0x7f09a814c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.337+0000 7f09a7e00640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8161560 0x7f09a814c310 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f09a001b430 tx=0x7f09a0003f60 comp rx=0 tx=0).stop 2024-09-10T21:11:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.337+0000 7f09a7e00640 1 -- 172.21.15.16:0/2633629799 shutdown_connections 2024-09-10T21:11:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.337+0000 7f09a7e00640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09840725e0 0x7f0984074aa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.337+0000 7f09a7e00640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8161560 0x7f09a814c310 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.337+0000 7f09a7e00640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09a80d0c80 0x7f09a80cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.337+0000 7f09a7e00640 1 --2- 172.21.15.16:0/2633629799 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09a80b2c20 0x7f09a80c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.337+0000 7f09a7e00640 1 -- 172.21.15.16:0/2633629799 >> 172.21.15.16:0/2633629799 conn(0x7f09a80d3f00 msgr2=0x7f09a80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.337+0000 7f09a7e00640 1 -- 172.21.15.16:0/2633629799 shutdown_connections 2024-09-10T21:11:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.337+0000 7f09a7e00640 1 -- 172.21.15.16:0/2633629799 wait complete. 2024-09-10T21:11:22.363 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-10T21:11:21.467205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253492,"kb_used_data":3976,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721228,"statfs":{"total":773094113280,"available":772834537472,"internally_reserved":0,"allocated":4071424,"data_stored":2268652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.261338"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208842+0000","last_change":"2024-09-10T21:11:13.244223+0000","last_active":"2024-09-10T21:11:14.208842+0000","last_peered":"2024-09-10T21:11:14.208842+0000","last_clean":"2024-09-10T21:11:14.208842+0000","last_became_active":"2024-09-10T21:11:13.243954+0000","last_became_peered":"2024-09-10T21:11:13.243954+0000","last_unstale":"2024-09-10T21:11:14.208842+0000","last_undegraded":"2024-09-10T21:11:14.208842+0000","last_fullsized":"2024-09-10T21:11:14.208842+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:00.486341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.209364+0000","last_change":"2024-09-10T21:11:13.247771+0000","last_active":"2024-09-10T21:11:14.209364+0000","last_peered":"2024-09-10T21:11:14.209364+0000","last_clean":"2024-09-10T21:11:14.209364+0000","last_became_active":"2024-09-10T21:11:13.247293+0000","last_became_peered":"2024-09-10T21:11:13.247293+0000","last_unstale":"2024-09-10T21:11:14.209364+0000","last_undegraded":"2024-09-10T21:11:14.209364+0000","last_fullsized":"2024-09-10T21:11:14.209364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:36.345868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.042833+0000","last_change":"2024-09-10T21:11:13.226646+0000","last_active":"2024-09-10T21:11:15.042833+0000","last_peered":"2024-09-10T21:11:15.042833+0000","last_clean":"2024-09-10T21:11:15.042833+0000","last_became_active":"2024-09-10T21:11:13.226274+0000","last_became_peered":"2024-09-10T21:11:13.226274+0000","last_unstale":"2024-09-10T21:11:15.042833+0000","last_undegraded":"2024-09-10T21:11:15.042833+0000","last_fullsized":"2024-09-10T21:11:15.042833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:14.847686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.209582+0000","last_change":"2024-09-10T21:11:13.233899+0000","last_active":"2024-09-10T21:11:14.209582+0000","last_peered":"2024-09-10T21:11:14.209582+0000","last_clean":"2024-09-10T21:11:14.209582+0000","last_became_active":"2024-09-10T21:11:13.219250+0000","last_became_peered":"2024-09-10T21:11:13.219250+0000","last_unstale":"2024-09-10T21:11:14.209582+0000","last_undegraded":"2024-09-10T21:11:14.209582+0000","last_fullsized":"2024-09-10T21:11:14.209582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:38.050493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208483+0000","last_change":"2024-09-10T21:11:13.235979+0000","last_active":"2024-09-10T21:11:14.208483+0000","last_peered":"2024-09-10T21:11:14.208483+0000","last_clean":"2024-09-10T21:11:14.208483+0000","last_became_active":"2024-09-10T21:11:13.233126+0000","last_became_peered":"2024-09-10T21:11:13.233126+0000","last_unstale":"2024-09-10T21:11:14.208483+0000","last_undegraded":"2024-09-10T21:11:14.208483+0000","last_fullsized":"2024-09-10T21:11:14.208483+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:21.959614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.216984+0000","last_change":"2024-09-10T21:11:13.226107+0000","last_active":"2024-09-10T21:11:14.216984+0000","last_peered":"2024-09-10T21:11:14.216984+0000","last_clean":"2024-09-10T21:11:14.216984+0000","last_became_active":"2024-09-10T21:11:13.225766+0000","last_became_peered":"2024-09-10T21:11:13.225766+0000","last_unstale":"2024-09-10T21:11:14.216984+0000","last_undegraded":"2024-09-10T21:11:14.216984+0000","last_fullsized":"2024-09-10T21:11:14.216984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:49.996905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208282+0000","last_change":"2024-09-10T21:11:13.232062+0000","last_active":"2024-09-10T21:11:14.208282+0000","last_peered":"2024-09-10T21:11:14.208282+0000","last_clean":"2024-09-10T21:11:14.208282+0000","last_became_active":"2024-09-10T21:11:13.231580+0000","last_became_peered":"2024-09-10T21:11:13.231580+0000","last_unstale":"2024-09-10T21:11:14.208282+0000","last_undegraded":"2024-09-10T21:11:14.208282+0000","last_fullsized":"2024-09-10T21:11:14.208282+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:54.238938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208081+0000","last_change":"2024-09-10T21:11:13.240369+0000","last_active":"2024-09-10T21:11:14.208081+0000","last_peered":"2024-09-10T21:11:14.208081+0000","last_clean":"2024-09-10T21:11:14.208081+0000","last_became_active":"2024-09-10T21:11:13.239717+0000","last_became_peered":"2024-09-10T21:11:13.239717+0000","last_unstale":"2024-09-10T21:11:14.208081+0000","last_undegraded":"2024-09-10T21:11:14.208081+0000","last_fullsized":"2024-09-10T21:11:14.208081+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:44.589671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.042790+0000","last_change":"2024-09-10T21:11:13.243835+0000","last_active":"2024-09-10T21:11:15.042790+0000","last_peered":"2024-09-10T21:11:15.042790+0000","last_clean":"2024-09-10T21:11:15.042790+0000","last_became_active":"2024-09-10T21:11:13.243572+0000","last_became_peered":"2024-09-10T21:11:13.243572+0000","last_unstale":"2024-09-10T21:11:15.042790+0000","last_undegraded":"2024-09-10T21:11:15.042790+0000","last_fullsized":"2024-09-10T21:11:15.042790+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:07.723291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.209954+0000","last_change":"2024-09-10T21:11:13.233269+0000","last_active":"2024-09-10T21:11:14.209954+0000","last_peered":"2024-09-10T21:11:14.209954+0000","last_clean":"2024-09-10T21:11:14.209954+0000","last_became_active":"2024-09-10T21:11:13.228592+0000","last_became_peered":"2024-09-10T21:11:13.228592+0000","last_unstale":"2024-09-10T21:11:14.209954+0000","last_undegraded":"2024-09-10T21:11:14.209954+0000","last_fullsized":"2024-09-10T21:11:14.209954+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:09.865906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.237315+0000","last_change":"2024-09-10T21:11:13.233758+0000","last_active":"2024-09-10T21:11:14.237315+0000","last_peered":"2024-09-10T21:11:14.237315+0000","last_clean":"2024-09-10T21:11:14.237315+0000","last_became_active":"2024-09-10T21:11:13.233244+0000","last_became_peered":"2024-09-10T21:11:13.233244+0000","last_unstale":"2024-09-10T21:11:14.237315+0000","last_undegraded":"2024-09-10T21:11:14.237315+0000","last_fullsized":"2024-09-10T21:11:14.237315+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:14.687731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.042673+0000","last_change":"2024-09-10T21:11:13.237008+0000","last_active":"2024-09-10T21:11:15.042673+0000","last_peered":"2024-09-10T21:11:15.042673+0000","last_clean":"2024-09-10T21:11:15.042673+0000","last_became_active":"2024-09-10T21:11:13.236683+0000","last_became_peered":"2024-09-10T21:11:13.236683+0000","last_unstale":"2024-09-10T21:11:15.042673+0000","last_undegraded":"2024-09-10T21:11:15.042673+0000","last_fullsized":"2024-09-10T21:11:15.042673+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:08.804915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041222+0000","last_change":"2024-09-10T21:11:13.250940+0000","last_active":"2024-09-10T21:11:15.041222+0000","last_peered":"2024-09-10T21:11:15.041222+0000","last_clean":"2024-09-10T21:11:15.041222+0000","last_became_active":"2024-09-10T21:11:13.250694+0000","last_became_peered":"2024-09-10T21:11:13.250694+0000","last_unstale":"2024-09-10T21:11:15.041222+0000","last_undegraded":"2024-09-10T21:11:15.041222+0000","last_fullsized":"2024-09-10T21:11:15.041222+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:59.977227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208995+0000","last_change":"2024-09-10T21:11:13.216431+0000","last_active":"2024-09-10T21:11:14.208995+0000","last_peered":"2024-09-10T21:11:14.208995+0000","last_clean":"2024-09-10T21:11:14.208995+0000","last_became_active":"2024-09-10T21:11:13.216126+0000","last_became_peered":"2024-09-10T21:11:13.216126+0000","last_unstale":"2024-09-10T21:11:14.208995+0000","last_undegraded":"2024-09-10T21:11:14.208995+0000","last_fullsized":"2024-09-10T21:11:14.208995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:02.913850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041660+0000","last_change":"2024-09-10T21:11:13.227625+0000","last_active":"2024-09-10T21:11:15.041660+0000","last_peered":"2024-09-10T21:11:15.041660+0000","last_clean":"2024-09-10T21:11:15.041660+0000","last_became_active":"2024-09-10T21:11:13.227357+0000","last_became_peered":"2024-09-10T21:11:13.227357+0000","last_unstale":"2024-09-10T21:11:15.041660+0000","last_undegraded":"2024-09-10T21:11:15.041660+0000","last_fullsized":"2024-09-10T21:11:15.041660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:36.262388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041554+0000","last_change":"2024-09-10T21:11:13.233228+0000","last_active":"2024-09-10T21:11:15.041554+0000","last_peered":"2024-09-10T21:11:15.041554+0000","last_clean":"2024-09-10T21:11:15.041554+0000","last_became_active":"2024-09-10T21:11:13.232879+0000","last_became_peered":"2024-09-10T21:11:13.232879+0000","last_unstale":"2024-09-10T21:11:15.041554+0000","last_undegraded":"2024-09-10T21:11:15.041554+0000","last_fullsized":"2024-09-10T21:11:15.041554+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:11.172911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.237477+0000","last_change":"2024-09-10T21:11:13.228537+0000","last_active":"2024-09-10T21:11:14.237477+0000","last_peered":"2024-09-10T21:11:14.237477+0000","last_clean":"2024-09-10T21:11:14.237477+0000","last_became_active":"2024-09-10T21:11:13.225545+0000","last_became_peered":"2024-09-10T21:11:13.225545+0000","last_unstale":"2024-09-10T21:11:14.237477+0000","last_undegraded":"2024-09-10T21:11:14.237477+0000","last_fullsized":"2024-09-10T21:11:14.237477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:52.268300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.216977+0000","last_change":"2024-09-10T21:11:13.239102+0000","last_active":"2024-09-10T21:11:14.216977+0000","last_peered":"2024-09-10T21:11:14.216977+0000","last_clean":"2024-09-10T21:11:14.216977+0000","last_became_active":"2024-09-10T21:11:13.238353+0000","last_became_peered":"2024-09-10T21:11:13.238353+0000","last_unstale":"2024-09-10T21:11:14.216977+0000","last_undegraded":"2024-09-10T21:11:14.216977+0000","last_fullsized":"2024-09-10T21:11:14.216977+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:08.804915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.216950+0000","last_change":"2024-09-10T21:11:13.234923+0000","last_active":"2024-09-10T21:11:14.216950+0000","last_peered":"2024-09-10T21:11:14.216950+0000","last_clean":"2024-09-10T21:11:14.216950+0000","last_became_active":"2024-09-10T21:11:13.234594+0000","last_became_peered":"2024-09-10T21:11:13.234594+0000","last_unstale":"2024-09-10T21:11:14.216950+0000","last_undegraded":"2024-09-10T21:11:14.216950+0000","last_fullsized":"2024-09-10T21:11:14.216950+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:17.679028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041449+0000","last_change":"2024-09-10T21:11:13.241250+0000","last_active":"2024-09-10T21:11:15.041449+0000","last_peered":"2024-09-10T21:11:15.041449+0000","last_clean":"2024-09-10T21:11:15.041449+0000","last_became_active":"2024-09-10T21:11:13.240973+0000","last_became_peered":"2024-09-10T21:11:13.240973+0000","last_unstale":"2024-09-10T21:11:15.041449+0000","last_undegraded":"2024-09-10T21:11:15.041449+0000","last_fullsized":"2024-09-10T21:11:15.041449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:17.285190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041893+0000","last_change":"2024-09-10T21:11:13.236702+0000","last_active":"2024-09-10T21:11:15.041893+0000","last_peered":"2024-09-10T21:11:15.041893+0000","last_clean":"2024-09-10T21:11:15.041893+0000","last_became_active":"2024-09-10T21:11:13.236376+0000","last_became_peered":"2024-09-10T21:11:13.236376+0000","last_unstale":"2024-09-10T21:11:15.041893+0000","last_undegraded":"2024-09-10T21:11:15.041893+0000","last_fullsized":"2024-09-10T21:11:15.041893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:37.339633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208525+0000","last_change":"2024-09-10T21:11:13.219569+0000","last_active":"2024-09-10T21:11:14.208525+0000","last_peered":"2024-09-10T21:11:14.208525+0000","last_clean":"2024-09-10T21:11:14.208525+0000","last_became_active":"2024-09-10T21:11:13.218463+0000","last_became_peered":"2024-09-10T21:11:13.218463+0000","last_unstale":"2024-09-10T21:11:14.208525+0000","last_undegraded":"2024-09-10T21:11:14.208525+0000","last_fullsized":"2024-09-10T21:11:14.208525+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:44.939754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.040957+0000","last_change":"2024-09-10T21:11:13.242608+0000","last_active":"2024-09-10T21:11:15.040957+0000","last_peered":"2024-09-10T21:11:15.040957+0000","last_clean":"2024-09-10T21:11:15.040957+0000","last_became_active":"2024-09-10T21:11:13.241793+0000","last_became_peered":"2024-09-10T21:11:13.241793+0000","last_unstale":"2024-09-10T21:11:15.040957+0000","last_undegraded":"2024-09-10T21:11:15.040957+0000","last_fullsized":"2024-09-10T21:11:15.040957+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:09.002865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208182+0000","last_change":"2024-09-10T21:11:13.240633+0000","last_active":"2024-09-10T21:11:14.208182+0000","last_peered":"2024-09-10T21:11:14.208182+0000","last_clean":"2024-09-10T21:11:14.208182+0000","last_became_active":"2024-09-10T21:11:13.240046+0000","last_became_peered":"2024-09-10T21:11:13.240046+0000","last_unstale":"2024-09-10T21:11:14.208182+0000","last_undegraded":"2024-09-10T21:11:14.208182+0000","last_fullsized":"2024-09-10T21:11:14.208182+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:47.399613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.210128+0000","last_change":"2024-09-10T21:11:13.219704+0000","last_active":"2024-09-10T21:11:14.210128+0000","last_peered":"2024-09-10T21:11:14.210128+0000","last_clean":"2024-09-10T21:11:14.210128+0000","last_became_active":"2024-09-10T21:11:13.218621+0000","last_became_peered":"2024-09-10T21:11:13.218621+0000","last_unstale":"2024-09-10T21:11:14.210128+0000","last_undegraded":"2024-09-10T21:11:14.210128+0000","last_fullsized":"2024-09-10T21:11:14.210128+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:57.096496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.210037+0000","last_change":"2024-09-10T21:11:13.232802+0000","last_active":"2024-09-10T21:11:14.210037+0000","last_peered":"2024-09-10T21:11:14.210037+0000","last_clean":"2024-09-10T21:11:14.210037+0000","last_became_active":"2024-09-10T21:11:13.227485+0000","last_became_peered":"2024-09-10T21:11:13.227485+0000","last_unstale":"2024-09-10T21:11:14.210037+0000","last_undegraded":"2024-09-10T21:11:14.210037+0000","last_fullsized":"2024-09-10T21:11:14.210037+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:17.312744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.209216+0000","last_change":"2024-09-10T21:11:13.239036+0000","last_active":"2024-09-10T21:11:14.209216+0000","last_peered":"2024-09-10T21:11:14.209216+0000","last_clean":"2024-09-10T21:11:14.209216+0000","last_became_active":"2024-09-10T21:11:13.238795+0000","last_became_peered":"2024-09-10T21:11:13.238795+0000","last_unstale":"2024-09-10T21:11:14.209216+0000","last_undegraded":"2024-09-10T21:11:14.209216+0000","last_fullsized":"2024-09-10T21:11:14.209216+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:40.006514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208632+0000","last_change":"2024-09-10T21:11:13.235338+0000","last_active":"2024-09-10T21:11:14.208632+0000","last_peered":"2024-09-10T21:11:14.208632+0000","last_clean":"2024-09-10T21:11:14.208632+0000","last_became_active":"2024-09-10T21:11:13.231675+0000","last_became_peered":"2024-09-10T21:11:13.231675+0000","last_unstale":"2024-09-10T21:11:14.208632+0000","last_undegraded":"2024-09-10T21:11:14.208632+0000","last_fullsized":"2024-09-10T21:11:14.208632+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:16.574001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041758+0000","last_change":"2024-09-10T21:11:13.249311+0000","last_active":"2024-09-10T21:11:15.041758+0000","last_peered":"2024-09-10T21:11:15.041758+0000","last_clean":"2024-09-10T21:11:15.041758+0000","last_became_active":"2024-09-10T21:11:13.248813+0000","last_became_peered":"2024-09-10T21:11:13.248813+0000","last_unstale":"2024-09-10T21:11:15.041758+0000","last_undegraded":"2024-09-10T21:11:15.041758+0000","last_fullsized":"2024-09-10T21:11:15.041758+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:00.750717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041767+0000","last_change":"2024-09-10T21:11:13.248118+0000","last_active":"2024-09-10T21:11:15.041767+0000","last_peered":"2024-09-10T21:11:15.041767+0000","last_clean":"2024-09-10T21:11:15.041767+0000","last_became_active":"2024-09-10T21:11:13.247828+0000","last_became_peered":"2024-09-10T21:11:13.247828+0000","last_unstale":"2024-09-10T21:11:15.041767+0000","last_undegraded":"2024-09-10T21:11:15.041767+0000","last_fullsized":"2024-09-10T21:11:15.041767+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:03.475245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208612+0000","last_change":"2024-09-10T21:11:13.231198+0000","last_active":"2024-09-10T21:11:14.208612+0000","last_peered":"2024-09-10T21:11:14.208612+0000","last_clean":"2024-09-10T21:11:14.208612+0000","last_became_active":"2024-09-10T21:11:13.229553+0000","last_became_peered":"2024-09-10T21:11:13.229553+0000","last_unstale":"2024-09-10T21:11:14.208612+0000","last_undegraded":"2024-09-10T21:11:14.208612+0000","last_fullsized":"2024-09-10T21:11:14.208612+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:40.277707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.236958+0000","last_change":"2024-09-10T21:11:13.220392+0000","last_active":"2024-09-10T21:11:14.236958+0000","last_peered":"2024-09-10T21:11:14.236958+0000","last_clean":"2024-09-10T21:11:14.236958+0000","last_became_active":"2024-09-10T21:11:13.220117+0000","last_became_peered":"2024-09-10T21:11:13.220117+0000","last_unstale":"2024-09-10T21:11:14.236958+0000","last_undegraded":"2024-09-10T21:11:14.236958+0000","last_fullsized":"2024-09-10T21:11:14.236958+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:16.797970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041833+0000","last_change":"2024-09-10T21:11:13.224627+0000","last_active":"2024-09-10T21:11:15.041833+0000","last_peered":"2024-09-10T21:11:15.041833+0000","last_clean":"2024-09-10T21:11:15.041833+0000","last_became_active":"2024-09-10T21:11:13.224165+0000","last_became_peered":"2024-09-10T21:11:13.224165+0000","last_unstale":"2024-09-10T21:11:15.041833+0000","last_undegraded":"2024-09-10T21:11:15.041833+0000","last_fullsized":"2024-09-10T21:11:15.041833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:31.506862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.237622+0000","last_change":"2024-09-10T21:11:12.225366+0000","last_active":"2024-09-10T21:11:14.237622+0000","last_peered":"2024-09-10T21:11:14.237622+0000","last_clean":"2024-09-10T21:11:14.237622+0000","last_became_active":"2024-09-10T21:11:12.223302+0000","last_became_peered":"2024-09-10T21:11:12.223302+0000","last_unstale":"2024-09-10T21:11:14.237622+0000","last_undegraded":"2024-09-10T21:11:14.237622+0000","last_fullsized":"2024-09-10T21:11:14.237622+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:27.847102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208924+0000","last_change":"2024-09-10T21:11:13.221051+0000","last_active":"2024-09-10T21:11:14.208924+0000","last_peered":"2024-09-10T21:11:14.208924+0000","last_clean":"2024-09-10T21:11:14.208924+0000","last_became_active":"2024-09-10T21:11:13.220762+0000","last_became_peered":"2024-09-10T21:11:13.220762+0000","last_unstale":"2024-09-10T21:11:14.208924+0000","last_undegraded":"2024-09-10T21:11:14.208924+0000","last_fullsized":"2024-09-10T21:11:14.208924+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:28.850594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.210189+0000","last_change":"2024-09-10T21:11:12.222288+0000","last_active":"2024-09-10T21:11:14.210189+0000","last_peered":"2024-09-10T21:11:14.210189+0000","last_clean":"2024-09-10T21:11:14.210189+0000","last_became_active":"2024-09-10T21:11:12.221928+0000","last_became_peered":"2024-09-10T21:11:12.221928+0000","last_unstale":"2024-09-10T21:11:14.210189+0000","last_undegraded":"2024-09-10T21:11:14.210189+0000","last_fullsized":"2024-09-10T21:11:14.210189+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:35.884414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208676+0000","last_change":"2024-09-10T21:11:13.219137+0000","last_active":"2024-09-10T21:11:14.208676+0000","last_peered":"2024-09-10T21:11:14.208676+0000","last_clean":"2024-09-10T21:11:14.208676+0000","last_became_active":"2024-09-10T21:11:13.218827+0000","last_became_peered":"2024-09-10T21:11:13.218827+0000","last_unstale":"2024-09-10T21:11:14.208676+0000","last_undegraded":"2024-09-10T21:11:14.208676+0000","last_fullsized":"2024-09-10T21:11:14.208676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:27.570320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.217263+0000","last_change":"2024-09-10T21:11:12.223630+0000","last_active":"2024-09-10T21:11:14.217263+0000","last_peered":"2024-09-10T21:11:14.217263+0000","last_clean":"2024-09-10T21:11:14.217263+0000","last_became_active":"2024-09-10T21:11:12.223352+0000","last_became_peered":"2024-09-10T21:11:12.223352+0000","last_unstale":"2024-09-10T21:11:14.217263+0000","last_undegraded":"2024-09-10T21:11:14.217263+0000","last_fullsized":"2024-09-10T21:11:14.217263+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:53.480628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.209903+0000","last_change":"2024-09-10T21:11:13.236750+0000","last_active":"2024-09-10T21:11:14.209903+0000","last_peered":"2024-09-10T21:11:14.209903+0000","last_clean":"2024-09-10T21:11:14.209903+0000","last_became_active":"2024-09-10T21:11:13.236417+0000","last_became_peered":"2024-09-10T21:11:13.236417+0000","last_unstale":"2024-09-10T21:11:14.209903+0000","last_undegraded":"2024-09-10T21:11:14.209903+0000","last_fullsized":"2024-09-10T21:11:14.209903+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:44.911928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.209863+0000","last_change":"2024-09-10T21:11:12.223783+0000","last_active":"2024-09-10T21:11:14.209863+0000","last_peered":"2024-09-10T21:11:14.209863+0000","last_clean":"2024-09-10T21:11:14.209863+0000","last_became_active":"2024-09-10T21:11:12.223509+0000","last_became_peered":"2024-09-10T21:11:12.223509+0000","last_unstale":"2024-09-10T21:11:14.209863+0000","last_undegraded":"2024-09-10T21:11:14.209863+0000","last_fullsized":"2024-09-10T21:11:14.209863+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:14.818515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.042482+0000","last_change":"2024-09-10T21:11:13.244681+0000","last_active":"2024-09-10T21:11:15.042482+0000","last_peered":"2024-09-10T21:11:15.042482+0000","last_clean":"2024-09-10T21:11:15.042482+0000","last_became_active":"2024-09-10T21:11:13.244379+0000","last_became_peered":"2024-09-10T21:11:13.244379+0000","last_unstale":"2024-09-10T21:11:15.042482+0000","last_undegraded":"2024-09-10T21:11:15.042482+0000","last_fullsized":"2024-09-10T21:11:15.042482+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:57.880649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208454+0000","last_change":"2024-09-10T21:11:12.212357+0000","last_active":"2024-09-10T21:11:14.208454+0000","last_peered":"2024-09-10T21:11:14.208454+0000","last_clean":"2024-09-10T21:11:14.208454+0000","last_became_active":"2024-09-10T21:11:12.209827+0000","last_became_peered":"2024-09-10T21:11:12.209827+0000","last_unstale":"2024-09-10T21:11:14.208454+0000","last_undegraded":"2024-09-10T21:11:14.208454+0000","last_fullsized":"2024-09-10T21:11:14.208454+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:38.717269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208305+0000","last_change":"2024-09-10T21:11:13.222468+0000","last_active":"2024-09-10T21:11:14.208305+0000","last_peered":"2024-09-10T21:11:14.208305+0000","last_clean":"2024-09-10T21:11:14.208305+0000","last_became_active":"2024-09-10T21:11:13.221546+0000","last_became_peered":"2024-09-10T21:11:13.221546+0000","last_unstale":"2024-09-10T21:11:14.208305+0000","last_undegraded":"2024-09-10T21:11:14.208305+0000","last_fullsized":"2024-09-10T21:11:14.208305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:03.475245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208220+0000","last_change":"2024-09-10T21:11:12.225033+0000","last_active":"2024-09-10T21:11:14.208220+0000","last_peered":"2024-09-10T21:11:14.208220+0000","last_clean":"2024-09-10T21:11:14.208220+0000","last_became_active":"2024-09-10T21:11:12.224007+0000","last_became_peered":"2024-09-10T21:11:12.224007+0000","last_unstale":"2024-09-10T21:11:14.208220+0000","last_undegraded":"2024-09-10T21:11:14.208220+0000","last_fullsized":"2024-09-10T21:11:14.208220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:37.774571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.042379+0000","last_change":"2024-09-10T21:11:13.230228+0000","last_active":"2024-09-10T21:11:15.042379+0000","last_peered":"2024-09-10T21:11:15.042379+0000","last_clean":"2024-09-10T21:11:15.042379+0000","last_became_active":"2024-09-10T21:11:13.229888+0000","last_became_peered":"2024-09-10T21:11:13.229888+0000","last_unstale":"2024-09-10T21:11:15.042379+0000","last_undegraded":"2024-09-10T21:11:15.042379+0000","last_fullsized":"2024-09-10T21:11:15.042379+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:41.659020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.216874+0000","last_change":"2024-09-10T21:11:12.221685+0000","last_active":"2024-09-10T21:11:14.216874+0000","last_peered":"2024-09-10T21:11:14.216874+0000","last_clean":"2024-09-10T21:11:14.216874+0000","last_became_active":"2024-09-10T21:11:12.221393+0000","last_became_peered":"2024-09-10T21:11:12.221393+0000","last_unstale":"2024-09-10T21:11:14.216874+0000","last_undegraded":"2024-09-10T21:11:14.216874+0000","last_fullsized":"2024-09-10T21:11:14.216874+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:26.336426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208496+0000","last_change":"2024-09-10T21:11:13.234906+0000","last_active":"2024-09-10T21:11:14.208496+0000","last_peered":"2024-09-10T21:11:14.208496+0000","last_clean":"2024-09-10T21:11:14.208496+0000","last_became_active":"2024-09-10T21:11:13.228823+0000","last_became_peered":"2024-09-10T21:11:13.228823+0000","last_unstale":"2024-09-10T21:11:14.208496+0000","last_undegraded":"2024-09-10T21:11:14.208496+0000","last_fullsized":"2024-09-10T21:11:14.208496+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:11.846750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208335+0000","last_change":"2024-09-10T21:11:12.224827+0000","last_active":"2024-09-10T21:11:14.208335+0000","last_peered":"2024-09-10T21:11:14.208335+0000","last_clean":"2024-09-10T21:11:14.208335+0000","last_became_active":"2024-09-10T21:11:12.224356+0000","last_became_peered":"2024-09-10T21:11:12.224356+0000","last_unstale":"2024-09-10T21:11:14.208335+0000","last_undegraded":"2024-09-10T21:11:14.208335+0000","last_fullsized":"2024-09-10T21:11:14.208335+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:00.054106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.237363+0000","last_change":"2024-09-10T21:11:13.233922+0000","last_active":"2024-09-10T21:11:14.237363+0000","last_peered":"2024-09-10T21:11:14.237363+0000","last_clean":"2024-09-10T21:11:14.237363+0000","last_became_active":"2024-09-10T21:11:13.230732+0000","last_became_peered":"2024-09-10T21:11:13.230732+0000","last_unstale":"2024-09-10T21:11:14.237363+0000","last_undegraded":"2024-09-10T21:11:14.237363+0000","last_fullsized":"2024-09-10T21:11:14.237363+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:25.973693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.209171+0000","last_change":"2024-09-10T21:11:12.221271+0000","last_active":"2024-09-10T21:11:14.209171+0000","last_peered":"2024-09-10T21:11:14.209171+0000","last_clean":"2024-09-10T21:11:14.209171+0000","last_became_active":"2024-09-10T21:11:12.220975+0000","last_became_peered":"2024-09-10T21:11:12.220975+0000","last_unstale":"2024-09-10T21:11:14.209171+0000","last_undegraded":"2024-09-10T21:11:14.209171+0000","last_fullsized":"2024-09-10T21:11:14.209171+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:29.280746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.210107+0000","last_change":"2024-09-10T21:11:13.234418+0000","last_active":"2024-09-10T21:11:14.210107+0000","last_peered":"2024-09-10T21:11:14.210107+0000","last_clean":"2024-09-10T21:11:14.210107+0000","last_became_active":"2024-09-10T21:11:13.234159+0000","last_became_peered":"2024-09-10T21:11:13.234159+0000","last_unstale":"2024-09-10T21:11:14.210107+0000","last_undegraded":"2024-09-10T21:11:14.210107+0000","last_fullsized":"2024-09-10T21:11:14.210107+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:50.936222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.209446+0000","last_change":"2024-09-10T21:11:12.222257+0000","last_active":"2024-09-10T21:11:14.209446+0000","last_peered":"2024-09-10T21:11:14.209446+0000","last_clean":"2024-09-10T21:11:14.209446+0000","last_became_active":"2024-09-10T21:11:12.221959+0000","last_became_peered":"2024-09-10T21:11:12.221959+0000","last_unstale":"2024-09-10T21:11:14.209446+0000","last_undegraded":"2024-09-10T21:11:14.209446+0000","last_fullsized":"2024-09-10T21:11:14.209446+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:35.482932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.209641+0000","last_change":"2024-09-10T21:11:13.220496+0000","last_active":"2024-09-10T21:11:14.209641+0000","last_peered":"2024-09-10T21:11:14.209641+0000","last_clean":"2024-09-10T21:11:14.209641+0000","last_became_active":"2024-09-10T21:11:13.218857+0000","last_became_peered":"2024-09-10T21:11:13.218857+0000","last_unstale":"2024-09-10T21:11:14.209641+0000","last_undegraded":"2024-09-10T21:11:14.209641+0000","last_fullsized":"2024-09-10T21:11:14.209641+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:10:15.992696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.237503+0000","last_change":"2024-09-10T21:11:12.229925+0000","last_active":"2024-09-10T21:11:14.237503+0000","last_peered":"2024-09-10T21:11:14.237503+0000","last_clean":"2024-09-10T21:11:14.237503+0000","last_became_active":"2024-09-10T21:11:12.229037+0000","last_became_peered":"2024-09-10T21:11:12.229037+0000","last_unstale":"2024-09-10T21:11:14.237503+0000","last_undegraded":"2024-09-10T21:11:14.237503+0000","last_fullsized":"2024-09-10T21:11:14.237503+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:43.936262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.237315+0000","last_change":"2024-09-10T21:11:13.233489+0000","last_active":"2024-09-10T21:11:14.237315+0000","last_peered":"2024-09-10T21:11:14.237315+0000","last_clean":"2024-09-10T21:11:14.237315+0000","last_became_active":"2024-09-10T21:11:13.232909+0000","last_became_peered":"2024-09-10T21:11:13.232909+0000","last_unstale":"2024-09-10T21:11:14.237315+0000","last_undegraded":"2024-09-10T21:11:14.237315+0000","last_fullsized":"2024-09-10T21:11:14.237315+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:44.221660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041496+0000","last_change":"2024-09-10T21:11:12.230626+0000","last_active":"2024-09-10T21:11:15.041496+0000","last_peered":"2024-09-10T21:11:15.041496+0000","last_clean":"2024-09-10T21:11:15.041496+0000","last_became_active":"2024-09-10T21:11:12.230308+0000","last_became_peered":"2024-09-10T21:11:12.230308+0000","last_unstale":"2024-09-10T21:11:15.041496+0000","last_undegraded":"2024-09-10T21:11:15.041496+0000","last_fullsized":"2024-09-10T21:11:15.041496+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:09.728148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.040933+0000","last_change":"2024-09-10T21:11:13.242994+0000","last_active":"2024-09-10T21:11:15.040933+0000","last_peered":"2024-09-10T21:11:15.040933+0000","last_clean":"2024-09-10T21:11:15.040933+0000","last_became_active":"2024-09-10T21:11:13.242683+0000","last_became_peered":"2024-09-10T21:11:13.242683+0000","last_unstale":"2024-09-10T21:11:15.040933+0000","last_undegraded":"2024-09-10T21:11:15.040933+0000","last_fullsized":"2024-09-10T21:11:15.040933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:16.706810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208491+0000","last_change":"2024-09-10T21:11:12.224674+0000","last_active":"2024-09-10T21:11:14.208491+0000","last_peered":"2024-09-10T21:11:14.208491+0000","last_clean":"2024-09-10T21:11:14.208491+0000","last_became_active":"2024-09-10T21:11:12.224325+0000","last_became_peered":"2024-09-10T21:11:12.224325+0000","last_unstale":"2024-09-10T21:11:14.208491+0000","last_undegraded":"2024-09-10T21:11:14.208491+0000","last_fullsized":"2024-09-10T21:11:14.208491+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:06.355342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208920+0000","last_change":"2024-09-10T21:11:13.236816+0000","last_active":"2024-09-10T21:11:14.208920+0000","last_peered":"2024-09-10T21:11:14.208920+0000","last_clean":"2024-09-10T21:11:14.208920+0000","last_became_active":"2024-09-10T21:11:13.236502+0000","last_became_peered":"2024-09-10T21:11:13.236502+0000","last_unstale":"2024-09-10T21:11:14.208920+0000","last_undegraded":"2024-09-10T21:11:14.208920+0000","last_fullsized":"2024-09-10T21:11:14.208920+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:50.522566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.209036+0000","last_change":"2024-09-10T21:11:12.226932+0000","last_active":"2024-09-10T21:11:14.209036+0000","last_peered":"2024-09-10T21:11:14.209036+0000","last_clean":"2024-09-10T21:11:14.209036+0000","last_became_active":"2024-09-10T21:11:12.226622+0000","last_became_peered":"2024-09-10T21:11:12.226622+0000","last_unstale":"2024-09-10T21:11:14.209036+0000","last_undegraded":"2024-09-10T21:11:14.209036+0000","last_fullsized":"2024-09-10T21:11:14.209036+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:15.020787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208313+0000","last_change":"2024-09-10T21:11:13.226536+0000","last_active":"2024-09-10T21:11:14.208313+0000","last_peered":"2024-09-10T21:11:14.208313+0000","last_clean":"2024-09-10T21:11:14.208313+0000","last_became_active":"2024-09-10T21:11:13.226204+0000","last_became_peered":"2024-09-10T21:11:13.226204+0000","last_unstale":"2024-09-10T21:11:14.208313+0000","last_undegraded":"2024-09-10T21:11:14.208313+0000","last_fullsized":"2024-09-10T21:11:14.208313+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:25.669128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208694+0000","last_change":"2024-09-10T21:11:12.222663+0000","last_active":"2024-09-10T21:11:14.208694+0000","last_peered":"2024-09-10T21:11:14.208694+0000","last_clean":"2024-09-10T21:11:14.208694+0000","last_became_active":"2024-09-10T21:11:12.222491+0000","last_became_peered":"2024-09-10T21:11:12.222491+0000","last_unstale":"2024-09-10T21:11:14.208694+0000","last_undegraded":"2024-09-10T21:11:14.208694+0000","last_fullsized":"2024-09-10T21:11:14.208694+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:26.481636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.237032+0000","last_change":"2024-09-10T21:11:13.218025+0000","last_active":"2024-09-10T21:11:14.237032+0000","last_peered":"2024-09-10T21:11:14.237032+0000","last_clean":"2024-09-10T21:11:14.237032+0000","last_became_active":"2024-09-10T21:11:13.217119+0000","last_became_peered":"2024-09-10T21:11:13.217119+0000","last_unstale":"2024-09-10T21:11:14.237032+0000","last_undegraded":"2024-09-10T21:11:14.237032+0000","last_fullsized":"2024-09-10T21:11:14.237032+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:00.011652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.216710+0000","last_change":"2024-09-10T21:11:12.225066+0000","last_active":"2024-09-10T21:11:14.216710+0000","last_peered":"2024-09-10T21:11:14.216710+0000","last_clean":"2024-09-10T21:11:14.216710+0000","last_became_active":"2024-09-10T21:11:12.224555+0000","last_became_peered":"2024-09-10T21:11:12.224555+0000","last_unstale":"2024-09-10T21:11:14.216710+0000","last_undegraded":"2024-09-10T21:11:14.216710+0000","last_fullsized":"2024-09-10T21:11:14.216710+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:39.490439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.237361+0000","last_change":"2024-09-10T21:11:13.238225+0000","last_active":"2024-09-10T21:11:14.237361+0000","last_peered":"2024-09-10T21:11:14.237361+0000","last_clean":"2024-09-10T21:11:14.237361+0000","last_became_active":"2024-09-10T21:11:13.236412+0000","last_became_peered":"2024-09-10T21:11:13.236412+0000","last_unstale":"2024-09-10T21:11:14.237361+0000","last_undegraded":"2024-09-10T21:11:14.237361+0000","last_fullsized":"2024-09-10T21:11:14.237361+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:46.174549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.217075+0000","last_change":"2024-09-10T21:11:12.228489+0000","last_active":"2024-09-10T21:11:14.217075+0000","last_peered":"2024-09-10T21:11:14.217075+0000","last_clean":"2024-09-10T21:11:14.217075+0000","last_became_active":"2024-09-10T21:11:12.228136+0000","last_became_peered":"2024-09-10T21:11:12.228136+0000","last_unstale":"2024-09-10T21:11:14.217075+0000","last_undegraded":"2024-09-10T21:11:14.217075+0000","last_fullsized":"2024-09-10T21:11:14.217075+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:04.507973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.042794+0000","last_change":"2024-09-10T21:11:13.239774+0000","last_active":"2024-09-10T21:11:15.042794+0000","last_peered":"2024-09-10T21:11:15.042794+0000","last_clean":"2024-09-10T21:11:15.042794+0000","last_became_active":"2024-09-10T21:11:13.239416+0000","last_became_peered":"2024-09-10T21:11:13.239416+0000","last_unstale":"2024-09-10T21:11:15.042794+0000","last_undegraded":"2024-09-10T21:11:15.042794+0000","last_fullsized":"2024-09-10T21:11:15.042794+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:19.338163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208760+0000","last_change":"2024-09-10T21:11:12.222819+0000","last_active":"2024-09-10T21:11:14.208760+0000","last_peered":"2024-09-10T21:11:14.208760+0000","last_clean":"2024-09-10T21:11:14.208760+0000","last_became_active":"2024-09-10T21:11:12.222499+0000","last_became_peered":"2024-09-10T21:11:12.222499+0000","last_unstale":"2024-09-10T21:11:14.208760+0000","last_undegraded":"2024-09-10T21:11:14.208760+0000","last_fullsized":"2024-09-10T21:11:14.208760+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:11.810704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.209382+0000","last_change":"2024-09-10T21:11:13.227904+0000","last_active":"2024-09-10T21:11:14.209382+0000","last_peered":"2024-09-10T21:11:14.209382+0000","last_clean":"2024-09-10T21:11:14.209382+0000","last_became_active":"2024-09-10T21:11:13.227548+0000","last_became_peered":"2024-09-10T21:11:13.227548+0000","last_unstale":"2024-09-10T21:11:14.209382+0000","last_undegraded":"2024-09-10T21:11:14.209382+0000","last_fullsized":"2024-09-10T21:11:14.209382+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:24.063070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.237502+0000","last_change":"2024-09-10T21:11:12.230259+0000","last_active":"2024-09-10T21:11:14.237502+0000","last_peered":"2024-09-10T21:11:14.237502+0000","last_clean":"2024-09-10T21:11:14.237502+0000","last_became_active":"2024-09-10T21:11:12.229526+0000","last_became_peered":"2024-09-10T21:11:12.229526+0000","last_unstale":"2024-09-10T21:11:14.237502+0000","last_undegraded":"2024-09-10T21:11:14.237502+0000","last_fullsized":"2024-09-10T21:11:14.237502+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:26.566827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.217121+0000","last_change":"2024-09-10T21:11:13.244342+0000","last_active":"2024-09-10T21:11:14.217121+0000","last_peered":"2024-09-10T21:11:14.217121+0000","last_clean":"2024-09-10T21:11:14.217121+0000","last_became_active":"2024-09-10T21:11:13.242903+0000","last_became_peered":"2024-09-10T21:11:13.242903+0000","last_unstale":"2024-09-10T21:11:14.217121+0000","last_undegraded":"2024-09-10T21:11:14.217121+0000","last_fullsized":"2024-09-10T21:11:14.217121+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:59.490751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041898+0000","last_change":"2024-09-10T21:11:12.230126+0000","last_active":"2024-09-10T21:11:15.041898+0000","last_peered":"2024-09-10T21:11:15.041898+0000","last_clean":"2024-09-10T21:11:15.041898+0000","last_became_active":"2024-09-10T21:11:12.229860+0000","last_became_peered":"2024-09-10T21:11:12.229860+0000","last_unstale":"2024-09-10T21:11:15.041898+0000","last_undegraded":"2024-09-10T21:11:15.041898+0000","last_fullsized":"2024-09-10T21:11:15.041898+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:09.292762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.040980+0000","last_change":"2024-09-10T21:11:13.241293+0000","last_active":"2024-09-10T21:11:15.040980+0000","last_peered":"2024-09-10T21:11:15.040980+0000","last_clean":"2024-09-10T21:11:15.040980+0000","last_became_active":"2024-09-10T21:11:13.240911+0000","last_became_peered":"2024-09-10T21:11:13.240911+0000","last_unstale":"2024-09-10T21:11:15.040980+0000","last_undegraded":"2024-09-10T21:11:15.040980+0000","last_fullsized":"2024-09-10T21:11:15.040980+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:36.486425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208858+0000","last_change":"2024-09-10T21:11:12.221176+0000","last_active":"2024-09-10T21:11:14.208858+0000","last_peered":"2024-09-10T21:11:14.208858+0000","last_clean":"2024-09-10T21:11:14.208858+0000","last_became_active":"2024-09-10T21:11:12.220902+0000","last_became_peered":"2024-09-10T21:11:12.220902+0000","last_unstale":"2024-09-10T21:11:14.208858+0000","last_undegraded":"2024-09-10T21:11:14.208858+0000","last_fullsized":"2024-09-10T21:11:14.208858+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:54.937881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208705+0000","last_change":"2024-09-10T21:11:13.224634+0000","last_active":"2024-09-10T21:11:14.208705+0000","last_peered":"2024-09-10T21:11:14.208705+0000","last_clean":"2024-09-10T21:11:14.208705+0000","last_became_active":"2024-09-10T21:11:13.224333+0000","last_became_peered":"2024-09-10T21:11:13.224333+0000","last_unstale":"2024-09-10T21:11:14.208705+0000","last_undegraded":"2024-09-10T21:11:14.208705+0000","last_fullsized":"2024-09-10T21:11:14.208705+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:24.043303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208327+0000","last_change":"2024-09-10T21:11:12.226970+0000","last_active":"2024-09-10T21:11:14.208327+0000","last_peered":"2024-09-10T21:11:14.208327+0000","last_clean":"2024-09-10T21:11:14.208327+0000","last_became_active":"2024-09-10T21:11:12.226548+0000","last_became_peered":"2024-09-10T21:11:12.226548+0000","last_unstale":"2024-09-10T21:11:14.208327+0000","last_undegraded":"2024-09-10T21:11:14.208327+0000","last_fullsized":"2024-09-10T21:11:14.208327+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:20.427638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.040996+0000","last_change":"2024-09-10T21:11:13.253121+0000","last_active":"2024-09-10T21:11:15.040996+0000","last_peered":"2024-09-10T21:11:15.040996+0000","last_clean":"2024-09-10T21:11:15.040996+0000","last_became_active":"2024-09-10T21:11:13.252833+0000","last_became_peered":"2024-09-10T21:11:13.252833+0000","last_unstale":"2024-09-10T21:11:15.040996+0000","last_undegraded":"2024-09-10T21:11:15.040996+0000","last_fullsized":"2024-09-10T21:11:15.040996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:00.129682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.042527+0000","last_change":"2024-09-10T21:11:12.215786+0000","last_active":"2024-09-10T21:11:15.042527+0000","last_peered":"2024-09-10T21:11:15.042527+0000","last_clean":"2024-09-10T21:11:15.042527+0000","last_became_active":"2024-09-10T21:11:12.215497+0000","last_became_peered":"2024-09-10T21:11:12.215497+0000","last_unstale":"2024-09-10T21:11:15.042527+0000","last_undegraded":"2024-09-10T21:11:15.042527+0000","last_fullsized":"2024-09-10T21:11:15.042527+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:39.490439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.042437+0000","last_change":"2024-09-10T21:11:13.230287+0000","last_active":"2024-09-10T21:11:15.042437+0000","last_peered":"2024-09-10T21:11:15.042437+0000","last_clean":"2024-09-10T21:11:15.042437+0000","last_became_active":"2024-09-10T21:11:13.229920+0000","last_became_peered":"2024-09-10T21:11:13.229920+0000","last_unstale":"2024-09-10T21:11:15.042437+0000","last_undegraded":"2024-09-10T21:11:15.042437+0000","last_fullsized":"2024-09-10T21:11:15.042437+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:08.712246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.236983+0000","last_change":"2024-09-10T21:11:12.234069+0000","last_active":"2024-09-10T21:11:14.236983+0000","last_peered":"2024-09-10T21:11:14.236983+0000","last_clean":"2024-09-10T21:11:14.236983+0000","last_became_active":"2024-09-10T21:11:12.233714+0000","last_became_peered":"2024-09-10T21:11:12.233714+0000","last_unstale":"2024-09-10T21:11:14.236983+0000","last_undegraded":"2024-09-10T21:11:14.236983+0000","last_fullsized":"2024-09-10T21:11:14.236983+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.570509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.217235+0000","last_change":"2024-09-10T21:11:13.239067+0000","last_active":"2024-09-10T21:11:14.217235+0000","last_peered":"2024-09-10T21:11:14.217235+0000","last_clean":"2024-09-10T21:11:14.217235+0000","last_became_active":"2024-09-10T21:11:13.238755+0000","last_became_peered":"2024-09-10T21:11:13.238755+0000","last_unstale":"2024-09-10T21:11:14.217235+0000","last_undegraded":"2024-09-10T21:11:14.217235+0000","last_fullsized":"2024-09-10T21:11:14.217235+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:09.036043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041755+0000","last_change":"2024-09-10T21:11:12.223676+0000","last_active":"2024-09-10T21:11:15.041755+0000","last_peered":"2024-09-10T21:11:15.041755+0000","last_clean":"2024-09-10T21:11:15.041755+0000","last_became_active":"2024-09-10T21:11:12.222053+0000","last_became_peered":"2024-09-10T21:11:12.222053+0000","last_unstale":"2024-09-10T21:11:15.041755+0000","last_undegraded":"2024-09-10T21:11:15.041755+0000","last_fullsized":"2024-09-10T21:11:15.041755+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:51.453841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.237668+0000","last_change":"2024-09-10T21:11:13.223973+0000","last_active":"2024-09-10T21:11:14.237668+0000","last_peered":"2024-09-10T21:11:14.237668+0000","last_clean":"2024-09-10T21:11:14.237668+0000","last_became_active":"2024-09-10T21:11:13.223701+0000","last_became_peered":"2024-09-10T21:11:13.223701+0000","last_unstale":"2024-09-10T21:11:14.237668+0000","last_undegraded":"2024-09-10T21:11:14.237668+0000","last_fullsized":"2024-09-10T21:11:14.237668+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:58.537711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.209603+0000","last_change":"2024-09-10T21:11:12.222106+0000","last_active":"2024-09-10T21:11:14.209603+0000","last_peered":"2024-09-10T21:11:14.209603+0000","last_clean":"2024-09-10T21:11:14.209603+0000","last_became_active":"2024-09-10T21:11:12.221770+0000","last_became_peered":"2024-09-10T21:11:12.221770+0000","last_unstale":"2024-09-10T21:11:14.209603+0000","last_undegraded":"2024-09-10T21:11:14.209603+0000","last_fullsized":"2024-09-10T21:11:14.209603+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:07.999372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.217295+0000","last_change":"2024-09-10T21:11:13.248078+0000","last_active":"2024-09-10T21:11:14.217295+0000","last_peered":"2024-09-10T21:11:14.217295+0000","last_clean":"2024-09-10T21:11:14.217295+0000","last_became_active":"2024-09-10T21:11:13.247741+0000","last_became_peered":"2024-09-10T21:11:13.247741+0000","last_unstale":"2024-09-10T21:11:14.217295+0000","last_undegraded":"2024-09-10T21:11:14.217295+0000","last_fullsized":"2024-09-10T21:11:14.217295+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:43.146814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041610+0000","last_change":"2024-09-10T21:11:12.229500+0000","last_active":"2024-09-10T21:11:15.041610+0000","last_peered":"2024-09-10T21:11:15.041610+0000","last_clean":"2024-09-10T21:11:15.041610+0000","last_became_active":"2024-09-10T21:11:12.229152+0000","last_became_peered":"2024-09-10T21:11:12.229152+0000","last_unstale":"2024-09-10T21:11:15.041610+0000","last_undegraded":"2024-09-10T21:11:15.041610+0000","last_fullsized":"2024-09-10T21:11:15.041610+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:11.772150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.042066+0000","last_change":"2024-09-10T21:11:13.232669+0000","last_active":"2024-09-10T21:11:15.042066+0000","last_peered":"2024-09-10T21:11:15.042066+0000","last_clean":"2024-09-10T21:11:15.042066+0000","last_became_active":"2024-09-10T21:11:13.232049+0000","last_became_peered":"2024-09-10T21:11:13.232049+0000","last_unstale":"2024-09-10T21:11:15.042066+0000","last_undegraded":"2024-09-10T21:11:15.042066+0000","last_fullsized":"2024-09-10T21:11:15.042066+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:08.113294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041294+0000","last_change":"2024-09-10T21:11:12.224196+0000","last_active":"2024-09-10T21:11:15.041294+0000","last_peered":"2024-09-10T21:11:15.041294+0000","last_clean":"2024-09-10T21:11:15.041294+0000","last_became_active":"2024-09-10T21:11:12.223853+0000","last_became_peered":"2024-09-10T21:11:12.223853+0000","last_unstale":"2024-09-10T21:11:15.041294+0000","last_undegraded":"2024-09-10T21:11:15.041294+0000","last_fullsized":"2024-09-10T21:11:15.041294+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:23.761112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208973+0000","last_change":"2024-09-10T21:11:13.236581+0000","last_active":"2024-09-10T21:11:14.208973+0000","last_peered":"2024-09-10T21:11:14.208973+0000","last_clean":"2024-09-10T21:11:14.208973+0000","last_became_active":"2024-09-10T21:11:13.236310+0000","last_became_peered":"2024-09-10T21:11:13.236310+0000","last_unstale":"2024-09-10T21:11:14.208973+0000","last_undegraded":"2024-09-10T21:11:14.208973+0000","last_fullsized":"2024-09-10T21:11:14.208973+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:44.291798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041252+0000","last_change":"2024-09-10T21:11:12.224815+0000","last_active":"2024-09-10T21:11:15.041252+0000","last_peered":"2024-09-10T21:11:15.041252+0000","last_clean":"2024-09-10T21:11:15.041252+0000","last_became_active":"2024-09-10T21:11:12.224504+0000","last_became_peered":"2024-09-10T21:11:12.224504+0000","last_unstale":"2024-09-10T21:11:15.041252+0000","last_undegraded":"2024-09-10T21:11:15.041252+0000","last_fullsized":"2024-09-10T21:11:15.041252+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:29.112283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208530+0000","last_change":"2024-09-10T21:11:13.226608+0000","last_active":"2024-09-10T21:11:14.208530+0000","last_peered":"2024-09-10T21:11:14.208530+0000","last_clean":"2024-09-10T21:11:14.208530+0000","last_became_active":"2024-09-10T21:11:13.226297+0000","last_became_peered":"2024-09-10T21:11:13.226297+0000","last_unstale":"2024-09-10T21:11:14.208530+0000","last_undegraded":"2024-09-10T21:11:14.208530+0000","last_fullsized":"2024-09-10T21:11:14.208530+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:47.378006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208452+0000","last_change":"2024-09-10T21:11:12.222940+0000","last_active":"2024-09-10T21:11:14.208452+0000","last_peered":"2024-09-10T21:11:14.208452+0000","last_clean":"2024-09-10T21:11:14.208452+0000","last_became_active":"2024-09-10T21:11:12.222474+0000","last_became_peered":"2024-09-10T21:11:12.222474+0000","last_unstale":"2024-09-10T21:11:14.208452+0000","last_undegraded":"2024-09-10T21:11:14.208452+0000","last_fullsized":"2024-09-10T21:11:14.208452+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:32.362391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.237093+0000","last_change":"2024-09-10T21:11:13.228335+0000","last_active":"2024-09-10T21:11:14.237093+0000","last_peered":"2024-09-10T21:11:14.237093+0000","last_clean":"2024-09-10T21:11:14.237093+0000","last_became_active":"2024-09-10T21:11:13.228013+0000","last_became_peered":"2024-09-10T21:11:13.228013+0000","last_unstale":"2024-09-10T21:11:14.237093+0000","last_undegraded":"2024-09-10T21:11:14.237093+0000","last_fullsized":"2024-09-10T21:11:14.237093+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:37.781739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.208672+0000","last_change":"2024-09-10T21:11:12.220700+0000","last_active":"2024-09-10T21:11:14.208672+0000","last_peered":"2024-09-10T21:11:14.208672+0000","last_clean":"2024-09-10T21:11:14.208672+0000","last_became_active":"2024-09-10T21:11:12.220364+0000","last_became_peered":"2024-09-10T21:11:12.220364+0000","last_unstale":"2024-09-10T21:11:14.208672+0000","last_undegraded":"2024-09-10T21:11:14.208672+0000","last_fullsized":"2024-09-10T21:11:14.208672+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:30.271305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.041869+0000","last_change":"2024-09-10T21:11:13.234881+0000","last_active":"2024-09-10T21:11:15.041869+0000","last_peered":"2024-09-10T21:11:15.041869+0000","last_clean":"2024-09-10T21:11:15.041869+0000","last_became_active":"2024-09-10T21:11:13.234666+0000","last_became_peered":"2024-09-10T21:11:13.234666+0000","last_unstale":"2024-09-10T21:11:15.041869+0000","last_undegraded":"2024-09-10T21:11:15.041869+0000","last_fullsized":"2024-09-10T21:11:15.041869+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:12.194803+0000","last_clean_scrub_stamp":"2024-09-10T21:11:12.194803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:20.005465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:14.237609+0000","last_change":"2024-09-10T21:11:12.230047+0000","last_active":"2024-09-10T21:11:14.237609+0000","last_peered":"2024-09-10T21:11:14.237609+0000","last_clean":"2024-09-10T21:11:14.237609+0000","last_became_active":"2024-09-10T21:11:12.229249+0000","last_became_peered":"2024-09-10T21:11:12.229249+0000","last_unstale":"2024-09-10T21:11:14.237609+0000","last_undegraded":"2024-09-10T21:11:14.237609+0000","last_fullsized":"2024-09-10T21:11:14.237609+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:11:11.191311+0000","last_clean_scrub_stamp":"2024-09-10T21:11:11.191311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:59.482848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":131,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:11:15.042337+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:11:15.042337+0000","last_peered":"2024-09-10T21:11:15.042337+0000","last_clean":"2024-09-10T21:11:15.042337+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:11:15.042337+0000","last_undegraded":"2024-09-10T21:11:15.042337+0000","last_fullsized":"2024-09-10T21:11:15.042337+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":364544,"data_stored":136999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":950272,"data_stored":726337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":368640,"data_stored":137021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":368640,"data_stored":136163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":356352,"data_stored":135421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":356352,"data_stored":135477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31540,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":356352,"data_stored":135409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":950272,"data_stored":725825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:11:22.371 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:11:22.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:11:22.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.557+0000 7fdddd000640 1 -- 172.21.15.16:0/700424343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd8072bf0 msgr2=0x7fddd815dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:22.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.557+0000 7fdddd000640 1 --2- 172.21.15.16:0/700424343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd8072bf0 0x7fddd815dd80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fddd80b2470 tx=0x7fddc001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.557+0000 7fdddd000640 1 -- 172.21.15.16:0/700424343 shutdown_connections 2024-09-10T21:11:22.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.557+0000 7fdddd000640 1 --2- 172.21.15.16:0/700424343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd8072bf0 0x7fddd815dd80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.557+0000 7fdddd000640 1 --2- 172.21.15.16:0/700424343 >> v2:172.21.15.52:3300/0 conn(0x7fddd8151210 0x7fddd80726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.557+0000 7fdddd000640 1 -- 172.21.15.16:0/700424343 >> 172.21.15.16:0/700424343 conn(0x7fddd80b16a0 msgr2=0x7fddd80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:22.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.557+0000 7fdddd000640 1 -- 172.21.15.16:0/700424343 shutdown_connections 2024-09-10T21:11:22.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.557+0000 7fdddd000640 1 -- 172.21.15.16:0/700424343 wait complete. 2024-09-10T21:11:22.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fdddd000640 1 Processor -- start 2024-09-10T21:11:22.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fdddd000640 1 -- start start 2024-09-10T21:11:22.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fdddd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd8072bf0 0x7fddd8139550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:22.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fdddd000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddd8151210 0x7fddd8148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:22.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fdddd000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fddd815e2c0 0x7fddd814f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:22.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fdddd000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddd815b310 con 0x7fddd8072bf0 2024-09-10T21:11:22.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fdddd000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fddd815b190 con 0x7fddd8151210 2024-09-10T21:11:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fdddd000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fddd815b490 con 0x7fddd815e2c0 2024-09-10T21:11:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddd5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd8072bf0 0x7fddd8139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddd5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd8072bf0 0x7fddd8139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43110/0 (socket says 172.21.15.16:43110) 2024-09-10T21:11:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddd5e00640 1 -- 172.21.15.16:0/1213997354 learned_addr learned my addr 172.21.15.16:0/1213997354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddcfe00640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddd8151210 0x7fddd8148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddd6800640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fddd815e2c0 0x7fddd814f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddd5e00640 1 -- 172.21.15.16:0/1213997354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fddd815e2c0 msgr2=0x7fddd814f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddd5e00640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fddd815e2c0 0x7fddd814f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddd5e00640 1 -- 172.21.15.16:0/1213997354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddd8151210 msgr2=0x7fddd8148c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddd5e00640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddd8151210 0x7fddd8148c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddd5e00640 1 -- 172.21.15.16:0/1213997354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd814fbe0 con 0x7fddd8072bf0 2024-09-10T21:11:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddd5e00640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd8072bf0 0x7fddd8139550 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fddc800d980 tx=0x7fddc800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddce000640 1 -- 172.21.15.16:0/1213997354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8017070 con 0x7fddd8072bf0 2024-09-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddce000640 1 -- 172.21.15.16:0/1213997354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddc8009d20 con 0x7fddd8072bf0 2024-09-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fddce000640 1 -- 172.21.15.16:0/1213997354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc80106c0 con 0x7fddd8072bf0 2024-09-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fdddd000640 1 -- 172.21.15.16:0/1213997354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd814fec0 con 0x7fddd8072bf0 2024-09-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.561+0000 7fdddd000640 1 -- 172.21.15.16:0/1213997354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd8147b10 con 0x7fddd8072bf0 2024-09-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.565+0000 7fdddd000640 1 -- 172.21.15.16:0/1213997354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd815dd80 con 0x7fddd8072bf0 2024-09-10T21:11:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.565+0000 7fddce000640 1 -- 172.21.15.16:0/1213997354 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fddc8010860 con 0x7fddd8072bf0 2024-09-10T21:11:22.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.565+0000 7fddce000640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fddb40725e0 0x7fddb4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:22.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.565+0000 7fddce000640 1 -- 172.21.15.16:0/1213997354 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fddc80935d0 con 0x7fddd8072bf0 2024-09-10T21:11:22.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.565+0000 7fddcfe00640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fddb40725e0 0x7fddb4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:22.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.565+0000 7fddcfe00640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fddb40725e0 0x7fddb4074aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fddc00af000 tx=0x7fddc001b100 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:22.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.569+0000 7fddce000640 1 -- 172.21.15.16:0/1213997354 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fddc805e650 con 0x7fddd8072bf0 2024-09-10T21:11:22.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.805+0000 7fdddd000640 1 -- 172.21.15.16:0/1213997354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fddd80b2c50 con 0x7fddd8072bf0 2024-09-10T21:11:22.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.809+0000 7fddce000640 1 -- 172.21.15.16:0/1213997354 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7fddc8063080 con 0x7fddd8072bf0 2024-09-10T21:11:22.814 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-10T21:11:22.814 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA== 2024-09-10T21:11:22.814 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-10T21:11:22.814 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-10T21:11:22.814 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:11:22.815 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-10T21:11:22.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.809+0000 7fdddd000640 1 -- 172.21.15.16:0/1213997354 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fddb40725e0 msgr2=0x7fddb4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:22.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.809+0000 7fdddd000640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fddb40725e0 0x7fddb4074aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fddc00af000 tx=0x7fddc001b100 comp rx=0 tx=0).stop 2024-09-10T21:11:22.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.809+0000 7fdddd000640 1 -- 172.21.15.16:0/1213997354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd8072bf0 msgr2=0x7fddd8139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:22.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.809+0000 7fdddd000640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd8072bf0 0x7fddd8139550 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fddc800d980 tx=0x7fddc800de40 comp rx=0 tx=0).stop 2024-09-10T21:11:22.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.809+0000 7fdddd000640 1 -- 172.21.15.16:0/1213997354 shutdown_connections 2024-09-10T21:11:22.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.809+0000 7fdddd000640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fddb40725e0 0x7fddb4074aa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.809+0000 7fdddd000640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fddd815e2c0 0x7fddd814f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.809+0000 7fdddd000640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddd8151210 0x7fddd8148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.809+0000 7fdddd000640 1 --2- 172.21.15.16:0/1213997354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd8072bf0 0x7fddd8139550 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.809+0000 7fdddd000640 1 -- 172.21.15.16:0/1213997354 >> 172.21.15.16:0/1213997354 conn(0x7fddd80b16a0 msgr2=0x7fddd806fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:22.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.809+0000 7fdddd000640 1 -- 172.21.15.16:0/1213997354 shutdown_connections 2024-09-10T21:11:22.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.809+0000 7fdddd000640 1 -- 172.21.15.16:0/1213997354 wait complete. 2024-09-10T21:11:22.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:11:22.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.977+0000 7f7d05a00640 1 -- 172.21.15.16:0/650408209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d000b13c0 msgr2=0x7f7d000afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:22.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.977+0000 7f7d05a00640 1 --2- 172.21.15.16:0/650408209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d000b13c0 0x7f7d000afac0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7d000698f0 tx=0x7f7cf801c7a0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.977+0000 7f7d05a00640 1 -- 172.21.15.16:0/650408209 shutdown_connections 2024-09-10T21:11:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.977+0000 7f7d05a00640 1 --2- 172.21.15.16:0/650408209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d000b13c0 0x7f7d000afac0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.977+0000 7f7d05a00640 1 --2- 172.21.15.16:0/650408209 >> v2:172.21.15.52:3300/0 conn(0x7f7d000716f0 0x7f7d000af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.977+0000 7f7d05a00640 1 -- 172.21.15.16:0/650408209 >> 172.21.15.16:0/650408209 conn(0x7f7d0009b130 msgr2=0x7f7d000d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.977+0000 7f7d05a00640 1 -- 172.21.15.16:0/650408209 shutdown_connections 2024-09-10T21:11:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.977+0000 7f7d05a00640 1 -- 172.21.15.16:0/650408209 wait complete. 2024-09-10T21:11:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7d05a00640 1 Processor -- start 2024-09-10T21:11:22.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7d05a00640 1 -- start start 2024-09-10T21:11:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7d05a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d000716f0 0x7f7d00148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7d05a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d000b13c0 0x7f7d0014bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7d05a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0015f400 0x7f7d00149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7d05a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d00168f60 con 0x7f7d0015f400 2024-09-10T21:11:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7d05a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7d00168de0 con 0x7f7d000716f0 2024-09-10T21:11:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7d05a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7d001690e0 con 0x7f7d000b13c0 2024-09-10T21:11:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cff400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0015f400 0x7f7d00149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cff400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0015f400 0x7f7d00149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43116/0 (socket says 172.21.15.16:43116) 2024-09-10T21:11:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cff400640 1 -- 172.21.15.16:0/65503099 learned_addr learned my addr 172.21.15.16:0/65503099 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cfea00640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d000716f0 0x7f7d00148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cf6000640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d000b13c0 0x7f7d0014bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cff400640 1 -- 172.21.15.16:0/65503099 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d000b13c0 msgr2=0x7f7d0014bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cff400640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d000b13c0 0x7f7d0014bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cff400640 1 -- 172.21.15.16:0/65503099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d000716f0 msgr2=0x7f7d00148a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cff400640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d000716f0 0x7f7d00148a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cff400640 1 -- 172.21.15.16:0/65503099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0014c220 con 0x7f7d0015f400 2024-09-10T21:11:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cf6000640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d000b13c0 0x7f7d0014bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cfea00640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d000716f0 0x7f7d00148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:22.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cff400640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0015f400 0x7f7d00149240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80b0040 tx=0x7f7cf8005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cf7400640 1 -- 172.21.15.16:0/65503099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf80b2040 con 0x7f7d0015f400 2024-09-10T21:11:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cf7400640 1 -- 172.21.15.16:0/65503099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cf80bd070 con 0x7f7d0015f400 2024-09-10T21:11:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7cf7400640 1 -- 172.21.15.16:0/65503099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8003600 con 0x7f7d0015f400 2024-09-10T21:11:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7d05a00640 1 -- 172.21.15.16:0/65503099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d000b4cc0 con 0x7f7d0015f400 2024-09-10T21:11:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7d05a00640 1 -- 172.21.15.16:0/65503099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d000b6980 con 0x7f7d0015f400 2024-09-10T21:11:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.981+0000 7f7d05a00640 1 -- 172.21.15.16:0/65503099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d000b51e0 con 0x7f7d0015f400 2024-09-10T21:11:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.985+0000 7f7cf7400640 1 -- 172.21.15.16:0/65503099 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7cf80037a0 con 0x7f7d0015f400 2024-09-10T21:11:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.985+0000 7f7cf7400640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7cdc0725e0 0x7f7cdc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.985+0000 7f7cf7400640 1 -- 172.21.15.16:0/65503099 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f7cf812db20 con 0x7f7d0015f400 2024-09-10T21:11:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.985+0000 7f7cfea00640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7cdc0725e0 0x7f7cdc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.985+0000 7f7cfea00640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7cdc0725e0 0x7f7cdc074aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0000fe0 tx=0x7f7cf0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:22.989+0000 7f7cf7400640 1 -- 172.21.15.16:0/65503099 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7d0014c220 con 0x7f7d0015f400 2024-09-10T21:11:23.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.201+0000 7f7d05a00640 1 -- 172.21.15.16:0/65503099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d00070fe0 con 0x7f7d0015f400 2024-09-10T21:11:23.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.201+0000 7f7cf7400640 1 -- 172.21.15.16:0/65503099 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5232 (secure 0 0 0) 0x7f7cf80164b0 con 0x7f7d0015f400 2024-09-10T21:11:23.209 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:11:23.209 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-10T21:11:16:232531+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/788814920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":788814920},{"type":"v1","addr":"172.21.15.16:6837","nonce":788814920}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2275087718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2275087718},{"type":"v1","addr":"172.21.15.16:6836","nonce":2275087718}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":134516,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3457135235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3457135235},{"type":"v1","addr":"172.21.15.52:6834","nonce":3457135235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:11:12.621848+0000","modified":"2024-09-10T21:11:15.636822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134519":{"gid":134519,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/4218881085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4218881085},{"type":"v1","addr":"172.21.15.52:6835","nonce":4218881085}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134519,"qdb_cluster":[134519]},"id":2}]} 2024-09-10T21:11:23.209 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-10T21:11:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.205+0000 7f7d05a00640 1 -- 172.21.15.16:0/65503099 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7cdc0725e0 msgr2=0x7f7cdc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.205+0000 7f7d05a00640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7cdc0725e0 0x7f7cdc074aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0000fe0 tx=0x7f7cf0009040 comp rx=0 tx=0).stop 2024-09-10T21:11:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.205+0000 7f7d05a00640 1 -- 172.21.15.16:0/65503099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0015f400 msgr2=0x7f7d00149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.205+0000 7f7d05a00640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0015f400 0x7f7d00149240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80b0040 tx=0x7f7cf8005e70 comp rx=0 tx=0).stop 2024-09-10T21:11:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.205+0000 7f7d05a00640 1 -- 172.21.15.16:0/65503099 shutdown_connections 2024-09-10T21:11:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.205+0000 7f7d05a00640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7cdc0725e0 0x7f7cdc074aa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.205+0000 7f7d05a00640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0015f400 0x7f7d00149240 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.205+0000 7f7d05a00640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d000b13c0 0x7f7d0014bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.205+0000 7f7d05a00640 1 --2- 172.21.15.16:0/65503099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d000716f0 0x7f7d00148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.205+0000 7f7d05a00640 1 -- 172.21.15.16:0/65503099 >> 172.21.15.16:0/65503099 conn(0x7f7d0009b130 msgr2=0x7f7d000ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.205+0000 7f7d05a00640 1 -- 172.21.15.16:0/65503099 shutdown_connections 2024-09-10T21:11:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.205+0000 7f7d05a00640 1 -- 172.21.15.16:0/65503099 wait complete. 2024-09-10T21:11:23.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:11:23.247 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:11:23.633 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:11:12.621848+0000', 'modified': '2024-09-10T21:11:15.636822+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 134519}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_134519': {'gid': 134519, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.52:6835/4218881085', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6833', 'nonce': 4218881085}, {'type': 'v1', 'addr': '172.21.15.52:6835', 'nonce': 4218881085}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 134519, 'qdb_cluster': [134519]} 2024-09-10T21:11:23.634 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:11:23.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T21:11:23.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.777+0000 7f2aa9000640 1 -- 172.21.15.16:0/3136824280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa40705e0 msgr2=0x7f2aa40ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.777+0000 7f2aa9000640 1 --2- 172.21.15.16:0/3136824280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa40705e0 0x7f2aa40ce450 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2aa406e130 tx=0x7f2a8c01c770 comp rx=0 tx=0).stop 2024-09-10T21:11:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.777+0000 7f2aa9000640 1 -- 172.21.15.16:0/3136824280 shutdown_connections 2024-09-10T21:11:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.777+0000 7f2aa9000640 1 --2- 172.21.15.16:0/3136824280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa40705e0 0x7f2aa40ce450 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.777+0000 7f2aa9000640 1 --2- 172.21.15.16:0/3136824280 >> v2:172.21.15.52:3300/0 conn(0x7f2aa415a270 0x7f2aa40700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.777+0000 7f2aa9000640 1 -- 172.21.15.16:0/3136824280 >> 172.21.15.16:0/3136824280 conn(0x7f2aa40cfb20 msgr2=0x7f2aa40cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.777+0000 7f2aa9000640 1 -- 172.21.15.16:0/3136824280 shutdown_connections 2024-09-10T21:11:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.777+0000 7f2aa9000640 1 -- 172.21.15.16:0/3136824280 wait complete. 2024-09-10T21:11:23.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2aa9000640 1 Processor -- start 2024-09-10T21:11:23.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2aa9000640 1 -- start start 2024-09-10T21:11:23.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2aa9000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2aa40705e0 0x7f2aa40af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2aa9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa40ce990 0x7f2aa40c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2aa9000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2aa415a270 0x7f2aa4163e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2aa9000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa4072ad0 con 0x7f2aa40ce990 2024-09-10T21:11:23.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2aa9000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa4072950 con 0x7f2aa40705e0 2024-09-10T21:11:23.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2aa9000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa4072c50 con 0x7f2aa415a270 2024-09-10T21:11:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2a9be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa40ce990 0x7f2aa40c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2a9be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa40ce990 0x7f2aa40c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43122/0 (socket says 172.21.15.16:43122) 2024-09-10T21:11:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2a9be00640 1 -- 172.21.15.16:0/91433775 learned_addr learned my addr 172.21.15.16:0/91433775 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2a9be00640 1 -- 172.21.15.16:0/91433775 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2aa415a270 msgr2=0x7f2aa4163e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2a9be00640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2aa415a270 0x7f2aa4163e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2a9be00640 1 -- 172.21.15.16:0/91433775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2aa40705e0 msgr2=0x7f2aa40af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2aa40705e0 0x7f2aa40af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2a9be00640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2aa40705e0 0x7f2aa40af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2a9be00640 1 -- 172.21.15.16:0/91433775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa40afbb0 con 0x7f2aa40ce990 2024-09-10T21:11:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2aa40705e0 0x7f2aa40af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2a9be00640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa40ce990 0x7f2aa40c9610 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c000c10 tx=0x7f2a8c0a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2a9a000640 1 -- 172.21.15.16:0/91433775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c0b2040 con 0x7f2aa40ce990 2024-09-10T21:11:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2a9a000640 1 -- 172.21.15.16:0/91433775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a8c004340 con 0x7f2aa40ce990 2024-09-10T21:11:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2a9a000640 1 -- 172.21.15.16:0/91433775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c0044f0 con 0x7f2aa40ce990 2024-09-10T21:11:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2aa9000640 1 -- 172.21.15.16:0/91433775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa40afe90 con 0x7f2aa40ce990 2024-09-10T21:11:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.781+0000 7f2aa9000640 1 -- 172.21.15.16:0/91433775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa40af7f0 con 0x7f2aa40ce990 2024-09-10T21:11:23.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.785+0000 7f2a9a000640 1 -- 172.21.15.16:0/91433775 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2a8c0b5070 con 0x7f2aa40ce990 2024-09-10T21:11:23.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.785+0000 7f2a9a000640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a800725e0 0x7f2a80074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.785+0000 7f2a9a000640 1 -- 172.21.15.16:0/91433775 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2a8c12d250 con 0x7f2aa40ce990 2024-09-10T21:11:23.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.785+0000 7f2aac49c000 1 Processor -- start 2024-09-10T21:11:23.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.785+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a800725e0 0x7f2a80074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.785+0000 7f2aac49c000 1 -- start start 2024-09-10T21:11:23.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.785+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a800725e0 0x7f2a80074aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2a94005fd0 tx=0x7f2a94005dc0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f2199d0 0x56504f219db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f21a2f0 0x56504f1cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 0x56504f1ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56504f0ee3f0 con 0x56504f1cde10 2024-09-10T21:11:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56504f02bb20 con 0x56504f2199d0 2024-09-10T21:11:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x56504f01af50 con 0x56504f21a2f0 2024-09-10T21:11:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aa2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 0x56504f1ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aa2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 0x56504f1ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43124/0 (socket says 172.21.15.16:43124) 2024-09-10T21:11:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aa2800640 1 -- 172.21.15.16:0/3647833226 learned_addr learned my addr 172.21.15.16:0/3647833226 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/3647833226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f2199d0 0x56504f219db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2a9be00640 1 --2- 172.21.15.16:0/3647833226 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f21a2f0 0x56504f1cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aa2800640 1 -- 172.21.15.16:0/3647833226 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f21a2f0 msgr2=0x56504f1cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:23.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aa2800640 1 --2- 172.21.15.16:0/3647833226 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f21a2f0 0x56504f1cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aa2800640 1 -- 172.21.15.16:0/3647833226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f2199d0 msgr2=0x56504f219db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:23.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aa2800640 1 --2- 172.21.15.16:0/3647833226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f2199d0 0x56504f219db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aa2800640 1 -- 172.21.15.16:0/3647833226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56504efe2590 con 0x56504f1cde10 2024-09-10T21:11:23.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2a9be00640 1 --2- 172.21.15.16:0/3647833226 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f21a2f0 0x56504f1cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:23.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aa2800640 1 --2- 172.21.15.16:0/3647833226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 0x56504f1ce1f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0020b0 tx=0x7f2a9c0021a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d5cce8dfa9d919 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:23.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2a92a00640 1 -- 172.21.15.16:0/3647833226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c00cc50 con 0x56504f1cde10 2024-09-10T21:11:23.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2a92a00640 1 -- 172.21.15.16:0/3647833226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a9c017070 con 0x56504f1cde10 2024-09-10T21:11:23.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2a92a00640 1 -- 172.21.15.16:0/3647833226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c012640 con 0x56504f1cde10 2024-09-10T21:11:23.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 -- 172.21.15.16:0/3647833226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 msgr2=0x56504f1ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 --2- 172.21.15.16:0/3647833226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 0x56504f1ce1f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0020b0 tx=0x7f2a9c0021a0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 -- 172.21.15.16:0/3647833226 shutdown_connections 2024-09-10T21:11:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 --2- 172.21.15.16:0/3647833226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 0x56504f1ce1f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 --2- 172.21.15.16:0/3647833226 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f21a2f0 0x56504f1cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 --2- 172.21.15.16:0/3647833226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f2199d0 0x56504f219db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 -- 172.21.15.16:0/3647833226 >> 172.21.15.16:0/3647833226 conn(0x56504f0b7130 msgr2=0x56504f1fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 -- 172.21.15.16:0/3647833226 shutdown_connections 2024-09-10T21:11:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 -- 172.21.15.16:0/3647833226 wait complete. 2024-09-10T21:11:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 -- start start 2024-09-10T21:11:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.789+0000 7f2aac49c000 1 Processor -- start 2024-09-10T21:11:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f1cde10 0x56504f1fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f2199d0 0x56504f1fae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f21a2f0 0x56504f1fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56504efb55d0 con 0x56504f2199d0 2024-09-10T21:11:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56504f01ed60 con 0x56504f1cde10 2024-09-10T21:11:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x56504efc1500 con 0x56504f21a2f0 2024-09-10T21:11:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 10 client.0 _ll_register_callbacks cb 0x56504efc5810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a9be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f2199d0 0x56504f1fae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a9be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f2199d0 0x56504f1fae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43128/0 (socket says 172.21.15.16:43128) 2024-09-10T21:11:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a9be00640 1 -- 172.21.15.16:0/1624013484 learned_addr learned my addr 172.21.15.16:0/1624013484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/1624013484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f1cde10 0x56504f1fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aa2800640 1 --2- 172.21.15.16:0/1624013484 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f21a2f0 0x56504f1fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a9be00640 1 -- 172.21.15.16:0/1624013484 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f21a2f0 msgr2=0x56504f1fb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a9be00640 1 --2- 172.21.15.16:0/1624013484 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f21a2f0 0x56504f1fb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a9be00640 1 -- 172.21.15.16:0/1624013484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f1cde10 msgr2=0x56504f1fa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a9be00640 1 --2- 172.21.15.16:0/1624013484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f1cde10 0x56504f1fa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a9be00640 1 -- 172.21.15.16:0/1624013484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504efb9170 con 0x56504f2199d0 2024-09-10T21:11:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a9be00640 1 --2- 172.21.15.16:0/1624013484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f2199d0 0x56504f1fae70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0f7f20 tx=0x7f2a8c0f7ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fdabe6084bc45d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a87400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:11:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a87400640 1 -- 172.21.15.16:0/1624013484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c101060 con 0x56504f2199d0 2024-09-10T21:11:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a87400640 1 -- 172.21.15.16:0/1624013484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2a8c11ad90 con 0x56504f2199d0 2024-09-10T21:11:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 1 -- 172.21.15.16:0/1624013484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56504efb30f0 con 0x56504f2199d0 2024-09-10T21:11:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a87400640 1 -- 172.21.15.16:0/1624013484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f2a8c0f7d30 con 0x56504f2199d0 2024-09-10T21:11:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 10 client.5016 fetch_fsmap learned FSMap version 16 2024-09-10T21:11:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 1 -- 172.21.15.16:0/1624013484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x56504f09fef0 con 0x56504f2199d0 2024-09-10T21:11:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a87400640 1 -- 172.21.15.16:0/1624013484 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56504efb30f0 con 0x56504f2199d0 2024-09-10T21:11:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a87400640 1 -- 172.21.15.16:0/1624013484 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7f2a8c0ff0b0 con 0x56504f2199d0 2024-09-10T21:11:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 10 client.5016 fetch_fsmap finished waiting for FSMap version 16 2024-09-10T21:11:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 10 client.5016 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:11:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 20 client.5016 populate_metadata read hostname 'smithi016' 2024-09-10T21:11:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 1 --2- 172.21.15.16:0/1624013484 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x56504f1fc0f0 0x56504f22bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 4 client.5016 mds_command: new command op to 134519 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:11:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 1 -- 172.21.15.16:0/1624013484 --> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56504f02da70 con 0x56504f1fc0f0 2024-09-10T21:11:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/1624013484 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x56504f1fc0f0 0x56504f22bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a87400640 10 client.5016 ms_handle_connect on v2:172.21.15.52:6833/4218881085 2024-09-10T21:11:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/1624013484 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x56504f1fc0f0 0x56504f22bc90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1078e750a17cf99 server_cookie=59c5e92239836e28 in_seq=0 out_seq=0 2024-09-10T21:11:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a87400640 1 -- 172.21.15.16:0/1624013484 <== mds.0 v2:172.21.15.52:6833/4218881085 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56504f02da70 con 0x56504f1fc0f0 2024-09-10T21:11:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2a87400640 10 client.5016 handle_command_reply: tid=0 2024-09-10T21:11:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.793+0000 7f2aac49c000 1 client.5016 shutdown 2024-09-10T21:11:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.797+0000 7f2aac49c000 1 -- 172.21.15.16:0/1624013484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f2199d0 msgr2=0x56504f1fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.797+0000 7f2aac49c000 1 --2- 172.21.15.16:0/1624013484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f2199d0 0x56504f1fae70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0f7f20 tx=0x7f2a8c0f7ff0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.797+0000 7f2aac49c000 1 -- 172.21.15.16:0/1624013484 shutdown_connections 2024-09-10T21:11:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.797+0000 7f2aac49c000 1 --2- 172.21.15.16:0/1624013484 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x56504f1fc0f0 0x56504f22bc90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.797+0000 7f2aac49c000 1 --2- 172.21.15.16:0/1624013484 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f21a2f0 0x56504f1fb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.797+0000 7f2aac49c000 1 --2- 172.21.15.16:0/1624013484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f2199d0 0x56504f1fae70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.797+0000 7f2aac49c000 1 --2- 172.21.15.16:0/1624013484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f1cde10 0x56504f1fa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.797+0000 7f2a87400640 0 client.5016 ms_handle_reset on v2:172.21.15.52:6833/4218881085 2024-09-10T21:11:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.797+0000 7f2aac49c000 1 -- 172.21.15.16:0/1624013484 >> 172.21.15.16:0/1624013484 conn(0x56504f0b7130 msgr2=0x56504f1ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.797+0000 7f2aac49c000 1 -- 172.21.15.16:0/1624013484 shutdown_connections 2024-09-10T21:11:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.797+0000 7f2aac49c000 1 -- 172.21.15.16:0/1624013484 wait complete. 2024-09-10T21:11:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.797+0000 7f2aac49c000 20 client.5016 trim_cache size 0 max 16384 2024-09-10T21:11:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.813+0000 7f2aac49c000 1 Processor -- start 2024-09-10T21:11:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.813+0000 7f2aac49c000 1 -- start start 2024-09-10T21:11:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 0x56504f28bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f2199d0 0x56504f28c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f21a2f0 0x56504f28c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa4203bf0 con 0x56504f1cde10 2024-09-10T21:11:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa4202cb0 con 0x56504f21a2f0 2024-09-10T21:11:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa41fbce0 con 0x56504f2199d0 2024-09-10T21:11:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aa1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 0x56504f28bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aa1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 0x56504f28bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43136/0 (socket says 172.21.15.16:43136) 2024-09-10T21:11:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aa1e00640 1 -- 172.21.15.16:0/3205803555 learned_addr learned my addr 172.21.15.16:0/3205803555 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aa2800640 1 --2- 172.21.15.16:0/3205803555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f21a2f0 0x56504f28c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2a9be00640 1 --2- 172.21.15.16:0/3205803555 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f2199d0 0x56504f28c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aa1e00640 1 -- 172.21.15.16:0/3205803555 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f2199d0 msgr2=0x56504f28c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/3205803555 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f2199d0 0x56504f28c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aa1e00640 1 -- 172.21.15.16:0/3205803555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f21a2f0 msgr2=0x56504f28c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/3205803555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f21a2f0 0x56504f28c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aa1e00640 1 -- 172.21.15.16:0/3205803555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56504f076d90 con 0x56504f1cde10 2024-09-10T21:11:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2a9be00640 1 --2- 172.21.15.16:0/3205803555 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f2199d0 0x56504f28c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/3205803555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 0x56504f28bbc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x56504efb3390 tx=0x7f2a94011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d37a9f90e2d2bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2a92000640 1 -- 172.21.15.16:0/3205803555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9401add0 con 0x56504f1cde10 2024-09-10T21:11:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2a92000640 1 -- 172.21.15.16:0/3205803555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a94004020 con 0x56504f1cde10 2024-09-10T21:11:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2a92000640 1 -- 172.21.15.16:0/3205803555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a940043d0 con 0x56504f1cde10 2024-09-10T21:11:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 -- 172.21.15.16:0/3205803555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 msgr2=0x56504f28bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 --2- 172.21.15.16:0/3205803555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 0x56504f28bbc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x56504efb3390 tx=0x7f2a94011000 comp rx=0 tx=0).stop 2024-09-10T21:11:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 -- 172.21.15.16:0/3205803555 shutdown_connections 2024-09-10T21:11:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 --2- 172.21.15.16:0/3205803555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f21a2f0 0x56504f28c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 --2- 172.21.15.16:0/3205803555 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f2199d0 0x56504f28c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 --2- 172.21.15.16:0/3205803555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f1cde10 0x56504f28bbc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 -- 172.21.15.16:0/3205803555 >> 172.21.15.16:0/3205803555 conn(0x56504f0b7130 msgr2=0x56504f226fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 -- 172.21.15.16:0/3205803555 shutdown_connections 2024-09-10T21:11:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 -- 172.21.15.16:0/3205803555 wait complete. 2024-09-10T21:11:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 -- start start 2024-09-10T21:11:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.817+0000 7f2aac49c000 1 Processor -- start 2024-09-10T21:11:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f1cde10 0x56504f28f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f2199d0 0x56504f28fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f21a2f0 0x56504f290050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56504efe6ec0 con 0x56504f21a2f0 2024-09-10T21:11:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56504f02a8f0 con 0x56504f2199d0 2024-09-10T21:11:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x56504f023390 con 0x56504f1cde10 2024-09-10T21:11:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 10 client.0 _ll_register_callbacks cb 0x7f2a8c009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aa2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f21a2f0 0x56504f290050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aa2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f21a2f0 0x56504f290050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43152/0 (socket says 172.21.15.16:43152) 2024-09-10T21:11:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aa2800640 1 -- 172.21.15.16:0/1628006029 learned_addr learned my addr 172.21.15.16:0/1628006029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2a9be00640 1 --2- 172.21.15.16:0/1628006029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f2199d0 0x56504f28fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/1628006029 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f1cde10 0x56504f28f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aa2800640 1 -- 172.21.15.16:0/1628006029 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f1cde10 msgr2=0x56504f28f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aa2800640 1 --2- 172.21.15.16:0/1628006029 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f1cde10 0x56504f28f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aa2800640 1 -- 172.21.15.16:0/1628006029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f2199d0 msgr2=0x56504f28fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aa2800640 1 --2- 172.21.15.16:0/1628006029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f2199d0 0x56504f28fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aa2800640 1 -- 172.21.15.16:0/1628006029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504efe1a90 con 0x56504f21a2f0 2024-09-10T21:11:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aa2800640 1 --2- 172.21.15.16:0/1628006029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f21a2f0 0x56504f290050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c014040 tx=0x7f2a9c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ab1938915ffb5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2a87400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:11:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2a87400640 1 -- 172.21.15.16:0/1628006029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c02a060 con 0x56504f21a2f0 2024-09-10T21:11:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2a87400640 1 -- 172.21.15.16:0/1628006029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2a9c012630 con 0x56504f21a2f0 2024-09-10T21:11:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 -- 172.21.15.16:0/1628006029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a94004020 con 0x56504f21a2f0 2024-09-10T21:11:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2a87400640 1 -- 172.21.15.16:0/1628006029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f2a9c030050 con 0x56504f21a2f0 2024-09-10T21:11:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 10 client.5022 fetch_fsmap learned FSMap version 16 2024-09-10T21:11:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 -- 172.21.15.16:0/1628006029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x56504efc0050 con 0x56504f21a2f0 2024-09-10T21:11:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2a87400640 1 -- 172.21.15.16:0/1628006029 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a94004020 con 0x56504f21a2f0 2024-09-10T21:11:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2a87400640 1 -- 172.21.15.16:0/1628006029 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7f2a9c033020 con 0x56504f21a2f0 2024-09-10T21:11:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 10 client.5022 fetch_fsmap finished waiting for FSMap version 16 2024-09-10T21:11:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 10 client.5022 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:11:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 20 client.5022 populate_metadata read hostname 'smithi016' 2024-09-10T21:11:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 --2- 172.21.15.16:0/1628006029 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x56504f22e7f0 0x56504f22ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 4 client.5022 mds_command: new command op to 134519 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:11:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 -- 172.21.15.16:0/1628006029 --> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] -- command(tid 0: {"prefix": "status"}) -- 0x56504f029520 con 0x56504f22e7f0 2024-09-10T21:11:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/1628006029 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x56504f22e7f0 0x56504f22ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aa1e00640 1 --2- 172.21.15.16:0/1628006029 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x56504f22e7f0 0x56504f22ebd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e94ba297271e6 server_cookie=6e53e6f65372a02b in_seq=0 out_seq=0 2024-09-10T21:11:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2a87400640 10 client.5022 ms_handle_connect on v2:172.21.15.52:6833/4218881085 2024-09-10T21:11:23.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2a87400640 1 -- 172.21.15.16:0/1628006029 <== mds.0 v2:172.21.15.52:6833/4218881085 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56504f029520 con 0x56504f22e7f0 2024-09-10T21:11:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2a87400640 10 client.5022 handle_command_reply: tid=0 2024-09-10T21:11:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 client.5022 shutdown 2024-09-10T21:11:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 -- 172.21.15.16:0/1628006029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f21a2f0 msgr2=0x56504f290050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 --2- 172.21.15.16:0/1628006029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f21a2f0 0x56504f290050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c014040 tx=0x7f2a9c00cd50 comp rx=0 tx=0).stop 2024-09-10T21:11:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 -- 172.21.15.16:0/1628006029 shutdown_connections 2024-09-10T21:11:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 --2- 172.21.15.16:0/1628006029 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x56504f22e7f0 0x56504f22ebd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 --2- 172.21.15.16:0/1628006029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56504f21a2f0 0x56504f290050 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 --2- 172.21.15.16:0/1628006029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504f2199d0 0x56504f28fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 --2- 172.21.15.16:0/1628006029 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x56504f1cde10 0x56504f28f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2a87400640 0 client.5022 ms_handle_reset on v2:172.21.15.52:6833/4218881085 2024-09-10T21:11:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 -- 172.21.15.16:0/1628006029 >> 172.21.15.16:0/1628006029 conn(0x56504f0b7130 msgr2=0x56504f1d5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 -- 172.21.15.16:0/1628006029 shutdown_connections 2024-09-10T21:11:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.821+0000 7f2aac49c000 1 -- 172.21.15.16:0/1628006029 wait complete. 2024-09-10T21:11:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.825+0000 7f2aac49c000 20 client.5022 trim_cache size 0 max 16384 2024-09-10T21:11:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:11:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "eb19be60-6d8f-4816-b778-3d7d6698304d", 2024-09-10T21:11:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-10T21:11:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 134519, 2024-09-10T21:11:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-10T21:11:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-10T21:11:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-10T21:11:23.838 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 10.620134734000001, 2024-09-10T21:11:23.838 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 16, 2024-09-10T21:11:23.838 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 32, 2024-09-10T21:11:23.838 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 32, 2024-09-10T21:11:23.838 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 18.000294601 2024-09-10T21:11:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:11:23.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.825+0000 7f2aa9000640 1 -- 172.21.15.16:0/91433775 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a800725e0 msgr2=0x7f2a80074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:23.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.825+0000 7f2aa9000640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a800725e0 0x7f2a80074aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2a94005fd0 tx=0x7f2a94005dc0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.825+0000 7f2aa9000640 1 -- 172.21.15.16:0/91433775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa40ce990 msgr2=0x7f2aa40c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.825+0000 7f2aa9000640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa40ce990 0x7f2aa40c9610 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c000c10 tx=0x7f2a8c0a5630 comp rx=0 tx=0).stop 2024-09-10T21:11:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.825+0000 7f2aa9000640 1 -- 172.21.15.16:0/91433775 shutdown_connections 2024-09-10T21:11:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.825+0000 7f2aa9000640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a800725e0 0x7f2a80074aa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.825+0000 7f2aa9000640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2aa415a270 0x7f2aa4163e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.825+0000 7f2aa9000640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa40ce990 0x7f2aa40c9610 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.825+0000 7f2aa9000640 1 --2- 172.21.15.16:0/91433775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2aa40705e0 0x7f2aa40af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.825+0000 7f2aa9000640 1 -- 172.21.15.16:0/91433775 >> 172.21.15.16:0/91433775 conn(0x7f2aa40cfb20 msgr2=0x7f2aa40b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.825+0000 7f2aa9000640 1 -- 172.21.15.16:0/91433775 shutdown_connections 2024-09-10T21:11:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.825+0000 7f2aa9000640 1 -- 172.21.15.16:0/91433775 wait complete. 2024-09-10T21:11:23.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:11:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.989+0000 7f3acf400640 1 -- 172.21.15.16:0/2198169601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ab0003f80 msgr2=0x7f3ab0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.989+0000 7f3acf400640 1 --2- 172.21.15.16:0/2198169601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ab0003f80 0x7f3ab0004810 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3ac809b860 tx=0x7f3ab401d1d0 comp rx=0 tx=0).stop 2024-09-10T21:11:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.989+0000 7f3acd400640 1 -- 172.21.15.16:0/2198169601 reap_dead start 2024-09-10T21:11:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.989+0000 7f3acf400640 1 -- 172.21.15.16:0/2198169601 shutdown_connections 2024-09-10T21:11:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.989+0000 7f3acf400640 1 -- 172.21.15.16:0/2198169601 >> 172.21.15.16:0/2198169601 conn(0x7f3ac809b130 msgr2=0x7f3ac80d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.989+0000 7f3acf400640 1 -- 172.21.15.16:0/2198169601 shutdown_connections 2024-09-10T21:11:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.989+0000 7f3acf400640 1 -- 172.21.15.16:0/2198169601 wait complete. 2024-09-10T21:11:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acf400640 1 Processor -- start 2024-09-10T21:11:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acf400640 1 -- start start 2024-09-10T21:11:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acf400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ab0003f80 0x7f3ac81f8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acf400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac8162740 0x7f3ac8152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac8160610 0x7f3ac8160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acf400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac8167f60 con 0x7f3ac8160610 2024-09-10T21:11:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acf400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac8167de0 con 0x7f3ac8162740 2024-09-10T21:11:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acf400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac81680e0 con 0x7f3ab0003f80 2024-09-10T21:11:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac8160610 0x7f3ac8160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acca00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ab0003f80 0x7f3ac81f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac8160610 0x7f3ac8160ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43156/0 (socket says 172.21.15.16:43156) 2024-09-10T21:11:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acd400640 1 -- 172.21.15.16:0/2296305770 learned_addr learned my addr 172.21.15.16:0/2296305770 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3ac7e00640 1 --2- 172.21.15.16:0/2296305770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac8162740 0x7f3ac8152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acd400640 1 -- 172.21.15.16:0/2296305770 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ab0003f80 msgr2=0x7f3ac81f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acd400640 1 --2- 172.21.15.16:0/2296305770 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ab0003f80 0x7f3ac81f8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acd400640 1 -- 172.21.15.16:0/2296305770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac8162740 msgr2=0x7f3ac8152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acd400640 1 --2- 172.21.15.16:0/2296305770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac8162740 0x7f3ac8152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acd400640 1 -- 172.21.15.16:0/2296305770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac815ff70 con 0x7f3ac8160610 2024-09-10T21:11:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acd400640 1 --2- 172.21.15.16:0/2296305770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac8160610 0x7f3ac8160ac0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80cc380 tx=0x7f3ac0006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3ac4c00640 1 -- 172.21.15.16:0/2296305770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac000fa50 con 0x7f3ac8160610 2024-09-10T21:11:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3ac4c00640 1 -- 172.21.15.16:0/2296305770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ac000fbf0 con 0x7f3ac8160610 2024-09-10T21:11:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3ac4c00640 1 -- 172.21.15.16:0/2296305770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0011430 con 0x7f3ac8160610 2024-09-10T21:11:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.993+0000 7f3acf400640 1 -- 172.21.15.16:0/2296305770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac8160250 con 0x7f3ac8160610 2024-09-10T21:11:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.997+0000 7f3acf400640 1 -- 172.21.15.16:0/2296305770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac81578f0 con 0x7f3ac8160610 2024-09-10T21:11:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.997+0000 7f3acf400640 1 -- 172.21.15.16:0/2296305770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac80afac0 con 0x7f3ac8160610 2024-09-10T21:11:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.997+0000 7f3ac4c00640 1 -- 172.21.15.16:0/2296305770 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3ac00117e0 con 0x7f3ac8160610 2024-09-10T21:11:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.997+0000 7f3ac4c00640 1 --2- 172.21.15.16:0/2296305770 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3aa8072610 0x7f3aa8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.997+0000 7f3ac4c00640 1 -- 172.21.15.16:0/2296305770 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3ac0093bc0 con 0x7f3ac8160610 2024-09-10T21:11:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:23.997+0000 7f3acca00640 1 --2- 172.21.15.16:0/2296305770 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3aa8072610 0x7f3aa8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.001+0000 7f3acca00640 1 --2- 172.21.15.16:0/2296305770 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3aa8072610 0x7f3aa8074ad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3abc001ec0 tx=0x7f3abc0066d0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.001+0000 7f3ac4c00640 1 -- 172.21.15.16:0/2296305770 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3ac005ebb0 con 0x7f3ac8160610 2024-09-10T21:11:24.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3acf400640 1 -- 172.21.15.16:0/2296305770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3ac8071430 con 0x7f3ac8160610 2024-09-10T21:11:24.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3ac4c00640 1 -- 172.21.15.16:0/2296305770 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f3ac00635e0 con 0x7f3ac8160610 2024-09-10T21:11:24.219 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:11:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3acf400640 1 -- 172.21.15.16:0/2296305770 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3aa8072610 msgr2=0x7f3aa8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3acf400640 1 --2- 172.21.15.16:0/2296305770 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3aa8072610 0x7f3aa8074ad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3abc001ec0 tx=0x7f3abc0066d0 comp rx=0 tx=0).stop 2024-09-10T21:11:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3acf400640 1 -- 172.21.15.16:0/2296305770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac8160610 msgr2=0x7f3ac8160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3acf400640 1 --2- 172.21.15.16:0/2296305770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac8160610 0x7f3ac8160ac0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80cc380 tx=0x7f3ac0006720 comp rx=0 tx=0).stop 2024-09-10T21:11:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3acf400640 1 -- 172.21.15.16:0/2296305770 shutdown_connections 2024-09-10T21:11:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3acf400640 1 --2- 172.21.15.16:0/2296305770 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3aa8072610 0x7f3aa8074ad0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3acf400640 1 --2- 172.21.15.16:0/2296305770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ac8160610 0x7f3ac8160ac0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:24.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3acf400640 1 --2- 172.21.15.16:0/2296305770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac8162740 0x7f3ac8152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:24.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3acf400640 1 --2- 172.21.15.16:0/2296305770 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ab0003f80 0x7f3ac81f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:24.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3acf400640 1 -- 172.21.15.16:0/2296305770 >> 172.21.15.16:0/2296305770 conn(0x7f3ac809b130 msgr2=0x7f3ac80d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:24.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3acf400640 1 -- 172.21.15.16:0/2296305770 shutdown_connections 2024-09-10T21:11:24.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24.213+0000 7f3acf400640 1 -- 172.21.15.16:0/2296305770 wait complete. 2024-09-10T21:11:24.241 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:11:24.242 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:11:24.242 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-10T21:11:24.242 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:11:24.242 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:11:24.242 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:11:24.242 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:11:24.242 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:11:24.242 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:11:24.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:24.242 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-10T21:11:24.252 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:11:24.253 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:11:24.253 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:11:24.253 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:24.253 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:11:24.253 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:24.253 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:11:24.253 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:11:24.253 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-10T21:11:24.253 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:11:24.253 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:11:24.253 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-10T21:11:24.254 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:11:24.254 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:11:24.254 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-10T21:11:24.254 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:11:24.254 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42365sec preferred_lft 42365sec 2024-09-10T21:11:24.254 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-10T21:11:24.254 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:24.254 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:11:24.254 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:11:24.254 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-10T21:11:24.254 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:11:24.254 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:30:fb:74:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:11:24.254 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:11:24.255 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:11:24.256 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:24.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:24.256 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:11:24.311 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-10T21:11:24.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:24.313 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-10T21:11:24.366 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-10T21:11:24.366 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:11:24.367 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:11:24.367 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:11:24.367 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:11:24.367 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:11:24.367 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:11:24.367 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:11:24.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:11:24.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:11:24.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:11:24.368 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:11:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:11:24.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:11:24.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:24.504 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:11:24.514 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:11:24.514 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:11:24.515 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:11:24.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:24.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-10T21:11:24.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:11:24.577 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:11:24.629 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:11:24.629 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:11:24.629 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:11:24.681 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:11:24.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:24.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:11:24.769 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:11:24.770 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:11:24.784 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:11:24.784 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:11:24.784 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:11:24.784 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:11:24.784 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:11:24.788 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:11:24.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:24.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:11:24.808 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:11:24.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:11:24.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:11:24.921 INFO:teuthology.orchestra.run.smithi016.stdout:client4939 2024-09-10T21:11:24.922 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:11:24.922 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:11:24.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:11:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.117+0000 7f375a200640 1 -- 172.21.15.16:0/3760584098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37540cfd40 msgr2=0x7f37540d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.117+0000 7f375a200640 1 --2- 172.21.15.16:0/3760584098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37540cfd40 0x7f37540d0120 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3754074370 tx=0x7f374401c830 comp rx=0 tx=0).stop 2024-09-10T21:11:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.117+0000 7f375a200640 1 -- 172.21.15.16:0/3760584098 shutdown_connections 2024-09-10T21:11:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.117+0000 7f375a200640 1 --2- 172.21.15.16:0/3760584098 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3754160120 0x7f3754160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.117+0000 7f375a200640 1 --2- 172.21.15.16:0/3760584098 >> v2:172.21.15.52:3300/0 conn(0x7f37540d2820 0x7f375415fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.117+0000 7f375a200640 1 --2- 172.21.15.16:0/3760584098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37540cfd40 0x7f37540d0120 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.117+0000 7f375a200640 1 -- 172.21.15.16:0/3760584098 >> 172.21.15.16:0/3760584098 conn(0x7f37540d1d00 msgr2=0x7f37540d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.117+0000 7f375a200640 1 -- 172.21.15.16:0/3760584098 shutdown_connections 2024-09-10T21:11:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.117+0000 7f375a200640 1 -- 172.21.15.16:0/3760584098 wait complete. 2024-09-10T21:11:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f375a200640 1 Processor -- start 2024-09-10T21:11:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f375a200640 1 -- start start 2024-09-10T21:11:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f375a200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37540cfd40 0x7f375414b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f375a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37540d2820 0x7f375415ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f375a200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3754160120 0x7f375415d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f375a200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37540b2610 con 0x7f37540d2820 2024-09-10T21:11:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f375a200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f37540b2490 con 0x7f37540cfd40 2024-09-10T21:11:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f375a200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f37540b2790 con 0x7f3754160120 2024-09-10T21:11:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f374be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37540d2820 0x7f375415ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f374be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37540d2820 0x7f375415ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43162/0 (socket says 172.21.15.16:43162) 2024-09-10T21:11:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f374be00640 1 -- 172.21.15.16:0/3697045910 learned_addr learned my addr 172.21.15.16:0/3697045910 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f3758e00640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37540cfd40 0x7f375414b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f3759800640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3754160120 0x7f375415d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f374be00640 1 -- 172.21.15.16:0/3697045910 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3754160120 msgr2=0x7f375415d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f374be00640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3754160120 0x7f375415d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f374be00640 1 -- 172.21.15.16:0/3697045910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37540cfd40 msgr2=0x7f375414b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f374be00640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37540cfd40 0x7f375414b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f374be00640 1 -- 172.21.15.16:0/3697045910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37540d25d0 con 0x7f37540d2820 2024-09-10T21:11:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f3758e00640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37540cfd40 0x7f375414b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f374be00640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37540d2820 0x7f375415ce70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f373c00d770 tx=0x7f373c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f3751600640 1 -- 172.21.15.16:0/3697045910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c017070 con 0x7f37540d2820 2024-09-10T21:11:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f3751600640 1 -- 172.21.15.16:0/3697045910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f373c009d20 con 0x7f37540d2820 2024-09-10T21:11:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f3751600640 1 -- 172.21.15.16:0/3697045910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c0106c0 con 0x7f37540d2820 2024-09-10T21:11:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f375a200640 1 -- 172.21.15.16:0/3697045910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37540caef0 con 0x7f37540d2820 2024-09-10T21:11:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f375a200640 1 -- 172.21.15.16:0/3697045910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37540cb4b0 con 0x7f37540d2820 2024-09-10T21:11:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.121+0000 7f375a200640 1 -- 172.21.15.16:0/3697045910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f375415fbe0 con 0x7f37540d2820 2024-09-10T21:11:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.125+0000 7f3751600640 1 -- 172.21.15.16:0/3697045910 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f373c010860 con 0x7f37540d2820 2024-09-10T21:11:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.125+0000 7f3751600640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3730072610 0x7f3730074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.125+0000 7f3751600640 1 -- 172.21.15.16:0/3697045910 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f373c093740 con 0x7f37540d2820 2024-09-10T21:11:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.125+0000 7f3758e00640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3730072610 0x7f3730074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.125+0000 7f3758e00640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3730072610 0x7f3730074ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f37440af000 tx=0x7f374401b070 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.129+0000 7f3751600640 1 -- 172.21.15.16:0/3697045910 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f373c05ed80 con 0x7f37540d2820 2024-09-10T21:11:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.341+0000 7f375a200640 1 -- 172.21.15.16:0/3697045910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f375415b7f0 con 0x7f37540d2820 2024-09-10T21:11:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.341+0000 7f3751600640 1 -- 172.21.15.16:0/3697045910 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f373c05e680 con 0x7f37540d2820 2024-09-10T21:11:25.348 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:11:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.345+0000 7f375a200640 1 -- 172.21.15.16:0/3697045910 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3730072610 msgr2=0x7f3730074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.345+0000 7f375a200640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3730072610 0x7f3730074ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f37440af000 tx=0x7f374401b070 comp rx=0 tx=0).stop 2024-09-10T21:11:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.345+0000 7f375a200640 1 -- 172.21.15.16:0/3697045910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37540d2820 msgr2=0x7f375415ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.345+0000 7f375a200640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37540d2820 0x7f375415ce70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f373c00d770 tx=0x7f373c00dc30 comp rx=0 tx=0).stop 2024-09-10T21:11:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.345+0000 7f375a200640 1 -- 172.21.15.16:0/3697045910 shutdown_connections 2024-09-10T21:11:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.345+0000 7f375a200640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3730072610 0x7f3730074ad0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.345+0000 7f375a200640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3754160120 0x7f375415d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.345+0000 7f375a200640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37540d2820 0x7f375415ce70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.345+0000 7f375a200640 1 --2- 172.21.15.16:0/3697045910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37540cfd40 0x7f375414b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:25.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.345+0000 7f375a200640 1 -- 172.21.15.16:0/3697045910 >> 172.21.15.16:0/3697045910 conn(0x7f37540d1d00 msgr2=0x7f3754070720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:25.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.345+0000 7f375a200640 1 -- 172.21.15.16:0/3697045910 shutdown_connections 2024-09-10T21:11:25.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:25.345+0000 7f375a200640 1 -- 172.21.15.16:0/3697045910 wait complete. 2024-09-10T21:11:25.369 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:11:25.369 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:11:25.369 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-09-10T21:11:25.369 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T21:11:25.369 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:11:25.369 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:11:25.369 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:11:25.370 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:11:25.370 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:11:25.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:25.370 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-09-10T21:11:25.380 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:11:25.381 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:11:25.381 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:11:25.381 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:25.381 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:11:25.381 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:25.381 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:11:25.381 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:11:25.381 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-09-10T21:11:25.381 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:11:25.381 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:11:25.381 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-09-10T21:11:25.382 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:11:25.382 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-09-10T21:11:25.382 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-09-10T21:11:25.382 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:11:25.382 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42365sec preferred_lft 42365sec 2024-09-10T21:11:25.382 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-09-10T21:11:25.382 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:25.382 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:11:25.382 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-09-10T21:11:25.382 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-09-10T21:11:25.382 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:11:25.382 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:7e:b4:71:a2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:11:25.382 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:11:25.383 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:25.383 INFO:teuthology.orchestra.run.smithi052.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:11:25.383 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:11:25.383 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:11:25.383 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:25.383 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:11:25.383 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:25.383 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:11:25.383 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:11:25.383 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:11:25.383 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:25.383 INFO:teuthology.orchestra.run.smithi052.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:11:25.384 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:11:25.384 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:11:25.384 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:11:25.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:25.385 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:11:25.445 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-09-10T21:11:25.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:25.448 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-09-10T21:11:25.504 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-09-10T21:11:25.504 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:11:25.504 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:11:25.504 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:11:25.504 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:11:25.504 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:11:25.505 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:11:25.505 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:11:25.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:11:25.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:11:25.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:11:25.505 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:11:25.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:11:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:11:25.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:11:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:11:25.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:25.651 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:11:25.661 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:11:25.661 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:11:25.661 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:11:25.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:25.662 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-09-10T21:11:25.726 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:11:25.726 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T21:11:25.778 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:11:25.778 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:11:25.778 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:11:25.833 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T21:11:25.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:25.833 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:11:25.927 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:11:25.927 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:11:25.940 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:11:25.940 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:11:25.940 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:11:25.940 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 1@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:11:25.940 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:11:25.945 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:11:25.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:11:25.945 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:11:25.965 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:11:26.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:11:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:11:26.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:11:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:11:26.068 INFO:teuthology.orchestra.run.smithi052.stdout:client134642 2024-09-10T21:11:26.068 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:11:26.068 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:11:26.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 1 2024-09-10T21:11:26.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.233+0000 7f3ef2800640 1 -- 172.21.15.16:0/786357653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec0ce040 msgr2=0x7f3eec1671a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:26.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.233+0000 7f3ef2800640 1 --2- 172.21.15.16:0/786357653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec0ce040 0x7f3eec1671a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0cda10 tx=0x7f3ee401f390 comp rx=0 tx=0).stop 2024-09-10T21:11:26.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.233+0000 7f3ef2800640 1 -- 172.21.15.16:0/786357653 shutdown_connections 2024-09-10T21:11:26.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.233+0000 7f3ef2800640 1 --2- 172.21.15.16:0/786357653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec0ce040 0x7f3eec1671a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.233+0000 7f3ef2800640 1 -- 172.21.15.16:0/786357653 >> 172.21.15.16:0/786357653 conn(0x7f3eec0d1df0 msgr2=0x7f3eec0d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:26.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.233+0000 7f3ef2800640 1 -- 172.21.15.16:0/786357653 shutdown_connections 2024-09-10T21:11:26.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.233+0000 7f3ef2800640 1 -- 172.21.15.16:0/786357653 wait complete. 2024-09-10T21:11:26.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ef2800640 1 Processor -- start 2024-09-10T21:11:26.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ef2800640 1 -- start start 2024-09-10T21:11:26.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ef2800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3eec0ce040 0x7f3eec1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:26.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ef2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eec0cfe30 0x7f3eec158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:26.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ef2800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec0d0210 0x7f3eec06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:26.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ef2800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec0d2a90 con 0x7f3eec0cfe30 2024-09-10T21:11:26.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ef2800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec0d2910 con 0x7f3eec0d0210 2024-09-10T21:11:26.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ef2800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3eec0b12e0 con 0x7f3eec0ce040 2024-09-10T21:11:26.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ee3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eec0cfe30 0x7f3eec158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:26.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ee3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eec0cfe30 0x7f3eec158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43172/0 (socket says 172.21.15.16:43172) 2024-09-10T21:11:26.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ee3e00640 1 -- 172.21.15.16:0/3197128934 learned_addr learned my addr 172.21.15.16:0/3197128934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:26.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3eebe00640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec0d0210 0x7f3eec06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:26.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3eeb400640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3eec0ce040 0x7f3eec1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:26.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ee3e00640 1 -- 172.21.15.16:0/3197128934 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3eec0ce040 msgr2=0x7f3eec1592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:26.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ee3e00640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3eec0ce040 0x7f3eec1592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ee3e00640 1 -- 172.21.15.16:0/3197128934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec0d0210 msgr2=0x7f3eec06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:26.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ee3e00640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec0d0210 0x7f3eec06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ee3e00640 1 -- 172.21.15.16:0/3197128934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec0c64b0 con 0x7f3eec0cfe30 2024-09-10T21:11:26.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ee3e00640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eec0cfe30 0x7f3eec158700 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400c420 tx=0x7f3ed400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:26.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ee3400640 1 -- 172.21.15.16:0/3197128934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4017070 con 0x7f3eec0cfe30 2024-09-10T21:11:26.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ee3400640 1 -- 172.21.15.16:0/3197128934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ed40053c0 con 0x7f3eec0cfe30 2024-09-10T21:11:26.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ef2800640 1 -- 172.21.15.16:0/3197128934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eec158fd0 con 0x7f3eec0cfe30 2024-09-10T21:11:26.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ee3400640 1 -- 172.21.15.16:0/3197128934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4003860 con 0x7f3eec0cfe30 2024-09-10T21:11:26.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ef2800640 1 -- 172.21.15.16:0/3197128934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec158c40 con 0x7f3eec0cfe30 2024-09-10T21:11:26.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.237+0000 7f3ef2800640 1 -- 172.21.15.16:0/3197128934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eec0ce590 con 0x7f3eec0cfe30 2024-09-10T21:11:26.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.241+0000 7f3ee3400640 1 -- 172.21.15.16:0/3197128934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3ed4003a00 con 0x7f3eec0cfe30 2024-09-10T21:11:26.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.241+0000 7f3ee3400640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3ec8072610 0x7f3ec8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:26.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.241+0000 7f3ee3400640 1 -- 172.21.15.16:0/3197128934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3ed40927f0 con 0x7f3eec0cfe30 2024-09-10T21:11:26.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.241+0000 7f3eeb400640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3ec8072610 0x7f3ec8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:26.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.241+0000 7f3eeb400640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3ec8072610 0x7f3ec8074ad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0046c0 tx=0x7f3edc007cd0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:26.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.245+0000 7f3ee3400640 1 -- 172.21.15.16:0/3197128934 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3ed405d870 con 0x7f3eec0cfe30 2024-09-10T21:11:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.421+0000 7f3ef2800640 1 -- 172.21.15.16:0/3197128934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) -- 0x7f3eec15e390 con 0x7f3eec0cfe30 2024-09-10T21:11:26.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.425+0000 7f3ee3400640 1 -- 172.21.15.16:0/3197128934 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v29)=0 v29) ==== 121+0+0 (secure 0 0 0) 0x7f3ed40622a0 con 0x7f3eec0cfe30 2024-09-10T21:11:26.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.429+0000 7f3ef2800640 1 -- 172.21.15.16:0/3197128934 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3ec8072610 msgr2=0x7f3ec8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:26.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.429+0000 7f3ef2800640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3ec8072610 0x7f3ec8074ad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0046c0 tx=0x7f3edc007cd0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.429+0000 7f3ef2800640 1 -- 172.21.15.16:0/3197128934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eec0cfe30 msgr2=0x7f3eec158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:26.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.429+0000 7f3ef2800640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eec0cfe30 0x7f3eec158700 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400c420 tx=0x7f3ed400c8e0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.429+0000 7f3ef2800640 1 -- 172.21.15.16:0/3197128934 shutdown_connections 2024-09-10T21:11:26.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.429+0000 7f3ef2800640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3ec8072610 0x7f3ec8074ad0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.429+0000 7f3ef2800640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec0d0210 0x7f3eec06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.429+0000 7f3ef2800640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eec0cfe30 0x7f3eec158700 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.429+0000 7f3ef2800640 1 --2- 172.21.15.16:0/3197128934 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3eec0ce040 0x7f3eec1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.429+0000 7f3ef2800640 1 -- 172.21.15.16:0/3197128934 >> 172.21.15.16:0/3197128934 conn(0x7f3eec0d1df0 msgr2=0x7f3eec0b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:26.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.429+0000 7f3ef2800640 1 -- 172.21.15.16:0/3197128934 shutdown_connections 2024-09-10T21:11:26.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.429+0000 7f3ef2800640 1 -- 172.21.15.16:0/3197128934 wait complete. 2024-09-10T21:11:26.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_cap_acquisition_throttle 37 2024-09-10T21:11:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 -- 172.21.15.16:0/3021417438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e8003c90 msgr2=0x7f15e8004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:26.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 --2- 172.21.15.16:0/3021417438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e8003c90 0x7f15e8004090 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009b60 tx=0x7f15f401d270 comp rx=0 tx=0).stop 2024-09-10T21:11:26.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 -- 172.21.15.16:0/3021417438 shutdown_connections 2024-09-10T21:11:26.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 --2- 172.21.15.16:0/3021417438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15e8003c90 0x7f15e8004090 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 -- 172.21.15.16:0/3021417438 >> 172.21.15.16:0/3021417438 conn(0x7f16040b16a0 msgr2=0x7f16040b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:26.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 -- 172.21.15.16:0/3021417438 shutdown_connections 2024-09-10T21:11:26.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 -- 172.21.15.16:0/3021417438 wait complete. 2024-09-10T21:11:26.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 Processor -- start 2024-09-10T21:11:26.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 -- start start 2024-09-10T21:11:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15e8003c90 0x7f16041f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16041f9c60 0x7f16041505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1604150b30 0x7f1604150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f160406f4c0 con 0x7f16041f9c60 2024-09-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1604073290 con 0x7f1604150b30 2024-09-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f160ba00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f160406f640 con 0x7f15e8003c90 2024-09-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f15fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16041f9c60 0x7f16041505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f15fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16041f9c60 0x7f16041505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43188/0 (socket says 172.21.15.16:43188) 2024-09-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.601+0000 7f15fbe00640 1 -- 172.21.15.16:0/2463608967 learned_addr learned my addr 172.21.15.16:0/2463608967 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f1609000640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15e8003c90 0x7f16041f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f1609a00640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1604150b30 0x7f1604150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f15fbe00640 1 -- 172.21.15.16:0/2463608967 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15e8003c90 msgr2=0x7f16041f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f15fbe00640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15e8003c90 0x7f16041f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f15fbe00640 1 -- 172.21.15.16:0/2463608967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1604150b30 msgr2=0x7f1604150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f15fbe00640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1604150b30 0x7f1604150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f15fbe00640 1 -- 172.21.15.16:0/2463608967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1604140ac0 con 0x7f16041f9c60 2024-09-10T21:11:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f15fbe00640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16041f9c60 0x7f16041505f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f15ec001470 tx=0x7f15ec001870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f1601600640 1 -- 172.21.15.16:0/2463608967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec015020 con 0x7f16041f9c60 2024-09-10T21:11:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f1601600640 1 -- 172.21.15.16:0/2463608967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15ec00f960 con 0x7f16041f9c60 2024-09-10T21:11:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f1601600640 1 -- 172.21.15.16:0/2463608967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec00fdb0 con 0x7f16041f9c60 2024-09-10T21:11:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f160ba00640 1 -- 172.21.15.16:0/2463608967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16041fa290 con 0x7f16041f9c60 2024-09-10T21:11:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f160ba00640 1 -- 172.21.15.16:0/2463608967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1604141070 con 0x7f16041f9c60 2024-09-10T21:11:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f160ba00640 1 -- 172.21.15.16:0/2463608967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1604161930 con 0x7f16041f9c60 2024-09-10T21:11:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f1601600640 1 -- 172.21.15.16:0/2463608967 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f15ec011430 con 0x7f16041f9c60 2024-09-10T21:11:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.605+0000 7f1601600640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15e00725e0 0x7f15e0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.609+0000 7f1601600640 1 -- 172.21.15.16:0/2463608967 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f15ec093e70 con 0x7f16041f9c60 2024-09-10T21:11:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.609+0000 7f1609000640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15e00725e0 0x7f15e0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.609+0000 7f1609000640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15e00725e0 0x7f15e0074aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f15f40ae000 tx=0x7f15f401a010 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.609+0000 7f1601600640 1 -- 172.21.15.16:0/2463608967 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f15ec01f020 con 0x7f16041f9c60 2024-09-10T21:11:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.789+0000 7f160ba00640 1 -- 172.21.15.16:0/2463608967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_session_cap_acquisition_throttle}] v 0) -- 0x7f1604139ed0 con 0x7f16041f9c60 2024-09-10T21:11:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.793+0000 7f1601600640 1 -- 172.21.15.16:0/2463608967 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_cap_acquisition_throttle}]=0 v30)=0 v30) ==== 135+0+0 (secure 0 0 0) 0x7f15ec05f4b0 con 0x7f16041f9c60 2024-09-10T21:11:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.793+0000 7f160ba00640 1 -- 172.21.15.16:0/2463608967 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15e00725e0 msgr2=0x7f15e0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.793+0000 7f160ba00640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15e00725e0 0x7f15e0074aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f15f40ae000 tx=0x7f15f401a010 comp rx=0 tx=0).stop 2024-09-10T21:11:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.793+0000 7f160ba00640 1 -- 172.21.15.16:0/2463608967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16041f9c60 msgr2=0x7f16041505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.793+0000 7f160ba00640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16041f9c60 0x7f16041505f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f15ec001470 tx=0x7f15ec001870 comp rx=0 tx=0).stop 2024-09-10T21:11:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.797+0000 7f160ba00640 1 -- 172.21.15.16:0/2463608967 shutdown_connections 2024-09-10T21:11:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.797+0000 7f160ba00640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15e00725e0 0x7f15e0074aa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.797+0000 7f160ba00640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1604150b30 0x7f1604150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.797+0000 7f160ba00640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16041f9c60 0x7f16041505f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.797+0000 7f160ba00640 1 --2- 172.21.15.16:0/2463608967 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15e8003c90 0x7f16041f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.797+0000 7f160ba00640 1 -- 172.21.15.16:0/2463608967 >> 172.21.15.16:0/2463608967 conn(0x7f16040b16a0 msgr2=0x7f160415c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.797+0000 7f160ba00640 1 -- 172.21.15.16:0/2463608967 shutdown_connections 2024-09-10T21:11:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:26.797+0000 7f160ba00640 1 -- 172.21.15.16:0/2463608967 wait complete. 2024-09-10T21:11:26.821 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:11:26.821 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:11:26.830 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-10T21:11:26.830 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-10T21:11:26.830 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-10T21:11:26.830 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:26.830 DEBUG:teuthology.orchestra.run.smithi016:> n = 25 2024-09-10T21:11:26.830 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-10T21:11:26.830 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-10T21:11:26.830 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/dir1/file" 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-10T21:11:26.831 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-10T21:11:26.832 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:11:26.832 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-10T21:11:26.832 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:11:26.832 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-10T21:11:26.832 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-10T21:11:26.832 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-10T21:11:26.832 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-10T21:11:26.832 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:11:26.832 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-10T21:11:26.832 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-10T21:11:26.832 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-10T21:11:26.832 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-10T21:11:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:11:27.105 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:11:27.105 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:11:27.158 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-10T21:11:27.158 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-10T21:11:27.159 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-10T21:11:27.159 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:27.159 DEBUG:teuthology.orchestra.run.smithi016:> n = 25 2024-09-10T21:11:27.159 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-10T21:11:27.159 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-10T21:11:27.159 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-10T21:11:27.159 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/dir2/file" 2024-09-10T21:11:27.159 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:27.159 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-10T21:11:27.159 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-10T21:11:27.160 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-10T21:11:27.160 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:27.160 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-10T21:11:27.160 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-10T21:11:27.160 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:27.160 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-10T21:11:27.160 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-10T21:11:27.160 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-10T21:11:27.160 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-10T21:11:27.161 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-10T21:11:27.161 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-10T21:11:27.161 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-10T21:11:27.161 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:11:27.161 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-10T21:11:27.161 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:11:27.161 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-10T21:11:27.161 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-10T21:11:27.161 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-10T21:11:27.162 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-10T21:11:27.162 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:11:27.162 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-10T21:11:27.162 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-10T21:11:27.162 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-10T21:11:27.162 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-10T21:11:27.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:11:27.418 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:11:27.418 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:11:27.470 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-10T21:11:27.470 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-10T21:11:27.470 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-10T21:11:27.470 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:27.470 DEBUG:teuthology.orchestra.run.smithi016:> n = 25 2024-09-10T21:11:27.470 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-10T21:11:27.470 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-10T21:11:27.470 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-10T21:11:27.470 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/dir3/file" 2024-09-10T21:11:27.470 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:27.470 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-10T21:11:27.471 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-10T21:11:27.471 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-10T21:11:27.471 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:27.471 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-10T21:11:27.471 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-10T21:11:27.471 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:27.471 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-10T21:11:27.471 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-10T21:11:27.471 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-10T21:11:27.471 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-10T21:11:27.471 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-10T21:11:27.471 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-10T21:11:27.472 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-10T21:11:27.472 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:11:27.472 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-10T21:11:27.472 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:11:27.472 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-10T21:11:27.472 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-10T21:11:27.472 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-10T21:11:27.472 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-10T21:11:27.472 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:11:27.472 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-10T21:11:27.472 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-10T21:11:27.472 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-10T21:11:27.473 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-10T21:11:27.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:11:27.723 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:11:27.724 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:11:27.774 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-10T21:11:27.774 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-10T21:11:27.774 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-10T21:11:27.774 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:27.774 DEBUG:teuthology.orchestra.run.smithi016:> n = 25 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/dir4/file" 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-10T21:11:27.775 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:11:27.776 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-10T21:11:27.777 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-10T21:11:27.777 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-10T21:11:27.777 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-10T21:11:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:11:28.030 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:11:28.030 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:11:28.082 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:11:28.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:11:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:11:28.183 INFO:teuthology.orchestra.run.smithi016.stdout:client4939 2024-09-10T21:11:28.184 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'find . | wc') 2024-09-10T21:11:28.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:11:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:11:53.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:11:53.306 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:12:07.260 INFO:teuthology.orchestra.run.smithi016.stdout: 105 105 1490 2024-09-10T21:12:07.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:12:07.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.420+0000 7fb4e5600640 1 -- 172.21.15.16:0/560356349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e00b3200 msgr2=0x7fb4e00d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.420+0000 7fb4e5600640 1 --2- 172.21.15.16:0/560356349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e00b3200 0x7fb4e00d01c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00b3810 tx=0x7fb4c801efd0 comp rx=0 tx=0).stop 2024-09-10T21:12:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.420+0000 7fb4e5600640 1 -- 172.21.15.16:0/560356349 shutdown_connections 2024-09-10T21:12:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.420+0000 7fb4e5600640 1 --2- 172.21.15.16:0/560356349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e00b3200 0x7fb4e00d01c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.420+0000 7fb4e5600640 1 --2- 172.21.15.16:0/560356349 >> v2:172.21.15.52:3300/0 conn(0x7fb4e015f450 0x7fb4e00cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.420+0000 7fb4e5600640 1 -- 172.21.15.16:0/560356349 >> 172.21.15.16:0/560356349 conn(0x7fb4e00d1d80 msgr2=0x7fb4e00d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.420+0000 7fb4e5600640 1 -- 172.21.15.16:0/560356349 shutdown_connections 2024-09-10T21:12:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.420+0000 7fb4e5600640 1 -- 172.21.15.16:0/560356349 wait complete. 2024-09-10T21:12:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.420+0000 7fb4e5600640 1 Processor -- start 2024-09-10T21:12:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.420+0000 7fb4e5600640 1 -- start start 2024-09-10T21:12:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4e5600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e00d0890 0x7fb4e014f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4e5600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4e015f450 0x7fb4e01f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4e5600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4e01559b0 0x7fb4e01f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4e5600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e00b2e30 con 0x7fb4e00d0890 2024-09-10T21:12:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4e5600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e00b2cb0 con 0x7fb4e01559b0 2024-09-10T21:12:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4e5600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e00b2fb0 con 0x7fb4e015f450 2024-09-10T21:12:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4dea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e00d0890 0x7fb4e014f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4dea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e00d0890 0x7fb4e014f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34050/0 (socket says 172.21.15.16:34050) 2024-09-10T21:12:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4dea00640 1 -- 172.21.15.16:0/3359433288 learned_addr learned my addr 172.21.15.16:0/3359433288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4df400640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4e01559b0 0x7fb4e01f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4d7e00640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4e015f450 0x7fb4e01f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4dea00640 1 -- 172.21.15.16:0/3359433288 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4e015f450 msgr2=0x7fb4e01f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4dea00640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4e015f450 0x7fb4e01f8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4dea00640 1 -- 172.21.15.16:0/3359433288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4e01559b0 msgr2=0x7fb4e01f86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4dea00640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4e01559b0 0x7fb4e01f86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4dea00640 1 -- 172.21.15.16:0/3359433288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e00ccac0 con 0x7fb4e00d0890 2024-09-10T21:12:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4dea00640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e00d0890 0x7fb4e014f270 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000d950 tx=0x7fb4d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4d6a00640 1 -- 172.21.15.16:0/3359433288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0017070 con 0x7fb4e00d0890 2024-09-10T21:12:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4d6a00640 1 -- 172.21.15.16:0/3359433288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4d0009d20 con 0x7fb4e00d0890 2024-09-10T21:12:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4d6a00640 1 -- 172.21.15.16:0/3359433288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d00106c0 con 0x7fb4e00d0890 2024-09-10T21:12:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4e5600640 1 -- 172.21.15.16:0/3359433288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e00ccda0 con 0x7fb4e00d0890 2024-09-10T21:12:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4e5600640 1 -- 172.21.15.16:0/3359433288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e00cd360 con 0x7fb4e00d0890 2024-09-10T21:12:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4e5600640 1 -- 172.21.15.16:0/3359433288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e00b1410 con 0x7fb4e00d0890 2024-09-10T21:12:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.424+0000 7fb4d6a00640 1 -- 172.21.15.16:0/3359433288 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb4d0010860 con 0x7fb4e00d0890 2024-09-10T21:12:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.428+0000 7fb4d6a00640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4bc0725e0 0x7fb4bc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.428+0000 7fb4d6a00640 1 -- 172.21.15.16:0/3359433288 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb4d00936e0 con 0x7fb4e00d0890 2024-09-10T21:12:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.428+0000 7fb4d7e00640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4bc0725e0 0x7fb4bc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.428+0000 7fb4d7e00640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4bc0725e0 0x7fb4bc074aa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80b2000 tx=0x7fb4c801d810 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:07.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.432+0000 7fb4d6a00640 1 -- 172.21.15.16:0/3359433288 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb4d005edb0 con 0x7fb4e00d0890 2024-09-10T21:12:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.640+0000 7fb4e5600640 1 -- 172.21.15.16:0/3359433288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4e00cc840 con 0x7fb4e00d0890 2024-09-10T21:12:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.640+0000 7fb4d6a00640 1 -- 172.21.15.16:0/3359433288 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5232 (secure 0 0 0) 0x7fb4e00cc840 con 0x7fb4e00d0890 2024-09-10T21:12:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:07.648 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-10T21:11:16:232531+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/788814920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":788814920},{"type":"v1","addr":"172.21.15.16:6837","nonce":788814920}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2275087718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2275087718},{"type":"v1","addr":"172.21.15.16:6836","nonce":2275087718}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":134516,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3457135235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3457135235},{"type":"v1","addr":"172.21.15.52:6834","nonce":3457135235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:11:12.621848+0000","modified":"2024-09-10T21:11:15.636822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134519":{"gid":134519,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/4218881085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4218881085},{"type":"v1","addr":"172.21.15.52:6835","nonce":4218881085}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134519,"qdb_cluster":[134519]},"id":2}]} 2024-09-10T21:12:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-10T21:12:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.644+0000 7fb4e5600640 1 -- 172.21.15.16:0/3359433288 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4bc0725e0 msgr2=0x7fb4bc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.644+0000 7fb4e5600640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4bc0725e0 0x7fb4bc074aa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80b2000 tx=0x7fb4c801d810 comp rx=0 tx=0).stop 2024-09-10T21:12:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.644+0000 7fb4e5600640 1 -- 172.21.15.16:0/3359433288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e00d0890 msgr2=0x7fb4e014f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.644+0000 7fb4e5600640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e00d0890 0x7fb4e014f270 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000d950 tx=0x7fb4d000de10 comp rx=0 tx=0).stop 2024-09-10T21:12:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.644+0000 7fb4e5600640 1 -- 172.21.15.16:0/3359433288 shutdown_connections 2024-09-10T21:12:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.644+0000 7fb4e5600640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4bc0725e0 0x7fb4bc074aa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.644+0000 7fb4e5600640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4e01559b0 0x7fb4e01f86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.644+0000 7fb4e5600640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4e015f450 0x7fb4e01f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.644+0000 7fb4e5600640 1 --2- 172.21.15.16:0/3359433288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e00d0890 0x7fb4e014f270 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.644+0000 7fb4e5600640 1 -- 172.21.15.16:0/3359433288 >> 172.21.15.16:0/3359433288 conn(0x7fb4e00d1d80 msgr2=0x7fb4e00cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.644+0000 7fb4e5600640 1 -- 172.21.15.16:0/3359433288 shutdown_connections 2024-09-10T21:12:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.644+0000 7fb4e5600640 1 -- 172.21.15.16:0/3359433288 wait complete. 2024-09-10T21:12:07.670 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json perf dump 2024-09-10T21:12:07.845 INFO:teuthology.orchestra.run.smithi052.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":469,"msgr_send_messages":534,"msgr_recv_bytes":237224,"msgr_send_bytes":283467,"msgr_created_connections":8,"msgr_active_connections":4,"msgr_running_total_time":0.062971009,"msgr_running_send_time":0.028436931,"msgr_running_recv_time":0.062614311,"msgr_running_fast_dispatch_time":0.002455758,"msgr_send_messages_queue_lat":{"avgcount":534,"sum":0.066290990,"avgtime":0.000124140},"msgr_handle_ack_lat":{"avgcount":757,"sum":0.000139466,"avgtime":0.000000184},"msgr_recv_encrypted_bytes":125376,"msgr_send_encrypted_bytes":18576},"AsyncMessenger::Worker-1":{"msgr_recv_messages":70,"msgr_send_messages":48,"msgr_recv_bytes":25439,"msgr_send_bytes":97512,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.018344323,"msgr_running_send_time":0.006653037,"msgr_running_recv_time":0.008876708,"msgr_running_fast_dispatch_time":0.002394541,"msgr_send_messages_queue_lat":{"avgcount":46,"sum":0.352979608,"avgtime":0.007673469},"msgr_handle_ack_lat":{"avgcount":62,"sum":0.000011908,"avgtime":0.000000192},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":78544},"AsyncMessenger::Worker-2":{"msgr_recv_messages":132,"msgr_send_messages":134,"msgr_recv_bytes":33002,"msgr_send_bytes":472997,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.021028199,"msgr_running_send_time":0.007307867,"msgr_running_recv_time":0.009067510,"msgr_running_fast_dispatch_time":0.003562522,"msgr_send_messages_queue_lat":{"avgcount":133,"sum":0.249769803,"avgtime":0.001877968},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":253,"sum":0.050585731,"avgtime":0.000199943}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000017516,"avgtime":0.000008758}},"mds":{"request":148,"reply":148,"reply_latency":{"avgcount":148,"sum":39.083041272,"avgtime":0.264074603},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":114,"inodes_top":0,"inodes_bottom":114,"inodes_pin_tail":0,"inodes_pinned":114,"inodes_expired":0,"inodes_with_caps":100,"caps":101,"subtrees":2,"traverse":188,"traverse_hit":174,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":100,"root_rbytes":160,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":4,"ceph_cap_op_grant":112,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":104,"handle_client_caps_dirty":100,"handle_client_cap_release":1,"process_request_cap_release":14},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":219,"evex":0,"evtrm":0,"ev":219,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4615103,"rdpos":0,"jlat":{"avgcount":110,"sum":0.183107677,"avgtime":0.001664615},"replayed":0},"mds_mem":{"ino":117,"ino+":117,"ino-":0,"dir":16,"dir+":16,"dir-":0,"dn":114,"dn+":115,"dn-":1,"cap":101,"cap+":106,"cap-":5,"rss":52780,"heap":267137024},"mds_server":{"dispatch_client_request":230,"dispatch_server_request":0,"handle_client_request":148,"handle_client_session":106,"handle_peer_request":0,"req_create_latency":{"avgcount":100,"sum":0.051750437,"avgtime":0.000517504},"req_getattr_latency":{"avgcount":32,"sum":0.006251282,"avgtime":0.000195352},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":4,"sum":0.000474722,"avgtime":0.000118680},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":4,"sum":0.005745847,"avgtime":0.001436461},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":2,"sum":0.000294422,"avgtime":0.000147211},"req_readdir_latency":{"avgcount":5,"sum":39.018242398,"avgtime":7.803648479},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000282164,"avgtime":0.000282164},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":78,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":2,"session_add":2,"session_remove":0,"sessions_open":2,"sessions_stale":0,"total_load":93,"average_load":46,"avg_session_uptime":40,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":226075,"buffer_anon_items":87,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5312,"osdmap_items":75,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":396416,"mds_co_items":3209,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":152,"op_send_bytes":438578,"op_resend":37,"op_reply":152,"op_latency":{"avgcount":152,"sum":1.575988280,"avgtime":0.010368343},"op_inflight":0,"oplen_avg":{"avgcount":152,"sum":179},"op":152,"op_r":0,"op_w":152,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":110,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":32,"map_full":0,"map_inc":1,"osd_sessions":6,"osd_session_open":6,"osd_session_close":0,"osd_laggy":0,"omap_wr":20,"omap_rd":0,"omap_del":2},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":667,"get_sum":230469,"get_or_fail_fail":0,"get_or_fail_success":667,"take":0,"take_sum":0,"put":667,"put_sum":230469,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":152,"take_sum":438578,"put":152,"put_sum":438578,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":152,"take_sum":152,"put":152,"put_sum":152,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55d2d1214500":{"val":0,"max":3758096384,"get_started":0,"get":219,"get_sum":420799,"get_or_fail_fail":0,"get_or_fail_success":219,"take":0,"take_sum":0,"put":110,"put_sum":420799,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":2}} 2024-09-10T21:12:07.849 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 4, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 757, "avgtime": 1.84e-07, "sum": 0.000139466 }, "msgr_recv_bytes": 237224, "msgr_recv_encrypted_bytes": 125376, "msgr_recv_messages": 469, "msgr_running_fast_dispatch_time": 0.002455758, "msgr_running_recv_time": 0.062614311, "msgr_running_send_time": 0.028436931, "msgr_running_total_time": 0.062971009, "msgr_send_bytes": 283467, "msgr_send_encrypted_bytes": 18576, "msgr_send_messages": 534, "msgr_send_messages_queue_lat": { "avgcount": 534, "avgtime": 0.00012414, "sum": 0.06629099 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 62, "avgtime": 1.92e-07, "sum": 1.1908e-05 }, "msgr_recv_bytes": 25439, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 70, "msgr_running_fast_dispatch_time": 0.002394541, "msgr_running_recv_time": 0.008876708, "msgr_running_send_time": 0.006653037, "msgr_running_total_time": 0.018344323, "msgr_send_bytes": 97512, "msgr_send_encrypted_bytes": 78544, "msgr_send_messages": 48, "msgr_send_messages_queue_lat": { "avgcount": 46, "avgtime": 0.007673469, "sum": 0.352979608 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 33002, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 132, "msgr_running_fast_dispatch_time": 0.003562522, "msgr_running_recv_time": 0.00906751, "msgr_running_send_time": 0.007307867, "msgr_running_total_time": 0.021028199, "msgr_send_bytes": 472997, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 134, "msgr_send_messages_queue_lat": { "avgcount": 133, "avgtime": 0.001877968, "sum": 0.249769803 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 253, "avgtime": 0.000199943, "sum": 0.050585731 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 8.758e-06, "sum": 1.7516e-05 }, "queue_len": 0 }, "mds": { "caps": 101, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 112, "ceph_cap_op_revoke": 4, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 1, "handle_client_caps": 104, "handle_client_caps_dirty": 100, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 114, "inodes_bottom": 114, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 114, "inodes_top": 0, "inodes_with_caps": 100, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 14, "q": 0, "reply": 148, "reply_latency": { "avgcount": 148, "avgtime": 0.264074603, "sum": 39.083041272 }, "request": 148, "root_rbytes": 160, "root_rfiles": 100, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 188, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 174, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 219, "evadd": 219, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 110, "avgtime": 0.001664615, "sum": 0.183107677 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4615103 }, "mds_mem": { "cap": 101, "cap+": 106, "cap-": 5, "dir": 16, "dir+": 16, "dir-": 0, "dn": 114, "dn+": 115, "dn-": 1, "heap": 267137024, "ino": 117, "ino+": 117, "ino-": 0, "rss": 52780 }, "mds_server": { "cap_acquisition_throttle": 78, "cap_revoke_eviction": 0, "dispatch_client_request": 230, "dispatch_server_request": 0, "handle_client_request": 148, "handle_client_session": 106, "handle_peer_request": 0, "req_create_latency": { "avgcount": 100, "avgtime": 0.000517504, "sum": 0.051750437 }, "req_getattr_latency": { "avgcount": 32, "avgtime": 0.000195352, "sum": 0.006251282 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 4, "avgtime": 0.00011868, "sum": 0.000474722 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 4, "avgtime": 0.001436461, "sum": 0.005745847 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 2, "avgtime": 0.000147211, "sum": 0.000294422 }, "req_readdir_latency": { "avgcount": 5, "avgtime": 7.803648479, "sum": 39.018242398 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000282164, "sum": 0.000282164 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 46, "avg_session_uptime": 40, "mdthresh_evicted": 0, "session_add": 2, "session_count": 2, "session_remove": 0, "sessions_open": 2, "sessions_stale": 0, "total_load": 93 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 226075, "buffer_anon_items": 87, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 396416, "mds_co_items": 3209, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5312, "osdmap_items": 75, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 32, "map_full": 0, "map_inc": 1, "omap_del": 2, "omap_rd": 0, "omap_wr": 20, "op": 152, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 152, "avgtime": 0.010368343, "sum": 1.57598828 }, "op_pg": 0, "op_r": 0, "op_reply": 152, "op_resend": 37, "op_rmw": 0, "op_send": 152, "op_send_bytes": 438578, "op_w": 152, "oplen_avg": { "avgcount": 152, "sum": 179 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 6, "osd_sessions": 6, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 110, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 667, "get_or_fail_fail": 0, "get_or_fail_success": 667, "get_started": 0, "get_sum": 230469, "max": 104857600, "put": 667, "put_sum": 230469, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 152, "put_sum": 438578, "take": 152, "take_sum": 438578, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 152, "put_sum": 152, "take": 152, "take_sum": 152, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55d2d1214500": { "get": 219, "get_or_fail_fail": 0, "get_or_fail_success": 219, "get_started": 0, "get_sum": 420799, "max": 3758096384, "put": 110, "put_sum": 420799, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 2 } } 2024-09-10T21:12:07.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.992+0000 7fde59600640 1 -- 172.21.15.16:0/1085861765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde38005290 msgr2=0x7fde3800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.992+0000 7fde59600640 1 --2- 172.21.15.16:0/1085861765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde38005290 0x7fde3800b9d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00f6f0 tx=0x7fde4c01e630 comp rx=0 tx=0).stop 2024-09-10T21:12:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.992+0000 7fde59600640 1 -- 172.21.15.16:0/1085861765 shutdown_connections 2024-09-10T21:12:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.992+0000 7fde59600640 1 --2- 172.21.15.16:0/1085861765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde38005290 0x7fde3800b9d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:07.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.992+0000 7fde59600640 1 -- 172.21.15.16:0/1085861765 >> 172.21.15.16:0/1085861765 conn(0x7fde540cfb20 msgr2=0x7fde540cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:08.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.992+0000 7fde59600640 1 -- 172.21.15.16:0/1085861765 shutdown_connections 2024-09-10T21:12:08.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.992+0000 7fde59600640 1 -- 172.21.15.16:0/1085861765 wait complete. 2024-09-10T21:12:08.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.992+0000 7fde59600640 1 Processor -- start 2024-09-10T21:12:08.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.992+0000 7fde59600640 1 -- start start 2024-09-10T21:12:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde59600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fde38005290 0x7fde540b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde59600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde541f7e90 0x7fde541f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde59600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde5415b920 0x7fde54158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde59600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde540ce610 con 0x7fde541f7e90 2024-09-10T21:12:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde59600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fde540ca810 con 0x7fde5415b920 2024-09-10T21:12:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde59600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fde540ce790 con 0x7fde38005290 2024-09-10T21:12:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde4be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde541f7e90 0x7fde541f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde4be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde541f7e90 0x7fde541f4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34070/0 (socket says 172.21.15.16:34070) 2024-09-10T21:12:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde4be00640 1 -- 172.21.15.16:0/2772147583 learned_addr learned my addr 172.21.15.16:0/2772147583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde53400640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde5415b920 0x7fde54158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde52a00640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fde38005290 0x7fde540b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde4be00640 1 -- 172.21.15.16:0/2772147583 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fde38005290 msgr2=0x7fde540b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde4be00640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fde38005290 0x7fde540b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde4be00640 1 -- 172.21.15.16:0/2772147583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde5415b920 msgr2=0x7fde54158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde4be00640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde5415b920 0x7fde54158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde4be00640 1 -- 172.21.15.16:0/2772147583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde541f4d90 con 0x7fde541f7e90 2024-09-10T21:12:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde52a00640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fde38005290 0x7fde540b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde4be00640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde541f7e90 0x7fde541f4850 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005ed0 tx=0x7fde3c011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde4aa00640 1 -- 172.21.15.16:0/2772147583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c005320 con 0x7fde541f7e90 2024-09-10T21:12:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde4aa00640 1 -- 172.21.15.16:0/2772147583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde3c018020 con 0x7fde541f7e90 2024-09-10T21:12:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde4aa00640 1 -- 172.21.15.16:0/2772147583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c003860 con 0x7fde541f7e90 2024-09-10T21:12:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde59600640 1 -- 172.21.15.16:0/2772147583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde54074140 con 0x7fde541f7e90 2024-09-10T21:12:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde59600640 1 -- 172.21.15.16:0/2772147583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde541f52f0 con 0x7fde541f7e90 2024-09-10T21:12:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde59600640 1 -- 172.21.15.16:0/2772147583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde5406e640 con 0x7fde541f7e90 2024-09-10T21:12:08.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:07.996+0000 7fde4aa00640 1 -- 172.21.15.16:0/2772147583 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fde3c010040 con 0x7fde541f7e90 2024-09-10T21:12:08.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.000+0000 7fde4aa00640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde30072610 0x7fde30074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.000+0000 7fde4aa00640 1 -- 172.21.15.16:0/2772147583 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fde3c0965e0 con 0x7fde541f7e90 2024-09-10T21:12:08.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.000+0000 7fde52a00640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde30072610 0x7fde30074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.000+0000 7fde52a00640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde30072610 0x7fde30074ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fde54158880 tx=0x7fde44006730 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.004+0000 7fde4aa00640 1 -- 172.21.15.16:0/2772147583 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fde3c022020 con 0x7fde541f7e90 2024-09-10T21:12:08.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.208+0000 7fde59600640 1 -- 172.21.15.16:0/2772147583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde540c3340 con 0x7fde541f7e90 2024-09-10T21:12:08.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.212+0000 7fde4aa00640 1 -- 172.21.15.16:0/2772147583 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5232 (secure 0 0 0) 0x7fde541f4d90 con 0x7fde541f7e90 2024-09-10T21:12:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:08.218 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-10T21:11:16:232531+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/788814920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":788814920},{"type":"v1","addr":"172.21.15.16:6837","nonce":788814920}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2275087718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2275087718},{"type":"v1","addr":"172.21.15.16:6836","nonce":2275087718}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":134516,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3457135235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3457135235},{"type":"v1","addr":"172.21.15.52:6834","nonce":3457135235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:11:12.621848+0000","modified":"2024-09-10T21:11:15.636822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134519":{"gid":134519,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/4218881085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4218881085},{"type":"v1","addr":"172.21.15.52:6835","nonce":4218881085}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134519,"qdb_cluster":[134519]},"id":2}]} 2024-09-10T21:12:08.218 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-10T21:12:08.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.212+0000 7fde59600640 1 -- 172.21.15.16:0/2772147583 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde30072610 msgr2=0x7fde30074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:08.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.212+0000 7fde59600640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde30072610 0x7fde30074ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fde54158880 tx=0x7fde44006730 comp rx=0 tx=0).stop 2024-09-10T21:12:08.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.212+0000 7fde59600640 1 -- 172.21.15.16:0/2772147583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde541f7e90 msgr2=0x7fde541f4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:08.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.212+0000 7fde59600640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde541f7e90 0x7fde541f4850 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005ed0 tx=0x7fde3c011740 comp rx=0 tx=0).stop 2024-09-10T21:12:08.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.212+0000 7fde59600640 1 -- 172.21.15.16:0/2772147583 shutdown_connections 2024-09-10T21:12:08.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.212+0000 7fde59600640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde30072610 0x7fde30074ad0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.212+0000 7fde59600640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde5415b920 0x7fde54158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.212+0000 7fde59600640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde541f7e90 0x7fde541f4850 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.212+0000 7fde59600640 1 --2- 172.21.15.16:0/2772147583 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fde38005290 0x7fde540b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.212+0000 7fde59600640 1 -- 172.21.15.16:0/2772147583 >> 172.21.15.16:0/2772147583 conn(0x7fde540cfb20 msgr2=0x7fde540b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:08.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.212+0000 7fde59600640 1 -- 172.21.15.16:0/2772147583 shutdown_connections 2024-09-10T21:12:08.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.212+0000 7fde59600640 1 -- 172.21.15.16:0/2772147583 wait complete. 2024-09-10T21:12:08.242 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-10T21:12:08.408 INFO:teuthology.orchestra.run.smithi052.stdout:[{"id":134642,"entity":{"name":{"type":"client","num":134642},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":336956311}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":2,"uptime":42.455676898999997,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":8.1841326621847763,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.134642 192.168.0.1:0/336956311","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi052","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":4939,"entity":{"name":{"type":"client","num":4939},"addr":{"type":"any","addr":"172.21.15.16:0","nonce":3091112520}},"state":"open","num_leases":1,"num_caps":100,"request_load_avg":89,"uptime":43.611668227999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":12.796580787072919,"halflife":60},"release_caps":{"value":3.2928406330665188,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":194.69582773420089,"halflife":300},"cap_acquisition":{"value":60.187845598409169,"halflife":30},"last_trim_completed_requests_tid":144,"last_trim_completed_flushes_tid":101,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.4939 172.21.15.16:0/3091112520","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000025c","length":899}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-10T21:12:08.409 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi052", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 336956311, "type": "any" }, "name": { "num": 134642, "type": "client" } }, "id": 134642, "inst": "client.134642 192.168.0.1:0/336956311", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 8.184132662184776 }, "state": "open", "uptime": 42.455676899 }, { "cap_acquisition": { "halflife": 30, "value": 60.18784559840917 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 3091112520, "type": "any" }, "name": { "num": 4939, "type": "client" } }, "id": 4939, "inst": "client.4939 172.21.15.16:0/3091112520", "last_trim_completed_flushes_tid": 101, "last_trim_completed_requests_tid": 144, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 899, "start": "0x1000000025c" } ], "recall_caps": { "halflife": 60, "value": 12.79658078707292 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 3.2928406330665188 }, "request_load_avg": 89, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 194.6958277342009 }, "state": "open", "uptime": 43.611668228 } ] 2024-09-10T21:12:08.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump_historic_ops 2024-09-10T21:12:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 -- 172.21.15.16:0/880939920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40cd070 msgr2=0x7fb1f40cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 --2- 172.21.15.16:0/880939920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40cd070 0x7fb1f40cbe90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb1f406dc80 tx=0x7fb1dc01c800 comp rx=0 tx=0).stop 2024-09-10T21:12:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 -- 172.21.15.16:0/880939920 shutdown_connections 2024-09-10T21:12:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 --2- 172.21.15.16:0/880939920 >> v2:172.21.15.52:3300/0 conn(0x7fb1f40c42c0 0x7fb1f4070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 --2- 172.21.15.16:0/880939920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f40cd070 0x7fb1f40cbe90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 -- 172.21.15.16:0/880939920 >> 172.21.15.16:0/880939920 conn(0x7fb1f40d0c30 msgr2=0x7fb1f40d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 -- 172.21.15.16:0/880939920 shutdown_connections 2024-09-10T21:12:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 -- 172.21.15.16:0/880939920 wait complete. 2024-09-10T21:12:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 Processor -- start 2024-09-10T21:12:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 -- start start 2024-09-10T21:12:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1f40c42c0 0x7fb1f415e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1f40cd070 0x7fb1f40c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f415be10 0x7fb1f40c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41692d0 con 0x7fb1f415be10 2024-09-10T21:12:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4169150 con 0x7fb1f40c42c0 2024-09-10T21:12:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1fa400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f4169450 con 0x7fb1f40cd070 2024-09-10T21:12:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f415be10 0x7fb1f40c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1f3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1f40c42c0 0x7fb1f415e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1f3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1f40c42c0 0x7fb1f415e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:54042/0 (socket says 172.21.15.16:54042) 2024-09-10T21:12:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f415be10 0x7fb1f40c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34082/0 (socket says 172.21.15.16:34082) 2024-09-10T21:12:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1f3400640 1 -- 172.21.15.16:0/379728775 learned_addr learned my addr 172.21.15.16:0/379728775 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1eaa00640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1f40cd070 0x7fb1f40c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1f3400640 1 -- 172.21.15.16:0/379728775 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1f40cd070 msgr2=0x7fb1f40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1f3400640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1f40cd070 0x7fb1f40c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1f3400640 1 -- 172.21.15.16:0/379728775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f415be10 msgr2=0x7fb1f40c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1f3400640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f415be10 0x7fb1f40c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1f3400640 1 -- 172.21.15.16:0/379728775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f415ebf0 con 0x7fb1f40c42c0 2024-09-10T21:12:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.552+0000 7fb1f3e00640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f415be10 0x7fb1f40c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.556+0000 7fb1f3400640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1f40c42c0 0x7fb1f415e3a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400c3c0 tx=0x7fb1e400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.556+0000 7fb1ebe00640 1 -- 172.21.15.16:0/379728775 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4017070 con 0x7fb1f40c42c0 2024-09-10T21:12:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.556+0000 7fb1ebe00640 1 -- 172.21.15.16:0/379728775 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1e40053c0 con 0x7fb1f40c42c0 2024-09-10T21:12:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.556+0000 7fb1fa400640 1 -- 172.21.15.16:0/379728775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f40c9350 con 0x7fb1f40c42c0 2024-09-10T21:12:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.556+0000 7fb1ebe00640 1 -- 172.21.15.16:0/379728775 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4003860 con 0x7fb1f40c42c0 2024-09-10T21:12:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.556+0000 7fb1fa400640 1 -- 172.21.15.16:0/379728775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f40c9910 con 0x7fb1f40c42c0 2024-09-10T21:12:08.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.556+0000 7fb1ebe00640 1 -- 172.21.15.16:0/379728775 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb1e4003a00 con 0x7fb1f40c42c0 2024-09-10T21:12:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 Processor -- start 2024-09-10T21:12:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 -- start start 2024-09-10T21:12:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1ebe00640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1d00725e0 0x7fb1d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1ebe00640 1 -- 172.21.15.16:0/379728775 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb1e4092860 con 0x7fb1f40c42c0 2024-09-10T21:12:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1f3e00640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1d00725e0 0x7fb1d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc26a0 0x557d70bc2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70bc2fc0 0x557d70b76110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1f3e00640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1d00725e0 0x7fb1d0074aa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb1f406f800 tx=0x7fb1ec009500 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 0x557d70b76a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557d709cfb70 con 0x557d70b76650 2024-09-10T21:12:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557d709befa0 con 0x557d70bc2fc0 2024-09-10T21:12:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x557d709c2db0 con 0x557d70bc26a0 2024-09-10T21:12:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 0x557d70b76a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 0x557d70b76a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34096/0 (socket says 172.21.15.16:34096) 2024-09-10T21:12:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1f3e00640 1 -- 172.21.15.16:0/416990354 learned_addr learned my addr 172.21.15.16:0/416990354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1eaa00640 1 --2- 172.21.15.16:0/416990354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70bc2fc0 0x557d70b76110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1f3400640 1 --2- 172.21.15.16:0/416990354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc26a0 0x557d70bc2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1f3e00640 1 -- 172.21.15.16:0/416990354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc26a0 msgr2=0x557d70bc2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1f3e00640 1 --2- 172.21.15.16:0/416990354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc26a0 0x557d70bc2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1f3e00640 1 -- 172.21.15.16:0/416990354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70bc2fc0 msgr2=0x557d70b76110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1f3e00640 1 --2- 172.21.15.16:0/416990354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70bc2fc0 0x557d70b76110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1f3e00640 1 -- 172.21.15.16:0/416990354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d709632a0 con 0x557d70b76650 2024-09-10T21:12:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1f3e00640 1 --2- 172.21.15.16:0/416990354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 0x557d70b76a30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec003c60 tx=0x7fb1ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5effcff05d8fa3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1e3e00640 1 -- 172.21.15.16:0/416990354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec004230 con 0x557d70b76650 2024-09-10T21:12:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1e3e00640 1 -- 172.21.15.16:0/416990354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1ec022070 con 0x557d70b76650 2024-09-10T21:12:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1e3e00640 1 -- 172.21.15.16:0/416990354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec004610 con 0x557d70b76650 2024-09-10T21:12:08.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 -- 172.21.15.16:0/416990354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 msgr2=0x557d70b76a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:08.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 --2- 172.21.15.16:0/416990354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 0x557d70b76a30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec003c60 tx=0x7fb1ec014000 comp rx=0 tx=0).stop 2024-09-10T21:12:08.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 -- 172.21.15.16:0/416990354 shutdown_connections 2024-09-10T21:12:08.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 --2- 172.21.15.16:0/416990354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 0x557d70b76a30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 --2- 172.21.15.16:0/416990354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70bc2fc0 0x557d70b76110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 --2- 172.21.15.16:0/416990354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc26a0 0x557d70bc2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 -- 172.21.15.16:0/416990354 >> 172.21.15.16:0/416990354 conn(0x557d70a5bec0 msgr2=0x557d70ba5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 -- 172.21.15.16:0/416990354 shutdown_connections 2024-09-10T21:12:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.560+0000 7fb1fd751000 1 -- 172.21.15.16:0/416990354 wait complete. 2024-09-10T21:12:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 1 -- start start 2024-09-10T21:12:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 1 Processor -- start 2024-09-10T21:12:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70b76650 0x557d70bc9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc26a0 0x557d70bc9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70bc2fc0 0x557d70bca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557d7096a050 con 0x557d70bc2fc0 2024-09-10T21:12:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557d7098a890 con 0x557d70b76650 2024-09-10T21:12:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x557d709ce940 con 0x557d70bc26a0 2024-09-10T21:12:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 10 client.0 _ll_register_callbacks cb 0x557d70b5fbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70bc2fc0 0x557d70bca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1f3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70b76650 0x557d70bc9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1f3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70b76650 0x557d70bc9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:54072/0 (socket says 172.21.15.16:54072) 2024-09-10T21:12:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1f3400640 1 -- 172.21.15.16:0/2593980834 learned_addr learned my addr 172.21.15.16:0/2593980834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1eaa00640 1 --2- 172.21.15.16:0/2593980834 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc26a0 0x557d70bc9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1f3400640 1 -- 172.21.15.16:0/2593980834 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc26a0 msgr2=0x557d70bc9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1f3400640 1 --2- 172.21.15.16:0/2593980834 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc26a0 0x557d70bc9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1f3400640 1 -- 172.21.15.16:0/2593980834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70bc2fc0 msgr2=0x557d70bca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1f3400640 1 --2- 172.21.15.16:0/2593980834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70bc2fc0 0x557d70bca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1f3400640 1 -- 172.21.15.16:0/2593980834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d7095d1d0 con 0x557d70b76650 2024-09-10T21:12:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1f3e00640 1 --2- 172.21.15.16:0/2593980834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70bc2fc0 0x557d70bca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1f3400640 1 --2- 172.21.15.16:0/2593980834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70b76650 0x557d70bc9820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb1e405d550 tx=0x7fb1e405d750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b342e5890d1271 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-09-10T21:12:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1e0c00640 1 -- 172.21.15.16:0/2593980834 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4062280 con 0x557d70b76650 2024-09-10T21:12:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1e0c00640 1 -- 172.21.15.16:0/2593980834 <== mon.1 v2:172.21.15.52:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb1e40591a0 con 0x557d70b76650 2024-09-10T21:12:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 1 -- 172.21.15.16:0/2593980834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557d70957150 con 0x557d70b76650 2024-09-10T21:12:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1e0c00640 1 -- 172.21.15.16:0/2593980834 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fb1e4066ca0 con 0x557d70b76650 2024-09-10T21:12:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 10 client.4993 fetch_fsmap learned FSMap version 16 2024-09-10T21:12:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 1 -- 172.21.15.16:0/2593980834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x557d70a449d0 con 0x557d70b76650 2024-09-10T21:12:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1e0c00640 1 -- 172.21.15.16:0/2593980834 <== mon.1 v2:172.21.15.52:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x557d70957150 con 0x557d70b76650 2024-09-10T21:12:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1e0c00640 1 -- 172.21.15.16:0/2593980834 <== mon.1 v2:172.21.15.52:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7fb1e40593a0 con 0x557d70b76650 2024-09-10T21:12:08.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 10 client.4993 fetch_fsmap finished waiting for FSMap version 16 2024-09-10T21:12:08.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 10 client.4993 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-09-10T21:12:08.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 20 client.4993 populate_metadata read hostname 'smithi016' 2024-09-10T21:12:08.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 1 --2- 172.21.15.16:0/2593980834 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x557d70bd1bf0 0x557d70bd3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 4 client.4993 mds_command: new command op to 134519 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:12:08.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1fd751000 1 -- 172.21.15.16:0/2593980834 --> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d709d1ac0 con 0x557d70bd1bf0 2024-09-10T21:12:08.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.564+0000 7fb1eaa00640 1 --2- 172.21.15.16:0/2593980834 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x557d70bd1bf0 0x557d70bd3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1eaa00640 1 --2- 172.21.15.16:0/2593980834 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x557d70bd1bf0 0x557d70bd3fe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17cc4446e7fc6b96 server_cookie=5b2d9432ae0453de in_seq=0 out_seq=0 2024-09-10T21:12:08.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1e0c00640 10 client.4993 ms_handle_connect on v2:172.21.15.52:6833/4218881085 2024-09-10T21:12:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1e0c00640 1 -- 172.21.15.16:0/2593980834 <== mds.0 v2:172.21.15.52:6833/4218881085 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557d709d1ac0 con 0x557d70bd1bf0 2024-09-10T21:12:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1e0c00640 10 client.4993 handle_command_reply: tid=0 2024-09-10T21:12:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1fd751000 1 client.4993 shutdown 2024-09-10T21:12:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1fd751000 1 -- 172.21.15.16:0/2593980834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70b76650 msgr2=0x557d70bc9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1fd751000 1 --2- 172.21.15.16:0/2593980834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70b76650 0x557d70bc9820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb1e405d550 tx=0x7fb1e405d750 comp rx=0 tx=0).stop 2024-09-10T21:12:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1fd751000 1 -- 172.21.15.16:0/2593980834 shutdown_connections 2024-09-10T21:12:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1fd751000 1 --2- 172.21.15.16:0/2593980834 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x557d70bd1bf0 0x557d70bd3fe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1fd751000 1 --2- 172.21.15.16:0/2593980834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70bc2fc0 0x557d70bca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1fd751000 1 --2- 172.21.15.16:0/2593980834 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc26a0 0x557d70bc9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1fd751000 1 --2- 172.21.15.16:0/2593980834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70b76650 0x557d70bc9820 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1fd751000 1 -- 172.21.15.16:0/2593980834 >> 172.21.15.16:0/2593980834 conn(0x557d70a5bec0 msgr2=0x557d70b75920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1e0c00640 0 client.4993 ms_handle_reset on v2:172.21.15.52:6833/4218881085 2024-09-10T21:12:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1fd751000 1 -- 172.21.15.16:0/2593980834 shutdown_connections 2024-09-10T21:12:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1fd751000 1 -- 172.21.15.16:0/2593980834 wait complete. 2024-09-10T21:12:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.568+0000 7fb1fd751000 20 client.4993 trim_cache size 0 max 16384 2024-09-10T21:12:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1fd751000 1 Processor -- start 2024-09-10T21:12:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1fd751000 1 -- start start 2024-09-10T21:12:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1fd751000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70b76650 0x557d70c358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1fd751000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70bc26a0 0x557d70c35e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1fd751000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc2fc0 0x557d70c36370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1fd751000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4154cd0 con 0x557d70bc26a0 2024-09-10T21:12:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1fd751000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f40cad60 con 0x557d70b76650 2024-09-10T21:12:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1fd751000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f4154680 con 0x557d70bc2fc0 2024-09-10T21:12:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1eaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70bc26a0 0x557d70c35e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1eaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70bc26a0 0x557d70c35e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34122/0 (socket says 172.21.15.16:34122) 2024-09-10T21:12:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1f3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70b76650 0x557d70c358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1eaa00640 1 -- 172.21.15.16:0/3335200475 learned_addr learned my addr 172.21.15.16:0/3335200475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1f3e00640 1 --2- 172.21.15.16:0/3335200475 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc2fc0 0x557d70c36370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1f3400640 1 -- 172.21.15.16:0/3335200475 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc2fc0 msgr2=0x557d70c36370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1f3400640 1 --2- 172.21.15.16:0/3335200475 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc2fc0 0x557d70c36370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1f3400640 1 -- 172.21.15.16:0/3335200475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70bc26a0 msgr2=0x557d70c35e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1f3400640 1 --2- 172.21.15.16:0/3335200475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70bc26a0 0x557d70c35e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1f3400640 1 -- 172.21.15.16:0/3335200475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d70a1bc90 con 0x557d70b76650 2024-09-10T21:12:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1eaa00640 1 --2- 172.21.15.16:0/3335200475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70bc26a0 0x557d70c35e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1f3400640 1 --2- 172.21.15.16:0/3335200475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70b76650 0x557d70c358f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4066e60 tx=0x7fb1e405d090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8a429bbe0e2ba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1e3400640 1 -- 172.21.15.16:0/3335200475 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4086070 con 0x557d70b76650 2024-09-10T21:12:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1e3400640 1 -- 172.21.15.16:0/3335200475 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1e4062280 con 0x557d70b76650 2024-09-10T21:12:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.584+0000 7fb1e3400640 1 -- 172.21.15.16:0/3335200475 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40591a0 con 0x557d70b76650 2024-09-10T21:12:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 -- 172.21.15.16:0/3335200475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70b76650 msgr2=0x557d70c358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 --2- 172.21.15.16:0/3335200475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70b76650 0x557d70c358f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4066e60 tx=0x7fb1e405d090 comp rx=0 tx=0).stop 2024-09-10T21:12:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 -- 172.21.15.16:0/3335200475 shutdown_connections 2024-09-10T21:12:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 --2- 172.21.15.16:0/3335200475 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc2fc0 0x557d70c36370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 --2- 172.21.15.16:0/3335200475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70bc26a0 0x557d70c35e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 --2- 172.21.15.16:0/3335200475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70b76650 0x557d70c358f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 -- 172.21.15.16:0/3335200475 >> 172.21.15.16:0/3335200475 conn(0x557d70a5bec0 msgr2=0x557d70bc9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 -- 172.21.15.16:0/3335200475 shutdown_connections 2024-09-10T21:12:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 -- 172.21.15.16:0/3335200475 wait complete. 2024-09-10T21:12:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 -- start start 2024-09-10T21:12:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 Processor -- start 2024-09-10T21:12:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 0x557d70c38f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70bc26a0 0x557d70c39490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc2fc0 0x557d70c399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557d709c20c0 con 0x557d70b76650 2024-09-10T21:12:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557d70981690 con 0x557d70bc26a0 2024-09-10T21:12:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x557d709c73e0 con 0x557d70bc2fc0 2024-09-10T21:12:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 10 client.0 _ll_register_callbacks cb 0x7fb1e400c310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1f3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 0x557d70c38f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1f3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 0x557d70c38f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34134/0 (socket says 172.21.15.16:34134) 2024-09-10T21:12:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1f3400640 1 -- 172.21.15.16:0/495181588 learned_addr learned my addr 172.21.15.16:0/495181588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1f3e00640 1 --2- 172.21.15.16:0/495181588 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc2fc0 0x557d70c399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1eaa00640 1 --2- 172.21.15.16:0/495181588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70bc26a0 0x557d70c39490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1f3400640 1 -- 172.21.15.16:0/495181588 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc2fc0 msgr2=0x557d70c399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1f3400640 1 --2- 172.21.15.16:0/495181588 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc2fc0 0x557d70c399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1f3400640 1 -- 172.21.15.16:0/495181588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70bc26a0 msgr2=0x557d70c39490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1f3400640 1 --2- 172.21.15.16:0/495181588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70bc26a0 0x557d70c39490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1f3400640 1 -- 172.21.15.16:0/495181588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d70985970 con 0x557d70b76650 2024-09-10T21:12:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1f3400640 1 --2- 172.21.15.16:0/495181588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 0x557d70c38f50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4066e60 tx=0x7fb1e4059d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d685a142644504f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:12:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1e0c00640 1 -- 172.21.15.16:0/495181588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4086070 con 0x557d70b76650 2024-09-10T21:12:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1e0c00640 1 -- 172.21.15.16:0/495181588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb1e405ead0 con 0x557d70b76650 2024-09-10T21:12:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 -- 172.21.15.16:0/495181588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e4062280 con 0x557d70b76650 2024-09-10T21:12:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1e0c00640 1 -- 172.21.15.16:0/495181588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fb1e407e050 con 0x557d70b76650 2024-09-10T21:12:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 10 client.5079 fetch_fsmap learned FSMap version 16 2024-09-10T21:12:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.588+0000 7fb1fd751000 1 -- 172.21.15.16:0/495181588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x557d709641c0 con 0x557d70b76650 2024-09-10T21:12:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1e0c00640 1 -- 172.21.15.16:0/495181588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1e4062280 con 0x557d70b76650 2024-09-10T21:12:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1e0c00640 1 -- 172.21.15.16:0/495181588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7fb1e405c520 con 0x557d70b76650 2024-09-10T21:12:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 10 client.5079 fetch_fsmap finished waiting for FSMap version 16 2024-09-10T21:12:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 10 client.5079 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.b 2024-09-10T21:12:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 20 client.5079 populate_metadata read hostname 'smithi016' 2024-09-10T21:12:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 1 --2- 172.21.15.16:0/495181588 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x557d70bcc5c0 0x557d70bcc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 4 client.5079 mds_command: new command op to 134519 tid=0 multi_id=0 [{"prefix": "dump_historic_ops"}] 2024-09-10T21:12:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 1 -- 172.21.15.16:0/495181588 --> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] -- command(tid 0: {"prefix": "dump_historic_ops"}) -- 0x557d709cd570 con 0x557d70bcc5c0 2024-09-10T21:12:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1eaa00640 1 --2- 172.21.15.16:0/495181588 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x557d70bcc5c0 0x557d70bcc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1eaa00640 1 --2- 172.21.15.16:0/495181588 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x557d70bcc5c0 0x557d70bcc9a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccd2e0169da3ac97 server_cookie=fffe42571b75cc84 in_seq=0 out_seq=0 2024-09-10T21:12:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1e0c00640 10 client.5079 ms_handle_connect on v2:172.21.15.52:6833/4218881085 2024-09-10T21:12:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1e0c00640 1 -- 172.21.15.16:0/495181588 <== mds.0 v2:172.21.15.52:6833/4218881085 1 ==== command_reply(tid 0: 0 ) ==== 8+0+71648 (crc 0 0 0) 0x557d709cd570 con 0x557d70bcc5c0 2024-09-10T21:12:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1e0c00640 10 client.5079 handle_command_reply: tid=0 2024-09-10T21:12:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 1 client.5079 shutdown 2024-09-10T21:12:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 1 -- 172.21.15.16:0/495181588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 msgr2=0x557d70c38f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 1 --2- 172.21.15.16:0/495181588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 0x557d70c38f50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4066e60 tx=0x7fb1e4059d70 comp rx=0 tx=0).stop 2024-09-10T21:12:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 1 -- 172.21.15.16:0/495181588 shutdown_connections 2024-09-10T21:12:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 1 --2- 172.21.15.16:0/495181588 >> [v2:172.21.15.52:6833/4218881085,v1:172.21.15.52:6835/4218881085] conn(0x557d70bcc5c0 0x557d70bcc9a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 1 --2- 172.21.15.16:0/495181588 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x557d70bc2fc0 0x557d70c399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 1 --2- 172.21.15.16:0/495181588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d70bc26a0 0x557d70c39490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1e0c00640 0 client.5079 ms_handle_reset on v2:172.21.15.52:6833/4218881085 2024-09-10T21:12:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 1 --2- 172.21.15.16:0/495181588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d70b76650 0x557d70c38f50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 1 -- 172.21.15.16:0/495181588 >> 172.21.15.16:0/495181588 conn(0x557d70a5bec0 msgr2=0x557d70ba31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 1 -- 172.21.15.16:0/495181588 shutdown_connections 2024-09-10T21:12:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 1 -- 172.21.15.16:0/495181588 wait complete. 2024-09-10T21:12:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.592+0000 7fb1fd751000 20 client.5079 trim_cache size 0 max 16384 2024-09-10T21:12:08.606 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:12:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "size": 20, 2024-09-10T21:12:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 600, 2024-09-10T21:12:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "ops": [ 2024-09-10T21:12:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:2 setattr mode=041777 #0x1 2024-09-10T21:11:24.801284+0000 caller_uid=0, caller_gid=0{0,})", 2024-09-10T21:12:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:24.802434+0000", 2024-09-10T21:12:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 43.792759879999998, 2024-09-10T21:12:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 1.028300676, 2024-09-10T21:12:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 2 2024-09-10T21:12:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:24.802434+0000", 2024-09-10T21:12:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:24.802434+0000", 2024-09-10T21:12:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:24.802434+0000", 2024-09-10T21:12:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:24.802441+0000", 2024-09-10T21:12:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:24.802497+0000", 2024-09-10T21:12:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:24.802637+0000", 2024-09-10T21:12:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:24.802719+0000", 2024-09-10T21:12:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:24.802719+0000", 2024-09-10T21:12:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:25.830616+0000", 2024-09-10T21:12:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:25.830647+0000", 2024-09-10T21:12:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:25.830716+0000", 2024-09-10T21:12:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:25.830734+0000", 2024-09-10T21:12:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:25.830734+0000", 2024-09-10T21:12:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:27 create owner_uid=1000, owner_gid=1271 #0x10000000000/file_0 2024-09-10T21:11:26.949261+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:26.950456+0000", 2024-09-10T21:12:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.644737640999999, 2024-09-10T21:12:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.010500023000000001, 2024-09-10T21:12:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 27 2024-09-10T21:12:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.950456+0000", 2024-09-10T21:12:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.950456+0000", 2024-09-10T21:12:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.950456+0000", 2024-09-10T21:12:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.950467+0000", 2024-09-10T21:12:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.950495+0000", 2024-09-10T21:12:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.950668+0000", 2024-09-10T21:12:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.950945+0000", 2024-09-10T21:12:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.950945+0000", 2024-09-10T21:12:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.960667+0000", 2024-09-10T21:12:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.960797+0000", 2024-09-10T21:12:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.960876+0000", 2024-09-10T21:12:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.960955+0000", 2024-09-10T21:12:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.960956+0000", 2024-09-10T21:12:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:28 create owner_uid=1000, owner_gid=1271 #0x10000000000/file_1 2024-09-10T21:11:26.957261+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:26.961536+0000", 2024-09-10T21:12:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.633657223, 2024-09-10T21:12:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0060813810000000003, 2024-09-10T21:12:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 28 2024-09-10T21:12:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.961536+0000", 2024-09-10T21:12:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.961536+0000", 2024-09-10T21:12:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.961537+0000", 2024-09-10T21:12:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.961546+0000", 2024-09-10T21:12:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.961573+0000", 2024-09-10T21:12:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.961746+0000", 2024-09-10T21:12:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.962063+0000", 2024-09-10T21:12:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.962063+0000", 2024-09-10T21:12:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.967331+0000", 2024-09-10T21:12:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.967456+0000", 2024-09-10T21:12:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.967537+0000", 2024-09-10T21:12:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.967617+0000", 2024-09-10T21:12:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.967618+0000", 2024-09-10T21:12:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:29 create owner_uid=1000, owner_gid=1271 #0x10000000000/file_2 2024-09-10T21:11:26.965260+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:26.968030+0000", 2024-09-10T21:12:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.627163996, 2024-09-10T21:12:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0062460240000000002, 2024-09-10T21:12:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 29 2024-09-10T21:12:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.968030+0000", 2024-09-10T21:12:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.968030+0000", 2024-09-10T21:12:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.968030+0000", 2024-09-10T21:12:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.968037+0000", 2024-09-10T21:12:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.968064+0000", 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.968220+0000", 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.968521+0000", 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.968522+0000", 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.973975+0000", 2024-09-10T21:12:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.974115+0000", 2024-09-10T21:12:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.974198+0000", 2024-09-10T21:12:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.974275+0000", 2024-09-10T21:12:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.974276+0000", 2024-09-10T21:12:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:34 create owner_uid=1000, owner_gid=1271 #0x10000000000/file_7 2024-09-10T21:11:26.993260+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:26.995585+0000", 2024-09-10T21:12:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.599608519, 2024-09-10T21:12:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0063265989999999996, 2024-09-10T21:12:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 34 2024-09-10T21:12:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.995585+0000", 2024-09-10T21:12:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.995585+0000", 2024-09-10T21:12:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.995585+0000", 2024-09-10T21:12:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.995592+0000", 2024-09-10T21:12:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.995618+0000", 2024-09-10T21:12:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.995778+0000", 2024-09-10T21:12:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.996070+0000", 2024-09-10T21:12:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:26.996070+0000", 2024-09-10T21:12:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.001623+0000", 2024-09-10T21:12:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.001751+0000", 2024-09-10T21:12:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.001831+0000", 2024-09-10T21:12:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.001911+0000", 2024-09-10T21:12:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.001912+0000", 2024-09-10T21:12:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:36 create owner_uid=1000, owner_gid=1271 #0x10000000000/file_9 2024-09-10T21:11:27.005260+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:27.006923+0000", 2024-09-10T21:12:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.588270340999998, 2024-09-10T21:12:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.005295653, 2024-09-10T21:12:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 36 2024-09-10T21:12:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.006923+0000", 2024-09-10T21:12:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.006923+0000", 2024-09-10T21:12:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.006924+0000", 2024-09-10T21:12:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.006941+0000", 2024-09-10T21:12:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.006966+0000", 2024-09-10T21:12:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.007123+0000", 2024-09-10T21:12:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.007411+0000", 2024-09-10T21:12:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.007411+0000", 2024-09-10T21:12:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.011926+0000", 2024-09-10T21:12:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.012066+0000", 2024-09-10T21:12:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.012144+0000", 2024-09-10T21:12:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.012218+0000", 2024-09-10T21:12:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.012219+0000", 2024-09-10T21:12:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:41 create owner_uid=1000, owner_gid=1271 #0x10000000000/file_14 2024-09-10T21:11:27.029260+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:27.033277+0000", 2024-09-10T21:12:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.561916134000001, 2024-09-10T21:12:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0056581670000000004, 2024-09-10T21:12:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 41 2024-09-10T21:12:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.033277+0000", 2024-09-10T21:12:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.033277+0000", 2024-09-10T21:12:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.033278+0000", 2024-09-10T21:12:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.033283+0000", 2024-09-10T21:12:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.033308+0000", 2024-09-10T21:12:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.033463+0000", 2024-09-10T21:12:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.033748+0000", 2024-09-10T21:12:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.033748+0000", 2024-09-10T21:12:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.038666+0000", 2024-09-10T21:12:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.038785+0000", 2024-09-10T21:12:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.038859+0000", 2024-09-10T21:12:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.038935+0000", 2024-09-10T21:12:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.038936+0000", 2024-09-10T21:12:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:42 create owner_uid=1000, owner_gid=1271 #0x10000000000/file_15 2024-09-10T21:11:27.037260+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:27.039092+0000", 2024-09-10T21:12:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.556101449000003, 2024-09-10T21:12:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0055811510000000003, 2024-09-10T21:12:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 42 2024-09-10T21:12:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.039092+0000", 2024-09-10T21:12:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.039092+0000", 2024-09-10T21:12:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.039093+0000", 2024-09-10T21:12:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.039098+0000", 2024-09-10T21:12:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.039125+0000", 2024-09-10T21:12:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.039280+0000", 2024-09-10T21:12:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.039597+0000", 2024-09-10T21:12:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.039597+0000", 2024-09-10T21:12:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.044374+0000", 2024-09-10T21:12:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.044515+0000", 2024-09-10T21:12:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.044595+0000", 2024-09-10T21:12:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.044673+0000", 2024-09-10T21:12:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.044673+0000", 2024-09-10T21:12:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:43 create owner_uid=1000, owner_gid=1271 #0x10000000000/file_16 2024-09-10T21:11:27.041260+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:27.045047+0000", 2024-09-10T21:12:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.550146429999998, 2024-09-10T21:12:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0052283820000000002, 2024-09-10T21:12:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 43 2024-09-10T21:12:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.045047+0000", 2024-09-10T21:12:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.045047+0000", 2024-09-10T21:12:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.045048+0000", 2024-09-10T21:12:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.045055+0000", 2024-09-10T21:12:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.045079+0000", 2024-09-10T21:12:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.045237+0000", 2024-09-10T21:12:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.045542+0000", 2024-09-10T21:12:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.045542+0000", 2024-09-10T21:12:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.049976+0000", 2024-09-10T21:12:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.050117+0000", 2024-09-10T21:12:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.050195+0000", 2024-09-10T21:12:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.050275+0000", 2024-09-10T21:12:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.050275+0000", 2024-09-10T21:12:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:47 create owner_uid=1000, owner_gid=1271 #0x10000000000/file_20 2024-09-10T21:11:27.061259+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:27.064978+0000", 2024-09-10T21:12:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.530215511999998, 2024-09-10T21:12:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0057419539999999996, 2024-09-10T21:12:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 47 2024-09-10T21:12:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.064978+0000", 2024-09-10T21:12:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.064978+0000", 2024-09-10T21:12:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.064978+0000", 2024-09-10T21:12:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.064985+0000", 2024-09-10T21:12:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.065008+0000", 2024-09-10T21:12:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.065175+0000", 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.065472+0000", 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.065473+0000", 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.070428+0000", 2024-09-10T21:12:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.070560+0000", 2024-09-10T21:12:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.070641+0000", 2024-09-10T21:12:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.070719+0000", 2024-09-10T21:12:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.070720+0000", 2024-09-10T21:12:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:54 mkdir owner_uid=1000, owner_gid=1271 #0x1/dir2 2024-09-10T21:11:27.269257+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:27.272914+0000", 2024-09-10T21:12:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.322280004, 2024-09-10T21:12:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0071314450000000001, 2024-09-10T21:12:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 54 2024-09-10T21:12:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.272914+0000", 2024-09-10T21:12:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.272914+0000", 2024-09-10T21:12:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.272914+0000", 2024-09-10T21:12:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.272918+0000", 2024-09-10T21:12:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.272936+0000", 2024-09-10T21:12:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.273122+0000", 2024-09-10T21:12:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "failed to wrlock, waiting" 2024-09-10T21:12:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.273536+0000", 2024-09-10T21:12:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.273782+0000", 2024-09-10T21:12:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.273783+0000", 2024-09-10T21:12:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.279838+0000", 2024-09-10T21:12:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.279936+0000", 2024-09-10T21:12:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280014+0000", 2024-09-10T21:12:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280045+0000", 2024-09-10T21:12:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280045+0000", 2024-09-10T21:12:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:55 create owner_uid=1000, owner_gid=1271 #0x1000000020e/file_0 2024-09-10T21:11:27.273257+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:27.274354+0000", 2024-09-10T21:12:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.320839173000003, 2024-09-10T21:12:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0059726559999999998, 2024-09-10T21:12:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 55 2024-09-10T21:12:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.274354+0000", 2024-09-10T21:12:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.274354+0000", 2024-09-10T21:12:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.274355+0000", 2024-09-10T21:12:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.274361+0000", 2024-09-10T21:12:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.274408+0000", 2024-09-10T21:12:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.274585+0000", 2024-09-10T21:12:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.274883+0000", 2024-09-10T21:12:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.274884+0000", 2024-09-10T21:12:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280047+0000", 2024-09-10T21:12:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280175+0000", 2024-09-10T21:12:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280250+0000", 2024-09-10T21:12:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280327+0000", 2024-09-10T21:12:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280327+0000", 2024-09-10T21:12:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:56 create owner_uid=1000, owner_gid=1271 #0x1000000020e/file_1 2024-09-10T21:11:27.277257+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:27.280556+0000", 2024-09-10T21:12:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.314637013999999, 2024-09-10T21:12:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0054328780000000004, 2024-09-10T21:12:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 56 2024-09-10T21:12:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280556+0000", 2024-09-10T21:12:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280556+0000", 2024-09-10T21:12:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280557+0000", 2024-09-10T21:12:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280561+0000", 2024-09-10T21:12:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280591+0000", 2024-09-10T21:12:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.280885+0000", 2024-09-10T21:12:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.281294+0000", 2024-09-10T21:12:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.281294+0000", 2024-09-10T21:12:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.285701+0000", 2024-09-10T21:12:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.285820+0000", 2024-09-10T21:12:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.285893+0000", 2024-09-10T21:12:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.285989+0000", 2024-09-10T21:12:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.285989+0000", 2024-09-10T21:12:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:82 mkdir owner_uid=1000, owner_gid=1271 #0x1/dir3 2024-09-10T21:11:27.581254+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:27.585762+0000", 2024-09-10T21:12:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.009431016000001, 2024-09-10T21:12:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0071601590000000001, 2024-09-10T21:12:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 82 2024-09-10T21:12:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.585762+0000", 2024-09-10T21:12:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.585762+0000", 2024-09-10T21:12:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.585763+0000", 2024-09-10T21:12:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.585766+0000", 2024-09-10T21:12:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.585786+0000", 2024-09-10T21:12:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.585976+0000", 2024-09-10T21:12:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "failed to wrlock, waiting" 2024-09-10T21:12:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.586350+0000", 2024-09-10T21:12:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.586606+0000", 2024-09-10T21:12:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.586607+0000", 2024-09-10T21:12:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.592710+0000", 2024-09-10T21:12:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.592811+0000", 2024-09-10T21:12:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.592892+0000", 2024-09-10T21:12:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.592922+0000", 2024-09-10T21:12:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.592923+0000", 2024-09-10T21:12:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:83 create owner_uid=1000, owner_gid=1271 #0x10000000228/file_0 2024-09-10T21:11:27.585254+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:27.587278+0000", 2024-09-10T21:12:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.007915957000002, 2024-09-10T21:12:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0059141319999999999, 2024-09-10T21:12:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 83 2024-09-10T21:12:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.587278+0000", 2024-09-10T21:12:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.587278+0000", 2024-09-10T21:12:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.587279+0000", 2024-09-10T21:12:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.587290+0000", 2024-09-10T21:12:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.587354+0000", 2024-09-10T21:12:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.587517+0000", 2024-09-10T21:12:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.587795+0000", 2024-09-10T21:12:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.587795+0000", 2024-09-10T21:12:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.592925+0000", 2024-09-10T21:12:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.593036+0000", 2024-09-10T21:12:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.593110+0000", 2024-09-10T21:12:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.593191+0000", 2024-09-10T21:12:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.593192+0000", 2024-09-10T21:12:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:98 create owner_uid=1000, owner_gid=1271 #0x10000000228/file_15 2024-09-10T21:11:27.657253+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:27.661178+0000", 2024-09-10T21:12:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.934015862000003, 2024-09-10T21:12:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0052389819999999997, 2024-09-10T21:12:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 98 2024-09-10T21:12:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.661178+0000", 2024-09-10T21:12:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.661178+0000", 2024-09-10T21:12:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.661178+0000", 2024-09-10T21:12:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.661182+0000", 2024-09-10T21:12:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.661221+0000", 2024-09-10T21:12:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.661400+0000", 2024-09-10T21:12:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.661736+0000", 2024-09-10T21:12:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.661737+0000", 2024-09-10T21:12:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.666111+0000", 2024-09-10T21:12:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.666237+0000", 2024-09-10T21:12:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.666327+0000", 2024-09-10T21:12:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.666416+0000", 2024-09-10T21:12:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.666417+0000", 2024-09-10T21:12:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:110 mkdir owner_uid=1000, owner_gid=1271 #0x1/dir4 2024-09-10T21:11:27.885250+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:27.889660+0000", 2024-09-10T21:12:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.705533678999998, 2024-09-10T21:12:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0072045080000000001, 2024-09-10T21:12:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.715 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.715 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.715 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.715 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.715 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 110 2024-09-10T21:12:08.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.715 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.716 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.716 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.716 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.889660+0000", 2024-09-10T21:12:08.716 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.716 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.716 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.889660+0000", 2024-09-10T21:12:08.716 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.716 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.717 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.889660+0000", 2024-09-10T21:12:08.717 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.717 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.717 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.889663+0000", 2024-09-10T21:12:08.717 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.717 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.717 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.889681+0000", 2024-09-10T21:12:08.717 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.718 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.718 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.889873+0000", 2024-09-10T21:12:08.718 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "failed to wrlock, waiting" 2024-09-10T21:12:08.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.718 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.718 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.890245+0000", 2024-09-10T21:12:08.718 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.718 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.718 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.890525+0000", 2024-09-10T21:12:08.719 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.719 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.719 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.890526+0000", 2024-09-10T21:12:08.719 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.719 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.719 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.896650+0000", 2024-09-10T21:12:08.719 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.719 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.720 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.896756+0000", 2024-09-10T21:12:08.720 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.720 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.720 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.896833+0000", 2024-09-10T21:12:08.720 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.720 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.720 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.896864+0000", 2024-09-10T21:12:08.720 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.721 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.721 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.896864+0000", 2024-09-10T21:12:08.721 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.721 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.721 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.721 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.721 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.721 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.721 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:111 create owner_uid=1000, owner_gid=1271 #0x10000000242/file_0 2024-09-10T21:11:27.889250+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.722 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:27.891058+0000", 2024-09-10T21:12:08.722 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.704135637999997, 2024-09-10T21:12:08.722 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0060919310000000001, 2024-09-10T21:12:08.722 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.722 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.722 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.722 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.723 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 111 2024-09-10T21:12:08.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.723 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.723 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.723 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.723 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.891058+0000", 2024-09-10T21:12:08.723 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.723 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.724 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.891058+0000", 2024-09-10T21:12:08.724 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.724 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.724 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.891059+0000", 2024-09-10T21:12:08.724 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.724 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.724 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.891067+0000", 2024-09-10T21:12:08.724 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.725 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.725 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.891160+0000", 2024-09-10T21:12:08.725 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.725 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.725 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.891553+0000", 2024-09-10T21:12:08.725 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.725 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.725 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.891833+0000", 2024-09-10T21:12:08.725 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-10T21:12:08.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.726 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.726 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.891833+0000", 2024-09-10T21:12:08.726 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-10T21:12:08.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.726 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.726 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.896867+0000", 2024-09-10T21:12:08.726 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-10T21:12:08.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.726 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.726 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.896983+0000", 2024-09-10T21:12:08.726 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.727 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.727 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.897064+0000", 2024-09-10T21:12:08.727 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.727 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.727 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.897149+0000", 2024-09-10T21:12:08.727 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.727 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.727 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:27.897150+0000", 2024-09-10T21:12:08.728 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.728 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.728 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.728 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.728 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.728 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.728 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:143 readdir #0x10000000242 2024-09-10T21:11:28.237247+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.728 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:28.238700+0000", 2024-09-10T21:12:08.728 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.356493051000001, 2024-09-10T21:12:08.728 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 16.507472661000001, 2024-09-10T21:12:08.729 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.729 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.729 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.729 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.729 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 143 2024-09-10T21:12:08.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.730 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.730 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.730 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.730 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:28.238700+0000", 2024-09-10T21:12:08.730 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.730 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.730 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:28.238700+0000", 2024-09-10T21:12:08.730 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.731 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.731 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:28.238701+0000", 2024-09-10T21:12:08.731 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.731 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.731 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:28.238705+0000", 2024-09-10T21:12:08.731 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.731 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.731 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:28.238749+0000", 2024-09-10T21:12:08.731 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.732 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.732 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:28.238834+0000", 2024-09-10T21:12:08.732 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.732 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.732 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:28.739025+0000", 2024-09-10T21:12:08.732 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.732 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.733 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:29.239118+0000", 2024-09-10T21:12:08.733 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.733 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.733 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:29.739320+0000", 2024-09-10T21:12:08.733 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.733 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.733 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:30.239483+0000", 2024-09-10T21:12:08.733 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.734 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.734 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:30.739694+0000", 2024-09-10T21:12:08.734 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.734 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.734 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:31.239899+0000", 2024-09-10T21:12:08.734 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.734 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.734 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:31.740118+0000", 2024-09-10T21:12:08.735 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.735 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.735 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:32.240290+0000", 2024-09-10T21:12:08.735 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.735 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.735 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:32.740477+0000", 2024-09-10T21:12:08.735 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.735 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.735 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:33.240691+0000", 2024-09-10T21:12:08.736 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.736 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.736 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:33.740897+0000", 2024-09-10T21:12:08.736 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.736 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.736 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:34.241102+0000", 2024-09-10T21:12:08.736 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.737 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.737 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:34.741262+0000", 2024-09-10T21:12:08.737 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.737 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.737 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:35.241470+0000", 2024-09-10T21:12:08.737 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.737 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.737 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:35.741666+0000", 2024-09-10T21:12:08.737 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.738 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.738 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:36.241876+0000", 2024-09-10T21:12:08.738 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.738 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.738 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:36.742058+0000", 2024-09-10T21:12:08.738 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.738 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.738 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:37.242161+0000", 2024-09-10T21:12:08.739 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.739 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.739 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:37.742336+0000", 2024-09-10T21:12:08.739 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.739 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.739 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:38.242490+0000", 2024-09-10T21:12:08.739 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.739 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.740 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:38.742674+0000", 2024-09-10T21:12:08.740 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.740 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.740 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:39.242872+0000", 2024-09-10T21:12:08.740 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.740 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.740 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:39.743082+0000", 2024-09-10T21:12:08.740 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.741 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.741 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:40.243292+0000", 2024-09-10T21:12:08.741 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.741 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.741 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:40.743506+0000", 2024-09-10T21:12:08.741 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.741 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.741 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:41.243713+0000", 2024-09-10T21:12:08.741 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.742 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.742 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:41.743927+0000", 2024-09-10T21:12:08.742 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.742 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.742 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:42.244141+0000", 2024-09-10T21:12:08.742 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.742 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.742 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:42.744331+0000", 2024-09-10T21:12:08.743 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.743 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.743 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:43.244452+0000", 2024-09-10T21:12:08.743 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.743 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.743 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:43.744639+0000", 2024-09-10T21:12:08.743 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.743 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.743 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:44.244841+0000", 2024-09-10T21:12:08.743 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.744 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.744 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:44.745247+0000", 2024-09-10T21:12:08.744 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.744 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.744 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:44.745960+0000", 2024-09-10T21:12:08.744 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.744 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.744 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:44.746054+0000", 2024-09-10T21:12:08.744 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.745 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.745 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:44.746172+0000", 2024-09-10T21:12:08.745 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.745 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.745 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:44.746173+0000", 2024-09-10T21:12:08.745 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.745 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.745 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.745 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.746 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.746 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.746 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.4939:144 readdir #0x10000000228 2024-09-10T21:11:44.745069+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-10T21:12:08.746 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-10T21:11:44.746762+0000", 2024-09-10T21:12:08.746 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 23.848431820999998, 2024-09-10T21:12:08.746 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 22.509502384000001, 2024-09-10T21:12:08.746 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-10T21:12:08.746 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-10T21:12:08.746 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-10T21:12:08.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-10T21:12:08.747 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-10T21:12:08.747 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-10T21:12:08.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-10T21:12:08.747 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4939 2024-09-10T21:12:08.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.747 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 144 2024-09-10T21:12:08.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.747 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-10T21:12:08.747 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-10T21:12:08.747 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.747 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:44.746762+0000", 2024-09-10T21:12:08.747 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-10T21:12:08.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.748 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.748 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:44.746762+0000", 2024-09-10T21:12:08.748 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-10T21:12:08.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.748 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.748 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:44.746762+0000", 2024-09-10T21:12:08.748 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-10T21:12:08.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.748 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.748 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:44.746773+0000", 2024-09-10T21:12:08.749 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-10T21:12:08.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.749 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.749 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:44.746961+0000", 2024-09-10T21:12:08.749 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-10T21:12:08.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.749 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.749 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:44.747274+0000", 2024-09-10T21:12:08.749 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.750 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.750 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:45.247376+0000", 2024-09-10T21:12:08.750 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.750 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.750 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:45.747542+0000", 2024-09-10T21:12:08.750 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.750 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.750 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:46.247724+0000", 2024-09-10T21:12:08.751 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.751 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.751 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:46.747917+0000", 2024-09-10T21:12:08.751 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.751 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.751 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:47.248100+0000", 2024-09-10T21:12:08.751 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.751 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.751 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:47.748312+0000", 2024-09-10T21:12:08.752 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.752 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.752 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:48.248431+0000", 2024-09-10T21:12:08.752 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.752 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.752 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:48.748620+0000", 2024-09-10T21:12:08.752 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.753 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.753 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:49.248732+0000", 2024-09-10T21:12:08.753 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.753 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.753 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:49.748878+0000", 2024-09-10T21:12:08.753 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.753 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.753 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:50.249029+0000", 2024-09-10T21:12:08.753 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.754 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.754 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:50.749259+0000", 2024-09-10T21:12:08.754 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.754 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.754 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:51.249427+0000", 2024-09-10T21:12:08.754 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.754 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.754 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:51.749588+0000", 2024-09-10T21:12:08.754 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:52.249766+0000", 2024-09-10T21:12:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:52.749967+0000", 2024-09-10T21:12:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:53.250043+0000", 2024-09-10T21:12:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.756 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.756 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:53.750239+0000", 2024-09-10T21:12:08.756 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.756 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.756 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:54.250337+0000", 2024-09-10T21:12:08.756 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.756 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.757 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:54.750539+0000", 2024-09-10T21:12:08.757 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.757 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.757 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:55.250736+0000", 2024-09-10T21:12:08.757 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.757 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.757 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:55.750942+0000", 2024-09-10T21:12:08.757 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:56.251122+0000", 2024-09-10T21:12:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:56.751345+0000", 2024-09-10T21:12:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:57.251551+0000", 2024-09-10T21:12:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:57.751763+0000", 2024-09-10T21:12:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:58.251960+0000", 2024-09-10T21:12:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:58.752163+0000", 2024-09-10T21:12:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:59.252351+0000", 2024-09-10T21:12:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:11:59.752521+0000", 2024-09-10T21:12:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:00.252634+0000", 2024-09-10T21:12:08.761 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.761 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.761 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:00.752850+0000", 2024-09-10T21:12:08.761 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.761 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.761 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:01.252973+0000", 2024-09-10T21:12:08.761 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.761 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.762 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:01.753169+0000", 2024-09-10T21:12:08.762 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.762 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.762 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:02.253345+0000", 2024-09-10T21:12:08.762 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.762 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.762 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:02.753552+0000", 2024-09-10T21:12:08.762 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.762 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.763 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:03.253687+0000", 2024-09-10T21:12:08.763 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.763 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.763 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:03.753892+0000", 2024-09-10T21:12:08.763 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.763 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.763 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:04.254082+0000", 2024-09-10T21:12:08.763 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.763 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.764 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:04.754289+0000", 2024-09-10T21:12:08.764 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.764 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.764 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:05.254500+0000", 2024-09-10T21:12:08.764 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.764 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.764 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:05.754651+0000", 2024-09-10T21:12:08.764 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.765 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.765 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:06.254772+0000", 2024-09-10T21:12:08.765 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.765 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.765 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:06.754979+0000", 2024-09-10T21:12:08.765 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-10T21:12:08.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.765 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.765 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:07.255655+0000", 2024-09-10T21:12:08.766 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-10T21:12:08.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.766 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.766 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:07.256115+0000", 2024-09-10T21:12:08.766 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-10T21:12:08.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.766 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.766 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:07.256169+0000", 2024-09-10T21:12:08.766 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-10T21:12:08.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.766 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.766 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:07.256263+0000", 2024-09-10T21:12:08.767 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-10T21:12:08.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:08.767 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:08.767 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-10T21:12:07.256264+0000", 2024-09-10T21:12:08.767 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-10T21:12:08.767 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.767 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-10T21:12:08.767 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-10T21:12:08.767 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.767 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:08.767 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-10T21:12:08.768 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:12:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.596+0000 7fb1fa400640 1 -- 172.21.15.16:0/379728775 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1d00725e0 msgr2=0x7fb1d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.596+0000 7fb1fa400640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1d00725e0 0x7fb1d0074aa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb1f406f800 tx=0x7fb1ec009500 comp rx=0 tx=0).stop 2024-09-10T21:12:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.596+0000 7fb1fa400640 1 -- 172.21.15.16:0/379728775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1f40c42c0 msgr2=0x7fb1f415e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.596+0000 7fb1fa400640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1f40c42c0 0x7fb1f415e3a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400c3c0 tx=0x7fb1e400c880 comp rx=0 tx=0).stop 2024-09-10T21:12:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.596+0000 7fb1fa400640 1 -- 172.21.15.16:0/379728775 shutdown_connections 2024-09-10T21:12:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.596+0000 7fb1fa400640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1d00725e0 0x7fb1d0074aa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.596+0000 7fb1fa400640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1f415be10 0x7fb1f40c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.596+0000 7fb1fa400640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1f40cd070 0x7fb1f40c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.596+0000 7fb1fa400640 1 --2- 172.21.15.16:0/379728775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1f40c42c0 0x7fb1f415e3a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.596+0000 7fb1fa400640 1 -- 172.21.15.16:0/379728775 >> 172.21.15.16:0/379728775 conn(0x7fb1f40d0c30 msgr2=0x7fb1f41579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.596+0000 7fb1fa400640 1 -- 172.21.15.16:0/379728775 shutdown_connections 2024-09-10T21:12:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:08.596+0000 7fb1fa400640 1 -- 172.21.15.16:0/379728775 wait complete. 2024-09-10T21:12:08.771 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:12:08.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:12:08.872 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:12:08.940 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:12:08.940 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:12:08.941 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:12:08.941 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:08.950 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:12:08.950 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:09.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:12:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.184+0000 7f37bb600640 1 -- 172.21.15.16:0/3089748691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40cffe0 msgr2=0x7f37b40ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.184+0000 7f37bb600640 1 --2- 172.21.15.16:0/3089748691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40cffe0 0x7f37b40ce2b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f37b40b4930 tx=0x7f37a801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.184+0000 7f37bb600640 1 -- 172.21.15.16:0/3089748691 shutdown_connections 2024-09-10T21:12:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.184+0000 7f37bb600640 1 --2- 172.21.15.16:0/3089748691 >> v2:172.21.15.52:3300/0 conn(0x7f37b40b2860 0x7f37b4160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.184+0000 7f37bb600640 1 --2- 172.21.15.16:0/3089748691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40cffe0 0x7f37b40ce2b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.184+0000 7f37bb600640 1 -- 172.21.15.16:0/3089748691 >> 172.21.15.16:0/3089748691 conn(0x7f37b40d1d80 msgr2=0x7f37b40d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.184+0000 7f37bb600640 1 -- 172.21.15.16:0/3089748691 shutdown_connections 2024-09-10T21:12:09.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.184+0000 7f37bb600640 1 -- 172.21.15.16:0/3089748691 wait complete. 2024-09-10T21:12:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.184+0000 7f37bb600640 1 Processor -- start 2024-09-10T21:12:09.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.184+0000 7f37bb600640 1 -- start start 2024-09-10T21:12:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37bb600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40b2860 0x7f37b414d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37bb600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37b40cffe0 0x7f37b40c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:09.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37bb600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37b40d03c0 0x7f37b40c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37bb600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37b40cd9d0 con 0x7f37b40b2860 2024-09-10T21:12:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37bb600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f37b40cd850 con 0x7f37b40d03c0 2024-09-10T21:12:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37bb600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f37b40cdb50 con 0x7f37b40cffe0 2024-09-10T21:12:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40b2860 0x7f37b414d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40b2860 0x7f37b414d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34154/0 (socket says 172.21.15.16:34154) 2024-09-10T21:12:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b8c00640 1 -- 172.21.15.16:0/1928271095 learned_addr learned my addr 172.21.15.16:0/1928271095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b3e00640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37b40cffe0 0x7f37b40c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b9600640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37b40d03c0 0x7f37b40c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b8c00640 1 -- 172.21.15.16:0/1928271095 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37b40cffe0 msgr2=0x7f37b40c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b8c00640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37b40cffe0 0x7f37b40c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b8c00640 1 -- 172.21.15.16:0/1928271095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37b40d03c0 msgr2=0x7f37b40c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b8c00640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37b40d03c0 0x7f37b40c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b8c00640 1 -- 172.21.15.16:0/1928271095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b4074bf0 con 0x7f37b40b2860 2024-09-10T21:12:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b9600640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37b40d03c0 0x7f37b40c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b8c00640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40b2860 0x7f37b414d470 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f37a80b0040 tx=0x7f37a8005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b0c00640 1 -- 172.21.15.16:0/1928271095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a80b2040 con 0x7f37b40b2860 2024-09-10T21:12:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b0c00640 1 -- 172.21.15.16:0/1928271095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37a80a6da0 con 0x7f37b40b2860 2024-09-10T21:12:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b0c00640 1 -- 172.21.15.16:0/1928271095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8005280 con 0x7f37b40b2860 2024-09-10T21:12:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37bb600640 1 -- 172.21.15.16:0/1928271095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b40b4480 con 0x7f37b40b2860 2024-09-10T21:12:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37bb600640 1 -- 172.21.15.16:0/1928271095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b40afe70 con 0x7f37b40b2860 2024-09-10T21:12:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37bb600640 1 -- 172.21.15.16:0/1928271095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b40ce2b0 con 0x7f37b40b2860 2024-09-10T21:12:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.188+0000 7f37b0c00640 1 -- 172.21.15.16:0/1928271095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f37a8005420 con 0x7f37b40b2860 2024-09-10T21:12:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.192+0000 7f37b0c00640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3794072610 0x7f3794074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.192+0000 7f37b0c00640 1 -- 172.21.15.16:0/1928271095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f37a812ce30 con 0x7f37b40b2860 2024-09-10T21:12:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.192+0000 7f37b3e00640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3794072610 0x7f3794074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.192+0000 7f37b3e00640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3794072610 0x7f3794074ad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f37b40c5eb0 tx=0x7f37a0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:09.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.196+0000 7f37b0c00640 1 -- 172.21.15.16:0/1928271095 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f37a80bc060 con 0x7f37b40b2860 2024-09-10T21:12:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.388+0000 7f37bb600640 1 -- 172.21.15.16:0/1928271095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f37b40cf950 con 0x7f37b40b2860 2024-09-10T21:12:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.388+0000 7f37b0c00640 1 -- 172.21.15.16:0/1928271095 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7f37a80164b0 con 0x7f37b40b2860 2024-09-10T21:12:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:12:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.388+0000 7f37bb600640 1 -- 172.21.15.16:0/1928271095 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3794072610 msgr2=0x7f3794074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.388+0000 7f37bb600640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3794072610 0x7f3794074ad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f37b40c5eb0 tx=0x7f37a0009040 comp rx=0 tx=0).stop 2024-09-10T21:12:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.388+0000 7f37bb600640 1 -- 172.21.15.16:0/1928271095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40b2860 msgr2=0x7f37b414d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.388+0000 7f37bb600640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40b2860 0x7f37b414d470 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f37a80b0040 tx=0x7f37a8005760 comp rx=0 tx=0).stop 2024-09-10T21:12:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.392+0000 7f37bb600640 1 -- 172.21.15.16:0/1928271095 shutdown_connections 2024-09-10T21:12:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.392+0000 7f37bb600640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3794072610 0x7f3794074ad0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.392+0000 7f37bb600640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37b40d03c0 0x7f37b40c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.392+0000 7f37bb600640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37b40cffe0 0x7f37b40c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.392+0000 7f37bb600640 1 --2- 172.21.15.16:0/1928271095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40b2860 0x7f37b414d470 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.392+0000 7f37bb600640 1 -- 172.21.15.16:0/1928271095 >> 172.21.15.16:0/1928271095 conn(0x7f37b40d1d80 msgr2=0x7f37b4071640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.392+0000 7f37bb600640 1 -- 172.21.15.16:0/1928271095 shutdown_connections 2024-09-10T21:12:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.392+0000 7f37bb600640 1 -- 172.21.15.16:0/1928271095 wait complete. 2024-09-10T21:12:09.416 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:12:09.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:09.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:12:09.504 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:12:09.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:09.505 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:12:09.515 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:12:09.515 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:12:09.515 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:09.525 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:12:09.525 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:09.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:12:09.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.720+0000 7f105a800640 1 -- 172.21.15.16:0/3989098309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10540d03c0 msgr2=0x7f10540ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:09.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.720+0000 7f105a800640 1 --2- 172.21.15.16:0/3989098309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10540d03c0 0x7f10540ce2b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f103c009900 tx=0x7f103c01c8e0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.720+0000 7f105a800640 1 -- 172.21.15.16:0/3989098309 shutdown_connections 2024-09-10T21:12:09.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.720+0000 7f105a800640 1 --2- 172.21.15.16:0/3989098309 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10540b2860 0x7f1054160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.720+0000 7f105a800640 1 --2- 172.21.15.16:0/3989098309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10540d03c0 0x7f10540ce2b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.720+0000 7f105a800640 1 -- 172.21.15.16:0/3989098309 >> 172.21.15.16:0/3989098309 conn(0x7f10540d1d80 msgr2=0x7f10540d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:09.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.720+0000 7f105a800640 1 -- 172.21.15.16:0/3989098309 shutdown_connections 2024-09-10T21:12:09.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.720+0000 7f105a800640 1 -- 172.21.15.16:0/3989098309 wait complete. 2024-09-10T21:12:09.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.720+0000 7f105a800640 1 Processor -- start 2024-09-10T21:12:09.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.720+0000 7f105a800640 1 -- start start 2024-09-10T21:12:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f105a800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10540b2860 0x7f10540b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f105a800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10540cffe0 0x7f1054152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f105a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10540d03c0 0x7f105415faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f105a800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10540cd930 con 0x7f10540d03c0 2024-09-10T21:12:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f105a800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f10540cd7b0 con 0x7f10540cffe0 2024-09-10T21:12:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f105a800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f10540cdab0 con 0x7f10540b2860 2024-09-10T21:12:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f1053e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10540d03c0 0x7f105415faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f1053400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10540b2860 0x7f10540b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f1053e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10540d03c0 0x7f105415faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34180/0 (socket says 172.21.15.16:34180) 2024-09-10T21:12:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f1053e00640 1 -- 172.21.15.16:0/1611407227 learned_addr learned my addr 172.21.15.16:0/1611407227 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f104be00640 1 --2- 172.21.15.16:0/1611407227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10540cffe0 0x7f1054152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f1053e00640 1 -- 172.21.15.16:0/1611407227 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10540b2860 msgr2=0x7f10540b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f1053e00640 1 --2- 172.21.15.16:0/1611407227 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10540b2860 0x7f10540b75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f1053e00640 1 -- 172.21.15.16:0/1611407227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10540cffe0 msgr2=0x7f1054152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f1053e00640 1 --2- 172.21.15.16:0/1611407227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10540cffe0 0x7f1054152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f1053e00640 1 -- 172.21.15.16:0/1611407227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1054152e80 con 0x7f10540d03c0 2024-09-10T21:12:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f1053e00640 1 --2- 172.21.15.16:0/1611407227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10540d03c0 0x7f105415faf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f104c00ea80 tx=0x7f104c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f104b400640 1 -- 172.21.15.16:0/1611407227 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c004b30 con 0x7f10540d03c0 2024-09-10T21:12:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f104b400640 1 -- 172.21.15.16:0/1611407227 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f104c004cf0 con 0x7f10540d03c0 2024-09-10T21:12:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f105a800640 1 -- 172.21.15.16:0/1611407227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10541f7610 con 0x7f10540d03c0 2024-09-10T21:12:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f104b400640 1 -- 172.21.15.16:0/1611407227 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c005280 con 0x7f10540d03c0 2024-09-10T21:12:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f105a800640 1 -- 172.21.15.16:0/1611407227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10541f7ba0 con 0x7f10540d03c0 2024-09-10T21:12:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f105a800640 1 -- 172.21.15.16:0/1611407227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10540cf950 con 0x7f10540d03c0 2024-09-10T21:12:09.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.724+0000 7f104b400640 1 -- 172.21.15.16:0/1611407227 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f104c005420 con 0x7f10540d03c0 2024-09-10T21:12:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.728+0000 7f104b400640 1 --2- 172.21.15.16:0/1611407227 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1030072610 0x7f1030074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.728+0000 7f104b400640 1 -- 172.21.15.16:0/1611407227 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5825+0+0 (secure 0 0 0) 0x7f104c093ec0 con 0x7f10540d03c0 2024-09-10T21:12:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.728+0000 7f1053400640 1 --2- 172.21.15.16:0/1611407227 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1030072610 0x7f1030074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.728+0000 7f1053400640 1 --2- 172.21.15.16:0/1611407227 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1030072610 0x7f1030074ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1044000f70 tx=0x7f1044009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:09.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.732+0000 7f104b400640 1 -- 172.21.15.16:0/1611407227 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f104c05eec0 con 0x7f10540d03c0 2024-09-10T21:12:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.916+0000 7f105a800640 1 -- 172.21.15.16:0/1611407227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1054076250 con 0x7f10540d03c0 2024-09-10T21:12:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.916+0000 7f104b400640 1 -- 172.21.15.16:0/1611407227 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f104c0638f0 con 0x7f10540d03c0 2024-09-10T21:12:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:12:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.920+0000 7f105a800640 1 -- 172.21.15.16:0/1611407227 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1030072610 msgr2=0x7f1030074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.920+0000 7f105a800640 1 --2- 172.21.15.16:0/1611407227 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1030072610 0x7f1030074ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1044000f70 tx=0x7f1044009040 comp rx=0 tx=0).stop 2024-09-10T21:12:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.920+0000 7f105a800640 1 -- 172.21.15.16:0/1611407227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10540d03c0 msgr2=0x7f105415faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.920+0000 7f105a800640 1 --2- 172.21.15.16:0/1611407227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10540d03c0 0x7f105415faf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f104c00ea80 tx=0x7f104c00ef40 comp rx=0 tx=0).stop 2024-09-10T21:12:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.920+0000 7f105a800640 1 -- 172.21.15.16:0/1611407227 shutdown_connections 2024-09-10T21:12:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.920+0000 7f105a800640 1 --2- 172.21.15.16:0/1611407227 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1030072610 0x7f1030074ad0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.920+0000 7f105a800640 1 --2- 172.21.15.16:0/1611407227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10540d03c0 0x7f105415faf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.920+0000 7f105a800640 1 --2- 172.21.15.16:0/1611407227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10540cffe0 0x7f1054152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.920+0000 7f105a800640 1 --2- 172.21.15.16:0/1611407227 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10540b2860 0x7f10540b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.920+0000 7f105a800640 1 -- 172.21.15.16:0/1611407227 >> 172.21.15.16:0/1611407227 conn(0x7f10540d1d80 msgr2=0x7f1054159640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.920+0000 7f105a800640 1 -- 172.21.15.16:0/1611407227 shutdown_connections 2024-09-10T21:12:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:09.920+0000 7f105a800640 1 -- 172.21.15.16:0/1611407227 wait complete. 2024-09-10T21:12:09.946 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T21:12:09.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:09.946 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T21:12:09.975 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:12:09.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:09.975 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T21:12:10.026 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:12:10.026 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:12:10.026 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:10.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:12:10.035 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:12:10.035 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:12:10.035 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:10.081 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:12:10.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.220+0000 7f10d4200640 1 -- 172.21.15.16:0/1547708441 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10cc15e200 msgr2=0x7f10cc0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.220+0000 7f10d4200640 1 --2- 172.21.15.16:0/1547708441 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10cc15e200 0x7f10cc0ceb80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009840 tx=0x7f10c001c710 comp rx=0 tx=0).stop 2024-09-10T21:12:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.220+0000 7f10d4200640 1 -- 172.21.15.16:0/1547708441 shutdown_connections 2024-09-10T21:12:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.220+0000 7f10d4200640 1 --2- 172.21.15.16:0/1547708441 >> v2:172.21.15.52:3300/0 conn(0x7f10cc0d4b00 0x7f10cc0cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.220+0000 7f10d4200640 1 --2- 172.21.15.16:0/1547708441 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10cc15e200 0x7f10cc0ceb80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.220+0000 7f10d4200640 1 -- 172.21.15.16:0/1547708441 >> 172.21.15.16:0/1547708441 conn(0x7f10cc0d3f70 msgr2=0x7f10cc0d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:10.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.220+0000 7f10d4200640 1 -- 172.21.15.16:0/1547708441 shutdown_connections 2024-09-10T21:12:10.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.220+0000 7f10d4200640 1 -- 172.21.15.16:0/1547708441 wait complete. 2024-09-10T21:12:10.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d4200640 1 Processor -- start 2024-09-10T21:12:10.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d4200640 1 -- start start 2024-09-10T21:12:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d4200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10cc0cf600 0x7f10cc0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d4200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10cc0d4b00 0x7f10cc1f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d4200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10cc15e200 0x7f10cc153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d4200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc16b4f0 con 0x7f10cc0cf600 2024-09-10T21:12:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d4200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc16b370 con 0x7f10cc15e200 2024-09-10T21:12:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d4200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc16b670 con 0x7f10cc0d4b00 2024-09-10T21:12:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10cc0cf600 0x7f10cc0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10cc0cf600 0x7f10cc0cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34196/0 (socket says 172.21.15.16:34196) 2024-09-10T21:12:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d1800640 1 -- 172.21.15.16:0/913388530 learned_addr learned my addr 172.21.15.16:0/913388530 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d0e00640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10cc0d4b00 0x7f10cc1f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d2200640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10cc15e200 0x7f10cc153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d1800640 1 -- 172.21.15.16:0/913388530 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10cc0d4b00 msgr2=0x7f10cc1f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d1800640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10cc0d4b00 0x7f10cc1f7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d1800640 1 -- 172.21.15.16:0/913388530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10cc15e200 msgr2=0x7f10cc153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d1800640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10cc15e200 0x7f10cc153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d1800640 1 -- 172.21.15.16:0/913388530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc1f8780 con 0x7f10cc0cf600 2024-09-10T21:12:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d1800640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10cc0cf600 0x7f10cc0cc770 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f10c001adc0 tx=0x7f10c0004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10c9600640 1 -- 172.21.15.16:0/913388530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c00b2040 con 0x7f10cc0cf600 2024-09-10T21:12:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10c9600640 1 -- 172.21.15.16:0/913388530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10c00a5b60 con 0x7f10cc0cf600 2024-09-10T21:12:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10c9600640 1 -- 172.21.15.16:0/913388530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0003600 con 0x7f10cc0cf600 2024-09-10T21:12:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d4200640 1 -- 172.21.15.16:0/913388530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc1f8a60 con 0x7f10cc0cf600 2024-09-10T21:12:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.224+0000 7f10d4200640 1 -- 172.21.15.16:0/913388530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc150eb0 con 0x7f10cc0cf600 2024-09-10T21:12:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.228+0000 7f10d4200640 1 -- 172.21.15.16:0/913388530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc1509e0 con 0x7f10cc0cf600 2024-09-10T21:12:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.228+0000 7f10c9600640 1 -- 172.21.15.16:0/913388530 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f10c00b5070 con 0x7f10cc0cf600 2024-09-10T21:12:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.228+0000 7f10c9600640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f10ac072610 0x7f10ac074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.228+0000 7f10c9600640 1 -- 172.21.15.16:0/913388530 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5825+0+0 (secure 0 0 0) 0x7f10c012c700 con 0x7f10cc0cf600 2024-09-10T21:12:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.228+0000 7f10d0e00640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f10ac072610 0x7f10ac074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.228+0000 7f10d0e00640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f10ac072610 0x7f10ac074ad0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009600 tx=0x7f10b80091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:10.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.232+0000 7f10c9600640 1 -- 172.21.15.16:0/913388530 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f10c0016330 con 0x7f10cc0cf600 2024-09-10T21:12:10.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.440+0000 7f10d4200640 1 -- 172.21.15.16:0/913388530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10cc0b9850 con 0x7f10cc0cf600 2024-09-10T21:12:10.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.444+0000 7f10c9600640 1 -- 172.21.15.16:0/913388530 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5232 (secure 0 0 0) 0x7f10c00f7700 con 0x7f10cc0cf600 2024-09-10T21:12:10.449 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:10.450 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-10T21:11:16:232531+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/788814920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":788814920},{"type":"v1","addr":"172.21.15.16:6837","nonce":788814920}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2275087718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2275087718},{"type":"v1","addr":"172.21.15.16:6836","nonce":2275087718}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":134516,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3457135235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3457135235},{"type":"v1","addr":"172.21.15.52:6834","nonce":3457135235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:11:12.621848+0000","modified":"2024-09-10T21:11:15.636822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134519":{"gid":134519,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/4218881085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4218881085},{"type":"v1","addr":"172.21.15.52:6835","nonce":4218881085}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134519,"qdb_cluster":[134519]},"id":2}]} 2024-09-10T21:12:10.450 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-10T21:12:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.444+0000 7f10d4200640 1 -- 172.21.15.16:0/913388530 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f10ac072610 msgr2=0x7f10ac074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.444+0000 7f10d4200640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f10ac072610 0x7f10ac074ad0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009600 tx=0x7f10b80091e0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.444+0000 7f10d4200640 1 -- 172.21.15.16:0/913388530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10cc0cf600 msgr2=0x7f10cc0cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.444+0000 7f10d4200640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10cc0cf600 0x7f10cc0cc770 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f10c001adc0 tx=0x7f10c0004620 comp rx=0 tx=0).stop 2024-09-10T21:12:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.444+0000 7f10d4200640 1 -- 172.21.15.16:0/913388530 shutdown_connections 2024-09-10T21:12:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.444+0000 7f10d4200640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f10ac072610 0x7f10ac074ad0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.444+0000 7f10d4200640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10cc15e200 0x7f10cc153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.444+0000 7f10d4200640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10cc0d4b00 0x7f10cc1f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.444+0000 7f10d4200640 1 --2- 172.21.15.16:0/913388530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10cc0cf600 0x7f10cc0cc770 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.444+0000 7f10d4200640 1 -- 172.21.15.16:0/913388530 >> 172.21.15.16:0/913388530 conn(0x7f10cc0d3f70 msgr2=0x7f10cc156b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.444+0000 7f10d4200640 1 -- 172.21.15.16:0/913388530 shutdown_connections 2024-09-10T21:12:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.444+0000 7f10d4200640 1 -- 172.21.15.16:0/913388530 wait complete. 2024-09-10T21:12:10.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 -- 172.21.15.16:0/2411031917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d00d06b0 msgr2=0x7f18d00700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 --2- 172.21.15.16:0/2411031917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d00d06b0 0x7f18d00700a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f18d006e1a0 tx=0x7f18b801e590 comp rx=0 tx=0).stop 2024-09-10T21:12:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 -- 172.21.15.16:0/2411031917 shutdown_connections 2024-09-10T21:12:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 --2- 172.21.15.16:0/2411031917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d00d06b0 0x7f18d00700a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 --2- 172.21.15.16:0/2411031917 >> v2:172.21.15.52:3300/0 conn(0x7f18d015a270 0x7f18d00ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 -- 172.21.15.16:0/2411031917 >> 172.21.15.16:0/2411031917 conn(0x7f18d00cfb20 msgr2=0x7f18d00cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 -- 172.21.15.16:0/2411031917 shutdown_connections 2024-09-10T21:12:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 -- 172.21.15.16:0/2411031917 wait complete. 2024-09-10T21:12:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 Processor -- start 2024-09-10T21:12:10.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 -- start start 2024-09-10T21:12:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d00705e0 0x7f18d00b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18d00d06b0 0x7f18d014c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18d015a270 0x7f18d00c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0072b30 con 0x7f18d00705e0 2024-09-10T21:12:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f18d00729b0 con 0x7f18d00d06b0 2024-09-10T21:12:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18d4c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f18d0072cb0 con 0x7f18d015a270 2024-09-10T21:12:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18cde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d00705e0 0x7f18d00b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18cde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d00705e0 0x7f18d00b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34222/0 (socket says 172.21.15.16:34222) 2024-09-10T21:12:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18cde00640 1 -- 172.21.15.16:0/2053172535 learned_addr learned my addr 172.21.15.16:0/2053172535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18c7e00640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18d00d06b0 0x7f18d014c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18ce800640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18d015a270 0x7f18d00c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18cde00640 1 -- 172.21.15.16:0/2053172535 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18d015a270 msgr2=0x7f18d00c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18cde00640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18d015a270 0x7f18d00c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18cde00640 1 -- 172.21.15.16:0/2053172535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18d00d06b0 msgr2=0x7f18d014c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18cde00640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18d00d06b0 0x7f18d014c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18cde00640 1 -- 172.21.15.16:0/2053172535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d00b54e0 con 0x7f18d00705e0 2024-09-10T21:12:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18c7e00640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18d00d06b0 0x7f18d014c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.620+0000 7f18cde00640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d00705e0 0x7f18d00b3ee0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f18c000d720 tx=0x7f18c000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.624+0000 7f18c6000640 1 -- 172.21.15.16:0/2053172535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0017070 con 0x7f18d00705e0 2024-09-10T21:12:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.624+0000 7f18c6000640 1 -- 172.21.15.16:0/2053172535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18c0009d20 con 0x7f18d00705e0 2024-09-10T21:12:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.624+0000 7f18c6000640 1 -- 172.21.15.16:0/2053172535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0005280 con 0x7f18d00705e0 2024-09-10T21:12:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.624+0000 7f18d4c00640 1 -- 172.21.15.16:0/2053172535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d00b57c0 con 0x7f18d00705e0 2024-09-10T21:12:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.624+0000 7f18d4c00640 1 -- 172.21.15.16:0/2053172535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d00b36e0 con 0x7f18d00705e0 2024-09-10T21:12:10.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.624+0000 7f18d4c00640 1 -- 172.21.15.16:0/2053172535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d00700a0 con 0x7f18d00705e0 2024-09-10T21:12:10.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.624+0000 7f18c6000640 1 -- 172.21.15.16:0/2053172535 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f18c0005420 con 0x7f18d00705e0 2024-09-10T21:12:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.624+0000 7f18c6000640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18ac0725e0 0x7f18ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.624+0000 7f18c6000640 1 -- 172.21.15.16:0/2053172535 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5825+0+0 (secure 0 0 0) 0x7f18c00933c0 con 0x7f18d00705e0 2024-09-10T21:12:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.624+0000 7f18c7e00640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18ac0725e0 0x7f18ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.628+0000 7f18c7e00640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18ac0725e0 0x7f18ac074aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f18b80b0000 tx=0x7f18b801c370 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.628+0000 7f18c6000640 1 -- 172.21.15.16:0/2053172535 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f18c005e330 con 0x7f18d00705e0 2024-09-10T21:12:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.840+0000 7f18d4c00640 1 -- 172.21.15.16:0/2053172535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18d00b5aa0 con 0x7f18d00705e0 2024-09-10T21:12:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.840+0000 7f18c6000640 1 -- 172.21.15.16:0/2053172535 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5232 (secure 0 0 0) 0x7f18c00670a0 con 0x7f18d00705e0 2024-09-10T21:12:10.848 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:10.848 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-10T21:11:16:232531+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/788814920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":788814920},{"type":"v1","addr":"172.21.15.16:6837","nonce":788814920}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2275087718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2275087718},{"type":"v1","addr":"172.21.15.16:6836","nonce":2275087718}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":134516,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3457135235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3457135235},{"type":"v1","addr":"172.21.15.52:6834","nonce":3457135235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:11:12.621848+0000","modified":"2024-09-10T21:11:15.636822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134519":{"gid":134519,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/4218881085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4218881085},{"type":"v1","addr":"172.21.15.52:6835","nonce":4218881085}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134519,"qdb_cluster":[134519]},"id":2}]} 2024-09-10T21:12:10.849 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-10T21:12:10.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.844+0000 7f18d4c00640 1 -- 172.21.15.16:0/2053172535 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18ac0725e0 msgr2=0x7f18ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:10.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.844+0000 7f18d4c00640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18ac0725e0 0x7f18ac074aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f18b80b0000 tx=0x7f18b801c370 comp rx=0 tx=0).stop 2024-09-10T21:12:10.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.844+0000 7f18d4c00640 1 -- 172.21.15.16:0/2053172535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d00705e0 msgr2=0x7f18d00b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:10.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.844+0000 7f18d4c00640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d00705e0 0x7f18d00b3ee0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f18c000d720 tx=0x7f18c000dbe0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.844+0000 7f18d4c00640 1 -- 172.21.15.16:0/2053172535 shutdown_connections 2024-09-10T21:12:10.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.844+0000 7f18d4c00640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18ac0725e0 0x7f18ac074aa0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.844+0000 7f18d4c00640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18d015a270 0x7f18d00c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.844+0000 7f18d4c00640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18d00d06b0 0x7f18d014c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.844+0000 7f18d4c00640 1 --2- 172.21.15.16:0/2053172535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d00705e0 0x7f18d00b3ee0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:10.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.844+0000 7f18d4c00640 1 -- 172.21.15.16:0/2053172535 >> 172.21.15.16:0/2053172535 conn(0x7f18d00cfb20 msgr2=0x7f18d00c3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:10.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.844+0000 7f18d4c00640 1 -- 172.21.15.16:0/2053172535 shutdown_connections 2024-09-10T21:12:10.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:10.844+0000 7f18d4c00640 1 -- 172.21.15.16:0/2053172535 wait complete. 2024-09-10T21:12:10.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:12:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.008+0000 7f8f14800640 1 -- 172.21.15.16:0/636598204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f0c0716f0 msgr2=0x7f8f0c0af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.008+0000 7f8f14800640 1 --2- 172.21.15.16:0/636598204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f0c0716f0 0x7f8f0c0af580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009900 tx=0x7f8efc01c7c0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.008+0000 7f8f14800640 1 -- 172.21.15.16:0/636598204 shutdown_connections 2024-09-10T21:12:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.008+0000 7f8f14800640 1 --2- 172.21.15.16:0/636598204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f0c0b13c0 0x7f8f0c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.008+0000 7f8f14800640 1 --2- 172.21.15.16:0/636598204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f0c0716f0 0x7f8f0c0af580 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.008+0000 7f8f14800640 1 -- 172.21.15.16:0/636598204 >> 172.21.15.16:0/636598204 conn(0x7f8f0c09b130 msgr2=0x7f8f0c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.008+0000 7f8f14800640 1 -- 172.21.15.16:0/636598204 shutdown_connections 2024-09-10T21:12:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.008+0000 7f8f14800640 1 -- 172.21.15.16:0/636598204 wait complete. 2024-09-10T21:12:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f14800640 1 Processor -- start 2024-09-10T21:12:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f14800640 1 -- start start 2024-09-10T21:12:11.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f14800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f0c0716f0 0x7f8f0c1471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f14800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f0c0b13c0 0x7f8f0c155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f14800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f0c15f400 0x7f8f0c156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f14800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c168fc0 con 0x7f8f0c15f400 2024-09-10T21:12:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f14800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c168e40 con 0x7f8f0c0716f0 2024-09-10T21:12:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f14800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8f0c169140 con 0x7f8f0c0b13c0 2024-09-10T21:12:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f12800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f0c15f400 0x7f8f0c156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f12800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f0c15f400 0x7f8f0c156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34242/0 (socket says 172.21.15.16:34242) 2024-09-10T21:12:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f12800640 1 -- 172.21.15.16:0/2770307832 learned_addr learned my addr 172.21.15.16:0/2770307832 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f11e00640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f0c0716f0 0x7f8f0c1471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f11400640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f0c0b13c0 0x7f8f0c155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f12800640 1 -- 172.21.15.16:0/2770307832 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f0c0b13c0 msgr2=0x7f8f0c155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f12800640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f0c0b13c0 0x7f8f0c155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f12800640 1 -- 172.21.15.16:0/2770307832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f0c0716f0 msgr2=0x7f8f0c1471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f12800640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f0c0716f0 0x7f8f0c1471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f12800640 1 -- 172.21.15.16:0/2770307832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0c155ad0 con 0x7f8f0c15f400 2024-09-10T21:12:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f11e00640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f0c0716f0 0x7f8f0c1471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f12800640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f0c15f400 0x7f8f0c156d80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800e9b0 tx=0x7f8f0800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f06000640 1 -- 172.21.15.16:0/2770307832 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08004d30 con 0x7f8f0c15f400 2024-09-10T21:12:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f06000640 1 -- 172.21.15.16:0/2770307832 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f0800cb20 con 0x7f8f0c15f400 2024-09-10T21:12:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f14800640 1 -- 172.21.15.16:0/2770307832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0c155db0 con 0x7f8f0c15f400 2024-09-10T21:12:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f06000640 1 -- 172.21.15.16:0/2770307832 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08005020 con 0x7f8f0c15f400 2024-09-10T21:12:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f14800640 1 -- 172.21.15.16:0/2770307832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f0c1567c0 con 0x7f8f0c15f400 2024-09-10T21:12:11.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.012+0000 7f8f14800640 1 -- 172.21.15.16:0/2770307832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0c0b7110 con 0x7f8f0c15f400 2024-09-10T21:12:11.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.016+0000 7f8f06000640 1 -- 172.21.15.16:0/2770307832 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8f08022060 con 0x7f8f0c15f400 2024-09-10T21:12:11.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.016+0000 7f8f06000640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ef00725e0 0x7f8ef0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:11.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.016+0000 7f8f06000640 1 -- 172.21.15.16:0/2770307832 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5825+0+0 (secure 0 0 0) 0x7f8f080937e0 con 0x7f8f0c15f400 2024-09-10T21:12:11.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.016+0000 7f8f11e00640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ef00725e0 0x7f8ef0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:11.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.016+0000 7f8f11e00640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ef00725e0 0x7f8ef0074aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8f00001700 tx=0x7f8f00009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:11.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.020+0000 7f8f06000640 1 -- 172.21.15.16:0/2770307832 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8f08016030 con 0x7f8f0c15f400 2024-09-10T21:12:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.192+0000 7f8f14800640 1 -- 172.21.15.16:0/2770307832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f0c1562d0 con 0x7f8f0c15f400 2024-09-10T21:12:11.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.196+0000 7f8f06000640 1 -- 172.21.15.16:0/2770307832 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16870 (secure 0 0 0) 0x7f8f0808ae20 con 0x7f8f0c15f400 2024-09-10T21:12:11.202 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:11.202 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":34,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:10.584294+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:11:10.338788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T21:11:11.578459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.5","mappings":[{"from":4,"to":5}]},{"pgid":"5.1a","mappings":[{"from":4,"to":5}]},{"pgid":"5.26","mappings":[{"from":4,"to":5}]},{"pgid":"5.35","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:12:11.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.196+0000 7f8f14800640 1 -- 172.21.15.16:0/2770307832 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ef00725e0 msgr2=0x7f8ef0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.196+0000 7f8f14800640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ef00725e0 0x7f8ef0074aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8f00001700 tx=0x7f8f00009040 comp rx=0 tx=0).stop 2024-09-10T21:12:11.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.196+0000 7f8f14800640 1 -- 172.21.15.16:0/2770307832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f0c15f400 msgr2=0x7f8f0c156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.196+0000 7f8f14800640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f0c15f400 0x7f8f0c156d80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800e9b0 tx=0x7f8f0800ee70 comp rx=0 tx=0).stop 2024-09-10T21:12:11.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.196+0000 7f8f14800640 1 -- 172.21.15.16:0/2770307832 shutdown_connections 2024-09-10T21:12:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.196+0000 7f8f14800640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ef00725e0 0x7f8ef0074aa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.196+0000 7f8f14800640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f0c15f400 0x7f8f0c156d80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.196+0000 7f8f14800640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f0c0b13c0 0x7f8f0c155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.196+0000 7f8f14800640 1 --2- 172.21.15.16:0/2770307832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f0c0716f0 0x7f8f0c1471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.196+0000 7f8f14800640 1 -- 172.21.15.16:0/2770307832 >> 172.21.15.16:0/2770307832 conn(0x7f8f0c09b130 msgr2=0x7f8f0c0af510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.196+0000 7f8f14800640 1 -- 172.21.15.16:0/2770307832 shutdown_connections 2024-09-10T21:12:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.196+0000 7f8f14800640 1 -- 172.21.15.16:0/2770307832 wait complete. 2024-09-10T21:12:11.223 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:12:11.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.364+0000 7fd0aa800640 1 -- 172.21.15.16:0/4285048988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd088003c70 msgr2=0x7fd088004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.364+0000 7fd0aa800640 1 --2- 172.21.15.16:0/4285048988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd088003c70 0x7fd088004070 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4076250 tx=0x7fd09401d2a0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.364+0000 7fd0a3e00640 1 -- 172.21.15.16:0/4285048988 reap_dead start 2024-09-10T21:12:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.364+0000 7fd0aa800640 1 -- 172.21.15.16:0/4285048988 shutdown_connections 2024-09-10T21:12:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.364+0000 7fd0aa800640 1 -- 172.21.15.16:0/4285048988 >> 172.21.15.16:0/4285048988 conn(0x7fd0a40d03a0 msgr2=0x7fd0a40d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.364+0000 7fd0aa800640 1 -- 172.21.15.16:0/4285048988 shutdown_connections 2024-09-10T21:12:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.364+0000 7fd0aa800640 1 -- 172.21.15.16:0/4285048988 wait complete. 2024-09-10T21:12:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0aa800640 1 Processor -- start 2024-09-10T21:12:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0aa800640 1 -- start start 2024-09-10T21:12:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0aa800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd088003c70 0x7fd0a40cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0aa800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0a4159970 0x7fd0a40cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0aa800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0a40ce4b0 0x7fd0a415e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0aa800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a40b13d0 con 0x7fd088003c70 2024-09-10T21:12:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0aa800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a40b1250 con 0x7fd0a40ce4b0 2024-09-10T21:12:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0aa800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a40b1550 con 0x7fd0a4159970 2024-09-10T21:12:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0a3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd088003c70 0x7fd0a40cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0a3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd088003c70 0x7fd0a40cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34262/0 (socket says 172.21.15.16:34262) 2024-09-10T21:12:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0a3400640 1 -- 172.21.15.16:0/1906068963 learned_addr learned my addr 172.21.15.16:0/1906068963 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd09aa00640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0a4159970 0x7fd0a40cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0a3e00640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0a40ce4b0 0x7fd0a415e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0a3400640 1 -- 172.21.15.16:0/1906068963 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0a4159970 msgr2=0x7fd0a40cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0a3400640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0a4159970 0x7fd0a40cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0a3400640 1 -- 172.21.15.16:0/1906068963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0a40ce4b0 msgr2=0x7fd0a415e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0a3400640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0a40ce4b0 0x7fd0a415e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0a3400640 1 -- 172.21.15.16:0/1906068963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a41fa060 con 0x7fd088003c70 2024-09-10T21:12:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0a3400640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd088003c70 0x7fd0a40cfa60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd0940af040 tx=0x7fd0940adeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd09be00640 1 -- 172.21.15.16:0/1906068963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0940b1040 con 0x7fd088003c70 2024-09-10T21:12:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd09be00640 1 -- 172.21.15.16:0/1906068963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0940be070 con 0x7fd088003c70 2024-09-10T21:12:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd09be00640 1 -- 172.21.15.16:0/1906068963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094005530 con 0x7fd088003c70 2024-09-10T21:12:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0aa800640 1 -- 172.21.15.16:0/1906068963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a40ce9c0 con 0x7fd088003c70 2024-09-10T21:12:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0aa800640 1 -- 172.21.15.16:0/1906068963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a40cef80 con 0x7fd088003c70 2024-09-10T21:12:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.368+0000 7fd0aa800640 1 -- 172.21.15.16:0/1906068963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a40b4600 con 0x7fd088003c70 2024-09-10T21:12:11.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.372+0000 7fd09be00640 1 -- 172.21.15.16:0/1906068963 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd0940046d0 con 0x7fd088003c70 2024-09-10T21:12:11.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.372+0000 7fd09be00640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd080072610 0x7fd080074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:11.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.372+0000 7fd09be00640 1 -- 172.21.15.16:0/1906068963 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5825+0+0 (secure 0 0 0) 0x7fd09412ecf0 con 0x7fd088003c70 2024-09-10T21:12:11.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.372+0000 7fd09aa00640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd080072610 0x7fd080074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:11.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.372+0000 7fd09aa00640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd080072610 0x7fd080074ad0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40b2100 tx=0x7fd08c001600 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.376+0000 7fd09be00640 1 -- 172.21.15.16:0/1906068963 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd094016fe0 con 0x7fd088003c70 2024-09-10T21:12:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.584+0000 7fd0aa800640 1 -- 172.21.15.16:0/1906068963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0a406eff0 con 0x7fd088003c70 2024-09-10T21:12:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.584+0000 7fd09be00640 1 -- 172.21.15.16:0/1906068963 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5232 (secure 0 0 0) 0x7fd0940fa220 con 0x7fd088003c70 2024-09-10T21:12:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:11.593 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-10T21:11:16:232531+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/788814920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":788814920},{"type":"v1","addr":"172.21.15.16:6837","nonce":788814920}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2275087718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2275087718},{"type":"v1","addr":"172.21.15.16:6836","nonce":2275087718}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":134516,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3457135235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3457135235},{"type":"v1","addr":"172.21.15.52:6834","nonce":3457135235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:11:12.621848+0000","modified":"2024-09-10T21:11:15.636822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134519":{"gid":134519,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/4218881085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4218881085},{"type":"v1","addr":"172.21.15.52:6835","nonce":4218881085}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134519,"qdb_cluster":[134519]},"id":2}]} 2024-09-10T21:12:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-10T21:12:11.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.588+0000 7fd0aa800640 1 -- 172.21.15.16:0/1906068963 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd080072610 msgr2=0x7fd080074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.588+0000 7fd0aa800640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd080072610 0x7fd080074ad0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40b2100 tx=0x7fd08c001600 comp rx=0 tx=0).stop 2024-09-10T21:12:11.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.588+0000 7fd0aa800640 1 -- 172.21.15.16:0/1906068963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd088003c70 msgr2=0x7fd0a40cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.588+0000 7fd0aa800640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd088003c70 0x7fd0a40cfa60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd0940af040 tx=0x7fd0940adeb0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.588+0000 7fd0aa800640 1 -- 172.21.15.16:0/1906068963 shutdown_connections 2024-09-10T21:12:11.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.588+0000 7fd0aa800640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd080072610 0x7fd080074ad0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.588+0000 7fd0aa800640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0a40ce4b0 0x7fd0a415e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.588+0000 7fd0aa800640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0a4159970 0x7fd0a40cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.588+0000 7fd0aa800640 1 --2- 172.21.15.16:0/1906068963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd088003c70 0x7fd0a40cfa60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.588+0000 7fd0aa800640 1 -- 172.21.15.16:0/1906068963 >> 172.21.15.16:0/1906068963 conn(0x7fd0a40d03a0 msgr2=0x7fd0a406f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:11.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.588+0000 7fd0aa800640 1 -- 172.21.15.16:0/1906068963 shutdown_connections 2024-09-10T21:12:11.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.588+0000 7fd0aa800640 1 -- 172.21.15.16:0/1906068963 wait complete. 2024-09-10T21:12:11.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.756+0000 7f5dbba00640 1 -- 172.21.15.16:0/3358652554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db415f400 msgr2=0x7f5db4070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.756+0000 7f5dbba00640 1 --2- 172.21.15.16:0/3358652554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db415f400 0x7f5db4070fe0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5da40098f0 tx=0x7f5da401c7c0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.756+0000 7f5dbba00640 1 -- 172.21.15.16:0/3358652554 shutdown_connections 2024-09-10T21:12:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.756+0000 7f5dbba00640 1 --2- 172.21.15.16:0/3358652554 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5db40b0590 0x7f5db4071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.756+0000 7f5dbba00640 1 --2- 172.21.15.16:0/3358652554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db415f400 0x7f5db4070fe0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.756+0000 7f5dbba00640 1 -- 172.21.15.16:0/3358652554 >> 172.21.15.16:0/3358652554 conn(0x7f5db409b130 msgr2=0x7f5db40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.756+0000 7f5dbba00640 1 -- 172.21.15.16:0/3358652554 shutdown_connections 2024-09-10T21:12:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.756+0000 7f5dbba00640 1 -- 172.21.15.16:0/3358652554 wait complete. 2024-09-10T21:12:11.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dbba00640 1 Processor -- start 2024-09-10T21:12:11.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dbba00640 1 -- start start 2024-09-10T21:12:11.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dbba00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5db40b0590 0x7f5db40c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:11.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dbba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db40b13c0 0x7f5db40b6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dbba00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5db415f400 0x7f5db415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dbba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4168f60 con 0x7f5db40b13c0 2024-09-10T21:12:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dbba00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4168de0 con 0x7f5db415f400 2024-09-10T21:12:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dbba00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5db41690e0 con 0x7f5db40b0590 2024-09-10T21:12:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dabe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db40b13c0 0x7f5db40b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dabe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db40b13c0 0x7f5db40b6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34276/0 (socket says 172.21.15.16:34276) 2024-09-10T21:12:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dabe00640 1 -- 172.21.15.16:0/2017615163 learned_addr learned my addr 172.21.15.16:0/2017615163 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5db9000640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5db40b0590 0x7f5db40c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5db9a00640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5db415f400 0x7f5db415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dabe00640 1 -- 172.21.15.16:0/2017615163 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5db40b0590 msgr2=0x7f5db40c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dabe00640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5db40b0590 0x7f5db40c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dabe00640 1 -- 172.21.15.16:0/2017615163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5db415f400 msgr2=0x7f5db415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dabe00640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5db415f400 0x7f5db415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dabe00640 1 -- 172.21.15.16:0/2017615163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db41f5910 con 0x7f5db40b13c0 2024-09-10T21:12:11.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dabe00640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db40b13c0 0x7f5db40b6930 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00c500 tx=0x7f5d9c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:11.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5db1600640 1 -- 172.21.15.16:0/2017615163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c017070 con 0x7f5db40b13c0 2024-09-10T21:12:11.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5db1600640 1 -- 172.21.15.16:0/2017615163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d9c003600 con 0x7f5db40b13c0 2024-09-10T21:12:11.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5db1600640 1 -- 172.21.15.16:0/2017615163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c003a90 con 0x7f5db40b13c0 2024-09-10T21:12:11.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dbba00640 1 -- 172.21.15.16:0/2017615163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db41f5bf0 con 0x7f5db40b13c0 2024-09-10T21:12:11.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dbba00640 1 -- 172.21.15.16:0/2017615163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db41f61b0 con 0x7f5db40b13c0 2024-09-10T21:12:11.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.760+0000 7f5dbba00640 1 -- 172.21.15.16:0/2017615163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db40cfc80 con 0x7f5db40b13c0 2024-09-10T21:12:11.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.764+0000 7f5db1600640 1 -- 172.21.15.16:0/2017615163 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5d9c003c30 con 0x7f5db40b13c0 2024-09-10T21:12:11.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.764+0000 7f5db1600640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5d900725e0 0x7f5d90074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:11.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.764+0000 7f5db1600640 1 -- 172.21.15.16:0/2017615163 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5825+0+0 (secure 0 0 0) 0x7f5d9c0929c0 con 0x7f5db40b13c0 2024-09-10T21:12:11.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.764+0000 7f5db9000640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5d900725e0 0x7f5d90074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:11.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.764+0000 7f5db9000640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5d900725e0 0x7f5d90074aa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5da40af000 tx=0x7f5da401b000 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:11.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.768+0000 7f5db1600640 1 -- 172.21.15.16:0/2017615163 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5d9c05da80 con 0x7f5db40b13c0 2024-09-10T21:12:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.976+0000 7f5dbba00640 1 -- 172.21.15.16:0/2017615163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5db40ce990 con 0x7f5db40b13c0 2024-09-10T21:12:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.976+0000 7f5db1600640 1 -- 172.21.15.16:0/2017615163 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5232 (secure 0 0 0) 0x7f5d9c0624b0 con 0x7f5db40b13c0 2024-09-10T21:12:11.984 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:11.984 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-10T21:11:16:232531+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/788814920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":788814920},{"type":"v1","addr":"172.21.15.16:6837","nonce":788814920}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2275087718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2275087718},{"type":"v1","addr":"172.21.15.16:6836","nonce":2275087718}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":134516,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3457135235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3457135235},{"type":"v1","addr":"172.21.15.52:6834","nonce":3457135235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:11:12.621848+0000","modified":"2024-09-10T21:11:15.636822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134519":{"gid":134519,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/4218881085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4218881085},{"type":"v1","addr":"172.21.15.52:6835","nonce":4218881085}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134519,"qdb_cluster":[134519]},"id":2}]} 2024-09-10T21:12:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-10T21:12:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.980+0000 7f5dbba00640 1 -- 172.21.15.16:0/2017615163 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5d900725e0 msgr2=0x7f5d90074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.980+0000 7f5dbba00640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5d900725e0 0x7f5d90074aa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5da40af000 tx=0x7f5da401b000 comp rx=0 tx=0).stop 2024-09-10T21:12:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.980+0000 7f5dbba00640 1 -- 172.21.15.16:0/2017615163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db40b13c0 msgr2=0x7f5db40b6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.980+0000 7f5dbba00640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db40b13c0 0x7f5db40b6930 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00c500 tx=0x7f5d9c00c9c0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.980+0000 7f5dbba00640 1 -- 172.21.15.16:0/2017615163 shutdown_connections 2024-09-10T21:12:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.980+0000 7f5dbba00640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5d900725e0 0x7f5d90074aa0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.980+0000 7f5dbba00640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5db415f400 0x7f5db415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.980+0000 7f5dbba00640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db40b13c0 0x7f5db40b6930 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.980+0000 7f5dbba00640 1 --2- 172.21.15.16:0/2017615163 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5db40b0590 0x7f5db40c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.980+0000 7f5dbba00640 1 -- 172.21.15.16:0/2017615163 >> 172.21.15.16:0/2017615163 conn(0x7f5db409b130 msgr2=0x7f5db40b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.980+0000 7f5dbba00640 1 -- 172.21.15.16:0/2017615163 shutdown_connections 2024-09-10T21:12:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:11.980+0000 7f5dbba00640 1 -- 172.21.15.16:0/2017615163 wait complete. 2024-09-10T21:12:12.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:12:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.144+0000 7f36ba200640 1 -- 172.21.15.16:0/3407743445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b40b13c0 msgr2=0x7f36b4071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.144+0000 7f36ba200640 1 --2- 172.21.15.16:0/3407743445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b40b13c0 0x7f36b4071520 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f36b4074cc0 tx=0x7f369c01c7a0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.144+0000 7f36ba200640 1 -- 172.21.15.16:0/3407743445 shutdown_connections 2024-09-10T21:12:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.144+0000 7f36ba200640 1 --2- 172.21.15.16:0/3407743445 >> v2:172.21.15.52:3300/0 conn(0x7f36b40cfc80 0x7f36b40d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.144+0000 7f36ba200640 1 --2- 172.21.15.16:0/3407743445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b40b13c0 0x7f36b4071520 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.144+0000 7f36ba200640 1 --2- 172.21.15.16:0/3407743445 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f36b415f400 0x7f36b4070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.144+0000 7f36ba200640 1 -- 172.21.15.16:0/3407743445 >> 172.21.15.16:0/3407743445 conn(0x7f36b409b130 msgr2=0x7f36b40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.144+0000 7f36ba200640 1 -- 172.21.15.16:0/3407743445 shutdown_connections 2024-09-10T21:12:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.144+0000 7f36ba200640 1 -- 172.21.15.16:0/3407743445 wait complete. 2024-09-10T21:12:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36ba200640 1 Processor -- start 2024-09-10T21:12:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36ba200640 1 -- start start 2024-09-10T21:12:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36ba200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f36b40b13c0 0x7f36b41546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36ba200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36b40cfc80 0x7f36b40b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36ba200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b415f400 0x7f36b40b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36ba200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36b40afd90 con 0x7f36b415f400 2024-09-10T21:12:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36ba200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f36b40d1260 con 0x7f36b40cfc80 2024-09-10T21:12:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36ba200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f36b40aff10 con 0x7f36b40b13c0 2024-09-10T21:12:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36b3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b415f400 0x7f36b40b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36b3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b415f400 0x7f36b40b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60696/0 (socket says 172.21.15.16:60696) 2024-09-10T21:12:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36b3e00640 1 -- 172.21.15.16:0/870974160 learned_addr learned my addr 172.21.15.16:0/870974160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36b3400640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f36b40b13c0 0x7f36b41546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36aaa00640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36b40cfc80 0x7f36b40b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36b3e00640 1 -- 172.21.15.16:0/870974160 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f36b40b13c0 msgr2=0x7f36b41546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36b3e00640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f36b40b13c0 0x7f36b41546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36b3e00640 1 -- 172.21.15.16:0/870974160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36b40cfc80 msgr2=0x7f36b40b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36b3e00640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36b40cfc80 0x7f36b40b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36b3e00640 1 -- 172.21.15.16:0/870974160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b4155bc0 con 0x7f36b415f400 2024-09-10T21:12:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36b3400640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f36b40b13c0 0x7f36b41546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36b3e00640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b415f400 0x7f36b40b32d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00b9a0 tx=0x7f36ac00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36abe00640 1 -- 172.21.15.16:0/870974160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac019070 con 0x7f36b415f400 2024-09-10T21:12:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36abe00640 1 -- 172.21.15.16:0/870974160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36ac00ccb0 con 0x7f36b415f400 2024-09-10T21:12:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36ba200640 1 -- 172.21.15.16:0/870974160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b4155ea0 con 0x7f36b415f400 2024-09-10T21:12:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36abe00640 1 -- 172.21.15.16:0/870974160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac0146a0 con 0x7f36b415f400 2024-09-10T21:12:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36ba200640 1 -- 172.21.15.16:0/870974160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b4154c30 con 0x7f36b415f400 2024-09-10T21:12:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.148+0000 7f36ba200640 1 -- 172.21.15.16:0/870974160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b40d00d0 con 0x7f36b415f400 2024-09-10T21:12:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.152+0000 7f36abe00640 1 -- 172.21.15.16:0/870974160 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f36ac01d3b0 con 0x7f36b415f400 2024-09-10T21:12:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.152+0000 7f36abe00640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f36900725e0 0x7f3690074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.152+0000 7f36abe00640 1 -- 172.21.15.16:0/870974160 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5825+0+0 (secure 0 0 0) 0x7f36ac0932f0 con 0x7f36b415f400 2024-09-10T21:12:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.152+0000 7f36b3400640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f36900725e0 0x7f3690074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.152+0000 7f36b3400640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f36900725e0 0x7f3690074aa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009890 tx=0x7f36a4008040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.156+0000 7f36abe00640 1 -- 172.21.15.16:0/870974160 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f36ac05e2f0 con 0x7f36b415f400 2024-09-10T21:12:12.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.328+0000 7f36ba200640 1 -- 172.21.15.16:0/870974160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36b415d5a0 con 0x7f36b415f400 2024-09-10T21:12:12.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.332+0000 7f36abe00640 1 -- 172.21.15.16:0/870974160 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16870 (secure 0 0 0) 0x7f36ac062d20 con 0x7f36b415f400 2024-09-10T21:12:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:12.338 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":34,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:10.584294+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:11:10.338788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T21:11:11.578459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.5","mappings":[{"from":4,"to":5}]},{"pgid":"5.1a","mappings":[{"from":4,"to":5}]},{"pgid":"5.26","mappings":[{"from":4,"to":5}]},{"pgid":"5.35","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:12:12.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.332+0000 7f36ba200640 1 -- 172.21.15.16:0/870974160 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f36900725e0 msgr2=0x7f3690074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:12.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.332+0000 7f36ba200640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f36900725e0 0x7f3690074aa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009890 tx=0x7f36a4008040 comp rx=0 tx=0).stop 2024-09-10T21:12:12.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.332+0000 7f36ba200640 1 -- 172.21.15.16:0/870974160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b415f400 msgr2=0x7f36b40b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:12.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.332+0000 7f36ba200640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b415f400 0x7f36b40b32d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00b9a0 tx=0x7f36ac00be60 comp rx=0 tx=0).stop 2024-09-10T21:12:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.332+0000 7f36ba200640 1 -- 172.21.15.16:0/870974160 shutdown_connections 2024-09-10T21:12:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.332+0000 7f36ba200640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f36900725e0 0x7f3690074aa0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.332+0000 7f36ba200640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b415f400 0x7f36b40b32d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.332+0000 7f36ba200640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36b40cfc80 0x7f36b40b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.332+0000 7f36ba200640 1 --2- 172.21.15.16:0/870974160 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f36b40b13c0 0x7f36b41546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.332+0000 7f36ba200640 1 -- 172.21.15.16:0/870974160 >> 172.21.15.16:0/870974160 conn(0x7f36b409b130 msgr2=0x7f36b406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.332+0000 7f36ba200640 1 -- 172.21.15.16:0/870974160 shutdown_connections 2024-09-10T21:12:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.332+0000 7f36ba200640 1 -- 172.21.15.16:0/870974160 wait complete. 2024-09-10T21:12:12.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:12:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.500+0000 7f2e48c00640 1 -- 172.21.15.16:0/3669881722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e440b0c90 msgr2=0x7f2e440aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.500+0000 7f2e48c00640 1 --2- 172.21.15.16:0/3669881722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e440b0c90 0x7f2e440aea50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009840 tx=0x7f2e2c01c820 comp rx=0 tx=0).stop 2024-09-10T21:12:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.500+0000 7f2e48c00640 1 -- 172.21.15.16:0/3669881722 shutdown_connections 2024-09-10T21:12:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.500+0000 7f2e48c00640 1 --2- 172.21.15.16:0/3669881722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e440b0c90 0x7f2e440aea50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.500+0000 7f2e48c00640 1 -- 172.21.15.16:0/3669881722 >> 172.21.15.16:0/3669881722 conn(0x7f2e440d1d80 msgr2=0x7f2e440d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.500+0000 7f2e48c00640 1 -- 172.21.15.16:0/3669881722 shutdown_connections 2024-09-10T21:12:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.500+0000 7f2e48c00640 1 -- 172.21.15.16:0/3669881722 wait complete. 2024-09-10T21:12:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e48c00640 1 Processor -- start 2024-09-10T21:12:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e48c00640 1 -- start start 2024-09-10T21:12:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e48c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2e440b0c90 0x7f2e440c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e48c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e440d1860 0x7f2e44155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e48c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e44161720 0x7f2e440c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e48c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4416ca90 con 0x7f2e44161720 2024-09-10T21:12:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e48c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4416c910 con 0x7f2e440d1860 2024-09-10T21:12:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e48c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4416cc10 con 0x7f2e440b0c90 2024-09-10T21:12:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e42800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e44161720 0x7f2e440c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e42800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e44161720 0x7f2e440c7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60706/0 (socket says 172.21.15.16:60706) 2024-09-10T21:12:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e42800640 1 -- 172.21.15.16:0/2987147292 learned_addr learned my addr 172.21.15.16:0/2987147292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e41e00640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2e440b0c90 0x7f2e440c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e3be00640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e440d1860 0x7f2e44155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e42800640 1 -- 172.21.15.16:0/2987147292 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2e440b0c90 msgr2=0x7f2e440c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e42800640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2e440b0c90 0x7f2e440c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e42800640 1 -- 172.21.15.16:0/2987147292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e440d1860 msgr2=0x7f2e44155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e42800640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e440d1860 0x7f2e44155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e42800640 1 -- 172.21.15.16:0/2987147292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e440c8250 con 0x7f2e44161720 2024-09-10T21:12:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e42800640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e44161720 0x7f2e440c7c20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00cba0 tx=0x7f2e3c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e3a000640 1 -- 172.21.15.16:0/2987147292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c016020 con 0x7f2e44161720 2024-09-10T21:12:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e3a000640 1 -- 172.21.15.16:0/2987147292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e3c00ab60 con 0x7f2e44161720 2024-09-10T21:12:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e3a000640 1 -- 172.21.15.16:0/2987147292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c012420 con 0x7f2e44161720 2024-09-10T21:12:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e48c00640 1 -- 172.21.15.16:0/2987147292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e440c8530 con 0x7f2e44161720 2024-09-10T21:12:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e48c00640 1 -- 172.21.15.16:0/2987147292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e44156890 con 0x7f2e44161720 2024-09-10T21:12:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.504+0000 7f2e48c00640 1 -- 172.21.15.16:0/2987147292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e44156a90 con 0x7f2e44161720 2024-09-10T21:12:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.508+0000 7f2e3a000640 1 -- 172.21.15.16:0/2987147292 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2e3c0125c0 con 0x7f2e44161720 2024-09-10T21:12:12.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.508+0000 7f2e3a000640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2e200725e0 0x7f2e20074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:12.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.508+0000 7f2e3a000640 1 -- 172.21.15.16:0/2987147292 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5825+0+0 (secure 0 0 0) 0x7f2e3c0939e0 con 0x7f2e44161720 2024-09-10T21:12:12.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.508+0000 7f2e41e00640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2e200725e0 0x7f2e20074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.508+0000 7f2e41e00640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2e200725e0 0x7f2e20074aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e34001130 tx=0x7f2e34009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.512+0000 7f2e3a000640 1 -- 172.21.15.16:0/2987147292 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2e44156a90 con 0x7f2e44161720 2024-09-10T21:12:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.720+0000 7f2e48c00640 1 -- 172.21.15.16:0/2987147292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2e440cf9f0 con 0x7f2e44161720 2024-09-10T21:12:12.729 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-10T21:12:12.724+0000 7ff429400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:12:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.724+0000 7f2e3a000640 1 -- 172.21.15.16:0/2987147292 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f2e3c020020 con 0x7f2e44161720 2024-09-10T21:12:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:12:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.728+0000 7f2e48c00640 1 -- 172.21.15.16:0/2987147292 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2e200725e0 msgr2=0x7f2e20074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.728+0000 7f2e48c00640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2e200725e0 0x7f2e20074aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e34001130 tx=0x7f2e34009040 comp rx=0 tx=0).stop 2024-09-10T21:12:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.728+0000 7f2e48c00640 1 -- 172.21.15.16:0/2987147292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e44161720 msgr2=0x7f2e440c7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.728+0000 7f2e48c00640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e44161720 0x7f2e440c7c20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00cba0 tx=0x7f2e3c00cc90 comp rx=0 tx=0).stop 2024-09-10T21:12:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.728+0000 7f2e48c00640 1 -- 172.21.15.16:0/2987147292 shutdown_connections 2024-09-10T21:12:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.728+0000 7f2e48c00640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2e200725e0 0x7f2e20074aa0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.728+0000 7f2e48c00640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e44161720 0x7f2e440c7c20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.728+0000 7f2e48c00640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e440d1860 0x7f2e44155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.728+0000 7f2e48c00640 1 --2- 172.21.15.16:0/2987147292 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2e440b0c90 0x7f2e440c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.728+0000 7f2e48c00640 1 -- 172.21.15.16:0/2987147292 >> 172.21.15.16:0/2987147292 conn(0x7f2e440d1d80 msgr2=0x7f2e44070960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.728+0000 7f2e48c00640 1 -- 172.21.15.16:0/2987147292 shutdown_connections 2024-09-10T21:12:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.728+0000 7f2e48c00640 1 -- 172.21.15.16:0/2987147292 wait complete. 2024-09-10T21:12:12.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:12:12.803 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:12:12.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.896+0000 7feeeea00640 1 -- 172.21.15.16:0/876693651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80b0980 msgr2=0x7feee80ae940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:12.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.896+0000 7feeeea00640 1 --2- 172.21.15.16:0/876693651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80b0980 0x7feee80ae940 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7feee80b0db0 tx=0x7feed401c890 comp rx=0 tx=0).stop 2024-09-10T21:12:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.896+0000 7feeeea00640 1 -- 172.21.15.16:0/876693651 shutdown_connections 2024-09-10T21:12:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.896+0000 7feeeea00640 1 --2- 172.21.15.16:0/876693651 >> v2:172.21.15.52:3300/0 conn(0x7feee80cf1e0 0x7feee80aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.896+0000 7feeeea00640 1 --2- 172.21.15.16:0/876693651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80b0980 0x7feee80ae940 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.896+0000 7feeeea00640 1 --2- 172.21.15.16:0/876693651 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee80ce8c0 0x7feee80ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.896+0000 7feeeea00640 1 -- 172.21.15.16:0/876693651 >> 172.21.15.16:0/876693651 conn(0x7feee80d1c70 msgr2=0x7feee80d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.896+0000 7feeeea00640 1 -- 172.21.15.16:0/876693651 shutdown_connections 2024-09-10T21:12:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.896+0000 7feeeea00640 1 -- 172.21.15.16:0/876693651 wait complete. 2024-09-10T21:12:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.896+0000 7feeeea00640 1 Processor -- start 2024-09-10T21:12:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.896+0000 7feeeea00640 1 -- start start 2024-09-10T21:12:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeeea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee80b0980 0x7feee8155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeeea00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee80ce8c0 0x7feee80b7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeeea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80cf1e0 0x7feee815e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeeea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feee8160000 con 0x7feee80cf1e0 2024-09-10T21:12:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeeea00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feee815fe80 con 0x7feee80b0980 2024-09-10T21:12:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeeea00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7feee8160180 con 0x7feee80ce8c0 2024-09-10T21:12:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeee000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80cf1e0 0x7feee815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeee000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80cf1e0 0x7feee815e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60724/0 (socket says 172.21.15.16:60724) 2024-09-10T21:12:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeee000640 1 -- 172.21.15.16:0/2306701943 learned_addr learned my addr 172.21.15.16:0/2306701943 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeecc00640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee80ce8c0 0x7feee80b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeed600640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee80b0980 0x7feee8155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeee000640 1 -- 172.21.15.16:0/2306701943 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee80ce8c0 msgr2=0x7feee80b7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeee000640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee80ce8c0 0x7feee80b7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeee000640 1 -- 172.21.15.16:0/2306701943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee80b0980 msgr2=0x7feee8155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeee000640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee80b0980 0x7feee8155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeee000640 1 -- 172.21.15.16:0/2306701943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee815db50 con 0x7feee80cf1e0 2024-09-10T21:12:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeee000640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80cf1e0 0x7feee815e720 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feee000e970 tx=0x7feee000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feee5600640 1 -- 172.21.15.16:0/2306701943 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0004d30 con 0x7feee80cf1e0 2024-09-10T21:12:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feee5600640 1 -- 172.21.15.16:0/2306701943 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feee000cb20 con 0x7feee80cf1e0 2024-09-10T21:12:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeeea00640 1 -- 172.21.15.16:0/2306701943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee80b79b0 con 0x7feee80cf1e0 2024-09-10T21:12:12.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feee5600640 1 -- 172.21.15.16:0/2306701943 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0005020 con 0x7feee80cf1e0 2024-09-10T21:12:12.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeeea00640 1 -- 172.21.15.16:0/2306701943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee815d590 con 0x7feee80cf1e0 2024-09-10T21:12:12.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feeeea00640 1 -- 172.21.15.16:0/2306701943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee81606a0 con 0x7feee80cf1e0 2024-09-10T21:12:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.900+0000 7feee5600640 1 -- 172.21.15.16:0/2306701943 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feee000f040 con 0x7feee80cf1e0 2024-09-10T21:12:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.904+0000 7feee5600640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec80725e0 0x7feec8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.904+0000 7feee5600640 1 -- 172.21.15.16:0/2306701943 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5911+0+0 (secure 0 0 0) 0x7feee00939d0 con 0x7feee80cf1e0 2024-09-10T21:12:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.904+0000 7feeed600640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec80725e0 0x7feec8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.904+0000 7feeed600640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec80725e0 0x7feec8074aa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feedc00bb70 tx=0x7feedc00b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:12.908+0000 7feee5600640 1 -- 172.21.15.16:0/2306701943 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feee0016030 con 0x7feee80cf1e0 2024-09-10T21:12:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.116+0000 7feeeea00640 1 -- 172.21.15.16:0/2306701943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7feee80b2200 con 0x7feee80cf1e0 2024-09-10T21:12:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.728+0000 7feee5600640 1 -- 172.21.15.16:0/2306701943 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7feee0093020 con 0x7feee80cf1e0 2024-09-10T21:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.732+0000 7feeeea00640 1 -- 172.21.15.16:0/2306701943 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec80725e0 msgr2=0x7feec8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.732+0000 7feeeea00640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec80725e0 0x7feec8074aa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feedc00bb70 tx=0x7feedc00b2e0 comp rx=0 tx=0).stop 2024-09-10T21:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.732+0000 7feeeea00640 1 -- 172.21.15.16:0/2306701943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80cf1e0 msgr2=0x7feee815e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.732+0000 7feeeea00640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80cf1e0 0x7feee815e720 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feee000e970 tx=0x7feee000ee30 comp rx=0 tx=0).stop 2024-09-10T21:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.732+0000 7feeeea00640 1 -- 172.21.15.16:0/2306701943 shutdown_connections 2024-09-10T21:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.732+0000 7feeeea00640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec80725e0 0x7feec8074aa0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.732+0000 7feeeea00640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80cf1e0 0x7feee815e720 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.732+0000 7feeeea00640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee80ce8c0 0x7feee80b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.732+0000 7feeeea00640 1 --2- 172.21.15.16:0/2306701943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee80b0980 0x7feee8155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.732+0000 7feeeea00640 1 -- 172.21.15.16:0/2306701943 >> 172.21.15.16:0/2306701943 conn(0x7feee80d1c70 msgr2=0x7feee80b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.732+0000 7feeeea00640 1 -- 172.21.15.16:0/2306701943 shutdown_connections 2024-09-10T21:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.732+0000 7feeeea00640 1 -- 172.21.15.16:0/2306701943 wait complete. 2024-09-10T21:12:13.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:12:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.912+0000 7f5f4cc00640 1 -- 172.21.15.16:0/1949719615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f48074fd0 msgr2=0x7f5f480cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.912+0000 7f5f4cc00640 1 --2- 172.21.15.16:0/1949719615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f48074fd0 0x7f5f480cc350 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009bc0 tx=0x7f5f4001cba0 comp rx=0 tx=0).stop 2024-09-10T21:12:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.912+0000 7f5f4cc00640 1 -- 172.21.15.16:0/1949719615 shutdown_connections 2024-09-10T21:12:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.912+0000 7f5f4cc00640 1 --2- 172.21.15.16:0/1949719615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f48074fd0 0x7f5f480cc350 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.912+0000 7f5f4cc00640 1 -- 172.21.15.16:0/1949719615 >> 172.21.15.16:0/1949719615 conn(0x7f5f48073470 msgr2=0x7f5f48074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.912+0000 7f5f4cc00640 1 -- 172.21.15.16:0/1949719615 shutdown_connections 2024-09-10T21:12:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.912+0000 7f5f4cc00640 1 -- 172.21.15.16:0/1949719615 wait complete. 2024-09-10T21:12:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.912+0000 7f5f4cc00640 1 Processor -- start 2024-09-10T21:12:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.912+0000 7f5f4cc00640 1 -- start start 2024-09-10T21:12:13.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f4cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f48074fd0 0x7f5f480c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:13.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f4cc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f480d0a30 0x7f5f48165840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:13.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f4cc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f4815bbe0 0x7f5f481fb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:13.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f4cc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4816a630 con 0x7f5f48074fd0 2024-09-10T21:12:13.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f4cc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4816a4b0 con 0x7f5f480d0a30 2024-09-10T21:12:13.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f4cc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5f4816a7b0 con 0x7f5f4815bbe0 2024-09-10T21:12:13.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f48074fd0 0x7f5f480c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:13.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f48074fd0 0x7f5f480c6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60754/0 (socket says 172.21.15.16:60754) 2024-09-10T21:12:13.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f45e00640 1 -- 172.21.15.16:0/2847558858 learned_addr learned my addr 172.21.15.16:0/2847558858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:13.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f46800640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f4815bbe0 0x7f5f481fb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:13.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f3fe00640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f480d0a30 0x7f5f48165840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:13.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f45e00640 1 -- 172.21.15.16:0/2847558858 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f4815bbe0 msgr2=0x7f5f481fb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:13.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f45e00640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f4815bbe0 0x7f5f481fb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:13.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f45e00640 1 -- 172.21.15.16:0/2847558858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f480d0a30 msgr2=0x7f5f48165840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:13.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f45e00640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f480d0a30 0x7f5f48165840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:13.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f45e00640 1 -- 172.21.15.16:0/2847558858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f48165ee0 con 0x7f5f48074fd0 2024-09-10T21:12:13.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f45e00640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f48074fd0 0x7f5f480c6520 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800c450 tx=0x7f5f3800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:13.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f3e000640 1 -- 172.21.15.16:0/2847558858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38016020 con 0x7f5f48074fd0 2024-09-10T21:12:13.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f3e000640 1 -- 172.21.15.16:0/2847558858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f38012920 con 0x7f5f48074fd0 2024-09-10T21:12:13.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f4cc00640 1 -- 172.21.15.16:0/2847558858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f481661c0 con 0x7f5f48074fd0 2024-09-10T21:12:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f3e000640 1 -- 172.21.15.16:0/2847558858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38012d50 con 0x7f5f48074fd0 2024-09-10T21:12:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f4cc00640 1 -- 172.21.15.16:0/2847558858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f481664a0 con 0x7f5f48074fd0 2024-09-10T21:12:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f4cc00640 1 -- 172.21.15.16:0/2847558858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f480af390 con 0x7f5f48074fd0 2024-09-10T21:12:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.916+0000 7f5f3e000640 1 -- 172.21.15.16:0/2847558858 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5f3801b400 con 0x7f5f48074fd0 2024-09-10T21:12:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.920+0000 7f5f3e000640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f24072610 0x7f5f24074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.920+0000 7f5f3e000640 1 -- 172.21.15.16:0/2847558858 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5911+0+0 (secure 0 0 0) 0x7f5f380935e0 con 0x7f5f48074fd0 2024-09-10T21:12:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.920+0000 7f5f3fe00640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f24072610 0x7f5f24074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.920+0000 7f5f3fe00640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f24072610 0x7f5f24074ad0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5f48157b10 tx=0x7f5f30009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:13.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:13.924+0000 7f5f3e000640 1 -- 172.21.15.16:0/2847558858 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f38020020 con 0x7f5f48074fd0 2024-09-10T21:12:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.100+0000 7f5f4cc00640 1 -- 172.21.15.16:0/2847558858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5f4806df10 con 0x7f5f48074fd0 2024-09-10T21:12:14.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.732+0000 7f5f3e000640 1 -- 172.21.15.16:0/2847558858 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) ==== 181+0+0 (secure 0 0 0) 0x7f5f3805eb50 con 0x7f5f48074fd0 2024-09-10T21:12:14.739 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:12:14.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.736+0000 7f5f4cc00640 1 -- 172.21.15.16:0/2847558858 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f24072610 msgr2=0x7f5f24074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:14.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.736+0000 7f5f4cc00640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f24072610 0x7f5f24074ad0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5f48157b10 tx=0x7f5f30009040 comp rx=0 tx=0).stop 2024-09-10T21:12:14.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.736+0000 7f5f4cc00640 1 -- 172.21.15.16:0/2847558858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f48074fd0 msgr2=0x7f5f480c6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:14.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.736+0000 7f5f4cc00640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f48074fd0 0x7f5f480c6520 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800c450 tx=0x7f5f3800c910 comp rx=0 tx=0).stop 2024-09-10T21:12:14.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.736+0000 7f5f4cc00640 1 -- 172.21.15.16:0/2847558858 shutdown_connections 2024-09-10T21:12:14.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.736+0000 7f5f4cc00640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f24072610 0x7f5f24074ad0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:14.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.736+0000 7f5f4cc00640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f4815bbe0 0x7f5f481fb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:14.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.736+0000 7f5f4cc00640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f480d0a30 0x7f5f48165840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:14.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.736+0000 7f5f4cc00640 1 --2- 172.21.15.16:0/2847558858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f48074fd0 0x7f5f480c6520 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:14.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.736+0000 7f5f4cc00640 1 -- 172.21.15.16:0/2847558858 >> 172.21.15.16:0/2847558858 conn(0x7f5f48073470 msgr2=0x7f5f4815b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:14.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.736+0000 7f5f4cc00640 1 -- 172.21.15.16:0/2847558858 shutdown_connections 2024-09-10T21:12:14.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.736+0000 7f5f4cc00640 1 -- 172.21.15.16:0/2847558858 wait complete. 2024-09-10T21:12:14.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:12:14.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.912+0000 7f39b5200640 1 -- 172.21.15.16:0/1507306368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b00cfc40 msgr2=0x7f39b015ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.912+0000 7f39b5200640 1 --2- 172.21.15.16:0/1507306368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b00cfc40 0x7f39b015ff50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f39b0073ae0 tx=0x7f399801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:12:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.912+0000 7f39b5200640 1 -- 172.21.15.16:0/1507306368 shutdown_connections 2024-09-10T21:12:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.912+0000 7f39b5200640 1 --2- 172.21.15.16:0/1507306368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b00cfc40 0x7f39b015ff50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.912+0000 7f39b5200640 1 --2- 172.21.15.16:0/1507306368 >> v2:172.21.15.52:3300/0 conn(0x7f39b00ce7c0 0x7f39b00ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.912+0000 7f39b5200640 1 -- 172.21.15.16:0/1507306368 >> 172.21.15.16:0/1507306368 conn(0x7f39b00d1e60 msgr2=0x7f39b00d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.912+0000 7f39b5200640 1 -- 172.21.15.16:0/1507306368 shutdown_connections 2024-09-10T21:12:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.912+0000 7f39b5200640 1 -- 172.21.15.16:0/1507306368 wait complete. 2024-09-10T21:12:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39b5200640 1 Processor -- start 2024-09-10T21:12:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39b5200640 1 -- start start 2024-09-10T21:12:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39b5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b00ce7c0 0x7f39b014de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39b5200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39b00cfc40 0x7f39b00b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39b5200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f39b0160490 0x7f39b00b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39b5200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01693a0 con 0x7f39b00ce7c0 2024-09-10T21:12:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39b5200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f39b0169220 con 0x7f39b00cfc40 2024-09-10T21:12:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39b5200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f39b0169520 con 0x7f39b0160490 2024-09-10T21:12:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39ade00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b00ce7c0 0x7f39b014de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39ade00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b00ce7c0 0x7f39b014de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60772/0 (socket says 172.21.15.16:60772) 2024-09-10T21:12:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39ade00640 1 -- 172.21.15.16:0/2851975704 learned_addr learned my addr 172.21.15.16:0/2851975704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39a7e00640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39b00cfc40 0x7f39b00b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39ae800640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f39b0160490 0x7f39b00b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39ade00640 1 -- 172.21.15.16:0/2851975704 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f39b0160490 msgr2=0x7f39b00b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39ade00640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f39b0160490 0x7f39b00b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39ade00640 1 -- 172.21.15.16:0/2851975704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39b00cfc40 msgr2=0x7f39b00b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39ade00640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39b00cfc40 0x7f39b00b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39ade00640 1 -- 172.21.15.16:0/2851975704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b00b4fa0 con 0x7f39b00ce7c0 2024-09-10T21:12:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39ade00640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b00ce7c0 0x7f39b014de60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f39a000d9b0 tx=0x7f39a000de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39a6000640 1 -- 172.21.15.16:0/2851975704 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0017070 con 0x7f39b00ce7c0 2024-09-10T21:12:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39a6000640 1 -- 172.21.15.16:0/2851975704 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39a0009cc0 con 0x7f39b00ce7c0 2024-09-10T21:12:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39b5200640 1 -- 172.21.15.16:0/2851975704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b014e5b0 con 0x7f39b00ce7c0 2024-09-10T21:12:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39a6000640 1 -- 172.21.15.16:0/2851975704 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0005280 con 0x7f39b00ce7c0 2024-09-10T21:12:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.916+0000 7f39b5200640 1 -- 172.21.15.16:0/2851975704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b00b66e0 con 0x7f39b00ce7c0 2024-09-10T21:12:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.920+0000 7f39b5200640 1 -- 172.21.15.16:0/2851975704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b00ceba0 con 0x7f39b00ce7c0 2024-09-10T21:12:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.920+0000 7f39a6000640 1 -- 172.21.15.16:0/2851975704 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f39a0005420 con 0x7f39b00ce7c0 2024-09-10T21:12:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.920+0000 7f39a6000640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f398c072610 0x7f398c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.920+0000 7f39a6000640 1 -- 172.21.15.16:0/2851975704 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5473+0+0 (secure 0 0 0) 0x7f39a00933e0 con 0x7f39b00ce7c0 2024-09-10T21:12:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.920+0000 7f39a7e00640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f398c072610 0x7f398c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.920+0000 7f39a7e00640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f398c072610 0x7f398c074ad0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f39980af000 tx=0x7f399801b100 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:14.924+0000 7f39a6000640 1 -- 172.21.15.16:0/2851975704 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f39b00b4fa0 con 0x7f39b00ce7c0 2024-09-10T21:12:15.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.100+0000 7f39b5200640 1 -- 172.21.15.16:0/2851975704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f39b00b0480 con 0x7f39b00ce7c0 2024-09-10T21:12:15.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.736+0000 7f39a6000640 1 -- 172.21.15.16:0/2851975704 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) ==== 169+0+0 (secure 0 0 0) 0x7f39a0067220 con 0x7f39b00ce7c0 2024-09-10T21:12:15.744 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-10T21:12:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.740+0000 7f39b5200640 1 -- 172.21.15.16:0/2851975704 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f398c072610 msgr2=0x7f398c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.740+0000 7f39b5200640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f398c072610 0x7f398c074ad0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f39980af000 tx=0x7f399801b100 comp rx=0 tx=0).stop 2024-09-10T21:12:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.740+0000 7f39b5200640 1 -- 172.21.15.16:0/2851975704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b00ce7c0 msgr2=0x7f39b014de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.740+0000 7f39b5200640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b00ce7c0 0x7f39b014de60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f39a000d9b0 tx=0x7f39a000de70 comp rx=0 tx=0).stop 2024-09-10T21:12:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.744+0000 7f39b5200640 1 -- 172.21.15.16:0/2851975704 shutdown_connections 2024-09-10T21:12:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.744+0000 7f39b5200640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f398c072610 0x7f398c074ad0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.744+0000 7f39b5200640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f39b0160490 0x7f39b00b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.744+0000 7f39b5200640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39b00cfc40 0x7f39b00b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.744+0000 7f39b5200640 1 --2- 172.21.15.16:0/2851975704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b00ce7c0 0x7f39b014de60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.744+0000 7f39b5200640 1 -- 172.21.15.16:0/2851975704 >> 172.21.15.16:0/2851975704 conn(0x7f39b00d1e60 msgr2=0x7f39b00b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.744+0000 7f39b5200640 1 -- 172.21.15.16:0/2851975704 shutdown_connections 2024-09-10T21:12:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.744+0000 7f39b5200640 1 -- 172.21.15.16:0/2851975704 wait complete. 2024-09-10T21:12:15.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:12:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.932+0000 7f0826a00640 1 -- 172.21.15.16:0/1886352152 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08100a6d50 con 0x7f08280ce450 2024-09-10T21:12:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.932+0000 7f082e200640 1 -- 172.21.15.16:0/1886352152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08280ce450 msgr2=0x7f08280ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.932+0000 7f082e200640 1 --2- 172.21.15.16:0/1886352152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08280ce450 0x7f08280ce8a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f08280cd5b0 tx=0x7f081001c770 comp rx=0 tx=0).stop 2024-09-10T21:12:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.932+0000 7f082e200640 1 -- 172.21.15.16:0/1886352152 shutdown_connections 2024-09-10T21:12:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.932+0000 7f082e200640 1 --2- 172.21.15.16:0/1886352152 >> v2:172.21.15.52:3300/0 conn(0x7f08280ca170 0x7f08280ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.932+0000 7f082e200640 1 --2- 172.21.15.16:0/1886352152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08280ce450 0x7f08280ce8a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.932+0000 7f082e200640 1 -- 172.21.15.16:0/1886352152 >> 172.21.15.16:0/1886352152 conn(0x7f08280cfb20 msgr2=0x7f08280cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.932+0000 7f082e200640 1 -- 172.21.15.16:0/1886352152 shutdown_connections 2024-09-10T21:12:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.932+0000 7f082e200640 1 -- 172.21.15.16:0/1886352152 wait complete. 2024-09-10T21:12:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f082e200640 1 Processor -- start 2024-09-10T21:12:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f082e200640 1 -- start start 2024-09-10T21:12:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f082e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08280ca170 0x7f0828152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f082e200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08280ce450 0x7f082814f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f082e200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f082815a270 0x7f0828150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f082e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0828072b80 con 0x7f08280ca170 2024-09-10T21:12:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f082e200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0828072a00 con 0x7f08280ce450 2024-09-10T21:12:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f082e200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0828072d00 con 0x7f082815a270 2024-09-10T21:12:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f0827400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08280ca170 0x7f0828152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f081ea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08280ce450 0x7f082814f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f0827400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08280ca170 0x7f0828152710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60798/0 (socket says 172.21.15.16:60798) 2024-09-10T21:12:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f0827400640 1 -- 172.21.15.16:0/3784693196 learned_addr learned my addr 172.21.15.16:0/3784693196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f0827400640 1 -- 172.21.15.16:0/3784693196 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f082815a270 msgr2=0x7f0828150c30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:12:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f0827400640 1 --2- 172.21.15.16:0/3784693196 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f082815a270 0x7f0828150c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f0827400640 1 -- 172.21.15.16:0/3784693196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08280ce450 msgr2=0x7f082814f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f0827400640 1 --2- 172.21.15.16:0/3784693196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08280ce450 0x7f082814f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.936+0000 7f0827400640 1 -- 172.21.15.16:0/3784693196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0828156510 con 0x7f08280ca170 2024-09-10T21:12:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.940+0000 7f0827400640 1 --2- 172.21.15.16:0/3784693196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08280ca170 0x7f0828152710 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f081800c430 tx=0x7f081800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.940+0000 7f081fe00640 1 -- 172.21.15.16:0/3784693196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818011790 con 0x7f08280ca170 2024-09-10T21:12:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.940+0000 7f081fe00640 1 -- 172.21.15.16:0/3784693196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0818017070 con 0x7f08280ca170 2024-09-10T21:12:15.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.940+0000 7f081fe00640 1 -- 172.21.15.16:0/3784693196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818011ce0 con 0x7f08280ca170 2024-09-10T21:12:15.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.940+0000 7f082e200640 1 -- 172.21.15.16:0/3784693196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08281567f0 con 0x7f08280ca170 2024-09-10T21:12:15.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.940+0000 7f082e200640 1 -- 172.21.15.16:0/3784693196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08281f7870 con 0x7f08280ca170 2024-09-10T21:12:15.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.944+0000 7f082e200640 1 -- 172.21.15.16:0/3784693196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08000052a0 con 0x7f08280ca170 2024-09-10T21:12:15.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.944+0000 7f081fe00640 1 -- 172.21.15.16:0/3784693196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f081801c460 con 0x7f08280ca170 2024-09-10T21:12:15.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.944+0000 7f081fe00640 1 --2- 172.21.15.16:0/3784693196 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0804072630 0x7f0804074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:15.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.944+0000 7f081fe00640 1 -- 172.21.15.16:0/3784693196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f0818092bc0 con 0x7f08280ca170 2024-09-10T21:12:15.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.944+0000 7f081ea00640 1 --2- 172.21.15.16:0/3784693196 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0804072630 0x7f0804074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.948+0000 7f081ea00640 1 --2- 172.21.15.16:0/3784693196 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0804072630 0x7f0804074af0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f08100af000 tx=0x7f081001afb0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:15.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:15.952+0000 7f081fe00640 1 -- 172.21.15.16:0/3784693196 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f081805e640 con 0x7f08280ca170 2024-09-10T21:12:16.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.124+0000 7f082e200640 1 -- 172.21.15.16:0/3784693196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f0800005590 con 0x7f08280ca170 2024-09-10T21:12:16.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.128+0000 7f081fe00640 1 -- 172.21.15.16:0/3784693196 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v31) ==== 71+0+0 (secure 0 0 0) 0x7f081805df40 con 0x7f08280ca170 2024-09-10T21:12:16.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.128+0000 7f082e200640 1 -- 172.21.15.16:0/3784693196 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0804072630 msgr2=0x7f0804074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:16.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.128+0000 7f082e200640 1 --2- 172.21.15.16:0/3784693196 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0804072630 0x7f0804074af0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f08100af000 tx=0x7f081001afb0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.128+0000 7f082e200640 1 -- 172.21.15.16:0/3784693196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08280ca170 msgr2=0x7f0828152710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:16.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.128+0000 7f082e200640 1 --2- 172.21.15.16:0/3784693196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08280ca170 0x7f0828152710 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f081800c430 tx=0x7f081800c8f0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.128+0000 7f082e200640 1 -- 172.21.15.16:0/3784693196 shutdown_connections 2024-09-10T21:12:16.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.128+0000 7f082e200640 1 --2- 172.21.15.16:0/3784693196 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0804072630 0x7f0804074af0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.128+0000 7f082e200640 1 --2- 172.21.15.16:0/3784693196 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f082815a270 0x7f0828150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.128+0000 7f082e200640 1 --2- 172.21.15.16:0/3784693196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08280ce450 0x7f082814f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.128+0000 7f082e200640 1 --2- 172.21.15.16:0/3784693196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08280ca170 0x7f0828152710 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.128+0000 7f082e200640 1 -- 172.21.15.16:0/3784693196 >> 172.21.15.16:0/3784693196 conn(0x7f08280cfb20 msgr2=0x7f08280734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:16.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.128+0000 7f082e200640 1 -- 172.21.15.16:0/3784693196 shutdown_connections 2024-09-10T21:12:16.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.128+0000 7f082e200640 1 -- 172.21.15.16:0/3784693196 wait complete. 2024-09-10T21:12:16.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2024-09-10T21:12:16.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.296+0000 7f2d82a00640 1 -- 172.21.15.16:0/1079848225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d60003a10 msgr2=0x7f2d60003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:16.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.296+0000 7f2d82a00640 1 --2- 172.21.15.16:0/1079848225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d60003a10 0x7f2d60003e30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009b50 tx=0x7f2d6c01d290 comp rx=0 tx=0).stop 2024-09-10T21:12:16.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.296+0000 7f2d80a00640 1 -- 172.21.15.16:0/1079848225 reap_dead start 2024-09-10T21:12:16.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.296+0000 7f2d82a00640 1 -- 172.21.15.16:0/1079848225 shutdown_connections 2024-09-10T21:12:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.296+0000 7f2d82a00640 1 -- 172.21.15.16:0/1079848225 >> 172.21.15.16:0/1079848225 conn(0x7f2d7c09b130 msgr2=0x7f2d7c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.296+0000 7f2d82a00640 1 -- 172.21.15.16:0/1079848225 shutdown_connections 2024-09-10T21:12:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.296+0000 7f2d82a00640 1 -- 172.21.15.16:0/1079848225 wait complete. 2024-09-10T21:12:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d82a00640 1 Processor -- start 2024-09-10T21:12:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d82a00640 1 -- start start 2024-09-10T21:12:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d82a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d60003a10 0x7f2d7c1f9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d82a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7c162640 0x7f2d7c0b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d82a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d7c0b3080 0x7f2d7c160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d82a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c0718b0 con 0x7f2d60003a10 2024-09-10T21:12:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d82a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c071730 con 0x7f2d7c162640 2024-09-10T21:12:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d82a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7c1673d0 con 0x7f2d7c0b3080 2024-09-10T21:12:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d60003a10 0x7f2d7c1f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d60003a10 0x7f2d7c1f9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60820/0 (socket says 172.21.15.16:60820) 2024-09-10T21:12:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d7be00640 1 -- 172.21.15.16:0/3519253046 learned_addr learned my addr 172.21.15.16:0/3519253046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d80a00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d7c0b3080 0x7f2d7c160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d73e00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7c162640 0x7f2d7c0b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d7be00640 1 -- 172.21.15.16:0/3519253046 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d7c0b3080 msgr2=0x7f2d7c160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d7be00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d7c0b3080 0x7f2d7c160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d7be00640 1 -- 172.21.15.16:0/3519253046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7c162640 msgr2=0x7f2d7c0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d7be00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7c162640 0x7f2d7c0b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d7be00640 1 -- 172.21.15.16:0/3519253046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7c1f96c0 con 0x7f2d60003a10 2024-09-10T21:12:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d73e00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7c162640 0x7f2d7c0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d80a00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d7c0b3080 0x7f2d7c160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d7be00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d60003a10 0x7f2d7c1f9180 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009aa0 tx=0x7f2d6c0048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d78c00640 1 -- 172.21.15.16:0/3519253046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c004cc0 con 0x7f2d60003a10 2024-09-10T21:12:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d78c00640 1 -- 172.21.15.16:0/3519253046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d6c005030 con 0x7f2d60003a10 2024-09-10T21:12:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d78c00640 1 -- 172.21.15.16:0/3519253046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c004e60 con 0x7f2d60003a10 2024-09-10T21:12:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d82a00640 1 -- 172.21.15.16:0/3519253046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7c0b2ab0 con 0x7f2d60003a10 2024-09-10T21:12:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.300+0000 7f2d82a00640 1 -- 172.21.15.16:0/3519253046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7c15fe90 con 0x7f2d60003a10 2024-09-10T21:12:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.304+0000 7f2d82a00640 1 -- 172.21.15.16:0/3519253046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d7c070fe0 con 0x7f2d60003a10 2024-09-10T21:12:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.304+0000 7f2d78c00640 1 -- 172.21.15.16:0/3519253046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2d6c0b9070 con 0x7f2d60003a10 2024-09-10T21:12:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.304+0000 7f2d78c00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d58072610 0x7f2d58074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.304+0000 7f2d78c00640 1 -- 172.21.15.16:0/3519253046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f2d6c12d5f0 con 0x7f2d60003a10 2024-09-10T21:12:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.304+0000 7f2d73e00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d58072610 0x7f2d58074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:16.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.304+0000 7f2d73e00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d58072610 0x7f2d58074ad0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2d64005fd0 tx=0x7f2d64005ee0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:16.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.308+0000 7f2d78c00640 1 -- 172.21.15.16:0/3519253046 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2d6c016fa0 con 0x7f2d60003a10 2024-09-10T21:12:16.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.544+0000 7f2d82a00640 1 -- 172.21.15.16:0/3519253046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) -- 0x7f2d7c0cd3e0 con 0x7f2d60003a10 2024-09-10T21:12:16.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.740+0000 7f2d78c00640 1 -- 172.21.15.16:0/3519253046 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v105) ==== 165+0+0 (secure 0 0 0) 0x7f2d6c0f8f30 con 0x7f2d60003a10 2024-09-10T21:12:16.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.740+0000 7f2d82a00640 1 -- 172.21.15.16:0/3519253046 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d58072610 msgr2=0x7f2d58074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:16.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.740+0000 7f2d82a00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d58072610 0x7f2d58074ad0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2d64005fd0 tx=0x7f2d64005ee0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.740+0000 7f2d82a00640 1 -- 172.21.15.16:0/3519253046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d60003a10 msgr2=0x7f2d7c1f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:16.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.740+0000 7f2d82a00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d60003a10 0x7f2d7c1f9180 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009aa0 tx=0x7f2d6c0048b0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.740+0000 7f2d82a00640 1 -- 172.21.15.16:0/3519253046 shutdown_connections 2024-09-10T21:12:16.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.740+0000 7f2d82a00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d58072610 0x7f2d58074ad0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.740+0000 7f2d82a00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d7c0b3080 0x7f2d7c160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.740+0000 7f2d82a00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7c162640 0x7f2d7c0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.740+0000 7f2d82a00640 1 --2- 172.21.15.16:0/3519253046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d60003a10 0x7f2d7c1f9180 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.740+0000 7f2d82a00640 1 -- 172.21.15.16:0/3519253046 >> 172.21.15.16:0/3519253046 conn(0x7f2d7c09b130 msgr2=0x7f2d7c15e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:16.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.744+0000 7f2d82a00640 1 -- 172.21.15.16:0/3519253046 shutdown_connections 2024-09-10T21:12:16.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.744+0000 7f2d82a00640 1 -- 172.21.15.16:0/3519253046 wait complete. 2024-09-10T21:12:16.769 INFO:tasks.cephfs_test_runner:Mostly readdir acquires caps faster than the mds recalls, so the cap ... ok 2024-09-10T21:12:16.769 INFO:tasks.cephfs_test_runner:Starting test: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) that a client is blocklisted when its encoded session metadata exceeds the 2024-09-10T21:12:16.769 INFO:tasks.cephfs_test_runner:test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-10T21:12:16.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:12:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 -- 172.21.15.16:0/3608363149 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f321015bc70 msgr2=0x7f321006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 --2- 172.21.15.16:0/3608363149 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f321015bc70 0x7f321006ffe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f31f8009840 tx=0x7f31f801c7c0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 -- 172.21.15.16:0/3608363149 shutdown_connections 2024-09-10T21:12:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 --2- 172.21.15.16:0/3608363149 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f321015bc70 0x7f321006ffe0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 --2- 172.21.15.16:0/3608363149 >> v2:172.21.15.52:3300/0 conn(0x7f3210154f90 0x7f3210072d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 -- 172.21.15.16:0/3608363149 >> 172.21.15.16:0/3608363149 conn(0x7f32100cfb20 msgr2=0x7f32100cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 -- 172.21.15.16:0/3608363149 shutdown_connections 2024-09-10T21:12:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 -- 172.21.15.16:0/3608363149 wait complete. 2024-09-10T21:12:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 Processor -- start 2024-09-10T21:12:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 -- start start 2024-09-10T21:12:16.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210070520 0x7f32100b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3210154f90 0x7f32100c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f321015bc70 0x7f32100c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3210167b50 con 0x7f3210070520 2024-09-10T21:12:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f32101679d0 con 0x7f321015bc70 2024-09-10T21:12:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3216600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3210167cd0 con 0x7f3210154f90 2024-09-10T21:12:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f320f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210070520 0x7f32100b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f320f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210070520 0x7f32100b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60830/0 (socket says 172.21.15.16:60830) 2024-09-10T21:12:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f320f400640 1 -- 172.21.15.16:0/4025036826 learned_addr learned my addr 172.21.15.16:0/4025036826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f3206a00640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3210154f90 0x7f32100c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f320fe00640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f321015bc70 0x7f32100c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f320f400640 1 -- 172.21.15.16:0/4025036826 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3210154f90 msgr2=0x7f32100c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f320f400640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3210154f90 0x7f32100c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f320f400640 1 -- 172.21.15.16:0/4025036826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f321015bc70 msgr2=0x7f32100c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f320f400640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f321015bc70 0x7f32100c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f320f400640 1 -- 172.21.15.16:0/4025036826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32100b39a0 con 0x7f3210070520 2024-09-10T21:12:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.912+0000 7f320fe00640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f321015bc70 0x7f32100c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.916+0000 7f320f400640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210070520 0x7f32100b4d20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f320000e9d0 tx=0x7f320000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.916+0000 7f3207e00640 1 -- 172.21.15.16:0/4025036826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320000cde0 con 0x7f3210070520 2024-09-10T21:12:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.916+0000 7f3207e00640 1 -- 172.21.15.16:0/4025036826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3200004020 con 0x7f3210070520 2024-09-10T21:12:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.916+0000 7f3207e00640 1 -- 172.21.15.16:0/4025036826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200004450 con 0x7f3210070520 2024-09-10T21:12:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.916+0000 7f3216600640 1 -- 172.21.15.16:0/4025036826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32100b3c80 con 0x7f3210070520 2024-09-10T21:12:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.916+0000 7f3216600640 1 -- 172.21.15.16:0/4025036826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32100b4240 con 0x7f3210070520 2024-09-10T21:12:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.916+0000 7f3216600640 1 -- 172.21.15.16:0/4025036826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3210072d40 con 0x7f3210070520 2024-09-10T21:12:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.916+0000 7f3207e00640 1 -- 172.21.15.16:0/4025036826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3200003600 con 0x7f3210070520 2024-09-10T21:12:16.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.916+0000 7f3207e00640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f31ec0725e0 0x7f31ec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:16.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.916+0000 7f3207e00640 1 -- 172.21.15.16:0/4025036826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f3200093960 con 0x7f3210070520 2024-09-10T21:12:16.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.916+0000 7f3206a00640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f31ec0725e0 0x7f31ec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:16.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.920+0000 7f3206a00640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f31ec0725e0 0x7f31ec074aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f31f80af000 tx=0x7f31f801b060 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:16.920+0000 7f3207e00640 1 -- 172.21.15.16:0/4025036826 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f320005f2b0 con 0x7f3210070520 2024-09-10T21:12:17.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.096+0000 7f3216600640 1 -- 172.21.15.16:0/4025036826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3210050fd0 con 0x7f3210070520 2024-09-10T21:12:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.096+0000 7f3207e00640 1 -- 172.21.15.16:0/4025036826 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+12984 (secure 0 0 0) 0x7f320005ebb0 con 0x7f3210070520 2024-09-10T21:12:17.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:17.104 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":37,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:15.736878+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6835/4218881085":"2024-09-11T21:12:12.727023+0000","172.21.15.52:6833/4218881085":"2024-09-11T21:12:12.727023+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:12:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.100+0000 7f3216600640 1 -- 172.21.15.16:0/4025036826 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f31ec0725e0 msgr2=0x7f31ec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.100+0000 7f3216600640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f31ec0725e0 0x7f31ec074aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f31f80af000 tx=0x7f31f801b060 comp rx=0 tx=0).stop 2024-09-10T21:12:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.100+0000 7f3216600640 1 -- 172.21.15.16:0/4025036826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210070520 msgr2=0x7f32100b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.100+0000 7f3216600640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210070520 0x7f32100b4d20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f320000e9d0 tx=0x7f320000ee90 comp rx=0 tx=0).stop 2024-09-10T21:12:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.100+0000 7f3216600640 1 -- 172.21.15.16:0/4025036826 shutdown_connections 2024-09-10T21:12:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.100+0000 7f3216600640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f31ec0725e0 0x7f31ec074aa0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.100+0000 7f3216600640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f321015bc70 0x7f32100c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.100+0000 7f3216600640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3210154f90 0x7f32100c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.100+0000 7f3216600640 1 --2- 172.21.15.16:0/4025036826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3210070520 0x7f32100b4d20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.100+0000 7f3216600640 1 -- 172.21.15.16:0/4025036826 >> 172.21.15.16:0/4025036826 conn(0x7f32100cfb20 msgr2=0x7f321015b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.100+0000 7f3216600640 1 -- 172.21.15.16:0/4025036826 shutdown_connections 2024-09-10T21:12:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.100+0000 7f3216600640 1 -- 172.21.15.16:0/4025036826 wait complete. 2024-09-10T21:12:17.126 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:10:17.493770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:12:17.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:12:17.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.268+0000 7fb65d000640 1 -- 172.21.15.16:0/1052775603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6580c8fd0 msgr2=0x7fb658164a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.268+0000 7fb65d000640 1 --2- 172.21.15.16:0/1052775603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6580c8fd0 0x7fb658164a30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb658071eb0 tx=0x7fb65001ccf0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 -- 172.21.15.16:0/1052775603 shutdown_connections 2024-09-10T21:12:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 --2- 172.21.15.16:0/1052775603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6580c8fd0 0x7fb658164a30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 --2- 172.21.15.16:0/1052775603 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6580c8650 0x7fb6580c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 --2- 172.21.15.16:0/1052775603 >> v2:172.21.15.52:3300/0 conn(0x7fb65815f420 0x7fb6580c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 -- 172.21.15.16:0/1052775603 >> 172.21.15.16:0/1052775603 conn(0x7fb6580d0390 msgr2=0x7fb6580d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 -- 172.21.15.16:0/1052775603 shutdown_connections 2024-09-10T21:12:17.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 -- 172.21.15.16:0/1052775603 wait complete. 2024-09-10T21:12:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 Processor -- start 2024-09-10T21:12:17.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 -- start start 2024-09-10T21:12:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6580c8650 0x7fb6580b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6580c8fd0 0x7fb6581573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb65815f420 0x7fb65815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb658167b20 con 0x7fb6580c8650 2024-09-10T21:12:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581679a0 con 0x7fb65815f420 2024-09-10T21:12:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb65d000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb658167ca0 con 0x7fb6580c8fd0 2024-09-10T21:12:17.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb655e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6580c8650 0x7fb6580b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb655e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6580c8650 0x7fb6580b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60842/0 (socket says 172.21.15.16:60842) 2024-09-10T21:12:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb655e00640 1 -- 172.21.15.16:0/2416801468 learned_addr learned my addr 172.21.15.16:0/2416801468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb656800640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb65815f420 0x7fb65815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:17.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb64fe00640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6580c8fd0 0x7fb6581573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb655e00640 1 -- 172.21.15.16:0/2416801468 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6580c8fd0 msgr2=0x7fb6581573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb655e00640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6580c8fd0 0x7fb6581573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb655e00640 1 -- 172.21.15.16:0/2416801468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb65815f420 msgr2=0x7fb65815c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb655e00640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb65815f420 0x7fb65815c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb655e00640 1 -- 172.21.15.16:0/2416801468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6580cd7b0 con 0x7fb6580c8650 2024-09-10T21:12:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.272+0000 7fb655e00640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6580c8650 0x7fb6580b4900 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb64800ca20 tx=0x7fb64800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:17.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.276+0000 7fb64e000640 1 -- 172.21.15.16:0/2416801468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648007c70 con 0x7fb6580c8650 2024-09-10T21:12:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.276+0000 7fb64e000640 1 -- 172.21.15.16:0/2416801468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb64800f450 con 0x7fb6580c8650 2024-09-10T21:12:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.276+0000 7fb64e000640 1 -- 172.21.15.16:0/2416801468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64800f8a0 con 0x7fb6580c8650 2024-09-10T21:12:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.276+0000 7fb65d000640 1 -- 172.21.15.16:0/2416801468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6580cf110 con 0x7fb6580c8650 2024-09-10T21:12:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.276+0000 7fb65d000640 1 -- 172.21.15.16:0/2416801468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65809ae40 con 0x7fb6580c8650 2024-09-10T21:12:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.276+0000 7fb65d000640 1 -- 172.21.15.16:0/2416801468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6580c8a70 con 0x7fb6580c8650 2024-09-10T21:12:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.276+0000 7fb64e000640 1 -- 172.21.15.16:0/2416801468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb64800fa40 con 0x7fb6580c8650 2024-09-10T21:12:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.276+0000 7fb64e000640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6340725e0 0x7fb634074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.276+0000 7fb64e000640 1 -- 172.21.15.16:0/2416801468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fb648093590 con 0x7fb6580c8650 2024-09-10T21:12:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.276+0000 7fb64fe00640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6340725e0 0x7fb634074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.280+0000 7fb64fe00640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6340725e0 0x7fb634074aa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb65815cc50 tx=0x7fb640009110 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:17.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.280+0000 7fb64e000640 1 -- 172.21.15.16:0/2416801468 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb648014030 con 0x7fb6580c8650 2024-09-10T21:12:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.460+0000 7fb65d000640 1 -- 172.21.15.16:0/2416801468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb65815e650 con 0x7fb6580c8650 2024-09-10T21:12:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.460+0000 7fb64e000640 1 -- 172.21.15.16:0/2416801468 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7fb64805e910 con 0x7fb6580c8650 2024-09-10T21:12:17.468 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-10T21:12:17.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.464+0000 7fb65d000640 1 -- 172.21.15.16:0/2416801468 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6340725e0 msgr2=0x7fb634074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:17.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.464+0000 7fb65d000640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6340725e0 0x7fb634074aa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb65815cc50 tx=0x7fb640009110 comp rx=0 tx=0).stop 2024-09-10T21:12:17.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.464+0000 7fb65d000640 1 -- 172.21.15.16:0/2416801468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6580c8650 msgr2=0x7fb6580b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:17.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.464+0000 7fb65d000640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6580c8650 0x7fb6580b4900 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb64800ca20 tx=0x7fb64800cee0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.464+0000 7fb65d000640 1 -- 172.21.15.16:0/2416801468 shutdown_connections 2024-09-10T21:12:17.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.464+0000 7fb65d000640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6340725e0 0x7fb634074aa0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.464+0000 7fb65d000640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb65815f420 0x7fb65815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.464+0000 7fb65d000640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6580c8fd0 0x7fb6581573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.464+0000 7fb65d000640 1 --2- 172.21.15.16:0/2416801468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6580c8650 0x7fb6580b4900 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.464+0000 7fb65d000640 1 -- 172.21.15.16:0/2416801468 >> 172.21.15.16:0/2416801468 conn(0x7fb6580d0390 msgr2=0x7fb6580cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.464+0000 7fb65d000640 1 -- 172.21.15.16:0/2416801468 shutdown_connections 2024-09-10T21:12:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.464+0000 7fb65d000640 1 -- 172.21.15.16:0/2416801468 wait complete. 2024-09-10T21:12:17.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2024-09-10T21:12:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 -- 172.21.15.16:0/2144363714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854004da0 msgr2=0x7f8854005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 --2- 172.21.15.16:0/2144363714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854004da0 0x7f8854005220 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f88700c3370 tx=0x7f886801d0f0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f886fe00640 1 -- 172.21.15.16:0/2144363714 reap_dead start 2024-09-10T21:12:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 -- 172.21.15.16:0/2144363714 shutdown_connections 2024-09-10T21:12:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 -- 172.21.15.16:0/2144363714 >> 172.21.15.16:0/2144363714 conn(0x7f88700cfb20 msgr2=0x7f88700cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 -- 172.21.15.16:0/2144363714 shutdown_connections 2024-09-10T21:12:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 -- 172.21.15.16:0/2144363714 wait complete. 2024-09-10T21:12:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 Processor -- start 2024-09-10T21:12:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 -- start start 2024-09-10T21:12:17.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854004da0 0x7f88700c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:17.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f88701f89f0 0x7f88700b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:17.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88700b2950 0x7f887015c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:17.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88701638d0 con 0x7f8854004da0 2024-09-10T21:12:17.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8870163750 con 0x7f88700b2950 2024-09-10T21:12:17.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8876400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8870163a50 con 0x7f88701f89f0 2024-09-10T21:12:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f886f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854004da0 0x7f88700c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f886f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854004da0 0x7f88700c2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60856/0 (socket says 172.21.15.16:60856) 2024-09-10T21:12:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f886f400640 1 -- 172.21.15.16:0/1521813307 learned_addr learned my addr 172.21.15.16:0/1521813307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f886fe00640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88700b2950 0x7f887015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f8867e00640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f88701f89f0 0x7f88700b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.632+0000 7f886f400640 1 -- 172.21.15.16:0/1521813307 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f88701f89f0 msgr2=0x7f88700b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f886f400640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f88701f89f0 0x7f88700b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f886f400640 1 -- 172.21.15.16:0/1521813307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88700b2950 msgr2=0x7f887015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f886f400640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88700b2950 0x7f887015c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f886f400640 1 -- 172.21.15.16:0/1521813307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88700b5970 con 0x7f8854004da0 2024-09-10T21:12:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f886f400640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854004da0 0x7f88700c2b20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8860006fd0 tx=0x7f8860003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:17.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f8867400640 1 -- 172.21.15.16:0/1521813307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860015070 con 0x7f8854004da0 2024-09-10T21:12:17.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f8867400640 1 -- 172.21.15.16:0/1521813307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f886000f9e0 con 0x7f8854004da0 2024-09-10T21:12:17.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f8876400640 1 -- 172.21.15.16:0/1521813307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88700c8260 con 0x7f8854004da0 2024-09-10T21:12:17.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f8867400640 1 -- 172.21.15.16:0/1521813307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886000fe30 con 0x7f8854004da0 2024-09-10T21:12:17.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f8876400640 1 -- 172.21.15.16:0/1521813307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88700c8820 con 0x7f8854004da0 2024-09-10T21:12:17.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f8876400640 1 -- 172.21.15.16:0/1521813307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8870072d40 con 0x7f8854004da0 2024-09-10T21:12:17.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f8867400640 1 -- 172.21.15.16:0/1521813307 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8860011430 con 0x7f8854004da0 2024-09-10T21:12:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f8867400640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f884c072610 0x7f884c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f8867400640 1 -- 172.21.15.16:0/1521813307 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f8860093d80 con 0x7f8854004da0 2024-09-10T21:12:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.636+0000 7f8867e00640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f884c072610 0x7f884c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.640+0000 7f8867e00640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f884c072610 0x7f884c074ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f88700c5d00 tx=0x7f8858006b80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:17.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.640+0000 7f8867400640 1 -- 172.21.15.16:0/1521813307 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f886005f750 con 0x7f8854004da0 2024-09-10T21:12:17.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:17.884+0000 7f8876400640 1 -- 172.21.15.16:0/1521813307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) -- 0x7f887015cff0 con 0x7f8854004da0 2024-09-10T21:12:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.748+0000 7f8867400640 1 -- 172.21.15.16:0/1521813307 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v107) ==== 165+0+0 (secure 0 0 0) 0x7f886005f050 con 0x7f8854004da0 2024-09-10T21:12:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.748+0000 7f8876400640 1 -- 172.21.15.16:0/1521813307 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f884c072610 msgr2=0x7f884c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.748+0000 7f8876400640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f884c072610 0x7f884c074ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f88700c5d00 tx=0x7f8858006b80 comp rx=0 tx=0).stop 2024-09-10T21:12:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.748+0000 7f8876400640 1 -- 172.21.15.16:0/1521813307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854004da0 msgr2=0x7f88700c2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.748+0000 7f8876400640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854004da0 0x7f88700c2b20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8860006fd0 tx=0x7f8860003a90 comp rx=0 tx=0).stop 2024-09-10T21:12:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.748+0000 7f8876400640 1 -- 172.21.15.16:0/1521813307 shutdown_connections 2024-09-10T21:12:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.748+0000 7f8876400640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f884c072610 0x7f884c074ad0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.748+0000 7f8876400640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88700b2950 0x7f887015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.748+0000 7f8876400640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f88701f89f0 0x7f88700b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.748+0000 7f8876400640 1 --2- 172.21.15.16:0/1521813307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8854004da0 0x7f88700c2b20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.748+0000 7f8876400640 1 -- 172.21.15.16:0/1521813307 >> 172.21.15.16:0/1521813307 conn(0x7f88700cfb20 msgr2=0x7f887015d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.748+0000 7f8876400640 1 -- 172.21.15.16:0/1521813307 shutdown_connections 2024-09-10T21:12:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.752+0000 7f8876400640 1 -- 172.21.15.16:0/1521813307 wait complete. 2024-09-10T21:12:18.782 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:12:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 -- 172.21.15.16:0/471459620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0051d0 msgr2=0x7f090c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 --2- 172.21.15.16:0/471459620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0051d0 0x7f090c00b9d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0924070900 tx=0x7f092001cfe0 comp rx=0 tx=0).stop 2024-09-10T21:12:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092a800640 1 -- 172.21.15.16:0/471459620 reap_dead start 2024-09-10T21:12:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 -- 172.21.15.16:0/471459620 shutdown_connections 2024-09-10T21:12:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 -- 172.21.15.16:0/471459620 >> 172.21.15.16:0/471459620 conn(0x7f09240cfb20 msgr2=0x7f09240cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 -- 172.21.15.16:0/471459620 shutdown_connections 2024-09-10T21:12:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 -- 172.21.15.16:0/471459620 wait complete. 2024-09-10T21:12:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 Processor -- start 2024-09-10T21:12:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 -- start start 2024-09-10T21:12:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0051d0 0x7f09241e3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09240cdd60 0x7f09240c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0924149b30 0x7f0924149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09240ca8c0 con 0x7f090c0051d0 2024-09-10T21:12:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09240ca740 con 0x7f0924149b30 2024-09-10T21:12:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.928+0000 7f092c800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f09240caa40 con 0x7f09240cdd60 2024-09-10T21:12:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f0929e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0051d0 0x7f09241e3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f0929e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0051d0 0x7f09241e3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60864/0 (socket says 172.21.15.16:60864) 2024-09-10T21:12:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f0929e00640 1 -- 172.21.15.16:0/1645023699 learned_addr learned my addr 172.21.15.16:0/1645023699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f092a800640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0924149b30 0x7f0924149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f0929400640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09240cdd60 0x7f09240c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f0929e00640 1 -- 172.21.15.16:0/1645023699 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09240cdd60 msgr2=0x7f09240c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f0929e00640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09240cdd60 0x7f09240c5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f0929e00640 1 -- 172.21.15.16:0/1645023699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0924149b30 msgr2=0x7f0924149fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f0929e00640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0924149b30 0x7f0924149fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f0929e00640 1 -- 172.21.15.16:0/1645023699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09240ce240 con 0x7f090c0051d0 2024-09-10T21:12:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f0929e00640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0051d0 0x7f09241e3b20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0918006580 tx=0x7f09180039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f091e000640 1 -- 172.21.15.16:0/1645023699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918015070 con 0x7f090c0051d0 2024-09-10T21:12:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f091e000640 1 -- 172.21.15.16:0/1645023699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0918011840 con 0x7f090c0051d0 2024-09-10T21:12:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f091e000640 1 -- 172.21.15.16:0/1645023699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918011c90 con 0x7f090c0051d0 2024-09-10T21:12:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f092c800640 1 -- 172.21.15.16:0/1645023699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09240ce520 con 0x7f090c0051d0 2024-09-10T21:12:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f092c800640 1 -- 172.21.15.16:0/1645023699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09240ceae0 con 0x7f090c0051d0 2024-09-10T21:12:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f092c800640 1 -- 172.21.15.16:0/1645023699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09240735d0 con 0x7f090c0051d0 2024-09-10T21:12:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.932+0000 7f091e000640 1 -- 172.21.15.16:0/1645023699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f09180103c0 con 0x7f090c0051d0 2024-09-10T21:12:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.936+0000 7f091e000640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09080725e0 0x7f0908074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.936+0000 7f091e000640 1 -- 172.21.15.16:0/1645023699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f0918093f00 con 0x7f090c0051d0 2024-09-10T21:12:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.936+0000 7f0929400640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09080725e0 0x7f0908074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.936+0000 7f0929400640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09080725e0 0x7f0908074aa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f09240b2de0 tx=0x7f09140091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:18.940+0000 7f091e000640 1 -- 172.21.15.16:0/1645023699 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f091805f280 con 0x7f090c0051d0 2024-09-10T21:12:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f092c800640 1 -- 172.21.15.16:0/1645023699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f092406f400 con 0x7f090c0051d0 2024-09-10T21:12:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f091e000640 1 -- 172.21.15.16:0/1645023699 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+1998 (secure 0 0 0) 0x7f0918063cb0 con 0x7f090c0051d0 2024-09-10T21:12:19.123 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:12:19.123 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:12:19.123 INFO:teuthology.orchestra.run.smithi016.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:12:19.124 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:12:19.124 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4226.427518 2024-09-10T21:12:19.124 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5426.620519 2024-09-10T21:12:19.124 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4255.210203 2024-09-10T21:12:19.124 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4255.734871 2024-09-10T21:12:19.124 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4192.833008 2024-09-10T21:12:19.125 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4136.940759 2024-09-10T21:12:19.125 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4217.384701 2024-09-10T21:12:19.125 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5220.337972 2024-09-10T21:12:19.125 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-10T21:12:19.125 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:12:19.125 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-10T21:12:19.125 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-10T21:12:19.125 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:12:19.126 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:12:19.126 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-10T21:12:19.126 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:12:19.126 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:12:19.126 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:12:19.126 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-10T21:12:19.126 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-10T21:12:19.126 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-10T21:12:19.127 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:12:19.127 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:12:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f092c800640 1 -- 172.21.15.16:0/1645023699 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09080725e0 msgr2=0x7f0908074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f092c800640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09080725e0 0x7f0908074aa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f09240b2de0 tx=0x7f09140091e0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f092c800640 1 -- 172.21.15.16:0/1645023699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0051d0 msgr2=0x7f09241e3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:19.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f092c800640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0051d0 0x7f09241e3b20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0918006580 tx=0x7f09180039d0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f092c800640 1 -- 172.21.15.16:0/1645023699 shutdown_connections 2024-09-10T21:12:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f092c800640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09080725e0 0x7f0908074aa0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f092c800640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0924149b30 0x7f0924149fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f092c800640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09240cdd60 0x7f09240c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f092c800640 1 --2- 172.21.15.16:0/1645023699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0051d0 0x7f09241e3b20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f092c800640 1 -- 172.21.15.16:0/1645023699 >> 172.21.15.16:0/1645023699 conn(0x7f09240cfb20 msgr2=0x7f09240b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f092c800640 1 -- 172.21.15.16:0/1645023699 shutdown_connections 2024-09-10T21:12:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.116+0000 7f092c800640 1 -- 172.21.15.16:0/1645023699 wait complete. 2024-09-10T21:12:19.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:12:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.288+0000 7f0b8ec00640 1 -- 172.21.15.16:0/142869594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c003a60 msgr2=0x7f0b6c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.288+0000 7f0b8ec00640 1 --2- 172.21.15.16:0/142869594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c003a60 0x7f0b6c003e80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800c2e0 tx=0x7f0b7801cf20 comp rx=0 tx=0).stop 2024-09-10T21:12:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.288+0000 7f0b8ec00640 1 -- 172.21.15.16:0/142869594 shutdown_connections 2024-09-10T21:12:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.288+0000 7f0b8ec00640 1 --2- 172.21.15.16:0/142869594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b6c003a60 0x7f0b6c003e80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.288+0000 7f0b8ec00640 1 -- 172.21.15.16:0/142869594 >> 172.21.15.16:0/142869594 conn(0x7f0b880cd6a0 msgr2=0x7f0b880cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.288+0000 7f0b8ec00640 1 -- 172.21.15.16:0/142869594 shutdown_connections 2024-09-10T21:12:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.288+0000 7f0b8ec00640 1 -- 172.21.15.16:0/142869594 wait complete. 2024-09-10T21:12:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8ec00640 1 Processor -- start 2024-09-10T21:12:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8ec00640 1 -- start start 2024-09-10T21:12:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8ec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b6c003a60 0x7f0b880c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8ec00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0b880c63a0 0x7f0b8815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8ec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b880c71e0 0x7f0b8815c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8ec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8806d570 con 0x7f0b880c71e0 2024-09-10T21:12:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8ec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8806d3f0 con 0x7f0b6c003a60 2024-09-10T21:12:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8ec00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8806d6f0 con 0x7f0b880c63a0 2024-09-10T21:12:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b880c71e0 0x7f0b8815c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b880c71e0 0x7f0b8815c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60880/0 (socket says 172.21.15.16:60880) 2024-09-10T21:12:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8cc00640 1 -- 172.21.15.16:0/959574956 learned_addr learned my addr 172.21.15.16:0/959574956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b87e00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b6c003a60 0x7f0b880c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b7fe00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0b880c63a0 0x7f0b8815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8cc00640 1 -- 172.21.15.16:0/959574956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0b880c63a0 msgr2=0x7f0b8815c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8cc00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0b880c63a0 0x7f0b8815c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8cc00640 1 -- 172.21.15.16:0/959574956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b6c003a60 msgr2=0x7f0b880c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8cc00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b6c003a60 0x7f0b880c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8cc00640 1 -- 172.21.15.16:0/959574956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8815df30 con 0x7f0b880c71e0 2024-09-10T21:12:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8cc00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b880c71e0 0x7f0b8815c700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0b880d0bd0 tx=0x7f0b80010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b84c00640 1 -- 172.21.15.16:0/959574956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80003980 con 0x7f0b880c71e0 2024-09-10T21:12:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b84c00640 1 -- 172.21.15.16:0/959574956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b80015070 con 0x7f0b880c71e0 2024-09-10T21:12:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8ec00640 1 -- 172.21.15.16:0/959574956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b881f5ca0 con 0x7f0b880c71e0 2024-09-10T21:12:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b84c00640 1 -- 172.21.15.16:0/959574956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80004030 con 0x7f0b880c71e0 2024-09-10T21:12:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8ec00640 1 -- 172.21.15.16:0/959574956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b881f6260 con 0x7f0b880c71e0 2024-09-10T21:12:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.292+0000 7f0b8ec00640 1 -- 172.21.15.16:0/959574956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b88071770 con 0x7f0b880c71e0 2024-09-10T21:12:19.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.296+0000 7f0b84c00640 1 -- 172.21.15.16:0/959574956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0b800041d0 con 0x7f0b880c71e0 2024-09-10T21:12:19.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.296+0000 7f0b84c00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0b64072610 0x7f0b64074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:19.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.296+0000 7f0b84c00640 1 -- 172.21.15.16:0/959574956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f0b80093960 con 0x7f0b880c71e0 2024-09-10T21:12:19.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.296+0000 7f0b87e00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0b64072610 0x7f0b64074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:19.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.296+0000 7f0b87e00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0b64072610 0x7f0b64074ad0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0b780ab000 tx=0x7f0b78019d60 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:19.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.300+0000 7f0b84c00640 1 -- 172.21.15.16:0/959574956 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0b8815df30 con 0x7f0b880c71e0 2024-09-10T21:12:19.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.472+0000 7f0b8ec00640 1 -- 172.21.15.16:0/959574956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0b880c41a0 con 0x7f0b880c71e0 2024-09-10T21:12:19.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.476+0000 7f0b84c00640 1 -- 172.21.15.16:0/959574956 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f0b8005ec50 con 0x7f0b880c71e0 2024-09-10T21:12:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.480+0000 7f0b8ec00640 1 -- 172.21.15.16:0/959574956 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0b64072610 msgr2=0x7f0b64074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.480+0000 7f0b8ec00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0b64072610 0x7f0b64074ad0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0b780ab000 tx=0x7f0b78019d60 comp rx=0 tx=0).stop 2024-09-10T21:12:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.480+0000 7f0b8ec00640 1 -- 172.21.15.16:0/959574956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b880c71e0 msgr2=0x7f0b8815c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.480+0000 7f0b8ec00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b880c71e0 0x7f0b8815c700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0b880d0bd0 tx=0x7f0b80010f40 comp rx=0 tx=0).stop 2024-09-10T21:12:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.480+0000 7f0b8ec00640 1 -- 172.21.15.16:0/959574956 shutdown_connections 2024-09-10T21:12:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.480+0000 7f0b8ec00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0b64072610 0x7f0b64074ad0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.480+0000 7f0b8ec00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b880c71e0 0x7f0b8815c700 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.480+0000 7f0b8ec00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0b880c63a0 0x7f0b8815c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.480+0000 7f0b8ec00640 1 --2- 172.21.15.16:0/959574956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b6c003a60 0x7f0b880c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.480+0000 7f0b8ec00640 1 -- 172.21.15.16:0/959574956 >> 172.21.15.16:0/959574956 conn(0x7f0b880cd6a0 msgr2=0x7f0b880cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.480+0000 7f0b8ec00640 1 -- 172.21.15.16:0/959574956 shutdown_connections 2024-09-10T21:12:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.480+0000 7f0b8ec00640 1 -- 172.21.15.16:0/959574956 wait complete. 2024-09-10T21:12:19.506 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:12:19.506 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:12:19.600 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:12:19.704 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:12:19.704 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:12:19.704 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:19.715 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:12:19.715 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:12:19.715 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:19.725 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:12:19.725 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:12:19.725 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:19.765 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:12:19.765 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:12:19.765 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:19.782 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:12:19.783 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:12:19.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 -- 172.21.15.16:0/734657719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec815e420 msgr2=0x7fcec80cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 --2- 172.21.15.16:0/734657719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec815e420 0x7fcec80cc480 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009900 tx=0x7fcebc01c850 comp rx=0 tx=0).stop 2024-09-10T21:12:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 -- 172.21.15.16:0/734657719 shutdown_connections 2024-09-10T21:12:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 --2- 172.21.15.16:0/734657719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec815e420 0x7fcec80cc480 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 -- 172.21.15.16:0/734657719 >> 172.21.15.16:0/734657719 conn(0x7fcec80d03a0 msgr2=0x7fcec80d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 -- 172.21.15.16:0/734657719 shutdown_connections 2024-09-10T21:12:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 -- 172.21.15.16:0/734657719 wait complete. 2024-09-10T21:12:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 Processor -- start 2024-09-10T21:12:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 -- start start 2024-09-10T21:12:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8068c10 0x7fcec80c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec80cc9c0 0x7fcec8098cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcec815e420 0x7fcec8099210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcec8169440 con 0x7fcec8068c10 2024-09-10T21:12:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81692c0 con 0x7fcec80cc9c0 2024-09-10T21:12:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecf400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fcec81695c0 con 0x7fcec815e420 2024-09-10T21:12:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8068c10 0x7fcec80c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8068c10 0x7fcec80c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60902/0 (socket says 172.21.15.16:60902) 2024-09-10T21:12:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecca00640 1 -- 172.21.15.16:0/3629325910 learned_addr learned my addr 172.21.15.16:0/3629325910 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcec7e00640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec80cc9c0 0x7fcec8098cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecd400640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcec815e420 0x7fcec8099210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecca00640 1 -- 172.21.15.16:0/3629325910 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcec815e420 msgr2=0x7fcec8099210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecca00640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcec815e420 0x7fcec8099210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecca00640 1 -- 172.21.15.16:0/3629325910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec80cc9c0 msgr2=0x7fcec8098cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecca00640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec80cc9c0 0x7fcec8098cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcecca00640 1 -- 172.21.15.16:0/3629325910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec80ca050 con 0x7fcec8068c10 2024-09-10T21:12:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.952+0000 7fcec7e00640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec80cc9c0 0x7fcec8098cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.956+0000 7fcecca00640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8068c10 0x7fcec80c6710 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009850 tx=0x7fcebc0a5760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.956+0000 7fcec4c00640 1 -- 172.21.15.16:0/3629325910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0b2040 con 0x7fcec8068c10 2024-09-10T21:12:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.956+0000 7fcec4c00640 1 -- 172.21.15.16:0/3629325910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcebc0043c0 con 0x7fcec8068c10 2024-09-10T21:12:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.956+0000 7fcec4c00640 1 -- 172.21.15.16:0/3629325910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc003860 con 0x7fcec8068c10 2024-09-10T21:12:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.956+0000 7fcecf400640 1 -- 172.21.15.16:0/3629325910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec80ca330 con 0x7fcec8068c10 2024-09-10T21:12:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.956+0000 7fcecf400640 1 -- 172.21.15.16:0/3629325910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec80c42e0 con 0x7fcec8068c10 2024-09-10T21:12:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.956+0000 7fcecf400640 1 -- 172.21.15.16:0/3629325910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec80cc480 con 0x7fcec8068c10 2024-09-10T21:12:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.956+0000 7fcec4c00640 1 -- 172.21.15.16:0/3629325910 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcebc003a00 con 0x7fcec8068c10 2024-09-10T21:12:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.956+0000 7fcec4c00640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcea80725e0 0x7fcea8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.956+0000 7fcec4c00640 1 -- 172.21.15.16:0/3629325910 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fcebc12d180 con 0x7fcec8068c10 2024-09-10T21:12:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.956+0000 7fcec7e00640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcea80725e0 0x7fcea8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.960+0000 7fcec7e00640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcea80725e0 0x7fcea8074aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcec80c5ec0 tx=0x7fceb4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:19.960+0000 7fcec4c00640 1 -- 172.21.15.16:0/3629325910 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcebc0164d0 con 0x7fcec8068c10 2024-09-10T21:12:20.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.172+0000 7fcecf400640 1 -- 172.21.15.16:0/3629325910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcec8072a30 con 0x7fcec8068c10 2024-09-10T21:12:20.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.172+0000 7fcec4c00640 1 -- 172.21.15.16:0/3629325910 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3907 (secure 0 0 0) 0x7fcebc0f8c10 con 0x7fcec8068c10 2024-09-10T21:12:20.181 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:20.181 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"btime":"2024-09-10T21:12:13:729625+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/788814920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":788814920},{"type":"v1","addr":"172.21.15.16:6837","nonce":788814920}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2275087718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2275087718},{"type":"v1","addr":"172.21.15.16:6836","nonce":2275087718}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/983262671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":983262671},{"type":"v1","addr":"172.21.15.52:6835","nonce":983262671}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":134516,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3457135235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3457135235},{"type":"v1","addr":"172.21.15.52:6834","nonce":3457135235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-09-10T21:12:20.181 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 18 2024-09-10T21:12:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.176+0000 7fcecf400640 1 -- 172.21.15.16:0/3629325910 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcea80725e0 msgr2=0x7fcea8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.176+0000 7fcecf400640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcea80725e0 0x7fcea8074aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcec80c5ec0 tx=0x7fceb4009040 comp rx=0 tx=0).stop 2024-09-10T21:12:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.176+0000 7fcecf400640 1 -- 172.21.15.16:0/3629325910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8068c10 msgr2=0x7fcec80c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.176+0000 7fcecf400640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8068c10 0x7fcec80c6710 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009850 tx=0x7fcebc0a5760 comp rx=0 tx=0).stop 2024-09-10T21:12:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.176+0000 7fcecf400640 1 -- 172.21.15.16:0/3629325910 shutdown_connections 2024-09-10T21:12:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.176+0000 7fcecf400640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcea80725e0 0x7fcea8074aa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.176+0000 7fcecf400640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcec815e420 0x7fcec8099210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.176+0000 7fcecf400640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec80cc9c0 0x7fcec8098cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.176+0000 7fcecf400640 1 --2- 172.21.15.16:0/3629325910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8068c10 0x7fcec80c6710 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.176+0000 7fcecf400640 1 -- 172.21.15.16:0/3629325910 >> 172.21.15.16:0/3629325910 conn(0x7fcec80d03a0 msgr2=0x7fcec80d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.176+0000 7fcecf400640 1 -- 172.21.15.16:0/3629325910 shutdown_connections 2024-09-10T21:12:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.176+0000 7fcecf400640 1 -- 172.21.15.16:0/3629325910 wait complete. 2024-09-10T21:12:20.202 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:12:20.203 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:12:20.203 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:12:20.203 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:12:20.203 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:12:20.203 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:12:20.203 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:12:20.203 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:12:20.204 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:12:20.204 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:12:20.204 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:12:20.204 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:12:20.204 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:12:20.204 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:12:20.205 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:12:20.205 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:12:20.216 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:12:20.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:12:20.217 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:12:20.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:12:20.219 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:12:20.219 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:12:20.220 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:12:20.220 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:12:20.264 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:12:20.264 DEBUG:teuthology.parallel:result is None 2024-09-10T21:12:20.265 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:12:20.265 DEBUG:teuthology.parallel:result is None 2024-09-10T21:12:20.269 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:12:20.269 DEBUG:teuthology.parallel:result is None 2024-09-10T21:12:20.270 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:12:20.271 DEBUG:teuthology.parallel:result is None 2024-09-10T21:12:20.271 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:12:20.352 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-10T21:12:20.352 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-10T21:12:20.354 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-09-10T21:12:20.348+0000 7fd441e00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-10T21:12:20.363 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:12:20.363 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:12:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 -- 172.21.15.16:0/1304525659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7f00ca880 msgr2=0x7fc7f0072860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 --2- 172.21.15.16:0/1304525659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7f00ca880 0x7fc7f0072860 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009bc0 tx=0x7fc7e801cba0 comp rx=0 tx=0).stop 2024-09-10T21:12:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 -- 172.21.15.16:0/1304525659 shutdown_connections 2024-09-10T21:12:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 --2- 172.21.15.16:0/1304525659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7f00ca880 0x7fc7f0072860 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 --2- 172.21.15.16:0/1304525659 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7f0157be0 0x7fc7f015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 -- 172.21.15.16:0/1304525659 >> 172.21.15.16:0/1304525659 conn(0x7fc7f00cfb20 msgr2=0x7fc7f00cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 -- 172.21.15.16:0/1304525659 shutdown_connections 2024-09-10T21:12:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 -- 172.21.15.16:0/1304525659 wait complete. 2024-09-10T21:12:20.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 Processor -- start 2024-09-10T21:12:20.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 -- start start 2024-09-10T21:12:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f00ca880 0x7fc7f00c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7f00cd6f0 0x7fc7f014be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7f0157be0 0x7fc7f00b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f0166f60 con 0x7fc7f00ca880 2024-09-10T21:12:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f0166de0 con 0x7fc7f00cd6f0 2024-09-10T21:12:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.416+0000 7fc7f4c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f01670e0 con 0x7fc7f0157be0 2024-09-10T21:12:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7ede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f00ca880 0x7fc7f00c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7ede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f00ca880 0x7fc7f00c64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60914/0 (socket says 172.21.15.16:60914) 2024-09-10T21:12:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7ede00640 1 -- 172.21.15.16:0/2319143084 learned_addr learned my addr 172.21.15.16:0/2319143084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7ede00640 1 -- 172.21.15.16:0/2319143084 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7f0157be0 msgr2=0x7fc7f00b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7ede00640 1 --2- 172.21.15.16:0/2319143084 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7f0157be0 0x7fc7f00b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7ede00640 1 -- 172.21.15.16:0/2319143084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7f00cd6f0 msgr2=0x7fc7f014be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7ede00640 1 --2- 172.21.15.16:0/2319143084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7f00cd6f0 0x7fc7f014be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7ede00640 1 -- 172.21.15.16:0/2319143084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f00b0840 con 0x7fc7f00ca880 2024-09-10T21:12:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7ede00640 1 --2- 172.21.15.16:0/2319143084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f00ca880 0x7fc7f00c64b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00095d0 tx=0x7fc7e0009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7e6000640 1 -- 172.21.15.16:0/2319143084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0018020 con 0x7fc7f00ca880 2024-09-10T21:12:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7e6000640 1 -- 172.21.15.16:0/2319143084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7e0014910 con 0x7fc7f00ca880 2024-09-10T21:12:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7e6000640 1 -- 172.21.15.16:0/2319143084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0014d40 con 0x7fc7f00ca880 2024-09-10T21:12:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7f4c00640 1 -- 172.21.15.16:0/2319143084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f01fadb0 con 0x7fc7f00ca880 2024-09-10T21:12:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7f4c00640 1 -- 172.21.15.16:0/2319143084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f01fb370 con 0x7fc7f00ca880 2024-09-10T21:12:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7f4c00640 1 -- 172.21.15.16:0/2319143084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f0072860 con 0x7fc7f00ca880 2024-09-10T21:12:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.420+0000 7fc7e6000640 1 -- 172.21.15.16:0/2319143084 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc7e001d400 con 0x7fc7f00ca880 2024-09-10T21:12:20.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.424+0000 7fc7e6000640 1 --2- 172.21.15.16:0/2319143084 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7cc0725e0 0x7fc7cc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:20.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.424+0000 7fc7e6000640 1 -- 172.21.15.16:0/2319143084 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fc7e00936a0 con 0x7fc7f00ca880 2024-09-10T21:12:20.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.424+0000 7fc7e7e00640 1 --2- 172.21.15.16:0/2319143084 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7cc0725e0 0x7fc7cc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:20.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.424+0000 7fc7e7e00640 1 --2- 172.21.15.16:0/2319143084 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7cc0725e0 0x7fc7cc074aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80046c0 tx=0x7fc7d80012c0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:20.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.424+0000 7fc7e6000640 1 -- 172.21.15.16:0/2319143084 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc7e0022020 con 0x7fc7f00ca880 2024-09-10T21:12:20.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:20.620+0000 7fc7f4c00640 1 -- 172.21.15.16:0/2319143084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc7f0069970 con 0x7fc7f00ca880 2024-09-10T21:12:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.488+0000 7fc7e6000640 1 -- 172.21.15.16:0/2319143084 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) ==== 97+0+0 (secure 0 0 0) 0x7fc7e0011070 con 0x7fc7f00ca880 2024-09-10T21:12:21.497 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-10T21:12:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.492+0000 7fc7f4c00640 1 -- 172.21.15.16:0/2319143084 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7cc0725e0 msgr2=0x7fc7cc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.492+0000 7fc7f4c00640 1 --2- 172.21.15.16:0/2319143084 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7cc0725e0 0x7fc7cc074aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80046c0 tx=0x7fc7d80012c0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.492+0000 7fc7f4c00640 1 -- 172.21.15.16:0/2319143084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f00ca880 msgr2=0x7fc7f00c64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.492+0000 7fc7f4c00640 1 --2- 172.21.15.16:0/2319143084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f00ca880 0x7fc7f00c64b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00095d0 tx=0x7fc7e0009a90 comp rx=0 tx=0).stop 2024-09-10T21:12:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.492+0000 7fc7f4c00640 1 -- 172.21.15.16:0/2319143084 shutdown_connections 2024-09-10T21:12:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.492+0000 7fc7f4c00640 1 --2- 172.21.15.16:0/2319143084 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7cc0725e0 0x7fc7cc074aa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.492+0000 7fc7f4c00640 1 --2- 172.21.15.16:0/2319143084 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7f0157be0 0x7fc7f00b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.492+0000 7fc7f4c00640 1 --2- 172.21.15.16:0/2319143084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7f00cd6f0 0x7fc7f014be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.492+0000 7fc7f4c00640 1 --2- 172.21.15.16:0/2319143084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f00ca880 0x7fc7f00c64b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.492+0000 7fc7f4c00640 1 -- 172.21.15.16:0/2319143084 >> 172.21.15.16:0/2319143084 conn(0x7fc7f00cfb20 msgr2=0x7fc7f01597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.492+0000 7fc7f4c00640 1 -- 172.21.15.16:0/2319143084 shutdown_connections 2024-09-10T21:12:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.492+0000 7fc7f4c00640 1 -- 172.21.15.16:0/2319143084 wait complete. 2024-09-10T21:12:21.524 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:12:21.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.668+0000 7f096d600640 1 -- 172.21.15.16:0/1838611169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09680af780 msgr2=0x7f096815f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:21.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.668+0000 7f096d600640 1 --2- 172.21.15.16:0/1838611169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09680af780 0x7f096815f3b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09680c7ae0 tx=0x7f095401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.668+0000 7f096d600640 1 -- 172.21.15.16:0/1838611169 shutdown_connections 2024-09-10T21:12:21.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.668+0000 7f096d600640 1 --2- 172.21.15.16:0/1838611169 >> v2:172.21.15.52:3300/0 conn(0x7f0968074280 0x7f09680cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.668+0000 7f096d600640 1 --2- 172.21.15.16:0/1838611169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09680af780 0x7f096815f3b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.668+0000 7f096d600640 1 -- 172.21.15.16:0/1838611169 >> 172.21.15.16:0/1838611169 conn(0x7f09680d0a60 msgr2=0x7f09680d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:21.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.668+0000 7f096d600640 1 -- 172.21.15.16:0/1838611169 shutdown_connections 2024-09-10T21:12:21.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.668+0000 7f096d600640 1 -- 172.21.15.16:0/1838611169 wait complete. 2024-09-10T21:12:21.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.668+0000 7f096d600640 1 Processor -- start 2024-09-10T21:12:21.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.668+0000 7f096d600640 1 -- start start 2024-09-10T21:12:21.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f096d600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09680737e0 0x7f09680ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:21.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f096d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0968074280 0x7f09680a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:21.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f096d600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09680af780 0x7f09680a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f096d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09680cfce0 con 0x7f0968074280 2024-09-10T21:12:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f096d600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09680cfb60 con 0x7f09680737e0 2024-09-10T21:12:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f096d600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0968167f70 con 0x7f09680af780 2024-09-10T21:12:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f0967400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0968074280 0x7f09680a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f0967400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0968074280 0x7f09680a9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60936/0 (socket says 172.21.15.16:60936) 2024-09-10T21:12:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f0967400640 1 -- 172.21.15.16:0/4269254121 learned_addr learned my addr 172.21.15.16:0/4269254121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f0967e00640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09680737e0 0x7f09680ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f096cc00640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09680af780 0x7f09680a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f0967400640 1 -- 172.21.15.16:0/4269254121 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09680af780 msgr2=0x7f09680a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f0967400640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09680af780 0x7f09680a9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f0967400640 1 -- 172.21.15.16:0/4269254121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09680737e0 msgr2=0x7f09680ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:21.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f0967400640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09680737e0 0x7f09680ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f0967400640 1 -- 172.21.15.16:0/4269254121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096815bd20 con 0x7f0968074280 2024-09-10T21:12:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f0967400640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0968074280 0x7f09680a9440 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f09540b0040 tx=0x7f09540a5580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:21.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f095fe00640 1 -- 172.21.15.16:0/4269254121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09540b2040 con 0x7f0968074280 2024-09-10T21:12:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f095fe00640 1 -- 172.21.15.16:0/4269254121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09540043b0 con 0x7f0968074280 2024-09-10T21:12:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f095fe00640 1 -- 172.21.15.16:0/4269254121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954005280 con 0x7f0968074280 2024-09-10T21:12:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f096d600640 1 -- 172.21.15.16:0/4269254121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09680b6290 con 0x7f0968074280 2024-09-10T21:12:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f096d600640 1 -- 172.21.15.16:0/4269254121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096815c2a0 con 0x7f0968074280 2024-09-10T21:12:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f096d600640 1 -- 172.21.15.16:0/4269254121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096806d180 con 0x7f0968074280 2024-09-10T21:12:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.672+0000 7f095fe00640 1 -- 172.21.15.16:0/4269254121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0954005420 con 0x7f0968074280 2024-09-10T21:12:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.676+0000 7f095fe00640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0948072610 0x7f0948074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.676+0000 7f095fe00640 1 -- 172.21.15.16:0/4269254121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f095412c900 con 0x7f0968074280 2024-09-10T21:12:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.676+0000 7f0967e00640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0948072610 0x7f0948074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.676+0000 7f0967e00640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0948072610 0x7f0948074ad0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0958007fd0 tx=0x7f0958009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.680+0000 7f095fe00640 1 -- 172.21.15.16:0/4269254121 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f09540164b0 con 0x7f0968074280 2024-09-10T21:12:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.916+0000 7f096d600640 1 -- 172.21.15.16:0/4269254121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f09680008e0 con 0x7f0968074280 2024-09-10T21:12:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.916+0000 7f095fe00640 1 -- 172.21.15.16:0/4269254121 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7f09540f7cd0 con 0x7f0968074280 2024-09-10T21:12:21.923 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:21.923 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:12:21.923 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-10T21:12:21.923 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.923 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-10T21:12:21.923 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmhg+lJBAAWjuYW75BGo2dmyAUi7s9+A==", 2024-09-10T21:12:21.923 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.923 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:12:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:12:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-10T21:12:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmQr7+MBAAOzXhWIirBOoi/7wKnrFgmg==", 2024-09-10T21:12:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:12:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:12:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.925 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.925 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-10T21:12:21.925 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmxxoALBAAgkmw91/VP1Uv2WMZsJKAyA==", 2024-09-10T21:12:21.925 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:21.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:12:21.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:12:21.925 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.926 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.926 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-10T21:12:21.926 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmA4tzOBAAiS+8n+Gokz6yXapVf2ecYg==", 2024-09-10T21:12:21.926 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:21.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:12:21.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:12:21.926 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.926 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.926 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.926 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-10T21:12:21.926 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHteBmRoahLRAA32PsUkqtGkeI7tMFRwas5Q==", 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCLteBmGmEMBhAAr1yQEf/NkiN4+ezAQ9VR2A==", 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.927 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.928 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.928 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-10T21:12:21.928 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCOteBmmMuAHBAAt+G/KApsZymjTLRSmyRbdw==", 2024-09-10T21:12:21.928 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.928 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:21.928 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:21.928 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.928 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.928 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.928 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-10T21:12:21.928 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCRteBmoQfNMBAA0vnIqlYoI13KNAAfNmJg3w==", 2024-09-10T21:12:21.929 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:21.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:21.929 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.929 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.929 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.929 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-10T21:12:21.929 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCWteBmc/7lDxAADlSi4UpcFJVNdp6YTkpdSw==", 2024-09-10T21:12:21.929 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:21.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:21.930 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.930 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.930 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-10T21:12:21.930 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZteBmBZ1LJhAAPcbZa1LQu4kNbSsJ/8Z81A==", 2024-09-10T21:12:21.930 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.930 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:21.930 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:21.930 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.931 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.931 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-10T21:12:21.931 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCcteBm7/czOxAAqREnY84+LF1N9dRXOhzz6g==", 2024-09-10T21:12:21.931 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.931 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:21.931 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:21.931 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.931 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.931 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-10T21:12:21.931 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgteBmG2J8FRAAB+cMFCnf9wJg1R3RD5SUJw==", 2024-09-10T21:12:21.932 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:21.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:21.932 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.932 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.932 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.932 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-10T21:12:21.932 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA==", 2024-09-10T21:12:21.932 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:21.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-10T21:12:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:12:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-10T21:12:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmuEm4ChAAe08QvhwIcN8ZHa3AoVcYww==", 2024-09-10T21:12:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:12:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:12:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:12:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.934 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.934 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-10T21:12:21.934 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBm50o1BxAAIFbVxVqU4rU8eKYpuZjTHA==", 2024-09-10T21:12:21.934 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.934 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:21.934 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:12:21.934 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:12:21.934 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:12:21.934 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.934 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.934 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-10T21:12:21.935 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmxtu0EBAAmN7+07lDiwjsIC/pNYUlSw==", 2024-09-10T21:12:21.935 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.935 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:21.935 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:12:21.935 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:12:21.935 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:12:21.935 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.935 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.936 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-10T21:12:21.936 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmfqgYCBAAm00tBe45phGieCcp1q46cw==", 2024-09-10T21:12:21.936 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.936 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:12:21.936 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:12:21.936 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:12:21.936 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.936 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.937 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.937 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.937 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-10T21:12:21.937 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmMVABIBAAoB72mPyXyUO6Y6k+fMqQQg==", 2024-09-10T21:12:21.937 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.937 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:12:21.937 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:12:21.937 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.937 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.937 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:21.937 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:21.937 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-10T21:12:21.937 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmeyYxHBAAvBkpTzjPQKAOlHzI1jQ46g==", 2024-09-10T21:12:21.938 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:21.938 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:12:21.938 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:12:21.938 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:21.938 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.938 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:21.938 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-10T21:12:21.938 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:12:21.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.916+0000 7f096d600640 1 -- 172.21.15.16:0/4269254121 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0948072610 msgr2=0x7f0948074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:21.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.916+0000 7f096d600640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0948072610 0x7f0948074ad0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0958007fd0 tx=0x7f0958009040 comp rx=0 tx=0).stop 2024-09-10T21:12:21.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.916+0000 7f096d600640 1 -- 172.21.15.16:0/4269254121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0968074280 msgr2=0x7f09680a9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:21.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.916+0000 7f096d600640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0968074280 0x7f09680a9440 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f09540b0040 tx=0x7f09540a5580 comp rx=0 tx=0).stop 2024-09-10T21:12:21.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.916+0000 7f096d600640 1 -- 172.21.15.16:0/4269254121 shutdown_connections 2024-09-10T21:12:21.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.916+0000 7f096d600640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0948072610 0x7f0948074ad0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.916+0000 7f096d600640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09680af780 0x7f09680a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.916+0000 7f096d600640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0968074280 0x7f09680a9440 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.916+0000 7f096d600640 1 --2- 172.21.15.16:0/4269254121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09680737e0 0x7f09680ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:21.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.916+0000 7f096d600640 1 -- 172.21.15.16:0/4269254121 >> 172.21.15.16:0/4269254121 conn(0x7f09680d0a60 msgr2=0x7f096815ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:21.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.920+0000 7f096d600640 1 -- 172.21.15.16:0/4269254121 shutdown_connections 2024-09-10T21:12:21.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:21.920+0000 7f096d600640 1 -- 172.21.15.16:0/4269254121 wait complete. 2024-09-10T21:12:21.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:12:22.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.084+0000 7ff82f000640 1 -- 172.21.15.16:0/632348487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815f400 msgr2=0x7ff828070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.084+0000 7ff82f000640 1 --2- 172.21.15.16:0/632348487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815f400 0x7ff828070fe0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff8280698f0 tx=0x7ff81801c780 comp rx=0 tx=0).stop 2024-09-10T21:12:22.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.084+0000 7ff82f000640 1 -- 172.21.15.16:0/632348487 shutdown_connections 2024-09-10T21:12:22.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.084+0000 7ff82f000640 1 --2- 172.21.15.16:0/632348487 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff8280cfc80 0x7ff8280d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.084+0000 7ff82f000640 1 --2- 172.21.15.16:0/632348487 >> v2:172.21.15.52:3300/0 conn(0x7ff8280b13c0 0x7ff828071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.084+0000 7ff82f000640 1 --2- 172.21.15.16:0/632348487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815f400 0x7ff828070fe0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.084+0000 7ff82f000640 1 -- 172.21.15.16:0/632348487 >> 172.21.15.16:0/632348487 conn(0x7ff82809b130 msgr2=0x7ff8280d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:22.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.084+0000 7ff82f000640 1 -- 172.21.15.16:0/632348487 shutdown_connections 2024-09-10T21:12:22.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.084+0000 7ff82f000640 1 -- 172.21.15.16:0/632348487 wait complete. 2024-09-10T21:12:22.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82f000640 1 Processor -- start 2024-09-10T21:12:22.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82f000640 1 -- start start 2024-09-10T21:12:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82f000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff8280b13c0 0x7ff8280c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82f000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8280cfc80 0x7ff82814b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82f000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815f400 0x7ff8281fb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82f000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8280afd90 con 0x7ff82815f400 2024-09-10T21:12:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82f000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff8280d1260 con 0x7ff8280cfc80 2024-09-10T21:12:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82f000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff8280aff10 con 0x7ff8280b13c0 2024-09-10T21:12:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815f400 0x7ff8281fb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815f400 0x7ff8281fb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57022/0 (socket says 172.21.15.16:57022) 2024-09-10T21:12:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82d000640 1 -- 172.21.15.16:0/1558095707 learned_addr learned my addr 172.21.15.16:0/1558095707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff827e00640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff8280b13c0 0x7ff8280c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff81fe00640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8280cfc80 0x7ff82814b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82d000640 1 -- 172.21.15.16:0/1558095707 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff8280b13c0 msgr2=0x7ff8280c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82d000640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff8280b13c0 0x7ff8280c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82d000640 1 -- 172.21.15.16:0/1558095707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8280cfc80 msgr2=0x7ff82814b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82d000640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8280cfc80 0x7ff82814b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82d000640 1 -- 172.21.15.16:0/1558095707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82814bdc0 con 0x7ff82815f400 2024-09-10T21:12:22.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff827e00640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff8280b13c0 0x7ff8280c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:22.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff81fe00640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8280cfc80 0x7ff82814b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:22.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82d000640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815f400 0x7ff8281fb350 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff82000daa0 tx=0x7ff82000df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:22.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff824c00640 1 -- 172.21.15.16:0/1558095707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820017070 con 0x7ff82815f400 2024-09-10T21:12:22.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff824c00640 1 -- 172.21.15.16:0/1558095707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff820004b30 con 0x7ff82815f400 2024-09-10T21:12:22.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82f000640 1 -- 172.21.15.16:0/1558095707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8280cb330 con 0x7ff82815f400 2024-09-10T21:12:22.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff824c00640 1 -- 172.21.15.16:0/1558095707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820005020 con 0x7ff82815f400 2024-09-10T21:12:22.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82f000640 1 -- 172.21.15.16:0/1558095707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8280cb8f0 con 0x7ff82815f400 2024-09-10T21:12:22.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.088+0000 7ff82f000640 1 -- 172.21.15.16:0/1558095707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8280d0110 con 0x7ff82815f400 2024-09-10T21:12:22.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.092+0000 7ff824c00640 1 -- 172.21.15.16:0/1558095707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff8200051c0 con 0x7ff82815f400 2024-09-10T21:12:22.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.092+0000 7ff824c00640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff8040725e0 0x7ff804074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.092+0000 7ff824c00640 1 -- 172.21.15.16:0/1558095707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7ff820093480 con 0x7ff82815f400 2024-09-10T21:12:22.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.092+0000 7ff827e00640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff8040725e0 0x7ff804074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.092+0000 7ff827e00640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff8040725e0 0x7ff804074aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff8180af000 tx=0x7ff81801afc0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:22.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.096+0000 7ff824c00640 1 -- 172.21.15.16:0/1558095707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff820020070 con 0x7ff82815f400 2024-09-10T21:12:22.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff82f000640 1 -- 172.21.15.16:0/1558095707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff8280b3180 con 0x7ff82815f400 2024-09-10T21:12:22.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff824c00640 1 -- 172.21.15.16:0/1558095707 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) ==== 84+0+65 (secure 0 0 0) 0x7ff82005ee10 con 0x7ff82815f400 2024-09-10T21:12:22.278 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:22.278 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-10T21:12:22.278 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:22.278 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-10T21:12:22.278 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-10T21:12:22.279 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:22.279 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-10T21:12:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff82f000640 1 -- 172.21.15.16:0/1558095707 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff8040725e0 msgr2=0x7ff804074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff82f000640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff8040725e0 0x7ff804074aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff8180af000 tx=0x7ff81801afc0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff82f000640 1 -- 172.21.15.16:0/1558095707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815f400 msgr2=0x7ff8281fb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff82f000640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815f400 0x7ff8281fb350 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff82000daa0 tx=0x7ff82000df60 comp rx=0 tx=0).stop 2024-09-10T21:12:22.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff82f000640 1 -- 172.21.15.16:0/1558095707 shutdown_connections 2024-09-10T21:12:22.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff82f000640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff8040725e0 0x7ff804074aa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff82f000640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815f400 0x7ff8281fb350 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff82f000640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8280cfc80 0x7ff82814b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff82f000640 1 --2- 172.21.15.16:0/1558095707 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff8280b13c0 0x7ff8280c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff82f000640 1 -- 172.21.15.16:0/1558095707 >> 172.21.15.16:0/1558095707 conn(0x7ff82809b130 msgr2=0x7ff82806f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:22.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff82f000640 1 -- 172.21.15.16:0/1558095707 shutdown_connections 2024-09-10T21:12:22.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.272+0000 7ff82f000640 1 -- 172.21.15.16:0/1558095707 wait complete. 2024-09-10T21:12:22.299 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:12:22.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:12:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.440+0000 7f66c9c00640 1 -- 172.21.15.16:0/1208095898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c415bf00 msgr2=0x7f66c40ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.440+0000 7f66c9c00640 1 --2- 172.21.15.16:0/1208095898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c415bf00 0x7f66c40ce7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f66c4074d00 tx=0x7f66b401c890 comp rx=0 tx=0).stop 2024-09-10T21:12:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.440+0000 7f66c9c00640 1 -- 172.21.15.16:0/1208095898 shutdown_connections 2024-09-10T21:12:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.440+0000 7f66c9c00640 1 --2- 172.21.15.16:0/1208095898 >> v2:172.21.15.52:3300/0 conn(0x7f66c40d28b0 0x7f66c40cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.440+0000 7f66c9c00640 1 --2- 172.21.15.16:0/1208095898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c415bf00 0x7f66c40ce7a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.440+0000 7f66c9c00640 1 -- 172.21.15.16:0/1208095898 >> 172.21.15.16:0/1208095898 conn(0x7f66c40d1d90 msgr2=0x7f66c40d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.440+0000 7f66c9c00640 1 -- 172.21.15.16:0/1208095898 shutdown_connections 2024-09-10T21:12:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66c9c00640 1 -- 172.21.15.16:0/1208095898 wait complete. 2024-09-10T21:12:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66c9c00640 1 Processor -- start 2024-09-10T21:12:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66c9c00640 1 -- start start 2024-09-10T21:12:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66c9c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66c40cf220 0x7f66c415ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66c9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c40d28b0 0x7f66c406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66c9c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c415bf00 0x7f66c40b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66c9c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66c4169210 con 0x7f66c40d28b0 2024-09-10T21:12:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66c9c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f66c4169090 con 0x7f66c415bf00 2024-09-10T21:12:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66c9c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f66c4169390 con 0x7f66c40cf220 2024-09-10T21:12:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66bbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c40d28b0 0x7f66c406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66bbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c40d28b0 0x7f66c406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57028/0 (socket says 172.21.15.16:57028) 2024-09-10T21:12:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66bbe00640 1 -- 172.21.15.16:0/2957220519 learned_addr learned my addr 172.21.15.16:0/2957220519 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66c2a00640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66c40cf220 0x7f66c415ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66c3400640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c415bf00 0x7f66c40b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66bbe00640 1 -- 172.21.15.16:0/2957220519 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66c40cf220 msgr2=0x7f66c415ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66bbe00640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66c40cf220 0x7f66c415ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66bbe00640 1 -- 172.21.15.16:0/2957220519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c415bf00 msgr2=0x7f66c40b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66bbe00640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c415bf00 0x7f66c40b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66bbe00640 1 -- 172.21.15.16:0/2957220519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c4159ed0 con 0x7f66c40d28b0 2024-09-10T21:12:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.444+0000 7f66bbe00640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c40d28b0 0x7f66c406dcf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00c9a0 tx=0x7f66ac00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.448+0000 7f66baa00640 1 -- 172.21.15.16:0/2957220519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac007c70 con 0x7f66c40d28b0 2024-09-10T21:12:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.448+0000 7f66baa00640 1 -- 172.21.15.16:0/2957220519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66ac00f450 con 0x7f66c40d28b0 2024-09-10T21:12:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.448+0000 7f66c9c00640 1 -- 172.21.15.16:0/2957220519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c415a1b0 con 0x7f66c40d28b0 2024-09-10T21:12:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.448+0000 7f66baa00640 1 -- 172.21.15.16:0/2957220519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00f8a0 con 0x7f66c40d28b0 2024-09-10T21:12:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.448+0000 7f66c9c00640 1 -- 172.21.15.16:0/2957220519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c415a770 con 0x7f66c40d28b0 2024-09-10T21:12:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.448+0000 7f66c9c00640 1 -- 172.21.15.16:0/2957220519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c40cc800 con 0x7f66c40d28b0 2024-09-10T21:12:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.448+0000 7f66baa00640 1 -- 172.21.15.16:0/2957220519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f66ac00fa40 con 0x7f66c40d28b0 2024-09-10T21:12:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.448+0000 7f66baa00640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66a00725e0 0x7f66a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.448+0000 7f66baa00640 1 -- 172.21.15.16:0/2957220519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f66ac0937f0 con 0x7f66c40d28b0 2024-09-10T21:12:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.448+0000 7f66c2a00640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66a00725e0 0x7f66a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.452+0000 7f66c2a00640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66a00725e0 0x7f66a0074aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f66b40af000 tx=0x7f66b401b0d0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.452+0000 7f66baa00640 1 -- 172.21.15.16:0/2957220519 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f66ac014030 con 0x7f66c40d28b0 2024-09-10T21:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.628+0000 7f66c9c00640 1 -- 172.21.15.16:0/2957220519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f66c4069e00 con 0x7f66c40d28b0 2024-09-10T21:12:22.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.768+0000 7f66baa00640 1 -- 172.21.15.16:0/2957220519 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) ==== 138+0+0 (secure 0 0 0) 0x7f66ac05f100 con 0x7f66c40d28b0 2024-09-10T21:12:22.774 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-10T21:12:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.768+0000 7f66c9c00640 1 -- 172.21.15.16:0/2957220519 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66a00725e0 msgr2=0x7f66a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.768+0000 7f66c9c00640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66a00725e0 0x7f66a0074aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f66b40af000 tx=0x7f66b401b0d0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.768+0000 7f66c9c00640 1 -- 172.21.15.16:0/2957220519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c40d28b0 msgr2=0x7f66c406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.768+0000 7f66c9c00640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c40d28b0 0x7f66c406dcf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00c9a0 tx=0x7f66ac00ce60 comp rx=0 tx=0).stop 2024-09-10T21:12:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.768+0000 7f66c9c00640 1 -- 172.21.15.16:0/2957220519 shutdown_connections 2024-09-10T21:12:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.768+0000 7f66c9c00640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66a00725e0 0x7f66a0074aa0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.768+0000 7f66c9c00640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c415bf00 0x7f66c40b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.768+0000 7f66c9c00640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c40d28b0 0x7f66c406dcf0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.768+0000 7f66c9c00640 1 --2- 172.21.15.16:0/2957220519 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66c40cf220 0x7f66c415ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.768+0000 7f66c9c00640 1 -- 172.21.15.16:0/2957220519 >> 172.21.15.16:0/2957220519 conn(0x7f66c40d1d90 msgr2=0x7f66c40af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:22.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.768+0000 7f66c9c00640 1 -- 172.21.15.16:0/2957220519 shutdown_connections 2024-09-10T21:12:22.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.768+0000 7f66c9c00640 1 -- 172.21.15.16:0/2957220519 wait complete. 2024-09-10T21:12:22.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:12:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.948+0000 7fe0a4800640 1 -- 172.21.15.16:0/4216046821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe09c0cbee0 msgr2=0x7fe09c0cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.948+0000 7fe0a4800640 1 --2- 172.21.15.16:0/4216046821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe09c0cbee0 0x7fe09c0cc330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0cf0d0 tx=0x7fe09801cd60 comp rx=0 tx=0).stop 2024-09-10T21:12:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.948+0000 7fe0a4800640 1 -- 172.21.15.16:0/4216046821 shutdown_connections 2024-09-10T21:12:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.948+0000 7fe0a4800640 1 --2- 172.21.15.16:0/4216046821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe09c0cbee0 0x7fe09c0cc330 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.948+0000 7fe0a4800640 1 --2- 172.21.15.16:0/4216046821 >> v2:172.21.15.52:3300/0 conn(0x7fe09c06f0e0 0x7fe09c06f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.948+0000 7fe0a4800640 1 -- 172.21.15.16:0/4216046821 >> 172.21.15.16:0/4216046821 conn(0x7fe09c0b16a0 msgr2=0x7fe09c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.948+0000 7fe0a4800640 1 -- 172.21.15.16:0/4216046821 shutdown_connections 2024-09-10T21:12:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.948+0000 7fe0a4800640 1 -- 172.21.15.16:0/4216046821 wait complete. 2024-09-10T21:12:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.948+0000 7fe0a4800640 1 Processor -- start 2024-09-10T21:12:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a4800640 1 -- start start 2024-09-10T21:12:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a4800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe09c06f0e0 0x7fe09c0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a4800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe09c0cbb00 0x7fe09c0b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a4800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe09c0cbee0 0x7fe09c155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a4800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c168d00 con 0x7fe09c0cbee0 2024-09-10T21:12:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a4800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c168b80 con 0x7fe09c06f0e0 2024-09-10T21:12:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a4800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe09c168e80 con 0x7fe09c0cbb00 2024-09-10T21:12:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe09c0cbee0 0x7fe09c155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe09c0cbee0 0x7fe09c155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57046/0 (socket says 172.21.15.16:57046) 2024-09-10T21:12:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a2800640 1 -- 172.21.15.16:0/1897377881 learned_addr learned my addr 172.21.15.16:0/1897377881 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a1e00640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe09c06f0e0 0x7fe09c0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a1400640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe09c0cbb00 0x7fe09c0b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a2800640 1 -- 172.21.15.16:0/1897377881 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe09c0cbb00 msgr2=0x7fe09c0b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a2800640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe09c0cbb00 0x7fe09c0b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a2800640 1 -- 172.21.15.16:0/1897377881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe09c06f0e0 msgr2=0x7fe09c0b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a2800640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe09c06f0e0 0x7fe09c0b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a2800640 1 -- 172.21.15.16:0/1897377881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c156230 con 0x7fe09c0cbee0 2024-09-10T21:12:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a2800640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe09c0cbee0 0x7fe09c155cf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe098019970 tx=0x7fe0980a5b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe096000640 1 -- 172.21.15.16:0/1897377881 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0980bb070 con 0x7fe09c0cbee0 2024-09-10T21:12:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe096000640 1 -- 172.21.15.16:0/1897377881 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe098004d40 con 0x7fe09c0cbee0 2024-09-10T21:12:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a4800640 1 -- 172.21.15.16:0/1897377881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09c0af0d0 con 0x7fe09c0cbee0 2024-09-10T21:12:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe096000640 1 -- 172.21.15.16:0/1897377881 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0980b2040 con 0x7fe09c0cbee0 2024-09-10T21:12:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a4800640 1 -- 172.21.15.16:0/1897377881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09c159280 con 0x7fe09c0cbee0 2024-09-10T21:12:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.952+0000 7fe0a4800640 1 -- 172.21.15.16:0/1897377881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe09c06f550 con 0x7fe09c0cbee0 2024-09-10T21:12:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.956+0000 7fe096000640 1 -- 172.21.15.16:0/1897377881 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe0980b5030 con 0x7fe09c0cbee0 2024-09-10T21:12:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.956+0000 7fe096000640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe080072610 0x7fe080074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.956+0000 7fe096000640 1 -- 172.21.15.16:0/1897377881 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5227+0+0 (secure 0 0 0) 0x7fe09812d100 con 0x7fe09c0cbee0 2024-09-10T21:12:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.956+0000 7fe0a1e00640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe080072610 0x7fe080074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.956+0000 7fe0a1e00640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe080072610 0x7fe080074ad0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe090000f70 tx=0x7fe090009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:22.960+0000 7fe096000640 1 -- 172.21.15.16:0/1897377881 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe098016950 con 0x7fe09c0cbee0 2024-09-10T21:12:23.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.136+0000 7fe0a4800640 1 -- 172.21.15.16:0/1897377881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe09c158d30 con 0x7fe09c0cbee0 2024-09-10T21:12:23.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:12:23.386 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:12:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.772+0000 7fe096000640 1 -- 172.21.15.16:0/1897377881 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) ==== 170+0+0 (secure 0 0 0) 0x7fe098101150 con 0x7fe09c0cbee0 2024-09-10T21:12:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-10T21:12:23.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.780+0000 7fe0a4800640 1 -- 172.21.15.16:0/1897377881 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe080072610 msgr2=0x7fe080074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:23.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.780+0000 7fe0a4800640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe080072610 0x7fe080074ad0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe090000f70 tx=0x7fe090009040 comp rx=0 tx=0).stop 2024-09-10T21:12:23.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.780+0000 7fe0a4800640 1 -- 172.21.15.16:0/1897377881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe09c0cbee0 msgr2=0x7fe09c155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:23.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.780+0000 7fe0a4800640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe09c0cbee0 0x7fe09c155cf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe098019970 tx=0x7fe0980a5b90 comp rx=0 tx=0).stop 2024-09-10T21:12:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.780+0000 7fe0a4800640 1 -- 172.21.15.16:0/1897377881 shutdown_connections 2024-09-10T21:12:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.780+0000 7fe0a4800640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe080072610 0x7fe080074ad0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.780+0000 7fe0a4800640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe09c0cbee0 0x7fe09c155cf0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.780+0000 7fe0a4800640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe09c0cbb00 0x7fe09c0b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.780+0000 7fe0a4800640 1 --2- 172.21.15.16:0/1897377881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe09c06f0e0 0x7fe09c0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:23.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.780+0000 7fe0a4800640 1 -- 172.21.15.16:0/1897377881 >> 172.21.15.16:0/1897377881 conn(0x7fe09c0b16a0 msgr2=0x7fe09c15b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:23.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.780+0000 7fe0a4800640 1 -- 172.21.15.16:0/1897377881 shutdown_connections 2024-09-10T21:12:23.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.780+0000 7fe0a4800640 1 -- 172.21.15.16:0/1897377881 wait complete. 2024-09-10T21:12:23.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:12:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.976+0000 7fb49bc00640 1 -- 172.21.15.16:0/547233403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4940ce040 msgr2=0x7fb4940b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.976+0000 7fb49bc00640 1 --2- 172.21.15.16:0/547233403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4940ce040 0x7fb4940b1710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb4940cda10 tx=0x7fb48c01f560 comp rx=0 tx=0).stop 2024-09-10T21:12:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.976+0000 7fb49bc00640 1 -- 172.21.15.16:0/547233403 shutdown_connections 2024-09-10T21:12:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.976+0000 7fb49bc00640 1 --2- 172.21.15.16:0/547233403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4940ce040 0x7fb4940b1710 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.976+0000 7fb49bc00640 1 --2- 172.21.15.16:0/547233403 >> v2:172.21.15.52:3300/0 conn(0x7fb4940d0210 0x7fb4940ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.976+0000 7fb49bc00640 1 -- 172.21.15.16:0/547233403 >> 172.21.15.16:0/547233403 conn(0x7fb4940d1df0 msgr2=0x7fb4940d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.976+0000 7fb49bc00640 1 -- 172.21.15.16:0/547233403 shutdown_connections 2024-09-10T21:12:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.976+0000 7fb49bc00640 1 -- 172.21.15.16:0/547233403 wait complete. 2024-09-10T21:12:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.976+0000 7fb49bc00640 1 Processor -- start 2024-09-10T21:12:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.976+0000 7fb49bc00640 1 -- start start 2024-09-10T21:12:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb49bc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4940ce040 0x7fb4941f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb49bc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4940cfe30 0x7fb4940b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb49bc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4940d0210 0x7fb4940cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb49bc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4940cf1e0 con 0x7fb4940cfe30 2024-09-10T21:12:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb49bc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb4940b14f0 con 0x7fb4940ce040 2024-09-10T21:12:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb49bc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb4940cf360 con 0x7fb4940d0210 2024-09-10T21:12:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb48be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4940cfe30 0x7fb4940b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb499c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4940d0210 0x7fb4940cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb48be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4940cfe30 0x7fb4940b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57052/0 (socket says 172.21.15.16:57052) 2024-09-10T21:12:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb48be00640 1 -- 172.21.15.16:0/2697384482 learned_addr learned my addr 172.21.15.16:0/2697384482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb499200640 1 --2- 172.21.15.16:0/2697384482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4940ce040 0x7fb4941f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb48be00640 1 -- 172.21.15.16:0/2697384482 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4940d0210 msgr2=0x7fb4940cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb48be00640 1 --2- 172.21.15.16:0/2697384482 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4940d0210 0x7fb4940cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb48be00640 1 -- 172.21.15.16:0/2697384482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4940ce040 msgr2=0x7fb4941f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb48be00640 1 --2- 172.21.15.16:0/2697384482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4940ce040 0x7fb4941f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb48be00640 1 -- 172.21.15.16:0/2697384482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4940b7380 con 0x7fb4940cfe30 2024-09-10T21:12:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb48be00640 1 --2- 172.21.15.16:0/2697384482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4940cfe30 0x7fb4940b4c20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00e7a0 tx=0x7fb47c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb491600640 1 -- 172.21.15.16:0/2697384482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c00ccc0 con 0x7fb4940cfe30 2024-09-10T21:12:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb491600640 1 -- 172.21.15.16:0/2697384482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb47c010430 con 0x7fb4940cfe30 2024-09-10T21:12:23.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb49bc00640 1 -- 172.21.15.16:0/2697384482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4940b7660 con 0x7fb4940cfe30 2024-09-10T21:12:23.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb491600640 1 -- 172.21.15.16:0/2697384482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c010860 con 0x7fb4940cfe30 2024-09-10T21:12:23.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb49bc00640 1 -- 172.21.15.16:0/2697384482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb49406de90 con 0x7fb4940cfe30 2024-09-10T21:12:23.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.980+0000 7fb49bc00640 1 -- 172.21.15.16:0/2697384482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4940b1710 con 0x7fb4940cfe30 2024-09-10T21:12:23.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.984+0000 7fb491600640 1 -- 172.21.15.16:0/2697384482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb47c010a00 con 0x7fb4940cfe30 2024-09-10T21:12:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.984+0000 7fb491600640 1 --2- 172.21.15.16:0/2697384482 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4700725e0 0x7fb470074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.984+0000 7fb491600640 1 -- 172.21.15.16:0/2697384482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5609+0+0 (secure 0 0 0) 0x7fb47c093c10 con 0x7fb4940cfe30 2024-09-10T21:12:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.984+0000 7fb499200640 1 --2- 172.21.15.16:0/2697384482 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4700725e0 0x7fb470074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:23.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.984+0000 7fb499200640 1 --2- 172.21.15.16:0/2697384482 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4700725e0 0x7fb470074aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4940b3090 tx=0x7fb484009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:23.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:23.988+0000 7fb491600640 1 -- 172.21.15.16:0/2697384482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb47c05ec70 con 0x7fb4940cfe30 2024-09-10T21:12:24.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.196+0000 7fb49bc00640 1 -- 172.21.15.16:0/2697384482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb4940723b0 con 0x7fb4940cfe30 2024-09-10T21:12:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.776+0000 7fb491600640 1 -- 172.21.15.16:0/2697384482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) ==== 172+0+0 (secure 0 0 0) 0x7fb47c0636a0 con 0x7fb4940cfe30 2024-09-10T21:12:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 6 and data pool 7 2024-09-10T21:12:24.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.780+0000 7fb49bc00640 1 -- 172.21.15.16:0/2697384482 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4700725e0 msgr2=0x7fb470074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:24.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.780+0000 7fb49bc00640 1 --2- 172.21.15.16:0/2697384482 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4700725e0 0x7fb470074aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4940b3090 tx=0x7fb484009040 comp rx=0 tx=0).stop 2024-09-10T21:12:24.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.780+0000 7fb49bc00640 1 -- 172.21.15.16:0/2697384482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4940cfe30 msgr2=0x7fb4940b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:24.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.780+0000 7fb49bc00640 1 --2- 172.21.15.16:0/2697384482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4940cfe30 0x7fb4940b4c20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00e7a0 tx=0x7fb47c00ec60 comp rx=0 tx=0).stop 2024-09-10T21:12:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.780+0000 7fb49bc00640 1 -- 172.21.15.16:0/2697384482 shutdown_connections 2024-09-10T21:12:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.780+0000 7fb49bc00640 1 --2- 172.21.15.16:0/2697384482 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4700725e0 0x7fb470074aa0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.780+0000 7fb49bc00640 1 --2- 172.21.15.16:0/2697384482 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4940d0210 0x7fb4940cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.780+0000 7fb49bc00640 1 --2- 172.21.15.16:0/2697384482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4940cfe30 0x7fb4940b4c20 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.780+0000 7fb49bc00640 1 --2- 172.21.15.16:0/2697384482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4940ce040 0x7fb4941f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.780+0000 7fb49bc00640 1 -- 172.21.15.16:0/2697384482 >> 172.21.15.16:0/2697384482 conn(0x7fb4940d1df0 msgr2=0x7fb4940aeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.780+0000 7fb49bc00640 1 -- 172.21.15.16:0/2697384482 shutdown_connections 2024-09-10T21:12:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.780+0000 7fb49bc00640 1 -- 172.21.15.16:0/2697384482 wait complete. 2024-09-10T21:12:24.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:12:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 -- 172.21.15.16:0/2322545488 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6d6c0b13c0 msgr2=0x7f6d6c0afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 --2- 172.21.15.16:0/2322545488 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6d6c0b13c0 0x7f6d6c0afc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009bf0 tx=0x7f6d6401cac0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 -- 172.21.15.16:0/2322545488 shutdown_connections 2024-09-10T21:12:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 --2- 172.21.15.16:0/2322545488 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6d6c0b13c0 0x7f6d6c0afc90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 -- 172.21.15.16:0/2322545488 >> 172.21.15.16:0/2322545488 conn(0x7f6d6c09b130 msgr2=0x7f6d6c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 -- 172.21.15.16:0/2322545488 shutdown_connections 2024-09-10T21:12:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 -- 172.21.15.16:0/2322545488 wait complete. 2024-09-10T21:12:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 Processor -- start 2024-09-10T21:12:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 -- start start 2024-09-10T21:12:24.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c0716f0 0x7f6d6c14af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:24.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6c0b13c0 0x7f6d6c1f9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:24.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6d6c15f400 0x7f6d6c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c16a0e0 con 0x7f6d6c0716f0 2024-09-10T21:12:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c169f60 con 0x7f6d6c0b13c0 2024-09-10T21:12:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.968+0000 7f6d73800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6d6c16a260 con 0x7f6d6c15f400 2024-09-10T21:12:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d70e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c0716f0 0x7f6d6c14af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d70e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c0716f0 0x7f6d6c14af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57060/0 (socket says 172.21.15.16:57060) 2024-09-10T21:12:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d70e00640 1 -- 172.21.15.16:0/3089758339 learned_addr learned my addr 172.21.15.16:0/3089758339 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d6be00640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6c0b13c0 0x7f6d6c1f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d71800640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6d6c15f400 0x7f6d6c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d70e00640 1 -- 172.21.15.16:0/3089758339 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6d6c15f400 msgr2=0x7f6d6c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d70e00640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6d6c15f400 0x7f6d6c0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d70e00640 1 -- 172.21.15.16:0/3089758339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6c0b13c0 msgr2=0x7f6d6c1f9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d70e00640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6c0b13c0 0x7f6d6c1f9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d70e00640 1 -- 172.21.15.16:0/3089758339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6c1f9db0 con 0x7f6d6c0716f0 2024-09-10T21:12:24.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d70e00640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c0716f0 0x7f6d6c14af20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000c460 tx=0x7f6d6000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:24.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d68c00640 1 -- 172.21.15.16:0/3089758339 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60005230 con 0x7f6d6c0716f0 2024-09-10T21:12:24.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d68c00640 1 -- 172.21.15.16:0/3089758339 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d600054e0 con 0x7f6d6c0716f0 2024-09-10T21:12:24.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d68c00640 1 -- 172.21.15.16:0/3089758339 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60004900 con 0x7f6d6c0716f0 2024-09-10T21:12:24.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d73800640 1 -- 172.21.15.16:0/3089758339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6c155f30 con 0x7f6d6c0716f0 2024-09-10T21:12:24.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d73800640 1 -- 172.21.15.16:0/3089758339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6c1564f0 con 0x7f6d6c0716f0 2024-09-10T21:12:24.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d73800640 1 -- 172.21.15.16:0/3089758339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d6c070fe0 con 0x7f6d6c0716f0 2024-09-10T21:12:24.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.972+0000 7f6d68c00640 1 -- 172.21.15.16:0/3089758339 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6d60023060 con 0x7f6d6c0716f0 2024-09-10T21:12:24.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.976+0000 7f6d68c00640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d4c0725e0 0x7f6d4c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.976+0000 7f6d68c00640 1 -- 172.21.15.16:0/3089758339 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f6d60092fe0 con 0x7f6d6c0716f0 2024-09-10T21:12:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.976+0000 7f6d6be00640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d4c0725e0 0x7f6d4c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.976+0000 7f6d6be00640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d4c0725e0 0x7f6d4c074aa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6d58000c10 tx=0x7f6d58009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:24.980+0000 7f6d68c00640 1 -- 172.21.15.16:0/3089758339 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6d6c1f9db0 con 0x7f6d6c0716f0 2024-09-10T21:12:25.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.156+0000 7f6d73800640 1 -- 172.21.15.16:0/3089758339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d6c0cddb0 con 0x7f6d6c0716f0 2024-09-10T21:12:25.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.156+0000 7f6d68c00640 1 -- 172.21.15.16:0/3089758339 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16634 (secure 0 0 0) 0x7f6d6005dfd0 con 0x7f6d6c0716f0 2024-09-10T21:12:25.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:25.163 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":45,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:24.201843+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:12:22.642253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T21:12:23.151552+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:12:25.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.160+0000 7f6d73800640 1 -- 172.21.15.16:0/3089758339 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d4c0725e0 msgr2=0x7f6d4c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.160+0000 7f6d73800640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d4c0725e0 0x7f6d4c074aa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6d58000c10 tx=0x7f6d58009040 comp rx=0 tx=0).stop 2024-09-10T21:12:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.160+0000 7f6d73800640 1 -- 172.21.15.16:0/3089758339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c0716f0 msgr2=0x7f6d6c14af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.160+0000 7f6d73800640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c0716f0 0x7f6d6c14af20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000c460 tx=0x7f6d6000c920 comp rx=0 tx=0).stop 2024-09-10T21:12:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.160+0000 7f6d73800640 1 -- 172.21.15.16:0/3089758339 shutdown_connections 2024-09-10T21:12:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.160+0000 7f6d73800640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d4c0725e0 0x7f6d4c074aa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.160+0000 7f6d73800640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6d6c15f400 0x7f6d6c0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.160+0000 7f6d73800640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6c0b13c0 0x7f6d6c1f9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.160+0000 7f6d73800640 1 --2- 172.21.15.16:0/3089758339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c0716f0 0x7f6d6c14af20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.160+0000 7f6d73800640 1 -- 172.21.15.16:0/3089758339 >> 172.21.15.16:0/3089758339 conn(0x7f6d6c09b130 msgr2=0x7f6d6c157830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.160+0000 7f6d73800640 1 -- 172.21.15.16:0/3089758339 shutdown_connections 2024-09-10T21:12:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.160+0000 7f6d73800640 1 -- 172.21.15.16:0/3089758339 wait complete. 2024-09-10T21:12:25.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:12:25.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.324+0000 7fa6f7800640 1 -- 172.21.15.16:0/2966354310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f015e200 msgr2=0x7fa6f00ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:25.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.324+0000 7fa6f7800640 1 --2- 172.21.15.16:0/2966354310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f015e200 0x7fa6f00ceb80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009840 tx=0x7fa6e401c760 comp rx=0 tx=0).stop 2024-09-10T21:12:25.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.324+0000 7fa6f7800640 1 -- 172.21.15.16:0/2966354310 shutdown_connections 2024-09-10T21:12:25.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.324+0000 7fa6f7800640 1 --2- 172.21.15.16:0/2966354310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f015e200 0x7fa6f00ceb80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.324+0000 7fa6f7800640 1 -- 172.21.15.16:0/2966354310 >> 172.21.15.16:0/2966354310 conn(0x7fa6f00d3f70 msgr2=0x7fa6f00d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:25.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.324+0000 7fa6f7800640 1 -- 172.21.15.16:0/2966354310 shutdown_connections 2024-09-10T21:12:25.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.324+0000 7fa6f7800640 1 -- 172.21.15.16:0/2966354310 wait complete. 2024-09-10T21:12:25.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f7800640 1 Processor -- start 2024-09-10T21:12:25.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f7800640 1 -- start start 2024-09-10T21:12:25.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f7800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6f00cf7d0 0x7fa6f014dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:25.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f7800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6f00d4b00 0x7fa6f014e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f7800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f015e200 0x7fa6f015e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f7800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f016c050 con 0x7fa6f015e200 2024-09-10T21:12:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f7800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f00d39e0 con 0x7fa6f00cf7d0 2024-09-10T21:12:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f7800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f016c1d0 con 0x7fa6f00d4b00 2024-09-10T21:12:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f015e200 0x7fa6f015e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f015e200 0x7fa6f015e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57078/0 (socket says 172.21.15.16:57078) 2024-09-10T21:12:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f5800640 1 -- 172.21.15.16:0/182346290 learned_addr learned my addr 172.21.15.16:0/182346290 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f4e00640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6f00cf7d0 0x7fa6f014dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6efe00640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6f00d4b00 0x7fa6f014e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f5800640 1 -- 172.21.15.16:0/182346290 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6f00d4b00 msgr2=0x7fa6f014e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f5800640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6f00d4b00 0x7fa6f014e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f5800640 1 -- 172.21.15.16:0/182346290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6f00cf7d0 msgr2=0x7fa6f014dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f5800640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6f00cf7d0 0x7fa6f014dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f5800640 1 -- 172.21.15.16:0/182346290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f01509d0 con 0x7fa6f015e200 2024-09-10T21:12:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f5800640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f015e200 0x7fa6f015e8e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c780 tx=0x7fa6e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6ecc00640 1 -- 172.21.15.16:0/182346290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8004b70 con 0x7fa6f015e200 2024-09-10T21:12:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6ecc00640 1 -- 172.21.15.16:0/182346290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6e8004e00 con 0x7fa6f015e200 2024-09-10T21:12:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6ecc00640 1 -- 172.21.15.16:0/182346290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e800aa90 con 0x7fa6f015e200 2024-09-10T21:12:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f7800640 1 -- 172.21.15.16:0/182346290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f014e930 con 0x7fa6f015e200 2024-09-10T21:12:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f7800640 1 -- 172.21.15.16:0/182346290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f014eef0 con 0x7fa6f015e200 2024-09-10T21:12:25.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.328+0000 7fa6f7800640 1 -- 172.21.15.16:0/182346290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f00d2710 con 0x7fa6f015e200 2024-09-10T21:12:25.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.332+0000 7fa6ecc00640 1 -- 172.21.15.16:0/182346290 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa6e800ac30 con 0x7fa6f015e200 2024-09-10T21:12:25.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.332+0000 7fa6ecc00640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa6d00725e0 0x7fa6d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:25.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.332+0000 7fa6ecc00640 1 -- 172.21.15.16:0/182346290 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fa6e80933c0 con 0x7fa6f015e200 2024-09-10T21:12:25.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.332+0000 7fa6f4e00640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa6d00725e0 0x7fa6d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:25.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.332+0000 7fa6f4e00640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa6d00725e0 0x7fa6d0074aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40af000 tx=0x7fa6e401afa0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:25.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.336+0000 7fa6ecc00640 1 -- 172.21.15.16:0/182346290 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa6e8020070 con 0x7fa6f015e200 2024-09-10T21:12:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.512+0000 7fa6f7800640 1 -- 172.21.15.16:0/182346290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6f015ee20 con 0x7fa6f015e200 2024-09-10T21:12:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.512+0000 7fa6ecc00640 1 -- 172.21.15.16:0/182346290 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16634 (secure 0 0 0) 0x7fa6e805e440 con 0x7fa6f015e200 2024-09-10T21:12:25.520 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:25.520 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":45,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:24.201843+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:12:22.642253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T21:12:23.151552+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:12:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.516+0000 7fa6f7800640 1 -- 172.21.15.16:0/182346290 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa6d00725e0 msgr2=0x7fa6d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.516+0000 7fa6f7800640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa6d00725e0 0x7fa6d0074aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40af000 tx=0x7fa6e401afa0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.516+0000 7fa6f7800640 1 -- 172.21.15.16:0/182346290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f015e200 msgr2=0x7fa6f015e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.516+0000 7fa6f7800640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f015e200 0x7fa6f015e8e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c780 tx=0x7fa6e800cc40 comp rx=0 tx=0).stop 2024-09-10T21:12:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.516+0000 7fa6f7800640 1 -- 172.21.15.16:0/182346290 shutdown_connections 2024-09-10T21:12:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.516+0000 7fa6f7800640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa6d00725e0 0x7fa6d0074aa0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.516+0000 7fa6f7800640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f015e200 0x7fa6f015e8e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.516+0000 7fa6f7800640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6f00d4b00 0x7fa6f014e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.516+0000 7fa6f7800640 1 --2- 172.21.15.16:0/182346290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6f00cf7d0 0x7fa6f014dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.516+0000 7fa6f7800640 1 -- 172.21.15.16:0/182346290 >> 172.21.15.16:0/182346290 conn(0x7fa6f00d3f70 msgr2=0x7fa6f0156ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.516+0000 7fa6f7800640 1 -- 172.21.15.16:0/182346290 shutdown_connections 2024-09-10T21:12:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.516+0000 7fa6f7800640 1 -- 172.21.15.16:0/182346290 wait complete. 2024-09-10T21:12:25.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:12:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.684+0000 7f2bb3c00640 1 -- 172.21.15.16:0/3779759895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bac160050 msgr2=0x7f2bac0d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.684+0000 7f2bb3c00640 1 --2- 172.21.15.16:0/3779759895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bac160050 0x7f2bac0d2800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009bc0 tx=0x7f2ba401c9a0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.684+0000 7f2bb3c00640 1 -- 172.21.15.16:0/3779759895 shutdown_connections 2024-09-10T21:12:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.684+0000 7f2bb3c00640 1 --2- 172.21.15.16:0/3779759895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bac160050 0x7f2bac0d2800 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.684+0000 7f2bb3c00640 1 -- 172.21.15.16:0/3779759895 >> 172.21.15.16:0/3779759895 conn(0x7f2bac0d1ce0 msgr2=0x7f2bac0d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.684+0000 7f2bb3c00640 1 -- 172.21.15.16:0/3779759895 shutdown_connections 2024-09-10T21:12:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.684+0000 7f2bb3c00640 1 -- 172.21.15.16:0/3779759895 wait complete. 2024-09-10T21:12:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.684+0000 7f2bb3c00640 1 Processor -- start 2024-09-10T21:12:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2bb3c00640 1 -- start start 2024-09-10T21:12:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2bb3c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bac0cd300 0x7f2bac0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2bb3c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac15bdc0 0x7f2bac1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2bb3c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2bac160050 0x7f2bac0708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2bb3c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac16a300 con 0x7f2bac15bdc0 2024-09-10T21:12:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2bb3c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac16a180 con 0x7f2bac0cd300 2024-09-10T21:12:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2bb3c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2bac16a480 con 0x7f2bac160050 2024-09-10T21:12:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2babe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac15bdc0 0x7f2bac1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2babe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac15bdc0 0x7f2bac1571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57080/0 (socket says 172.21.15.16:57080) 2024-09-10T21:12:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2babe00640 1 -- 172.21.15.16:0/2655357917 learned_addr learned my addr 172.21.15.16:0/2655357917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2bb1c00640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2bac160050 0x7f2bac0708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2bb1200640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bac0cd300 0x7f2bac0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2babe00640 1 -- 172.21.15.16:0/2655357917 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2bac160050 msgr2=0x7f2bac0708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2babe00640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2bac160050 0x7f2bac0708d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2babe00640 1 -- 172.21.15.16:0/2655357917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bac0cd300 msgr2=0x7f2bac0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:25.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2babe00640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bac0cd300 0x7f2bac0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:25.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2babe00640 1 -- 172.21.15.16:0/2655357917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bac1fb7d0 con 0x7f2bac15bdc0 2024-09-10T21:12:25.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2bb1200640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bac0cd300 0x7f2bac0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:25.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2babe00640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac15bdc0 0x7f2bac1571a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800c700 tx=0x7f2b9800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:25.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2ba8c00640 1 -- 172.21.15.16:0/2655357917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98016020 con 0x7f2bac15bdc0 2024-09-10T21:12:25.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2ba8c00640 1 -- 172.21.15.16:0/2655357917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b9800a930 con 0x7f2bac15bdc0 2024-09-10T21:12:25.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2bb3c00640 1 -- 172.21.15.16:0/2655357917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bac0ce890 con 0x7f2bac15bdc0 2024-09-10T21:12:25.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2ba8c00640 1 -- 172.21.15.16:0/2655357917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9800ad60 con 0x7f2bac15bdc0 2024-09-10T21:12:25.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2bb3c00640 1 -- 172.21.15.16:0/2655357917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bac0c75b0 con 0x7f2bac15bdc0 2024-09-10T21:12:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.688+0000 7f2bb3c00640 1 -- 172.21.15.16:0/2655357917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bac0c77b0 con 0x7f2bac15bdc0 2024-09-10T21:12:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.692+0000 7f2ba8c00640 1 -- 172.21.15.16:0/2655357917 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b98004020 con 0x7f2bac15bdc0 2024-09-10T21:12:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.692+0000 7f2ba8c00640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b8c072610 0x7f2b8c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.692+0000 7f2ba8c00640 1 -- 172.21.15.16:0/2655357917 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f2b98093390 con 0x7f2bac15bdc0 2024-09-10T21:12:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.692+0000 7f2bb1200640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b8c072610 0x7f2b8c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.692+0000 7f2bb1200640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b8c072610 0x7f2b8c074ad0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0000f70 tx=0x7f2ba0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.696+0000 7f2ba8c00640 1 -- 172.21.15.16:0/2655357917 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2bac0c77b0 con 0x7f2bac15bdc0 2024-09-10T21:12:25.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:25.908+0000 7f2bb3c00640 1 -- 172.21.15.16:0/2655357917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2bac0d2800 con 0x7f2bac15bdc0 2024-09-10T21:12:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.800+0000 7f2ba8c00640 1 -- 172.21.15.16:0/2655357917 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7f2b98020030 con 0x7f2bac15bdc0 2024-09-10T21:12:26.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.800+0000 7f2bb3c00640 1 -- 172.21.15.16:0/2655357917 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b8c072610 msgr2=0x7f2b8c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.800+0000 7f2bb3c00640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b8c072610 0x7f2b8c074ad0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0000f70 tx=0x7f2ba0009040 comp rx=0 tx=0).stop 2024-09-10T21:12:26.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.800+0000 7f2bb3c00640 1 -- 172.21.15.16:0/2655357917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac15bdc0 msgr2=0x7f2bac1571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.800+0000 7f2bb3c00640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac15bdc0 0x7f2bac1571a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800c700 tx=0x7f2b9800cbc0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.800+0000 7f2bb3c00640 1 -- 172.21.15.16:0/2655357917 shutdown_connections 2024-09-10T21:12:26.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.800+0000 7f2bb3c00640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b8c072610 0x7f2b8c074ad0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.800+0000 7f2bb3c00640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2bac160050 0x7f2bac0708d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.800+0000 7f2bb3c00640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac15bdc0 0x7f2bac1571a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.800+0000 7f2bb3c00640 1 --2- 172.21.15.16:0/2655357917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bac0cd300 0x7f2bac0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.800+0000 7f2bb3c00640 1 -- 172.21.15.16:0/2655357917 >> 172.21.15.16:0/2655357917 conn(0x7f2bac0d1ce0 msgr2=0x7f2bac0b55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.800+0000 7f2bb3c00640 1 -- 172.21.15.16:0/2655357917 shutdown_connections 2024-09-10T21:12:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.800+0000 7f2bb3c00640 1 -- 172.21.15.16:0/2655357917 wait complete. 2024-09-10T21:12:26.833 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:12:26.833 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:12:26.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.980+0000 7f67aba00640 1 -- 172.21.15.16:0/2225590067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a406f0e0 msgr2=0x7f67a406f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.980+0000 7f67aba00640 1 --2- 172.21.15.16:0/2225590067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a406f0e0 0x7f67a406f520 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6790009900 tx=0x7f679001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.980+0000 7f67aba00640 1 -- 172.21.15.16:0/2225590067 shutdown_connections 2024-09-10T21:12:26.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.980+0000 7f67aba00640 1 --2- 172.21.15.16:0/2225590067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a406f0e0 0x7f67a406f520 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.980+0000 7f67aba00640 1 --2- 172.21.15.16:0/2225590067 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67a40cbb00 0x7f67a40cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.980+0000 7f67aba00640 1 -- 172.21.15.16:0/2225590067 >> 172.21.15.16:0/2225590067 conn(0x7f67a40b16a0 msgr2=0x7f67a40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.980+0000 7f67aba00640 1 -- 172.21.15.16:0/2225590067 shutdown_connections 2024-09-10T21:12:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.980+0000 7f67aba00640 1 -- 172.21.15.16:0/2225590067 wait complete. 2024-09-10T21:12:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.980+0000 7f67aba00640 1 Processor -- start 2024-09-10T21:12:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67aba00640 1 -- start start 2024-09-10T21:12:26.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67aba00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67a406f0e0 0x7f67a41fb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67aba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a40cbb00 0x7f67a415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67aba00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67a40cbee0 0x7f67a40b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67aba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67a4168d00 con 0x7f67a40cbb00 2024-09-10T21:12:26.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67aba00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f67a4168b80 con 0x7f67a40cbee0 2024-09-10T21:12:26.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67aba00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f67a4168e80 con 0x7f67a406f0e0 2024-09-10T21:12:26.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a40cbb00 0x7f67a415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a40cbb00 0x7f67a415aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57108/0 (socket says 172.21.15.16:57108) 2024-09-10T21:12:26.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a3e00640 1 -- 172.21.15.16:0/773874372 learned_addr learned my addr 172.21.15.16:0/773874372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:26.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a9000640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67a406f0e0 0x7f67a41fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a9a00640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67a40cbee0 0x7f67a40b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a3e00640 1 -- 172.21.15.16:0/773874372 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67a406f0e0 msgr2=0x7f67a41fb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a3e00640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67a406f0e0 0x7f67a41fb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a3e00640 1 -- 172.21.15.16:0/773874372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67a40cbee0 msgr2=0x7f67a40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a3e00640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67a40cbee0 0x7f67a40b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a3e00640 1 -- 172.21.15.16:0/773874372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a41fbab0 con 0x7f67a40cbb00 2024-09-10T21:12:26.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a3e00640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a40cbb00 0x7f67a415aaa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6790009850 tx=0x7f67900a5720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:26.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a0c00640 1 -- 172.21.15.16:0/773874372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67900b2040 con 0x7f67a40cbb00 2024-09-10T21:12:26.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a0c00640 1 -- 172.21.15.16:0/773874372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6790005030 con 0x7f67a40cbb00 2024-09-10T21:12:26.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67aba00640 1 -- 172.21.15.16:0/773874372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a413f000 con 0x7f67a40cbb00 2024-09-10T21:12:26.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67a0c00640 1 -- 172.21.15.16:0/773874372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67900044f0 con 0x7f67a40cbb00 2024-09-10T21:12:26.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67aba00640 1 -- 172.21.15.16:0/773874372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a413f5c0 con 0x7f67a40cbb00 2024-09-10T21:12:26.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.984+0000 7f67aba00640 1 -- 172.21.15.16:0/773874372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a406f520 con 0x7f67a40cbb00 2024-09-10T21:12:26.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.988+0000 7f67a0c00640 1 -- 172.21.15.16:0/773874372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6790005460 con 0x7f67a40cbb00 2024-09-10T21:12:26.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.988+0000 7f67a0c00640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67840725e0 0x7f6784074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.988+0000 7f67a0c00640 1 -- 172.21.15.16:0/773874372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f679012d070 con 0x7f67a40cbb00 2024-09-10T21:12:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.988+0000 7f67a9000640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67840725e0 0x7f6784074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.988+0000 7f67a9000640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67840725e0 0x7f6784074aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6798004630 tx=0x7f679800b390 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:26.992+0000 7f67a0c00640 1 -- 172.21.15.16:0/773874372 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f67900be060 con 0x7f67a40cbb00 2024-09-10T21:12:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.204+0000 7f67aba00640 1 -- 172.21.15.16:0/773874372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f67a413e7f0 con 0x7f67a40cbb00 2024-09-10T21:12:27.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.804+0000 7f67a0c00640 1 -- 172.21.15.16:0/773874372 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f67a41fbab0 con 0x7f67a40cbb00 2024-09-10T21:12:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.808+0000 7f67aba00640 1 -- 172.21.15.16:0/773874372 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67840725e0 msgr2=0x7f6784074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.808+0000 7f67aba00640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67840725e0 0x7f6784074aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6798004630 tx=0x7f679800b390 comp rx=0 tx=0).stop 2024-09-10T21:12:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.808+0000 7f67aba00640 1 -- 172.21.15.16:0/773874372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a40cbb00 msgr2=0x7f67a415aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.808+0000 7f67aba00640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a40cbb00 0x7f67a415aaa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6790009850 tx=0x7f67900a5720 comp rx=0 tx=0).stop 2024-09-10T21:12:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.808+0000 7f67aba00640 1 -- 172.21.15.16:0/773874372 shutdown_connections 2024-09-10T21:12:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.808+0000 7f67aba00640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67840725e0 0x7f6784074aa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.808+0000 7f67aba00640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67a40cbee0 0x7f67a40b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.808+0000 7f67aba00640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a40cbb00 0x7f67a415aaa0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.808+0000 7f67aba00640 1 --2- 172.21.15.16:0/773874372 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67a406f0e0 0x7f67a41fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.808+0000 7f67aba00640 1 -- 172.21.15.16:0/773874372 >> 172.21.15.16:0/773874372 conn(0x7f67a40b16a0 msgr2=0x7f67a406e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.808+0000 7f67aba00640 1 -- 172.21.15.16:0/773874372 shutdown_connections 2024-09-10T21:12:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.808+0000 7f67aba00640 1 -- 172.21.15.16:0/773874372 wait complete. 2024-09-10T21:12:27.838 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:12:27.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 -- 172.21.15.16:0/3380364697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e80b3200 msgr2=0x7fb1e80d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 --2- 172.21.15.16:0/3380364697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e80b3200 0x7fb1e80d01c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009900 tx=0x7fb1d401f070 comp rx=0 tx=0).stop 2024-09-10T21:12:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 -- 172.21.15.16:0/3380364697 shutdown_connections 2024-09-10T21:12:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 --2- 172.21.15.16:0/3380364697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e80b3200 0x7fb1e80d01c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 --2- 172.21.15.16:0/3380364697 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1e815f450 0x7fb1e80cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 -- 172.21.15.16:0/3380364697 >> 172.21.15.16:0/3380364697 conn(0x7fb1e80d1d80 msgr2=0x7fb1e80d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 -- 172.21.15.16:0/3380364697 shutdown_connections 2024-09-10T21:12:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 -- 172.21.15.16:0/3380364697 wait complete. 2024-09-10T21:12:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 Processor -- start 2024-09-10T21:12:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 -- start start 2024-09-10T21:12:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e80d0890 0x7fb1e80ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1e815f450 0x7fb1e80c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e8155c50 0x7fb1e81f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e80b2e30 con 0x7fb1e80d0890 2024-09-10T21:12:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e80b2cb0 con 0x7fb1e8155c50 2024-09-10T21:12:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1efe00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e80b2fb0 con 0x7fb1e815f450 2024-09-10T21:12:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1ed400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e80d0890 0x7fb1e80ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1ed400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e80d0890 0x7fb1e80ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57114/0 (socket says 172.21.15.16:57114) 2024-09-10T21:12:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1ed400640 1 -- 172.21.15.16:0/1983664407 learned_addr learned my addr 172.21.15.16:0/1983664407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1ede00640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e8155c50 0x7fb1e81f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1eca00640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1e815f450 0x7fb1e80c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1ed400640 1 -- 172.21.15.16:0/1983664407 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1e815f450 msgr2=0x7fb1e80c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1ed400640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1e815f450 0x7fb1e80c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1ed400640 1 -- 172.21.15.16:0/1983664407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e8155c50 msgr2=0x7fb1e81f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1ed400640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e8155c50 0x7fb1e81f3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1ed400640 1 -- 172.21.15.16:0/1983664407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e80c8af0 con 0x7fb1e80d0890 2024-09-10T21:12:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1eca00640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1e815f450 0x7fb1e80c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.984+0000 7fb1ede00640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e8155c50 0x7fb1e81f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.988+0000 7fb1ed400640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e80d0890 0x7fb1e80ce2d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00d950 tx=0x7fb1dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.988+0000 7fb1e5600640 1 -- 172.21.15.16:0/1983664407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc017070 con 0x7fb1e80d0890 2024-09-10T21:12:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.988+0000 7fb1e5600640 1 -- 172.21.15.16:0/1983664407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1dc009d20 con 0x7fb1e80d0890 2024-09-10T21:12:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.988+0000 7fb1efe00640 1 -- 172.21.15.16:0/1983664407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e80ce980 con 0x7fb1e80d0890 2024-09-10T21:12:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.988+0000 7fb1e5600640 1 -- 172.21.15.16:0/1983664407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc0106c0 con 0x7fb1e80d0890 2024-09-10T21:12:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.988+0000 7fb1efe00640 1 -- 172.21.15.16:0/1983664407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e80cf1c0 con 0x7fb1e80d0890 2024-09-10T21:12:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.988+0000 7fb1efe00640 1 -- 172.21.15.16:0/1983664407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e80b1e90 con 0x7fb1e80d0890 2024-09-10T21:12:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.988+0000 7fb1e5600640 1 -- 172.21.15.16:0/1983664407 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb1dc010860 con 0x7fb1e80d0890 2024-09-10T21:12:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.988+0000 7fb1e5600640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1c80725e0 0x7fb1c8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.988+0000 7fb1e5600640 1 -- 172.21.15.16:0/1983664407 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fb1dc0937e0 con 0x7fb1e80d0890 2024-09-10T21:12:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.988+0000 7fb1eca00640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1c80725e0 0x7fb1c8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.992+0000 7fb1eca00640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1c80725e0 0x7fb1c8074aa0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40b2000 tx=0x7fb1d401d910 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:27.992+0000 7fb1e5600640 1 -- 172.21.15.16:0/1983664407 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb1e80ce980 con 0x7fb1e80d0890 2024-09-10T21:12:28.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.204+0000 7fb1efe00640 1 -- 172.21.15.16:0/1983664407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1e8074ba0 con 0x7fb1e80d0890 2024-09-10T21:12:28.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.204+0000 7fb1e5600640 1 -- 172.21.15.16:0/1983664407 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5235 (secure 0 0 0) 0x7fb1e80c8af0 con 0x7fb1e80d0890 2024-09-10T21:12:28.212 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:28.213 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-10T21:12:27:803754+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3694613625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3694613625},{"type":"v1","addr":"172.21.15.16:6836","nonce":3694613625}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1264024538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1264024538},{"type":"v1","addr":"172.21.15.52:6835","nonce":1264024538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":134768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2899191048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2899191048},{"type":"v1","addr":"172.21.15.16:6837","nonce":2899191048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:12:24.201810+0000","modified":"2024-09-10T21:12:27.207587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134771":{"gid":134771,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1834750661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1834750661},{"type":"v1","addr":"172.21.15.52:6834","nonce":1834750661}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134771,"qdb_cluster":[134771]},"id":3}]} 2024-09-10T21:12:28.213 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-10T21:12:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.208+0000 7fb1efe00640 1 -- 172.21.15.16:0/1983664407 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1c80725e0 msgr2=0x7fb1c8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.208+0000 7fb1efe00640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1c80725e0 0x7fb1c8074aa0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40b2000 tx=0x7fb1d401d910 comp rx=0 tx=0).stop 2024-09-10T21:12:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.208+0000 7fb1efe00640 1 -- 172.21.15.16:0/1983664407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e80d0890 msgr2=0x7fb1e80ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.208+0000 7fb1efe00640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e80d0890 0x7fb1e80ce2d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00d950 tx=0x7fb1dc00de10 comp rx=0 tx=0).stop 2024-09-10T21:12:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.208+0000 7fb1efe00640 1 -- 172.21.15.16:0/1983664407 shutdown_connections 2024-09-10T21:12:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.208+0000 7fb1efe00640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb1c80725e0 0x7fb1c8074aa0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.208+0000 7fb1efe00640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1e8155c50 0x7fb1e81f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.208+0000 7fb1efe00640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1e815f450 0x7fb1e80c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.208+0000 7fb1efe00640 1 --2- 172.21.15.16:0/1983664407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1e80d0890 0x7fb1e80ce2d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.208+0000 7fb1efe00640 1 -- 172.21.15.16:0/1983664407 >> 172.21.15.16:0/1983664407 conn(0x7fb1e80d1d80 msgr2=0x7fb1e815da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.208+0000 7fb1efe00640 1 -- 172.21.15.16:0/1983664407 shutdown_connections 2024-09-10T21:12:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.208+0000 7fb1efe00640 1 -- 172.21.15.16:0/1983664407 wait complete. 2024-09-10T21:12:28.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:12:28.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 -- 172.21.15.16:0/1846048135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b340d06b0 msgr2=0x7f6b340700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:28.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 --2- 172.21.15.16:0/1846048135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b340d06b0 0x7f6b340700a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6b3406e1a0 tx=0x7f6b2001d1a0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 -- 172.21.15.16:0/1846048135 shutdown_connections 2024-09-10T21:12:28.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 --2- 172.21.15.16:0/1846048135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b340d06b0 0x7f6b340700a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 --2- 172.21.15.16:0/1846048135 >> v2:172.21.15.52:3300/0 conn(0x7f6b3415a270 0x7f6b340ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 -- 172.21.15.16:0/1846048135 >> 172.21.15.16:0/1846048135 conn(0x7f6b340cfb20 msgr2=0x7f6b340cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 -- 172.21.15.16:0/1846048135 shutdown_connections 2024-09-10T21:12:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 -- 172.21.15.16:0/1846048135 wait complete. 2024-09-10T21:12:28.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 Processor -- start 2024-09-10T21:12:28.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 -- start start 2024-09-10T21:12:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6b340705e0 0x7f6b341533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b340d06b0 0x7f6b34157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3415a270 0x7f6b34157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34072b30 con 0x7f6b340d06b0 2024-09-10T21:12:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6b340729b0 con 0x7f6b3415a270 2024-09-10T21:12:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b3ba00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6b34072cb0 con 0x7f6b340705e0 2024-09-10T21:12:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b340d06b0 0x7f6b34157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b340d06b0 0x7f6b34157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57136/0 (socket says 172.21.15.16:57136) 2024-09-10T21:12:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.376+0000 7f6b33e00640 1 -- 172.21.15.16:0/2792981294 learned_addr learned my addr 172.21.15.16:0/2792981294 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b39000640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6b340705e0 0x7f6b341533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b39a00640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3415a270 0x7f6b34157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b33e00640 1 -- 172.21.15.16:0/2792981294 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6b340705e0 msgr2=0x7f6b341533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b33e00640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6b340705e0 0x7f6b341533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b33e00640 1 -- 172.21.15.16:0/2792981294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3415a270 msgr2=0x7f6b34157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b33e00640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3415a270 0x7f6b34157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b33e00640 1 -- 172.21.15.16:0/2792981294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b34153920 con 0x7f6b340d06b0 2024-09-10T21:12:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b33e00640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b340d06b0 0x7f6b34157870 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6b20000c10 tx=0x7f6b200a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b30c00640 1 -- 172.21.15.16:0/2792981294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b200bc070 con 0x7f6b340d06b0 2024-09-10T21:12:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b30c00640 1 -- 172.21.15.16:0/2792981294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b200043b0 con 0x7f6b340d06b0 2024-09-10T21:12:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b30c00640 1 -- 172.21.15.16:0/2792981294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b200b2040 con 0x7f6b340d06b0 2024-09-10T21:12:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b3ba00640 1 -- 172.21.15.16:0/2792981294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b341543b0 con 0x7f6b340d06b0 2024-09-10T21:12:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b3ba00640 1 -- 172.21.15.16:0/2792981294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b340c9600 con 0x7f6b340d06b0 2024-09-10T21:12:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b3ba00640 1 -- 172.21.15.16:0/2792981294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b340700a0 con 0x7f6b340d06b0 2024-09-10T21:12:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b30c00640 1 -- 172.21.15.16:0/2792981294 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6b200b5070 con 0x7f6b340d06b0 2024-09-10T21:12:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.380+0000 7f6b30c00640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b140725e0 0x7f6b14074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.384+0000 7f6b30c00640 1 -- 172.21.15.16:0/2792981294 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f6b2012c910 con 0x7f6b340d06b0 2024-09-10T21:12:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.384+0000 7f6b39000640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b140725e0 0x7f6b14074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.384+0000 7f6b39000640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b140725e0 0x7f6b14074aa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800ab40 tx=0x7f6b2800a2f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.384+0000 7f6b30c00640 1 -- 172.21.15.16:0/2792981294 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6b341543b0 con 0x7f6b340d06b0 2024-09-10T21:12:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.560+0000 7f6b3ba00640 1 -- 172.21.15.16:0/2792981294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b340b5aa0 con 0x7f6b340d06b0 2024-09-10T21:12:28.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.560+0000 7f6b30c00640 1 -- 172.21.15.16:0/2792981294 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16621 (secure 0 0 0) 0x7f6b200163f0 con 0x7f6b340d06b0 2024-09-10T21:12:28.569 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:28.569 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:25.780513+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:12:22.642253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T21:12:23.151552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:12:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.564+0000 7f6b3ba00640 1 -- 172.21.15.16:0/2792981294 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b140725e0 msgr2=0x7f6b14074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.564+0000 7f6b3ba00640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b140725e0 0x7f6b14074aa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800ab40 tx=0x7f6b2800a2f0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.564+0000 7f6b3ba00640 1 -- 172.21.15.16:0/2792981294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b340d06b0 msgr2=0x7f6b34157870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.564+0000 7f6b3ba00640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b340d06b0 0x7f6b34157870 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6b20000c10 tx=0x7f6b200a5620 comp rx=0 tx=0).stop 2024-09-10T21:12:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.564+0000 7f6b3ba00640 1 -- 172.21.15.16:0/2792981294 shutdown_connections 2024-09-10T21:12:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.564+0000 7f6b3ba00640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b140725e0 0x7f6b14074aa0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.564+0000 7f6b3ba00640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3415a270 0x7f6b34157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.564+0000 7f6b3ba00640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b340d06b0 0x7f6b34157870 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.564+0000 7f6b3ba00640 1 --2- 172.21.15.16:0/2792981294 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6b340705e0 0x7f6b341533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.564+0000 7f6b3ba00640 1 -- 172.21.15.16:0/2792981294 >> 172.21.15.16:0/2792981294 conn(0x7f6b340cfb20 msgr2=0x7f6b340b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.564+0000 7f6b3ba00640 1 -- 172.21.15.16:0/2792981294 shutdown_connections 2024-09-10T21:12:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.564+0000 7f6b3ba00640 1 -- 172.21.15.16:0/2792981294 wait complete. 2024-09-10T21:12:28.591 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:12:28.591 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:12:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.732+0000 7f42f6400640 1 -- 172.21.15.16:0/4070847716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d4004810 msgr2=0x7f42d4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.732+0000 7f42f6400640 1 --2- 172.21.15.16:0/4070847716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d4004810 0x7f42d4004c90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009b60 tx=0x7f42d801d2b0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.732+0000 7f42efe00640 1 -- 172.21.15.16:0/4070847716 reap_dead start 2024-09-10T21:12:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.732+0000 7f42f6400640 1 -- 172.21.15.16:0/4070847716 shutdown_connections 2024-09-10T21:12:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.732+0000 7f42f6400640 1 -- 172.21.15.16:0/4070847716 >> 172.21.15.16:0/4070847716 conn(0x7f42f00cfb20 msgr2=0x7f42f00cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.732+0000 7f42f6400640 1 -- 172.21.15.16:0/4070847716 shutdown_connections 2024-09-10T21:12:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.732+0000 7f42f6400640 1 -- 172.21.15.16:0/4070847716 wait complete. 2024-09-10T21:12:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42f6400640 1 Processor -- start 2024-09-10T21:12:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42f6400640 1 -- start start 2024-09-10T21:12:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42f6400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f42d4004810 0x7f42f014d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42f6400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f00c90b0 0x7f42f00c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42f6400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f014f8d0 0x7f42f014fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42f6400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42f006f260 con 0x7f42f014f8d0 2024-09-10T21:12:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42f6400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f42f006f0e0 con 0x7f42f00c90b0 2024-09-10T21:12:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42f6400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f42f006f3e0 con 0x7f42d4004810 2024-09-10T21:12:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f014f8d0 0x7f42f014fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f014f8d0 0x7f42f014fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57150/0 (socket says 172.21.15.16:57150) 2024-09-10T21:12:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42efe00640 1 -- 172.21.15.16:0/1596517894 learned_addr learned my addr 172.21.15.16:0/1596517894 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42e7e00640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f00c90b0 0x7f42f00c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42ef400640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f42d4004810 0x7f42f014d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42efe00640 1 -- 172.21.15.16:0/1596517894 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f42d4004810 msgr2=0x7f42f014d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42efe00640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f42d4004810 0x7f42f014d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42efe00640 1 -- 172.21.15.16:0/1596517894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f00c90b0 msgr2=0x7f42f00c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42efe00640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f00c90b0 0x7f42f00c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42efe00640 1 -- 172.21.15.16:0/1596517894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f00c9a70 con 0x7f42f014f8d0 2024-09-10T21:12:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42ef400640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f42d4004810 0x7f42f014d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42efe00640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f014f8d0 0x7f42f014fd50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f42f00c3350 tx=0x7f42e800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42e7400640 1 -- 172.21.15.16:0/1596517894 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8017070 con 0x7f42f014f8d0 2024-09-10T21:12:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42e7400640 1 -- 172.21.15.16:0/1596517894 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42e8015070 con 0x7f42f014f8d0 2024-09-10T21:12:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42e7400640 1 -- 172.21.15.16:0/1596517894 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e800eea0 con 0x7f42f014f8d0 2024-09-10T21:12:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42f6400640 1 -- 172.21.15.16:0/1596517894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f014db00 con 0x7f42f014f8d0 2024-09-10T21:12:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42f6400640 1 -- 172.21.15.16:0/1596517894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f014e0c0 con 0x7f42f014f8d0 2024-09-10T21:12:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.736+0000 7f42f6400640 1 -- 172.21.15.16:0/1596517894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f0072d40 con 0x7f42f014f8d0 2024-09-10T21:12:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.740+0000 7f42e7400640 1 -- 172.21.15.16:0/1596517894 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f42e8005280 con 0x7f42f014f8d0 2024-09-10T21:12:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.740+0000 7f42e7400640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42cc072610 0x7f42cc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.740+0000 7f42e7400640 1 -- 172.21.15.16:0/1596517894 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f42e8093750 con 0x7f42f014f8d0 2024-09-10T21:12:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.740+0000 7f42ef400640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42cc072610 0x7f42cc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.740+0000 7f42ef400640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42cc072610 0x7f42cc074ad0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f42e0000fc0 tx=0x7f42e0001ba0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:28.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.744+0000 7f42e7400640 1 -- 172.21.15.16:0/1596517894 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f42e8022030 con 0x7f42f014f8d0 2024-09-10T21:12:28.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.916+0000 7f42f6400640 1 -- 172.21.15.16:0/1596517894 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42f014f640 con 0x7f42cc072610 2024-09-10T21:12:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.920+0000 7f42e7400640 1 -- 172.21.15.16:0/1596517894 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245278 (secure 0 0 0) 0x7f42f014f640 con 0x7f42cc072610 2024-09-10T21:12:28.932 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:12:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.924+0000 7f42f6400640 1 -- 172.21.15.16:0/1596517894 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42cc072610 msgr2=0x7f42cc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.924+0000 7f42f6400640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42cc072610 0x7f42cc074ad0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f42e0000fc0 tx=0x7f42e0001ba0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.924+0000 7f42f6400640 1 -- 172.21.15.16:0/1596517894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f014f8d0 msgr2=0x7f42f014fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.924+0000 7f42f6400640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f014f8d0 0x7f42f014fd50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f42f00c3350 tx=0x7f42e800e720 comp rx=0 tx=0).stop 2024-09-10T21:12:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.924+0000 7f42f6400640 1 -- 172.21.15.16:0/1596517894 shutdown_connections 2024-09-10T21:12:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.924+0000 7f42f6400640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42cc072610 0x7f42cc074ad0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.924+0000 7f42f6400640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f014f8d0 0x7f42f014fd50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.924+0000 7f42f6400640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f00c90b0 0x7f42f00c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.924+0000 7f42f6400640 1 --2- 172.21.15.16:0/1596517894 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f42d4004810 0x7f42f014d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.924+0000 7f42f6400640 1 -- 172.21.15.16:0/1596517894 >> 172.21.15.16:0/1596517894 conn(0x7f42f00cfb20 msgr2=0x7f42f00cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.924+0000 7f42f6400640 1 -- 172.21.15.16:0/1596517894 shutdown_connections 2024-09-10T21:12:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:28.924+0000 7f42f6400640 1 -- 172.21.15.16:0/1596517894 wait complete. 2024-09-10T21:12:28.951 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-10T21:12:27.483412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":168,"acting":168,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254536,"kb_used_data":5008,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720184,"statfs":{"total":773094113280,"available":772833468416,"internally_reserved":0,"allocated":5128192,"data_stored":2746650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.710827"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787978+0000","last_change":"2024-09-10T21:12:23.787978+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787978+0000","last_undegraded":"2024-09-10T21:12:23.787978+0000","last_fullsized":"2024-09-10T21:12:23.787978+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787245+0000","last_change":"2024-09-10T21:12:23.787245+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787245+0000","last_undegraded":"2024-09-10T21:12:23.787245+0000","last_fullsized":"2024-09-10T21:12:23.787245+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782754+0000","last_change":"2024-09-10T21:12:23.782754+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782754+0000","last_undegraded":"2024-09-10T21:12:23.782754+0000","last_fullsized":"2024-09-10T21:12:23.782754+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.783698+0000","last_change":"2024-09-10T21:12:23.783698+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.783698+0000","last_undegraded":"2024-09-10T21:12:23.783698+0000","last_fullsized":"2024-09-10T21:12:23.783698+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.791512+0000","last_change":"2024-09-10T21:12:23.791512+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.791512+0000","last_undegraded":"2024-09-10T21:12:23.791512+0000","last_fullsized":"2024-09-10T21:12:23.791512+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.790560+0000","last_change":"2024-09-10T21:12:23.790560+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.790560+0000","last_undegraded":"2024-09-10T21:12:23.790560+0000","last_fullsized":"2024-09-10T21:12:23.790560+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.784962+0000","last_change":"2024-09-10T21:12:23.784962+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.784962+0000","last_undegraded":"2024-09-10T21:12:23.784962+0000","last_fullsized":"2024-09-10T21:12:23.784962+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.789970+0000","last_change":"2024-09-10T21:12:23.789970+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.789970+0000","last_undegraded":"2024-09-10T21:12:23.789970+0000","last_fullsized":"2024-09-10T21:12:23.789970+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.788313+0000","last_change":"2024-09-10T21:12:23.788313+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.788313+0000","last_undegraded":"2024-09-10T21:12:23.788313+0000","last_fullsized":"2024-09-10T21:12:23.788313+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.783286+0000","last_change":"2024-09-10T21:12:23.783286+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.783286+0000","last_undegraded":"2024-09-10T21:12:23.783286+0000","last_fullsized":"2024-09-10T21:12:23.783286+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.788387+0000","last_change":"2024-09-10T21:12:23.788387+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.788387+0000","last_undegraded":"2024-09-10T21:12:23.788387+0000","last_fullsized":"2024-09-10T21:12:23.788387+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.791309+0000","last_change":"2024-09-10T21:12:23.791309+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.791309+0000","last_undegraded":"2024-09-10T21:12:23.791309+0000","last_fullsized":"2024-09-10T21:12:23.791309+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.781799+0000","last_change":"2024-09-10T21:12:23.781799+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.781799+0000","last_undegraded":"2024-09-10T21:12:23.781799+0000","last_fullsized":"2024-09-10T21:12:23.781799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.783664+0000","last_change":"2024-09-10T21:12:23.783664+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.783664+0000","last_undegraded":"2024-09-10T21:12:23.783664+0000","last_fullsized":"2024-09-10T21:12:23.783664+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.784042+0000","last_change":"2024-09-10T21:12:23.784042+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.784042+0000","last_undegraded":"2024-09-10T21:12:23.784042+0000","last_fullsized":"2024-09-10T21:12:23.784042+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.789083+0000","last_change":"2024-09-10T21:12:23.789083+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.789083+0000","last_undegraded":"2024-09-10T21:12:23.789083+0000","last_fullsized":"2024-09-10T21:12:23.789083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.783192+0000","last_change":"2024-09-10T21:12:23.783192+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.783192+0000","last_undegraded":"2024-09-10T21:12:23.783192+0000","last_fullsized":"2024-09-10T21:12:23.783192+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787963+0000","last_change":"2024-09-10T21:12:23.787963+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787963+0000","last_undegraded":"2024-09-10T21:12:23.787963+0000","last_fullsized":"2024-09-10T21:12:23.787963+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.781815+0000","last_change":"2024-09-10T21:12:23.781815+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.781815+0000","last_undegraded":"2024-09-10T21:12:23.781815+0000","last_fullsized":"2024-09-10T21:12:23.781815+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.789677+0000","last_change":"2024-09-10T21:12:23.789677+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.789677+0000","last_undegraded":"2024-09-10T21:12:23.789677+0000","last_fullsized":"2024-09-10T21:12:23.789677+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787835+0000","last_change":"2024-09-10T21:12:23.787835+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787835+0000","last_undegraded":"2024-09-10T21:12:23.787835+0000","last_fullsized":"2024-09-10T21:12:23.787835+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.785962+0000","last_change":"2024-09-10T21:12:23.785962+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.785962+0000","last_undegraded":"2024-09-10T21:12:23.785962+0000","last_fullsized":"2024-09-10T21:12:23.785962+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782740+0000","last_change":"2024-09-10T21:12:23.782740+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782740+0000","last_undegraded":"2024-09-10T21:12:23.782740+0000","last_fullsized":"2024-09-10T21:12:23.782740+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.790668+0000","last_change":"2024-09-10T21:12:23.790668+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.790668+0000","last_undegraded":"2024-09-10T21:12:23.790668+0000","last_fullsized":"2024-09-10T21:12:23.790668+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.789308+0000","last_change":"2024-09-10T21:12:23.789308+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.789308+0000","last_undegraded":"2024-09-10T21:12:23.789308+0000","last_fullsized":"2024-09-10T21:12:23.789308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.783262+0000","last_change":"2024-09-10T21:12:23.783262+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.783262+0000","last_undegraded":"2024-09-10T21:12:23.783262+0000","last_fullsized":"2024-09-10T21:12:23.783262+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787684+0000","last_change":"2024-09-10T21:12:23.787684+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787684+0000","last_undegraded":"2024-09-10T21:12:23.787684+0000","last_fullsized":"2024-09-10T21:12:23.787684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.790234+0000","last_change":"2024-09-10T21:12:23.790234+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.790234+0000","last_undegraded":"2024-09-10T21:12:23.790234+0000","last_fullsized":"2024-09-10T21:12:23.790234+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.784473+0000","last_change":"2024-09-10T21:12:23.784473+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.784473+0000","last_undegraded":"2024-09-10T21:12:23.784473+0000","last_fullsized":"2024-09-10T21:12:23.784473+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.798094+0000","last_change":"2024-09-10T21:12:23.795968+0000","last_active":"2024-09-10T21:12:23.798094+0000","last_peered":"2024-09-10T21:12:23.798094+0000","last_clean":"2024-09-10T21:12:23.798094+0000","last_became_active":"2024-09-10T21:12:23.795674+0000","last_became_peered":"2024-09-10T21:12:23.795674+0000","last_unstale":"2024-09-10T21:12:23.798094+0000","last_undegraded":"2024-09-10T21:12:23.798094+0000","last_fullsized":"2024-09-10T21:12:23.798094+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:07.666553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782603+0000","last_change":"2024-09-10T21:12:23.782603+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782603+0000","last_undegraded":"2024-09-10T21:12:23.782603+0000","last_fullsized":"2024-09-10T21:12:23.782603+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803757+0000","last_change":"2024-09-10T21:12:23.799107+0000","last_active":"2024-09-10T21:12:23.803757+0000","last_peered":"2024-09-10T21:12:23.803757+0000","last_clean":"2024-09-10T21:12:23.803757+0000","last_became_active":"2024-09-10T21:12:23.798815+0000","last_became_peered":"2024-09-10T21:12:23.798815+0000","last_unstale":"2024-09-10T21:12:23.803757+0000","last_undegraded":"2024-09-10T21:12:23.803757+0000","last_fullsized":"2024-09-10T21:12:23.803757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:58.625840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787946+0000","last_change":"2024-09-10T21:12:23.787946+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787946+0000","last_undegraded":"2024-09-10T21:12:23.787946+0000","last_fullsized":"2024-09-10T21:12:23.787946+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.797239+0000","last_change":"2024-09-10T21:12:23.795368+0000","last_active":"2024-09-10T21:12:23.797239+0000","last_peered":"2024-09-10T21:12:23.797239+0000","last_clean":"2024-09-10T21:12:23.797239+0000","last_became_active":"2024-09-10T21:12:23.795086+0000","last_became_peered":"2024-09-10T21:12:23.795086+0000","last_unstale":"2024-09-10T21:12:23.797239+0000","last_undegraded":"2024-09-10T21:12:23.797239+0000","last_fullsized":"2024-09-10T21:12:23.797239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:01.506279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782629+0000","last_change":"2024-09-10T21:12:23.782629+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782629+0000","last_undegraded":"2024-09-10T21:12:23.782629+0000","last_fullsized":"2024-09-10T21:12:23.782629+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803772+0000","last_change":"2024-09-10T21:12:23.799265+0000","last_active":"2024-09-10T21:12:23.803772+0000","last_peered":"2024-09-10T21:12:23.803772+0000","last_clean":"2024-09-10T21:12:23.803772+0000","last_became_active":"2024-09-10T21:12:23.798990+0000","last_became_peered":"2024-09-10T21:12:23.798990+0000","last_unstale":"2024-09-10T21:12:23.803772+0000","last_undegraded":"2024-09-10T21:12:23.803772+0000","last_fullsized":"2024-09-10T21:12:23.803772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:10.371918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.788203+0000","last_change":"2024-09-10T21:12:23.788203+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.788203+0000","last_undegraded":"2024-09-10T21:12:23.788203+0000","last_fullsized":"2024-09-10T21:12:23.788203+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.802217+0000","last_change":"2024-09-10T21:12:23.798175+0000","last_active":"2024-09-10T21:12:23.802217+0000","last_peered":"2024-09-10T21:12:23.802217+0000","last_clean":"2024-09-10T21:12:23.802217+0000","last_became_active":"2024-09-10T21:12:23.797844+0000","last_became_peered":"2024-09-10T21:12:23.797844+0000","last_unstale":"2024-09-10T21:12:23.802217+0000","last_undegraded":"2024-09-10T21:12:23.802217+0000","last_fullsized":"2024-09-10T21:12:23.802217+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:17.620823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787187+0000","last_change":"2024-09-10T21:12:23.787187+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787187+0000","last_undegraded":"2024-09-10T21:12:23.787187+0000","last_fullsized":"2024-09-10T21:12:23.787187+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.808390+0000","last_change":"2024-09-10T21:12:23.803893+0000","last_active":"2024-09-10T21:12:23.808390+0000","last_peered":"2024-09-10T21:12:23.808390+0000","last_clean":"2024-09-10T21:12:23.808390+0000","last_became_active":"2024-09-10T21:12:23.803599+0000","last_became_peered":"2024-09-10T21:12:23.803599+0000","last_unstale":"2024-09-10T21:12:23.808390+0000","last_undegraded":"2024-09-10T21:12:23.808390+0000","last_fullsized":"2024-09-10T21:12:23.808390+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:07.284335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787410+0000","last_change":"2024-09-10T21:12:23.787410+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787410+0000","last_undegraded":"2024-09-10T21:12:23.787410+0000","last_fullsized":"2024-09-10T21:12:23.787410+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.795748+0000","last_change":"2024-09-10T21:12:23.793275+0000","last_active":"2024-09-10T21:12:23.795748+0000","last_peered":"2024-09-10T21:12:23.795748+0000","last_clean":"2024-09-10T21:12:23.795748+0000","last_became_active":"2024-09-10T21:12:23.792867+0000","last_became_peered":"2024-09-10T21:12:23.792867+0000","last_unstale":"2024-09-10T21:12:23.795748+0000","last_undegraded":"2024-09-10T21:12:23.795748+0000","last_fullsized":"2024-09-10T21:12:23.795748+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:25.165637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.790132+0000","last_change":"2024-09-10T21:12:23.790132+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.790132+0000","last_undegraded":"2024-09-10T21:12:23.790132+0000","last_fullsized":"2024-09-10T21:12:23.790132+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803065+0000","last_change":"2024-09-10T21:12:23.798078+0000","last_active":"2024-09-10T21:12:23.803065+0000","last_peered":"2024-09-10T21:12:23.803065+0000","last_clean":"2024-09-10T21:12:23.803065+0000","last_became_active":"2024-09-10T21:12:23.797773+0000","last_became_peered":"2024-09-10T21:12:23.797773+0000","last_unstale":"2024-09-10T21:12:23.803065+0000","last_undegraded":"2024-09-10T21:12:23.803065+0000","last_fullsized":"2024-09-10T21:12:23.803065+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:39.813884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-10T21:12:22.779766+0000","last_change":"2024-09-10T21:12:22.779766+0000","last_active":"2024-09-10T21:12:22.764860+0000","last_peered":"2024-09-10T21:12:22.764860+0000","last_clean":"2024-09-10T21:12:22.764860+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:22.779766+0000","last_undegraded":"2024-09-10T21:12:22.779766+0000","last_fullsized":"2024-09-10T21:12:22.779766+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.808783+0000","last_change":"2024-09-10T21:12:23.807246+0000","last_active":"2024-09-10T21:12:23.808783+0000","last_peered":"2024-09-10T21:12:23.808783+0000","last_clean":"2024-09-10T21:12:23.808783+0000","last_became_active":"2024-09-10T21:12:23.806959+0000","last_became_peered":"2024-09-10T21:12:23.806959+0000","last_unstale":"2024-09-10T21:12:23.808783+0000","last_undegraded":"2024-09-10T21:12:23.808783+0000","last_fullsized":"2024-09-10T21:12:23.808783+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:45.079126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782139+0000","last_change":"2024-09-10T21:12:23.782139+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782139+0000","last_undegraded":"2024-09-10T21:12:23.782139+0000","last_fullsized":"2024-09-10T21:12:23.782139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.797152+0000","last_change":"2024-09-10T21:12:23.790411+0000","last_active":"2024-09-10T21:12:23.797152+0000","last_peered":"2024-09-10T21:12:23.797152+0000","last_clean":"2024-09-10T21:12:23.797152+0000","last_became_active":"2024-09-10T21:12:23.790109+0000","last_became_peered":"2024-09-10T21:12:23.790109+0000","last_unstale":"2024-09-10T21:12:23.797152+0000","last_undegraded":"2024-09-10T21:12:23.797152+0000","last_fullsized":"2024-09-10T21:12:23.797152+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:48.351796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.790243+0000","last_change":"2024-09-10T21:12:23.790243+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.790243+0000","last_undegraded":"2024-09-10T21:12:23.790243+0000","last_fullsized":"2024-09-10T21:12:23.790243+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803914+0000","last_change":"2024-09-10T21:12:23.802815+0000","last_active":"2024-09-10T21:12:23.803914+0000","last_peered":"2024-09-10T21:12:23.803914+0000","last_clean":"2024-09-10T21:12:23.803914+0000","last_became_active":"2024-09-10T21:12:23.802540+0000","last_became_peered":"2024-09-10T21:12:23.802540+0000","last_unstale":"2024-09-10T21:12:23.803914+0000","last_undegraded":"2024-09-10T21:12:23.803914+0000","last_fullsized":"2024-09-10T21:12:23.803914+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:44.108373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-10T21:12:22.780457+0000","last_change":"2024-09-10T21:12:22.780457+0000","last_active":"2024-09-10T21:12:22.764860+0000","last_peered":"2024-09-10T21:12:22.764860+0000","last_clean":"2024-09-10T21:12:22.764860+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:22.780457+0000","last_undegraded":"2024-09-10T21:12:22.780457+0000","last_fullsized":"2024-09-10T21:12:22.780457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.807879+0000","last_change":"2024-09-10T21:12:23.800966+0000","last_active":"2024-09-10T21:12:23.807879+0000","last_peered":"2024-09-10T21:12:23.807879+0000","last_clean":"2024-09-10T21:12:23.807879+0000","last_became_active":"2024-09-10T21:12:23.800647+0000","last_became_peered":"2024-09-10T21:12:23.800647+0000","last_unstale":"2024-09-10T21:12:23.807879+0000","last_undegraded":"2024-09-10T21:12:23.807879+0000","last_fullsized":"2024-09-10T21:12:23.807879+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:10.581708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.808518+0000","last_change":"2024-09-10T21:12:23.806810+0000","last_active":"2024-09-10T21:12:23.808518+0000","last_peered":"2024-09-10T21:12:23.808518+0000","last_clean":"2024-09-10T21:12:23.808518+0000","last_became_active":"2024-09-10T21:12:23.806235+0000","last_became_peered":"2024-09-10T21:12:23.806235+0000","last_unstale":"2024-09-10T21:12:23.808518+0000","last_undegraded":"2024-09-10T21:12:23.808518+0000","last_fullsized":"2024-09-10T21:12:23.808518+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:56.825780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.786442+0000","last_change":"2024-09-10T21:12:23.786442+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.786442+0000","last_undegraded":"2024-09-10T21:12:23.786442+0000","last_fullsized":"2024-09-10T21:12:23.786442+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803041+0000","last_change":"2024-09-10T21:12:23.796942+0000","last_active":"2024-09-10T21:12:23.803041+0000","last_peered":"2024-09-10T21:12:23.803041+0000","last_clean":"2024-09-10T21:12:23.803041+0000","last_became_active":"2024-09-10T21:12:23.796661+0000","last_became_peered":"2024-09-10T21:12:23.796661+0000","last_unstale":"2024-09-10T21:12:23.803041+0000","last_undegraded":"2024-09-10T21:12:23.803041+0000","last_fullsized":"2024-09-10T21:12:23.803041+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:34.156674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.799963+0000","last_change":"2024-09-10T21:12:23.797369+0000","last_active":"2024-09-10T21:12:23.799963+0000","last_peered":"2024-09-10T21:12:23.799963+0000","last_clean":"2024-09-10T21:12:23.799963+0000","last_became_active":"2024-09-10T21:12:23.796137+0000","last_became_peered":"2024-09-10T21:12:23.796137+0000","last_unstale":"2024-09-10T21:12:23.799963+0000","last_undegraded":"2024-09-10T21:12:23.799963+0000","last_fullsized":"2024-09-10T21:12:23.799963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:34.576509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-10T21:12:22.779809+0000","last_change":"2024-09-10T21:12:22.779809+0000","last_active":"2024-09-10T21:12:22.764860+0000","last_peered":"2024-09-10T21:12:22.764860+0000","last_clean":"2024-09-10T21:12:22.764860+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:22.779809+0000","last_undegraded":"2024-09-10T21:12:22.779809+0000","last_fullsized":"2024-09-10T21:12:22.779809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.785775+0000","last_change":"2024-09-10T21:12:23.785775+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.785775+0000","last_undegraded":"2024-09-10T21:12:23.785775+0000","last_fullsized":"2024-09-10T21:12:23.785775+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-10T21:12:22.779822+0000","last_change":"2024-09-10T21:12:22.779822+0000","last_active":"2024-09-10T21:12:22.764860+0000","last_peered":"2024-09-10T21:12:22.764860+0000","last_clean":"2024-09-10T21:12:22.764860+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:22.779822+0000","last_undegraded":"2024-09-10T21:12:22.779822+0000","last_fullsized":"2024-09-10T21:12:22.779822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.807611+0000","last_change":"2024-09-10T21:12:23.806377+0000","last_active":"2024-09-10T21:12:23.807611+0000","last_peered":"2024-09-10T21:12:23.807611+0000","last_clean":"2024-09-10T21:12:23.807611+0000","last_became_active":"2024-09-10T21:12:23.806032+0000","last_became_peered":"2024-09-10T21:12:23.806032+0000","last_unstale":"2024-09-10T21:12:23.807611+0000","last_undegraded":"2024-09-10T21:12:23.807611+0000","last_fullsized":"2024-09-10T21:12:23.807611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:16.384124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787190+0000","last_change":"2024-09-10T21:12:23.787190+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787190+0000","last_undegraded":"2024-09-10T21:12:23.787190+0000","last_fullsized":"2024-09-10T21:12:23.787190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.806081+0000","last_change":"2024-09-10T21:12:23.804977+0000","last_active":"2024-09-10T21:12:23.806081+0000","last_peered":"2024-09-10T21:12:23.806081+0000","last_clean":"2024-09-10T21:12:23.806081+0000","last_became_active":"2024-09-10T21:12:23.804671+0000","last_became_peered":"2024-09-10T21:12:23.804671+0000","last_unstale":"2024-09-10T21:12:23.806081+0000","last_undegraded":"2024-09-10T21:12:23.806081+0000","last_fullsized":"2024-09-10T21:12:23.806081+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:30.301081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.790788+0000","last_change":"2024-09-10T21:12:23.790788+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.790788+0000","last_undegraded":"2024-09-10T21:12:23.790788+0000","last_fullsized":"2024-09-10T21:12:23.790788+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.810505+0000","last_change":"2024-09-10T21:12:23.808671+0000","last_active":"2024-09-10T21:12:23.810505+0000","last_peered":"2024-09-10T21:12:23.810505+0000","last_clean":"2024-09-10T21:12:23.810505+0000","last_became_active":"2024-09-10T21:12:23.808137+0000","last_became_peered":"2024-09-10T21:12:23.808137+0000","last_unstale":"2024-09-10T21:12:23.810505+0000","last_undegraded":"2024-09-10T21:12:23.810505+0000","last_fullsized":"2024-09-10T21:12:23.810505+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:48.351796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.781907+0000","last_change":"2024-09-10T21:12:23.781907+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.781907+0000","last_undegraded":"2024-09-10T21:12:23.781907+0000","last_fullsized":"2024-09-10T21:12:23.781907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.809301+0000","last_change":"2024-09-10T21:12:23.808302+0000","last_active":"2024-09-10T21:12:23.809301+0000","last_peered":"2024-09-10T21:12:23.809301+0000","last_clean":"2024-09-10T21:12:23.809301+0000","last_became_active":"2024-09-10T21:12:23.807962+0000","last_became_peered":"2024-09-10T21:12:23.807962+0000","last_unstale":"2024-09-10T21:12:23.809301+0000","last_undegraded":"2024-09-10T21:12:23.809301+0000","last_fullsized":"2024-09-10T21:12:23.809301+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:19.002126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.781820+0000","last_change":"2024-09-10T21:12:23.781820+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.781820+0000","last_undegraded":"2024-09-10T21:12:23.781820+0000","last_fullsized":"2024-09-10T21:12:23.781820+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.806261+0000","last_change":"2024-09-10T21:12:23.801081+0000","last_active":"2024-09-10T21:12:23.806261+0000","last_peered":"2024-09-10T21:12:23.806261+0000","last_clean":"2024-09-10T21:12:23.806261+0000","last_became_active":"2024-09-10T21:12:23.800792+0000","last_became_peered":"2024-09-10T21:12:23.800792+0000","last_unstale":"2024-09-10T21:12:23.806261+0000","last_undegraded":"2024-09-10T21:12:23.806261+0000","last_fullsized":"2024-09-10T21:12:23.806261+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:21.137294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787679+0000","last_change":"2024-09-10T21:12:23.787679+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787679+0000","last_undegraded":"2024-09-10T21:12:23.787679+0000","last_fullsized":"2024-09-10T21:12:23.787679+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.802492+0000","last_change":"2024-09-10T21:12:23.800595+0000","last_active":"2024-09-10T21:12:23.802492+0000","last_peered":"2024-09-10T21:12:23.802492+0000","last_clean":"2024-09-10T21:12:23.802492+0000","last_became_active":"2024-09-10T21:12:23.800279+0000","last_became_peered":"2024-09-10T21:12:23.800279+0000","last_unstale":"2024-09-10T21:12:23.802492+0000","last_undegraded":"2024-09-10T21:12:23.802492+0000","last_fullsized":"2024-09-10T21:12:23.802492+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:11.902707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787422+0000","last_change":"2024-09-10T21:12:23.787422+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787422+0000","last_undegraded":"2024-09-10T21:12:23.787422+0000","last_fullsized":"2024-09-10T21:12:23.787422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.811868+0000","last_change":"2024-09-10T21:12:23.810567+0000","last_active":"2024-09-10T21:12:23.811868+0000","last_peered":"2024-09-10T21:12:23.811868+0000","last_clean":"2024-09-10T21:12:23.811868+0000","last_became_active":"2024-09-10T21:12:23.810246+0000","last_became_peered":"2024-09-10T21:12:23.810246+0000","last_unstale":"2024-09-10T21:12:23.811868+0000","last_undegraded":"2024-09-10T21:12:23.811868+0000","last_fullsized":"2024-09-10T21:12:23.811868+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:25.257787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.786865+0000","last_change":"2024-09-10T21:12:23.786865+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.786865+0000","last_undegraded":"2024-09-10T21:12:23.786865+0000","last_fullsized":"2024-09-10T21:12:23.786865+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-10T21:12:22.780918+0000","last_change":"2024-09-10T21:12:22.780918+0000","last_active":"2024-09-10T21:12:22.764860+0000","last_peered":"2024-09-10T21:12:22.764860+0000","last_clean":"2024-09-10T21:12:22.764860+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:22.780918+0000","last_undegraded":"2024-09-10T21:12:22.780918+0000","last_fullsized":"2024-09-10T21:12:22.780918+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.786563+0000","last_change":"2024-09-10T21:12:23.786563+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.786563+0000","last_undegraded":"2024-09-10T21:12:23.786563+0000","last_fullsized":"2024-09-10T21:12:23.786563+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.798439+0000","last_change":"2024-09-10T21:12:23.795791+0000","last_active":"2024-09-10T21:12:23.798439+0000","last_peered":"2024-09-10T21:12:23.798439+0000","last_clean":"2024-09-10T21:12:23.798439+0000","last_became_active":"2024-09-10T21:12:23.795074+0000","last_became_peered":"2024-09-10T21:12:23.795074+0000","last_unstale":"2024-09-10T21:12:23.798439+0000","last_undegraded":"2024-09-10T21:12:23.798439+0000","last_fullsized":"2024-09-10T21:12:23.798439+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:27.368027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782827+0000","last_change":"2024-09-10T21:12:23.782827+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782827+0000","last_undegraded":"2024-09-10T21:12:23.782827+0000","last_fullsized":"2024-09-10T21:12:23.782827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803593+0000","last_change":"2024-09-10T21:12:23.797756+0000","last_active":"2024-09-10T21:12:23.803593+0000","last_peered":"2024-09-10T21:12:23.803593+0000","last_clean":"2024-09-10T21:12:23.803593+0000","last_became_active":"2024-09-10T21:12:23.797410+0000","last_became_peered":"2024-09-10T21:12:23.797410+0000","last_unstale":"2024-09-10T21:12:23.803593+0000","last_undegraded":"2024-09-10T21:12:23.803593+0000","last_fullsized":"2024-09-10T21:12:23.803593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:51.769273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803619+0000","last_change":"2024-09-10T21:12:23.800295+0000","last_active":"2024-09-10T21:12:23.803619+0000","last_peered":"2024-09-10T21:12:23.803619+0000","last_clean":"2024-09-10T21:12:23.803619+0000","last_became_active":"2024-09-10T21:12:23.799896+0000","last_became_peered":"2024-09-10T21:12:23.799896+0000","last_unstale":"2024-09-10T21:12:23.803619+0000","last_undegraded":"2024-09-10T21:12:23.803619+0000","last_fullsized":"2024-09-10T21:12:23.803619+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:13.981787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.785822+0000","last_change":"2024-09-10T21:12:23.785822+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.785822+0000","last_undegraded":"2024-09-10T21:12:23.785822+0000","last_fullsized":"2024-09-10T21:12:23.785822+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-10T21:12:22.780401+0000","last_change":"2024-09-10T21:12:22.780401+0000","last_active":"2024-09-10T21:12:22.764860+0000","last_peered":"2024-09-10T21:12:22.764860+0000","last_clean":"2024-09-10T21:12:22.764860+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:22.780401+0000","last_undegraded":"2024-09-10T21:12:22.780401+0000","last_fullsized":"2024-09-10T21:12:22.780401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782233+0000","last_change":"2024-09-10T21:12:23.782233+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782233+0000","last_undegraded":"2024-09-10T21:12:23.782233+0000","last_fullsized":"2024-09-10T21:12:23.782233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.808346+0000","last_change":"2024-09-10T21:12:23.803236+0000","last_active":"2024-09-10T21:12:23.808346+0000","last_peered":"2024-09-10T21:12:23.808346+0000","last_clean":"2024-09-10T21:12:23.808346+0000","last_became_active":"2024-09-10T21:12:23.802939+0000","last_became_peered":"2024-09-10T21:12:23.802939+0000","last_unstale":"2024-09-10T21:12:23.808346+0000","last_undegraded":"2024-09-10T21:12:23.808346+0000","last_fullsized":"2024-09-10T21:12:23.808346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:28.616824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":149,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T21:12:22.780416+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:12:22.780416+0000","last_peered":"2024-09-10T21:12:22.780416+0000","last_clean":"2024-09-10T21:12:22.780416+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:12:22.780416+0000","last_undegraded":"2024-09-10T21:12:22.780416+0000","last_fullsized":"2024-09-10T21:12:22.780416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":102,"acting":102,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":507904,"data_stored":203325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1089536,"data_stored":787418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":495616,"data_stored":197050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509467,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":197050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":197050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":197050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":483328,"data_stored":191233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1064960,"data_stored":776474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:12:29.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:12:29.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3489000831 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd99c0d06b0 msgr2=0x7fd99c0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 --2- 172.21.15.16:0/3489000831 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd99c0d06b0 0x7fd99c0700a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd9840097f0 tx=0x7fd98401cf80 comp rx=0 tx=0).stop 2024-09-10T21:12:29.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3489000831 shutdown_connections 2024-09-10T21:12:29.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 --2- 172.21.15.16:0/3489000831 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd99c0d06b0 0x7fd99c0700a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3489000831 >> 172.21.15.16:0/3489000831 conn(0x7fd99c0cfb20 msgr2=0x7fd99c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:29.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3489000831 shutdown_connections 2024-09-10T21:12:29.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3489000831 wait complete. 2024-09-10T21:12:29.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 Processor -- start 2024-09-10T21:12:29.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 -- start start 2024-09-10T21:12:29.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd99c0705e0 0x7fd99c1fa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd99c0d06b0 0x7fd99c160720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd99c15a270 0x7fd99c160c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c072ed0 con 0x7fd99c15a270 2024-09-10T21:12:29.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c072d50 con 0x7fd99c0705e0 2024-09-10T21:12:29.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd9a0c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd99c073050 con 0x7fd99c0d06b0 2024-09-10T21:12:29.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.156+0000 7fd99a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd99c15a270 0x7fd99c160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd99a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd99c15a270 0x7fd99c160c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57152/0 (socket says 172.21.15.16:57152) 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd99a800640 1 -- 172.21.15.16:0/3152547149 learned_addr learned my addr 172.21.15.16:0/3152547149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd999e00640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd99c0705e0 0x7fd99c1fa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd991400640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd99c0d06b0 0x7fd99c160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd99a800640 1 -- 172.21.15.16:0/3152547149 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd99c0d06b0 msgr2=0x7fd99c160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd99a800640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd99c0d06b0 0x7fd99c160720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd99a800640 1 -- 172.21.15.16:0/3152547149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd99c0705e0 msgr2=0x7fd99c1fa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd99a800640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd99c0705e0 0x7fd99c1fa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd99a800640 1 -- 172.21.15.16:0/3152547149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c1faef0 con 0x7fd99c15a270 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd99a800640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd99c15a270 0x7fd99c160c60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd99400caa0 tx=0x7fd99400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd992a00640 1 -- 172.21.15.16:0/3152547149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994017070 con 0x7fd99c15a270 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd992a00640 1 -- 172.21.15.16:0/3152547149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd994005070 con 0x7fd99c15a270 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3152547149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99c1602c0 con 0x7fd99c15a270 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd992a00640 1 -- 172.21.15.16:0/3152547149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9940054c0 con 0x7fd99c15a270 2024-09-10T21:12:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3152547149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99c0c7ab0 con 0x7fd99c15a270 2024-09-10T21:12:29.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3152547149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd99c0700a0 con 0x7fd99c15a270 2024-09-10T21:12:29.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.160+0000 7fd992a00640 1 -- 172.21.15.16:0/3152547149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd994005660 con 0x7fd99c15a270 2024-09-10T21:12:29.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.164+0000 7fd992a00640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9780725e0 0x7fd978074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.164+0000 7fd992a00640 1 -- 172.21.15.16:0/3152547149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fd994092db0 con 0x7fd99c15a270 2024-09-10T21:12:29.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.164+0000 7fd999e00640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9780725e0 0x7fd978074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.164+0000 7fd999e00640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9780725e0 0x7fd978074aa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd98c007fd0 tx=0x7fd98c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.164+0000 7fd992a00640 1 -- 172.21.15.16:0/3152547149 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd994097070 con 0x7fd99c15a270 2024-09-10T21:12:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.336+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3152547149 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd99c0ceb00 con 0x7fd9780725e0 2024-09-10T21:12:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.340+0000 7fd992a00640 1 -- 172.21.15.16:0/3152547149 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245278 (secure 0 0 0) 0x7fd99c0ceb00 con 0x7fd9780725e0 2024-09-10T21:12:29.350 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:29.351 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:12:29.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.344+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3152547149 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9780725e0 msgr2=0x7fd978074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.344+0000 7fd9a0c00640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9780725e0 0x7fd978074aa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd98c007fd0 tx=0x7fd98c009040 comp rx=0 tx=0).stop 2024-09-10T21:12:29.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.344+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3152547149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd99c15a270 msgr2=0x7fd99c160c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.344+0000 7fd9a0c00640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd99c15a270 0x7fd99c160c60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd99400caa0 tx=0x7fd99400cf60 comp rx=0 tx=0).stop 2024-09-10T21:12:29.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.344+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3152547149 shutdown_connections 2024-09-10T21:12:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.344+0000 7fd9a0c00640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9780725e0 0x7fd978074aa0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.344+0000 7fd9a0c00640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd99c15a270 0x7fd99c160c60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.344+0000 7fd9a0c00640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd99c0d06b0 0x7fd99c160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.344+0000 7fd9a0c00640 1 --2- 172.21.15.16:0/3152547149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd99c0705e0 0x7fd99c1fa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.344+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3152547149 >> 172.21.15.16:0/3152547149 conn(0x7fd99c0cfb20 msgr2=0x7fd99c0b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.344+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3152547149 shutdown_connections 2024-09-10T21:12:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.344+0000 7fd9a0c00640 1 -- 172.21.15.16:0/3152547149 wait complete. 2024-09-10T21:12:29.371 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-10T21:12:27.483412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":168,"acting":168,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254536,"kb_used_data":5008,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720184,"statfs":{"total":773094113280,"available":772833468416,"internally_reserved":0,"allocated":5128192,"data_stored":2746650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.710827"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787978+0000","last_change":"2024-09-10T21:12:23.787978+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787978+0000","last_undegraded":"2024-09-10T21:12:23.787978+0000","last_fullsized":"2024-09-10T21:12:23.787978+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787245+0000","last_change":"2024-09-10T21:12:23.787245+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787245+0000","last_undegraded":"2024-09-10T21:12:23.787245+0000","last_fullsized":"2024-09-10T21:12:23.787245+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782754+0000","last_change":"2024-09-10T21:12:23.782754+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782754+0000","last_undegraded":"2024-09-10T21:12:23.782754+0000","last_fullsized":"2024-09-10T21:12:23.782754+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.783698+0000","last_change":"2024-09-10T21:12:23.783698+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.783698+0000","last_undegraded":"2024-09-10T21:12:23.783698+0000","last_fullsized":"2024-09-10T21:12:23.783698+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.791512+0000","last_change":"2024-09-10T21:12:23.791512+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.791512+0000","last_undegraded":"2024-09-10T21:12:23.791512+0000","last_fullsized":"2024-09-10T21:12:23.791512+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.790560+0000","last_change":"2024-09-10T21:12:23.790560+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.790560+0000","last_undegraded":"2024-09-10T21:12:23.790560+0000","last_fullsized":"2024-09-10T21:12:23.790560+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.784962+0000","last_change":"2024-09-10T21:12:23.784962+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.784962+0000","last_undegraded":"2024-09-10T21:12:23.784962+0000","last_fullsized":"2024-09-10T21:12:23.784962+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.789970+0000","last_change":"2024-09-10T21:12:23.789970+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.789970+0000","last_undegraded":"2024-09-10T21:12:23.789970+0000","last_fullsized":"2024-09-10T21:12:23.789970+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.788313+0000","last_change":"2024-09-10T21:12:23.788313+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.788313+0000","last_undegraded":"2024-09-10T21:12:23.788313+0000","last_fullsized":"2024-09-10T21:12:23.788313+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.783286+0000","last_change":"2024-09-10T21:12:23.783286+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.783286+0000","last_undegraded":"2024-09-10T21:12:23.783286+0000","last_fullsized":"2024-09-10T21:12:23.783286+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.788387+0000","last_change":"2024-09-10T21:12:23.788387+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.788387+0000","last_undegraded":"2024-09-10T21:12:23.788387+0000","last_fullsized":"2024-09-10T21:12:23.788387+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.791309+0000","last_change":"2024-09-10T21:12:23.791309+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.791309+0000","last_undegraded":"2024-09-10T21:12:23.791309+0000","last_fullsized":"2024-09-10T21:12:23.791309+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.781799+0000","last_change":"2024-09-10T21:12:23.781799+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.781799+0000","last_undegraded":"2024-09-10T21:12:23.781799+0000","last_fullsized":"2024-09-10T21:12:23.781799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.783664+0000","last_change":"2024-09-10T21:12:23.783664+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.783664+0000","last_undegraded":"2024-09-10T21:12:23.783664+0000","last_fullsized":"2024-09-10T21:12:23.783664+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.784042+0000","last_change":"2024-09-10T21:12:23.784042+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.784042+0000","last_undegraded":"2024-09-10T21:12:23.784042+0000","last_fullsized":"2024-09-10T21:12:23.784042+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.789083+0000","last_change":"2024-09-10T21:12:23.789083+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.789083+0000","last_undegraded":"2024-09-10T21:12:23.789083+0000","last_fullsized":"2024-09-10T21:12:23.789083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.783192+0000","last_change":"2024-09-10T21:12:23.783192+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.783192+0000","last_undegraded":"2024-09-10T21:12:23.783192+0000","last_fullsized":"2024-09-10T21:12:23.783192+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787963+0000","last_change":"2024-09-10T21:12:23.787963+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787963+0000","last_undegraded":"2024-09-10T21:12:23.787963+0000","last_fullsized":"2024-09-10T21:12:23.787963+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.781815+0000","last_change":"2024-09-10T21:12:23.781815+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.781815+0000","last_undegraded":"2024-09-10T21:12:23.781815+0000","last_fullsized":"2024-09-10T21:12:23.781815+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.789677+0000","last_change":"2024-09-10T21:12:23.789677+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.789677+0000","last_undegraded":"2024-09-10T21:12:23.789677+0000","last_fullsized":"2024-09-10T21:12:23.789677+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787835+0000","last_change":"2024-09-10T21:12:23.787835+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787835+0000","last_undegraded":"2024-09-10T21:12:23.787835+0000","last_fullsized":"2024-09-10T21:12:23.787835+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.785962+0000","last_change":"2024-09-10T21:12:23.785962+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.785962+0000","last_undegraded":"2024-09-10T21:12:23.785962+0000","last_fullsized":"2024-09-10T21:12:23.785962+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782740+0000","last_change":"2024-09-10T21:12:23.782740+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782740+0000","last_undegraded":"2024-09-10T21:12:23.782740+0000","last_fullsized":"2024-09-10T21:12:23.782740+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.790668+0000","last_change":"2024-09-10T21:12:23.790668+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.790668+0000","last_undegraded":"2024-09-10T21:12:23.790668+0000","last_fullsized":"2024-09-10T21:12:23.790668+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.789308+0000","last_change":"2024-09-10T21:12:23.789308+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.789308+0000","last_undegraded":"2024-09-10T21:12:23.789308+0000","last_fullsized":"2024-09-10T21:12:23.789308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.783262+0000","last_change":"2024-09-10T21:12:23.783262+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.783262+0000","last_undegraded":"2024-09-10T21:12:23.783262+0000","last_fullsized":"2024-09-10T21:12:23.783262+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787684+0000","last_change":"2024-09-10T21:12:23.787684+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787684+0000","last_undegraded":"2024-09-10T21:12:23.787684+0000","last_fullsized":"2024-09-10T21:12:23.787684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.790234+0000","last_change":"2024-09-10T21:12:23.790234+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.790234+0000","last_undegraded":"2024-09-10T21:12:23.790234+0000","last_fullsized":"2024-09-10T21:12:23.790234+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.784473+0000","last_change":"2024-09-10T21:12:23.784473+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.784473+0000","last_undegraded":"2024-09-10T21:12:23.784473+0000","last_fullsized":"2024-09-10T21:12:23.784473+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.798094+0000","last_change":"2024-09-10T21:12:23.795968+0000","last_active":"2024-09-10T21:12:23.798094+0000","last_peered":"2024-09-10T21:12:23.798094+0000","last_clean":"2024-09-10T21:12:23.798094+0000","last_became_active":"2024-09-10T21:12:23.795674+0000","last_became_peered":"2024-09-10T21:12:23.795674+0000","last_unstale":"2024-09-10T21:12:23.798094+0000","last_undegraded":"2024-09-10T21:12:23.798094+0000","last_fullsized":"2024-09-10T21:12:23.798094+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:07.666553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782603+0000","last_change":"2024-09-10T21:12:23.782603+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782603+0000","last_undegraded":"2024-09-10T21:12:23.782603+0000","last_fullsized":"2024-09-10T21:12:23.782603+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803757+0000","last_change":"2024-09-10T21:12:23.799107+0000","last_active":"2024-09-10T21:12:23.803757+0000","last_peered":"2024-09-10T21:12:23.803757+0000","last_clean":"2024-09-10T21:12:23.803757+0000","last_became_active":"2024-09-10T21:12:23.798815+0000","last_became_peered":"2024-09-10T21:12:23.798815+0000","last_unstale":"2024-09-10T21:12:23.803757+0000","last_undegraded":"2024-09-10T21:12:23.803757+0000","last_fullsized":"2024-09-10T21:12:23.803757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:58.625840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787946+0000","last_change":"2024-09-10T21:12:23.787946+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787946+0000","last_undegraded":"2024-09-10T21:12:23.787946+0000","last_fullsized":"2024-09-10T21:12:23.787946+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.797239+0000","last_change":"2024-09-10T21:12:23.795368+0000","last_active":"2024-09-10T21:12:23.797239+0000","last_peered":"2024-09-10T21:12:23.797239+0000","last_clean":"2024-09-10T21:12:23.797239+0000","last_became_active":"2024-09-10T21:12:23.795086+0000","last_became_peered":"2024-09-10T21:12:23.795086+0000","last_unstale":"2024-09-10T21:12:23.797239+0000","last_undegraded":"2024-09-10T21:12:23.797239+0000","last_fullsized":"2024-09-10T21:12:23.797239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:01.506279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782629+0000","last_change":"2024-09-10T21:12:23.782629+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782629+0000","last_undegraded":"2024-09-10T21:12:23.782629+0000","last_fullsized":"2024-09-10T21:12:23.782629+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803772+0000","last_change":"2024-09-10T21:12:23.799265+0000","last_active":"2024-09-10T21:12:23.803772+0000","last_peered":"2024-09-10T21:12:23.803772+0000","last_clean":"2024-09-10T21:12:23.803772+0000","last_became_active":"2024-09-10T21:12:23.798990+0000","last_became_peered":"2024-09-10T21:12:23.798990+0000","last_unstale":"2024-09-10T21:12:23.803772+0000","last_undegraded":"2024-09-10T21:12:23.803772+0000","last_fullsized":"2024-09-10T21:12:23.803772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:10.371918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.788203+0000","last_change":"2024-09-10T21:12:23.788203+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.788203+0000","last_undegraded":"2024-09-10T21:12:23.788203+0000","last_fullsized":"2024-09-10T21:12:23.788203+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.802217+0000","last_change":"2024-09-10T21:12:23.798175+0000","last_active":"2024-09-10T21:12:23.802217+0000","last_peered":"2024-09-10T21:12:23.802217+0000","last_clean":"2024-09-10T21:12:23.802217+0000","last_became_active":"2024-09-10T21:12:23.797844+0000","last_became_peered":"2024-09-10T21:12:23.797844+0000","last_unstale":"2024-09-10T21:12:23.802217+0000","last_undegraded":"2024-09-10T21:12:23.802217+0000","last_fullsized":"2024-09-10T21:12:23.802217+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:17.620823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787187+0000","last_change":"2024-09-10T21:12:23.787187+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787187+0000","last_undegraded":"2024-09-10T21:12:23.787187+0000","last_fullsized":"2024-09-10T21:12:23.787187+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.808390+0000","last_change":"2024-09-10T21:12:23.803893+0000","last_active":"2024-09-10T21:12:23.808390+0000","last_peered":"2024-09-10T21:12:23.808390+0000","last_clean":"2024-09-10T21:12:23.808390+0000","last_became_active":"2024-09-10T21:12:23.803599+0000","last_became_peered":"2024-09-10T21:12:23.803599+0000","last_unstale":"2024-09-10T21:12:23.808390+0000","last_undegraded":"2024-09-10T21:12:23.808390+0000","last_fullsized":"2024-09-10T21:12:23.808390+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:07.284335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787410+0000","last_change":"2024-09-10T21:12:23.787410+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787410+0000","last_undegraded":"2024-09-10T21:12:23.787410+0000","last_fullsized":"2024-09-10T21:12:23.787410+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.795748+0000","last_change":"2024-09-10T21:12:23.793275+0000","last_active":"2024-09-10T21:12:23.795748+0000","last_peered":"2024-09-10T21:12:23.795748+0000","last_clean":"2024-09-10T21:12:23.795748+0000","last_became_active":"2024-09-10T21:12:23.792867+0000","last_became_peered":"2024-09-10T21:12:23.792867+0000","last_unstale":"2024-09-10T21:12:23.795748+0000","last_undegraded":"2024-09-10T21:12:23.795748+0000","last_fullsized":"2024-09-10T21:12:23.795748+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:25.165637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.790132+0000","last_change":"2024-09-10T21:12:23.790132+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.790132+0000","last_undegraded":"2024-09-10T21:12:23.790132+0000","last_fullsized":"2024-09-10T21:12:23.790132+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803065+0000","last_change":"2024-09-10T21:12:23.798078+0000","last_active":"2024-09-10T21:12:23.803065+0000","last_peered":"2024-09-10T21:12:23.803065+0000","last_clean":"2024-09-10T21:12:23.803065+0000","last_became_active":"2024-09-10T21:12:23.797773+0000","last_became_peered":"2024-09-10T21:12:23.797773+0000","last_unstale":"2024-09-10T21:12:23.803065+0000","last_undegraded":"2024-09-10T21:12:23.803065+0000","last_fullsized":"2024-09-10T21:12:23.803065+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:39.813884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-10T21:12:22.779766+0000","last_change":"2024-09-10T21:12:22.779766+0000","last_active":"2024-09-10T21:12:22.764860+0000","last_peered":"2024-09-10T21:12:22.764860+0000","last_clean":"2024-09-10T21:12:22.764860+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:22.779766+0000","last_undegraded":"2024-09-10T21:12:22.779766+0000","last_fullsized":"2024-09-10T21:12:22.779766+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.808783+0000","last_change":"2024-09-10T21:12:23.807246+0000","last_active":"2024-09-10T21:12:23.808783+0000","last_peered":"2024-09-10T21:12:23.808783+0000","last_clean":"2024-09-10T21:12:23.808783+0000","last_became_active":"2024-09-10T21:12:23.806959+0000","last_became_peered":"2024-09-10T21:12:23.806959+0000","last_unstale":"2024-09-10T21:12:23.808783+0000","last_undegraded":"2024-09-10T21:12:23.808783+0000","last_fullsized":"2024-09-10T21:12:23.808783+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:45.079126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782139+0000","last_change":"2024-09-10T21:12:23.782139+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782139+0000","last_undegraded":"2024-09-10T21:12:23.782139+0000","last_fullsized":"2024-09-10T21:12:23.782139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.797152+0000","last_change":"2024-09-10T21:12:23.790411+0000","last_active":"2024-09-10T21:12:23.797152+0000","last_peered":"2024-09-10T21:12:23.797152+0000","last_clean":"2024-09-10T21:12:23.797152+0000","last_became_active":"2024-09-10T21:12:23.790109+0000","last_became_peered":"2024-09-10T21:12:23.790109+0000","last_unstale":"2024-09-10T21:12:23.797152+0000","last_undegraded":"2024-09-10T21:12:23.797152+0000","last_fullsized":"2024-09-10T21:12:23.797152+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:48.351796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.790243+0000","last_change":"2024-09-10T21:12:23.790243+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.790243+0000","last_undegraded":"2024-09-10T21:12:23.790243+0000","last_fullsized":"2024-09-10T21:12:23.790243+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803914+0000","last_change":"2024-09-10T21:12:23.802815+0000","last_active":"2024-09-10T21:12:23.803914+0000","last_peered":"2024-09-10T21:12:23.803914+0000","last_clean":"2024-09-10T21:12:23.803914+0000","last_became_active":"2024-09-10T21:12:23.802540+0000","last_became_peered":"2024-09-10T21:12:23.802540+0000","last_unstale":"2024-09-10T21:12:23.803914+0000","last_undegraded":"2024-09-10T21:12:23.803914+0000","last_fullsized":"2024-09-10T21:12:23.803914+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:44.108373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-10T21:12:22.780457+0000","last_change":"2024-09-10T21:12:22.780457+0000","last_active":"2024-09-10T21:12:22.764860+0000","last_peered":"2024-09-10T21:12:22.764860+0000","last_clean":"2024-09-10T21:12:22.764860+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:22.780457+0000","last_undegraded":"2024-09-10T21:12:22.780457+0000","last_fullsized":"2024-09-10T21:12:22.780457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.807879+0000","last_change":"2024-09-10T21:12:23.800966+0000","last_active":"2024-09-10T21:12:23.807879+0000","last_peered":"2024-09-10T21:12:23.807879+0000","last_clean":"2024-09-10T21:12:23.807879+0000","last_became_active":"2024-09-10T21:12:23.800647+0000","last_became_peered":"2024-09-10T21:12:23.800647+0000","last_unstale":"2024-09-10T21:12:23.807879+0000","last_undegraded":"2024-09-10T21:12:23.807879+0000","last_fullsized":"2024-09-10T21:12:23.807879+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:10.581708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.808518+0000","last_change":"2024-09-10T21:12:23.806810+0000","last_active":"2024-09-10T21:12:23.808518+0000","last_peered":"2024-09-10T21:12:23.808518+0000","last_clean":"2024-09-10T21:12:23.808518+0000","last_became_active":"2024-09-10T21:12:23.806235+0000","last_became_peered":"2024-09-10T21:12:23.806235+0000","last_unstale":"2024-09-10T21:12:23.808518+0000","last_undegraded":"2024-09-10T21:12:23.808518+0000","last_fullsized":"2024-09-10T21:12:23.808518+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:56.825780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.786442+0000","last_change":"2024-09-10T21:12:23.786442+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.786442+0000","last_undegraded":"2024-09-10T21:12:23.786442+0000","last_fullsized":"2024-09-10T21:12:23.786442+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803041+0000","last_change":"2024-09-10T21:12:23.796942+0000","last_active":"2024-09-10T21:12:23.803041+0000","last_peered":"2024-09-10T21:12:23.803041+0000","last_clean":"2024-09-10T21:12:23.803041+0000","last_became_active":"2024-09-10T21:12:23.796661+0000","last_became_peered":"2024-09-10T21:12:23.796661+0000","last_unstale":"2024-09-10T21:12:23.803041+0000","last_undegraded":"2024-09-10T21:12:23.803041+0000","last_fullsized":"2024-09-10T21:12:23.803041+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:34.156674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.799963+0000","last_change":"2024-09-10T21:12:23.797369+0000","last_active":"2024-09-10T21:12:23.799963+0000","last_peered":"2024-09-10T21:12:23.799963+0000","last_clean":"2024-09-10T21:12:23.799963+0000","last_became_active":"2024-09-10T21:12:23.796137+0000","last_became_peered":"2024-09-10T21:12:23.796137+0000","last_unstale":"2024-09-10T21:12:23.799963+0000","last_undegraded":"2024-09-10T21:12:23.799963+0000","last_fullsized":"2024-09-10T21:12:23.799963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:34.576509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-10T21:12:22.779809+0000","last_change":"2024-09-10T21:12:22.779809+0000","last_active":"2024-09-10T21:12:22.764860+0000","last_peered":"2024-09-10T21:12:22.764860+0000","last_clean":"2024-09-10T21:12:22.764860+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:22.779809+0000","last_undegraded":"2024-09-10T21:12:22.779809+0000","last_fullsized":"2024-09-10T21:12:22.779809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.785775+0000","last_change":"2024-09-10T21:12:23.785775+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.785775+0000","last_undegraded":"2024-09-10T21:12:23.785775+0000","last_fullsized":"2024-09-10T21:12:23.785775+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-10T21:12:22.779822+0000","last_change":"2024-09-10T21:12:22.779822+0000","last_active":"2024-09-10T21:12:22.764860+0000","last_peered":"2024-09-10T21:12:22.764860+0000","last_clean":"2024-09-10T21:12:22.764860+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:22.779822+0000","last_undegraded":"2024-09-10T21:12:22.779822+0000","last_fullsized":"2024-09-10T21:12:22.779822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.807611+0000","last_change":"2024-09-10T21:12:23.806377+0000","last_active":"2024-09-10T21:12:23.807611+0000","last_peered":"2024-09-10T21:12:23.807611+0000","last_clean":"2024-09-10T21:12:23.807611+0000","last_became_active":"2024-09-10T21:12:23.806032+0000","last_became_peered":"2024-09-10T21:12:23.806032+0000","last_unstale":"2024-09-10T21:12:23.807611+0000","last_undegraded":"2024-09-10T21:12:23.807611+0000","last_fullsized":"2024-09-10T21:12:23.807611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:16.384124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787190+0000","last_change":"2024-09-10T21:12:23.787190+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787190+0000","last_undegraded":"2024-09-10T21:12:23.787190+0000","last_fullsized":"2024-09-10T21:12:23.787190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.806081+0000","last_change":"2024-09-10T21:12:23.804977+0000","last_active":"2024-09-10T21:12:23.806081+0000","last_peered":"2024-09-10T21:12:23.806081+0000","last_clean":"2024-09-10T21:12:23.806081+0000","last_became_active":"2024-09-10T21:12:23.804671+0000","last_became_peered":"2024-09-10T21:12:23.804671+0000","last_unstale":"2024-09-10T21:12:23.806081+0000","last_undegraded":"2024-09-10T21:12:23.806081+0000","last_fullsized":"2024-09-10T21:12:23.806081+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:30.301081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.790788+0000","last_change":"2024-09-10T21:12:23.790788+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.790788+0000","last_undegraded":"2024-09-10T21:12:23.790788+0000","last_fullsized":"2024-09-10T21:12:23.790788+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.810505+0000","last_change":"2024-09-10T21:12:23.808671+0000","last_active":"2024-09-10T21:12:23.810505+0000","last_peered":"2024-09-10T21:12:23.810505+0000","last_clean":"2024-09-10T21:12:23.810505+0000","last_became_active":"2024-09-10T21:12:23.808137+0000","last_became_peered":"2024-09-10T21:12:23.808137+0000","last_unstale":"2024-09-10T21:12:23.810505+0000","last_undegraded":"2024-09-10T21:12:23.810505+0000","last_fullsized":"2024-09-10T21:12:23.810505+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:48.351796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.781907+0000","last_change":"2024-09-10T21:12:23.781907+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.781907+0000","last_undegraded":"2024-09-10T21:12:23.781907+0000","last_fullsized":"2024-09-10T21:12:23.781907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.809301+0000","last_change":"2024-09-10T21:12:23.808302+0000","last_active":"2024-09-10T21:12:23.809301+0000","last_peered":"2024-09-10T21:12:23.809301+0000","last_clean":"2024-09-10T21:12:23.809301+0000","last_became_active":"2024-09-10T21:12:23.807962+0000","last_became_peered":"2024-09-10T21:12:23.807962+0000","last_unstale":"2024-09-10T21:12:23.809301+0000","last_undegraded":"2024-09-10T21:12:23.809301+0000","last_fullsized":"2024-09-10T21:12:23.809301+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:19.002126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.781820+0000","last_change":"2024-09-10T21:12:23.781820+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.781820+0000","last_undegraded":"2024-09-10T21:12:23.781820+0000","last_fullsized":"2024-09-10T21:12:23.781820+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.806261+0000","last_change":"2024-09-10T21:12:23.801081+0000","last_active":"2024-09-10T21:12:23.806261+0000","last_peered":"2024-09-10T21:12:23.806261+0000","last_clean":"2024-09-10T21:12:23.806261+0000","last_became_active":"2024-09-10T21:12:23.800792+0000","last_became_peered":"2024-09-10T21:12:23.800792+0000","last_unstale":"2024-09-10T21:12:23.806261+0000","last_undegraded":"2024-09-10T21:12:23.806261+0000","last_fullsized":"2024-09-10T21:12:23.806261+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:21.137294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787679+0000","last_change":"2024-09-10T21:12:23.787679+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787679+0000","last_undegraded":"2024-09-10T21:12:23.787679+0000","last_fullsized":"2024-09-10T21:12:23.787679+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.802492+0000","last_change":"2024-09-10T21:12:23.800595+0000","last_active":"2024-09-10T21:12:23.802492+0000","last_peered":"2024-09-10T21:12:23.802492+0000","last_clean":"2024-09-10T21:12:23.802492+0000","last_became_active":"2024-09-10T21:12:23.800279+0000","last_became_peered":"2024-09-10T21:12:23.800279+0000","last_unstale":"2024-09-10T21:12:23.802492+0000","last_undegraded":"2024-09-10T21:12:23.802492+0000","last_fullsized":"2024-09-10T21:12:23.802492+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:11.902707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.787422+0000","last_change":"2024-09-10T21:12:23.787422+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.787422+0000","last_undegraded":"2024-09-10T21:12:23.787422+0000","last_fullsized":"2024-09-10T21:12:23.787422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.811868+0000","last_change":"2024-09-10T21:12:23.810567+0000","last_active":"2024-09-10T21:12:23.811868+0000","last_peered":"2024-09-10T21:12:23.811868+0000","last_clean":"2024-09-10T21:12:23.811868+0000","last_became_active":"2024-09-10T21:12:23.810246+0000","last_became_peered":"2024-09-10T21:12:23.810246+0000","last_unstale":"2024-09-10T21:12:23.811868+0000","last_undegraded":"2024-09-10T21:12:23.811868+0000","last_fullsized":"2024-09-10T21:12:23.811868+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:25.257787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.786865+0000","last_change":"2024-09-10T21:12:23.786865+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.786865+0000","last_undegraded":"2024-09-10T21:12:23.786865+0000","last_fullsized":"2024-09-10T21:12:23.786865+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-10T21:12:22.780918+0000","last_change":"2024-09-10T21:12:22.780918+0000","last_active":"2024-09-10T21:12:22.764860+0000","last_peered":"2024-09-10T21:12:22.764860+0000","last_clean":"2024-09-10T21:12:22.764860+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:22.780918+0000","last_undegraded":"2024-09-10T21:12:22.780918+0000","last_fullsized":"2024-09-10T21:12:22.780918+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.786563+0000","last_change":"2024-09-10T21:12:23.786563+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.786563+0000","last_undegraded":"2024-09-10T21:12:23.786563+0000","last_fullsized":"2024-09-10T21:12:23.786563+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.798439+0000","last_change":"2024-09-10T21:12:23.795791+0000","last_active":"2024-09-10T21:12:23.798439+0000","last_peered":"2024-09-10T21:12:23.798439+0000","last_clean":"2024-09-10T21:12:23.798439+0000","last_became_active":"2024-09-10T21:12:23.795074+0000","last_became_peered":"2024-09-10T21:12:23.795074+0000","last_unstale":"2024-09-10T21:12:23.798439+0000","last_undegraded":"2024-09-10T21:12:23.798439+0000","last_fullsized":"2024-09-10T21:12:23.798439+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:27.368027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782827+0000","last_change":"2024-09-10T21:12:23.782827+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782827+0000","last_undegraded":"2024-09-10T21:12:23.782827+0000","last_fullsized":"2024-09-10T21:12:23.782827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803593+0000","last_change":"2024-09-10T21:12:23.797756+0000","last_active":"2024-09-10T21:12:23.803593+0000","last_peered":"2024-09-10T21:12:23.803593+0000","last_clean":"2024-09-10T21:12:23.803593+0000","last_became_active":"2024-09-10T21:12:23.797410+0000","last_became_peered":"2024-09-10T21:12:23.797410+0000","last_unstale":"2024-09-10T21:12:23.803593+0000","last_undegraded":"2024-09-10T21:12:23.803593+0000","last_fullsized":"2024-09-10T21:12:23.803593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:51.769273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:12:23.772267+0000","last_change":"2024-09-10T21:12:23.772267+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.772267+0000","last_undegraded":"2024-09-10T21:12:23.772267+0000","last_fullsized":"2024-09-10T21:12:23.772267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.803619+0000","last_change":"2024-09-10T21:12:23.800295+0000","last_active":"2024-09-10T21:12:23.803619+0000","last_peered":"2024-09-10T21:12:23.803619+0000","last_clean":"2024-09-10T21:12:23.803619+0000","last_became_active":"2024-09-10T21:12:23.799896+0000","last_became_peered":"2024-09-10T21:12:23.799896+0000","last_unstale":"2024-09-10T21:12:23.803619+0000","last_undegraded":"2024-09-10T21:12:23.803619+0000","last_fullsized":"2024-09-10T21:12:23.803619+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:13.981787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.785822+0000","last_change":"2024-09-10T21:12:23.785822+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.785822+0000","last_undegraded":"2024-09-10T21:12:23.785822+0000","last_fullsized":"2024-09-10T21:12:23.785822+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-10T21:12:22.780401+0000","last_change":"2024-09-10T21:12:22.780401+0000","last_active":"2024-09-10T21:12:22.764860+0000","last_peered":"2024-09-10T21:12:22.764860+0000","last_clean":"2024-09-10T21:12:22.764860+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:22.780401+0000","last_undegraded":"2024-09-10T21:12:22.780401+0000","last_fullsized":"2024-09-10T21:12:22.780401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-10T21:12:23.782233+0000","last_change":"2024-09-10T21:12:23.782233+0000","last_active":"2024-09-10T21:12:23.772267+0000","last_peered":"2024-09-10T21:12:23.772267+0000","last_clean":"2024-09-10T21:12:23.772267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:23.782233+0000","last_undegraded":"2024-09-10T21:12:23.782233+0000","last_fullsized":"2024-09-10T21:12:23.782233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:12:23.808346+0000","last_change":"2024-09-10T21:12:23.803236+0000","last_active":"2024-09-10T21:12:23.808346+0000","last_peered":"2024-09-10T21:12:23.808346+0000","last_clean":"2024-09-10T21:12:23.808346+0000","last_became_active":"2024-09-10T21:12:23.802939+0000","last_became_peered":"2024-09-10T21:12:23.802939+0000","last_unstale":"2024-09-10T21:12:23.808346+0000","last_undegraded":"2024-09-10T21:12:23.808346+0000","last_fullsized":"2024-09-10T21:12:23.808346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:28.616824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":149,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T21:12:22.780416+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:12:22.780416+0000","last_peered":"2024-09-10T21:12:22.780416+0000","last_clean":"2024-09-10T21:12:22.780416+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:12:22.780416+0000","last_undegraded":"2024-09-10T21:12:22.780416+0000","last_fullsized":"2024-09-10T21:12:22.780416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":102,"acting":102,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":507904,"data_stored":203325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1089536,"data_stored":787418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":495616,"data_stored":197050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509467,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":197050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":197050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":197050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":483328,"data_stored":191233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1064960,"data_stored":776474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:12:29.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:12:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.556+0000 7ff471e00640 1 -- 172.21.15.16:0/2921602878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46c0b13c0 msgr2=0x7ff46c071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.556+0000 7ff471e00640 1 --2- 172.21.15.16:0/2921602878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46c0b13c0 0x7ff46c071520 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff46c073290 tx=0x7ff45401c6e0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.556+0000 7ff471e00640 1 -- 172.21.15.16:0/2921602878 shutdown_connections 2024-09-10T21:12:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.556+0000 7ff471e00640 1 --2- 172.21.15.16:0/2921602878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46c0b13c0 0x7ff46c071520 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.556+0000 7ff471e00640 1 --2- 172.21.15.16:0/2921602878 >> v2:172.21.15.52:3300/0 conn(0x7ff46c15f400 0x7ff46c070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.556+0000 7ff471e00640 1 -- 172.21.15.16:0/2921602878 >> 172.21.15.16:0/2921602878 conn(0x7ff46c09b130 msgr2=0x7ff46c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.556+0000 7ff471e00640 1 -- 172.21.15.16:0/2921602878 shutdown_connections 2024-09-10T21:12:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.556+0000 7ff471e00640 1 -- 172.21.15.16:0/2921602878 wait complete. 2024-09-10T21:12:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.556+0000 7ff471e00640 1 Processor -- start 2024-09-10T21:12:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.556+0000 7ff471e00640 1 -- start start 2024-09-10T21:12:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff471e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46c0b13c0 0x7ff46c14b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff471e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c0cfc80 0x7ff46c0b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff471e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff46c15f400 0x7ff46c15e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff471e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c169560 con 0x7ff46c0b13c0 2024-09-10T21:12:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff471e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1693e0 con 0x7ff46c0cfc80 2024-09-10T21:12:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff471e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff46c1696e0 con 0x7ff46c15f400 2024-09-10T21:12:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff46aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46c0b13c0 0x7ff46c14b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff46aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46c0b13c0 0x7ff46c14b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57170/0 (socket says 172.21.15.16:57170) 2024-09-10T21:12:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff46aa00640 1 -- 172.21.15.16:0/1482430269 learned_addr learned my addr 172.21.15.16:0/1482430269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff46b400640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff46c15f400 0x7ff46c15e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff463e00640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c0cfc80 0x7ff46c0b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff46aa00640 1 -- 172.21.15.16:0/1482430269 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff46c15f400 msgr2=0x7ff46c15e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff46aa00640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff46c15f400 0x7ff46c15e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff46aa00640 1 -- 172.21.15.16:0/1482430269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c0cfc80 msgr2=0x7ff46c0b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff46aa00640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c0cfc80 0x7ff46c0b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff46aa00640 1 -- 172.21.15.16:0/1482430269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46c0b61c0 con 0x7ff46c0b13c0 2024-09-10T21:12:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff46aa00640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46c0b13c0 0x7ff46c14b3c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00d700 tx=0x7ff45c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff463400640 1 -- 172.21.15.16:0/1482430269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c017070 con 0x7ff46c0b13c0 2024-09-10T21:12:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff463400640 1 -- 172.21.15.16:0/1482430269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff45c009cc0 con 0x7ff46c0b13c0 2024-09-10T21:12:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff471e00640 1 -- 172.21.15.16:0/1482430269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46c0b64a0 con 0x7ff46c0b13c0 2024-09-10T21:12:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff463400640 1 -- 172.21.15.16:0/1482430269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0106c0 con 0x7ff46c0b13c0 2024-09-10T21:12:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff471e00640 1 -- 172.21.15.16:0/1482430269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46c0b6a60 con 0x7ff46c0b13c0 2024-09-10T21:12:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff471e00640 1 -- 172.21.15.16:0/1482430269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff46c0d00d0 con 0x7ff46c0b13c0 2024-09-10T21:12:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.560+0000 7ff463400640 1 -- 172.21.15.16:0/1482430269 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff45c010860 con 0x7ff46c0b13c0 2024-09-10T21:12:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.564+0000 7ff463400640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff4480725e0 0x7ff448074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.564+0000 7ff463400640 1 -- 172.21.15.16:0/1482430269 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff45c093670 con 0x7ff46c0b13c0 2024-09-10T21:12:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.564+0000 7ff463e00640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff4480725e0 0x7ff448074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.564+0000 7ff463e00640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff4480725e0 0x7ff448074aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff4540af000 tx=0x7ff45401af20 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.568+0000 7ff463400640 1 -- 172.21.15.16:0/1482430269 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff45c05ed40 con 0x7ff46c0b13c0 2024-09-10T21:12:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.736+0000 7ff471e00640 1 -- 172.21.15.16:0/1482430269 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff46c0ce4c0 con 0x7ff4480725e0 2024-09-10T21:12:29.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.740+0000 7ff463400640 1 -- 172.21.15.16:0/1482430269 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7ff46c0ce4c0 con 0x7ff4480725e0 2024-09-10T21:12:29.751 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:29.752 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:12:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.744+0000 7ff471e00640 1 -- 172.21.15.16:0/1482430269 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff4480725e0 msgr2=0x7ff448074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.744+0000 7ff471e00640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff4480725e0 0x7ff448074aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff4540af000 tx=0x7ff45401af20 comp rx=0 tx=0).stop 2024-09-10T21:12:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.744+0000 7ff471e00640 1 -- 172.21.15.16:0/1482430269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46c0b13c0 msgr2=0x7ff46c14b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.744+0000 7ff471e00640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46c0b13c0 0x7ff46c14b3c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00d700 tx=0x7ff45c00dbc0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.744+0000 7ff471e00640 1 -- 172.21.15.16:0/1482430269 shutdown_connections 2024-09-10T21:12:29.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.744+0000 7ff471e00640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff4480725e0 0x7ff448074aa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.744+0000 7ff471e00640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff46c15f400 0x7ff46c15e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.744+0000 7ff471e00640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff46c0cfc80 0x7ff46c0b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.744+0000 7ff471e00640 1 --2- 172.21.15.16:0/1482430269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff46c0b13c0 0x7ff46c14b3c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.744+0000 7ff471e00640 1 -- 172.21.15.16:0/1482430269 >> 172.21.15.16:0/1482430269 conn(0x7ff46c09b130 msgr2=0x7ff46c0af510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:29.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.744+0000 7ff471e00640 1 -- 172.21.15.16:0/1482430269 shutdown_connections 2024-09-10T21:12:29.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.744+0000 7ff471e00640 1 -- 172.21.15.16:0/1482430269 wait complete. 2024-09-10T21:12:29.772 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-10T21:12:29.483873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254896,"kb_used_data":5384,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719824,"statfs":{"total":773094113280,"available":772833099776,"internally_reserved":0,"allocated":5513216,"data_stored":2916756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001312"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428163+0000","last_change":"2024-09-10T21:12:24.807699+0000","last_active":"2024-09-10T21:12:26.428163+0000","last_peered":"2024-09-10T21:12:26.428163+0000","last_clean":"2024-09-10T21:12:26.428163+0000","last_became_active":"2024-09-10T21:12:24.807378+0000","last_became_peered":"2024-09-10T21:12:24.807378+0000","last_unstale":"2024-09-10T21:12:26.428163+0000","last_undegraded":"2024-09-10T21:12:26.428163+0000","last_fullsized":"2024-09-10T21:12:26.428163+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:06.104316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785215+0000","last_change":"2024-09-10T21:12:24.801034+0000","last_active":"2024-09-10T21:12:25.785215+0000","last_peered":"2024-09-10T21:12:25.785215+0000","last_clean":"2024-09-10T21:12:25.785215+0000","last_became_active":"2024-09-10T21:12:24.799842+0000","last_became_peered":"2024-09-10T21:12:24.799842+0000","last_unstale":"2024-09-10T21:12:25.785215+0000","last_undegraded":"2024-09-10T21:12:25.785215+0000","last_fullsized":"2024-09-10T21:12:25.785215+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:51.766507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429089+0000","last_change":"2024-09-10T21:12:24.811161+0000","last_active":"2024-09-10T21:12:26.429089+0000","last_peered":"2024-09-10T21:12:26.429089+0000","last_clean":"2024-09-10T21:12:26.429089+0000","last_became_active":"2024-09-10T21:12:24.810838+0000","last_became_peered":"2024-09-10T21:12:24.810838+0000","last_unstale":"2024-09-10T21:12:26.429089+0000","last_undegraded":"2024-09-10T21:12:26.429089+0000","last_fullsized":"2024-09-10T21:12:26.429089+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:14.399311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785704+0000","last_change":"2024-09-10T21:12:24.807996+0000","last_active":"2024-09-10T21:12:25.785704+0000","last_peered":"2024-09-10T21:12:25.785704+0000","last_clean":"2024-09-10T21:12:25.785704+0000","last_became_active":"2024-09-10T21:12:24.807614+0000","last_became_peered":"2024-09-10T21:12:24.807614+0000","last_unstale":"2024-09-10T21:12:25.785704+0000","last_undegraded":"2024-09-10T21:12:25.785704+0000","last_fullsized":"2024-09-10T21:12:25.785704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:58.826777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428361+0000","last_change":"2024-09-10T21:12:24.814793+0000","last_active":"2024-09-10T21:12:26.428361+0000","last_peered":"2024-09-10T21:12:26.428361+0000","last_clean":"2024-09-10T21:12:26.428361+0000","last_became_active":"2024-09-10T21:12:24.814494+0000","last_became_peered":"2024-09-10T21:12:24.814494+0000","last_unstale":"2024-09-10T21:12:26.428361+0000","last_undegraded":"2024-09-10T21:12:26.428361+0000","last_fullsized":"2024-09-10T21:12:26.428361+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:21.261528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785736+0000","last_change":"2024-09-10T21:12:24.827330+0000","last_active":"2024-09-10T21:12:25.785736+0000","last_peered":"2024-09-10T21:12:25.785736+0000","last_clean":"2024-09-10T21:12:25.785736+0000","last_became_active":"2024-09-10T21:12:24.827158+0000","last_became_peered":"2024-09-10T21:12:24.827158+0000","last_unstale":"2024-09-10T21:12:25.785736+0000","last_undegraded":"2024-09-10T21:12:25.785736+0000","last_fullsized":"2024-09-10T21:12:25.785736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:13.628817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427734+0000","last_change":"2024-09-10T21:12:24.808616+0000","last_active":"2024-09-10T21:12:26.427734+0000","last_peered":"2024-09-10T21:12:26.427734+0000","last_clean":"2024-09-10T21:12:26.427734+0000","last_became_active":"2024-09-10T21:12:24.808341+0000","last_became_peered":"2024-09-10T21:12:24.808341+0000","last_unstale":"2024-09-10T21:12:26.427734+0000","last_undegraded":"2024-09-10T21:12:26.427734+0000","last_fullsized":"2024-09-10T21:12:26.427734+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:23.140730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785688+0000","last_change":"2024-09-10T21:12:24.823724+0000","last_active":"2024-09-10T21:12:25.785688+0000","last_peered":"2024-09-10T21:12:25.785688+0000","last_clean":"2024-09-10T21:12:25.785688+0000","last_became_active":"2024-09-10T21:12:24.823452+0000","last_became_peered":"2024-09-10T21:12:24.823452+0000","last_unstale":"2024-09-10T21:12:25.785688+0000","last_undegraded":"2024-09-10T21:12:25.785688+0000","last_fullsized":"2024-09-10T21:12:25.785688+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:22.748738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.787140+0000","last_change":"2024-09-10T21:12:24.822052+0000","last_active":"2024-09-10T21:12:25.787140+0000","last_peered":"2024-09-10T21:12:25.787140+0000","last_clean":"2024-09-10T21:12:25.787140+0000","last_became_active":"2024-09-10T21:12:24.821688+0000","last_became_peered":"2024-09-10T21:12:24.821688+0000","last_unstale":"2024-09-10T21:12:25.787140+0000","last_undegraded":"2024-09-10T21:12:25.787140+0000","last_fullsized":"2024-09-10T21:12:25.787140+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:52.341058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428221+0000","last_change":"2024-09-10T21:12:24.813099+0000","last_active":"2024-09-10T21:12:26.428221+0000","last_peered":"2024-09-10T21:12:26.428221+0000","last_clean":"2024-09-10T21:12:26.428221+0000","last_became_active":"2024-09-10T21:12:24.812570+0000","last_became_peered":"2024-09-10T21:12:24.812570+0000","last_unstale":"2024-09-10T21:12:26.428221+0000","last_undegraded":"2024-09-10T21:12:26.428221+0000","last_fullsized":"2024-09-10T21:12:26.428221+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:25.911130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786429+0000","last_change":"2024-09-10T21:12:24.830636+0000","last_active":"2024-09-10T21:12:25.786429+0000","last_peered":"2024-09-10T21:12:25.786429+0000","last_clean":"2024-09-10T21:12:25.786429+0000","last_became_active":"2024-09-10T21:12:24.830369+0000","last_became_peered":"2024-09-10T21:12:24.830369+0000","last_unstale":"2024-09-10T21:12:25.786429+0000","last_undegraded":"2024-09-10T21:12:25.786429+0000","last_fullsized":"2024-09-10T21:12:25.786429+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:39.747387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784766+0000","last_change":"2024-09-10T21:12:24.788829+0000","last_active":"2024-09-10T21:12:25.784766+0000","last_peered":"2024-09-10T21:12:25.784766+0000","last_clean":"2024-09-10T21:12:25.784766+0000","last_became_active":"2024-09-10T21:12:24.788547+0000","last_became_peered":"2024-09-10T21:12:24.788547+0000","last_unstale":"2024-09-10T21:12:25.784766+0000","last_undegraded":"2024-09-10T21:12:25.784766+0000","last_fullsized":"2024-09-10T21:12:25.784766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:15.277562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785343+0000","last_change":"2024-09-10T21:12:24.826783+0000","last_active":"2024-09-10T21:12:25.785343+0000","last_peered":"2024-09-10T21:12:25.785343+0000","last_clean":"2024-09-10T21:12:25.785343+0000","last_became_active":"2024-09-10T21:12:24.826470+0000","last_became_peered":"2024-09-10T21:12:24.826470+0000","last_unstale":"2024-09-10T21:12:25.785343+0000","last_undegraded":"2024-09-10T21:12:25.785343+0000","last_fullsized":"2024-09-10T21:12:25.785343+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:37.303480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785877+0000","last_change":"2024-09-10T21:12:24.820100+0000","last_active":"2024-09-10T21:12:25.785877+0000","last_peered":"2024-09-10T21:12:25.785877+0000","last_clean":"2024-09-10T21:12:25.785877+0000","last_became_active":"2024-09-10T21:12:24.819477+0000","last_became_peered":"2024-09-10T21:12:24.819477+0000","last_unstale":"2024-09-10T21:12:25.785877+0000","last_undegraded":"2024-09-10T21:12:25.785877+0000","last_fullsized":"2024-09-10T21:12:25.785877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:22.872734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786381+0000","last_change":"2024-09-10T21:12:24.807623+0000","last_active":"2024-09-10T21:12:25.786381+0000","last_peered":"2024-09-10T21:12:25.786381+0000","last_clean":"2024-09-10T21:12:25.786381+0000","last_became_active":"2024-09-10T21:12:24.807290+0000","last_became_peered":"2024-09-10T21:12:24.807290+0000","last_unstale":"2024-09-10T21:12:25.786381+0000","last_undegraded":"2024-09-10T21:12:25.786381+0000","last_fullsized":"2024-09-10T21:12:25.786381+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:32.446130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428017+0000","last_change":"2024-09-10T21:12:24.822563+0000","last_active":"2024-09-10T21:12:26.428017+0000","last_peered":"2024-09-10T21:12:26.428017+0000","last_clean":"2024-09-10T21:12:26.428017+0000","last_became_active":"2024-09-10T21:12:24.822299+0000","last_became_peered":"2024-09-10T21:12:24.822299+0000","last_unstale":"2024-09-10T21:12:26.428017+0000","last_undegraded":"2024-09-10T21:12:26.428017+0000","last_fullsized":"2024-09-10T21:12:26.428017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:15.277562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784799+0000","last_change":"2024-09-10T21:12:24.804138+0000","last_active":"2024-09-10T21:12:25.784799+0000","last_peered":"2024-09-10T21:12:25.784799+0000","last_clean":"2024-09-10T21:12:25.784799+0000","last_became_active":"2024-09-10T21:12:24.803868+0000","last_became_peered":"2024-09-10T21:12:24.803868+0000","last_unstale":"2024-09-10T21:12:25.784799+0000","last_undegraded":"2024-09-10T21:12:25.784799+0000","last_fullsized":"2024-09-10T21:12:25.784799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:06.104316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429476+0000","last_change":"2024-09-10T21:12:24.806841+0000","last_active":"2024-09-10T21:12:26.429476+0000","last_peered":"2024-09-10T21:12:26.429476+0000","last_clean":"2024-09-10T21:12:26.429476+0000","last_became_active":"2024-09-10T21:12:24.804956+0000","last_became_peered":"2024-09-10T21:12:24.804956+0000","last_unstale":"2024-09-10T21:12:26.429476+0000","last_undegraded":"2024-09-10T21:12:26.429476+0000","last_fullsized":"2024-09-10T21:12:26.429476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:46.274444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428515+0000","last_change":"2024-09-10T21:12:24.817361+0000","last_active":"2024-09-10T21:12:26.428515+0000","last_peered":"2024-09-10T21:12:26.428515+0000","last_clean":"2024-09-10T21:12:26.428515+0000","last_became_active":"2024-09-10T21:12:24.815622+0000","last_became_peered":"2024-09-10T21:12:24.815622+0000","last_unstale":"2024-09-10T21:12:26.428515+0000","last_undegraded":"2024-09-10T21:12:26.428515+0000","last_fullsized":"2024-09-10T21:12:26.428515+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:11.907167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428188+0000","last_change":"2024-09-10T21:12:24.792792+0000","last_active":"2024-09-10T21:12:26.428188+0000","last_peered":"2024-09-10T21:12:26.428188+0000","last_clean":"2024-09-10T21:12:26.428188+0000","last_became_active":"2024-09-10T21:12:24.792220+0000","last_became_peered":"2024-09-10T21:12:24.792220+0000","last_unstale":"2024-09-10T21:12:26.428188+0000","last_undegraded":"2024-09-10T21:12:26.428188+0000","last_fullsized":"2024-09-10T21:12:26.428188+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:39.747387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786232+0000","last_change":"2024-09-10T21:12:24.824108+0000","last_active":"2024-09-10T21:12:25.786232+0000","last_peered":"2024-09-10T21:12:25.786232+0000","last_clean":"2024-09-10T21:12:25.786232+0000","last_became_active":"2024-09-10T21:12:24.823125+0000","last_became_peered":"2024-09-10T21:12:24.823125+0000","last_unstale":"2024-09-10T21:12:25.786232+0000","last_undegraded":"2024-09-10T21:12:25.786232+0000","last_fullsized":"2024-09-10T21:12:25.786232+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:41.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427209+0000","last_change":"2024-09-10T21:12:24.803379+0000","last_active":"2024-09-10T21:12:26.427209+0000","last_peered":"2024-09-10T21:12:26.427209+0000","last_clean":"2024-09-10T21:12:26.427209+0000","last_became_active":"2024-09-10T21:12:24.802088+0000","last_became_peered":"2024-09-10T21:12:24.802088+0000","last_unstale":"2024-09-10T21:12:26.427209+0000","last_undegraded":"2024-09-10T21:12:26.427209+0000","last_fullsized":"2024-09-10T21:12:26.427209+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:45.236002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428095+0000","last_change":"2024-09-10T21:12:24.800438+0000","last_active":"2024-09-10T21:12:26.428095+0000","last_peered":"2024-09-10T21:12:26.428095+0000","last_clean":"2024-09-10T21:12:26.428095+0000","last_became_active":"2024-09-10T21:12:24.799628+0000","last_became_peered":"2024-09-10T21:12:24.799628+0000","last_unstale":"2024-09-10T21:12:26.428095+0000","last_undegraded":"2024-09-10T21:12:26.428095+0000","last_fullsized":"2024-09-10T21:12:26.428095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:22.261573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784410+0000","last_change":"2024-09-10T21:12:24.822685+0000","last_active":"2024-09-10T21:12:25.784410+0000","last_peered":"2024-09-10T21:12:25.784410+0000","last_clean":"2024-09-10T21:12:25.784410+0000","last_became_active":"2024-09-10T21:12:24.821398+0000","last_became_peered":"2024-09-10T21:12:24.821398+0000","last_unstale":"2024-09-10T21:12:25.784410+0000","last_undegraded":"2024-09-10T21:12:25.784410+0000","last_fullsized":"2024-09-10T21:12:25.784410+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:02.665366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428387+0000","last_change":"2024-09-10T21:12:24.807404+0000","last_active":"2024-09-10T21:12:26.428387+0000","last_peered":"2024-09-10T21:12:26.428387+0000","last_clean":"2024-09-10T21:12:26.428387+0000","last_became_active":"2024-09-10T21:12:24.807144+0000","last_became_peered":"2024-09-10T21:12:24.807144+0000","last_unstale":"2024-09-10T21:12:26.428387+0000","last_undegraded":"2024-09-10T21:12:26.428387+0000","last_fullsized":"2024-09-10T21:12:26.428387+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:37.987597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786744+0000","last_change":"2024-09-10T21:12:24.829538+0000","last_active":"2024-09-10T21:12:25.786744+0000","last_peered":"2024-09-10T21:12:25.786744+0000","last_clean":"2024-09-10T21:12:25.786744+0000","last_became_active":"2024-09-10T21:12:24.829250+0000","last_became_peered":"2024-09-10T21:12:24.829250+0000","last_unstale":"2024-09-10T21:12:25.786744+0000","last_undegraded":"2024-09-10T21:12:25.786744+0000","last_fullsized":"2024-09-10T21:12:25.786744+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:47.207849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427811+0000","last_change":"2024-09-10T21:12:24.795084+0000","last_active":"2024-09-10T21:12:26.427811+0000","last_peered":"2024-09-10T21:12:26.427811+0000","last_clean":"2024-09-10T21:12:26.427811+0000","last_became_active":"2024-09-10T21:12:24.794823+0000","last_became_peered":"2024-09-10T21:12:24.794823+0000","last_unstale":"2024-09-10T21:12:26.427811+0000","last_undegraded":"2024-09-10T21:12:26.427811+0000","last_fullsized":"2024-09-10T21:12:26.427811+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:26.691816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428069+0000","last_change":"2024-09-10T21:12:24.793348+0000","last_active":"2024-09-10T21:12:26.428069+0000","last_peered":"2024-09-10T21:12:26.428069+0000","last_clean":"2024-09-10T21:12:26.428069+0000","last_became_active":"2024-09-10T21:12:24.793089+0000","last_became_peered":"2024-09-10T21:12:24.793089+0000","last_unstale":"2024-09-10T21:12:26.428069+0000","last_undegraded":"2024-09-10T21:12:26.428069+0000","last_fullsized":"2024-09-10T21:12:26.428069+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:03.310670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428591+0000","last_change":"2024-09-10T21:12:24.815439+0000","last_active":"2024-09-10T21:12:26.428591+0000","last_peered":"2024-09-10T21:12:26.428591+0000","last_clean":"2024-09-10T21:12:26.428591+0000","last_became_active":"2024-09-10T21:12:24.815191+0000","last_became_peered":"2024-09-10T21:12:24.815191+0000","last_unstale":"2024-09-10T21:12:26.428591+0000","last_undegraded":"2024-09-10T21:12:26.428591+0000","last_fullsized":"2024-09-10T21:12:26.428591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:14.014096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784439+0000","last_change":"2024-09-10T21:12:24.792836+0000","last_active":"2024-09-10T21:12:25.784439+0000","last_peered":"2024-09-10T21:12:25.784439+0000","last_clean":"2024-09-10T21:12:25.784439+0000","last_became_active":"2024-09-10T21:12:24.792569+0000","last_became_peered":"2024-09-10T21:12:24.792569+0000","last_unstale":"2024-09-10T21:12:25.784439+0000","last_undegraded":"2024-09-10T21:12:25.784439+0000","last_fullsized":"2024-09-10T21:12:25.784439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:31.753172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785090+0000","last_change":"2024-09-10T21:12:24.808015+0000","last_active":"2024-09-10T21:12:25.785090+0000","last_peered":"2024-09-10T21:12:25.785090+0000","last_clean":"2024-09-10T21:12:25.785090+0000","last_became_active":"2024-09-10T21:12:24.807723+0000","last_became_peered":"2024-09-10T21:12:24.807723+0000","last_unstale":"2024-09-10T21:12:25.785090+0000","last_undegraded":"2024-09-10T21:12:25.785090+0000","last_fullsized":"2024-09-10T21:12:25.785090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:26.164431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785759+0000","last_change":"2024-09-10T21:12:24.812500+0000","last_active":"2024-09-10T21:12:25.785759+0000","last_peered":"2024-09-10T21:12:25.785759+0000","last_clean":"2024-09-10T21:12:25.785759+0000","last_became_active":"2024-09-10T21:12:24.812143+0000","last_became_peered":"2024-09-10T21:12:24.812143+0000","last_unstale":"2024-09-10T21:12:25.785759+0000","last_undegraded":"2024-09-10T21:12:25.785759+0000","last_fullsized":"2024-09-10T21:12:25.785759+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:33.667666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427841+0000","last_change":"2024-09-10T21:12:24.817360+0000","last_active":"2024-09-10T21:12:26.427841+0000","last_peered":"2024-09-10T21:12:26.427841+0000","last_clean":"2024-09-10T21:12:26.427841+0000","last_became_active":"2024-09-10T21:12:24.817059+0000","last_became_peered":"2024-09-10T21:12:24.817059+0000","last_unstale":"2024-09-10T21:12:26.427841+0000","last_undegraded":"2024-09-10T21:12:26.427841+0000","last_fullsized":"2024-09-10T21:12:26.427841+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:33.659623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.787046+0000","last_change":"2024-09-10T21:12:23.795968+0000","last_active":"2024-09-10T21:12:25.787046+0000","last_peered":"2024-09-10T21:12:25.787046+0000","last_clean":"2024-09-10T21:12:25.787046+0000","last_became_active":"2024-09-10T21:12:23.795674+0000","last_became_peered":"2024-09-10T21:12:23.795674+0000","last_unstale":"2024-09-10T21:12:25.787046+0000","last_undegraded":"2024-09-10T21:12:25.787046+0000","last_fullsized":"2024-09-10T21:12:25.787046+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:34.576509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428635+0000","last_change":"2024-09-10T21:12:24.801727+0000","last_active":"2024-09-10T21:12:26.428635+0000","last_peered":"2024-09-10T21:12:26.428635+0000","last_clean":"2024-09-10T21:12:26.428635+0000","last_became_active":"2024-09-10T21:12:24.801282+0000","last_became_peered":"2024-09-10T21:12:24.801282+0000","last_unstale":"2024-09-10T21:12:26.428635+0000","last_undegraded":"2024-09-10T21:12:26.428635+0000","last_fullsized":"2024-09-10T21:12:26.428635+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:52.776680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428530+0000","last_change":"2024-09-10T21:12:23.799107+0000","last_active":"2024-09-10T21:12:26.428530+0000","last_peered":"2024-09-10T21:12:26.428530+0000","last_clean":"2024-09-10T21:12:26.428530+0000","last_became_active":"2024-09-10T21:12:23.798815+0000","last_became_peered":"2024-09-10T21:12:23.798815+0000","last_unstale":"2024-09-10T21:12:26.428530+0000","last_undegraded":"2024-09-10T21:12:26.428530+0000","last_fullsized":"2024-09-10T21:12:26.428530+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:41.715763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785035+0000","last_change":"2024-09-10T21:12:24.795186+0000","last_active":"2024-09-10T21:12:25.785035+0000","last_peered":"2024-09-10T21:12:25.785035+0000","last_clean":"2024-09-10T21:12:25.785035+0000","last_became_active":"2024-09-10T21:12:24.794735+0000","last_became_peered":"2024-09-10T21:12:24.794735+0000","last_unstale":"2024-09-10T21:12:25.785035+0000","last_undegraded":"2024-09-10T21:12:25.785035+0000","last_fullsized":"2024-09-10T21:12:25.785035+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:37.987597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786956+0000","last_change":"2024-09-10T21:12:23.795368+0000","last_active":"2024-09-10T21:12:25.786956+0000","last_peered":"2024-09-10T21:12:25.786956+0000","last_clean":"2024-09-10T21:12:25.786956+0000","last_became_active":"2024-09-10T21:12:23.795086+0000","last_became_peered":"2024-09-10T21:12:23.795086+0000","last_unstale":"2024-09-10T21:12:25.786956+0000","last_undegraded":"2024-09-10T21:12:25.786956+0000","last_fullsized":"2024-09-10T21:12:25.786956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:32.008963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786672+0000","last_change":"2024-09-10T21:12:24.808003+0000","last_active":"2024-09-10T21:12:25.786672+0000","last_peered":"2024-09-10T21:12:25.786672+0000","last_clean":"2024-09-10T21:12:25.786672+0000","last_became_active":"2024-09-10T21:12:24.807702+0000","last_became_peered":"2024-09-10T21:12:24.807702+0000","last_unstale":"2024-09-10T21:12:25.786672+0000","last_undegraded":"2024-09-10T21:12:25.786672+0000","last_fullsized":"2024-09-10T21:12:25.786672+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:26.038549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428269+0000","last_change":"2024-09-10T21:12:23.799265+0000","last_active":"2024-09-10T21:12:26.428269+0000","last_peered":"2024-09-10T21:12:26.428269+0000","last_clean":"2024-09-10T21:12:26.428269+0000","last_became_active":"2024-09-10T21:12:23.798990+0000","last_became_peered":"2024-09-10T21:12:23.798990+0000","last_unstale":"2024-09-10T21:12:26.428269+0000","last_undegraded":"2024-09-10T21:12:26.428269+0000","last_fullsized":"2024-09-10T21:12:26.428269+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:18.976586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429200+0000","last_change":"2024-09-10T21:12:24.797557+0000","last_active":"2024-09-10T21:12:26.429200+0000","last_peered":"2024-09-10T21:12:26.429200+0000","last_clean":"2024-09-10T21:12:26.429200+0000","last_became_active":"2024-09-10T21:12:24.797284+0000","last_became_peered":"2024-09-10T21:12:24.797284+0000","last_unstale":"2024-09-10T21:12:26.429200+0000","last_undegraded":"2024-09-10T21:12:26.429200+0000","last_fullsized":"2024-09-10T21:12:26.429200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:51:34.878779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786779+0000","last_change":"2024-09-10T21:12:23.798175+0000","last_active":"2024-09-10T21:12:25.786779+0000","last_peered":"2024-09-10T21:12:25.786779+0000","last_clean":"2024-09-10T21:12:25.786779+0000","last_became_active":"2024-09-10T21:12:23.797844+0000","last_became_peered":"2024-09-10T21:12:23.797844+0000","last_unstale":"2024-09-10T21:12:25.786779+0000","last_undegraded":"2024-09-10T21:12:25.786779+0000","last_fullsized":"2024-09-10T21:12:25.786779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:57.897857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427917+0000","last_change":"2024-09-10T21:12:24.820636+0000","last_active":"2024-09-10T21:12:26.427917+0000","last_peered":"2024-09-10T21:12:26.427917+0000","last_clean":"2024-09-10T21:12:26.427917+0000","last_became_active":"2024-09-10T21:12:24.820280+0000","last_became_peered":"2024-09-10T21:12:24.820280+0000","last_unstale":"2024-09-10T21:12:26.427917+0000","last_undegraded":"2024-09-10T21:12:26.427917+0000","last_fullsized":"2024-09-10T21:12:26.427917+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:06.763115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429325+0000","last_change":"2024-09-10T21:12:23.803893+0000","last_active":"2024-09-10T21:12:26.429325+0000","last_peered":"2024-09-10T21:12:26.429325+0000","last_clean":"2024-09-10T21:12:26.429325+0000","last_became_active":"2024-09-10T21:12:23.803599+0000","last_became_peered":"2024-09-10T21:12:23.803599+0000","last_unstale":"2024-09-10T21:12:26.429325+0000","last_undegraded":"2024-09-10T21:12:26.429325+0000","last_fullsized":"2024-09-10T21:12:26.429325+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:46.177434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427703+0000","last_change":"2024-09-10T21:12:24.813412+0000","last_active":"2024-09-10T21:12:26.427703+0000","last_peered":"2024-09-10T21:12:26.427703+0000","last_clean":"2024-09-10T21:12:26.427703+0000","last_became_active":"2024-09-10T21:12:24.802579+0000","last_became_peered":"2024-09-10T21:12:24.802579+0000","last_unstale":"2024-09-10T21:12:26.427703+0000","last_undegraded":"2024-09-10T21:12:26.427703+0000","last_fullsized":"2024-09-10T21:12:26.427703+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:51.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427719+0000","last_change":"2024-09-10T21:12:23.793275+0000","last_active":"2024-09-10T21:12:26.427719+0000","last_peered":"2024-09-10T21:12:26.427719+0000","last_clean":"2024-09-10T21:12:26.427719+0000","last_became_active":"2024-09-10T21:12:23.792867+0000","last_became_peered":"2024-09-10T21:12:23.792867+0000","last_unstale":"2024-09-10T21:12:26.427719+0000","last_undegraded":"2024-09-10T21:12:26.427719+0000","last_fullsized":"2024-09-10T21:12:26.427719+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:54.791716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427761+0000","last_change":"2024-09-10T21:12:24.806299+0000","last_active":"2024-09-10T21:12:26.427761+0000","last_peered":"2024-09-10T21:12:26.427761+0000","last_clean":"2024-09-10T21:12:26.427761+0000","last_became_active":"2024-09-10T21:12:24.806025+0000","last_became_peered":"2024-09-10T21:12:24.806025+0000","last_unstale":"2024-09-10T21:12:26.427761+0000","last_undegraded":"2024-09-10T21:12:26.427761+0000","last_fullsized":"2024-09-10T21:12:26.427761+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:19.306195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427757+0000","last_change":"2024-09-10T21:12:23.798078+0000","last_active":"2024-09-10T21:12:26.427757+0000","last_peered":"2024-09-10T21:12:26.427757+0000","last_clean":"2024-09-10T21:12:26.427757+0000","last_became_active":"2024-09-10T21:12:23.797773+0000","last_became_peered":"2024-09-10T21:12:23.797773+0000","last_unstale":"2024-09-10T21:12:26.427757+0000","last_undegraded":"2024-09-10T21:12:26.427757+0000","last_fullsized":"2024-09-10T21:12:26.427757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:34.190967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429612+0000","last_change":"2024-09-10T21:12:24.804022+0000","last_active":"2024-09-10T21:12:26.429612+0000","last_peered":"2024-09-10T21:12:26.429612+0000","last_clean":"2024-09-10T21:12:26.429612+0000","last_became_active":"2024-09-10T21:12:24.803828+0000","last_became_peered":"2024-09-10T21:12:24.803828+0000","last_unstale":"2024-09-10T21:12:26.429612+0000","last_undegraded":"2024-09-10T21:12:26.429612+0000","last_fullsized":"2024-09-10T21:12:26.429612+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:33.659623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428656+0000","last_change":"2024-09-10T21:12:23.808653+0000","last_active":"2024-09-10T21:12:26.428656+0000","last_peered":"2024-09-10T21:12:26.428656+0000","last_clean":"2024-09-10T21:12:26.428656+0000","last_became_active":"2024-09-10T21:12:23.808383+0000","last_became_peered":"2024-09-10T21:12:23.808383+0000","last_unstale":"2024-09-10T21:12:26.428656+0000","last_undegraded":"2024-09-10T21:12:26.428656+0000","last_fullsized":"2024-09-10T21:12:26.428656+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:55.562523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428596+0000","last_change":"2024-09-10T21:12:24.802802+0000","last_active":"2024-09-10T21:12:26.428596+0000","last_peered":"2024-09-10T21:12:26.428596+0000","last_clean":"2024-09-10T21:12:26.428596+0000","last_became_active":"2024-09-10T21:12:24.802449+0000","last_became_peered":"2024-09-10T21:12:24.802449+0000","last_unstale":"2024-09-10T21:12:26.428596+0000","last_undegraded":"2024-09-10T21:12:26.428596+0000","last_fullsized":"2024-09-10T21:12:26.428596+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:37.497080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428077+0000","last_change":"2024-09-10T21:12:23.807246+0000","last_active":"2024-09-10T21:12:26.428077+0000","last_peered":"2024-09-10T21:12:26.428077+0000","last_clean":"2024-09-10T21:12:26.428077+0000","last_became_active":"2024-09-10T21:12:23.806959+0000","last_became_peered":"2024-09-10T21:12:23.806959+0000","last_unstale":"2024-09-10T21:12:26.428077+0000","last_undegraded":"2024-09-10T21:12:26.428077+0000","last_fullsized":"2024-09-10T21:12:26.428077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:46.804078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786038+0000","last_change":"2024-09-10T21:12:24.822928+0000","last_active":"2024-09-10T21:12:25.786038+0000","last_peered":"2024-09-10T21:12:25.786038+0000","last_clean":"2024-09-10T21:12:25.786038+0000","last_became_active":"2024-09-10T21:12:24.822626+0000","last_became_peered":"2024-09-10T21:12:24.822626+0000","last_unstale":"2024-09-10T21:12:25.786038+0000","last_undegraded":"2024-09-10T21:12:25.786038+0000","last_fullsized":"2024-09-10T21:12:25.786038+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:06.763115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784521+0000","last_change":"2024-09-10T21:12:23.790411+0000","last_active":"2024-09-10T21:12:25.784521+0000","last_peered":"2024-09-10T21:12:25.784521+0000","last_clean":"2024-09-10T21:12:25.784521+0000","last_became_active":"2024-09-10T21:12:23.790109+0000","last_became_peered":"2024-09-10T21:12:23.790109+0000","last_unstale":"2024-09-10T21:12:25.784521+0000","last_undegraded":"2024-09-10T21:12:25.784521+0000","last_fullsized":"2024-09-10T21:12:25.784521+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:25.257787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428953+0000","last_change":"2024-09-10T21:12:24.808556+0000","last_active":"2024-09-10T21:12:26.428953+0000","last_peered":"2024-09-10T21:12:26.428953+0000","last_clean":"2024-09-10T21:12:26.428953+0000","last_became_active":"2024-09-10T21:12:24.808281+0000","last_became_peered":"2024-09-10T21:12:24.808281+0000","last_unstale":"2024-09-10T21:12:26.428953+0000","last_undegraded":"2024-09-10T21:12:26.428953+0000","last_fullsized":"2024-09-10T21:12:26.428953+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:04.882193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428411+0000","last_change":"2024-09-10T21:12:23.802815+0000","last_active":"2024-09-10T21:12:26.428411+0000","last_peered":"2024-09-10T21:12:26.428411+0000","last_clean":"2024-09-10T21:12:26.428411+0000","last_became_active":"2024-09-10T21:12:23.802540+0000","last_became_peered":"2024-09-10T21:12:23.802540+0000","last_unstale":"2024-09-10T21:12:26.428411+0000","last_undegraded":"2024-09-10T21:12:26.428411+0000","last_fullsized":"2024-09-10T21:12:26.428411+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:22.630049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428740+0000","last_change":"2024-09-10T21:12:24.822282+0000","last_active":"2024-09-10T21:12:26.428740+0000","last_peered":"2024-09-10T21:12:26.428740+0000","last_clean":"2024-09-10T21:12:26.428740+0000","last_became_active":"2024-09-10T21:12:24.821907+0000","last_became_peered":"2024-09-10T21:12:24.821907+0000","last_unstale":"2024-09-10T21:12:26.428740+0000","last_undegraded":"2024-09-10T21:12:26.428740+0000","last_fullsized":"2024-09-10T21:12:26.428740+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:45.236002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427043+0000","last_change":"2024-09-10T21:12:23.800329+0000","last_active":"2024-09-10T21:12:26.427043+0000","last_peered":"2024-09-10T21:12:26.427043+0000","last_clean":"2024-09-10T21:12:26.427043+0000","last_became_active":"2024-09-10T21:12:23.799441+0000","last_became_peered":"2024-09-10T21:12:23.799441+0000","last_unstale":"2024-09-10T21:12:26.427043+0000","last_undegraded":"2024-09-10T21:12:26.427043+0000","last_fullsized":"2024-09-10T21:12:26.427043+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:41.839052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427386+0000","last_change":"2024-09-10T21:12:24.799963+0000","last_active":"2024-09-10T21:12:26.427386+0000","last_peered":"2024-09-10T21:12:26.427386+0000","last_clean":"2024-09-10T21:12:26.427386+0000","last_became_active":"2024-09-10T21:12:24.799685+0000","last_became_peered":"2024-09-10T21:12:24.799685+0000","last_unstale":"2024-09-10T21:12:26.427386+0000","last_undegraded":"2024-09-10T21:12:26.427386+0000","last_fullsized":"2024-09-10T21:12:26.427386+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:14.399311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427958+0000","last_change":"2024-09-10T21:12:23.800966+0000","last_active":"2024-09-10T21:12:26.427958+0000","last_peered":"2024-09-10T21:12:26.427958+0000","last_clean":"2024-09-10T21:12:26.427958+0000","last_became_active":"2024-09-10T21:12:23.800647+0000","last_became_peered":"2024-09-10T21:12:23.800647+0000","last_unstale":"2024-09-10T21:12:26.427958+0000","last_undegraded":"2024-09-10T21:12:26.427958+0000","last_fullsized":"2024-09-10T21:12:26.427958+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:56.744606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427189+0000","last_change":"2024-09-10T21:12:24.799381+0000","last_active":"2024-09-10T21:12:26.427189+0000","last_peered":"2024-09-10T21:12:26.427189+0000","last_clean":"2024-09-10T21:12:26.427189+0000","last_became_active":"2024-09-10T21:12:24.798305+0000","last_became_peered":"2024-09-10T21:12:24.798305+0000","last_unstale":"2024-09-10T21:12:26.427189+0000","last_undegraded":"2024-09-10T21:12:26.427189+0000","last_fullsized":"2024-09-10T21:12:26.427189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:37.497080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784996+0000","last_change":"2024-09-10T21:12:23.806810+0000","last_active":"2024-09-10T21:12:25.784996+0000","last_peered":"2024-09-10T21:12:25.784996+0000","last_clean":"2024-09-10T21:12:25.784996+0000","last_became_active":"2024-09-10T21:12:23.806235+0000","last_became_peered":"2024-09-10T21:12:23.806235+0000","last_unstale":"2024-09-10T21:12:25.784996+0000","last_undegraded":"2024-09-10T21:12:25.784996+0000","last_fullsized":"2024-09-10T21:12:25.784996+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:11.902707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784660+0000","last_change":"2024-09-10T21:12:24.800682+0000","last_active":"2024-09-10T21:12:25.784660+0000","last_peered":"2024-09-10T21:12:25.784660+0000","last_clean":"2024-09-10T21:12:25.784660+0000","last_became_active":"2024-09-10T21:12:24.800333+0000","last_became_peered":"2024-09-10T21:12:24.800333+0000","last_unstale":"2024-09-10T21:12:25.784660+0000","last_undegraded":"2024-09-10T21:12:25.784660+0000","last_fullsized":"2024-09-10T21:12:25.784660+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:49.709340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":23,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427889+0000","last_change":"2024-09-10T21:12:23.796942+0000","last_active":"2024-09-10T21:12:26.427889+0000","last_peered":"2024-09-10T21:12:26.427889+0000","last_clean":"2024-09-10T21:12:26.427889+0000","last_became_active":"2024-09-10T21:12:23.796661+0000","last_became_peered":"2024-09-10T21:12:23.796661+0000","last_unstale":"2024-09-10T21:12:26.427889+0000","last_undegraded":"2024-09-10T21:12:26.427889+0000","last_fullsized":"2024-09-10T21:12:26.427889+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:05.892601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427615+0000","last_change":"2024-09-10T21:12:24.800375+0000","last_active":"2024-09-10T21:12:26.427615+0000","last_peered":"2024-09-10T21:12:26.427615+0000","last_clean":"2024-09-10T21:12:26.427615+0000","last_became_active":"2024-09-10T21:12:24.800087+0000","last_became_peered":"2024-09-10T21:12:24.800087+0000","last_unstale":"2024-09-10T21:12:26.427615+0000","last_undegraded":"2024-09-10T21:12:26.427615+0000","last_fullsized":"2024-09-10T21:12:26.427615+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:46.274444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786575+0000","last_change":"2024-09-10T21:12:23.797369+0000","last_active":"2024-09-10T21:12:25.786575+0000","last_peered":"2024-09-10T21:12:25.786575+0000","last_clean":"2024-09-10T21:12:25.786575+0000","last_became_active":"2024-09-10T21:12:23.796137+0000","last_became_peered":"2024-09-10T21:12:23.796137+0000","last_unstale":"2024-09-10T21:12:25.786575+0000","last_undegraded":"2024-09-10T21:12:25.786575+0000","last_fullsized":"2024-09-10T21:12:25.786575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:19.161900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427399+0000","last_change":"2024-09-10T21:12:24.800698+0000","last_active":"2024-09-10T21:12:26.427399+0000","last_peered":"2024-09-10T21:12:26.427399+0000","last_clean":"2024-09-10T21:12:26.427399+0000","last_became_active":"2024-09-10T21:12:24.799724+0000","last_became_peered":"2024-09-10T21:12:24.799724+0000","last_unstale":"2024-09-10T21:12:26.427399+0000","last_undegraded":"2024-09-10T21:12:26.427399+0000","last_fullsized":"2024-09-10T21:12:26.427399+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:44.428772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429684+0000","last_change":"2024-09-10T21:12:23.807367+0000","last_active":"2024-09-10T21:12:26.429684+0000","last_peered":"2024-09-10T21:12:26.429684+0000","last_clean":"2024-09-10T21:12:26.429684+0000","last_became_active":"2024-09-10T21:12:23.806960+0000","last_became_peered":"2024-09-10T21:12:23.806960+0000","last_unstale":"2024-09-10T21:12:26.429684+0000","last_undegraded":"2024-09-10T21:12:26.429684+0000","last_fullsized":"2024-09-10T21:12:26.429684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:21.476191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785216+0000","last_change":"2024-09-10T21:12:24.821547+0000","last_active":"2024-09-10T21:12:25.785216+0000","last_peered":"2024-09-10T21:12:25.785216+0000","last_clean":"2024-09-10T21:12:25.785216+0000","last_became_active":"2024-09-10T21:12:24.821285+0000","last_became_peered":"2024-09-10T21:12:24.821285+0000","last_unstale":"2024-09-10T21:12:25.785216+0000","last_undegraded":"2024-09-10T21:12:25.785216+0000","last_fullsized":"2024-09-10T21:12:25.785216+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:11.907167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429276+0000","last_change":"2024-09-10T21:12:23.810651+0000","last_active":"2024-09-10T21:12:26.429276+0000","last_peered":"2024-09-10T21:12:26.429276+0000","last_clean":"2024-09-10T21:12:26.429276+0000","last_became_active":"2024-09-10T21:12:23.809721+0000","last_became_peered":"2024-09-10T21:12:23.809721+0000","last_unstale":"2024-09-10T21:12:26.429276+0000","last_undegraded":"2024-09-10T21:12:26.429276+0000","last_fullsized":"2024-09-10T21:12:26.429276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:13.969390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429203+0000","last_change":"2024-09-10T21:12:24.802031+0000","last_active":"2024-09-10T21:12:26.429203+0000","last_peered":"2024-09-10T21:12:26.429203+0000","last_clean":"2024-09-10T21:12:26.429203+0000","last_became_active":"2024-09-10T21:12:24.801761+0000","last_became_peered":"2024-09-10T21:12:24.801761+0000","last_unstale":"2024-09-10T21:12:26.429203+0000","last_undegraded":"2024-09-10T21:12:26.429203+0000","last_fullsized":"2024-09-10T21:12:26.429203+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:30.695194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786779+0000","last_change":"2024-09-10T21:12:23.806377+0000","last_active":"2024-09-10T21:12:25.786779+0000","last_peered":"2024-09-10T21:12:25.786779+0000","last_clean":"2024-09-10T21:12:25.786779+0000","last_became_active":"2024-09-10T21:12:23.806032+0000","last_became_peered":"2024-09-10T21:12:23.806032+0000","last_unstale":"2024-09-10T21:12:25.786779+0000","last_undegraded":"2024-09-10T21:12:25.786779+0000","last_fullsized":"2024-09-10T21:12:25.786779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:18.235286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428188+0000","last_change":"2024-09-10T21:12:24.823192+0000","last_active":"2024-09-10T21:12:26.428188+0000","last_peered":"2024-09-10T21:12:26.428188+0000","last_clean":"2024-09-10T21:12:26.428188+0000","last_became_active":"2024-09-10T21:12:24.822927+0000","last_became_peered":"2024-09-10T21:12:24.822927+0000","last_unstale":"2024-09-10T21:12:26.428188+0000","last_undegraded":"2024-09-10T21:12:26.428188+0000","last_fullsized":"2024-09-10T21:12:26.428188+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:16.729677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427725+0000","last_change":"2024-09-10T21:12:23.804977+0000","last_active":"2024-09-10T21:12:26.427725+0000","last_peered":"2024-09-10T21:12:26.427725+0000","last_clean":"2024-09-10T21:12:26.427725+0000","last_became_active":"2024-09-10T21:12:23.804671+0000","last_became_peered":"2024-09-10T21:12:23.804671+0000","last_unstale":"2024-09-10T21:12:26.427725+0000","last_undegraded":"2024-09-10T21:12:26.427725+0000","last_fullsized":"2024-09-10T21:12:26.427725+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:44.061600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786247+0000","last_change":"2024-09-10T21:12:24.804621+0000","last_active":"2024-09-10T21:12:25.786247+0000","last_peered":"2024-09-10T21:12:25.786247+0000","last_clean":"2024-09-10T21:12:25.786247+0000","last_became_active":"2024-09-10T21:12:24.804273+0000","last_became_peered":"2024-09-10T21:12:24.804273+0000","last_unstale":"2024-09-10T21:12:25.786247+0000","last_undegraded":"2024-09-10T21:12:25.786247+0000","last_fullsized":"2024-09-10T21:12:25.786247+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:14.393333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428072+0000","last_change":"2024-09-10T21:12:23.808671+0000","last_active":"2024-09-10T21:12:26.428072+0000","last_peered":"2024-09-10T21:12:26.428072+0000","last_clean":"2024-09-10T21:12:26.428072+0000","last_became_active":"2024-09-10T21:12:23.808137+0000","last_became_peered":"2024-09-10T21:12:23.808137+0000","last_unstale":"2024-09-10T21:12:26.428072+0000","last_undegraded":"2024-09-10T21:12:26.428072+0000","last_fullsized":"2024-09-10T21:12:26.428072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:02.838356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786010+0000","last_change":"2024-09-10T21:12:24.812457+0000","last_active":"2024-09-10T21:12:25.786010+0000","last_peered":"2024-09-10T21:12:25.786010+0000","last_clean":"2024-09-10T21:12:25.786010+0000","last_became_active":"2024-09-10T21:12:24.812056+0000","last_became_peered":"2024-09-10T21:12:24.812056+0000","last_unstale":"2024-09-10T21:12:25.786010+0000","last_undegraded":"2024-09-10T21:12:25.786010+0000","last_fullsized":"2024-09-10T21:12:25.786010+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:51.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427529+0000","last_change":"2024-09-10T21:12:23.808302+0000","last_active":"2024-09-10T21:12:26.427529+0000","last_peered":"2024-09-10T21:12:26.427529+0000","last_clean":"2024-09-10T21:12:26.427529+0000","last_became_active":"2024-09-10T21:12:23.807962+0000","last_became_peered":"2024-09-10T21:12:23.807962+0000","last_unstale":"2024-09-10T21:12:26.427529+0000","last_undegraded":"2024-09-10T21:12:26.427529+0000","last_fullsized":"2024-09-10T21:12:26.427529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:39.325026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786383+0000","last_change":"2024-09-10T21:12:24.819374+0000","last_active":"2024-09-10T21:12:25.786383+0000","last_peered":"2024-09-10T21:12:25.786383+0000","last_clean":"2024-09-10T21:12:25.786383+0000","last_became_active":"2024-09-10T21:12:24.819016+0000","last_became_peered":"2024-09-10T21:12:24.819016+0000","last_unstale":"2024-09-10T21:12:25.786383+0000","last_undegraded":"2024-09-10T21:12:25.786383+0000","last_fullsized":"2024-09-10T21:12:25.786383+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:16.729677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427497+0000","last_change":"2024-09-10T21:12:23.801081+0000","last_active":"2024-09-10T21:12:26.427497+0000","last_peered":"2024-09-10T21:12:26.427497+0000","last_clean":"2024-09-10T21:12:26.427497+0000","last_became_active":"2024-09-10T21:12:23.800792+0000","last_became_peered":"2024-09-10T21:12:23.800792+0000","last_unstale":"2024-09-10T21:12:26.427497+0000","last_undegraded":"2024-09-10T21:12:26.427497+0000","last_fullsized":"2024-09-10T21:12:26.427497+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:24.670926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429005+0000","last_change":"2024-09-10T21:12:24.805825+0000","last_active":"2024-09-10T21:12:26.429005+0000","last_peered":"2024-09-10T21:12:26.429005+0000","last_clean":"2024-09-10T21:12:26.429005+0000","last_became_active":"2024-09-10T21:12:24.805527+0000","last_became_peered":"2024-09-10T21:12:24.805527+0000","last_unstale":"2024-09-10T21:12:26.429005+0000","last_undegraded":"2024-09-10T21:12:26.429005+0000","last_fullsized":"2024-09-10T21:12:26.429005+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:17.948171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786931+0000","last_change":"2024-09-10T21:12:23.800595+0000","last_active":"2024-09-10T21:12:25.786931+0000","last_peered":"2024-09-10T21:12:25.786931+0000","last_clean":"2024-09-10T21:12:25.786931+0000","last_became_active":"2024-09-10T21:12:23.800279+0000","last_became_peered":"2024-09-10T21:12:23.800279+0000","last_unstale":"2024-09-10T21:12:25.786931+0000","last_undegraded":"2024-09-10T21:12:25.786931+0000","last_fullsized":"2024-09-10T21:12:25.786931+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:25.846794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428151+0000","last_change":"2024-09-10T21:12:24.805436+0000","last_active":"2024-09-10T21:12:26.428151+0000","last_peered":"2024-09-10T21:12:26.428151+0000","last_clean":"2024-09-10T21:12:26.428151+0000","last_became_active":"2024-09-10T21:12:24.805106+0000","last_became_peered":"2024-09-10T21:12:24.805106+0000","last_unstale":"2024-09-10T21:12:26.428151+0000","last_undegraded":"2024-09-10T21:12:26.428151+0000","last_fullsized":"2024-09-10T21:12:26.428151+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:32.446130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428690+0000","last_change":"2024-09-10T21:12:23.810567+0000","last_active":"2024-09-10T21:12:26.428690+0000","last_peered":"2024-09-10T21:12:26.428690+0000","last_clean":"2024-09-10T21:12:26.428690+0000","last_became_active":"2024-09-10T21:12:23.810246+0000","last_became_peered":"2024-09-10T21:12:23.810246+0000","last_unstale":"2024-09-10T21:12:26.428690+0000","last_undegraded":"2024-09-10T21:12:26.428690+0000","last_fullsized":"2024-09-10T21:12:26.428690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:09.107768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428041+0000","last_change":"2024-09-10T21:12:24.808843+0000","last_active":"2024-09-10T21:12:26.428041+0000","last_peered":"2024-09-10T21:12:26.428041+0000","last_clean":"2024-09-10T21:12:26.428041+0000","last_became_active":"2024-09-10T21:12:24.808570+0000","last_became_peered":"2024-09-10T21:12:24.808570+0000","last_unstale":"2024-09-10T21:12:26.428041+0000","last_undegraded":"2024-09-10T21:12:26.428041+0000","last_fullsized":"2024-09-10T21:12:26.428041+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:50.496957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427295+0000","last_change":"2024-09-10T21:12:23.807419+0000","last_active":"2024-09-10T21:12:26.427295+0000","last_peered":"2024-09-10T21:12:26.427295+0000","last_clean":"2024-09-10T21:12:26.427295+0000","last_became_active":"2024-09-10T21:12:23.806848+0000","last_became_peered":"2024-09-10T21:12:23.806848+0000","last_unstale":"2024-09-10T21:12:26.427295+0000","last_undegraded":"2024-09-10T21:12:26.427295+0000","last_fullsized":"2024-09-10T21:12:26.427295+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:46.663095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428169+0000","last_change":"2024-09-10T21:12:24.818146+0000","last_active":"2024-09-10T21:12:26.428169+0000","last_peered":"2024-09-10T21:12:26.428169+0000","last_clean":"2024-09-10T21:12:26.428169+0000","last_became_active":"2024-09-10T21:12:24.817782+0000","last_became_peered":"2024-09-10T21:12:24.817782+0000","last_unstale":"2024-09-10T21:12:26.428169+0000","last_undegraded":"2024-09-10T21:12:26.428169+0000","last_fullsized":"2024-09-10T21:12:26.428169+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:41.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428688+0000","last_change":"2024-09-10T21:12:23.795791+0000","last_active":"2024-09-10T21:12:26.428688+0000","last_peered":"2024-09-10T21:12:26.428688+0000","last_clean":"2024-09-10T21:12:26.428688+0000","last_became_active":"2024-09-10T21:12:23.795074+0000","last_became_peered":"2024-09-10T21:12:23.795074+0000","last_unstale":"2024-09-10T21:12:26.428688+0000","last_undegraded":"2024-09-10T21:12:26.428688+0000","last_fullsized":"2024-09-10T21:12:26.428688+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:36.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785827+0000","last_change":"2024-09-10T21:12:24.823887+0000","last_active":"2024-09-10T21:12:25.785827+0000","last_peered":"2024-09-10T21:12:25.785827+0000","last_clean":"2024-09-10T21:12:25.785827+0000","last_became_active":"2024-09-10T21:12:24.823427+0000","last_became_peered":"2024-09-10T21:12:24.823427+0000","last_unstale":"2024-09-10T21:12:25.785827+0000","last_undegraded":"2024-09-10T21:12:25.785827+0000","last_fullsized":"2024-09-10T21:12:25.785827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:04.280622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429384+0000","last_change":"2024-09-10T21:12:23.797756+0000","last_active":"2024-09-10T21:12:26.429384+0000","last_peered":"2024-09-10T21:12:26.429384+0000","last_clean":"2024-09-10T21:12:26.429384+0000","last_became_active":"2024-09-10T21:12:23.797410+0000","last_became_peered":"2024-09-10T21:12:23.797410+0000","last_unstale":"2024-09-10T21:12:26.429384+0000","last_undegraded":"2024-09-10T21:12:26.429384+0000","last_fullsized":"2024-09-10T21:12:26.429384+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:58.733833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427546+0000","last_change":"2024-09-10T21:12:24.816020+0000","last_active":"2024-09-10T21:12:26.427546+0000","last_peered":"2024-09-10T21:12:26.427546+0000","last_clean":"2024-09-10T21:12:26.427546+0000","last_became_active":"2024-09-10T21:12:24.815116+0000","last_became_peered":"2024-09-10T21:12:24.815116+0000","last_unstale":"2024-09-10T21:12:26.427546+0000","last_undegraded":"2024-09-10T21:12:26.427546+0000","last_fullsized":"2024-09-10T21:12:26.427546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:19.498294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785094+0000","last_change":"2024-09-10T21:12:23.800295+0000","last_active":"2024-09-10T21:12:25.785094+0000","last_peered":"2024-09-10T21:12:25.785094+0000","last_clean":"2024-09-10T21:12:25.785094+0000","last_became_active":"2024-09-10T21:12:23.799896+0000","last_became_peered":"2024-09-10T21:12:23.799896+0000","last_unstale":"2024-09-10T21:12:25.785094+0000","last_undegraded":"2024-09-10T21:12:25.785094+0000","last_fullsized":"2024-09-10T21:12:25.785094+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:25.846794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784955+0000","last_change":"2024-09-10T21:12:24.798401+0000","last_active":"2024-09-10T21:12:25.784955+0000","last_peered":"2024-09-10T21:12:25.784955+0000","last_clean":"2024-09-10T21:12:25.784955+0000","last_became_active":"2024-09-10T21:12:24.798177+0000","last_became_peered":"2024-09-10T21:12:24.798177+0000","last_unstale":"2024-09-10T21:12:25.784955+0000","last_undegraded":"2024-09-10T21:12:25.784955+0000","last_fullsized":"2024-09-10T21:12:25.784955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:21.261528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427060+0000","last_change":"2024-09-10T21:12:23.799895+0000","last_active":"2024-09-10T21:12:26.427060+0000","last_peered":"2024-09-10T21:12:26.427060+0000","last_clean":"2024-09-10T21:12:26.427060+0000","last_became_active":"2024-09-10T21:12:23.799596+0000","last_became_peered":"2024-09-10T21:12:23.799596+0000","last_unstale":"2024-09-10T21:12:26.427060+0000","last_undegraded":"2024-09-10T21:12:26.427060+0000","last_fullsized":"2024-09-10T21:12:26.427060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:21.476191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786669+0000","last_change":"2024-09-10T21:12:24.823727+0000","last_active":"2024-09-10T21:12:25.786669+0000","last_peered":"2024-09-10T21:12:25.786669+0000","last_clean":"2024-09-10T21:12:25.786669+0000","last_became_active":"2024-09-10T21:12:24.823434+0000","last_became_peered":"2024-09-10T21:12:24.823434+0000","last_unstale":"2024-09-10T21:12:25.786669+0000","last_undegraded":"2024-09-10T21:12:25.786669+0000","last_fullsized":"2024-09-10T21:12:25.786669+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:09.337686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429146+0000","last_change":"2024-09-10T21:12:23.803236+0000","last_active":"2024-09-10T21:12:26.429146+0000","last_peered":"2024-09-10T21:12:26.429146+0000","last_clean":"2024-09-10T21:12:26.429146+0000","last_became_active":"2024-09-10T21:12:23.802939+0000","last_became_peered":"2024-09-10T21:12:23.802939+0000","last_unstale":"2024-09-10T21:12:26.429146+0000","last_undegraded":"2024-09-10T21:12:26.429146+0000","last_fullsized":"2024-09-10T21:12:26.429146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:35.914109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427619+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:12:26.427619+0000","last_peered":"2024-09-10T21:12:26.427619+0000","last_clean":"2024-09-10T21:12:26.427619+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:12:26.427619+0000","last_undegraded":"2024-09-10T21:12:26.427619+0000","last_fullsized":"2024-09-10T21:12:26.427619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1130496,"data_stored":806790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509469,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":218104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":544768,"data_stored":218012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":216490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":216434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":806838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:12:29.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T21:12:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.956+0000 7f6f50400640 1 -- 172.21.15.16:0/3746175131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f4815f290 msgr2=0x7f6f480cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.956+0000 7f6f50400640 1 --2- 172.21.15.16:0/3746175131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f4815f290 0x7f6f480cd120 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009930 tx=0x7f6f3c01c7c0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.956+0000 7f6f50400640 1 -- 172.21.15.16:0/3746175131 shutdown_connections 2024-09-10T21:12:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.956+0000 7f6f50400640 1 --2- 172.21.15.16:0/3746175131 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f480cb990 0x7f6f480737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.956+0000 7f6f50400640 1 --2- 172.21.15.16:0/3746175131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f4815f290 0x7f6f480cd120 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.956+0000 7f6f50400640 1 -- 172.21.15.16:0/3746175131 >> 172.21.15.16:0/3746175131 conn(0x7f6f480d0c30 msgr2=0x7f6f480d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.956+0000 7f6f50400640 1 -- 172.21.15.16:0/3746175131 shutdown_connections 2024-09-10T21:12:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.956+0000 7f6f50400640 1 -- 172.21.15.16:0/3746175131 wait complete. 2024-09-10T21:12:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.956+0000 7f6f50400640 1 Processor -- start 2024-09-10T21:12:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.956+0000 7f6f50400640 1 -- start start 2024-09-10T21:12:29.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f50400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f480c41d0 0x7f6f481561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f50400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f480cb990 0x7f6f4815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f50400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4815f290 0x7f6f480c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f50400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f48169190 con 0x7f6f480cb990 2024-09-10T21:12:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f50400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6f48169010 con 0x7f6f4815f290 2024-09-10T21:12:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f50400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6f48169310 con 0x7f6f480c41d0 2024-09-10T21:12:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f4d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f480cb990 0x7f6f4815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f4d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f480cb990 0x7f6f4815ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57196/0 (socket says 172.21.15.16:57196) 2024-09-10T21:12:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f4d000640 1 -- 172.21.15.16:0/1650535867 learned_addr learned my addr 172.21.15.16:0/1650535867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f4e400640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4815f290 0x7f6f480c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f4d000640 1 -- 172.21.15.16:0/1650535867 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f480c41d0 msgr2=0x7f6f481561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f4da00640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f480c41d0 0x7f6f481561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f4d000640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f480c41d0 0x7f6f481561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f4d000640 1 -- 172.21.15.16:0/1650535867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4815f290 msgr2=0x7f6f480c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f4d000640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4815f290 0x7f6f480c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f4d000640 1 -- 172.21.15.16:0/1650535867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f480c6770 con 0x7f6f480cb990 2024-09-10T21:12:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f4da00640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f480c41d0 0x7f6f481561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f4e400640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4815f290 0x7f6f480c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f4d000640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f480cb990 0x7f6f4815ddb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400c450 tx=0x7f6f3400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f45600640 1 -- 172.21.15.16:0/1650535867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34012830 con 0x7f6f480cb990 2024-09-10T21:12:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f45600640 1 -- 172.21.15.16:0/1650535867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f34017070 con 0x7f6f480cb990 2024-09-10T21:12:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f45600640 1 -- 172.21.15.16:0/1650535867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34012d80 con 0x7f6f480cb990 2024-09-10T21:12:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f50400640 1 -- 172.21.15.16:0/1650535867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f480c56d0 con 0x7f6f480cb990 2024-09-10T21:12:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f50400640 1 -- 172.21.15.16:0/1650535867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f480c5c90 con 0x7f6f480cb990 2024-09-10T21:12:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.960+0000 7f6f50400640 1 -- 172.21.15.16:0/1650535867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f480737c0 con 0x7f6f480cb990 2024-09-10T21:12:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.964+0000 7f6f45600640 1 -- 172.21.15.16:0/1650535867 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6f3401c460 con 0x7f6f480cb990 2024-09-10T21:12:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.964+0000 7f6f45600640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f280725e0 0x7f6f28074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.964+0000 7f6f45600640 1 -- 172.21.15.16:0/1650535867 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f6f34093080 con 0x7f6f480cb990 2024-09-10T21:12:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.964+0000 7f6f4da00640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f280725e0 0x7f6f28074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.964+0000 7f6f4da00640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f280725e0 0x7f6f28074aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0af000 tx=0x7f6f3c01b000 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:29.968+0000 7f6f45600640 1 -- 172.21.15.16:0/1650535867 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6f3405e6c0 con 0x7f6f480cb990 2024-09-10T21:12:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.204+0000 7f6f50400640 1 -- 172.21.15.16:0/1650535867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f6f4815ea50 con 0x7f6f480cb990 2024-09-10T21:12:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.204+0000 7f6f45600640 1 -- 172.21.15.16:0/1650535867 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1057 (secure 0 0 0) 0x7f6f3405dfc0 con 0x7f6f480cb990 2024-09-10T21:12:30.211 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:30.211 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":145,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":8,"num_up_osds":8,"osd_up_since":1726002618,"num_in_osds":8,"osd_in_since":1726002608,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":261013504,"bytes_avail":772833099776,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":24,"btime":"2024-09-10T21:12:27:803754+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"d","status":"up:active","gid":134771}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-10T21:12:13.480543+0000","services":{}},"progress_events":{}} 2024-09-10T21:12:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.204+0000 7f6f50400640 1 -- 172.21.15.16:0/1650535867 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f280725e0 msgr2=0x7f6f28074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.204+0000 7f6f50400640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f280725e0 0x7f6f28074aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0af000 tx=0x7f6f3c01b000 comp rx=0 tx=0).stop 2024-09-10T21:12:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.204+0000 7f6f50400640 1 -- 172.21.15.16:0/1650535867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f480cb990 msgr2=0x7f6f4815ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.204+0000 7f6f50400640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f480cb990 0x7f6f4815ddb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400c450 tx=0x7f6f3400c910 comp rx=0 tx=0).stop 2024-09-10T21:12:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.204+0000 7f6f50400640 1 -- 172.21.15.16:0/1650535867 shutdown_connections 2024-09-10T21:12:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.204+0000 7f6f50400640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f280725e0 0x7f6f28074aa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.204+0000 7f6f50400640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4815f290 0x7f6f480c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.204+0000 7f6f50400640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f480cb990 0x7f6f4815ddb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.204+0000 7f6f50400640 1 --2- 172.21.15.16:0/1650535867 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f480c41d0 0x7f6f481561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.204+0000 7f6f50400640 1 -- 172.21.15.16:0/1650535867 >> 172.21.15.16:0/1650535867 conn(0x7f6f480d0c30 msgr2=0x7f6f48071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.208+0000 7f6f50400640 1 -- 172.21.15.16:0/1650535867 shutdown_connections 2024-09-10T21:12:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.208+0000 7f6f50400640 1 -- 172.21.15.16:0/1650535867 wait complete. 2024-09-10T21:12:30.233 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T21:12:30.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:12:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.372+0000 7fb876200640 1 -- 172.21.15.16:0/2897124378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8700b3200 msgr2=0x7fb8700afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.372+0000 7fb876200640 1 --2- 172.21.15.16:0/2897124378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8700b3200 0x7fb8700afae0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb858009840 tx=0x7fb85801c820 comp rx=0 tx=0).stop 2024-09-10T21:12:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.372+0000 7fb876200640 1 -- 172.21.15.16:0/2897124378 shutdown_connections 2024-09-10T21:12:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.372+0000 7fb876200640 1 --2- 172.21.15.16:0/2897124378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8700b3200 0x7fb8700afae0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.372+0000 7fb876200640 1 -- 172.21.15.16:0/2897124378 >> 172.21.15.16:0/2897124378 conn(0x7fb8700d1d80 msgr2=0x7fb8700d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb876200640 1 -- 172.21.15.16:0/2897124378 shutdown_connections 2024-09-10T21:12:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb876200640 1 -- 172.21.15.16:0/2897124378 wait complete. 2024-09-10T21:12:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb876200640 1 Processor -- start 2024-09-10T21:12:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb876200640 1 -- start start 2024-09-10T21:12:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb876200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8700d0890 0x7fb8700c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb876200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87015f450 0x7fb87014adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb876200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb87014cad0 0x7fb87014b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb876200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8700b0730 con 0x7fb87015f450 2024-09-10T21:12:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb876200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb8700b08b0 con 0x7fb8700d0890 2024-09-10T21:12:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb876200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb8700b0a30 con 0x7fb87014cad0 2024-09-10T21:12:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb866a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87015f450 0x7fb87014adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb866a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87015f450 0x7fb87014adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57220/0 (socket says 172.21.15.16:57220) 2024-09-10T21:12:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb866a00640 1 -- 172.21.15.16:0/1072240150 learned_addr learned my addr 172.21.15.16:0/1072240150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb86f400640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8700d0890 0x7fb8700c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb86fe00640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb87014cad0 0x7fb87014b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb866a00640 1 -- 172.21.15.16:0/1072240150 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb87014cad0 msgr2=0x7fb87014b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb866a00640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb87014cad0 0x7fb87014b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb866a00640 1 -- 172.21.15.16:0/1072240150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8700d0890 msgr2=0x7fb8700c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb866a00640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8700d0890 0x7fb8700c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb866a00640 1 -- 172.21.15.16:0/1072240150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb870152900 con 0x7fb87015f450 2024-09-10T21:12:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb86fe00640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb87014cad0 0x7fb87014b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb866a00640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87015f450 0x7fb87014adc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb8580b0040 tx=0x7fb858004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb867e00640 1 -- 172.21.15.16:0/1072240150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858004440 con 0x7fb87015f450 2024-09-10T21:12:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb867e00640 1 -- 172.21.15.16:0/1072240150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8580036f0 con 0x7fb87015f450 2024-09-10T21:12:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb867e00640 1 -- 172.21.15.16:0/1072240150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858003b20 con 0x7fb87015f450 2024-09-10T21:12:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.376+0000 7fb876200640 1 -- 172.21.15.16:0/1072240150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb870152be0 con 0x7fb87015f450 2024-09-10T21:12:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.380+0000 7fb876200640 1 -- 172.21.15.16:0/1072240150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8701532a0 con 0x7fb87015f450 2024-09-10T21:12:30.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.380+0000 7fb876200640 1 -- 172.21.15.16:0/1072240150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8700cfc80 con 0x7fb87015f450 2024-09-10T21:12:30.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.380+0000 7fb867e00640 1 -- 172.21.15.16:0/1072240150 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb858003cc0 con 0x7fb87015f450 2024-09-10T21:12:30.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.380+0000 7fb867e00640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb84c0725e0 0x7fb84c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.380+0000 7fb867e00640 1 -- 172.21.15.16:0/1072240150 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fb85812e360 con 0x7fb87015f450 2024-09-10T21:12:30.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.380+0000 7fb86f400640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb84c0725e0 0x7fb84c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.384+0000 7fb86f400640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb84c0725e0 0x7fb84c074aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb860007fd0 tx=0x7fb860007ec0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:30.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.384+0000 7fb867e00640 1 -- 172.21.15.16:0/1072240150 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb8580c9020 con 0x7fb87015f450 2024-09-10T21:12:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.560+0000 7fb876200640 1 -- 172.21.15.16:0/1072240150 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb870050fd0 con 0x7fb84c0725e0 2024-09-10T21:12:30.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.564+0000 7fb867e00640 1 -- 172.21.15.16:0/1072240150 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7fb870050fd0 con 0x7fb84c0725e0 2024-09-10T21:12:30.572 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:12:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.568+0000 7fb876200640 1 -- 172.21.15.16:0/1072240150 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb84c0725e0 msgr2=0x7fb84c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.568+0000 7fb876200640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb84c0725e0 0x7fb84c074aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb860007fd0 tx=0x7fb860007ec0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.568+0000 7fb876200640 1 -- 172.21.15.16:0/1072240150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87015f450 msgr2=0x7fb87014adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.568+0000 7fb876200640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87015f450 0x7fb87014adc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb8580b0040 tx=0x7fb858004040 comp rx=0 tx=0).stop 2024-09-10T21:12:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.568+0000 7fb876200640 1 -- 172.21.15.16:0/1072240150 shutdown_connections 2024-09-10T21:12:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.568+0000 7fb876200640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb84c0725e0 0x7fb84c074aa0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.568+0000 7fb876200640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb87014cad0 0x7fb87014b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.568+0000 7fb876200640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb87015f450 0x7fb87014adc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.568+0000 7fb876200640 1 --2- 172.21.15.16:0/1072240150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8700d0890 0x7fb8700c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.568+0000 7fb876200640 1 -- 172.21.15.16:0/1072240150 >> 172.21.15.16:0/1072240150 conn(0x7fb8700d1d80 msgr2=0x7fb8700afa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:30.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.568+0000 7fb876200640 1 -- 172.21.15.16:0/1072240150 shutdown_connections 2024-09-10T21:12:30.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:30.568+0000 7fb876200640 1 -- 172.21.15.16:0/1072240150 wait complete. 2024-09-10T21:12:30.593 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-10T21:12:29.483873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254896,"kb_used_data":5384,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719824,"statfs":{"total":773094113280,"available":772833099776,"internally_reserved":0,"allocated":5513216,"data_stored":2916756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001312"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428163+0000","last_change":"2024-09-10T21:12:24.807699+0000","last_active":"2024-09-10T21:12:26.428163+0000","last_peered":"2024-09-10T21:12:26.428163+0000","last_clean":"2024-09-10T21:12:26.428163+0000","last_became_active":"2024-09-10T21:12:24.807378+0000","last_became_peered":"2024-09-10T21:12:24.807378+0000","last_unstale":"2024-09-10T21:12:26.428163+0000","last_undegraded":"2024-09-10T21:12:26.428163+0000","last_fullsized":"2024-09-10T21:12:26.428163+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:06.104316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785215+0000","last_change":"2024-09-10T21:12:24.801034+0000","last_active":"2024-09-10T21:12:25.785215+0000","last_peered":"2024-09-10T21:12:25.785215+0000","last_clean":"2024-09-10T21:12:25.785215+0000","last_became_active":"2024-09-10T21:12:24.799842+0000","last_became_peered":"2024-09-10T21:12:24.799842+0000","last_unstale":"2024-09-10T21:12:25.785215+0000","last_undegraded":"2024-09-10T21:12:25.785215+0000","last_fullsized":"2024-09-10T21:12:25.785215+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:51.766507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429089+0000","last_change":"2024-09-10T21:12:24.811161+0000","last_active":"2024-09-10T21:12:26.429089+0000","last_peered":"2024-09-10T21:12:26.429089+0000","last_clean":"2024-09-10T21:12:26.429089+0000","last_became_active":"2024-09-10T21:12:24.810838+0000","last_became_peered":"2024-09-10T21:12:24.810838+0000","last_unstale":"2024-09-10T21:12:26.429089+0000","last_undegraded":"2024-09-10T21:12:26.429089+0000","last_fullsized":"2024-09-10T21:12:26.429089+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:14.399311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785704+0000","last_change":"2024-09-10T21:12:24.807996+0000","last_active":"2024-09-10T21:12:25.785704+0000","last_peered":"2024-09-10T21:12:25.785704+0000","last_clean":"2024-09-10T21:12:25.785704+0000","last_became_active":"2024-09-10T21:12:24.807614+0000","last_became_peered":"2024-09-10T21:12:24.807614+0000","last_unstale":"2024-09-10T21:12:25.785704+0000","last_undegraded":"2024-09-10T21:12:25.785704+0000","last_fullsized":"2024-09-10T21:12:25.785704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:58.826777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428361+0000","last_change":"2024-09-10T21:12:24.814793+0000","last_active":"2024-09-10T21:12:26.428361+0000","last_peered":"2024-09-10T21:12:26.428361+0000","last_clean":"2024-09-10T21:12:26.428361+0000","last_became_active":"2024-09-10T21:12:24.814494+0000","last_became_peered":"2024-09-10T21:12:24.814494+0000","last_unstale":"2024-09-10T21:12:26.428361+0000","last_undegraded":"2024-09-10T21:12:26.428361+0000","last_fullsized":"2024-09-10T21:12:26.428361+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:21.261528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785736+0000","last_change":"2024-09-10T21:12:24.827330+0000","last_active":"2024-09-10T21:12:25.785736+0000","last_peered":"2024-09-10T21:12:25.785736+0000","last_clean":"2024-09-10T21:12:25.785736+0000","last_became_active":"2024-09-10T21:12:24.827158+0000","last_became_peered":"2024-09-10T21:12:24.827158+0000","last_unstale":"2024-09-10T21:12:25.785736+0000","last_undegraded":"2024-09-10T21:12:25.785736+0000","last_fullsized":"2024-09-10T21:12:25.785736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:13.628817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427734+0000","last_change":"2024-09-10T21:12:24.808616+0000","last_active":"2024-09-10T21:12:26.427734+0000","last_peered":"2024-09-10T21:12:26.427734+0000","last_clean":"2024-09-10T21:12:26.427734+0000","last_became_active":"2024-09-10T21:12:24.808341+0000","last_became_peered":"2024-09-10T21:12:24.808341+0000","last_unstale":"2024-09-10T21:12:26.427734+0000","last_undegraded":"2024-09-10T21:12:26.427734+0000","last_fullsized":"2024-09-10T21:12:26.427734+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:23.140730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785688+0000","last_change":"2024-09-10T21:12:24.823724+0000","last_active":"2024-09-10T21:12:25.785688+0000","last_peered":"2024-09-10T21:12:25.785688+0000","last_clean":"2024-09-10T21:12:25.785688+0000","last_became_active":"2024-09-10T21:12:24.823452+0000","last_became_peered":"2024-09-10T21:12:24.823452+0000","last_unstale":"2024-09-10T21:12:25.785688+0000","last_undegraded":"2024-09-10T21:12:25.785688+0000","last_fullsized":"2024-09-10T21:12:25.785688+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:22.748738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.787140+0000","last_change":"2024-09-10T21:12:24.822052+0000","last_active":"2024-09-10T21:12:25.787140+0000","last_peered":"2024-09-10T21:12:25.787140+0000","last_clean":"2024-09-10T21:12:25.787140+0000","last_became_active":"2024-09-10T21:12:24.821688+0000","last_became_peered":"2024-09-10T21:12:24.821688+0000","last_unstale":"2024-09-10T21:12:25.787140+0000","last_undegraded":"2024-09-10T21:12:25.787140+0000","last_fullsized":"2024-09-10T21:12:25.787140+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:52.341058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428221+0000","last_change":"2024-09-10T21:12:24.813099+0000","last_active":"2024-09-10T21:12:26.428221+0000","last_peered":"2024-09-10T21:12:26.428221+0000","last_clean":"2024-09-10T21:12:26.428221+0000","last_became_active":"2024-09-10T21:12:24.812570+0000","last_became_peered":"2024-09-10T21:12:24.812570+0000","last_unstale":"2024-09-10T21:12:26.428221+0000","last_undegraded":"2024-09-10T21:12:26.428221+0000","last_fullsized":"2024-09-10T21:12:26.428221+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:25.911130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786429+0000","last_change":"2024-09-10T21:12:24.830636+0000","last_active":"2024-09-10T21:12:25.786429+0000","last_peered":"2024-09-10T21:12:25.786429+0000","last_clean":"2024-09-10T21:12:25.786429+0000","last_became_active":"2024-09-10T21:12:24.830369+0000","last_became_peered":"2024-09-10T21:12:24.830369+0000","last_unstale":"2024-09-10T21:12:25.786429+0000","last_undegraded":"2024-09-10T21:12:25.786429+0000","last_fullsized":"2024-09-10T21:12:25.786429+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:39.747387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784766+0000","last_change":"2024-09-10T21:12:24.788829+0000","last_active":"2024-09-10T21:12:25.784766+0000","last_peered":"2024-09-10T21:12:25.784766+0000","last_clean":"2024-09-10T21:12:25.784766+0000","last_became_active":"2024-09-10T21:12:24.788547+0000","last_became_peered":"2024-09-10T21:12:24.788547+0000","last_unstale":"2024-09-10T21:12:25.784766+0000","last_undegraded":"2024-09-10T21:12:25.784766+0000","last_fullsized":"2024-09-10T21:12:25.784766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:15.277562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785343+0000","last_change":"2024-09-10T21:12:24.826783+0000","last_active":"2024-09-10T21:12:25.785343+0000","last_peered":"2024-09-10T21:12:25.785343+0000","last_clean":"2024-09-10T21:12:25.785343+0000","last_became_active":"2024-09-10T21:12:24.826470+0000","last_became_peered":"2024-09-10T21:12:24.826470+0000","last_unstale":"2024-09-10T21:12:25.785343+0000","last_undegraded":"2024-09-10T21:12:25.785343+0000","last_fullsized":"2024-09-10T21:12:25.785343+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:37.303480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785877+0000","last_change":"2024-09-10T21:12:24.820100+0000","last_active":"2024-09-10T21:12:25.785877+0000","last_peered":"2024-09-10T21:12:25.785877+0000","last_clean":"2024-09-10T21:12:25.785877+0000","last_became_active":"2024-09-10T21:12:24.819477+0000","last_became_peered":"2024-09-10T21:12:24.819477+0000","last_unstale":"2024-09-10T21:12:25.785877+0000","last_undegraded":"2024-09-10T21:12:25.785877+0000","last_fullsized":"2024-09-10T21:12:25.785877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:22.872734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786381+0000","last_change":"2024-09-10T21:12:24.807623+0000","last_active":"2024-09-10T21:12:25.786381+0000","last_peered":"2024-09-10T21:12:25.786381+0000","last_clean":"2024-09-10T21:12:25.786381+0000","last_became_active":"2024-09-10T21:12:24.807290+0000","last_became_peered":"2024-09-10T21:12:24.807290+0000","last_unstale":"2024-09-10T21:12:25.786381+0000","last_undegraded":"2024-09-10T21:12:25.786381+0000","last_fullsized":"2024-09-10T21:12:25.786381+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:32.446130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428017+0000","last_change":"2024-09-10T21:12:24.822563+0000","last_active":"2024-09-10T21:12:26.428017+0000","last_peered":"2024-09-10T21:12:26.428017+0000","last_clean":"2024-09-10T21:12:26.428017+0000","last_became_active":"2024-09-10T21:12:24.822299+0000","last_became_peered":"2024-09-10T21:12:24.822299+0000","last_unstale":"2024-09-10T21:12:26.428017+0000","last_undegraded":"2024-09-10T21:12:26.428017+0000","last_fullsized":"2024-09-10T21:12:26.428017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:15.277562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784799+0000","last_change":"2024-09-10T21:12:24.804138+0000","last_active":"2024-09-10T21:12:25.784799+0000","last_peered":"2024-09-10T21:12:25.784799+0000","last_clean":"2024-09-10T21:12:25.784799+0000","last_became_active":"2024-09-10T21:12:24.803868+0000","last_became_peered":"2024-09-10T21:12:24.803868+0000","last_unstale":"2024-09-10T21:12:25.784799+0000","last_undegraded":"2024-09-10T21:12:25.784799+0000","last_fullsized":"2024-09-10T21:12:25.784799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:06.104316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429476+0000","last_change":"2024-09-10T21:12:24.806841+0000","last_active":"2024-09-10T21:12:26.429476+0000","last_peered":"2024-09-10T21:12:26.429476+0000","last_clean":"2024-09-10T21:12:26.429476+0000","last_became_active":"2024-09-10T21:12:24.804956+0000","last_became_peered":"2024-09-10T21:12:24.804956+0000","last_unstale":"2024-09-10T21:12:26.429476+0000","last_undegraded":"2024-09-10T21:12:26.429476+0000","last_fullsized":"2024-09-10T21:12:26.429476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:46.274444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428515+0000","last_change":"2024-09-10T21:12:24.817361+0000","last_active":"2024-09-10T21:12:26.428515+0000","last_peered":"2024-09-10T21:12:26.428515+0000","last_clean":"2024-09-10T21:12:26.428515+0000","last_became_active":"2024-09-10T21:12:24.815622+0000","last_became_peered":"2024-09-10T21:12:24.815622+0000","last_unstale":"2024-09-10T21:12:26.428515+0000","last_undegraded":"2024-09-10T21:12:26.428515+0000","last_fullsized":"2024-09-10T21:12:26.428515+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:11.907167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428188+0000","last_change":"2024-09-10T21:12:24.792792+0000","last_active":"2024-09-10T21:12:26.428188+0000","last_peered":"2024-09-10T21:12:26.428188+0000","last_clean":"2024-09-10T21:12:26.428188+0000","last_became_active":"2024-09-10T21:12:24.792220+0000","last_became_peered":"2024-09-10T21:12:24.792220+0000","last_unstale":"2024-09-10T21:12:26.428188+0000","last_undegraded":"2024-09-10T21:12:26.428188+0000","last_fullsized":"2024-09-10T21:12:26.428188+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:39.747387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786232+0000","last_change":"2024-09-10T21:12:24.824108+0000","last_active":"2024-09-10T21:12:25.786232+0000","last_peered":"2024-09-10T21:12:25.786232+0000","last_clean":"2024-09-10T21:12:25.786232+0000","last_became_active":"2024-09-10T21:12:24.823125+0000","last_became_peered":"2024-09-10T21:12:24.823125+0000","last_unstale":"2024-09-10T21:12:25.786232+0000","last_undegraded":"2024-09-10T21:12:25.786232+0000","last_fullsized":"2024-09-10T21:12:25.786232+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:41.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427209+0000","last_change":"2024-09-10T21:12:24.803379+0000","last_active":"2024-09-10T21:12:26.427209+0000","last_peered":"2024-09-10T21:12:26.427209+0000","last_clean":"2024-09-10T21:12:26.427209+0000","last_became_active":"2024-09-10T21:12:24.802088+0000","last_became_peered":"2024-09-10T21:12:24.802088+0000","last_unstale":"2024-09-10T21:12:26.427209+0000","last_undegraded":"2024-09-10T21:12:26.427209+0000","last_fullsized":"2024-09-10T21:12:26.427209+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:45.236002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428095+0000","last_change":"2024-09-10T21:12:24.800438+0000","last_active":"2024-09-10T21:12:26.428095+0000","last_peered":"2024-09-10T21:12:26.428095+0000","last_clean":"2024-09-10T21:12:26.428095+0000","last_became_active":"2024-09-10T21:12:24.799628+0000","last_became_peered":"2024-09-10T21:12:24.799628+0000","last_unstale":"2024-09-10T21:12:26.428095+0000","last_undegraded":"2024-09-10T21:12:26.428095+0000","last_fullsized":"2024-09-10T21:12:26.428095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:22.261573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784410+0000","last_change":"2024-09-10T21:12:24.822685+0000","last_active":"2024-09-10T21:12:25.784410+0000","last_peered":"2024-09-10T21:12:25.784410+0000","last_clean":"2024-09-10T21:12:25.784410+0000","last_became_active":"2024-09-10T21:12:24.821398+0000","last_became_peered":"2024-09-10T21:12:24.821398+0000","last_unstale":"2024-09-10T21:12:25.784410+0000","last_undegraded":"2024-09-10T21:12:25.784410+0000","last_fullsized":"2024-09-10T21:12:25.784410+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:02.665366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428387+0000","last_change":"2024-09-10T21:12:24.807404+0000","last_active":"2024-09-10T21:12:26.428387+0000","last_peered":"2024-09-10T21:12:26.428387+0000","last_clean":"2024-09-10T21:12:26.428387+0000","last_became_active":"2024-09-10T21:12:24.807144+0000","last_became_peered":"2024-09-10T21:12:24.807144+0000","last_unstale":"2024-09-10T21:12:26.428387+0000","last_undegraded":"2024-09-10T21:12:26.428387+0000","last_fullsized":"2024-09-10T21:12:26.428387+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:37.987597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786744+0000","last_change":"2024-09-10T21:12:24.829538+0000","last_active":"2024-09-10T21:12:25.786744+0000","last_peered":"2024-09-10T21:12:25.786744+0000","last_clean":"2024-09-10T21:12:25.786744+0000","last_became_active":"2024-09-10T21:12:24.829250+0000","last_became_peered":"2024-09-10T21:12:24.829250+0000","last_unstale":"2024-09-10T21:12:25.786744+0000","last_undegraded":"2024-09-10T21:12:25.786744+0000","last_fullsized":"2024-09-10T21:12:25.786744+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:47.207849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427811+0000","last_change":"2024-09-10T21:12:24.795084+0000","last_active":"2024-09-10T21:12:26.427811+0000","last_peered":"2024-09-10T21:12:26.427811+0000","last_clean":"2024-09-10T21:12:26.427811+0000","last_became_active":"2024-09-10T21:12:24.794823+0000","last_became_peered":"2024-09-10T21:12:24.794823+0000","last_unstale":"2024-09-10T21:12:26.427811+0000","last_undegraded":"2024-09-10T21:12:26.427811+0000","last_fullsized":"2024-09-10T21:12:26.427811+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:26.691816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428069+0000","last_change":"2024-09-10T21:12:24.793348+0000","last_active":"2024-09-10T21:12:26.428069+0000","last_peered":"2024-09-10T21:12:26.428069+0000","last_clean":"2024-09-10T21:12:26.428069+0000","last_became_active":"2024-09-10T21:12:24.793089+0000","last_became_peered":"2024-09-10T21:12:24.793089+0000","last_unstale":"2024-09-10T21:12:26.428069+0000","last_undegraded":"2024-09-10T21:12:26.428069+0000","last_fullsized":"2024-09-10T21:12:26.428069+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:03.310670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428591+0000","last_change":"2024-09-10T21:12:24.815439+0000","last_active":"2024-09-10T21:12:26.428591+0000","last_peered":"2024-09-10T21:12:26.428591+0000","last_clean":"2024-09-10T21:12:26.428591+0000","last_became_active":"2024-09-10T21:12:24.815191+0000","last_became_peered":"2024-09-10T21:12:24.815191+0000","last_unstale":"2024-09-10T21:12:26.428591+0000","last_undegraded":"2024-09-10T21:12:26.428591+0000","last_fullsized":"2024-09-10T21:12:26.428591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:14.014096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784439+0000","last_change":"2024-09-10T21:12:24.792836+0000","last_active":"2024-09-10T21:12:25.784439+0000","last_peered":"2024-09-10T21:12:25.784439+0000","last_clean":"2024-09-10T21:12:25.784439+0000","last_became_active":"2024-09-10T21:12:24.792569+0000","last_became_peered":"2024-09-10T21:12:24.792569+0000","last_unstale":"2024-09-10T21:12:25.784439+0000","last_undegraded":"2024-09-10T21:12:25.784439+0000","last_fullsized":"2024-09-10T21:12:25.784439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:31.753172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785090+0000","last_change":"2024-09-10T21:12:24.808015+0000","last_active":"2024-09-10T21:12:25.785090+0000","last_peered":"2024-09-10T21:12:25.785090+0000","last_clean":"2024-09-10T21:12:25.785090+0000","last_became_active":"2024-09-10T21:12:24.807723+0000","last_became_peered":"2024-09-10T21:12:24.807723+0000","last_unstale":"2024-09-10T21:12:25.785090+0000","last_undegraded":"2024-09-10T21:12:25.785090+0000","last_fullsized":"2024-09-10T21:12:25.785090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:26.164431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785759+0000","last_change":"2024-09-10T21:12:24.812500+0000","last_active":"2024-09-10T21:12:25.785759+0000","last_peered":"2024-09-10T21:12:25.785759+0000","last_clean":"2024-09-10T21:12:25.785759+0000","last_became_active":"2024-09-10T21:12:24.812143+0000","last_became_peered":"2024-09-10T21:12:24.812143+0000","last_unstale":"2024-09-10T21:12:25.785759+0000","last_undegraded":"2024-09-10T21:12:25.785759+0000","last_fullsized":"2024-09-10T21:12:25.785759+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:33.667666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427841+0000","last_change":"2024-09-10T21:12:24.817360+0000","last_active":"2024-09-10T21:12:26.427841+0000","last_peered":"2024-09-10T21:12:26.427841+0000","last_clean":"2024-09-10T21:12:26.427841+0000","last_became_active":"2024-09-10T21:12:24.817059+0000","last_became_peered":"2024-09-10T21:12:24.817059+0000","last_unstale":"2024-09-10T21:12:26.427841+0000","last_undegraded":"2024-09-10T21:12:26.427841+0000","last_fullsized":"2024-09-10T21:12:26.427841+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:33.659623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.787046+0000","last_change":"2024-09-10T21:12:23.795968+0000","last_active":"2024-09-10T21:12:25.787046+0000","last_peered":"2024-09-10T21:12:25.787046+0000","last_clean":"2024-09-10T21:12:25.787046+0000","last_became_active":"2024-09-10T21:12:23.795674+0000","last_became_peered":"2024-09-10T21:12:23.795674+0000","last_unstale":"2024-09-10T21:12:25.787046+0000","last_undegraded":"2024-09-10T21:12:25.787046+0000","last_fullsized":"2024-09-10T21:12:25.787046+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:34.576509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428635+0000","last_change":"2024-09-10T21:12:24.801727+0000","last_active":"2024-09-10T21:12:26.428635+0000","last_peered":"2024-09-10T21:12:26.428635+0000","last_clean":"2024-09-10T21:12:26.428635+0000","last_became_active":"2024-09-10T21:12:24.801282+0000","last_became_peered":"2024-09-10T21:12:24.801282+0000","last_unstale":"2024-09-10T21:12:26.428635+0000","last_undegraded":"2024-09-10T21:12:26.428635+0000","last_fullsized":"2024-09-10T21:12:26.428635+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:52.776680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428530+0000","last_change":"2024-09-10T21:12:23.799107+0000","last_active":"2024-09-10T21:12:26.428530+0000","last_peered":"2024-09-10T21:12:26.428530+0000","last_clean":"2024-09-10T21:12:26.428530+0000","last_became_active":"2024-09-10T21:12:23.798815+0000","last_became_peered":"2024-09-10T21:12:23.798815+0000","last_unstale":"2024-09-10T21:12:26.428530+0000","last_undegraded":"2024-09-10T21:12:26.428530+0000","last_fullsized":"2024-09-10T21:12:26.428530+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:41.715763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785035+0000","last_change":"2024-09-10T21:12:24.795186+0000","last_active":"2024-09-10T21:12:25.785035+0000","last_peered":"2024-09-10T21:12:25.785035+0000","last_clean":"2024-09-10T21:12:25.785035+0000","last_became_active":"2024-09-10T21:12:24.794735+0000","last_became_peered":"2024-09-10T21:12:24.794735+0000","last_unstale":"2024-09-10T21:12:25.785035+0000","last_undegraded":"2024-09-10T21:12:25.785035+0000","last_fullsized":"2024-09-10T21:12:25.785035+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:37.987597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786956+0000","last_change":"2024-09-10T21:12:23.795368+0000","last_active":"2024-09-10T21:12:25.786956+0000","last_peered":"2024-09-10T21:12:25.786956+0000","last_clean":"2024-09-10T21:12:25.786956+0000","last_became_active":"2024-09-10T21:12:23.795086+0000","last_became_peered":"2024-09-10T21:12:23.795086+0000","last_unstale":"2024-09-10T21:12:25.786956+0000","last_undegraded":"2024-09-10T21:12:25.786956+0000","last_fullsized":"2024-09-10T21:12:25.786956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:32.008963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786672+0000","last_change":"2024-09-10T21:12:24.808003+0000","last_active":"2024-09-10T21:12:25.786672+0000","last_peered":"2024-09-10T21:12:25.786672+0000","last_clean":"2024-09-10T21:12:25.786672+0000","last_became_active":"2024-09-10T21:12:24.807702+0000","last_became_peered":"2024-09-10T21:12:24.807702+0000","last_unstale":"2024-09-10T21:12:25.786672+0000","last_undegraded":"2024-09-10T21:12:25.786672+0000","last_fullsized":"2024-09-10T21:12:25.786672+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:26.038549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428269+0000","last_change":"2024-09-10T21:12:23.799265+0000","last_active":"2024-09-10T21:12:26.428269+0000","last_peered":"2024-09-10T21:12:26.428269+0000","last_clean":"2024-09-10T21:12:26.428269+0000","last_became_active":"2024-09-10T21:12:23.798990+0000","last_became_peered":"2024-09-10T21:12:23.798990+0000","last_unstale":"2024-09-10T21:12:26.428269+0000","last_undegraded":"2024-09-10T21:12:26.428269+0000","last_fullsized":"2024-09-10T21:12:26.428269+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:18.976586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429200+0000","last_change":"2024-09-10T21:12:24.797557+0000","last_active":"2024-09-10T21:12:26.429200+0000","last_peered":"2024-09-10T21:12:26.429200+0000","last_clean":"2024-09-10T21:12:26.429200+0000","last_became_active":"2024-09-10T21:12:24.797284+0000","last_became_peered":"2024-09-10T21:12:24.797284+0000","last_unstale":"2024-09-10T21:12:26.429200+0000","last_undegraded":"2024-09-10T21:12:26.429200+0000","last_fullsized":"2024-09-10T21:12:26.429200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:51:34.878779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786779+0000","last_change":"2024-09-10T21:12:23.798175+0000","last_active":"2024-09-10T21:12:25.786779+0000","last_peered":"2024-09-10T21:12:25.786779+0000","last_clean":"2024-09-10T21:12:25.786779+0000","last_became_active":"2024-09-10T21:12:23.797844+0000","last_became_peered":"2024-09-10T21:12:23.797844+0000","last_unstale":"2024-09-10T21:12:25.786779+0000","last_undegraded":"2024-09-10T21:12:25.786779+0000","last_fullsized":"2024-09-10T21:12:25.786779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:57.897857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427917+0000","last_change":"2024-09-10T21:12:24.820636+0000","last_active":"2024-09-10T21:12:26.427917+0000","last_peered":"2024-09-10T21:12:26.427917+0000","last_clean":"2024-09-10T21:12:26.427917+0000","last_became_active":"2024-09-10T21:12:24.820280+0000","last_became_peered":"2024-09-10T21:12:24.820280+0000","last_unstale":"2024-09-10T21:12:26.427917+0000","last_undegraded":"2024-09-10T21:12:26.427917+0000","last_fullsized":"2024-09-10T21:12:26.427917+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:06.763115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429325+0000","last_change":"2024-09-10T21:12:23.803893+0000","last_active":"2024-09-10T21:12:26.429325+0000","last_peered":"2024-09-10T21:12:26.429325+0000","last_clean":"2024-09-10T21:12:26.429325+0000","last_became_active":"2024-09-10T21:12:23.803599+0000","last_became_peered":"2024-09-10T21:12:23.803599+0000","last_unstale":"2024-09-10T21:12:26.429325+0000","last_undegraded":"2024-09-10T21:12:26.429325+0000","last_fullsized":"2024-09-10T21:12:26.429325+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:46.177434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427703+0000","last_change":"2024-09-10T21:12:24.813412+0000","last_active":"2024-09-10T21:12:26.427703+0000","last_peered":"2024-09-10T21:12:26.427703+0000","last_clean":"2024-09-10T21:12:26.427703+0000","last_became_active":"2024-09-10T21:12:24.802579+0000","last_became_peered":"2024-09-10T21:12:24.802579+0000","last_unstale":"2024-09-10T21:12:26.427703+0000","last_undegraded":"2024-09-10T21:12:26.427703+0000","last_fullsized":"2024-09-10T21:12:26.427703+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:51.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427719+0000","last_change":"2024-09-10T21:12:23.793275+0000","last_active":"2024-09-10T21:12:26.427719+0000","last_peered":"2024-09-10T21:12:26.427719+0000","last_clean":"2024-09-10T21:12:26.427719+0000","last_became_active":"2024-09-10T21:12:23.792867+0000","last_became_peered":"2024-09-10T21:12:23.792867+0000","last_unstale":"2024-09-10T21:12:26.427719+0000","last_undegraded":"2024-09-10T21:12:26.427719+0000","last_fullsized":"2024-09-10T21:12:26.427719+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:54.791716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427761+0000","last_change":"2024-09-10T21:12:24.806299+0000","last_active":"2024-09-10T21:12:26.427761+0000","last_peered":"2024-09-10T21:12:26.427761+0000","last_clean":"2024-09-10T21:12:26.427761+0000","last_became_active":"2024-09-10T21:12:24.806025+0000","last_became_peered":"2024-09-10T21:12:24.806025+0000","last_unstale":"2024-09-10T21:12:26.427761+0000","last_undegraded":"2024-09-10T21:12:26.427761+0000","last_fullsized":"2024-09-10T21:12:26.427761+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:19.306195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427757+0000","last_change":"2024-09-10T21:12:23.798078+0000","last_active":"2024-09-10T21:12:26.427757+0000","last_peered":"2024-09-10T21:12:26.427757+0000","last_clean":"2024-09-10T21:12:26.427757+0000","last_became_active":"2024-09-10T21:12:23.797773+0000","last_became_peered":"2024-09-10T21:12:23.797773+0000","last_unstale":"2024-09-10T21:12:26.427757+0000","last_undegraded":"2024-09-10T21:12:26.427757+0000","last_fullsized":"2024-09-10T21:12:26.427757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:34.190967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429612+0000","last_change":"2024-09-10T21:12:24.804022+0000","last_active":"2024-09-10T21:12:26.429612+0000","last_peered":"2024-09-10T21:12:26.429612+0000","last_clean":"2024-09-10T21:12:26.429612+0000","last_became_active":"2024-09-10T21:12:24.803828+0000","last_became_peered":"2024-09-10T21:12:24.803828+0000","last_unstale":"2024-09-10T21:12:26.429612+0000","last_undegraded":"2024-09-10T21:12:26.429612+0000","last_fullsized":"2024-09-10T21:12:26.429612+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:33.659623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428656+0000","last_change":"2024-09-10T21:12:23.808653+0000","last_active":"2024-09-10T21:12:26.428656+0000","last_peered":"2024-09-10T21:12:26.428656+0000","last_clean":"2024-09-10T21:12:26.428656+0000","last_became_active":"2024-09-10T21:12:23.808383+0000","last_became_peered":"2024-09-10T21:12:23.808383+0000","last_unstale":"2024-09-10T21:12:26.428656+0000","last_undegraded":"2024-09-10T21:12:26.428656+0000","last_fullsized":"2024-09-10T21:12:26.428656+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:55.562523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428596+0000","last_change":"2024-09-10T21:12:24.802802+0000","last_active":"2024-09-10T21:12:26.428596+0000","last_peered":"2024-09-10T21:12:26.428596+0000","last_clean":"2024-09-10T21:12:26.428596+0000","last_became_active":"2024-09-10T21:12:24.802449+0000","last_became_peered":"2024-09-10T21:12:24.802449+0000","last_unstale":"2024-09-10T21:12:26.428596+0000","last_undegraded":"2024-09-10T21:12:26.428596+0000","last_fullsized":"2024-09-10T21:12:26.428596+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:37.497080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428077+0000","last_change":"2024-09-10T21:12:23.807246+0000","last_active":"2024-09-10T21:12:26.428077+0000","last_peered":"2024-09-10T21:12:26.428077+0000","last_clean":"2024-09-10T21:12:26.428077+0000","last_became_active":"2024-09-10T21:12:23.806959+0000","last_became_peered":"2024-09-10T21:12:23.806959+0000","last_unstale":"2024-09-10T21:12:26.428077+0000","last_undegraded":"2024-09-10T21:12:26.428077+0000","last_fullsized":"2024-09-10T21:12:26.428077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:46.804078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786038+0000","last_change":"2024-09-10T21:12:24.822928+0000","last_active":"2024-09-10T21:12:25.786038+0000","last_peered":"2024-09-10T21:12:25.786038+0000","last_clean":"2024-09-10T21:12:25.786038+0000","last_became_active":"2024-09-10T21:12:24.822626+0000","last_became_peered":"2024-09-10T21:12:24.822626+0000","last_unstale":"2024-09-10T21:12:25.786038+0000","last_undegraded":"2024-09-10T21:12:25.786038+0000","last_fullsized":"2024-09-10T21:12:25.786038+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:06.763115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784521+0000","last_change":"2024-09-10T21:12:23.790411+0000","last_active":"2024-09-10T21:12:25.784521+0000","last_peered":"2024-09-10T21:12:25.784521+0000","last_clean":"2024-09-10T21:12:25.784521+0000","last_became_active":"2024-09-10T21:12:23.790109+0000","last_became_peered":"2024-09-10T21:12:23.790109+0000","last_unstale":"2024-09-10T21:12:25.784521+0000","last_undegraded":"2024-09-10T21:12:25.784521+0000","last_fullsized":"2024-09-10T21:12:25.784521+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:25.257787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428953+0000","last_change":"2024-09-10T21:12:24.808556+0000","last_active":"2024-09-10T21:12:26.428953+0000","last_peered":"2024-09-10T21:12:26.428953+0000","last_clean":"2024-09-10T21:12:26.428953+0000","last_became_active":"2024-09-10T21:12:24.808281+0000","last_became_peered":"2024-09-10T21:12:24.808281+0000","last_unstale":"2024-09-10T21:12:26.428953+0000","last_undegraded":"2024-09-10T21:12:26.428953+0000","last_fullsized":"2024-09-10T21:12:26.428953+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:04.882193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428411+0000","last_change":"2024-09-10T21:12:23.802815+0000","last_active":"2024-09-10T21:12:26.428411+0000","last_peered":"2024-09-10T21:12:26.428411+0000","last_clean":"2024-09-10T21:12:26.428411+0000","last_became_active":"2024-09-10T21:12:23.802540+0000","last_became_peered":"2024-09-10T21:12:23.802540+0000","last_unstale":"2024-09-10T21:12:26.428411+0000","last_undegraded":"2024-09-10T21:12:26.428411+0000","last_fullsized":"2024-09-10T21:12:26.428411+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:22.630049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428740+0000","last_change":"2024-09-10T21:12:24.822282+0000","last_active":"2024-09-10T21:12:26.428740+0000","last_peered":"2024-09-10T21:12:26.428740+0000","last_clean":"2024-09-10T21:12:26.428740+0000","last_became_active":"2024-09-10T21:12:24.821907+0000","last_became_peered":"2024-09-10T21:12:24.821907+0000","last_unstale":"2024-09-10T21:12:26.428740+0000","last_undegraded":"2024-09-10T21:12:26.428740+0000","last_fullsized":"2024-09-10T21:12:26.428740+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:45.236002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427043+0000","last_change":"2024-09-10T21:12:23.800329+0000","last_active":"2024-09-10T21:12:26.427043+0000","last_peered":"2024-09-10T21:12:26.427043+0000","last_clean":"2024-09-10T21:12:26.427043+0000","last_became_active":"2024-09-10T21:12:23.799441+0000","last_became_peered":"2024-09-10T21:12:23.799441+0000","last_unstale":"2024-09-10T21:12:26.427043+0000","last_undegraded":"2024-09-10T21:12:26.427043+0000","last_fullsized":"2024-09-10T21:12:26.427043+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:41.839052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427386+0000","last_change":"2024-09-10T21:12:24.799963+0000","last_active":"2024-09-10T21:12:26.427386+0000","last_peered":"2024-09-10T21:12:26.427386+0000","last_clean":"2024-09-10T21:12:26.427386+0000","last_became_active":"2024-09-10T21:12:24.799685+0000","last_became_peered":"2024-09-10T21:12:24.799685+0000","last_unstale":"2024-09-10T21:12:26.427386+0000","last_undegraded":"2024-09-10T21:12:26.427386+0000","last_fullsized":"2024-09-10T21:12:26.427386+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:14.399311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427958+0000","last_change":"2024-09-10T21:12:23.800966+0000","last_active":"2024-09-10T21:12:26.427958+0000","last_peered":"2024-09-10T21:12:26.427958+0000","last_clean":"2024-09-10T21:12:26.427958+0000","last_became_active":"2024-09-10T21:12:23.800647+0000","last_became_peered":"2024-09-10T21:12:23.800647+0000","last_unstale":"2024-09-10T21:12:26.427958+0000","last_undegraded":"2024-09-10T21:12:26.427958+0000","last_fullsized":"2024-09-10T21:12:26.427958+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:56.744606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427189+0000","last_change":"2024-09-10T21:12:24.799381+0000","last_active":"2024-09-10T21:12:26.427189+0000","last_peered":"2024-09-10T21:12:26.427189+0000","last_clean":"2024-09-10T21:12:26.427189+0000","last_became_active":"2024-09-10T21:12:24.798305+0000","last_became_peered":"2024-09-10T21:12:24.798305+0000","last_unstale":"2024-09-10T21:12:26.427189+0000","last_undegraded":"2024-09-10T21:12:26.427189+0000","last_fullsized":"2024-09-10T21:12:26.427189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:37.497080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784996+0000","last_change":"2024-09-10T21:12:23.806810+0000","last_active":"2024-09-10T21:12:25.784996+0000","last_peered":"2024-09-10T21:12:25.784996+0000","last_clean":"2024-09-10T21:12:25.784996+0000","last_became_active":"2024-09-10T21:12:23.806235+0000","last_became_peered":"2024-09-10T21:12:23.806235+0000","last_unstale":"2024-09-10T21:12:25.784996+0000","last_undegraded":"2024-09-10T21:12:25.784996+0000","last_fullsized":"2024-09-10T21:12:25.784996+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:11.902707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784660+0000","last_change":"2024-09-10T21:12:24.800682+0000","last_active":"2024-09-10T21:12:25.784660+0000","last_peered":"2024-09-10T21:12:25.784660+0000","last_clean":"2024-09-10T21:12:25.784660+0000","last_became_active":"2024-09-10T21:12:24.800333+0000","last_became_peered":"2024-09-10T21:12:24.800333+0000","last_unstale":"2024-09-10T21:12:25.784660+0000","last_undegraded":"2024-09-10T21:12:25.784660+0000","last_fullsized":"2024-09-10T21:12:25.784660+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:49.709340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":23,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427889+0000","last_change":"2024-09-10T21:12:23.796942+0000","last_active":"2024-09-10T21:12:26.427889+0000","last_peered":"2024-09-10T21:12:26.427889+0000","last_clean":"2024-09-10T21:12:26.427889+0000","last_became_active":"2024-09-10T21:12:23.796661+0000","last_became_peered":"2024-09-10T21:12:23.796661+0000","last_unstale":"2024-09-10T21:12:26.427889+0000","last_undegraded":"2024-09-10T21:12:26.427889+0000","last_fullsized":"2024-09-10T21:12:26.427889+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:05.892601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427615+0000","last_change":"2024-09-10T21:12:24.800375+0000","last_active":"2024-09-10T21:12:26.427615+0000","last_peered":"2024-09-10T21:12:26.427615+0000","last_clean":"2024-09-10T21:12:26.427615+0000","last_became_active":"2024-09-10T21:12:24.800087+0000","last_became_peered":"2024-09-10T21:12:24.800087+0000","last_unstale":"2024-09-10T21:12:26.427615+0000","last_undegraded":"2024-09-10T21:12:26.427615+0000","last_fullsized":"2024-09-10T21:12:26.427615+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:46.274444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786575+0000","last_change":"2024-09-10T21:12:23.797369+0000","last_active":"2024-09-10T21:12:25.786575+0000","last_peered":"2024-09-10T21:12:25.786575+0000","last_clean":"2024-09-10T21:12:25.786575+0000","last_became_active":"2024-09-10T21:12:23.796137+0000","last_became_peered":"2024-09-10T21:12:23.796137+0000","last_unstale":"2024-09-10T21:12:25.786575+0000","last_undegraded":"2024-09-10T21:12:25.786575+0000","last_fullsized":"2024-09-10T21:12:25.786575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:19.161900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427399+0000","last_change":"2024-09-10T21:12:24.800698+0000","last_active":"2024-09-10T21:12:26.427399+0000","last_peered":"2024-09-10T21:12:26.427399+0000","last_clean":"2024-09-10T21:12:26.427399+0000","last_became_active":"2024-09-10T21:12:24.799724+0000","last_became_peered":"2024-09-10T21:12:24.799724+0000","last_unstale":"2024-09-10T21:12:26.427399+0000","last_undegraded":"2024-09-10T21:12:26.427399+0000","last_fullsized":"2024-09-10T21:12:26.427399+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:44.428772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429684+0000","last_change":"2024-09-10T21:12:23.807367+0000","last_active":"2024-09-10T21:12:26.429684+0000","last_peered":"2024-09-10T21:12:26.429684+0000","last_clean":"2024-09-10T21:12:26.429684+0000","last_became_active":"2024-09-10T21:12:23.806960+0000","last_became_peered":"2024-09-10T21:12:23.806960+0000","last_unstale":"2024-09-10T21:12:26.429684+0000","last_undegraded":"2024-09-10T21:12:26.429684+0000","last_fullsized":"2024-09-10T21:12:26.429684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:21.476191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785216+0000","last_change":"2024-09-10T21:12:24.821547+0000","last_active":"2024-09-10T21:12:25.785216+0000","last_peered":"2024-09-10T21:12:25.785216+0000","last_clean":"2024-09-10T21:12:25.785216+0000","last_became_active":"2024-09-10T21:12:24.821285+0000","last_became_peered":"2024-09-10T21:12:24.821285+0000","last_unstale":"2024-09-10T21:12:25.785216+0000","last_undegraded":"2024-09-10T21:12:25.785216+0000","last_fullsized":"2024-09-10T21:12:25.785216+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:11.907167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429276+0000","last_change":"2024-09-10T21:12:23.810651+0000","last_active":"2024-09-10T21:12:26.429276+0000","last_peered":"2024-09-10T21:12:26.429276+0000","last_clean":"2024-09-10T21:12:26.429276+0000","last_became_active":"2024-09-10T21:12:23.809721+0000","last_became_peered":"2024-09-10T21:12:23.809721+0000","last_unstale":"2024-09-10T21:12:26.429276+0000","last_undegraded":"2024-09-10T21:12:26.429276+0000","last_fullsized":"2024-09-10T21:12:26.429276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:13.969390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429203+0000","last_change":"2024-09-10T21:12:24.802031+0000","last_active":"2024-09-10T21:12:26.429203+0000","last_peered":"2024-09-10T21:12:26.429203+0000","last_clean":"2024-09-10T21:12:26.429203+0000","last_became_active":"2024-09-10T21:12:24.801761+0000","last_became_peered":"2024-09-10T21:12:24.801761+0000","last_unstale":"2024-09-10T21:12:26.429203+0000","last_undegraded":"2024-09-10T21:12:26.429203+0000","last_fullsized":"2024-09-10T21:12:26.429203+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:30.695194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786779+0000","last_change":"2024-09-10T21:12:23.806377+0000","last_active":"2024-09-10T21:12:25.786779+0000","last_peered":"2024-09-10T21:12:25.786779+0000","last_clean":"2024-09-10T21:12:25.786779+0000","last_became_active":"2024-09-10T21:12:23.806032+0000","last_became_peered":"2024-09-10T21:12:23.806032+0000","last_unstale":"2024-09-10T21:12:25.786779+0000","last_undegraded":"2024-09-10T21:12:25.786779+0000","last_fullsized":"2024-09-10T21:12:25.786779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:18.235286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428188+0000","last_change":"2024-09-10T21:12:24.823192+0000","last_active":"2024-09-10T21:12:26.428188+0000","last_peered":"2024-09-10T21:12:26.428188+0000","last_clean":"2024-09-10T21:12:26.428188+0000","last_became_active":"2024-09-10T21:12:24.822927+0000","last_became_peered":"2024-09-10T21:12:24.822927+0000","last_unstale":"2024-09-10T21:12:26.428188+0000","last_undegraded":"2024-09-10T21:12:26.428188+0000","last_fullsized":"2024-09-10T21:12:26.428188+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:16.729677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427725+0000","last_change":"2024-09-10T21:12:23.804977+0000","last_active":"2024-09-10T21:12:26.427725+0000","last_peered":"2024-09-10T21:12:26.427725+0000","last_clean":"2024-09-10T21:12:26.427725+0000","last_became_active":"2024-09-10T21:12:23.804671+0000","last_became_peered":"2024-09-10T21:12:23.804671+0000","last_unstale":"2024-09-10T21:12:26.427725+0000","last_undegraded":"2024-09-10T21:12:26.427725+0000","last_fullsized":"2024-09-10T21:12:26.427725+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:44.061600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786247+0000","last_change":"2024-09-10T21:12:24.804621+0000","last_active":"2024-09-10T21:12:25.786247+0000","last_peered":"2024-09-10T21:12:25.786247+0000","last_clean":"2024-09-10T21:12:25.786247+0000","last_became_active":"2024-09-10T21:12:24.804273+0000","last_became_peered":"2024-09-10T21:12:24.804273+0000","last_unstale":"2024-09-10T21:12:25.786247+0000","last_undegraded":"2024-09-10T21:12:25.786247+0000","last_fullsized":"2024-09-10T21:12:25.786247+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:14.393333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428072+0000","last_change":"2024-09-10T21:12:23.808671+0000","last_active":"2024-09-10T21:12:26.428072+0000","last_peered":"2024-09-10T21:12:26.428072+0000","last_clean":"2024-09-10T21:12:26.428072+0000","last_became_active":"2024-09-10T21:12:23.808137+0000","last_became_peered":"2024-09-10T21:12:23.808137+0000","last_unstale":"2024-09-10T21:12:26.428072+0000","last_undegraded":"2024-09-10T21:12:26.428072+0000","last_fullsized":"2024-09-10T21:12:26.428072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:02.838356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786010+0000","last_change":"2024-09-10T21:12:24.812457+0000","last_active":"2024-09-10T21:12:25.786010+0000","last_peered":"2024-09-10T21:12:25.786010+0000","last_clean":"2024-09-10T21:12:25.786010+0000","last_became_active":"2024-09-10T21:12:24.812056+0000","last_became_peered":"2024-09-10T21:12:24.812056+0000","last_unstale":"2024-09-10T21:12:25.786010+0000","last_undegraded":"2024-09-10T21:12:25.786010+0000","last_fullsized":"2024-09-10T21:12:25.786010+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:51.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427529+0000","last_change":"2024-09-10T21:12:23.808302+0000","last_active":"2024-09-10T21:12:26.427529+0000","last_peered":"2024-09-10T21:12:26.427529+0000","last_clean":"2024-09-10T21:12:26.427529+0000","last_became_active":"2024-09-10T21:12:23.807962+0000","last_became_peered":"2024-09-10T21:12:23.807962+0000","last_unstale":"2024-09-10T21:12:26.427529+0000","last_undegraded":"2024-09-10T21:12:26.427529+0000","last_fullsized":"2024-09-10T21:12:26.427529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:39.325026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786383+0000","last_change":"2024-09-10T21:12:24.819374+0000","last_active":"2024-09-10T21:12:25.786383+0000","last_peered":"2024-09-10T21:12:25.786383+0000","last_clean":"2024-09-10T21:12:25.786383+0000","last_became_active":"2024-09-10T21:12:24.819016+0000","last_became_peered":"2024-09-10T21:12:24.819016+0000","last_unstale":"2024-09-10T21:12:25.786383+0000","last_undegraded":"2024-09-10T21:12:25.786383+0000","last_fullsized":"2024-09-10T21:12:25.786383+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:16.729677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427497+0000","last_change":"2024-09-10T21:12:23.801081+0000","last_active":"2024-09-10T21:12:26.427497+0000","last_peered":"2024-09-10T21:12:26.427497+0000","last_clean":"2024-09-10T21:12:26.427497+0000","last_became_active":"2024-09-10T21:12:23.800792+0000","last_became_peered":"2024-09-10T21:12:23.800792+0000","last_unstale":"2024-09-10T21:12:26.427497+0000","last_undegraded":"2024-09-10T21:12:26.427497+0000","last_fullsized":"2024-09-10T21:12:26.427497+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:24.670926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429005+0000","last_change":"2024-09-10T21:12:24.805825+0000","last_active":"2024-09-10T21:12:26.429005+0000","last_peered":"2024-09-10T21:12:26.429005+0000","last_clean":"2024-09-10T21:12:26.429005+0000","last_became_active":"2024-09-10T21:12:24.805527+0000","last_became_peered":"2024-09-10T21:12:24.805527+0000","last_unstale":"2024-09-10T21:12:26.429005+0000","last_undegraded":"2024-09-10T21:12:26.429005+0000","last_fullsized":"2024-09-10T21:12:26.429005+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:17.948171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786931+0000","last_change":"2024-09-10T21:12:23.800595+0000","last_active":"2024-09-10T21:12:25.786931+0000","last_peered":"2024-09-10T21:12:25.786931+0000","last_clean":"2024-09-10T21:12:25.786931+0000","last_became_active":"2024-09-10T21:12:23.800279+0000","last_became_peered":"2024-09-10T21:12:23.800279+0000","last_unstale":"2024-09-10T21:12:25.786931+0000","last_undegraded":"2024-09-10T21:12:25.786931+0000","last_fullsized":"2024-09-10T21:12:25.786931+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:25.846794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428151+0000","last_change":"2024-09-10T21:12:24.805436+0000","last_active":"2024-09-10T21:12:26.428151+0000","last_peered":"2024-09-10T21:12:26.428151+0000","last_clean":"2024-09-10T21:12:26.428151+0000","last_became_active":"2024-09-10T21:12:24.805106+0000","last_became_peered":"2024-09-10T21:12:24.805106+0000","last_unstale":"2024-09-10T21:12:26.428151+0000","last_undegraded":"2024-09-10T21:12:26.428151+0000","last_fullsized":"2024-09-10T21:12:26.428151+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:32.446130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428690+0000","last_change":"2024-09-10T21:12:23.810567+0000","last_active":"2024-09-10T21:12:26.428690+0000","last_peered":"2024-09-10T21:12:26.428690+0000","last_clean":"2024-09-10T21:12:26.428690+0000","last_became_active":"2024-09-10T21:12:23.810246+0000","last_became_peered":"2024-09-10T21:12:23.810246+0000","last_unstale":"2024-09-10T21:12:26.428690+0000","last_undegraded":"2024-09-10T21:12:26.428690+0000","last_fullsized":"2024-09-10T21:12:26.428690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:09.107768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428041+0000","last_change":"2024-09-10T21:12:24.808843+0000","last_active":"2024-09-10T21:12:26.428041+0000","last_peered":"2024-09-10T21:12:26.428041+0000","last_clean":"2024-09-10T21:12:26.428041+0000","last_became_active":"2024-09-10T21:12:24.808570+0000","last_became_peered":"2024-09-10T21:12:24.808570+0000","last_unstale":"2024-09-10T21:12:26.428041+0000","last_undegraded":"2024-09-10T21:12:26.428041+0000","last_fullsized":"2024-09-10T21:12:26.428041+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:50.496957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427295+0000","last_change":"2024-09-10T21:12:23.807419+0000","last_active":"2024-09-10T21:12:26.427295+0000","last_peered":"2024-09-10T21:12:26.427295+0000","last_clean":"2024-09-10T21:12:26.427295+0000","last_became_active":"2024-09-10T21:12:23.806848+0000","last_became_peered":"2024-09-10T21:12:23.806848+0000","last_unstale":"2024-09-10T21:12:26.427295+0000","last_undegraded":"2024-09-10T21:12:26.427295+0000","last_fullsized":"2024-09-10T21:12:26.427295+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:46.663095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428169+0000","last_change":"2024-09-10T21:12:24.818146+0000","last_active":"2024-09-10T21:12:26.428169+0000","last_peered":"2024-09-10T21:12:26.428169+0000","last_clean":"2024-09-10T21:12:26.428169+0000","last_became_active":"2024-09-10T21:12:24.817782+0000","last_became_peered":"2024-09-10T21:12:24.817782+0000","last_unstale":"2024-09-10T21:12:26.428169+0000","last_undegraded":"2024-09-10T21:12:26.428169+0000","last_fullsized":"2024-09-10T21:12:26.428169+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:41.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428688+0000","last_change":"2024-09-10T21:12:23.795791+0000","last_active":"2024-09-10T21:12:26.428688+0000","last_peered":"2024-09-10T21:12:26.428688+0000","last_clean":"2024-09-10T21:12:26.428688+0000","last_became_active":"2024-09-10T21:12:23.795074+0000","last_became_peered":"2024-09-10T21:12:23.795074+0000","last_unstale":"2024-09-10T21:12:26.428688+0000","last_undegraded":"2024-09-10T21:12:26.428688+0000","last_fullsized":"2024-09-10T21:12:26.428688+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:36.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785827+0000","last_change":"2024-09-10T21:12:24.823887+0000","last_active":"2024-09-10T21:12:25.785827+0000","last_peered":"2024-09-10T21:12:25.785827+0000","last_clean":"2024-09-10T21:12:25.785827+0000","last_became_active":"2024-09-10T21:12:24.823427+0000","last_became_peered":"2024-09-10T21:12:24.823427+0000","last_unstale":"2024-09-10T21:12:25.785827+0000","last_undegraded":"2024-09-10T21:12:25.785827+0000","last_fullsized":"2024-09-10T21:12:25.785827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:04.280622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429384+0000","last_change":"2024-09-10T21:12:23.797756+0000","last_active":"2024-09-10T21:12:26.429384+0000","last_peered":"2024-09-10T21:12:26.429384+0000","last_clean":"2024-09-10T21:12:26.429384+0000","last_became_active":"2024-09-10T21:12:23.797410+0000","last_became_peered":"2024-09-10T21:12:23.797410+0000","last_unstale":"2024-09-10T21:12:26.429384+0000","last_undegraded":"2024-09-10T21:12:26.429384+0000","last_fullsized":"2024-09-10T21:12:26.429384+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:58.733833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427546+0000","last_change":"2024-09-10T21:12:24.816020+0000","last_active":"2024-09-10T21:12:26.427546+0000","last_peered":"2024-09-10T21:12:26.427546+0000","last_clean":"2024-09-10T21:12:26.427546+0000","last_became_active":"2024-09-10T21:12:24.815116+0000","last_became_peered":"2024-09-10T21:12:24.815116+0000","last_unstale":"2024-09-10T21:12:26.427546+0000","last_undegraded":"2024-09-10T21:12:26.427546+0000","last_fullsized":"2024-09-10T21:12:26.427546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:19.498294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785094+0000","last_change":"2024-09-10T21:12:23.800295+0000","last_active":"2024-09-10T21:12:25.785094+0000","last_peered":"2024-09-10T21:12:25.785094+0000","last_clean":"2024-09-10T21:12:25.785094+0000","last_became_active":"2024-09-10T21:12:23.799896+0000","last_became_peered":"2024-09-10T21:12:23.799896+0000","last_unstale":"2024-09-10T21:12:25.785094+0000","last_undegraded":"2024-09-10T21:12:25.785094+0000","last_fullsized":"2024-09-10T21:12:25.785094+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:25.846794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784955+0000","last_change":"2024-09-10T21:12:24.798401+0000","last_active":"2024-09-10T21:12:25.784955+0000","last_peered":"2024-09-10T21:12:25.784955+0000","last_clean":"2024-09-10T21:12:25.784955+0000","last_became_active":"2024-09-10T21:12:24.798177+0000","last_became_peered":"2024-09-10T21:12:24.798177+0000","last_unstale":"2024-09-10T21:12:25.784955+0000","last_undegraded":"2024-09-10T21:12:25.784955+0000","last_fullsized":"2024-09-10T21:12:25.784955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:21.261528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427060+0000","last_change":"2024-09-10T21:12:23.799895+0000","last_active":"2024-09-10T21:12:26.427060+0000","last_peered":"2024-09-10T21:12:26.427060+0000","last_clean":"2024-09-10T21:12:26.427060+0000","last_became_active":"2024-09-10T21:12:23.799596+0000","last_became_peered":"2024-09-10T21:12:23.799596+0000","last_unstale":"2024-09-10T21:12:26.427060+0000","last_undegraded":"2024-09-10T21:12:26.427060+0000","last_fullsized":"2024-09-10T21:12:26.427060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:21.476191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786669+0000","last_change":"2024-09-10T21:12:24.823727+0000","last_active":"2024-09-10T21:12:25.786669+0000","last_peered":"2024-09-10T21:12:25.786669+0000","last_clean":"2024-09-10T21:12:25.786669+0000","last_became_active":"2024-09-10T21:12:24.823434+0000","last_became_peered":"2024-09-10T21:12:24.823434+0000","last_unstale":"2024-09-10T21:12:25.786669+0000","last_undegraded":"2024-09-10T21:12:25.786669+0000","last_fullsized":"2024-09-10T21:12:25.786669+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:09.337686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429146+0000","last_change":"2024-09-10T21:12:23.803236+0000","last_active":"2024-09-10T21:12:26.429146+0000","last_peered":"2024-09-10T21:12:26.429146+0000","last_clean":"2024-09-10T21:12:26.429146+0000","last_became_active":"2024-09-10T21:12:23.802939+0000","last_became_peered":"2024-09-10T21:12:23.802939+0000","last_unstale":"2024-09-10T21:12:26.429146+0000","last_undegraded":"2024-09-10T21:12:26.429146+0000","last_fullsized":"2024-09-10T21:12:26.429146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:35.914109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427619+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:12:26.427619+0000","last_peered":"2024-09-10T21:12:26.427619+0000","last_clean":"2024-09-10T21:12:26.427619+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:12:26.427619+0000","last_undegraded":"2024-09-10T21:12:26.427619+0000","last_fullsized":"2024-09-10T21:12:26.427619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1130496,"data_stored":806790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509469,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":218104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":544768,"data_stored":218012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":216490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":216434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":806838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:12:30.766 INFO:tasks.ceph.mds.d.smithi052.stderr:2024-09-10T21:12:30.759+0000 7f0bd1200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:12:33.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:12:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 -- 172.21.15.16:0/736568305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552006e640 msgr2=0x7f55200ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 --2- 172.21.15.16:0/736568305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552006e640 0x7f55200ca170 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f550c009900 tx=0x7f550c01c850 comp rx=0 tx=0).stop 2024-09-10T21:12:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 -- 172.21.15.16:0/736568305 shutdown_connections 2024-09-10T21:12:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 --2- 172.21.15.16:0/736568305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552006e640 0x7f55200ca170 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 --2- 172.21.15.16:0/736568305 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55201599d0 0x7f552015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 -- 172.21.15.16:0/736568305 >> 172.21.15.16:0/736568305 conn(0x7f55200cfb20 msgr2=0x7f55200cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 -- 172.21.15.16:0/736568305 shutdown_connections 2024-09-10T21:12:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 -- 172.21.15.16:0/736568305 wait complete. 2024-09-10T21:12:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 Processor -- start 2024-09-10T21:12:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 -- start start 2024-09-10T21:12:33.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f552006e640 0x7f5520157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:33.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200ca6b0 0x7f55200c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:33.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55201599d0 0x7f5520156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:33.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5520168bd0 con 0x7f55200ca6b0 2024-09-10T21:12:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5520073010 con 0x7f552006e640 2024-09-10T21:12:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5527400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5520168d50 con 0x7f55201599d0 2024-09-10T21:12:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f551fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200ca6b0 0x7f55200c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f551fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200ca6b0 0x7f55200c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53642/0 (socket says 172.21.15.16:53642) 2024-09-10T21:12:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f551fe00640 1 -- 172.21.15.16:0/2424374293 learned_addr learned my addr 172.21.15.16:0/2424374293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.752+0000 7f5524a00640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f552006e640 0x7f5520157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f5525400640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55201599d0 0x7f5520156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f551fe00640 1 -- 172.21.15.16:0/2424374293 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55201599d0 msgr2=0x7f5520156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f551fe00640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55201599d0 0x7f5520156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f551fe00640 1 -- 172.21.15.16:0/2424374293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f552006e640 msgr2=0x7f5520157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f551fe00640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f552006e640 0x7f5520157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f551fe00640 1 -- 172.21.15.16:0/2424374293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55201572c0 con 0x7f55200ca6b0 2024-09-10T21:12:33.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f5524a00640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f552006e640 0x7f5520157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:33.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f551fe00640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200ca6b0 0x7f55200c98a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f550c01c680 tx=0x7f550c0a6b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f551cc00640 1 -- 172.21.15.16:0/2424374293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c0a6d60 con 0x7f55200ca6b0 2024-09-10T21:12:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f551cc00640 1 -- 172.21.15.16:0/2424374293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f550c004040 con 0x7f55200ca6b0 2024-09-10T21:12:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f551cc00640 1 -- 172.21.15.16:0/2424374293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c004470 con 0x7f55200ca6b0 2024-09-10T21:12:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f5527400640 1 -- 172.21.15.16:0/2424374293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55201fae30 con 0x7f55200ca6b0 2024-09-10T21:12:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f5527400640 1 -- 172.21.15.16:0/2424374293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55201fb3f0 con 0x7f55200ca6b0 2024-09-10T21:12:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f5527400640 1 -- 172.21.15.16:0/2424374293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55200ca170 con 0x7f55200ca6b0 2024-09-10T21:12:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f551cc00640 1 -- 172.21.15.16:0/2424374293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f550c003600 con 0x7f55200ca6b0 2024-09-10T21:12:33.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f551cc00640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55000725e0 0x7f5500074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:33.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.756+0000 7f551cc00640 1 -- 172.21.15.16:0/2424374293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f550c12de70 con 0x7f55200ca6b0 2024-09-10T21:12:33.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.760+0000 7f5524a00640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55000725e0 0x7f5500074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.760+0000 7f5524a00640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55000725e0 0x7f5500074aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f55200c6ea0 tx=0x7f5514005eb0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:33.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.764+0000 7f551cc00640 1 -- 172.21.15.16:0/2424374293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f550c0c8020 con 0x7f55200ca6b0 2024-09-10T21:12:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.932+0000 7f5527400640 1 -- 172.21.15.16:0/2424374293 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55201fabe0 con 0x7f55000725e0 2024-09-10T21:12:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.936+0000 7f551cc00640 1 -- 172.21.15.16:0/2424374293 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251924 (secure 0 0 0) 0x7f55201fabe0 con 0x7f55000725e0 2024-09-10T21:12:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:12:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.944+0000 7f5527400640 1 -- 172.21.15.16:0/2424374293 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55000725e0 msgr2=0x7f5500074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.944+0000 7f5527400640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55000725e0 0x7f5500074aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f55200c6ea0 tx=0x7f5514005eb0 comp rx=0 tx=0).stop 2024-09-10T21:12:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.944+0000 7f5527400640 1 -- 172.21.15.16:0/2424374293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200ca6b0 msgr2=0x7f55200c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.944+0000 7f5527400640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200ca6b0 0x7f55200c98a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f550c01c680 tx=0x7f550c0a6b70 comp rx=0 tx=0).stop 2024-09-10T21:12:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.944+0000 7f5527400640 1 -- 172.21.15.16:0/2424374293 shutdown_connections 2024-09-10T21:12:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.944+0000 7f5527400640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55000725e0 0x7f5500074aa0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.944+0000 7f5527400640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55201599d0 0x7f5520156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.944+0000 7f5527400640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55200ca6b0 0x7f55200c98a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.944+0000 7f5527400640 1 --2- 172.21.15.16:0/2424374293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f552006e640 0x7f5520157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.944+0000 7f5527400640 1 -- 172.21.15.16:0/2424374293 >> 172.21.15.16:0/2424374293 conn(0x7f55200cfb20 msgr2=0x7f55200b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.944+0000 7f5527400640 1 -- 172.21.15.16:0/2424374293 shutdown_connections 2024-09-10T21:12:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:33.944+0000 7f5527400640 1 -- 172.21.15.16:0/2424374293 wait complete. 2024-09-10T21:12:33.969 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-10T21:12:33.484898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254896,"kb_used_data":5384,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719824,"statfs":{"total":773094113280,"available":772833099776,"internally_reserved":0,"allocated":5513216,"data_stored":2916756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.702912"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428163+0000","last_change":"2024-09-10T21:12:24.807699+0000","last_active":"2024-09-10T21:12:26.428163+0000","last_peered":"2024-09-10T21:12:26.428163+0000","last_clean":"2024-09-10T21:12:26.428163+0000","last_became_active":"2024-09-10T21:12:24.807378+0000","last_became_peered":"2024-09-10T21:12:24.807378+0000","last_unstale":"2024-09-10T21:12:26.428163+0000","last_undegraded":"2024-09-10T21:12:26.428163+0000","last_fullsized":"2024-09-10T21:12:26.428163+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:06.104316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785215+0000","last_change":"2024-09-10T21:12:24.801034+0000","last_active":"2024-09-10T21:12:25.785215+0000","last_peered":"2024-09-10T21:12:25.785215+0000","last_clean":"2024-09-10T21:12:25.785215+0000","last_became_active":"2024-09-10T21:12:24.799842+0000","last_became_peered":"2024-09-10T21:12:24.799842+0000","last_unstale":"2024-09-10T21:12:25.785215+0000","last_undegraded":"2024-09-10T21:12:25.785215+0000","last_fullsized":"2024-09-10T21:12:25.785215+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:51.766507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429089+0000","last_change":"2024-09-10T21:12:24.811161+0000","last_active":"2024-09-10T21:12:26.429089+0000","last_peered":"2024-09-10T21:12:26.429089+0000","last_clean":"2024-09-10T21:12:26.429089+0000","last_became_active":"2024-09-10T21:12:24.810838+0000","last_became_peered":"2024-09-10T21:12:24.810838+0000","last_unstale":"2024-09-10T21:12:26.429089+0000","last_undegraded":"2024-09-10T21:12:26.429089+0000","last_fullsized":"2024-09-10T21:12:26.429089+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:14.399311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785704+0000","last_change":"2024-09-10T21:12:24.807996+0000","last_active":"2024-09-10T21:12:25.785704+0000","last_peered":"2024-09-10T21:12:25.785704+0000","last_clean":"2024-09-10T21:12:25.785704+0000","last_became_active":"2024-09-10T21:12:24.807614+0000","last_became_peered":"2024-09-10T21:12:24.807614+0000","last_unstale":"2024-09-10T21:12:25.785704+0000","last_undegraded":"2024-09-10T21:12:25.785704+0000","last_fullsized":"2024-09-10T21:12:25.785704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:58.826777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428361+0000","last_change":"2024-09-10T21:12:24.814793+0000","last_active":"2024-09-10T21:12:26.428361+0000","last_peered":"2024-09-10T21:12:26.428361+0000","last_clean":"2024-09-10T21:12:26.428361+0000","last_became_active":"2024-09-10T21:12:24.814494+0000","last_became_peered":"2024-09-10T21:12:24.814494+0000","last_unstale":"2024-09-10T21:12:26.428361+0000","last_undegraded":"2024-09-10T21:12:26.428361+0000","last_fullsized":"2024-09-10T21:12:26.428361+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:21.261528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785736+0000","last_change":"2024-09-10T21:12:24.827330+0000","last_active":"2024-09-10T21:12:25.785736+0000","last_peered":"2024-09-10T21:12:25.785736+0000","last_clean":"2024-09-10T21:12:25.785736+0000","last_became_active":"2024-09-10T21:12:24.827158+0000","last_became_peered":"2024-09-10T21:12:24.827158+0000","last_unstale":"2024-09-10T21:12:25.785736+0000","last_undegraded":"2024-09-10T21:12:25.785736+0000","last_fullsized":"2024-09-10T21:12:25.785736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:13.628817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427734+0000","last_change":"2024-09-10T21:12:24.808616+0000","last_active":"2024-09-10T21:12:26.427734+0000","last_peered":"2024-09-10T21:12:26.427734+0000","last_clean":"2024-09-10T21:12:26.427734+0000","last_became_active":"2024-09-10T21:12:24.808341+0000","last_became_peered":"2024-09-10T21:12:24.808341+0000","last_unstale":"2024-09-10T21:12:26.427734+0000","last_undegraded":"2024-09-10T21:12:26.427734+0000","last_fullsized":"2024-09-10T21:12:26.427734+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:23.140730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785688+0000","last_change":"2024-09-10T21:12:24.823724+0000","last_active":"2024-09-10T21:12:25.785688+0000","last_peered":"2024-09-10T21:12:25.785688+0000","last_clean":"2024-09-10T21:12:25.785688+0000","last_became_active":"2024-09-10T21:12:24.823452+0000","last_became_peered":"2024-09-10T21:12:24.823452+0000","last_unstale":"2024-09-10T21:12:25.785688+0000","last_undegraded":"2024-09-10T21:12:25.785688+0000","last_fullsized":"2024-09-10T21:12:25.785688+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:22.748738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.787140+0000","last_change":"2024-09-10T21:12:24.822052+0000","last_active":"2024-09-10T21:12:25.787140+0000","last_peered":"2024-09-10T21:12:25.787140+0000","last_clean":"2024-09-10T21:12:25.787140+0000","last_became_active":"2024-09-10T21:12:24.821688+0000","last_became_peered":"2024-09-10T21:12:24.821688+0000","last_unstale":"2024-09-10T21:12:25.787140+0000","last_undegraded":"2024-09-10T21:12:25.787140+0000","last_fullsized":"2024-09-10T21:12:25.787140+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:52.341058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428221+0000","last_change":"2024-09-10T21:12:24.813099+0000","last_active":"2024-09-10T21:12:26.428221+0000","last_peered":"2024-09-10T21:12:26.428221+0000","last_clean":"2024-09-10T21:12:26.428221+0000","last_became_active":"2024-09-10T21:12:24.812570+0000","last_became_peered":"2024-09-10T21:12:24.812570+0000","last_unstale":"2024-09-10T21:12:26.428221+0000","last_undegraded":"2024-09-10T21:12:26.428221+0000","last_fullsized":"2024-09-10T21:12:26.428221+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:25.911130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786429+0000","last_change":"2024-09-10T21:12:24.830636+0000","last_active":"2024-09-10T21:12:25.786429+0000","last_peered":"2024-09-10T21:12:25.786429+0000","last_clean":"2024-09-10T21:12:25.786429+0000","last_became_active":"2024-09-10T21:12:24.830369+0000","last_became_peered":"2024-09-10T21:12:24.830369+0000","last_unstale":"2024-09-10T21:12:25.786429+0000","last_undegraded":"2024-09-10T21:12:25.786429+0000","last_fullsized":"2024-09-10T21:12:25.786429+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:39.747387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784766+0000","last_change":"2024-09-10T21:12:24.788829+0000","last_active":"2024-09-10T21:12:25.784766+0000","last_peered":"2024-09-10T21:12:25.784766+0000","last_clean":"2024-09-10T21:12:25.784766+0000","last_became_active":"2024-09-10T21:12:24.788547+0000","last_became_peered":"2024-09-10T21:12:24.788547+0000","last_unstale":"2024-09-10T21:12:25.784766+0000","last_undegraded":"2024-09-10T21:12:25.784766+0000","last_fullsized":"2024-09-10T21:12:25.784766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:15.277562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785343+0000","last_change":"2024-09-10T21:12:24.826783+0000","last_active":"2024-09-10T21:12:25.785343+0000","last_peered":"2024-09-10T21:12:25.785343+0000","last_clean":"2024-09-10T21:12:25.785343+0000","last_became_active":"2024-09-10T21:12:24.826470+0000","last_became_peered":"2024-09-10T21:12:24.826470+0000","last_unstale":"2024-09-10T21:12:25.785343+0000","last_undegraded":"2024-09-10T21:12:25.785343+0000","last_fullsized":"2024-09-10T21:12:25.785343+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:37.303480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785877+0000","last_change":"2024-09-10T21:12:24.820100+0000","last_active":"2024-09-10T21:12:25.785877+0000","last_peered":"2024-09-10T21:12:25.785877+0000","last_clean":"2024-09-10T21:12:25.785877+0000","last_became_active":"2024-09-10T21:12:24.819477+0000","last_became_peered":"2024-09-10T21:12:24.819477+0000","last_unstale":"2024-09-10T21:12:25.785877+0000","last_undegraded":"2024-09-10T21:12:25.785877+0000","last_fullsized":"2024-09-10T21:12:25.785877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:22.872734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786381+0000","last_change":"2024-09-10T21:12:24.807623+0000","last_active":"2024-09-10T21:12:25.786381+0000","last_peered":"2024-09-10T21:12:25.786381+0000","last_clean":"2024-09-10T21:12:25.786381+0000","last_became_active":"2024-09-10T21:12:24.807290+0000","last_became_peered":"2024-09-10T21:12:24.807290+0000","last_unstale":"2024-09-10T21:12:25.786381+0000","last_undegraded":"2024-09-10T21:12:25.786381+0000","last_fullsized":"2024-09-10T21:12:25.786381+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:32.446130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428017+0000","last_change":"2024-09-10T21:12:24.822563+0000","last_active":"2024-09-10T21:12:26.428017+0000","last_peered":"2024-09-10T21:12:26.428017+0000","last_clean":"2024-09-10T21:12:26.428017+0000","last_became_active":"2024-09-10T21:12:24.822299+0000","last_became_peered":"2024-09-10T21:12:24.822299+0000","last_unstale":"2024-09-10T21:12:26.428017+0000","last_undegraded":"2024-09-10T21:12:26.428017+0000","last_fullsized":"2024-09-10T21:12:26.428017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:15.277562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784799+0000","last_change":"2024-09-10T21:12:24.804138+0000","last_active":"2024-09-10T21:12:25.784799+0000","last_peered":"2024-09-10T21:12:25.784799+0000","last_clean":"2024-09-10T21:12:25.784799+0000","last_became_active":"2024-09-10T21:12:24.803868+0000","last_became_peered":"2024-09-10T21:12:24.803868+0000","last_unstale":"2024-09-10T21:12:25.784799+0000","last_undegraded":"2024-09-10T21:12:25.784799+0000","last_fullsized":"2024-09-10T21:12:25.784799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:06.104316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429476+0000","last_change":"2024-09-10T21:12:24.806841+0000","last_active":"2024-09-10T21:12:26.429476+0000","last_peered":"2024-09-10T21:12:26.429476+0000","last_clean":"2024-09-10T21:12:26.429476+0000","last_became_active":"2024-09-10T21:12:24.804956+0000","last_became_peered":"2024-09-10T21:12:24.804956+0000","last_unstale":"2024-09-10T21:12:26.429476+0000","last_undegraded":"2024-09-10T21:12:26.429476+0000","last_fullsized":"2024-09-10T21:12:26.429476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:46.274444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428515+0000","last_change":"2024-09-10T21:12:24.817361+0000","last_active":"2024-09-10T21:12:26.428515+0000","last_peered":"2024-09-10T21:12:26.428515+0000","last_clean":"2024-09-10T21:12:26.428515+0000","last_became_active":"2024-09-10T21:12:24.815622+0000","last_became_peered":"2024-09-10T21:12:24.815622+0000","last_unstale":"2024-09-10T21:12:26.428515+0000","last_undegraded":"2024-09-10T21:12:26.428515+0000","last_fullsized":"2024-09-10T21:12:26.428515+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:11.907167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428188+0000","last_change":"2024-09-10T21:12:24.792792+0000","last_active":"2024-09-10T21:12:26.428188+0000","last_peered":"2024-09-10T21:12:26.428188+0000","last_clean":"2024-09-10T21:12:26.428188+0000","last_became_active":"2024-09-10T21:12:24.792220+0000","last_became_peered":"2024-09-10T21:12:24.792220+0000","last_unstale":"2024-09-10T21:12:26.428188+0000","last_undegraded":"2024-09-10T21:12:26.428188+0000","last_fullsized":"2024-09-10T21:12:26.428188+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:39.747387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786232+0000","last_change":"2024-09-10T21:12:24.824108+0000","last_active":"2024-09-10T21:12:25.786232+0000","last_peered":"2024-09-10T21:12:25.786232+0000","last_clean":"2024-09-10T21:12:25.786232+0000","last_became_active":"2024-09-10T21:12:24.823125+0000","last_became_peered":"2024-09-10T21:12:24.823125+0000","last_unstale":"2024-09-10T21:12:25.786232+0000","last_undegraded":"2024-09-10T21:12:25.786232+0000","last_fullsized":"2024-09-10T21:12:25.786232+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:41.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427209+0000","last_change":"2024-09-10T21:12:24.803379+0000","last_active":"2024-09-10T21:12:26.427209+0000","last_peered":"2024-09-10T21:12:26.427209+0000","last_clean":"2024-09-10T21:12:26.427209+0000","last_became_active":"2024-09-10T21:12:24.802088+0000","last_became_peered":"2024-09-10T21:12:24.802088+0000","last_unstale":"2024-09-10T21:12:26.427209+0000","last_undegraded":"2024-09-10T21:12:26.427209+0000","last_fullsized":"2024-09-10T21:12:26.427209+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:45.236002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428095+0000","last_change":"2024-09-10T21:12:24.800438+0000","last_active":"2024-09-10T21:12:26.428095+0000","last_peered":"2024-09-10T21:12:26.428095+0000","last_clean":"2024-09-10T21:12:26.428095+0000","last_became_active":"2024-09-10T21:12:24.799628+0000","last_became_peered":"2024-09-10T21:12:24.799628+0000","last_unstale":"2024-09-10T21:12:26.428095+0000","last_undegraded":"2024-09-10T21:12:26.428095+0000","last_fullsized":"2024-09-10T21:12:26.428095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:22.261573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784410+0000","last_change":"2024-09-10T21:12:24.822685+0000","last_active":"2024-09-10T21:12:25.784410+0000","last_peered":"2024-09-10T21:12:25.784410+0000","last_clean":"2024-09-10T21:12:25.784410+0000","last_became_active":"2024-09-10T21:12:24.821398+0000","last_became_peered":"2024-09-10T21:12:24.821398+0000","last_unstale":"2024-09-10T21:12:25.784410+0000","last_undegraded":"2024-09-10T21:12:25.784410+0000","last_fullsized":"2024-09-10T21:12:25.784410+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:02.665366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428387+0000","last_change":"2024-09-10T21:12:24.807404+0000","last_active":"2024-09-10T21:12:26.428387+0000","last_peered":"2024-09-10T21:12:26.428387+0000","last_clean":"2024-09-10T21:12:26.428387+0000","last_became_active":"2024-09-10T21:12:24.807144+0000","last_became_peered":"2024-09-10T21:12:24.807144+0000","last_unstale":"2024-09-10T21:12:26.428387+0000","last_undegraded":"2024-09-10T21:12:26.428387+0000","last_fullsized":"2024-09-10T21:12:26.428387+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:37.987597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786744+0000","last_change":"2024-09-10T21:12:24.829538+0000","last_active":"2024-09-10T21:12:25.786744+0000","last_peered":"2024-09-10T21:12:25.786744+0000","last_clean":"2024-09-10T21:12:25.786744+0000","last_became_active":"2024-09-10T21:12:24.829250+0000","last_became_peered":"2024-09-10T21:12:24.829250+0000","last_unstale":"2024-09-10T21:12:25.786744+0000","last_undegraded":"2024-09-10T21:12:25.786744+0000","last_fullsized":"2024-09-10T21:12:25.786744+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:47.207849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427811+0000","last_change":"2024-09-10T21:12:24.795084+0000","last_active":"2024-09-10T21:12:26.427811+0000","last_peered":"2024-09-10T21:12:26.427811+0000","last_clean":"2024-09-10T21:12:26.427811+0000","last_became_active":"2024-09-10T21:12:24.794823+0000","last_became_peered":"2024-09-10T21:12:24.794823+0000","last_unstale":"2024-09-10T21:12:26.427811+0000","last_undegraded":"2024-09-10T21:12:26.427811+0000","last_fullsized":"2024-09-10T21:12:26.427811+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:26.691816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428069+0000","last_change":"2024-09-10T21:12:24.793348+0000","last_active":"2024-09-10T21:12:26.428069+0000","last_peered":"2024-09-10T21:12:26.428069+0000","last_clean":"2024-09-10T21:12:26.428069+0000","last_became_active":"2024-09-10T21:12:24.793089+0000","last_became_peered":"2024-09-10T21:12:24.793089+0000","last_unstale":"2024-09-10T21:12:26.428069+0000","last_undegraded":"2024-09-10T21:12:26.428069+0000","last_fullsized":"2024-09-10T21:12:26.428069+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:03.310670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428591+0000","last_change":"2024-09-10T21:12:24.815439+0000","last_active":"2024-09-10T21:12:26.428591+0000","last_peered":"2024-09-10T21:12:26.428591+0000","last_clean":"2024-09-10T21:12:26.428591+0000","last_became_active":"2024-09-10T21:12:24.815191+0000","last_became_peered":"2024-09-10T21:12:24.815191+0000","last_unstale":"2024-09-10T21:12:26.428591+0000","last_undegraded":"2024-09-10T21:12:26.428591+0000","last_fullsized":"2024-09-10T21:12:26.428591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:14.014096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784439+0000","last_change":"2024-09-10T21:12:24.792836+0000","last_active":"2024-09-10T21:12:25.784439+0000","last_peered":"2024-09-10T21:12:25.784439+0000","last_clean":"2024-09-10T21:12:25.784439+0000","last_became_active":"2024-09-10T21:12:24.792569+0000","last_became_peered":"2024-09-10T21:12:24.792569+0000","last_unstale":"2024-09-10T21:12:25.784439+0000","last_undegraded":"2024-09-10T21:12:25.784439+0000","last_fullsized":"2024-09-10T21:12:25.784439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:31.753172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785090+0000","last_change":"2024-09-10T21:12:24.808015+0000","last_active":"2024-09-10T21:12:25.785090+0000","last_peered":"2024-09-10T21:12:25.785090+0000","last_clean":"2024-09-10T21:12:25.785090+0000","last_became_active":"2024-09-10T21:12:24.807723+0000","last_became_peered":"2024-09-10T21:12:24.807723+0000","last_unstale":"2024-09-10T21:12:25.785090+0000","last_undegraded":"2024-09-10T21:12:25.785090+0000","last_fullsized":"2024-09-10T21:12:25.785090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:26.164431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785759+0000","last_change":"2024-09-10T21:12:24.812500+0000","last_active":"2024-09-10T21:12:25.785759+0000","last_peered":"2024-09-10T21:12:25.785759+0000","last_clean":"2024-09-10T21:12:25.785759+0000","last_became_active":"2024-09-10T21:12:24.812143+0000","last_became_peered":"2024-09-10T21:12:24.812143+0000","last_unstale":"2024-09-10T21:12:25.785759+0000","last_undegraded":"2024-09-10T21:12:25.785759+0000","last_fullsized":"2024-09-10T21:12:25.785759+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:33.667666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427841+0000","last_change":"2024-09-10T21:12:24.817360+0000","last_active":"2024-09-10T21:12:26.427841+0000","last_peered":"2024-09-10T21:12:26.427841+0000","last_clean":"2024-09-10T21:12:26.427841+0000","last_became_active":"2024-09-10T21:12:24.817059+0000","last_became_peered":"2024-09-10T21:12:24.817059+0000","last_unstale":"2024-09-10T21:12:26.427841+0000","last_undegraded":"2024-09-10T21:12:26.427841+0000","last_fullsized":"2024-09-10T21:12:26.427841+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:33.659623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.787046+0000","last_change":"2024-09-10T21:12:23.795968+0000","last_active":"2024-09-10T21:12:25.787046+0000","last_peered":"2024-09-10T21:12:25.787046+0000","last_clean":"2024-09-10T21:12:25.787046+0000","last_became_active":"2024-09-10T21:12:23.795674+0000","last_became_peered":"2024-09-10T21:12:23.795674+0000","last_unstale":"2024-09-10T21:12:25.787046+0000","last_undegraded":"2024-09-10T21:12:25.787046+0000","last_fullsized":"2024-09-10T21:12:25.787046+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:34.576509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428635+0000","last_change":"2024-09-10T21:12:24.801727+0000","last_active":"2024-09-10T21:12:26.428635+0000","last_peered":"2024-09-10T21:12:26.428635+0000","last_clean":"2024-09-10T21:12:26.428635+0000","last_became_active":"2024-09-10T21:12:24.801282+0000","last_became_peered":"2024-09-10T21:12:24.801282+0000","last_unstale":"2024-09-10T21:12:26.428635+0000","last_undegraded":"2024-09-10T21:12:26.428635+0000","last_fullsized":"2024-09-10T21:12:26.428635+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:52.776680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428530+0000","last_change":"2024-09-10T21:12:23.799107+0000","last_active":"2024-09-10T21:12:26.428530+0000","last_peered":"2024-09-10T21:12:26.428530+0000","last_clean":"2024-09-10T21:12:26.428530+0000","last_became_active":"2024-09-10T21:12:23.798815+0000","last_became_peered":"2024-09-10T21:12:23.798815+0000","last_unstale":"2024-09-10T21:12:26.428530+0000","last_undegraded":"2024-09-10T21:12:26.428530+0000","last_fullsized":"2024-09-10T21:12:26.428530+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:41.715763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785035+0000","last_change":"2024-09-10T21:12:24.795186+0000","last_active":"2024-09-10T21:12:25.785035+0000","last_peered":"2024-09-10T21:12:25.785035+0000","last_clean":"2024-09-10T21:12:25.785035+0000","last_became_active":"2024-09-10T21:12:24.794735+0000","last_became_peered":"2024-09-10T21:12:24.794735+0000","last_unstale":"2024-09-10T21:12:25.785035+0000","last_undegraded":"2024-09-10T21:12:25.785035+0000","last_fullsized":"2024-09-10T21:12:25.785035+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:37.987597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786956+0000","last_change":"2024-09-10T21:12:23.795368+0000","last_active":"2024-09-10T21:12:25.786956+0000","last_peered":"2024-09-10T21:12:25.786956+0000","last_clean":"2024-09-10T21:12:25.786956+0000","last_became_active":"2024-09-10T21:12:23.795086+0000","last_became_peered":"2024-09-10T21:12:23.795086+0000","last_unstale":"2024-09-10T21:12:25.786956+0000","last_undegraded":"2024-09-10T21:12:25.786956+0000","last_fullsized":"2024-09-10T21:12:25.786956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:32.008963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786672+0000","last_change":"2024-09-10T21:12:24.808003+0000","last_active":"2024-09-10T21:12:25.786672+0000","last_peered":"2024-09-10T21:12:25.786672+0000","last_clean":"2024-09-10T21:12:25.786672+0000","last_became_active":"2024-09-10T21:12:24.807702+0000","last_became_peered":"2024-09-10T21:12:24.807702+0000","last_unstale":"2024-09-10T21:12:25.786672+0000","last_undegraded":"2024-09-10T21:12:25.786672+0000","last_fullsized":"2024-09-10T21:12:25.786672+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:26.038549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428269+0000","last_change":"2024-09-10T21:12:23.799265+0000","last_active":"2024-09-10T21:12:26.428269+0000","last_peered":"2024-09-10T21:12:26.428269+0000","last_clean":"2024-09-10T21:12:26.428269+0000","last_became_active":"2024-09-10T21:12:23.798990+0000","last_became_peered":"2024-09-10T21:12:23.798990+0000","last_unstale":"2024-09-10T21:12:26.428269+0000","last_undegraded":"2024-09-10T21:12:26.428269+0000","last_fullsized":"2024-09-10T21:12:26.428269+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:18.976586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429200+0000","last_change":"2024-09-10T21:12:24.797557+0000","last_active":"2024-09-10T21:12:26.429200+0000","last_peered":"2024-09-10T21:12:26.429200+0000","last_clean":"2024-09-10T21:12:26.429200+0000","last_became_active":"2024-09-10T21:12:24.797284+0000","last_became_peered":"2024-09-10T21:12:24.797284+0000","last_unstale":"2024-09-10T21:12:26.429200+0000","last_undegraded":"2024-09-10T21:12:26.429200+0000","last_fullsized":"2024-09-10T21:12:26.429200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:51:34.878779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786779+0000","last_change":"2024-09-10T21:12:23.798175+0000","last_active":"2024-09-10T21:12:25.786779+0000","last_peered":"2024-09-10T21:12:25.786779+0000","last_clean":"2024-09-10T21:12:25.786779+0000","last_became_active":"2024-09-10T21:12:23.797844+0000","last_became_peered":"2024-09-10T21:12:23.797844+0000","last_unstale":"2024-09-10T21:12:25.786779+0000","last_undegraded":"2024-09-10T21:12:25.786779+0000","last_fullsized":"2024-09-10T21:12:25.786779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:57.897857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427917+0000","last_change":"2024-09-10T21:12:24.820636+0000","last_active":"2024-09-10T21:12:26.427917+0000","last_peered":"2024-09-10T21:12:26.427917+0000","last_clean":"2024-09-10T21:12:26.427917+0000","last_became_active":"2024-09-10T21:12:24.820280+0000","last_became_peered":"2024-09-10T21:12:24.820280+0000","last_unstale":"2024-09-10T21:12:26.427917+0000","last_undegraded":"2024-09-10T21:12:26.427917+0000","last_fullsized":"2024-09-10T21:12:26.427917+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:06.763115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429325+0000","last_change":"2024-09-10T21:12:23.803893+0000","last_active":"2024-09-10T21:12:26.429325+0000","last_peered":"2024-09-10T21:12:26.429325+0000","last_clean":"2024-09-10T21:12:26.429325+0000","last_became_active":"2024-09-10T21:12:23.803599+0000","last_became_peered":"2024-09-10T21:12:23.803599+0000","last_unstale":"2024-09-10T21:12:26.429325+0000","last_undegraded":"2024-09-10T21:12:26.429325+0000","last_fullsized":"2024-09-10T21:12:26.429325+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:46.177434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427703+0000","last_change":"2024-09-10T21:12:24.813412+0000","last_active":"2024-09-10T21:12:26.427703+0000","last_peered":"2024-09-10T21:12:26.427703+0000","last_clean":"2024-09-10T21:12:26.427703+0000","last_became_active":"2024-09-10T21:12:24.802579+0000","last_became_peered":"2024-09-10T21:12:24.802579+0000","last_unstale":"2024-09-10T21:12:26.427703+0000","last_undegraded":"2024-09-10T21:12:26.427703+0000","last_fullsized":"2024-09-10T21:12:26.427703+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:51.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427719+0000","last_change":"2024-09-10T21:12:23.793275+0000","last_active":"2024-09-10T21:12:26.427719+0000","last_peered":"2024-09-10T21:12:26.427719+0000","last_clean":"2024-09-10T21:12:26.427719+0000","last_became_active":"2024-09-10T21:12:23.792867+0000","last_became_peered":"2024-09-10T21:12:23.792867+0000","last_unstale":"2024-09-10T21:12:26.427719+0000","last_undegraded":"2024-09-10T21:12:26.427719+0000","last_fullsized":"2024-09-10T21:12:26.427719+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:54.791716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427761+0000","last_change":"2024-09-10T21:12:24.806299+0000","last_active":"2024-09-10T21:12:26.427761+0000","last_peered":"2024-09-10T21:12:26.427761+0000","last_clean":"2024-09-10T21:12:26.427761+0000","last_became_active":"2024-09-10T21:12:24.806025+0000","last_became_peered":"2024-09-10T21:12:24.806025+0000","last_unstale":"2024-09-10T21:12:26.427761+0000","last_undegraded":"2024-09-10T21:12:26.427761+0000","last_fullsized":"2024-09-10T21:12:26.427761+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:19.306195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427757+0000","last_change":"2024-09-10T21:12:23.798078+0000","last_active":"2024-09-10T21:12:26.427757+0000","last_peered":"2024-09-10T21:12:26.427757+0000","last_clean":"2024-09-10T21:12:26.427757+0000","last_became_active":"2024-09-10T21:12:23.797773+0000","last_became_peered":"2024-09-10T21:12:23.797773+0000","last_unstale":"2024-09-10T21:12:26.427757+0000","last_undegraded":"2024-09-10T21:12:26.427757+0000","last_fullsized":"2024-09-10T21:12:26.427757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:34.190967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429612+0000","last_change":"2024-09-10T21:12:24.804022+0000","last_active":"2024-09-10T21:12:26.429612+0000","last_peered":"2024-09-10T21:12:26.429612+0000","last_clean":"2024-09-10T21:12:26.429612+0000","last_became_active":"2024-09-10T21:12:24.803828+0000","last_became_peered":"2024-09-10T21:12:24.803828+0000","last_unstale":"2024-09-10T21:12:26.429612+0000","last_undegraded":"2024-09-10T21:12:26.429612+0000","last_fullsized":"2024-09-10T21:12:26.429612+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:33.659623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428656+0000","last_change":"2024-09-10T21:12:23.808653+0000","last_active":"2024-09-10T21:12:26.428656+0000","last_peered":"2024-09-10T21:12:26.428656+0000","last_clean":"2024-09-10T21:12:26.428656+0000","last_became_active":"2024-09-10T21:12:23.808383+0000","last_became_peered":"2024-09-10T21:12:23.808383+0000","last_unstale":"2024-09-10T21:12:26.428656+0000","last_undegraded":"2024-09-10T21:12:26.428656+0000","last_fullsized":"2024-09-10T21:12:26.428656+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:55.562523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428596+0000","last_change":"2024-09-10T21:12:24.802802+0000","last_active":"2024-09-10T21:12:26.428596+0000","last_peered":"2024-09-10T21:12:26.428596+0000","last_clean":"2024-09-10T21:12:26.428596+0000","last_became_active":"2024-09-10T21:12:24.802449+0000","last_became_peered":"2024-09-10T21:12:24.802449+0000","last_unstale":"2024-09-10T21:12:26.428596+0000","last_undegraded":"2024-09-10T21:12:26.428596+0000","last_fullsized":"2024-09-10T21:12:26.428596+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:37.497080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428077+0000","last_change":"2024-09-10T21:12:23.807246+0000","last_active":"2024-09-10T21:12:26.428077+0000","last_peered":"2024-09-10T21:12:26.428077+0000","last_clean":"2024-09-10T21:12:26.428077+0000","last_became_active":"2024-09-10T21:12:23.806959+0000","last_became_peered":"2024-09-10T21:12:23.806959+0000","last_unstale":"2024-09-10T21:12:26.428077+0000","last_undegraded":"2024-09-10T21:12:26.428077+0000","last_fullsized":"2024-09-10T21:12:26.428077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:46.804078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786038+0000","last_change":"2024-09-10T21:12:24.822928+0000","last_active":"2024-09-10T21:12:25.786038+0000","last_peered":"2024-09-10T21:12:25.786038+0000","last_clean":"2024-09-10T21:12:25.786038+0000","last_became_active":"2024-09-10T21:12:24.822626+0000","last_became_peered":"2024-09-10T21:12:24.822626+0000","last_unstale":"2024-09-10T21:12:25.786038+0000","last_undegraded":"2024-09-10T21:12:25.786038+0000","last_fullsized":"2024-09-10T21:12:25.786038+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:06.763115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784521+0000","last_change":"2024-09-10T21:12:23.790411+0000","last_active":"2024-09-10T21:12:25.784521+0000","last_peered":"2024-09-10T21:12:25.784521+0000","last_clean":"2024-09-10T21:12:25.784521+0000","last_became_active":"2024-09-10T21:12:23.790109+0000","last_became_peered":"2024-09-10T21:12:23.790109+0000","last_unstale":"2024-09-10T21:12:25.784521+0000","last_undegraded":"2024-09-10T21:12:25.784521+0000","last_fullsized":"2024-09-10T21:12:25.784521+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:25.257787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428953+0000","last_change":"2024-09-10T21:12:24.808556+0000","last_active":"2024-09-10T21:12:26.428953+0000","last_peered":"2024-09-10T21:12:26.428953+0000","last_clean":"2024-09-10T21:12:26.428953+0000","last_became_active":"2024-09-10T21:12:24.808281+0000","last_became_peered":"2024-09-10T21:12:24.808281+0000","last_unstale":"2024-09-10T21:12:26.428953+0000","last_undegraded":"2024-09-10T21:12:26.428953+0000","last_fullsized":"2024-09-10T21:12:26.428953+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:04.882193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428411+0000","last_change":"2024-09-10T21:12:23.802815+0000","last_active":"2024-09-10T21:12:26.428411+0000","last_peered":"2024-09-10T21:12:26.428411+0000","last_clean":"2024-09-10T21:12:26.428411+0000","last_became_active":"2024-09-10T21:12:23.802540+0000","last_became_peered":"2024-09-10T21:12:23.802540+0000","last_unstale":"2024-09-10T21:12:26.428411+0000","last_undegraded":"2024-09-10T21:12:26.428411+0000","last_fullsized":"2024-09-10T21:12:26.428411+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:22.630049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428740+0000","last_change":"2024-09-10T21:12:24.822282+0000","last_active":"2024-09-10T21:12:26.428740+0000","last_peered":"2024-09-10T21:12:26.428740+0000","last_clean":"2024-09-10T21:12:26.428740+0000","last_became_active":"2024-09-10T21:12:24.821907+0000","last_became_peered":"2024-09-10T21:12:24.821907+0000","last_unstale":"2024-09-10T21:12:26.428740+0000","last_undegraded":"2024-09-10T21:12:26.428740+0000","last_fullsized":"2024-09-10T21:12:26.428740+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:45.236002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427043+0000","last_change":"2024-09-10T21:12:23.800329+0000","last_active":"2024-09-10T21:12:26.427043+0000","last_peered":"2024-09-10T21:12:26.427043+0000","last_clean":"2024-09-10T21:12:26.427043+0000","last_became_active":"2024-09-10T21:12:23.799441+0000","last_became_peered":"2024-09-10T21:12:23.799441+0000","last_unstale":"2024-09-10T21:12:26.427043+0000","last_undegraded":"2024-09-10T21:12:26.427043+0000","last_fullsized":"2024-09-10T21:12:26.427043+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:41.839052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427386+0000","last_change":"2024-09-10T21:12:24.799963+0000","last_active":"2024-09-10T21:12:26.427386+0000","last_peered":"2024-09-10T21:12:26.427386+0000","last_clean":"2024-09-10T21:12:26.427386+0000","last_became_active":"2024-09-10T21:12:24.799685+0000","last_became_peered":"2024-09-10T21:12:24.799685+0000","last_unstale":"2024-09-10T21:12:26.427386+0000","last_undegraded":"2024-09-10T21:12:26.427386+0000","last_fullsized":"2024-09-10T21:12:26.427386+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:14.399311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427958+0000","last_change":"2024-09-10T21:12:23.800966+0000","last_active":"2024-09-10T21:12:26.427958+0000","last_peered":"2024-09-10T21:12:26.427958+0000","last_clean":"2024-09-10T21:12:26.427958+0000","last_became_active":"2024-09-10T21:12:23.800647+0000","last_became_peered":"2024-09-10T21:12:23.800647+0000","last_unstale":"2024-09-10T21:12:26.427958+0000","last_undegraded":"2024-09-10T21:12:26.427958+0000","last_fullsized":"2024-09-10T21:12:26.427958+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:56.744606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427189+0000","last_change":"2024-09-10T21:12:24.799381+0000","last_active":"2024-09-10T21:12:26.427189+0000","last_peered":"2024-09-10T21:12:26.427189+0000","last_clean":"2024-09-10T21:12:26.427189+0000","last_became_active":"2024-09-10T21:12:24.798305+0000","last_became_peered":"2024-09-10T21:12:24.798305+0000","last_unstale":"2024-09-10T21:12:26.427189+0000","last_undegraded":"2024-09-10T21:12:26.427189+0000","last_fullsized":"2024-09-10T21:12:26.427189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:37.497080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784996+0000","last_change":"2024-09-10T21:12:23.806810+0000","last_active":"2024-09-10T21:12:25.784996+0000","last_peered":"2024-09-10T21:12:25.784996+0000","last_clean":"2024-09-10T21:12:25.784996+0000","last_became_active":"2024-09-10T21:12:23.806235+0000","last_became_peered":"2024-09-10T21:12:23.806235+0000","last_unstale":"2024-09-10T21:12:25.784996+0000","last_undegraded":"2024-09-10T21:12:25.784996+0000","last_fullsized":"2024-09-10T21:12:25.784996+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:11.902707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784660+0000","last_change":"2024-09-10T21:12:24.800682+0000","last_active":"2024-09-10T21:12:25.784660+0000","last_peered":"2024-09-10T21:12:25.784660+0000","last_clean":"2024-09-10T21:12:25.784660+0000","last_became_active":"2024-09-10T21:12:24.800333+0000","last_became_peered":"2024-09-10T21:12:24.800333+0000","last_unstale":"2024-09-10T21:12:25.784660+0000","last_undegraded":"2024-09-10T21:12:25.784660+0000","last_fullsized":"2024-09-10T21:12:25.784660+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:49.709340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":23,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427889+0000","last_change":"2024-09-10T21:12:23.796942+0000","last_active":"2024-09-10T21:12:26.427889+0000","last_peered":"2024-09-10T21:12:26.427889+0000","last_clean":"2024-09-10T21:12:26.427889+0000","last_became_active":"2024-09-10T21:12:23.796661+0000","last_became_peered":"2024-09-10T21:12:23.796661+0000","last_unstale":"2024-09-10T21:12:26.427889+0000","last_undegraded":"2024-09-10T21:12:26.427889+0000","last_fullsized":"2024-09-10T21:12:26.427889+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:05.892601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427615+0000","last_change":"2024-09-10T21:12:24.800375+0000","last_active":"2024-09-10T21:12:26.427615+0000","last_peered":"2024-09-10T21:12:26.427615+0000","last_clean":"2024-09-10T21:12:26.427615+0000","last_became_active":"2024-09-10T21:12:24.800087+0000","last_became_peered":"2024-09-10T21:12:24.800087+0000","last_unstale":"2024-09-10T21:12:26.427615+0000","last_undegraded":"2024-09-10T21:12:26.427615+0000","last_fullsized":"2024-09-10T21:12:26.427615+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:46.274444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786575+0000","last_change":"2024-09-10T21:12:23.797369+0000","last_active":"2024-09-10T21:12:25.786575+0000","last_peered":"2024-09-10T21:12:25.786575+0000","last_clean":"2024-09-10T21:12:25.786575+0000","last_became_active":"2024-09-10T21:12:23.796137+0000","last_became_peered":"2024-09-10T21:12:23.796137+0000","last_unstale":"2024-09-10T21:12:25.786575+0000","last_undegraded":"2024-09-10T21:12:25.786575+0000","last_fullsized":"2024-09-10T21:12:25.786575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:19.161900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427399+0000","last_change":"2024-09-10T21:12:24.800698+0000","last_active":"2024-09-10T21:12:26.427399+0000","last_peered":"2024-09-10T21:12:26.427399+0000","last_clean":"2024-09-10T21:12:26.427399+0000","last_became_active":"2024-09-10T21:12:24.799724+0000","last_became_peered":"2024-09-10T21:12:24.799724+0000","last_unstale":"2024-09-10T21:12:26.427399+0000","last_undegraded":"2024-09-10T21:12:26.427399+0000","last_fullsized":"2024-09-10T21:12:26.427399+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:44.428772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429684+0000","last_change":"2024-09-10T21:12:23.807367+0000","last_active":"2024-09-10T21:12:26.429684+0000","last_peered":"2024-09-10T21:12:26.429684+0000","last_clean":"2024-09-10T21:12:26.429684+0000","last_became_active":"2024-09-10T21:12:23.806960+0000","last_became_peered":"2024-09-10T21:12:23.806960+0000","last_unstale":"2024-09-10T21:12:26.429684+0000","last_undegraded":"2024-09-10T21:12:26.429684+0000","last_fullsized":"2024-09-10T21:12:26.429684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:21.476191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785216+0000","last_change":"2024-09-10T21:12:24.821547+0000","last_active":"2024-09-10T21:12:25.785216+0000","last_peered":"2024-09-10T21:12:25.785216+0000","last_clean":"2024-09-10T21:12:25.785216+0000","last_became_active":"2024-09-10T21:12:24.821285+0000","last_became_peered":"2024-09-10T21:12:24.821285+0000","last_unstale":"2024-09-10T21:12:25.785216+0000","last_undegraded":"2024-09-10T21:12:25.785216+0000","last_fullsized":"2024-09-10T21:12:25.785216+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:11.907167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429276+0000","last_change":"2024-09-10T21:12:23.810651+0000","last_active":"2024-09-10T21:12:26.429276+0000","last_peered":"2024-09-10T21:12:26.429276+0000","last_clean":"2024-09-10T21:12:26.429276+0000","last_became_active":"2024-09-10T21:12:23.809721+0000","last_became_peered":"2024-09-10T21:12:23.809721+0000","last_unstale":"2024-09-10T21:12:26.429276+0000","last_undegraded":"2024-09-10T21:12:26.429276+0000","last_fullsized":"2024-09-10T21:12:26.429276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:13.969390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429203+0000","last_change":"2024-09-10T21:12:24.802031+0000","last_active":"2024-09-10T21:12:26.429203+0000","last_peered":"2024-09-10T21:12:26.429203+0000","last_clean":"2024-09-10T21:12:26.429203+0000","last_became_active":"2024-09-10T21:12:24.801761+0000","last_became_peered":"2024-09-10T21:12:24.801761+0000","last_unstale":"2024-09-10T21:12:26.429203+0000","last_undegraded":"2024-09-10T21:12:26.429203+0000","last_fullsized":"2024-09-10T21:12:26.429203+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:30.695194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786779+0000","last_change":"2024-09-10T21:12:23.806377+0000","last_active":"2024-09-10T21:12:25.786779+0000","last_peered":"2024-09-10T21:12:25.786779+0000","last_clean":"2024-09-10T21:12:25.786779+0000","last_became_active":"2024-09-10T21:12:23.806032+0000","last_became_peered":"2024-09-10T21:12:23.806032+0000","last_unstale":"2024-09-10T21:12:25.786779+0000","last_undegraded":"2024-09-10T21:12:25.786779+0000","last_fullsized":"2024-09-10T21:12:25.786779+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:18.235286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428188+0000","last_change":"2024-09-10T21:12:24.823192+0000","last_active":"2024-09-10T21:12:26.428188+0000","last_peered":"2024-09-10T21:12:26.428188+0000","last_clean":"2024-09-10T21:12:26.428188+0000","last_became_active":"2024-09-10T21:12:24.822927+0000","last_became_peered":"2024-09-10T21:12:24.822927+0000","last_unstale":"2024-09-10T21:12:26.428188+0000","last_undegraded":"2024-09-10T21:12:26.428188+0000","last_fullsized":"2024-09-10T21:12:26.428188+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:16.729677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427725+0000","last_change":"2024-09-10T21:12:23.804977+0000","last_active":"2024-09-10T21:12:26.427725+0000","last_peered":"2024-09-10T21:12:26.427725+0000","last_clean":"2024-09-10T21:12:26.427725+0000","last_became_active":"2024-09-10T21:12:23.804671+0000","last_became_peered":"2024-09-10T21:12:23.804671+0000","last_unstale":"2024-09-10T21:12:26.427725+0000","last_undegraded":"2024-09-10T21:12:26.427725+0000","last_fullsized":"2024-09-10T21:12:26.427725+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:44.061600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786247+0000","last_change":"2024-09-10T21:12:24.804621+0000","last_active":"2024-09-10T21:12:25.786247+0000","last_peered":"2024-09-10T21:12:25.786247+0000","last_clean":"2024-09-10T21:12:25.786247+0000","last_became_active":"2024-09-10T21:12:24.804273+0000","last_became_peered":"2024-09-10T21:12:24.804273+0000","last_unstale":"2024-09-10T21:12:25.786247+0000","last_undegraded":"2024-09-10T21:12:25.786247+0000","last_fullsized":"2024-09-10T21:12:25.786247+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:14.393333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428072+0000","last_change":"2024-09-10T21:12:23.808671+0000","last_active":"2024-09-10T21:12:26.428072+0000","last_peered":"2024-09-10T21:12:26.428072+0000","last_clean":"2024-09-10T21:12:26.428072+0000","last_became_active":"2024-09-10T21:12:23.808137+0000","last_became_peered":"2024-09-10T21:12:23.808137+0000","last_unstale":"2024-09-10T21:12:26.428072+0000","last_undegraded":"2024-09-10T21:12:26.428072+0000","last_fullsized":"2024-09-10T21:12:26.428072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:02.838356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786010+0000","last_change":"2024-09-10T21:12:24.812457+0000","last_active":"2024-09-10T21:12:25.786010+0000","last_peered":"2024-09-10T21:12:25.786010+0000","last_clean":"2024-09-10T21:12:25.786010+0000","last_became_active":"2024-09-10T21:12:24.812056+0000","last_became_peered":"2024-09-10T21:12:24.812056+0000","last_unstale":"2024-09-10T21:12:25.786010+0000","last_undegraded":"2024-09-10T21:12:25.786010+0000","last_fullsized":"2024-09-10T21:12:25.786010+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:51.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427529+0000","last_change":"2024-09-10T21:12:23.808302+0000","last_active":"2024-09-10T21:12:26.427529+0000","last_peered":"2024-09-10T21:12:26.427529+0000","last_clean":"2024-09-10T21:12:26.427529+0000","last_became_active":"2024-09-10T21:12:23.807962+0000","last_became_peered":"2024-09-10T21:12:23.807962+0000","last_unstale":"2024-09-10T21:12:26.427529+0000","last_undegraded":"2024-09-10T21:12:26.427529+0000","last_fullsized":"2024-09-10T21:12:26.427529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:39.325026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786383+0000","last_change":"2024-09-10T21:12:24.819374+0000","last_active":"2024-09-10T21:12:25.786383+0000","last_peered":"2024-09-10T21:12:25.786383+0000","last_clean":"2024-09-10T21:12:25.786383+0000","last_became_active":"2024-09-10T21:12:24.819016+0000","last_became_peered":"2024-09-10T21:12:24.819016+0000","last_unstale":"2024-09-10T21:12:25.786383+0000","last_undegraded":"2024-09-10T21:12:25.786383+0000","last_fullsized":"2024-09-10T21:12:25.786383+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:16.729677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427497+0000","last_change":"2024-09-10T21:12:23.801081+0000","last_active":"2024-09-10T21:12:26.427497+0000","last_peered":"2024-09-10T21:12:26.427497+0000","last_clean":"2024-09-10T21:12:26.427497+0000","last_became_active":"2024-09-10T21:12:23.800792+0000","last_became_peered":"2024-09-10T21:12:23.800792+0000","last_unstale":"2024-09-10T21:12:26.427497+0000","last_undegraded":"2024-09-10T21:12:26.427497+0000","last_fullsized":"2024-09-10T21:12:26.427497+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:24.670926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429005+0000","last_change":"2024-09-10T21:12:24.805825+0000","last_active":"2024-09-10T21:12:26.429005+0000","last_peered":"2024-09-10T21:12:26.429005+0000","last_clean":"2024-09-10T21:12:26.429005+0000","last_became_active":"2024-09-10T21:12:24.805527+0000","last_became_peered":"2024-09-10T21:12:24.805527+0000","last_unstale":"2024-09-10T21:12:26.429005+0000","last_undegraded":"2024-09-10T21:12:26.429005+0000","last_fullsized":"2024-09-10T21:12:26.429005+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:17.948171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786931+0000","last_change":"2024-09-10T21:12:23.800595+0000","last_active":"2024-09-10T21:12:25.786931+0000","last_peered":"2024-09-10T21:12:25.786931+0000","last_clean":"2024-09-10T21:12:25.786931+0000","last_became_active":"2024-09-10T21:12:23.800279+0000","last_became_peered":"2024-09-10T21:12:23.800279+0000","last_unstale":"2024-09-10T21:12:25.786931+0000","last_undegraded":"2024-09-10T21:12:25.786931+0000","last_fullsized":"2024-09-10T21:12:25.786931+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:25.846794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428151+0000","last_change":"2024-09-10T21:12:24.805436+0000","last_active":"2024-09-10T21:12:26.428151+0000","last_peered":"2024-09-10T21:12:26.428151+0000","last_clean":"2024-09-10T21:12:26.428151+0000","last_became_active":"2024-09-10T21:12:24.805106+0000","last_became_peered":"2024-09-10T21:12:24.805106+0000","last_unstale":"2024-09-10T21:12:26.428151+0000","last_undegraded":"2024-09-10T21:12:26.428151+0000","last_fullsized":"2024-09-10T21:12:26.428151+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:32.446130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428690+0000","last_change":"2024-09-10T21:12:23.810567+0000","last_active":"2024-09-10T21:12:26.428690+0000","last_peered":"2024-09-10T21:12:26.428690+0000","last_clean":"2024-09-10T21:12:26.428690+0000","last_became_active":"2024-09-10T21:12:23.810246+0000","last_became_peered":"2024-09-10T21:12:23.810246+0000","last_unstale":"2024-09-10T21:12:26.428690+0000","last_undegraded":"2024-09-10T21:12:26.428690+0000","last_fullsized":"2024-09-10T21:12:26.428690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:09.107768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428041+0000","last_change":"2024-09-10T21:12:24.808843+0000","last_active":"2024-09-10T21:12:26.428041+0000","last_peered":"2024-09-10T21:12:26.428041+0000","last_clean":"2024-09-10T21:12:26.428041+0000","last_became_active":"2024-09-10T21:12:24.808570+0000","last_became_peered":"2024-09-10T21:12:24.808570+0000","last_unstale":"2024-09-10T21:12:26.428041+0000","last_undegraded":"2024-09-10T21:12:26.428041+0000","last_fullsized":"2024-09-10T21:12:26.428041+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:50.496957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427295+0000","last_change":"2024-09-10T21:12:23.807419+0000","last_active":"2024-09-10T21:12:26.427295+0000","last_peered":"2024-09-10T21:12:26.427295+0000","last_clean":"2024-09-10T21:12:26.427295+0000","last_became_active":"2024-09-10T21:12:23.806848+0000","last_became_peered":"2024-09-10T21:12:23.806848+0000","last_unstale":"2024-09-10T21:12:26.427295+0000","last_undegraded":"2024-09-10T21:12:26.427295+0000","last_fullsized":"2024-09-10T21:12:26.427295+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:46.663095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428169+0000","last_change":"2024-09-10T21:12:24.818146+0000","last_active":"2024-09-10T21:12:26.428169+0000","last_peered":"2024-09-10T21:12:26.428169+0000","last_clean":"2024-09-10T21:12:26.428169+0000","last_became_active":"2024-09-10T21:12:24.817782+0000","last_became_peered":"2024-09-10T21:12:24.817782+0000","last_unstale":"2024-09-10T21:12:26.428169+0000","last_undegraded":"2024-09-10T21:12:26.428169+0000","last_fullsized":"2024-09-10T21:12:26.428169+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:41.825084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.428688+0000","last_change":"2024-09-10T21:12:23.795791+0000","last_active":"2024-09-10T21:12:26.428688+0000","last_peered":"2024-09-10T21:12:26.428688+0000","last_clean":"2024-09-10T21:12:26.428688+0000","last_became_active":"2024-09-10T21:12:23.795074+0000","last_became_peered":"2024-09-10T21:12:23.795074+0000","last_unstale":"2024-09-10T21:12:26.428688+0000","last_undegraded":"2024-09-10T21:12:26.428688+0000","last_fullsized":"2024-09-10T21:12:26.428688+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:36.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785827+0000","last_change":"2024-09-10T21:12:24.823887+0000","last_active":"2024-09-10T21:12:25.785827+0000","last_peered":"2024-09-10T21:12:25.785827+0000","last_clean":"2024-09-10T21:12:25.785827+0000","last_became_active":"2024-09-10T21:12:24.823427+0000","last_became_peered":"2024-09-10T21:12:24.823427+0000","last_unstale":"2024-09-10T21:12:25.785827+0000","last_undegraded":"2024-09-10T21:12:25.785827+0000","last_fullsized":"2024-09-10T21:12:25.785827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:04.280622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429384+0000","last_change":"2024-09-10T21:12:23.797756+0000","last_active":"2024-09-10T21:12:26.429384+0000","last_peered":"2024-09-10T21:12:26.429384+0000","last_clean":"2024-09-10T21:12:26.429384+0000","last_became_active":"2024-09-10T21:12:23.797410+0000","last_became_peered":"2024-09-10T21:12:23.797410+0000","last_unstale":"2024-09-10T21:12:26.429384+0000","last_undegraded":"2024-09-10T21:12:26.429384+0000","last_fullsized":"2024-09-10T21:12:26.429384+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:58.733833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427546+0000","last_change":"2024-09-10T21:12:24.816020+0000","last_active":"2024-09-10T21:12:26.427546+0000","last_peered":"2024-09-10T21:12:26.427546+0000","last_clean":"2024-09-10T21:12:26.427546+0000","last_became_active":"2024-09-10T21:12:24.815116+0000","last_became_peered":"2024-09-10T21:12:24.815116+0000","last_unstale":"2024-09-10T21:12:26.427546+0000","last_undegraded":"2024-09-10T21:12:26.427546+0000","last_fullsized":"2024-09-10T21:12:26.427546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:19.498294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.785094+0000","last_change":"2024-09-10T21:12:23.800295+0000","last_active":"2024-09-10T21:12:25.785094+0000","last_peered":"2024-09-10T21:12:25.785094+0000","last_clean":"2024-09-10T21:12:25.785094+0000","last_became_active":"2024-09-10T21:12:23.799896+0000","last_became_peered":"2024-09-10T21:12:23.799896+0000","last_unstale":"2024-09-10T21:12:25.785094+0000","last_undegraded":"2024-09-10T21:12:25.785094+0000","last_fullsized":"2024-09-10T21:12:25.785094+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:25.846794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.784955+0000","last_change":"2024-09-10T21:12:24.798401+0000","last_active":"2024-09-10T21:12:25.784955+0000","last_peered":"2024-09-10T21:12:25.784955+0000","last_clean":"2024-09-10T21:12:25.784955+0000","last_became_active":"2024-09-10T21:12:24.798177+0000","last_became_peered":"2024-09-10T21:12:24.798177+0000","last_unstale":"2024-09-10T21:12:25.784955+0000","last_undegraded":"2024-09-10T21:12:25.784955+0000","last_fullsized":"2024-09-10T21:12:25.784955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:21.261528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427060+0000","last_change":"2024-09-10T21:12:23.799895+0000","last_active":"2024-09-10T21:12:26.427060+0000","last_peered":"2024-09-10T21:12:26.427060+0000","last_clean":"2024-09-10T21:12:26.427060+0000","last_became_active":"2024-09-10T21:12:23.799596+0000","last_became_peered":"2024-09-10T21:12:23.799596+0000","last_unstale":"2024-09-10T21:12:26.427060+0000","last_undegraded":"2024-09-10T21:12:26.427060+0000","last_fullsized":"2024-09-10T21:12:26.427060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:21.476191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:25.786669+0000","last_change":"2024-09-10T21:12:24.823727+0000","last_active":"2024-09-10T21:12:25.786669+0000","last_peered":"2024-09-10T21:12:25.786669+0000","last_clean":"2024-09-10T21:12:25.786669+0000","last_became_active":"2024-09-10T21:12:24.823434+0000","last_became_peered":"2024-09-10T21:12:24.823434+0000","last_unstale":"2024-09-10T21:12:25.786669+0000","last_undegraded":"2024-09-10T21:12:25.786669+0000","last_fullsized":"2024-09-10T21:12:25.786669+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:23.772267+0000","last_clean_scrub_stamp":"2024-09-10T21:12:23.772267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:09.337686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.429146+0000","last_change":"2024-09-10T21:12:23.803236+0000","last_active":"2024-09-10T21:12:26.429146+0000","last_peered":"2024-09-10T21:12:26.429146+0000","last_clean":"2024-09-10T21:12:26.429146+0000","last_became_active":"2024-09-10T21:12:23.802939+0000","last_became_peered":"2024-09-10T21:12:23.802939+0000","last_unstale":"2024-09-10T21:12:26.429146+0000","last_undegraded":"2024-09-10T21:12:26.429146+0000","last_fullsized":"2024-09-10T21:12:26.429146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:22.764860+0000","last_clean_scrub_stamp":"2024-09-10T21:12:22.764860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:35.914109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T21:12:26.427619+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:12:26.427619+0000","last_peered":"2024-09-10T21:12:26.427619+0000","last_clean":"2024-09-10T21:12:26.427619+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:12:26.427619+0000","last_undegraded":"2024-09-10T21:12:26.427619+0000","last_fullsized":"2024-09-10T21:12:26.427619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1130496,"data_stored":806790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509469,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":218104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":544768,"data_stored":218012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":216490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":217072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":216434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":806838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:12:33.973 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:12:33.973 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:12:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.164+0000 7f8079c00640 1 -- 172.21.15.16:0/3880295445 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80740cfe30 msgr2=0x7f80740ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.164+0000 7f8079c00640 1 --2- 172.21.15.16:0/3880295445 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80740cfe30 0x7f80740ce590 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8064009840 tx=0x7f806401f010 comp rx=0 tx=0).stop 2024-09-10T21:12:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.164+0000 7f8079c00640 1 -- 172.21.15.16:0/3880295445 shutdown_connections 2024-09-10T21:12:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.164+0000 7f8079c00640 1 --2- 172.21.15.16:0/3880295445 >> v2:172.21.15.52:3300/0 conn(0x7f80740d0210 0x7f80740cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.164+0000 7f8079c00640 1 --2- 172.21.15.16:0/3880295445 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80740cfe30 0x7f80740ce590 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.164+0000 7f8079c00640 1 -- 172.21.15.16:0/3880295445 >> 172.21.15.16:0/3880295445 conn(0x7f80740d1df0 msgr2=0x7f80740d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.164+0000 7f8079c00640 1 -- 172.21.15.16:0/3880295445 shutdown_connections 2024-09-10T21:12:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.164+0000 7f8079c00640 1 -- 172.21.15.16:0/3880295445 wait complete. 2024-09-10T21:12:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8079c00640 1 Processor -- start 2024-09-10T21:12:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8079c00640 1 -- start start 2024-09-10T21:12:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8079c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80740ce040 0x7f80741f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8079c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80740cfe30 0x7f807415b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8079c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80740d0210 0x7f80740b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8079c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80740b12e0 con 0x7f80740d0210 2024-09-10T21:12:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8079c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f80740d2b20 con 0x7f80740cfe30 2024-09-10T21:12:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8079c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f80740b1460 con 0x7f80740ce040 2024-09-10T21:12:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8073400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80740d0210 0x7f80740b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8073400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80740d0210 0x7f80740b6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53650/0 (socket says 172.21.15.16:53650) 2024-09-10T21:12:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8073400640 1 -- 172.21.15.16:0/853652457 learned_addr learned my addr 172.21.15.16:0/853652457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f806be00640 1 --2- 172.21.15.16:0/853652457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80740cfe30 0x7f807415b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8072a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80740ce040 0x7f80741f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8073400640 1 -- 172.21.15.16:0/853652457 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80740ce040 msgr2=0x7f80741f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8073400640 1 --2- 172.21.15.16:0/853652457 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80740ce040 0x7f80741f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8073400640 1 -- 172.21.15.16:0/853652457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80740cfe30 msgr2=0x7f807415b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8073400640 1 --2- 172.21.15.16:0/853652457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80740cfe30 0x7f807415b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8073400640 1 -- 172.21.15.16:0/853652457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807415bd50 con 0x7f80740d0210 2024-09-10T21:12:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8073400640 1 --2- 172.21.15.16:0/853652457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80740d0210 0x7f80740b6ca0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f806c00c790 tx=0x7f806c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f806aa00640 1 -- 172.21.15.16:0/853652457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c003b80 con 0x7f80740d0210 2024-09-10T21:12:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f806aa00640 1 -- 172.21.15.16:0/853652457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f806c003d40 con 0x7f80740d0210 2024-09-10T21:12:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f806aa00640 1 -- 172.21.15.16:0/853652457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c00aab0 con 0x7f80740d0210 2024-09-10T21:12:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8079c00640 1 -- 172.21.15.16:0/853652457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80740c7090 con 0x7f80740d0210 2024-09-10T21:12:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8079c00640 1 -- 172.21.15.16:0/853652457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80740c7650 con 0x7f80740d0210 2024-09-10T21:12:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.168+0000 7f8079c00640 1 -- 172.21.15.16:0/853652457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80740cf2b0 con 0x7f80740d0210 2024-09-10T21:12:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.172+0000 7f806aa00640 1 -- 172.21.15.16:0/853652457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f806c00ac50 con 0x7f80740d0210 2024-09-10T21:12:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.172+0000 7f806aa00640 1 --2- 172.21.15.16:0/853652457 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f80500725e0 0x7f8050074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.172+0000 7f806aa00640 1 -- 172.21.15.16:0/853652457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f806c092f80 con 0x7f80740d0210 2024-09-10T21:12:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.172+0000 7f8072a00640 1 --2- 172.21.15.16:0/853652457 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f80500725e0 0x7f8050074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.172+0000 7f8072a00640 1 --2- 172.21.15.16:0/853652457 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f80500725e0 0x7f8050074aa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f80640b2000 tx=0x7f806401d850 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.176+0000 7f806aa00640 1 -- 172.21.15.16:0/853652457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f807415bd50 con 0x7f80740d0210 2024-09-10T21:12:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.412+0000 7f8079c00640 1 -- 172.21.15.16:0/853652457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f80740b3d30 con 0x7f80740d0210 2024-09-10T21:12:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.416+0000 7f806aa00640 1 -- 172.21.15.16:0/853652457 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7f806c05e000 con 0x7f80740d0210 2024-09-10T21:12:34.421 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-10T21:12:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA== 2024-09-10T21:12:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-10T21:12:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-10T21:12:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:12:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-10T21:12:34.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.416+0000 7f8079c00640 1 -- 172.21.15.16:0/853652457 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f80500725e0 msgr2=0x7f8050074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.416+0000 7f8079c00640 1 --2- 172.21.15.16:0/853652457 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f80500725e0 0x7f8050074aa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f80640b2000 tx=0x7f806401d850 comp rx=0 tx=0).stop 2024-09-10T21:12:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.416+0000 7f8079c00640 1 -- 172.21.15.16:0/853652457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80740d0210 msgr2=0x7f80740b6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.416+0000 7f8079c00640 1 --2- 172.21.15.16:0/853652457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80740d0210 0x7f80740b6ca0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f806c00c790 tx=0x7f806c00cc50 comp rx=0 tx=0).stop 2024-09-10T21:12:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.416+0000 7f8079c00640 1 -- 172.21.15.16:0/853652457 shutdown_connections 2024-09-10T21:12:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.416+0000 7f8079c00640 1 --2- 172.21.15.16:0/853652457 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f80500725e0 0x7f8050074aa0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.416+0000 7f8079c00640 1 --2- 172.21.15.16:0/853652457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80740d0210 0x7f80740b6ca0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.416+0000 7f8079c00640 1 --2- 172.21.15.16:0/853652457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80740cfe30 0x7f807415b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.416+0000 7f8079c00640 1 --2- 172.21.15.16:0/853652457 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80740ce040 0x7f80741f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.416+0000 7f8079c00640 1 -- 172.21.15.16:0/853652457 >> 172.21.15.16:0/853652457 conn(0x7f80740d1df0 msgr2=0x7f80740aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.416+0000 7f8079c00640 1 -- 172.21.15.16:0/853652457 shutdown_connections 2024-09-10T21:12:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.416+0000 7f8079c00640 1 -- 172.21.15.16:0/853652457 wait complete. 2024-09-10T21:12:34.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.584+0000 7f181ba00640 1 -- 172.21.15.16:0/2636133465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18140d06b0 msgr2=0x7f18140700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.584+0000 7f181ba00640 1 --2- 172.21.15.16:0/2636133465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18140d06b0 0x7f18140700a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f181406e1a0 tx=0x7f17fc01d260 comp rx=0 tx=0).stop 2024-09-10T21:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.584+0000 7f181ba00640 1 -- 172.21.15.16:0/2636133465 shutdown_connections 2024-09-10T21:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.584+0000 7f181ba00640 1 --2- 172.21.15.16:0/2636133465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18140d06b0 0x7f18140700a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.584+0000 7f181ba00640 1 --2- 172.21.15.16:0/2636133465 >> v2:172.21.15.52:3300/0 conn(0x7f181415a270 0x7f18140ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.584+0000 7f181ba00640 1 -- 172.21.15.16:0/2636133465 >> 172.21.15.16:0/2636133465 conn(0x7f18140cfb20 msgr2=0x7f18140cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.584+0000 7f181ba00640 1 -- 172.21.15.16:0/2636133465 shutdown_connections 2024-09-10T21:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.584+0000 7f181ba00640 1 -- 172.21.15.16:0/2636133465 wait complete. 2024-09-10T21:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.584+0000 7f181ba00640 1 Processor -- start 2024-09-10T21:12:34.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.584+0000 7f181ba00640 1 -- start start 2024-09-10T21:12:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f181ba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18140705e0 0x7f181415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f181ba00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18140d06b0 0x7f18140b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f181ba00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f181415a270 0x7f18140b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f181ba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1814072b30 con 0x7f18140705e0 2024-09-10T21:12:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f181ba00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f18140729b0 con 0x7f181415a270 2024-09-10T21:12:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f181ba00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f1814072cb0 con 0x7f18140d06b0 2024-09-10T21:12:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1819000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18140705e0 0x7f181415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1819000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18140705e0 0x7f181415dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53660/0 (socket says 172.21.15.16:53660) 2024-09-10T21:12:34.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1819000640 1 -- 172.21.15.16:0/3449526016 learned_addr learned my addr 172.21.15.16:0/3449526016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:34.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1819a00640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f181415a270 0x7f18140b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f180be00640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18140d06b0 0x7f18140b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1819000640 1 -- 172.21.15.16:0/3449526016 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18140d06b0 msgr2=0x7f18140b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1819000640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18140d06b0 0x7f18140b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1819000640 1 -- 172.21.15.16:0/3449526016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f181415a270 msgr2=0x7f18140b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1819000640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f181415a270 0x7f18140b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1819000640 1 -- 172.21.15.16:0/3449526016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18140b3510 con 0x7f18140705e0 2024-09-10T21:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1819000640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18140705e0 0x7f181415dd90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f18040058d0 tx=0x7f180400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1811600640 1 -- 172.21.15.16:0/3449526016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804017070 con 0x7f18140705e0 2024-09-10T21:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1811600640 1 -- 172.21.15.16:0/3449526016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1804009d20 con 0x7f18140705e0 2024-09-10T21:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f181ba00640 1 -- 172.21.15.16:0/3449526016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18140b37f0 con 0x7f18140705e0 2024-09-10T21:12:34.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1811600640 1 -- 172.21.15.16:0/3449526016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18040106c0 con 0x7f18140705e0 2024-09-10T21:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f181ba00640 1 -- 172.21.15.16:0/3449526016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18140b3db0 con 0x7f18140705e0 2024-09-10T21:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f181ba00640 1 -- 172.21.15.16:0/3449526016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18140700a0 con 0x7f18140705e0 2024-09-10T21:12:34.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.588+0000 7f1811600640 1 -- 172.21.15.16:0/3449526016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1804010860 con 0x7f18140705e0 2024-09-10T21:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.592+0000 7f1811600640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f17f00725e0 0x7f17f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.592+0000 7f1811600640 1 -- 172.21.15.16:0/3449526016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f1804093700 con 0x7f18140705e0 2024-09-10T21:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.592+0000 7f180be00640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f17f00725e0 0x7f17f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.592+0000 7f180be00640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f17f00725e0 0x7f17f0074aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0af000 tx=0x7f17fc0a7040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.596+0000 7f1811600640 1 -- 172.21.15.16:0/3449526016 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f180405e6f0 con 0x7f18140705e0 2024-09-10T21:12:34.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.804+0000 7f181ba00640 1 -- 172.21.15.16:0/3449526016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18140b5aa0 con 0x7f18140705e0 2024-09-10T21:12:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.804+0000 7f1811600640 1 -- 172.21.15.16:0/3449526016 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5235 (secure 0 0 0) 0x7f1804063120 con 0x7f18140705e0 2024-09-10T21:12:34.811 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:34.811 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-10T21:12:27:803754+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3694613625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3694613625},{"type":"v1","addr":"172.21.15.16:6836","nonce":3694613625}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1264024538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1264024538},{"type":"v1","addr":"172.21.15.52:6835","nonce":1264024538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":134768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2899191048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2899191048},{"type":"v1","addr":"172.21.15.16:6837","nonce":2899191048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:12:24.201810+0000","modified":"2024-09-10T21:12:27.207587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134771":{"gid":134771,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1834750661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1834750661},{"type":"v1","addr":"172.21.15.52:6834","nonce":1834750661}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134771,"qdb_cluster":[134771]},"id":3}]} 2024-09-10T21:12:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-10T21:12:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.804+0000 7f181ba00640 1 -- 172.21.15.16:0/3449526016 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f17f00725e0 msgr2=0x7f17f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.804+0000 7f181ba00640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f17f00725e0 0x7f17f0074aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0af000 tx=0x7f17fc0a7040 comp rx=0 tx=0).stop 2024-09-10T21:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.808+0000 7f181ba00640 1 -- 172.21.15.16:0/3449526016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18140705e0 msgr2=0x7f181415dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.808+0000 7f181ba00640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18140705e0 0x7f181415dd90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f18040058d0 tx=0x7f180400dde0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.808+0000 7f181ba00640 1 -- 172.21.15.16:0/3449526016 shutdown_connections 2024-09-10T21:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.808+0000 7f181ba00640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f17f00725e0 0x7f17f0074aa0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.808+0000 7f181ba00640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f181415a270 0x7f18140b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.808+0000 7f181ba00640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18140d06b0 0x7f18140b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.808+0000 7f181ba00640 1 --2- 172.21.15.16:0/3449526016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18140705e0 0x7f181415dd90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.808+0000 7f181ba00640 1 -- 172.21.15.16:0/3449526016 >> 172.21.15.16:0/3449526016 conn(0x7f18140cfb20 msgr2=0x7f18140cde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.808+0000 7f181ba00640 1 -- 172.21.15.16:0/3449526016 shutdown_connections 2024-09-10T21:12:34.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:34.808+0000 7f181ba00640 1 -- 172.21.15.16:0/3449526016 wait complete. 2024-09-10T21:12:35.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:12:24.201810+0000', 'modified': '2024-09-10T21:12:27.207587+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 134771}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_134771': {'gid': 134771, 'name': 'd', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.52:6834/1834750661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6832', 'nonce': 1834750661}, {'type': 'v1', 'addr': '172.21.15.52:6834', 'nonce': 1834750661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 134771, 'qdb_cluster': [134771]} 2024-09-10T21:12:35.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:12:35.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T21:12:35.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.384+0000 7fbde5000640 1 -- 172.21.15.16:0/2257866766 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbde00aed30 msgr2=0x7fbde00af1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.384+0000 7fbde5000640 1 --2- 172.21.15.16:0/2257866766 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbde00aed30 0x7fbde00af1c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009c80 tx=0x7fbdd801cce0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.384+0000 7fbde5000640 1 -- 172.21.15.16:0/2257866766 shutdown_connections 2024-09-10T21:12:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.384+0000 7fbde5000640 1 --2- 172.21.15.16:0/2257866766 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbde00aed30 0x7fbde00af1c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.384+0000 7fbde5000640 1 --2- 172.21.15.16:0/2257866766 >> v2:172.21.15.52:3300/0 conn(0x7fbde00cd130 0x7fbde00d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.384+0000 7fbde5000640 1 -- 172.21.15.16:0/2257866766 >> 172.21.15.16:0/2257866766 conn(0x7fbde00d1d80 msgr2=0x7fbde00d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.384+0000 7fbde5000640 1 -- 172.21.15.16:0/2257866766 shutdown_connections 2024-09-10T21:12:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.384+0000 7fbde5000640 1 -- 172.21.15.16:0/2257866766 wait complete. 2024-09-10T21:12:35.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.384+0000 7fbde5000640 1 Processor -- start 2024-09-10T21:12:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.384+0000 7fbde5000640 1 -- start start 2024-09-10T21:12:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbde5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbde00aed30 0x7fbde01f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbde5000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbde00cd130 0x7fbde00cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbde5000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbde015f350 0x7fbde0148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbde5000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0168e60 con 0x7fbde00aed30 2024-09-10T21:12:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbde5000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0168ce0 con 0x7fbde00cd130 2024-09-10T21:12:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbde5000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fbde0168fe0 con 0x7fbde015f350 2024-09-10T21:12:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbddde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbde00aed30 0x7fbde01f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbddde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbde00aed30 0x7fbde01f8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53666/0 (socket says 172.21.15.16:53666) 2024-09-10T21:12:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbddde00640 1 -- 172.21.15.16:0/4048498785 learned_addr learned my addr 172.21.15.16:0/4048498785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbdde800640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbde015f350 0x7fbde0148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbdd7e00640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbde00cd130 0x7fbde00cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbddde00640 1 -- 172.21.15.16:0/4048498785 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbde015f350 msgr2=0x7fbde0148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbddde00640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbde015f350 0x7fbde0148cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbddde00640 1 -- 172.21.15.16:0/4048498785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbde00cd130 msgr2=0x7fbde00cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbddde00640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbde00cd130 0x7fbde00cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbddde00640 1 -- 172.21.15.16:0/4048498785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde01f9de0 con 0x7fbde00aed30 2024-09-10T21:12:35.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbddde00640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbde00aed30 0x7fbde01f8f20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000c3c0 tx=0x7fbdd000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:35.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbdd6000640 1 -- 172.21.15.16:0/4048498785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0017070 con 0x7fbde00aed30 2024-09-10T21:12:35.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbdd6000640 1 -- 172.21.15.16:0/4048498785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdd00053c0 con 0x7fbde00aed30 2024-09-10T21:12:35.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbdd6000640 1 -- 172.21.15.16:0/4048498785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0003860 con 0x7fbde00aed30 2024-09-10T21:12:35.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbde5000640 1 -- 172.21.15.16:0/4048498785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde01f9b00 con 0x7fbde00aed30 2024-09-10T21:12:35.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.388+0000 7fbde5000640 1 -- 172.21.15.16:0/4048498785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde00b61e0 con 0x7fbde00aed30 2024-09-10T21:12:35.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdd6000640 1 -- 172.21.15.16:0/4048498785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbdd0003a00 con 0x7fbde00aed30 2024-09-10T21:12:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdd6000640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbdbc0725e0 0x7fbdbc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdd6000640 1 -- 172.21.15.16:0/4048498785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fbdd0092830 con 0x7fbde00aed30 2024-09-10T21:12:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbde850b000 1 Processor -- start 2024-09-10T21:12:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdd7e00640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbdbc0725e0 0x7fbdbc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbde850b000 1 -- start start 2024-09-10T21:12:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdd7e00640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbdbc0725e0 0x7fbdbc074aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbde00c6070 tx=0x7fbdc80091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbde850b000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f8f00 0x55ba372f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbde850b000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372f9820 0x55ba372aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbde850b000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 0x55ba372af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbde850b000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ba371c93f0 con 0x55ba372af230 2024-09-10T21:12:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbde850b000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ba37106b20 con 0x55ba372f9820 2024-09-10T21:12:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbde850b000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55ba370f5f50 con 0x55ba372f8f00 2024-09-10T21:12:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdde800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 0x55ba372af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdde800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 0x55ba372af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53674/0 (socket says 172.21.15.16:53674) 2024-09-10T21:12:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdde800640 1 -- 172.21.15.16:0/730015375 learned_addr learned my addr 172.21.15.16:0/730015375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbddde00640 1 --2- 172.21.15.16:0/730015375 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f8f00 0x55ba372f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdd7e00640 1 --2- 172.21.15.16:0/730015375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372f9820 0x55ba372aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdde800640 1 -- 172.21.15.16:0/730015375 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f8f00 msgr2=0x55ba372f92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdde800640 1 --2- 172.21.15.16:0/730015375 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f8f00 0x55ba372f92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdde800640 1 -- 172.21.15.16:0/730015375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372f9820 msgr2=0x55ba372aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdde800640 1 --2- 172.21.15.16:0/730015375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372f9820 0x55ba372aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdde800640 1 -- 172.21.15.16:0/730015375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba370bd590 con 0x55ba372af230 2024-09-10T21:12:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.392+0000 7fbdde800640 1 --2- 172.21.15.16:0/730015375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 0x55ba372af610 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80b0040 tx=0x7fbdd8019a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d710f46dd8125f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbdcea00640 1 -- 172.21.15.16:0/730015375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80b2040 con 0x55ba372af230 2024-09-10T21:12:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbdcea00640 1 -- 172.21.15.16:0/730015375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdd80bc070 con 0x55ba372af230 2024-09-10T21:12:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbdcea00640 1 -- 172.21.15.16:0/730015375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8004cb0 con 0x55ba372af230 2024-09-10T21:12:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 -- 172.21.15.16:0/730015375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 msgr2=0x55ba372af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 --2- 172.21.15.16:0/730015375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 0x55ba372af610 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80b0040 tx=0x7fbdd8019a80 comp rx=0 tx=0).stop 2024-09-10T21:12:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 -- 172.21.15.16:0/730015375 shutdown_connections 2024-09-10T21:12:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 --2- 172.21.15.16:0/730015375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 0x55ba372af610 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 --2- 172.21.15.16:0/730015375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372f9820 0x55ba372aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 --2- 172.21.15.16:0/730015375 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f8f00 0x55ba372f92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 -- 172.21.15.16:0/730015375 >> 172.21.15.16:0/730015375 conn(0x55ba37192130 msgr2=0x55ba372b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 -- 172.21.15.16:0/730015375 shutdown_connections 2024-09-10T21:12:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 -- 172.21.15.16:0/730015375 wait complete. 2024-09-10T21:12:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 -- start start 2024-09-10T21:12:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 Processor -- start 2024-09-10T21:12:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372af230 0x55ba372e3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372f8f00 0x55ba372e3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f9820 0x55ba372e42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ba370905d0 con 0x55ba372f8f00 2024-09-10T21:12:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ba370f9d60 con 0x55ba372af230 2024-09-10T21:12:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55ba3709c500 con 0x55ba372f9820 2024-09-10T21:12:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbde850b000 10 client.0 _ll_register_callbacks cb 0x55ba370a0810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbdd7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372f8f00 0x55ba372e3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbddde00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372af230 0x55ba372e3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbdd7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372f8f00 0x55ba372e3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53680/0 (socket says 172.21.15.16:53680) 2024-09-10T21:12:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbdd7e00640 1 -- 172.21.15.16:0/1838624818 learned_addr learned my addr 172.21.15.16:0/1838624818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbdde800640 1 --2- 172.21.15.16:0/1838624818 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f9820 0x55ba372e42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbdd7e00640 1 -- 172.21.15.16:0/1838624818 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f9820 msgr2=0x55ba372e42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbdd7e00640 1 --2- 172.21.15.16:0/1838624818 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f9820 0x55ba372e42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbdd7e00640 1 -- 172.21.15.16:0/1838624818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372af230 msgr2=0x55ba372e3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbdd7e00640 1 --2- 172.21.15.16:0/1838624818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372af230 0x55ba372e3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.396+0000 7fbdd7e00640 1 -- 172.21.15.16:0/1838624818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba37094170 con 0x55ba372f8f00 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbdd7e00640 1 --2- 172.21.15.16:0/1838624818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372f8f00 0x55ba372e3db0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80038b0 tx=0x7fbdc8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f671b0b36a5ca39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbdc3400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbdc3400640 1 -- 172.21.15.16:0/1838624818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc801d6e0 con 0x55ba372f8f00 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbdc3400640 1 -- 172.21.15.16:0/1838624818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fbdc8028ad0 con 0x55ba372f8f00 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 -- 172.21.15.16:0/1838624818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ba3708e0f0 con 0x55ba372f8f00 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbdc3400640 1 -- 172.21.15.16:0/1838624818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbdc8028cd0 con 0x55ba372f8f00 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 10 client.5400 fetch_fsmap learned FSMap version 24 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 -- 172.21.15.16:0/1838624818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55ba3717aef0 con 0x55ba372f8f00 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbdc3400640 1 -- 172.21.15.16:0/1838624818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ba3708e0f0 con 0x55ba372f8f00 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbdc3400640 1 -- 172.21.15.16:0/1838624818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7fbdc802f050 con 0x55ba372f8f00 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 10 client.5400 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:12:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 10 client.5400 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:12:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 20 client.5400 populate_metadata read hostname 'smithi016' 2024-09-10T21:12:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 --2- 172.21.15.16:0/1838624818 >> [v2:172.21.15.52:6832/1834750661,v1:172.21.15.52:6834/1834750661] conn(0x55ba372e4d00 0x55ba372e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 4 client.5400 mds_command: new command op to 134771 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:12:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 -- 172.21.15.16:0/1838624818 --> [v2:172.21.15.52:6832/1834750661,v1:172.21.15.52:6834/1834750661] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba37108a70 con 0x55ba372e4d00 2024-09-10T21:12:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbddde00640 1 --2- 172.21.15.16:0/1838624818 >> [v2:172.21.15.52:6832/1834750661,v1:172.21.15.52:6834/1834750661] conn(0x55ba372e4d00 0x55ba372e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbddde00640 1 --2- 172.21.15.16:0/1838624818 >> [v2:172.21.15.52:6832/1834750661,v1:172.21.15.52:6834/1834750661] conn(0x55ba372e4d00 0x55ba372e50e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=417e364a84451ad0 server_cookie=68a611acbb0e14f3 in_seq=0 out_seq=0 2024-09-10T21:12:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbdc3400640 10 client.5400 ms_handle_connect on v2:172.21.15.52:6832/1834750661 2024-09-10T21:12:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbdc3400640 1 -- 172.21.15.16:0/1838624818 <== mds.0 v2:172.21.15.52:6832/1834750661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba37108a70 con 0x55ba372e4d00 2024-09-10T21:12:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbdc3400640 10 client.5400 handle_command_reply: tid=0 2024-09-10T21:12:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 client.5400 shutdown 2024-09-10T21:12:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 -- 172.21.15.16:0/1838624818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372f8f00 msgr2=0x55ba372e3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 --2- 172.21.15.16:0/1838624818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372f8f00 0x55ba372e3db0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80038b0 tx=0x7fbdc8004740 comp rx=0 tx=0).stop 2024-09-10T21:12:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 -- 172.21.15.16:0/1838624818 shutdown_connections 2024-09-10T21:12:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 --2- 172.21.15.16:0/1838624818 >> [v2:172.21.15.52:6832/1834750661,v1:172.21.15.52:6834/1834750661] conn(0x55ba372e4d00 0x55ba372e50e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 --2- 172.21.15.16:0/1838624818 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f9820 0x55ba372e42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 --2- 172.21.15.16:0/1838624818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372f8f00 0x55ba372e3db0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbdc3400640 0 client.5400 ms_handle_reset on v2:172.21.15.52:6832/1834750661 2024-09-10T21:12:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 --2- 172.21.15.16:0/1838624818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372af230 0x55ba372e3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 -- 172.21.15.16:0/1838624818 >> 172.21.15.16:0/1838624818 conn(0x55ba37192130 msgr2=0x55ba372af610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 -- 172.21.15.16:0/1838624818 shutdown_connections 2024-09-10T21:12:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 1 -- 172.21.15.16:0/1838624818 wait complete. 2024-09-10T21:12:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.400+0000 7fbde850b000 20 client.5400 trim_cache size 0 max 16384 2024-09-10T21:12:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbde850b000 1 Processor -- start 2024-09-10T21:12:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbde850b000 1 -- start start 2024-09-10T21:12:35.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbde850b000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba37192130 0x55ba37310630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbde850b000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 0x55ba37310b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbde850b000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f8f00 0x55ba37305e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbde850b000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbde006fc00 con 0x55ba372af230 2024-09-10T21:12:35.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbde850b000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbde00b7920 con 0x55ba37192130 2024-09-10T21:12:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbde850b000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fbde00b5830 con 0x55ba372f8f00 2024-09-10T21:12:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbdd7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 0x55ba37310b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbdd7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 0x55ba37310b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53684/0 (socket says 172.21.15.16:53684) 2024-09-10T21:12:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbdd7e00640 1 -- 172.21.15.16:0/2720034524 learned_addr learned my addr 172.21.15.16:0/2720034524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbddde00640 1 --2- 172.21.15.16:0/2720034524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba37192130 0x55ba37310630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbdde800640 1 --2- 172.21.15.16:0/2720034524 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f8f00 0x55ba37305e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbdd7e00640 1 -- 172.21.15.16:0/2720034524 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f8f00 msgr2=0x55ba37305e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbdd7e00640 1 --2- 172.21.15.16:0/2720034524 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f8f00 0x55ba37305e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbdd7e00640 1 -- 172.21.15.16:0/2720034524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba37192130 msgr2=0x55ba37310630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbdd7e00640 1 --2- 172.21.15.16:0/2720034524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba37192130 0x55ba37310630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.420+0000 7fbdd7e00640 1 -- 172.21.15.16:0/2720034524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba372e4660 con 0x55ba372af230 2024-09-10T21:12:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbdd7e00640 1 --2- 172.21.15.16:0/2720034524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 0x55ba37310b70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8000c10 tx=0x7fbdc80044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7547029c3cde9748 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbdce000640 1 -- 172.21.15.16:0/2720034524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8028530 con 0x55ba372af230 2024-09-10T21:12:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbdce000640 1 -- 172.21.15.16:0/2720034524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdc80286f0 con 0x55ba372af230 2024-09-10T21:12:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbdce000640 1 -- 172.21.15.16:0/2720034524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc801d6e0 con 0x55ba372af230 2024-09-10T21:12:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 -- 172.21.15.16:0/2720034524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 msgr2=0x55ba37310b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 --2- 172.21.15.16:0/2720034524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 0x55ba37310b70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8000c10 tx=0x7fbdc80044a0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 -- 172.21.15.16:0/2720034524 shutdown_connections 2024-09-10T21:12:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 --2- 172.21.15.16:0/2720034524 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372f8f00 0x55ba37305e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 --2- 172.21.15.16:0/2720034524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba372af230 0x55ba37310b70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 --2- 172.21.15.16:0/2720034524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba37192130 0x55ba37310630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 -- 172.21.15.16:0/2720034524 >> 172.21.15.16:0/2720034524 conn(0x55ba372e4280 msgr2=0x55ba372ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 -- 172.21.15.16:0/2720034524 shutdown_connections 2024-09-10T21:12:35.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 -- 172.21.15.16:0/2720034524 wait complete. 2024-09-10T21:12:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 -- start start 2024-09-10T21:12:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 Processor -- start 2024-09-10T21:12:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba37192130 0x55ba3730d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372af230 0x55ba3730dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372f8f00 0x55ba3730e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ba370c1ec0 con 0x55ba37192130 2024-09-10T21:12:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbde00b72d0 con 0x55ba372f8f00 2024-09-10T21:12:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55ba370af1c0 con 0x55ba372af230 2024-09-10T21:12:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 10 client.0 _ll_register_callbacks cb 0x7fbdc8005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbddde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba37192130 0x55ba3730d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbddde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba37192130 0x55ba3730d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53696/0 (socket says 172.21.15.16:53696) 2024-09-10T21:12:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbddde00640 1 -- 172.21.15.16:0/1287194376 learned_addr learned my addr 172.21.15.16:0/1287194376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbdd7e00640 1 --2- 172.21.15.16:0/1287194376 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372af230 0x55ba3730dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbdde800640 1 --2- 172.21.15.16:0/1287194376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372f8f00 0x55ba3730e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbddde00640 1 -- 172.21.15.16:0/1287194376 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372af230 msgr2=0x55ba3730dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbddde00640 1 --2- 172.21.15.16:0/1287194376 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372af230 0x55ba3730dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbddde00640 1 -- 172.21.15.16:0/1287194376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372f8f00 msgr2=0x55ba3730e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbddde00640 1 --2- 172.21.15.16:0/1287194376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372f8f00 0x55ba3730e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbddde00640 1 -- 172.21.15.16:0/1287194376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba372f9650 con 0x55ba37192130 2024-09-10T21:12:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbddde00640 1 --2- 172.21.15.16:0/1287194376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba37192130 0x55ba3730d960 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbdd005aa90 tx=0x7fbdd0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d070dc80019c06d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbdc3400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:12:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbdc3400640 1 -- 172.21.15.16:0/1287194376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd008e060 con 0x55ba37192130 2024-09-10T21:12:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbdc3400640 1 -- 172.21.15.16:0/1287194376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fbdd00669e0 con 0x55ba37192130 2024-09-10T21:12:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.424+0000 7fbde850b000 1 -- 172.21.15.16:0/1287194376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc80286f0 con 0x55ba37192130 2024-09-10T21:12:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbdc3400640 1 -- 172.21.15.16:0/1287194376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbdd0089e60 con 0x55ba37192130 2024-09-10T21:12:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 10 client.5406 fetch_fsmap learned FSMap version 24 2024-09-10T21:12:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 -- 172.21.15.16:0/1287194376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55ba37151d90 con 0x55ba37192130 2024-09-10T21:12:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbdc3400640 1 -- 172.21.15.16:0/1287194376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdc80286f0 con 0x55ba37192130 2024-09-10T21:12:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbdc3400640 1 -- 172.21.15.16:0/1287194376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7fbdd0082b30 con 0x55ba37192130 2024-09-10T21:12:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 10 client.5406 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:12:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 10 client.5406 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 20 client.5406 populate_metadata read hostname 'smithi016' 2024-09-10T21:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 --2- 172.21.15.16:0/1287194376 >> [v2:172.21.15.52:6832/1834750661,v1:172.21.15.52:6834/1834750661] conn(0x55ba372f9820 0x55ba3730e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 4 client.5406 mds_command: new command op to 134771 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 -- 172.21.15.16:0/1287194376 --> [v2:172.21.15.52:6832/1834750661,v1:172.21.15.52:6834/1834750661] -- command(tid 0: {"prefix": "status"}) -- 0x55ba37104520 con 0x55ba372f9820 2024-09-10T21:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbdd7e00640 1 --2- 172.21.15.16:0/1287194376 >> [v2:172.21.15.52:6832/1834750661,v1:172.21.15.52:6834/1834750661] conn(0x55ba372f9820 0x55ba3730e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbdd7e00640 1 --2- 172.21.15.16:0/1287194376 >> [v2:172.21.15.52:6832/1834750661,v1:172.21.15.52:6834/1834750661] conn(0x55ba372f9820 0x55ba3730e920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f73a989b730b7f2e server_cookie=dbd24951a65199ad in_seq=0 out_seq=0 2024-09-10T21:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbdc3400640 10 client.5406 ms_handle_connect on v2:172.21.15.52:6832/1834750661 2024-09-10T21:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbdc3400640 1 -- 172.21.15.16:0/1287194376 <== mds.0 v2:172.21.15.52:6832/1834750661 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ba37104520 con 0x55ba372f9820 2024-09-10T21:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbdc3400640 10 client.5406 handle_command_reply: tid=0 2024-09-10T21:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 client.5406 shutdown 2024-09-10T21:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 -- 172.21.15.16:0/1287194376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba37192130 msgr2=0x55ba3730d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 --2- 172.21.15.16:0/1287194376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba37192130 0x55ba3730d960 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbdd005aa90 tx=0x7fbdd0000bd0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 -- 172.21.15.16:0/1287194376 shutdown_connections 2024-09-10T21:12:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 --2- 172.21.15.16:0/1287194376 >> [v2:172.21.15.52:6832/1834750661,v1:172.21.15.52:6834/1834750661] conn(0x55ba372f9820 0x55ba3730e920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 --2- 172.21.15.16:0/1287194376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba372f8f00 0x55ba3730e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 --2- 172.21.15.16:0/1287194376 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ba372af230 0x55ba3730dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 --2- 172.21.15.16:0/1287194376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba37192130 0x55ba3730d960 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbdc3400640 0 client.5406 ms_handle_reset on v2:172.21.15.52:6832/1834750661 2024-09-10T21:12:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 -- 172.21.15.16:0/1287194376 >> 172.21.15.16:0/1287194376 conn(0x55ba372e4280 msgr2=0x55ba37305e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 -- 172.21.15.16:0/1287194376 shutdown_connections 2024-09-10T21:12:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 1 -- 172.21.15.16:0/1287194376 wait complete. 2024-09-10T21:12:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde850b000 20 client.5406 trim_cache size 0 max 16384 2024-09-10T21:12:35.441 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:12:35.441 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "eb19be60-6d8f-4816-b778-3d7d6698304d", 2024-09-10T21:12:35.441 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-10T21:12:35.441 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 134771, 2024-09-10T21:12:35.441 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-10T21:12:35.441 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-10T21:12:35.441 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-10T21:12:35.441 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 10.650318823999999, 2024-09-10T21:12:35.441 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-09-10T21:12:35.441 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 46, 2024-09-10T21:12:35.441 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 46, 2024-09-10T21:12:35.441 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 15.068384704 2024-09-10T21:12:35.442 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:12:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde5000640 1 -- 172.21.15.16:0/4048498785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbdbc0725e0 msgr2=0x7fbdbc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde5000640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbdbc0725e0 0x7fbdbc074aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbde00c6070 tx=0x7fbdc80091e0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde5000640 1 -- 172.21.15.16:0/4048498785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbde00aed30 msgr2=0x7fbde01f8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde5000640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbde00aed30 0x7fbde01f8f20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000c3c0 tx=0x7fbdd000c880 comp rx=0 tx=0).stop 2024-09-10T21:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde5000640 1 -- 172.21.15.16:0/4048498785 shutdown_connections 2024-09-10T21:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde5000640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbdbc0725e0 0x7fbdbc074aa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde5000640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbde015f350 0x7fbde0148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde5000640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbde00cd130 0x7fbde00cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde5000640 1 --2- 172.21.15.16:0/4048498785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbde00aed30 0x7fbde01f8f20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde5000640 1 -- 172.21.15.16:0/4048498785 >> 172.21.15.16:0/4048498785 conn(0x7fbde00d1d80 msgr2=0x7fbde00cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde5000640 1 -- 172.21.15.16:0/4048498785 shutdown_connections 2024-09-10T21:12:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.428+0000 7fbde5000640 1 -- 172.21.15.16:0/4048498785 wait complete. 2024-09-10T21:12:35.455 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:12:35.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.596+0000 7ff61d800640 1 -- 172.21.15.16:0/2533413151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff618154f90 msgr2=0x7ff618072d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.596+0000 7ff61d800640 1 --2- 172.21.15.16:0/2533413151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff618154f90 0x7ff618072d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff608009840 tx=0x7ff60801c710 comp rx=0 tx=0).stop 2024-09-10T21:12:35.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.596+0000 7ff61d800640 1 -- 172.21.15.16:0/2533413151 shutdown_connections 2024-09-10T21:12:35.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.596+0000 7ff61d800640 1 --2- 172.21.15.16:0/2533413151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff618154f90 0x7ff618072d40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.596+0000 7ff61d800640 1 -- 172.21.15.16:0/2533413151 >> 172.21.15.16:0/2533413151 conn(0x7ff6180cfb20 msgr2=0x7ff6180cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:35.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.596+0000 7ff61d800640 1 -- 172.21.15.16:0/2533413151 shutdown_connections 2024-09-10T21:12:35.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.596+0000 7ff61d800640 1 -- 172.21.15.16:0/2533413151 wait complete. 2024-09-10T21:12:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.596+0000 7ff61d800640 1 Processor -- start 2024-09-10T21:12:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.596+0000 7ff61d800640 1 -- start start 2024-09-10T21:12:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff61d800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6180706f0 0x7ff6180c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff61d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618154f90 0x7ff618147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff61d800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff61815bc70 0x7ff61814de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff61d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff618168b60 con 0x7ff618154f90 2024-09-10T21:12:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff61d800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6181689e0 con 0x7ff6180706f0 2024-09-10T21:12:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff61d800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff618168ce0 con 0x7ff61815bc70 2024-09-10T21:12:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618154f90 0x7ff618147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618154f90 0x7ff618147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53710/0 (socket says 172.21.15.16:53710) 2024-09-10T21:12:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff60fe00640 1 -- 172.21.15.16:0/3561464511 learned_addr learned my addr 172.21.15.16:0/3561464511 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff616a00640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6180706f0 0x7ff6180c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff617400640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff61815bc70 0x7ff61814de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff60fe00640 1 -- 172.21.15.16:0/3561464511 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff61815bc70 msgr2=0x7ff61814de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff60fe00640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff61815bc70 0x7ff61814de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff60fe00640 1 -- 172.21.15.16:0/3561464511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6180706f0 msgr2=0x7ff6180c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff60fe00640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6180706f0 0x7ff6180c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff60fe00640 1 -- 172.21.15.16:0/3561464511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61814d090 con 0x7ff618154f90 2024-09-10T21:12:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff616a00640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6180706f0 0x7ff6180c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff60fe00640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618154f90 0x7ff618147250 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff600001230 tx=0x7ff60000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff60ea00640 1 -- 172.21.15.16:0/3561464511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600016020 con 0x7ff618154f90 2024-09-10T21:12:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff60ea00640 1 -- 172.21.15.16:0/3561464511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff600012920 con 0x7ff618154f90 2024-09-10T21:12:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff61d800640 1 -- 172.21.15.16:0/3561464511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff618147ae0 con 0x7ff618154f90 2024-09-10T21:12:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff60ea00640 1 -- 172.21.15.16:0/3561464511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600012db0 con 0x7ff618154f90 2024-09-10T21:12:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff61d800640 1 -- 172.21.15.16:0/3561464511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61814d910 con 0x7ff618154f90 2024-09-10T21:12:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.600+0000 7ff61d800640 1 -- 172.21.15.16:0/3561464511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff61814db10 con 0x7ff618154f90 2024-09-10T21:12:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.604+0000 7ff60ea00640 1 -- 172.21.15.16:0/3561464511 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff60001a3b0 con 0x7ff618154f90 2024-09-10T21:12:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.604+0000 7ff60ea00640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff5f40725e0 0x7ff5f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.604+0000 7ff60ea00640 1 -- 172.21.15.16:0/3561464511 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff600093370 con 0x7ff618154f90 2024-09-10T21:12:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.604+0000 7ff616a00640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff5f40725e0 0x7ff5f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.604+0000 7ff616a00640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff5f40725e0 0x7ff5f4074aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff6080af000 tx=0x7ff60801af50 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.608+0000 7ff60ea00640 1 -- 172.21.15.16:0/3561464511 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff61814db10 con 0x7ff618154f90 2024-09-10T21:12:35.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.816+0000 7ff61d800640 1 -- 172.21.15.16:0/3561464511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff618072d40 con 0x7ff618154f90 2024-09-10T21:12:35.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.820+0000 7ff60ea00640 1 -- 172.21.15.16:0/3561464511 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7ff600020030 con 0x7ff618154f90 2024-09-10T21:12:35.825 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:12:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.820+0000 7ff61d800640 1 -- 172.21.15.16:0/3561464511 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff5f40725e0 msgr2=0x7ff5f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.820+0000 7ff61d800640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff5f40725e0 0x7ff5f4074aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff6080af000 tx=0x7ff60801af50 comp rx=0 tx=0).stop 2024-09-10T21:12:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.820+0000 7ff61d800640 1 -- 172.21.15.16:0/3561464511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618154f90 msgr2=0x7ff618147250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.820+0000 7ff61d800640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618154f90 0x7ff618147250 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff600001230 tx=0x7ff60000c980 comp rx=0 tx=0).stop 2024-09-10T21:12:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.820+0000 7ff61d800640 1 -- 172.21.15.16:0/3561464511 shutdown_connections 2024-09-10T21:12:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.820+0000 7ff61d800640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff5f40725e0 0x7ff5f4074aa0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.820+0000 7ff61d800640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff61815bc70 0x7ff61814de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.820+0000 7ff61d800640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618154f90 0x7ff618147250 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.820+0000 7ff61d800640 1 --2- 172.21.15.16:0/3561464511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6180706f0 0x7ff6180c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.820+0000 7ff61d800640 1 -- 172.21.15.16:0/3561464511 >> 172.21.15.16:0/3561464511 conn(0x7ff6180cfb20 msgr2=0x7ff61806ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.820+0000 7ff61d800640 1 -- 172.21.15.16:0/3561464511 shutdown_connections 2024-09-10T21:12:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:35.820+0000 7ff61d800640 1 -- 172.21.15.16:0/3561464511 wait complete. 2024-09-10T21:12:35.848 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:12:35.848 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:12:35.848 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-10T21:12:35.848 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:12:35.848 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:12:35.848 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:12:35.848 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:12:35.848 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:12:35.848 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:12:35.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:35.849 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-10T21:12:35.859 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:12:35.859 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:12:35.860 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:12:35.860 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:35.860 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:12:35.860 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:35.860 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:12:35.860 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:12:35.860 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-10T21:12:35.860 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:12:35.860 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:12:35.860 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-10T21:12:35.861 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:12:35.861 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:12:35.861 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-10T21:12:35.861 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:12:35.861 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42294sec preferred_lft 42294sec 2024-09-10T21:12:35.861 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-10T21:12:35.861 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:35.861 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:12:35.861 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:12:35.861 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-10T21:12:35.861 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:12:35.861 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:30:fb:74:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:12:35.862 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:12:35.862 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:35.862 INFO:teuthology.orchestra.run.smithi016.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:12:35.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:12:35.862 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:12:35.862 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:35.862 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:12:35.862 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:35.862 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:12:35.863 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:12:35.863 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:12:35.863 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:12:35.863 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:12:35.863 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:12:35.863 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:35.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:35.863 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:12:35.919 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-10T21:12:35.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:35.921 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-10T21:12:35.974 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-10T21:12:35.974 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:12:35.974 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:12:35.974 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:12:35.974 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:12:35.974 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:12:35.975 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:12:35.975 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:12:35.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:12:35.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:12:35.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:12:35.975 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:12:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:12:36.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:12:36.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:36.114 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:12:36.126 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:12:36.127 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:12:36.127 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:12:36.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:36.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-10T21:12:36.189 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:12:36.189 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:12:36.241 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:12:36.241 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:12:36.241 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:12:36.296 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:36.296 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:12:36.386 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:12:36.386 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:12:36.400 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:12:36.400 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:12:36.400 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:12:36.400 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:12:36.400 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:12:36.404 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:12:36.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:36.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:12:36.424 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:12:36.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:12:36.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:12:36.536 INFO:teuthology.orchestra.run.smithi016.stdout:client134873 2024-09-10T21:12:36.536 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:12:36.536 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:36.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:12:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.728+0000 7fa186a00640 1 -- 172.21.15.16:0/4036507438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa164004d50 msgr2=0x7fa1640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.728+0000 7fa186a00640 1 --2- 172.21.15.16:0/4036507438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa164004d50 0x7fa1640051d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa178009ee0 tx=0x7fa17801d5f0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.728+0000 7fa184a00640 1 -- 172.21.15.16:0/4036507438 reap_dead start 2024-09-10T21:12:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.728+0000 7fa186a00640 1 -- 172.21.15.16:0/4036507438 shutdown_connections 2024-09-10T21:12:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.728+0000 7fa186a00640 1 -- 172.21.15.16:0/4036507438 >> 172.21.15.16:0/4036507438 conn(0x7fa1800cfb20 msgr2=0x7fa1800cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.728+0000 7fa186a00640 1 -- 172.21.15.16:0/4036507438 shutdown_connections 2024-09-10T21:12:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.728+0000 7fa186a00640 1 -- 172.21.15.16:0/4036507438 wait complete. 2024-09-10T21:12:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.728+0000 7fa186a00640 1 Processor -- start 2024-09-10T21:12:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa186a00640 1 -- start start 2024-09-10T21:12:36.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa186a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164004d50 0x7fa1801584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa186a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1800c6bf0 0x7fa1800c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa186a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa180158c40 0x7fa1800c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa186a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa180072ac0 con 0x7fa1800c6bf0 2024-09-10T21:12:36.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa186a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa180072940 con 0x7fa164004d50 2024-09-10T21:12:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa186a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa180072c40 con 0x7fa180158c40 2024-09-10T21:12:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa177e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1800c6bf0 0x7fa1800c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa177e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1800c6bf0 0x7fa1800c9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53720/0 (socket says 172.21.15.16:53720) 2024-09-10T21:12:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa177e00640 1 -- 172.21.15.16:0/1206798645 learned_addr learned my addr 172.21.15.16:0/1206798645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa17fe00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164004d50 0x7fa1801584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa184a00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa180158c40 0x7fa1800c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa177e00640 1 -- 172.21.15.16:0/1206798645 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa180158c40 msgr2=0x7fa1800c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa177e00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa180158c40 0x7fa1800c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa177e00640 1 -- 172.21.15.16:0/1206798645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164004d50 msgr2=0x7fa1801584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa177e00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164004d50 0x7fa1801584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa177e00640 1 -- 172.21.15.16:0/1206798645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa180158a10 con 0x7fa1800c6bf0 2024-09-10T21:12:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa177e00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1800c6bf0 0x7fa1800c9120 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa168011b10 tx=0x7fa168009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa17cc00640 1 -- 172.21.15.16:0/1206798645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16800fd00 con 0x7fa1800c6bf0 2024-09-10T21:12:36.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa17cc00640 1 -- 172.21.15.16:0/1206798645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa168004020 con 0x7fa1800c6bf0 2024-09-10T21:12:36.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa186a00640 1 -- 172.21.15.16:0/1206798645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1800c6980 con 0x7fa1800c6bf0 2024-09-10T21:12:36.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa17cc00640 1 -- 172.21.15.16:0/1206798645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16800fea0 con 0x7fa1800c6bf0 2024-09-10T21:12:36.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa186a00640 1 -- 172.21.15.16:0/1206798645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1800c7600 con 0x7fa1800c6bf0 2024-09-10T21:12:36.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.732+0000 7fa186a00640 1 -- 172.21.15.16:0/1206798645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa18015fc70 con 0x7fa1800c6bf0 2024-09-10T21:12:36.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.736+0000 7fa17cc00640 1 -- 172.21.15.16:0/1206798645 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa168003600 con 0x7fa1800c6bf0 2024-09-10T21:12:36.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.736+0000 7fa17cc00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa15c072610 0x7fa15c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.736+0000 7fa17cc00640 1 -- 172.21.15.16:0/1206798645 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fa168093e00 con 0x7fa1800c6bf0 2024-09-10T21:12:36.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.736+0000 7fa17fe00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa15c072610 0x7fa15c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.736+0000 7fa17fe00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa15c072610 0x7fa15c074ad0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa170001870 tx=0x7fa170001e60 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:36.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.740+0000 7fa17cc00640 1 -- 172.21.15.16:0/1206798645 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa168004470 con 0x7fa1800c6bf0 2024-09-10T21:12:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.952+0000 7fa186a00640 1 -- 172.21.15.16:0/1206798645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa1800744b0 con 0x7fa1800c6bf0 2024-09-10T21:12:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.952+0000 7fa17cc00640 1 -- 172.21.15.16:0/1206798645 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7fa16805f440 con 0x7fa1800c6bf0 2024-09-10T21:12:36.960 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:12:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.956+0000 7fa186a00640 1 -- 172.21.15.16:0/1206798645 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa15c072610 msgr2=0x7fa15c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.956+0000 7fa186a00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa15c072610 0x7fa15c074ad0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa170001870 tx=0x7fa170001e60 comp rx=0 tx=0).stop 2024-09-10T21:12:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.956+0000 7fa186a00640 1 -- 172.21.15.16:0/1206798645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1800c6bf0 msgr2=0x7fa1800c9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.956+0000 7fa186a00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1800c6bf0 0x7fa1800c9120 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa168011b10 tx=0x7fa168009f20 comp rx=0 tx=0).stop 2024-09-10T21:12:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.956+0000 7fa186a00640 1 -- 172.21.15.16:0/1206798645 shutdown_connections 2024-09-10T21:12:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.956+0000 7fa186a00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa15c072610 0x7fa15c074ad0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.956+0000 7fa186a00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa180158c40 0x7fa1800c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.956+0000 7fa186a00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1800c6bf0 0x7fa1800c9120 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.956+0000 7fa186a00640 1 --2- 172.21.15.16:0/1206798645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164004d50 0x7fa1801584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.956+0000 7fa186a00640 1 -- 172.21.15.16:0/1206798645 >> 172.21.15.16:0/1206798645 conn(0x7fa1800cfb20 msgr2=0x7fa1800ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.956+0000 7fa186a00640 1 -- 172.21.15.16:0/1206798645 shutdown_connections 2024-09-10T21:12:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:36.956+0000 7fa186a00640 1 -- 172.21.15.16:0/1206798645 wait complete. 2024-09-10T21:12:36.980 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:12:36.981 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:12:36.981 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-09-10T21:12:36.981 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T21:12:36.981 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:12:36.981 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:12:36.981 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:12:36.981 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:12:36.981 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:12:36.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:36.981 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-09-10T21:12:36.992 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:12:36.992 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:12:36.992 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:12:36.992 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:36.992 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:12:36.992 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:36.992 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:12:36.992 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:12:36.992 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-09-10T21:12:36.992 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:12:36.993 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:12:36.993 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-09-10T21:12:36.993 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:12:36.993 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-09-10T21:12:36.993 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-09-10T21:12:36.993 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:12:36.993 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42294sec preferred_lft 42294sec 2024-09-10T21:12:36.993 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-09-10T21:12:36.993 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:36.993 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:12:36.993 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-09-10T21:12:36.993 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-09-10T21:12:36.993 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:12:36.994 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:7e:b4:71:a2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:12:36.994 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:12:36.994 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:36.994 INFO:teuthology.orchestra.run.smithi052.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:12:36.994 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:12:36.994 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:12:36.994 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:36.994 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:12:36.994 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:36.994 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:12:36.994 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:12:36.994 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:12:36.994 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:36.995 INFO:teuthology.orchestra.run.smithi052.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:12:36.995 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:12:36.995 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:12:36.995 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:12:36.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:36.996 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:12:37.061 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-09-10T21:12:37.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:37.064 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-09-10T21:12:37.115 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-09-10T21:12:37.116 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:12:37.116 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:12:37.116 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:12:37.116 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:12:37.116 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:12:37.116 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:12:37.116 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:12:37.116 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:12:37.116 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:12:37.117 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:12:37.117 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:12:37.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:12:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:12:37.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:12:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:12:37.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:37.268 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:12:37.277 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:12:37.277 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:12:37.278 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:12:37.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:37.278 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-09-10T21:12:37.346 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:12:37.346 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T21:12:37.397 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:12:37.397 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:12:37.397 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:12:37.448 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T21:12:37.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:37.449 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:12:37.544 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:12:37.545 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:12:37.556 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:12:37.557 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:12:37.557 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:12:37.557 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 1@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:12:37.557 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:12:37.560 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:12:37.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:37.561 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:12:37.581 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:12:37.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:12:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:12:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:12:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:12:37.682 INFO:teuthology.orchestra.run.smithi052.stdout:client134882 2024-09-10T21:12:37.683 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:12:37.683 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:37.692 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:12:37.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:12:37.784 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:12:37.864 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:12:37.864 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:12:37.864 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:12:37.864 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:37.874 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:12:37.875 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:37.930 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:12:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.128+0000 7f8a1b200640 1 -- 172.21.15.16:0/575105206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a140cffe0 msgr2=0x7f8a140ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.128+0000 7f8a1b200640 1 --2- 172.21.15.16:0/575105206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a140cffe0 0x7f8a140ce2b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8a140b4930 tx=0x7f8a0401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.128+0000 7f8a1b200640 1 -- 172.21.15.16:0/575105206 shutdown_connections 2024-09-10T21:12:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.128+0000 7f8a1b200640 1 --2- 172.21.15.16:0/575105206 >> v2:172.21.15.52:3300/0 conn(0x7f8a140b2860 0x7f8a14160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.128+0000 7f8a1b200640 1 --2- 172.21.15.16:0/575105206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a140cffe0 0x7f8a140ce2b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.128+0000 7f8a1b200640 1 -- 172.21.15.16:0/575105206 >> 172.21.15.16:0/575105206 conn(0x7f8a140d1d80 msgr2=0x7f8a140d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:38.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.128+0000 7f8a1b200640 1 -- 172.21.15.16:0/575105206 shutdown_connections 2024-09-10T21:12:38.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.128+0000 7f8a1b200640 1 -- 172.21.15.16:0/575105206 wait complete. 2024-09-10T21:12:38.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a1b200640 1 Processor -- start 2024-09-10T21:12:38.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a1b200640 1 -- start start 2024-09-10T21:12:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a1b200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a140b2860 0x7f8a14070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a1b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a140cffe0 0x7f8a141fae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a1b200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a140d03c0 0x7f8a14151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a1b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a140cd930 con 0x7f8a140cffe0 2024-09-10T21:12:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a1b200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a140cd7b0 con 0x7f8a140b2860 2024-09-10T21:12:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a1b200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8a140cdab0 con 0x7f8a140d03c0 2024-09-10T21:12:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a0be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a140cffe0 0x7f8a141fae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a0be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a140cffe0 0x7f8a141fae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53736/0 (socket says 172.21.15.16:53736) 2024-09-10T21:12:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a0be00640 1 -- 172.21.15.16:0/1974864056 learned_addr learned my addr 172.21.15.16:0/1974864056 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a13e00640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a140b2860 0x7f8a14070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a19200640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a140d03c0 0x7f8a14151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a0be00640 1 -- 172.21.15.16:0/1974864056 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a140d03c0 msgr2=0x7f8a14151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a0be00640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a140d03c0 0x7f8a14151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a0be00640 1 -- 172.21.15.16:0/1974864056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a140b2860 msgr2=0x7f8a14070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a0be00640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a140b2860 0x7f8a14070bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a0be00640 1 -- 172.21.15.16:0/1974864056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a141f7a00 con 0x7f8a140cffe0 2024-09-10T21:12:38.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a0be00640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a140cffe0 0x7f8a141fae00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c4e0 tx=0x7f89fc00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:38.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a10c00640 1 -- 172.21.15.16:0/1974864056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0052d0 con 0x7f8a140cffe0 2024-09-10T21:12:38.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a10c00640 1 -- 172.21.15.16:0/1974864056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89fc005470 con 0x7f8a140cffe0 2024-09-10T21:12:38.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a10c00640 1 -- 172.21.15.16:0/1974864056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc012890 con 0x7f8a140cffe0 2024-09-10T21:12:38.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a1b200640 1 -- 172.21.15.16:0/1974864056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a141fb570 con 0x7f8a140cffe0 2024-09-10T21:12:38.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a1b200640 1 -- 172.21.15.16:0/1974864056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a141fbb30 con 0x7f8a140cffe0 2024-09-10T21:12:38.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.132+0000 7f8a1b200640 1 -- 172.21.15.16:0/1974864056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a141f7810 con 0x7f8a140cffe0 2024-09-10T21:12:38.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.136+0000 7f8a10c00640 1 -- 172.21.15.16:0/1974864056 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f89fc003600 con 0x7f8a140cffe0 2024-09-10T21:12:38.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.136+0000 7f8a10c00640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f89f00725e0 0x7f89f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:38.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.136+0000 7f8a10c00640 1 -- 172.21.15.16:0/1974864056 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f89fc092dc0 con 0x7f8a140cffe0 2024-09-10T21:12:38.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.136+0000 7f8a13e00640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f89f00725e0 0x7f89f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:38.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.136+0000 7f8a13e00640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f89f00725e0 0x7f89f0074aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8a040af000 tx=0x7f8a0401b100 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.140+0000 7f8a10c00640 1 -- 172.21.15.16:0/1974864056 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f89fc05e400 con 0x7f8a140cffe0 2024-09-10T21:12:38.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.324+0000 7f8a1b200640 1 -- 172.21.15.16:0/1974864056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8a141f7bd0 con 0x7f8a140cffe0 2024-09-10T21:12:38.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.324+0000 7f8a10c00640 1 -- 172.21.15.16:0/1974864056 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7f89fc05dd00 con 0x7f8a140cffe0 2024-09-10T21:12:38.332 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:12:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.328+0000 7f8a1b200640 1 -- 172.21.15.16:0/1974864056 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f89f00725e0 msgr2=0x7f89f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.328+0000 7f8a1b200640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f89f00725e0 0x7f89f0074aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8a040af000 tx=0x7f8a0401b100 comp rx=0 tx=0).stop 2024-09-10T21:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.328+0000 7f8a1b200640 1 -- 172.21.15.16:0/1974864056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a140cffe0 msgr2=0x7f8a141fae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.328+0000 7f8a1b200640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a140cffe0 0x7f8a141fae00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c4e0 tx=0x7f89fc00c9a0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.328+0000 7f8a1b200640 1 -- 172.21.15.16:0/1974864056 shutdown_connections 2024-09-10T21:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.328+0000 7f8a1b200640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f89f00725e0 0x7f89f0074aa0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.328+0000 7f8a1b200640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a140d03c0 0x7f8a14151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.328+0000 7f8a1b200640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a140cffe0 0x7f8a141fae00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.328+0000 7f8a1b200640 1 --2- 172.21.15.16:0/1974864056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a140b2860 0x7f8a14070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.328+0000 7f8a1b200640 1 -- 172.21.15.16:0/1974864056 >> 172.21.15.16:0/1974864056 conn(0x7f8a140d1d80 msgr2=0x7f8a1415c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.328+0000 7f8a1b200640 1 -- 172.21.15.16:0/1974864056 shutdown_connections 2024-09-10T21:12:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.328+0000 7f8a1b200640 1 -- 172.21.15.16:0/1974864056 wait complete. 2024-09-10T21:12:38.354 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:12:38.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:38.355 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:12:38.382 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:12:38.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:38.382 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:12:38.434 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:12:38.435 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:12:38.435 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:38.444 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:12:38.444 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:38.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:12:38.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 -- 172.21.15.16:0/417314502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10a00b5050 msgr2=0x7f10a015f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 --2- 172.21.15.16:0/417314502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10a00b5050 0x7f10a015f850 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f10a01568d0 tx=0x7f109001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 -- 172.21.15.16:0/417314502 shutdown_connections 2024-09-10T21:12:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 --2- 172.21.15.16:0/417314502 >> v2:172.21.15.52:3300/0 conn(0x7f10a00ca170 0x7f10a00ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 --2- 172.21.15.16:0/417314502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10a00b5050 0x7f10a015f850 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 -- 172.21.15.16:0/417314502 >> 172.21.15.16:0/417314502 conn(0x7f10a00cfb20 msgr2=0x7f10a00cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 -- 172.21.15.16:0/417314502 shutdown_connections 2024-09-10T21:12:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 -- 172.21.15.16:0/417314502 wait complete. 2024-09-10T21:12:38.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 Processor -- start 2024-09-10T21:12:38.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 -- start start 2024-09-10T21:12:38.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10a006e640 0x7f10a0158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:38.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10a00b5050 0x7f10a01f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:38.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10a00ca170 0x7f10a0155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:38.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10a0168e80 con 0x7f10a00ca170 2024-09-10T21:12:38.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f10a0168d00 con 0x7f10a00b5050 2024-09-10T21:12:38.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f10a5000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f10a0169000 con 0x7f10a006e640 2024-09-10T21:12:38.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f109e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10a00ca170 0x7f10a0155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:38.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f109e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10a00ca170 0x7f10a0155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53756/0 (socket says 172.21.15.16:53756) 2024-09-10T21:12:38.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f109e800640 1 -- 172.21.15.16:0/2591449919 learned_addr learned my addr 172.21.15.16:0/2591449919 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:38.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f1097e00640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10a00b5050 0x7f10a01f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:38.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f109de00640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10a006e640 0x7f10a0158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:38.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f109e800640 1 -- 172.21.15.16:0/2591449919 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10a006e640 msgr2=0x7f10a0158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:38.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f109e800640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10a006e640 0x7f10a0158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f109e800640 1 -- 172.21.15.16:0/2591449919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10a00b5050 msgr2=0x7f10a01f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:38.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f109e800640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10a00b5050 0x7f10a01f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.640+0000 7f109e800640 1 -- 172.21.15.16:0/2591449919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a01f5d70 con 0x7f10a00ca170 2024-09-10T21:12:38.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.644+0000 7f109e800640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10a00ca170 0x7f10a0155cb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f109800cce0 tx=0x7f109800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:38.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.644+0000 7f1096000640 1 -- 172.21.15.16:0/2591449919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109800f450 con 0x7f10a00ca170 2024-09-10T21:12:38.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.644+0000 7f1096000640 1 -- 172.21.15.16:0/2591449919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f109800f6e0 con 0x7f10a00ca170 2024-09-10T21:12:38.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.644+0000 7f1096000640 1 -- 172.21.15.16:0/2591449919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109800fb30 con 0x7f10a00ca170 2024-09-10T21:12:38.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.644+0000 7f10a5000640 1 -- 172.21.15.16:0/2591449919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a01f6050 con 0x7f10a00ca170 2024-09-10T21:12:38.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.644+0000 7f10a5000640 1 -- 172.21.15.16:0/2591449919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a01561f0 con 0x7f10a00ca170 2024-09-10T21:12:38.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.644+0000 7f10a5000640 1 -- 172.21.15.16:0/2591449919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a015f850 con 0x7f10a00ca170 2024-09-10T21:12:38.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.644+0000 7f1096000640 1 -- 172.21.15.16:0/2591449919 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f109800fcd0 con 0x7f10a00ca170 2024-09-10T21:12:38.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.644+0000 7f1096000640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f107c0725e0 0x7f107c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:38.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.644+0000 7f1096000640 1 -- 172.21.15.16:0/2591449919 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f1098093d10 con 0x7f10a00ca170 2024-09-10T21:12:38.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.644+0000 7f109de00640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f107c0725e0 0x7f107c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:38.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.648+0000 7f109de00640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f107c0725e0 0x7f107c074aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f10900af000 tx=0x7f109001b100 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:38.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.648+0000 7f1096000640 1 -- 172.21.15.16:0/2591449919 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1098099020 con 0x7f10a00ca170 2024-09-10T21:12:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.832+0000 7f10a5000640 1 -- 172.21.15.16:0/2591449919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f10a015e280 con 0x7f10a00ca170 2024-09-10T21:12:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.836+0000 7f1096000640 1 -- 172.21.15.16:0/2591449919 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7f1098014030 con 0x7f10a00ca170 2024-09-10T21:12:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:12:38.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.840+0000 7f10a5000640 1 -- 172.21.15.16:0/2591449919 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f107c0725e0 msgr2=0x7f107c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:38.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.840+0000 7f10a5000640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f107c0725e0 0x7f107c074aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f10900af000 tx=0x7f109001b100 comp rx=0 tx=0).stop 2024-09-10T21:12:38.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.840+0000 7f10a5000640 1 -- 172.21.15.16:0/2591449919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10a00ca170 msgr2=0x7f10a0155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:38.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.840+0000 7f10a5000640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10a00ca170 0x7f10a0155cb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f109800cce0 tx=0x7f109800a620 comp rx=0 tx=0).stop 2024-09-10T21:12:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.840+0000 7f10a5000640 1 -- 172.21.15.16:0/2591449919 shutdown_connections 2024-09-10T21:12:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.840+0000 7f10a5000640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f107c0725e0 0x7f107c074aa0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.840+0000 7f10a5000640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10a00ca170 0x7f10a0155cb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.840+0000 7f10a5000640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10a00b5050 0x7f10a01f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.840+0000 7f10a5000640 1 --2- 172.21.15.16:0/2591449919 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10a006e640 0x7f10a0158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.840+0000 7f10a5000640 1 -- 172.21.15.16:0/2591449919 >> 172.21.15.16:0/2591449919 conn(0x7f10a00cfb20 msgr2=0x7f10a00736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.840+0000 7f10a5000640 1 -- 172.21.15.16:0/2591449919 shutdown_connections 2024-09-10T21:12:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:38.840+0000 7f10a5000640 1 -- 172.21.15.16:0/2591449919 wait complete. 2024-09-10T21:12:38.865 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T21:12:38.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:38.866 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T21:12:38.894 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:12:38.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:12:38.894 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T21:12:38.950 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:12:38.950 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:12:38.950 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:38.959 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:12:38.959 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:12:38.959 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:12:38.959 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:39.005 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:12:39.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:39.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.144+0000 7f5a93200640 1 -- 172.21.15.16:0/3777901904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940b2c20 msgr2=0x7f5a940d0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:39.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.144+0000 7f5a93200640 1 --2- 172.21.15.16:0/3777901904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940b2c20 0x7f5a940d0740 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5a9408e590 tx=0x7f5a7c01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.144+0000 7f5a93200640 1 -- 172.21.15.16:0/3777901904 shutdown_connections 2024-09-10T21:12:39.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.144+0000 7f5a93200640 1 --2- 172.21.15.16:0/3777901904 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5a940d0c80 0x7f5a940cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.144+0000 7f5a93200640 1 --2- 172.21.15.16:0/3777901904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940b2c20 0x7f5a940d0740 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.144+0000 7f5a93200640 1 --2- 172.21.15.16:0/3777901904 >> v2:172.21.15.52:3300/0 conn(0x7f5a94161560 0x7f5a940d4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.144+0000 7f5a93200640 1 -- 172.21.15.16:0/3777901904 >> 172.21.15.16:0/3777901904 conn(0x7f5a940d3f00 msgr2=0x7f5a940d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:39.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.144+0000 7f5a93200640 1 -- 172.21.15.16:0/3777901904 shutdown_connections 2024-09-10T21:12:39.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.144+0000 7f5a93200640 1 -- 172.21.15.16:0/3777901904 wait complete. 2024-09-10T21:12:39.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a93200640 1 Processor -- start 2024-09-10T21:12:39.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a93200640 1 -- start start 2024-09-10T21:12:39.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a93200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a940b2c20 0x7f5a94154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:39.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a93200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5a940d0c80 0x7f5a940b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:39.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a93200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a94161560 0x7f5a940b9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:39.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a93200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a940b18e0 con 0x7f5a94161560 2024-09-10T21:12:39.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a93200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5a940b1760 con 0x7f5a940b2c20 2024-09-10T21:12:39.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a93200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5a940b1a60 con 0x7f5a940d0c80 2024-09-10T21:12:39.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a92800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a94161560 0x7f5a940b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:39.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a92800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a94161560 0x7f5a940b9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53778/0 (socket says 172.21.15.16:53778) 2024-09-10T21:12:39.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a92800640 1 -- 172.21.15.16:0/2370633483 learned_addr learned my addr 172.21.15.16:0/2370633483 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:39.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a91e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a940b2c20 0x7f5a94154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:39.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a8be00640 1 --2- 172.21.15.16:0/2370633483 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5a940d0c80 0x7f5a940b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:39.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a92800640 1 -- 172.21.15.16:0/2370633483 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5a940d0c80 msgr2=0x7f5a940b8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:39.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a92800640 1 --2- 172.21.15.16:0/2370633483 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5a940d0c80 0x7f5a940b8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a92800640 1 -- 172.21.15.16:0/2370633483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a940b2c20 msgr2=0x7f5a94154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:39.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a92800640 1 --2- 172.21.15.16:0/2370633483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a940b2c20 0x7f5a94154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a92800640 1 -- 172.21.15.16:0/2370633483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a940cd670 con 0x7f5a94161560 2024-09-10T21:12:39.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a92800640 1 --2- 172.21.15.16:0/2370633483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a94161560 0x7f5a940b9420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00e990 tx=0x7f5a8c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:39.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a8a000640 1 -- 172.21.15.16:0/2370633483 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c004d30 con 0x7f5a94161560 2024-09-10T21:12:39.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a8a000640 1 -- 172.21.15.16:0/2370633483 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a8c00cb00 con 0x7f5a94161560 2024-09-10T21:12:39.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a93200640 1 -- 172.21.15.16:0/2370633483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a940cd950 con 0x7f5a94161560 2024-09-10T21:12:39.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a8a000640 1 -- 172.21.15.16:0/2370633483 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c005020 con 0x7f5a94161560 2024-09-10T21:12:39.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.148+0000 7f5a93200640 1 -- 172.21.15.16:0/2370633483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a940cdf10 con 0x7f5a94161560 2024-09-10T21:12:39.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.152+0000 7f5a93200640 1 -- 172.21.15.16:0/2370633483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a940b1f80 con 0x7f5a94161560 2024-09-10T21:12:39.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.152+0000 7f5a8a000640 1 -- 172.21.15.16:0/2370633483 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5a8c00f040 con 0x7f5a94161560 2024-09-10T21:12:39.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.152+0000 7f5a8a000640 1 --2- 172.21.15.16:0/2370633483 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5a700725e0 0x7f5a70074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:39.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.152+0000 7f5a8a000640 1 -- 172.21.15.16:0/2370633483 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f5a8c093730 con 0x7f5a94161560 2024-09-10T21:12:39.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.152+0000 7f5a91e00640 1 --2- 172.21.15.16:0/2370633483 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5a700725e0 0x7f5a70074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:39.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.152+0000 7f5a91e00640 1 --2- 172.21.15.16:0/2370633483 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5a700725e0 0x7f5a70074aa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400bb50 tx=0x7f5a8400b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:39.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.156+0000 7f5a8a000640 1 -- 172.21.15.16:0/2370633483 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5a8c016030 con 0x7f5a94161560 2024-09-10T21:12:39.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a93200640 1 -- 172.21.15.16:0/2370633483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a9415c7c0 con 0x7f5a94161560 2024-09-10T21:12:39.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a8a000640 1 -- 172.21.15.16:0/2370633483 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5235 (secure 0 0 0) 0x7f5a8c05e830 con 0x7f5a94161560 2024-09-10T21:12:39.375 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-10T21:12:27:803754+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3694613625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3694613625},{"type":"v1","addr":"172.21.15.16:6836","nonce":3694613625}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1264024538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1264024538},{"type":"v1","addr":"172.21.15.52:6835","nonce":1264024538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":134768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2899191048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2899191048},{"type":"v1","addr":"172.21.15.16:6837","nonce":2899191048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:12:24.201810+0000","modified":"2024-09-10T21:12:27.207587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134771":{"gid":134771,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1834750661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1834750661},{"type":"v1","addr":"172.21.15.52:6834","nonce":1834750661}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134771,"qdb_cluster":[134771]},"id":3}]} 2024-09-10T21:12:39.375 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-10T21:12:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a93200640 1 -- 172.21.15.16:0/2370633483 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5a700725e0 msgr2=0x7f5a70074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a93200640 1 --2- 172.21.15.16:0/2370633483 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5a700725e0 0x7f5a70074aa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400bb50 tx=0x7f5a8400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a93200640 1 -- 172.21.15.16:0/2370633483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a94161560 msgr2=0x7f5a940b9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a93200640 1 --2- 172.21.15.16:0/2370633483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a94161560 0x7f5a940b9420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00e990 tx=0x7f5a8c00ee50 comp rx=0 tx=0).stop 2024-09-10T21:12:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a93200640 1 -- 172.21.15.16:0/2370633483 shutdown_connections 2024-09-10T21:12:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a93200640 1 --2- 172.21.15.16:0/2370633483 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5a700725e0 0x7f5a70074aa0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a93200640 1 --2- 172.21.15.16:0/2370633483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a94161560 0x7f5a940b9420 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a93200640 1 --2- 172.21.15.16:0/2370633483 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5a940d0c80 0x7f5a940b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a93200640 1 --2- 172.21.15.16:0/2370633483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a940b2c20 0x7f5a94154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a93200640 1 -- 172.21.15.16:0/2370633483 >> 172.21.15.16:0/2370633483 conn(0x7f5a940d3f00 msgr2=0x7f5a940b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a93200640 1 -- 172.21.15.16:0/2370633483 shutdown_connections 2024-09-10T21:12:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.368+0000 7f5a93200640 1 -- 172.21.15.16:0/2370633483 wait complete. 2024-09-10T21:12:39.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.536+0000 7fb743200640 1 -- 172.21.15.16:0/3314755733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb720004d50 msgr2=0x7fb7200051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.536+0000 7fb743200640 1 --2- 172.21.15.16:0/3314755733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb720004d50 0x7fb7200051d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb734009ee0 tx=0x7fb73401d630 comp rx=0 tx=0).stop 2024-09-10T21:12:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.536+0000 7fb741200640 1 -- 172.21.15.16:0/3314755733 reap_dead start 2024-09-10T21:12:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.536+0000 7fb743200640 1 -- 172.21.15.16:0/3314755733 shutdown_connections 2024-09-10T21:12:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.536+0000 7fb743200640 1 -- 172.21.15.16:0/3314755733 >> 172.21.15.16:0/3314755733 conn(0x7fb73c0cfb20 msgr2=0x7fb73c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.536+0000 7fb743200640 1 -- 172.21.15.16:0/3314755733 shutdown_connections 2024-09-10T21:12:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.536+0000 7fb743200640 1 -- 172.21.15.16:0/3314755733 wait complete. 2024-09-10T21:12:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb743200640 1 Processor -- start 2024-09-10T21:12:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb743200640 1 -- start start 2024-09-10T21:12:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb743200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb720004d50 0x7fb73c0c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb743200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c0c36c0 0x7fb73c1f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb743200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb73c1f8640 0x7fb73c1f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb743200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c157ef0 con 0x7fb720004d50 2024-09-10T21:12:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb743200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c157d70 con 0x7fb73c0c36c0 2024-09-10T21:12:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb743200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb73c0707a0 con 0x7fb73c1f8640 2024-09-10T21:12:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb73be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb720004d50 0x7fb73c0c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb73be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb720004d50 0x7fb73c0c5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53804/0 (socket says 172.21.15.16:53804) 2024-09-10T21:12:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb73be00640 1 -- 172.21.15.16:0/935209941 learned_addr learned my addr 172.21.15.16:0/935209941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb73be00640 1 -- 172.21.15.16:0/935209941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb73c1f8640 msgr2=0x7fb73c1f8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb73be00640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb73c1f8640 0x7fb73c1f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb733e00640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c0c36c0 0x7fb73c1f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb73be00640 1 -- 172.21.15.16:0/935209941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c0c36c0 msgr2=0x7fb73c1f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb73be00640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c0c36c0 0x7fb73c1f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb73be00640 1 -- 172.21.15.16:0/935209941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73c0af610 con 0x7fb720004d50 2024-09-10T21:12:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb733e00640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c0c36c0 0x7fb73c1f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb73be00640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb720004d50 0x7fb73c0c5190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00e540 tx=0x7fb72c00ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb738c00640 1 -- 172.21.15.16:0/935209941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c015070 con 0x7fb720004d50 2024-09-10T21:12:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb738c00640 1 -- 172.21.15.16:0/935209941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb72c010980 con 0x7fb720004d50 2024-09-10T21:12:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb743200640 1 -- 172.21.15.16:0/935209941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73c0c59e0 con 0x7fb720004d50 2024-09-10T21:12:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb738c00640 1 -- 172.21.15.16:0/935209941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c010d90 con 0x7fb720004d50 2024-09-10T21:12:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.540+0000 7fb743200640 1 -- 172.21.15.16:0/935209941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb73c0c5c80 con 0x7fb720004d50 2024-09-10T21:12:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.544+0000 7fb743200640 1 -- 172.21.15.16:0/935209941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb73c0ca550 con 0x7fb720004d50 2024-09-10T21:12:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.544+0000 7fb738c00640 1 -- 172.21.15.16:0/935209941 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb72c011460 con 0x7fb720004d50 2024-09-10T21:12:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.544+0000 7fb738c00640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7180725e0 0x7fb718074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.544+0000 7fb738c00640 1 -- 172.21.15.16:0/935209941 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fb72c0939d0 con 0x7fb720004d50 2024-09-10T21:12:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.544+0000 7fb733e00640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7180725e0 0x7fb718074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.544+0000 7fb733e00640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7180725e0 0x7fb718074aa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb73c1f5c40 tx=0x7fb72400c040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.548+0000 7fb738c00640 1 -- 172.21.15.16:0/935209941 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb72c05f010 con 0x7fb720004d50 2024-09-10T21:12:39.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.760+0000 7fb743200640 1 -- 172.21.15.16:0/935209941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb73c0b5aa0 con 0x7fb720004d50 2024-09-10T21:12:39.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.760+0000 7fb738c00640 1 -- 172.21.15.16:0/935209941 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5235 (secure 0 0 0) 0x7fb72c05e910 con 0x7fb720004d50 2024-09-10T21:12:39.769 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:39.769 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-10T21:12:27:803754+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3694613625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3694613625},{"type":"v1","addr":"172.21.15.16:6836","nonce":3694613625}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1264024538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1264024538},{"type":"v1","addr":"172.21.15.52:6835","nonce":1264024538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":134768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2899191048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2899191048},{"type":"v1","addr":"172.21.15.16:6837","nonce":2899191048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:12:24.201810+0000","modified":"2024-09-10T21:12:27.207587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134771":{"gid":134771,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1834750661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1834750661},{"type":"v1","addr":"172.21.15.52:6834","nonce":1834750661}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134771,"qdb_cluster":[134771]},"id":3}]} 2024-09-10T21:12:39.769 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-10T21:12:39.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.764+0000 7fb743200640 1 -- 172.21.15.16:0/935209941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7180725e0 msgr2=0x7fb718074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:39.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.764+0000 7fb743200640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7180725e0 0x7fb718074aa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb73c1f5c40 tx=0x7fb72400c040 comp rx=0 tx=0).stop 2024-09-10T21:12:39.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.764+0000 7fb743200640 1 -- 172.21.15.16:0/935209941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb720004d50 msgr2=0x7fb73c0c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:39.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.764+0000 7fb743200640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb720004d50 0x7fb73c0c5190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00e540 tx=0x7fb72c00ea00 comp rx=0 tx=0).stop 2024-09-10T21:12:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.764+0000 7fb743200640 1 -- 172.21.15.16:0/935209941 shutdown_connections 2024-09-10T21:12:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.764+0000 7fb743200640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7180725e0 0x7fb718074aa0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.764+0000 7fb743200640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb73c1f8640 0x7fb73c1f8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.764+0000 7fb743200640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb73c0c36c0 0x7fb73c1f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.764+0000 7fb743200640 1 --2- 172.21.15.16:0/935209941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb720004d50 0x7fb73c0c5190 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.764+0000 7fb743200640 1 -- 172.21.15.16:0/935209941 >> 172.21.15.16:0/935209941 conn(0x7fb73c0cfb20 msgr2=0x7fb73c1591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.764+0000 7fb743200640 1 -- 172.21.15.16:0/935209941 shutdown_connections 2024-09-10T21:12:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.764+0000 7fb743200640 1 -- 172.21.15.16:0/935209941 wait complete. 2024-09-10T21:12:39.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:12:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.932+0000 7f2893a00640 1 -- 172.21.15.16:0/3197302283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288c15f400 msgr2=0x7f288c070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.932+0000 7f2893a00640 1 --2- 172.21.15.16:0/3197302283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288c15f400 0x7f288c070fe0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f288c0698f0 tx=0x7f288001c780 comp rx=0 tx=0).stop 2024-09-10T21:12:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.932+0000 7f2893a00640 1 -- 172.21.15.16:0/3197302283 shutdown_connections 2024-09-10T21:12:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.932+0000 7f2893a00640 1 --2- 172.21.15.16:0/3197302283 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f288c0cfc80 0x7f288c0d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.932+0000 7f2893a00640 1 --2- 172.21.15.16:0/3197302283 >> v2:172.21.15.52:3300/0 conn(0x7f288c0b13c0 0x7f288c071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.932+0000 7f2893a00640 1 --2- 172.21.15.16:0/3197302283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288c15f400 0x7f288c070fe0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.932+0000 7f2893a00640 1 -- 172.21.15.16:0/3197302283 >> 172.21.15.16:0/3197302283 conn(0x7f288c09b130 msgr2=0x7f288c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.932+0000 7f2893a00640 1 -- 172.21.15.16:0/3197302283 shutdown_connections 2024-09-10T21:12:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.932+0000 7f2893a00640 1 -- 172.21.15.16:0/3197302283 wait complete. 2024-09-10T21:12:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.932+0000 7f2893a00640 1 Processor -- start 2024-09-10T21:12:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.932+0000 7f2893a00640 1 -- start start 2024-09-10T21:12:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2893a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288c0b13c0 0x7f288c14b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2893a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f288c0cfc80 0x7f288c1fb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2893a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f288c15f400 0x7f288c1fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2893a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f288c0afd90 con 0x7f288c0b13c0 2024-09-10T21:12:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2893a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f288c0d1260 con 0x7f288c0cfc80 2024-09-10T21:12:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2893a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f288c0aff10 con 0x7f288c15f400 2024-09-10T21:12:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2891000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288c0b13c0 0x7f288c14b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2891000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288c0b13c0 0x7f288c14b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53810/0 (socket says 172.21.15.16:53810) 2024-09-10T21:12:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2891000640 1 -- 172.21.15.16:0/1060906542 learned_addr learned my addr 172.21.15.16:0/1060906542 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f288be00640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f288c0cfc80 0x7f288c1fb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2891a00640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f288c15f400 0x7f288c1fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2891000640 1 -- 172.21.15.16:0/1060906542 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f288c15f400 msgr2=0x7f288c1fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2891000640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f288c15f400 0x7f288c1fb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2891000640 1 -- 172.21.15.16:0/1060906542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f288c0cfc80 msgr2=0x7f288c1fb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2891000640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f288c0cfc80 0x7f288c1fb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2891000640 1 -- 172.21.15.16:0/1060906542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c0cc170 con 0x7f288c0b13c0 2024-09-10T21:12:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f288be00640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f288c0cfc80 0x7f288c1fb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2891000640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288c0b13c0 0x7f288c14b8d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2880000c10 tx=0x7f28800a67d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2888c00640 1 -- 172.21.15.16:0/1060906542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28800bd020 con 0x7f288c0b13c0 2024-09-10T21:12:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2888c00640 1 -- 172.21.15.16:0/1060906542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2880004450 con 0x7f288c0b13c0 2024-09-10T21:12:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2888c00640 1 -- 172.21.15.16:0/1060906542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880003860 con 0x7f288c0b13c0 2024-09-10T21:12:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2893a00640 1 -- 172.21.15.16:0/1060906542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288c0c92a0 con 0x7f288c0b13c0 2024-09-10T21:12:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2893a00640 1 -- 172.21.15.16:0/1060906542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f288c0cb970 con 0x7f288c0b13c0 2024-09-10T21:12:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.936+0000 7f2893a00640 1 -- 172.21.15.16:0/1060906542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288c0d0110 con 0x7f288c0b13c0 2024-09-10T21:12:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.940+0000 7f2888c00640 1 -- 172.21.15.16:0/1060906542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2880003a00 con 0x7f288c0b13c0 2024-09-10T21:12:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.940+0000 7f2888c00640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f286c0725e0 0x7f286c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.940+0000 7f2888c00640 1 -- 172.21.15.16:0/1060906542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f288012e9d0 con 0x7f288c0b13c0 2024-09-10T21:12:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.940+0000 7f288be00640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f286c0725e0 0x7f286c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.940+0000 7f288be00640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f286c0725e0 0x7f286c074aa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f288c15e210 tx=0x7f2878005d30 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:39.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:39.944+0000 7f2888c00640 1 -- 172.21.15.16:0/1060906542 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f28800c7030 con 0x7f288c0b13c0 2024-09-10T21:12:40.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.116+0000 7f2893a00640 1 -- 172.21.15.16:0/1060906542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f288c0af280 con 0x7f288c0b13c0 2024-09-10T21:12:40.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.120+0000 7f2888c00640 1 -- 172.21.15.16:0/1060906542 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16621 (secure 0 0 0) 0x7f2880016490 con 0x7f288c0b13c0 2024-09-10T21:12:40.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:40.125 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:25.780513+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:12:22.642253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T21:12:23.151552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:12:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.120+0000 7f2893a00640 1 -- 172.21.15.16:0/1060906542 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f286c0725e0 msgr2=0x7f286c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.120+0000 7f2893a00640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f286c0725e0 0x7f286c074aa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f288c15e210 tx=0x7f2878005d30 comp rx=0 tx=0).stop 2024-09-10T21:12:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.120+0000 7f2893a00640 1 -- 172.21.15.16:0/1060906542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288c0b13c0 msgr2=0x7f288c14b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.120+0000 7f2893a00640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288c0b13c0 0x7f288c14b8d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2880000c10 tx=0x7f28800a67d0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.120+0000 7f2893a00640 1 -- 172.21.15.16:0/1060906542 shutdown_connections 2024-09-10T21:12:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.120+0000 7f2893a00640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f286c0725e0 0x7f286c074aa0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.120+0000 7f2893a00640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f288c15f400 0x7f288c1fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.120+0000 7f2893a00640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f288c0cfc80 0x7f288c1fb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.120+0000 7f2893a00640 1 --2- 172.21.15.16:0/1060906542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f288c0b13c0 0x7f288c14b8d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.120+0000 7f2893a00640 1 -- 172.21.15.16:0/1060906542 >> 172.21.15.16:0/1060906542 conn(0x7f288c09b130 msgr2=0x7f288c0aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.120+0000 7f2893a00640 1 -- 172.21.15.16:0/1060906542 shutdown_connections 2024-09-10T21:12:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.120+0000 7f2893a00640 1 -- 172.21.15.16:0/1060906542 wait complete. 2024-09-10T21:12:40.148 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:12:40.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:40.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.288+0000 7f81f6400640 1 -- 172.21.15.16:0/2606959542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f015f400 msgr2=0x7f81f0070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:40.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.288+0000 7f81f6400640 1 --2- 172.21.15.16:0/2606959542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f015f400 0x7f81f0070fe0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f81e0005fd0 tx=0x7f81e001c700 comp rx=0 tx=0).stop 2024-09-10T21:12:40.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.288+0000 7f81f6400640 1 -- 172.21.15.16:0/2606959542 shutdown_connections 2024-09-10T21:12:40.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.288+0000 7f81f6400640 1 --2- 172.21.15.16:0/2606959542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f015f400 0x7f81f0070fe0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.288+0000 7f81f6400640 1 -- 172.21.15.16:0/2606959542 >> 172.21.15.16:0/2606959542 conn(0x7f81f009b130 msgr2=0x7f81f00d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:40.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.288+0000 7f81f6400640 1 -- 172.21.15.16:0/2606959542 shutdown_connections 2024-09-10T21:12:40.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.288+0000 7f81f6400640 1 -- 172.21.15.16:0/2606959542 wait complete. 2024-09-10T21:12:40.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.288+0000 7f81f6400640 1 Processor -- start 2024-09-10T21:12:40.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81f6400640 1 -- start start 2024-09-10T21:12:40.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81f6400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f81f00b0590 0x7f81f01f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:40.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81f6400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f00b13c0 0x7f81f01fc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:40.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81f6400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f015f400 0x7f81f0158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81f6400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81f016a800 con 0x7f81f00b13c0 2024-09-10T21:12:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81f6400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f81f00d1260 con 0x7f81f015f400 2024-09-10T21:12:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81f6400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f81f016a980 con 0x7f81f00b0590 2024-09-10T21:12:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f00b13c0 0x7f81f01fc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81ef400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f81f00b0590 0x7f81f01f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f00b13c0 0x7f81f01fc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53816/0 (socket says 172.21.15.16:53816) 2024-09-10T21:12:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81e7e00640 1 -- 172.21.15.16:0/2991635861 learned_addr learned my addr 172.21.15.16:0/2991635861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81efe00640 1 --2- 172.21.15.16:0/2991635861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f015f400 0x7f81f0158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81e7e00640 1 -- 172.21.15.16:0/2991635861 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f81f00b0590 msgr2=0x7f81f01f8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81e7e00640 1 --2- 172.21.15.16:0/2991635861 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f81f00b0590 0x7f81f01f8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81e7e00640 1 -- 172.21.15.16:0/2991635861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f015f400 msgr2=0x7f81f0158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81e7e00640 1 --2- 172.21.15.16:0/2991635861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f015f400 0x7f81f0158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81e7e00640 1 -- 172.21.15.16:0/2991635861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f015c300 con 0x7f81f00b13c0 2024-09-10T21:12:40.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81e7e00640 1 --2- 172.21.15.16:0/2991635861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f00b13c0 0x7f81f01fc050 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f81d800c4b0 tx=0x7f81d800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:40.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81e7400640 1 -- 172.21.15.16:0/2991635861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d80052d0 con 0x7f81f00b13c0 2024-09-10T21:12:40.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81e7400640 1 -- 172.21.15.16:0/2991635861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81d8005470 con 0x7f81f00b13c0 2024-09-10T21:12:40.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81e7400640 1 -- 172.21.15.16:0/2991635861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8012870 con 0x7f81f00b13c0 2024-09-10T21:12:40.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81f6400640 1 -- 172.21.15.16:0/2991635861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f01f8160 con 0x7f81f00b13c0 2024-09-10T21:12:40.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81f6400640 1 -- 172.21.15.16:0/2991635861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f015c5e0 con 0x7f81f00b13c0 2024-09-10T21:12:40.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.292+0000 7f81f6400640 1 -- 172.21.15.16:0/2991635861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f015cc70 con 0x7f81f00b13c0 2024-09-10T21:12:40.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.296+0000 7f81e7400640 1 -- 172.21.15.16:0/2991635861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f81d8003600 con 0x7f81f00b13c0 2024-09-10T21:12:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.296+0000 7f81e7400640 1 --2- 172.21.15.16:0/2991635861 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f81cc072610 0x7f81cc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.296+0000 7f81e7400640 1 -- 172.21.15.16:0/2991635861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f81d8092c60 con 0x7f81f00b13c0 2024-09-10T21:12:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.296+0000 7f81ef400640 1 --2- 172.21.15.16:0/2991635861 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f81cc072610 0x7f81cc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.296+0000 7f81ef400640 1 --2- 172.21.15.16:0/2991635861 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f81cc072610 0x7f81cc074ad0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f81e00ae000 tx=0x7f81e00b2040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.300+0000 7f81e7400640 1 -- 172.21.15.16:0/2991635861 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f81d805dce0 con 0x7f81f00b13c0 2024-09-10T21:12:40.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.512+0000 7f81f6400640 1 -- 172.21.15.16:0/2991635861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81f00b33b0 con 0x7f81f00b13c0 2024-09-10T21:12:40.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.512+0000 7f81e7400640 1 -- 172.21.15.16:0/2991635861 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5235 (secure 0 0 0) 0x7f81d8062710 con 0x7f81f00b13c0 2024-09-10T21:12:40.521 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:40.521 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-10T21:12:27:803754+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3694613625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3694613625},{"type":"v1","addr":"172.21.15.16:6836","nonce":3694613625}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1264024538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1264024538},{"type":"v1","addr":"172.21.15.52:6835","nonce":1264024538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":134768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2899191048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2899191048},{"type":"v1","addr":"172.21.15.16:6837","nonce":2899191048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:12:24.201810+0000","modified":"2024-09-10T21:12:27.207587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134771":{"gid":134771,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1834750661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1834750661},{"type":"v1","addr":"172.21.15.52:6834","nonce":1834750661}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134771,"qdb_cluster":[134771]},"id":3}]} 2024-09-10T21:12:40.521 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-10T21:12:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.516+0000 7f81f6400640 1 -- 172.21.15.16:0/2991635861 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f81cc072610 msgr2=0x7f81cc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.516+0000 7f81f6400640 1 --2- 172.21.15.16:0/2991635861 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f81cc072610 0x7f81cc074ad0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f81e00ae000 tx=0x7f81e00b2040 comp rx=0 tx=0).stop 2024-09-10T21:12:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.516+0000 7f81f6400640 1 -- 172.21.15.16:0/2991635861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f00b13c0 msgr2=0x7f81f01fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.516+0000 7f81f6400640 1 --2- 172.21.15.16:0/2991635861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f00b13c0 0x7f81f01fc050 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f81d800c4b0 tx=0x7f81d800c970 comp rx=0 tx=0).stop 2024-09-10T21:12:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.516+0000 7f81f6400640 1 -- 172.21.15.16:0/2991635861 shutdown_connections 2024-09-10T21:12:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.516+0000 7f81f6400640 1 --2- 172.21.15.16:0/2991635861 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f81cc072610 0x7f81cc074ad0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.516+0000 7f81f6400640 1 --2- 172.21.15.16:0/2991635861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f015f400 0x7f81f0158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.516+0000 7f81f6400640 1 --2- 172.21.15.16:0/2991635861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f00b13c0 0x7f81f01fc050 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.516+0000 7f81f6400640 1 --2- 172.21.15.16:0/2991635861 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f81f00b0590 0x7f81f01f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.516+0000 7f81f6400640 1 -- 172.21.15.16:0/2991635861 >> 172.21.15.16:0/2991635861 conn(0x7f81f009b130 msgr2=0x7f81f00afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:40.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.516+0000 7f81f6400640 1 -- 172.21.15.16:0/2991635861 shutdown_connections 2024-09-10T21:12:40.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.516+0000 7f81f6400640 1 -- 172.21.15.16:0/2991635861 wait complete. 2024-09-10T21:12:40.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:40.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 -- 172.21.15.16:0/1485431368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4440b1b40 msgr2=0x7fe4440b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:40.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 --2- 172.21.15.16:0/1485431368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4440b1b40 0x7fe4440b1f80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009bc0 tx=0x7fe43c01cb10 comp rx=0 tx=0).stop 2024-09-10T21:12:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 -- 172.21.15.16:0/1485431368 shutdown_connections 2024-09-10T21:12:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 --2- 172.21.15.16:0/1485431368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4440b1b40 0x7fe4440b1f80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 --2- 172.21.15.16:0/1485431368 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe4440d2820 0x7fe4440b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 -- 172.21.15.16:0/1485431368 >> 172.21.15.16:0/1485431368 conn(0x7fe4440d1d00 msgr2=0x7fe4440d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 -- 172.21.15.16:0/1485431368 shutdown_connections 2024-09-10T21:12:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 -- 172.21.15.16:0/1485431368 wait complete. 2024-09-10T21:12:40.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 Processor -- start 2024-09-10T21:12:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 -- start start 2024-09-10T21:12:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe4440b1b40 0x7fe4440b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4440cfd40 0x7fe4441500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4440d2820 0x7fe444071770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440b06c0 con 0x7fe4440cfd40 2024-09-10T21:12:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440b0540 con 0x7fe4440d2820 2024-09-10T21:12:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.684+0000 7fe449800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe444168eb0 con 0x7fe4440b1b40 2024-09-10T21:12:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe43be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4440cfd40 0x7fe4441500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe43be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4440cfd40 0x7fe4441500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53828/0 (socket says 172.21.15.16:53828) 2024-09-10T21:12:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe43be00640 1 -- 172.21.15.16:0/2798666726 learned_addr learned my addr 172.21.15.16:0/2798666726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe443e00640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe4440b1b40 0x7fe4440b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe448e00640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4440d2820 0x7fe444071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe43be00640 1 -- 172.21.15.16:0/2798666726 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe4440b1b40 msgr2=0x7fe4440b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe43be00640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe4440b1b40 0x7fe4440b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe43be00640 1 -- 172.21.15.16:0/2798666726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4440d2820 msgr2=0x7fe444071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe43be00640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4440d2820 0x7fe444071770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe43be00640 1 -- 172.21.15.16:0/2798666726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4440b5b40 con 0x7fe4440cfd40 2024-09-10T21:12:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe443e00640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe4440b1b40 0x7fe4440b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe448e00640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4440d2820 0x7fe444071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe43be00640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4440cfd40 0x7fe4441500e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00d950 tx=0x7fe42c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe440c00640 1 -- 172.21.15.16:0/2798666726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c017070 con 0x7fe4440cfd40 2024-09-10T21:12:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe440c00640 1 -- 172.21.15.16:0/2798666726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe42c009d20 con 0x7fe4440cfd40 2024-09-10T21:12:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe440c00640 1 -- 172.21.15.16:0/2798666726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c0106c0 con 0x7fe4440cfd40 2024-09-10T21:12:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe449800640 1 -- 172.21.15.16:0/2798666726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4440b52a0 con 0x7fe4440cfd40 2024-09-10T21:12:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe449800640 1 -- 172.21.15.16:0/2798666726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4440b5860 con 0x7fe4440cfd40 2024-09-10T21:12:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe449800640 1 -- 172.21.15.16:0/2798666726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4440b1f80 con 0x7fe4440cfd40 2024-09-10T21:12:40.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.688+0000 7fe440c00640 1 -- 172.21.15.16:0/2798666726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe42c010860 con 0x7fe4440cfd40 2024-09-10T21:12:40.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.692+0000 7fe440c00640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4200725e0 0x7fe420074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:40.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.692+0000 7fe440c00640 1 -- 172.21.15.16:0/2798666726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fe42c093770 con 0x7fe4440cfd40 2024-09-10T21:12:40.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.692+0000 7fe443e00640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4200725e0 0x7fe420074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.692+0000 7fe443e00640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4200725e0 0x7fe420074aa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe4340016d0 tx=0x7fe434009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.696+0000 7fe440c00640 1 -- 172.21.15.16:0/2798666726 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe42c05edb0 con 0x7fe4440cfd40 2024-09-10T21:12:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.908+0000 7fe449800640 1 -- 172.21.15.16:0/2798666726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4440d2280 con 0x7fe4440cfd40 2024-09-10T21:12:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.908+0000 7fe440c00640 1 -- 172.21.15.16:0/2798666726 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5235 (secure 0 0 0) 0x7fe42c05e6b0 con 0x7fe4440cfd40 2024-09-10T21:12:40.917 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-10T21:12:27:803754+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3694613625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3694613625},{"type":"v1","addr":"172.21.15.16:6836","nonce":3694613625}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1264024538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1264024538},{"type":"v1","addr":"172.21.15.52:6835","nonce":1264024538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":134768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2899191048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2899191048},{"type":"v1","addr":"172.21.15.16:6837","nonce":2899191048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:12:24.201810+0000","modified":"2024-09-10T21:12:27.207587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":134771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_134771":{"gid":134771,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1834750661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1834750661},{"type":"v1","addr":"172.21.15.52:6834","nonce":1834750661}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":134771,"qdb_cluster":[134771]},"id":3}]} 2024-09-10T21:12:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-10T21:12:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.912+0000 7fe449800640 1 -- 172.21.15.16:0/2798666726 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4200725e0 msgr2=0x7fe420074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.912+0000 7fe449800640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4200725e0 0x7fe420074aa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe4340016d0 tx=0x7fe434009040 comp rx=0 tx=0).stop 2024-09-10T21:12:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.912+0000 7fe449800640 1 -- 172.21.15.16:0/2798666726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4440cfd40 msgr2=0x7fe4441500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.912+0000 7fe449800640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4440cfd40 0x7fe4441500e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00d950 tx=0x7fe42c00de10 comp rx=0 tx=0).stop 2024-09-10T21:12:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.912+0000 7fe449800640 1 -- 172.21.15.16:0/2798666726 shutdown_connections 2024-09-10T21:12:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.912+0000 7fe449800640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4200725e0 0x7fe420074aa0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.912+0000 7fe449800640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4440d2820 0x7fe444071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.912+0000 7fe449800640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4440cfd40 0x7fe4441500e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.912+0000 7fe449800640 1 --2- 172.21.15.16:0/2798666726 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe4440b1b40 0x7fe4440b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.912+0000 7fe449800640 1 -- 172.21.15.16:0/2798666726 >> 172.21.15.16:0/2798666726 conn(0x7fe4440d1d00 msgr2=0x7fe44415f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.912+0000 7fe449800640 1 -- 172.21.15.16:0/2798666726 shutdown_connections 2024-09-10T21:12:40.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:40.912+0000 7fe449800640 1 -- 172.21.15.16:0/2798666726 wait complete. 2024-09-10T21:12:40.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:12:41.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.076+0000 7f5a21c00640 1 -- 172.21.15.16:0/3196318768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c15f400 msgr2=0x7f5a1c070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.076+0000 7f5a21c00640 1 --2- 172.21.15.16:0/3196318768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c15f400 0x7f5a1c070fe0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009bb0 tx=0x7f5a0c01ca80 comp rx=0 tx=0).stop 2024-09-10T21:12:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.076+0000 7f5a21c00640 1 -- 172.21.15.16:0/3196318768 shutdown_connections 2024-09-10T21:12:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.076+0000 7f5a21c00640 1 --2- 172.21.15.16:0/3196318768 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5a1c0b0590 0x7f5a1c071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.076+0000 7f5a21c00640 1 --2- 172.21.15.16:0/3196318768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c15f400 0x7f5a1c070fe0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.076+0000 7f5a21c00640 1 -- 172.21.15.16:0/3196318768 >> 172.21.15.16:0/3196318768 conn(0x7f5a1c09b130 msgr2=0x7f5a1c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.076+0000 7f5a21c00640 1 -- 172.21.15.16:0/3196318768 shutdown_connections 2024-09-10T21:12:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.076+0000 7f5a21c00640 1 -- 172.21.15.16:0/3196318768 wait complete. 2024-09-10T21:12:41.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a21c00640 1 Processor -- start 2024-09-10T21:12:41.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a21c00640 1 -- start start 2024-09-10T21:12:41.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a21c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a1c0b0590 0x7f5a1c1fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:41.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a21c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c0b13c0 0x7f5a1c14ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:41.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a21c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5a1c15f400 0x7f5a1c15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a21c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c168f60 con 0x7f5a1c0b13c0 2024-09-10T21:12:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a21c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c168de0 con 0x7f5a1c0b0590 2024-09-10T21:12:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a21c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1c1690e0 con 0x7f5a1c15f400 2024-09-10T21:12:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a12000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c0b13c0 0x7f5a1c14ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a12000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c0b13c0 0x7f5a1c14ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53858/0 (socket says 172.21.15.16:53858) 2024-09-10T21:12:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a12000640 1 -- 172.21.15.16:0/4292905132 learned_addr learned my addr 172.21.15.16:0/4292905132 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a1aa00640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a1c0b0590 0x7f5a1c1fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a1b400640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5a1c15f400 0x7f5a1c15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a12000640 1 -- 172.21.15.16:0/4292905132 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5a1c15f400 msgr2=0x7f5a1c15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a12000640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5a1c15f400 0x7f5a1c15b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a12000640 1 -- 172.21.15.16:0/4292905132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a1c0b0590 msgr2=0x7f5a1c1fb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:41.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a12000640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a1c0b0590 0x7f5a1c1fb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a12000640 1 -- 172.21.15.16:0/4292905132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c14f2d0 con 0x7f5a1c0b13c0 2024-09-10T21:12:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a1aa00640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a1c0b0590 0x7f5a1c1fb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a12000640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c0b13c0 0x7f5a1c14ed90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c500 tx=0x7f5a0400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a13400640 1 -- 172.21.15.16:0/4292905132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04017070 con 0x7f5a1c0b13c0 2024-09-10T21:12:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a13400640 1 -- 172.21.15.16:0/4292905132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a04003600 con 0x7f5a1c0b13c0 2024-09-10T21:12:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a21c00640 1 -- 172.21.15.16:0/4292905132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c0b6740 con 0x7f5a1c0b13c0 2024-09-10T21:12:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a13400640 1 -- 172.21.15.16:0/4292905132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04003a90 con 0x7f5a1c0b13c0 2024-09-10T21:12:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.080+0000 7f5a21c00640 1 -- 172.21.15.16:0/4292905132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c0b6d00 con 0x7f5a1c0b13c0 2024-09-10T21:12:41.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.084+0000 7f5a21c00640 1 -- 172.21.15.16:0/4292905132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1c0cfc80 con 0x7f5a1c0b13c0 2024-09-10T21:12:41.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.084+0000 7f5a13400640 1 -- 172.21.15.16:0/4292905132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5a04003c30 con 0x7f5a1c0b13c0 2024-09-10T21:12:41.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.084+0000 7f5a13400640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59f8072610 0x7f59f8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:41.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.084+0000 7f5a13400640 1 -- 172.21.15.16:0/4292905132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f5a040929d0 con 0x7f5a1c0b13c0 2024-09-10T21:12:41.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.084+0000 7f5a1aa00640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59f8072610 0x7f59f8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:41.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.084+0000 7f5a1aa00640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59f8072610 0x7f59f8074ad0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0af000 tx=0x7f5a0c01b2c0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:41.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.088+0000 7f5a13400640 1 -- 172.21.15.16:0/4292905132 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5a0405e160 con 0x7f5a1c0b13c0 2024-09-10T21:12:41.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.264+0000 7f5a21c00640 1 -- 172.21.15.16:0/4292905132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a1c0ce990 con 0x7f5a1c0b13c0 2024-09-10T21:12:41.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.264+0000 7f5a13400640 1 -- 172.21.15.16:0/4292905132 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16621 (secure 0 0 0) 0x7f5a0405da60 con 0x7f5a1c0b13c0 2024-09-10T21:12:41.272 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:41.272 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:25.780513+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:12:22.642253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T21:12:23.151552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:12:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.268+0000 7f5a21c00640 1 -- 172.21.15.16:0/4292905132 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59f8072610 msgr2=0x7f59f8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.268+0000 7f5a21c00640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59f8072610 0x7f59f8074ad0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0af000 tx=0x7f5a0c01b2c0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.268+0000 7f5a21c00640 1 -- 172.21.15.16:0/4292905132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c0b13c0 msgr2=0x7f5a1c14ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.268+0000 7f5a21c00640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c0b13c0 0x7f5a1c14ed90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c500 tx=0x7f5a0400c9c0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.268+0000 7f5a21c00640 1 -- 172.21.15.16:0/4292905132 shutdown_connections 2024-09-10T21:12:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.268+0000 7f5a21c00640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59f8072610 0x7f59f8074ad0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.268+0000 7f5a21c00640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5a1c15f400 0x7f5a1c15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.268+0000 7f5a21c00640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c0b13c0 0x7f5a1c14ed90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.268+0000 7f5a21c00640 1 --2- 172.21.15.16:0/4292905132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a1c0b0590 0x7f5a1c1fb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.268+0000 7f5a21c00640 1 -- 172.21.15.16:0/4292905132 >> 172.21.15.16:0/4292905132 conn(0x7f5a1c09b130 msgr2=0x7f5a1c0b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:41.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.268+0000 7f5a21c00640 1 -- 172.21.15.16:0/4292905132 shutdown_connections 2024-09-10T21:12:41.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.268+0000 7f5a21c00640 1 -- 172.21.15.16:0/4292905132 wait complete. 2024-09-10T21:12:41.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:12:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 -- 172.21.15.16:0/3796771789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c15be10 msgr2=0x7ffa2c0cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 --2- 172.21.15.16:0/3796771789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c15be10 0x7ffa2c0cbe90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c06f000 tx=0x7ffa1c01c800 comp rx=0 tx=0).stop 2024-09-10T21:12:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 -- 172.21.15.16:0/3796771789 shutdown_connections 2024-09-10T21:12:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 --2- 172.21.15.16:0/3796771789 >> v2:172.21.15.52:3300/0 conn(0x7ffa2c0cd070 0x7ffa2c070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 --2- 172.21.15.16:0/3796771789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c15be10 0x7ffa2c0cbe90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 -- 172.21.15.16:0/3796771789 >> 172.21.15.16:0/3796771789 conn(0x7ffa2c0d0c30 msgr2=0x7ffa2c0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 -- 172.21.15.16:0/3796771789 shutdown_connections 2024-09-10T21:12:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 -- 172.21.15.16:0/3796771789 wait complete. 2024-09-10T21:12:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 Processor -- start 2024-09-10T21:12:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 -- start start 2024-09-10T21:12:41.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ffa2c0c42c0 0x7ffa2c0c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:41.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2c0cd070 0x7ffa2c0c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:41.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c15be10 0x7ffa2c0c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:41.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c169330 con 0x7ffa2c15be10 2024-09-10T21:12:41.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1691b0 con 0x7ffa2c0cd070 2024-09-10T21:12:41.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.436+0000 7ffa31200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2c1694b0 con 0x7ffa2c0c42c0 2024-09-10T21:12:41.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa2a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c15be10 0x7ffa2c0c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa2a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c15be10 0x7ffa2c0c6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53882/0 (socket says 172.21.15.16:53882) 2024-09-10T21:12:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa2a800640 1 -- 172.21.15.16:0/3968231024 learned_addr learned my addr 172.21.15.16:0/3968231024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa23e00640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2c0cd070 0x7ffa2c0c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa29e00640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ffa2c0c42c0 0x7ffa2c0c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa2a800640 1 -- 172.21.15.16:0/3968231024 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ffa2c0c42c0 msgr2=0x7ffa2c0c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa2a800640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ffa2c0c42c0 0x7ffa2c0c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa2a800640 1 -- 172.21.15.16:0/3968231024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2c0cd070 msgr2=0x7ffa2c0c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa2a800640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2c0cd070 0x7ffa2c0c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa2a800640 1 -- 172.21.15.16:0/3968231024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c1546a0 con 0x7ffa2c15be10 2024-09-10T21:12:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa23e00640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2c0cd070 0x7ffa2c0c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa2a800640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c15be10 0x7ffa2c0c6e80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400cce0 tx=0x7ffa2400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa22000640 1 -- 172.21.15.16:0/3968231024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24003980 con 0x7ffa2c15be10 2024-09-10T21:12:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa22000640 1 -- 172.21.15.16:0/3968231024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa24003c10 con 0x7ffa2c15be10 2024-09-10T21:12:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa31200640 1 -- 172.21.15.16:0/3968231024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2c154980 con 0x7ffa2c15be10 2024-09-10T21:12:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa22000640 1 -- 172.21.15.16:0/3968231024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24004030 con 0x7ffa2c15be10 2024-09-10T21:12:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa31200640 1 -- 172.21.15.16:0/3968231024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2c154210 con 0x7ffa2c15be10 2024-09-10T21:12:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa31200640 1 -- 172.21.15.16:0/3968231024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa2c070b70 con 0x7ffa2c15be10 2024-09-10T21:12:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.440+0000 7ffa22000640 1 -- 172.21.15.16:0/3968231024 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ffa240041d0 con 0x7ffa2c15be10 2024-09-10T21:12:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.444+0000 7ffa22000640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ffa080725e0 0x7ffa08074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.444+0000 7ffa22000640 1 -- 172.21.15.16:0/3968231024 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ffa24093a60 con 0x7ffa2c15be10 2024-09-10T21:12:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.444+0000 7ffa29e00640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ffa080725e0 0x7ffa08074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.444+0000 7ffa29e00640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ffa080725e0 0x7ffa08074aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0af000 tx=0x7ffa1c01b040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.448+0000 7ffa22000640 1 -- 172.21.15.16:0/3968231024 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffa24014070 con 0x7ffa2c15be10 2024-09-10T21:12:41.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.656+0000 7ffa31200640 1 -- 172.21.15.16:0/3968231024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ffa2c0b6bb0 con 0x7ffa2c15be10 2024-09-10T21:12:41.665 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-10T21:12:41.660+0000 7ff429400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:12:41.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.660+0000 7ffa22000640 1 -- 172.21.15.16:0/3968231024 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7ffa2405ea60 con 0x7ffa2c15be10 2024-09-10T21:12:41.667 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:12:41.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.664+0000 7ffa31200640 1 -- 172.21.15.16:0/3968231024 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ffa080725e0 msgr2=0x7ffa08074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:41.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.664+0000 7ffa31200640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ffa080725e0 0x7ffa08074aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0af000 tx=0x7ffa1c01b040 comp rx=0 tx=0).stop 2024-09-10T21:12:41.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.664+0000 7ffa31200640 1 -- 172.21.15.16:0/3968231024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c15be10 msgr2=0x7ffa2c0c6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.664+0000 7ffa31200640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c15be10 0x7ffa2c0c6e80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400cce0 tx=0x7ffa2400a620 comp rx=0 tx=0).stop 2024-09-10T21:12:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.664+0000 7ffa31200640 1 -- 172.21.15.16:0/3968231024 shutdown_connections 2024-09-10T21:12:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.664+0000 7ffa31200640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ffa080725e0 0x7ffa08074aa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.664+0000 7ffa31200640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c15be10 0x7ffa2c0c6e80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.664+0000 7ffa31200640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa2c0cd070 0x7ffa2c0c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.664+0000 7ffa31200640 1 --2- 172.21.15.16:0/3968231024 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ffa2c0c42c0 0x7ffa2c0c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.664+0000 7ffa31200640 1 -- 172.21.15.16:0/3968231024 >> 172.21.15.16:0/3968231024 conn(0x7ffa2c0d0c30 msgr2=0x7ffa2c157900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.664+0000 7ffa31200640 1 -- 172.21.15.16:0/3968231024 shutdown_connections 2024-09-10T21:12:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.664+0000 7ffa31200640 1 -- 172.21.15.16:0/3968231024 wait complete. 2024-09-10T21:12:41.679 INFO:tasks.ceph.mds.d.smithi052.stderr: -2131> 2024-09-10T21:12:30.759+0000 7f0bd1200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:12:41.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:12:41.726 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:12:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.832+0000 7f01e8e00640 1 -- 172.21.15.16:0/3720320705 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01e406f800 msgr2=0x7f01e406fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.832+0000 7f01e8e00640 1 --2- 172.21.15.16:0/3720320705 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01e406f800 0x7f01e406fc70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009c80 tx=0x7f01dc01cda0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.832+0000 7f01e8e00640 1 -- 172.21.15.16:0/3720320705 shutdown_connections 2024-09-10T21:12:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.832+0000 7f01e8e00640 1 --2- 172.21.15.16:0/3720320705 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01e406f800 0x7f01e406fc70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.832+0000 7f01e8e00640 1 --2- 172.21.15.16:0/3720320705 >> v2:172.21.15.52:3300/0 conn(0x7f01e415dd80 0x7f01e415e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.832+0000 7f01e8e00640 1 -- 172.21.15.16:0/3720320705 >> 172.21.15.16:0/3720320705 conn(0x7f01e40b16a0 msgr2=0x7f01e40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.832+0000 7f01e8e00640 1 -- 172.21.15.16:0/3720320705 shutdown_connections 2024-09-10T21:12:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.832+0000 7f01e8e00640 1 -- 172.21.15.16:0/3720320705 wait complete. 2024-09-10T21:12:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.832+0000 7f01e8e00640 1 Processor -- start 2024-09-10T21:12:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.832+0000 7f01e8e00640 1 -- start start 2024-09-10T21:12:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e8e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01e406f800 0x7f01e4146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e8e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01e4151210 0x7f01e415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e415dd80 0x7f01e4165f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e8e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01e406eaf0 con 0x7f01e415dd80 2024-09-10T21:12:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e8e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f01e406e970 con 0x7f01e4151210 2024-09-10T21:12:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e8e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f01e406ec70 con 0x7f01e406f800 2024-09-10T21:12:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e1e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01e406f800 0x7f01e4146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e1e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01e406f800 0x7f01e4146160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:43904/0 (socket says 172.21.15.16:43904) 2024-09-10T21:12:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e1e00640 1 -- 172.21.15.16:0/3610222620 learned_addr learned my addr 172.21.15.16:0/3610222620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e2800640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e415dd80 0x7f01e4165f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01dbe00640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01e4151210 0x7f01e415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e1e00640 1 -- 172.21.15.16:0/3610222620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01e4151210 msgr2=0x7f01e415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e1e00640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01e4151210 0x7f01e415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e1e00640 1 -- 172.21.15.16:0/3610222620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e415dd80 msgr2=0x7f01e4165f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e1e00640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e415dd80 0x7f01e4165f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e1e00640 1 -- 172.21.15.16:0/3610222620 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e4050ef0 con 0x7f01e406f800 2024-09-10T21:12:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01dbe00640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01e4151210 0x7f01e415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:41.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e1e00640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01e406f800 0x7f01e4146160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01d400c400 tx=0x7f01d400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:41.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01da000640 1 -- 172.21.15.16:0/3610222620 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4017070 con 0x7f01e406f800 2024-09-10T21:12:41.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01da000640 1 -- 172.21.15.16:0/3610222620 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01d40053c0 con 0x7f01e406f800 2024-09-10T21:12:41.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01da000640 1 -- 172.21.15.16:0/3610222620 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4003880 con 0x7f01e406f800 2024-09-10T21:12:41.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e8e00640 1 -- 172.21.15.16:0/3610222620 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e415c1d0 con 0x7f01e406f800 2024-09-10T21:12:41.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e8e00640 1 -- 172.21.15.16:0/3610222620 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e41f7e70 con 0x7f01e406f800 2024-09-10T21:12:41.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01e8e00640 1 -- 172.21.15.16:0/3610222620 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e41f79a0 con 0x7f01e406f800 2024-09-10T21:12:41.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.836+0000 7f01da000640 1 -- 172.21.15.16:0/3610222620 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f01d4037a50 con 0x7f01e406f800 2024-09-10T21:12:41.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.840+0000 7f01da000640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f01c00725e0 0x7f01c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:41.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.840+0000 7f01da000640 1 -- 172.21.15.16:0/3610222620 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(47..47 src has 1..47) ==== 5795+0+0 (secure 0 0 0) 0x7f01d40928c0 con 0x7f01e406f800 2024-09-10T21:12:41.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.840+0000 7f01dbe00640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f01c00725e0 0x7f01c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:41.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.840+0000 7f01dbe00640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f01c00725e0 0x7f01c0074aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f01e40af410 tx=0x7f01cc0091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:41.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:41.844+0000 7f01da000640 1 -- 172.21.15.16:0/3610222620 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f01e415e6d0 con 0x7f01e406f800 2024-09-10T21:12:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.068+0000 7f01e8e00640 1 -- 172.21.15.16:0/3610222620 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f01e406fc70 con 0x7f01e406f800 2024-09-10T21:12:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.664+0000 7f01da000640 1 -- 172.21.15.16:0/3610222620 <== mon.2 v2:172.21.15.52:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7f01d4001130 con 0x7f01e406f800 2024-09-10T21:12:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.668+0000 7f01e8e00640 1 -- 172.21.15.16:0/3610222620 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f01c00725e0 msgr2=0x7f01c0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.668+0000 7f01e8e00640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f01c00725e0 0x7f01c0074aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f01e40af410 tx=0x7f01cc0091e0 comp rx=0 tx=0).stop 2024-09-10T21:12:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.668+0000 7f01e8e00640 1 -- 172.21.15.16:0/3610222620 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01e406f800 msgr2=0x7f01e4146160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.668+0000 7f01e8e00640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01e406f800 0x7f01e4146160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01d400c400 tx=0x7f01d400c8c0 comp rx=0 tx=0).stop 2024-09-10T21:12:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.668+0000 7f01e8e00640 1 -- 172.21.15.16:0/3610222620 shutdown_connections 2024-09-10T21:12:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.668+0000 7f01e8e00640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f01c00725e0 0x7f01c0074aa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.668+0000 7f01e8e00640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01e415dd80 0x7f01e4165f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.668+0000 7f01e8e00640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01e4151210 0x7f01e415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.668+0000 7f01e8e00640 1 --2- 172.21.15.16:0/3610222620 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01e406f800 0x7f01e4146160 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.668+0000 7f01e8e00640 1 -- 172.21.15.16:0/3610222620 >> 172.21.15.16:0/3610222620 conn(0x7f01e40b16a0 msgr2=0x7f01e40ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:42.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.668+0000 7f01e8e00640 1 -- 172.21.15.16:0/3610222620 shutdown_connections 2024-09-10T21:12:42.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.668+0000 7f01e8e00640 1 -- 172.21.15.16:0/3610222620 wait complete. 2024-09-10T21:12:42.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:12:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.848+0000 7f777fa00640 1 -- 172.21.15.16:0/3657990359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7760004d50 msgr2=0x7f77600051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.848+0000 7f777fa00640 1 --2- 172.21.15.16:0/3657990359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7760004d50 0x7f77600051d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7770009ee0 tx=0x7f777001d5f0 comp rx=0 tx=0).stop 2024-09-10T21:12:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.848+0000 7f777da00640 1 -- 172.21.15.16:0/3657990359 reap_dead start 2024-09-10T21:12:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.848+0000 7f777fa00640 1 -- 172.21.15.16:0/3657990359 shutdown_connections 2024-09-10T21:12:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.848+0000 7f777fa00640 1 -- 172.21.15.16:0/3657990359 >> 172.21.15.16:0/3657990359 conn(0x7f77780d1d90 msgr2=0x7f77780d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.848+0000 7f777fa00640 1 -- 172.21.15.16:0/3657990359 shutdown_connections 2024-09-10T21:12:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.848+0000 7f777fa00640 1 -- 172.21.15.16:0/3657990359 wait complete. 2024-09-10T21:12:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f777fa00640 1 Processor -- start 2024-09-10T21:12:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f777fa00640 1 -- start start 2024-09-10T21:12:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f777fa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7760004d50 0x7f7778159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f777fa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777806dcf0 0x7f77780b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f777fa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77780b45b0 0x7f77780b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f777fa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7778164d20 con 0x7f777806dcf0 2024-09-10T21:12:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f777fa00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7778164ba0 con 0x7f77780b45b0 2024-09-10T21:12:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f777fa00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7778164ea0 con 0x7f7760004d50 2024-09-10T21:12:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f7777e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777806dcf0 0x7f77780b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f7777e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777806dcf0 0x7f77780b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47692/0 (socket says 172.21.15.16:47692) 2024-09-10T21:12:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f7777e00640 1 -- 172.21.15.16:0/2716327010 learned_addr learned my addr 172.21.15.16:0/2716327010 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f777d000640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7760004d50 0x7f7778159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f777da00640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77780b45b0 0x7f77780b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f7777e00640 1 -- 172.21.15.16:0/2716327010 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7760004d50 msgr2=0x7f7778159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f7777e00640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7760004d50 0x7f7778159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f7777e00640 1 -- 172.21.15.16:0/2716327010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77780b45b0 msgr2=0x7f77780b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f7777e00640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77780b45b0 0x7f77780b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f7777e00640 1 -- 172.21.15.16:0/2716327010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777806e180 con 0x7f777806dcf0 2024-09-10T21:12:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f7777e00640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777806dcf0 0x7f77780b3310 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7764001890 tx=0x7f7764012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f7774c00640 1 -- 172.21.15.16:0/2716327010 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764010b40 con 0x7f777806dcf0 2024-09-10T21:12:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f7774c00640 1 -- 172.21.15.16:0/2716327010 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7764017030 con 0x7f777806dcf0 2024-09-10T21:12:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f777fa00640 1 -- 172.21.15.16:0/2716327010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777806e460 con 0x7f777806dcf0 2024-09-10T21:12:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f7774c00640 1 -- 172.21.15.16:0/2716327010 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764012e60 con 0x7f777806dcf0 2024-09-10T21:12:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f777fa00640 1 -- 172.21.15.16:0/2716327010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f777806ea20 con 0x7f777806dcf0 2024-09-10T21:12:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.852+0000 7f777fa00640 1 -- 172.21.15.16:0/2716327010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77780cfbd0 con 0x7f777806dcf0 2024-09-10T21:12:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.856+0000 7f7774c00640 1 -- 172.21.15.16:0/2716327010 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7764013450 con 0x7f777806dcf0 2024-09-10T21:12:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.856+0000 7f7774c00640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77580725e0 0x7f7758074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.856+0000 7f7774c00640 1 -- 172.21.15.16:0/2716327010 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5795+0+0 (secure 0 0 0) 0x7f7764096df0 con 0x7f777806dcf0 2024-09-10T21:12:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.856+0000 7f777d000640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77580725e0 0x7f7758074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.856+0000 7f777d000640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77580725e0 0x7f7758074aa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7778164590 tx=0x7f776c006d00 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:42.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:42.860+0000 7f7774c00640 1 -- 172.21.15.16:0/2716327010 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7764061eb0 con 0x7f777806dcf0 2024-09-10T21:12:43.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.036+0000 7f777fa00640 1 -- 172.21.15.16:0/2716327010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7778069e00 con 0x7f777806dcf0 2024-09-10T21:12:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.852+0000 7f7774c00640 1 -- 172.21.15.16:0/2716327010 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) ==== 181+0+0 (secure 0 0 0) 0x7f77640668e0 con 0x7f777806dcf0 2024-09-10T21:12:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:12:43.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.856+0000 7f777fa00640 1 -- 172.21.15.16:0/2716327010 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77580725e0 msgr2=0x7f7758074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:43.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.856+0000 7f777fa00640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77580725e0 0x7f7758074aa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7778164590 tx=0x7f776c006d00 comp rx=0 tx=0).stop 2024-09-10T21:12:43.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.856+0000 7f777fa00640 1 -- 172.21.15.16:0/2716327010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777806dcf0 msgr2=0x7f77780b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:43.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.856+0000 7f777fa00640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777806dcf0 0x7f77780b3310 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7764001890 tx=0x7f7764012b20 comp rx=0 tx=0).stop 2024-09-10T21:12:43.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.856+0000 7f777fa00640 1 -- 172.21.15.16:0/2716327010 shutdown_connections 2024-09-10T21:12:43.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.856+0000 7f777fa00640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77580725e0 0x7f7758074aa0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:43.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.856+0000 7f777fa00640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77780b45b0 0x7f77780b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.856+0000 7f777fa00640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777806dcf0 0x7f77780b3310 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.856+0000 7f777fa00640 1 --2- 172.21.15.16:0/2716327010 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7760004d50 0x7f7778159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.856+0000 7f777fa00640 1 -- 172.21.15.16:0/2716327010 >> 172.21.15.16:0/2716327010 conn(0x7f77780d1d90 msgr2=0x7f777815a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.860+0000 7f777fa00640 1 -- 172.21.15.16:0/2716327010 shutdown_connections 2024-09-10T21:12:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:43.860+0000 7f777fa00640 1 -- 172.21.15.16:0/2716327010 wait complete. 2024-09-10T21:12:43.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:12:44.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.040+0000 7f2e0b200640 1 -- 172.21.15.16:0/211942315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e0c0d1bf0 msgr2=0x7f2e0c162610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:44.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.040+0000 7f2e0b200640 1 --2- 172.21.15.16:0/211942315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e0c0d1bf0 0x7f2e0c162610 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0d29d0 tx=0x7f2df801c760 comp rx=0 tx=0).stop 2024-09-10T21:12:44.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.040+0000 7f2e0b200640 1 -- 172.21.15.16:0/211942315 shutdown_connections 2024-09-10T21:12:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.040+0000 7f2e0b200640 1 --2- 172.21.15.16:0/211942315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e0c0d1bf0 0x7f2e0c162610 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.040+0000 7f2e0b200640 1 -- 172.21.15.16:0/211942315 >> 172.21.15.16:0/211942315 conn(0x7f2e0c0d3e10 msgr2=0x7f2e0c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.040+0000 7f2e0b200640 1 -- 172.21.15.16:0/211942315 shutdown_connections 2024-09-10T21:12:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.040+0000 7f2e0b200640 1 -- 172.21.15.16:0/211942315 wait complete. 2024-09-10T21:12:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.040+0000 7f2e0b200640 1 Processor -- start 2024-09-10T21:12:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.040+0000 7f2e0b200640 1 -- start start 2024-09-10T21:12:44.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e0b200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e0c0b3920 0x7f2e0c14a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:44.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e0b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e0c0b4bb0 0x7f2e0c14a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:44.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e0b200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2e0c0d1bf0 0x7f2e0c148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:44.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e0b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c16c720 con 0x7f2e0c0b4bb0 2024-09-10T21:12:44.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e0b200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c162d90 con 0x7f2e0c0b3920 2024-09-10T21:12:44.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e0b200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0c16c8a0 con 0x7f2e0c0d1bf0 2024-09-10T21:12:44.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e09400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e0c0b4bb0 0x7f2e0c14a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:44.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e09400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e0c0b4bb0 0x7f2e0c14a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47722/0 (socket says 172.21.15.16:47722) 2024-09-10T21:12:44.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e09400640 1 -- 172.21.15.16:0/1409357028 learned_addr learned my addr 172.21.15.16:0/1409357028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:44.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e09e00640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e0c0b3920 0x7f2e0c14a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:44.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e0a800640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2e0c0d1bf0 0x7f2e0c148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:44.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e09400640 1 -- 172.21.15.16:0/1409357028 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2e0c0d1bf0 msgr2=0x7f2e0c148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:44.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e09400640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2e0c0d1bf0 0x7f2e0c148ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:44.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e09400640 1 -- 172.21.15.16:0/1409357028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e0c0b3920 msgr2=0x7f2e0c14a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:44.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e09400640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e0c0b3920 0x7f2e0c14a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:44.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e09400640 1 -- 172.21.15.16:0/1409357028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c0b9ab0 con 0x7f2e0c0b4bb0 2024-09-10T21:12:44.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e09400640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e0c0b4bb0 0x7f2e0c14a630 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2df8000c10 tx=0x7f2df8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:44.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e02000640 1 -- 172.21.15.16:0/1409357028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80bb070 con 0x7f2e0c0b4bb0 2024-09-10T21:12:44.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e02000640 1 -- 172.21.15.16:0/1409357028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2df8005030 con 0x7f2e0c0b4bb0 2024-09-10T21:12:44.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e02000640 1 -- 172.21.15.16:0/1409357028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80b2040 con 0x7f2e0c0b4bb0 2024-09-10T21:12:44.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e0b200640 1 -- 172.21.15.16:0/1409357028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0c147fe0 con 0x7f2e0c0b4bb0 2024-09-10T21:12:44.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e0b200640 1 -- 172.21.15.16:0/1409357028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e0c0b9710 con 0x7f2e0c0b4bb0 2024-09-10T21:12:44.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.044+0000 7f2e0b200640 1 -- 172.21.15.16:0/1409357028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e0c147e00 con 0x7f2e0c0b4bb0 2024-09-10T21:12:44.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.048+0000 7f2e02000640 1 -- 172.21.15.16:0/1409357028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2df80b5030 con 0x7f2e0c0b4bb0 2024-09-10T21:12:44.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.048+0000 7f2e02000640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2dec0725e0 0x7f2dec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:44.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.048+0000 7f2e02000640 1 -- 172.21.15.16:0/1409357028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5357+0+0 (secure 0 0 0) 0x7f2df812c8d0 con 0x7f2e0c0b4bb0 2024-09-10T21:12:44.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.048+0000 7f2e09e00640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2dec0725e0 0x7f2dec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:44.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.048+0000 7f2e09e00640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2dec0725e0 0x7f2dec074aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc000f70 tx=0x7f2dfc009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:44.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.052+0000 7f2e02000640 1 -- 172.21.15.16:0/1409357028 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2e0c147e00 con 0x7f2e0c0b4bb0 2024-09-10T21:12:44.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.228+0000 7f2e0b200640 1 -- 172.21.15.16:0/1409357028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2e0c161f00 con 0x7f2e0c0b4bb0 2024-09-10T21:12:44.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.856+0000 7f2e02000640 1 -- 172.21.15.16:0/1409357028 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) ==== 169+0+0 (secure 0 0 0) 0x7f2df8016410 con 0x7f2e0c0b4bb0 2024-09-10T21:12:44.862 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-10T21:12:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.860+0000 7f2e0b200640 1 -- 172.21.15.16:0/1409357028 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2dec0725e0 msgr2=0x7f2dec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.860+0000 7f2e0b200640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2dec0725e0 0x7f2dec074aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc000f70 tx=0x7f2dfc009040 comp rx=0 tx=0).stop 2024-09-10T21:12:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.860+0000 7f2e0b200640 1 -- 172.21.15.16:0/1409357028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e0c0b4bb0 msgr2=0x7f2e0c14a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.860+0000 7f2e0b200640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e0c0b4bb0 0x7f2e0c14a630 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2df8000c10 tx=0x7f2df8004040 comp rx=0 tx=0).stop 2024-09-10T21:12:44.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.860+0000 7f2e0b200640 1 -- 172.21.15.16:0/1409357028 shutdown_connections 2024-09-10T21:12:44.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.860+0000 7f2e0b200640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2dec0725e0 0x7f2dec074aa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:44.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.860+0000 7f2e0b200640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2e0c0d1bf0 0x7f2e0c148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:44.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.860+0000 7f2e0b200640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e0c0b4bb0 0x7f2e0c14a630 secure :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2df8000c10 tx=0x7f2df8004040 comp rx=0 tx=0).stop 2024-09-10T21:12:44.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.860+0000 7f2e0b200640 1 --2- 172.21.15.16:0/1409357028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e0c0b3920 0x7f2e0c14a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:44.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.860+0000 7f2e0b200640 1 -- 172.21.15.16:0/1409357028 >> 172.21.15.16:0/1409357028 conn(0x7f2e0c0d3e10 msgr2=0x7f2e0c0ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:44.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.864+0000 7f2e0b200640 1 -- 172.21.15.16:0/1409357028 shutdown_connections 2024-09-10T21:12:44.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:44.864+0000 7f2e0b200640 1 -- 172.21.15.16:0/1409357028 wait complete. 2024-09-10T21:12:44.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:12:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.040+0000 7ff3c6800640 1 -- 172.21.15.16:0/2436349538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3a4003a10 msgr2=0x7ff3a4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.040+0000 7ff3c6800640 1 --2- 172.21.15.16:0/2436349538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3a4003a10 0x7ff3a4003e30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0070880 tx=0x7ff3b001d290 comp rx=0 tx=0).stop 2024-09-10T21:12:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.040+0000 7ff3bfe00640 1 -- 172.21.15.16:0/2436349538 reap_dead start 2024-09-10T21:12:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.040+0000 7ff3c6800640 1 -- 172.21.15.16:0/2436349538 shutdown_connections 2024-09-10T21:12:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.040+0000 7ff3c6800640 1 -- 172.21.15.16:0/2436349538 >> 172.21.15.16:0/2436349538 conn(0x7ff3c009b130 msgr2=0x7ff3c00d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.040+0000 7ff3c6800640 1 -- 172.21.15.16:0/2436349538 shutdown_connections 2024-09-10T21:12:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.040+0000 7ff3c6800640 1 -- 172.21.15.16:0/2436349538 wait complete. 2024-09-10T21:12:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.040+0000 7ff3c6800640 1 Processor -- start 2024-09-10T21:12:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.040+0000 7ff3c6800640 1 -- start start 2024-09-10T21:12:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3c6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3a4003a10 0x7ff3c00cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3c6800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3c0165770 0x7ff3c01f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3c6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c01f78e0 0x7ff3c01fb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3c6800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0166dd0 con 0x7ff3c01f78e0 2024-09-10T21:12:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3c6800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0166f50 con 0x7ff3a4003a10 2024-09-10T21:12:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3c6800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c01670d0 con 0x7ff3c0165770 2024-09-10T21:12:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c01f78e0 0x7ff3c01fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c01f78e0 0x7ff3c01fb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47738/0 (socket says 172.21.15.16:47738) 2024-09-10T21:12:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3bfe00640 1 -- 172.21.15.16:0/4048997426 learned_addr learned my addr 172.21.15.16:0/4048997426 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3b7e00640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3c0165770 0x7ff3c01f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3bf400640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3a4003a10 0x7ff3c00cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3bfe00640 1 -- 172.21.15.16:0/4048997426 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3c0165770 msgr2=0x7ff3c01f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3bfe00640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3c0165770 0x7ff3c01f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3bfe00640 1 -- 172.21.15.16:0/4048997426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3a4003a10 msgr2=0x7ff3c00cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3bfe00640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3a4003a10 0x7ff3c00cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3bfe00640 1 -- 172.21.15.16:0/4048997426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c00cb390 con 0x7ff3c01f78e0 2024-09-10T21:12:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3bfe00640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c01f78e0 0x7ff3c01fb800 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00cf300 tx=0x7ff3b800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3b7400640 1 -- 172.21.15.16:0/4048997426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8004b30 con 0x7ff3c01f78e0 2024-09-10T21:12:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3b7400640 1 -- 172.21.15.16:0/4048997426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3b8004cd0 con 0x7ff3c01f78e0 2024-09-10T21:12:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3b7400640 1 -- 172.21.15.16:0/4048997426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8003be0 con 0x7ff3c01f78e0 2024-09-10T21:12:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3c6800640 1 -- 172.21.15.16:0/4048997426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c00c8f80 con 0x7ff3c01f78e0 2024-09-10T21:12:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3c6800640 1 -- 172.21.15.16:0/4048997426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c0153090 con 0x7ff3c01f78e0 2024-09-10T21:12:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.044+0000 7ff3c6800640 1 -- 172.21.15.16:0/4048997426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c0071500 con 0x7ff3c01f78e0 2024-09-10T21:12:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.048+0000 7ff3b7400640 1 -- 172.21.15.16:0/4048997426 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff3b8018070 con 0x7ff3c01f78e0 2024-09-10T21:12:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.048+0000 7ff3b7400640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff39c0725e0 0x7ff39c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.048+0000 7ff3b7400640 1 -- 172.21.15.16:0/4048997426 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7ff3b8093700 con 0x7ff3c01f78e0 2024-09-10T21:12:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.048+0000 7ff3bf400640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff39c0725e0 0x7ff39c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.048+0000 7ff3bf400640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff39c0725e0 0x7ff39c074aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00ae000 tx=0x7ff3b001a030 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.052+0000 7ff3b7400640 1 -- 172.21.15.16:0/4048997426 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff3b805ea80 con 0x7ff3c01f78e0 2024-09-10T21:12:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.228+0000 7ff3c6800640 1 -- 172.21.15.16:0/4048997426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7ff3c0071f20 con 0x7ff3c01f78e0 2024-09-10T21:12:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.232+0000 7ff3b7400640 1 -- 172.21.15.16:0/4048997426 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7ff3b80634b0 con 0x7ff3c01f78e0 2024-09-10T21:12:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.236+0000 7ff3c6800640 1 -- 172.21.15.16:0/4048997426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff39c0725e0 msgr2=0x7ff39c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.236+0000 7ff3c6800640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff39c0725e0 0x7ff39c074aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00ae000 tx=0x7ff3b001a030 comp rx=0 tx=0).stop 2024-09-10T21:12:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.236+0000 7ff3c6800640 1 -- 172.21.15.16:0/4048997426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c01f78e0 msgr2=0x7ff3c01fb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.236+0000 7ff3c6800640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c01f78e0 0x7ff3c01fb800 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00cf300 tx=0x7ff3b800f3c0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.236+0000 7ff3c6800640 1 -- 172.21.15.16:0/4048997426 shutdown_connections 2024-09-10T21:12:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.236+0000 7ff3c6800640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff39c0725e0 0x7ff39c074aa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.236+0000 7ff3c6800640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c01f78e0 0x7ff3c01fb800 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.236+0000 7ff3c6800640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3c0165770 0x7ff3c01f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.236+0000 7ff3c6800640 1 --2- 172.21.15.16:0/4048997426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3a4003a10 0x7ff3c00cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.236+0000 7ff3c6800640 1 -- 172.21.15.16:0/4048997426 >> 172.21.15.16:0/4048997426 conn(0x7ff3c009b130 msgr2=0x7ff3c00ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:45.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.236+0000 7ff3c6800640 1 -- 172.21.15.16:0/4048997426 shutdown_connections 2024-09-10T21:12:45.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.236+0000 7ff3c6800640 1 -- 172.21.15.16:0/4048997426 wait complete. 2024-09-10T21:12:45.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2024-09-10T21:12:45.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.400+0000 7f74b1600640 1 -- 172.21.15.16:0/2672249350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ac0b2c20 msgr2=0x7f74ac0d0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:45.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.400+0000 7f74b1600640 1 --2- 172.21.15.16:0/2672249350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ac0b2c20 0x7f74ac0d0740 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f74ac161ac0 tx=0x7f749401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.400+0000 7f74b1600640 1 -- 172.21.15.16:0/2672249350 shutdown_connections 2024-09-10T21:12:45.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.400+0000 7f74b1600640 1 --2- 172.21.15.16:0/2672249350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ac0b2c20 0x7f74ac0d0740 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.400+0000 7f74b1600640 1 --2- 172.21.15.16:0/2672249350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f74ac161560 0x7f74ac0d4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.400+0000 7f74b1600640 1 -- 172.21.15.16:0/2672249350 >> 172.21.15.16:0/2672249350 conn(0x7f74ac0d3f00 msgr2=0x7f74ac0d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:45.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.400+0000 7f74b1600640 1 -- 172.21.15.16:0/2672249350 shutdown_connections 2024-09-10T21:12:45.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.400+0000 7f74b1600640 1 -- 172.21.15.16:0/2672249350 wait complete. 2024-09-10T21:12:45.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b1600640 1 Processor -- start 2024-09-10T21:12:45.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b1600640 1 -- start start 2024-09-10T21:12:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b1600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f74ac0b2c20 0x7f74ac14d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b1600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74ac0d0c80 0x7f74ac14dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ac161560 0x7f74ac0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b1600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac16b540 con 0x7f74ac161560 2024-09-10T21:12:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b1600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac16b3c0 con 0x7f74ac0d0c80 2024-09-10T21:12:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b1600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f74ac16b6c0 con 0x7f74ac0b2c20 2024-09-10T21:12:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ac161560 0x7f74ac0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ac161560 0x7f74ac0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47756/0 (socket says 172.21.15.16:47756) 2024-09-10T21:12:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b0c00640 1 -- 172.21.15.16:0/149887548 learned_addr learned my addr 172.21.15.16:0/149887548 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74abe00640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f74ac0b2c20 0x7f74ac14d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74a3e00640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74ac0d0c80 0x7f74ac14dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b0c00640 1 -- 172.21.15.16:0/149887548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f74ac0b2c20 msgr2=0x7f74ac14d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b0c00640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f74ac0b2c20 0x7f74ac14d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b0c00640 1 -- 172.21.15.16:0/149887548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74ac0d0c80 msgr2=0x7f74ac14dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b0c00640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74ac0d0c80 0x7f74ac14dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b0c00640 1 -- 172.21.15.16:0/149887548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ac157500 con 0x7f74ac161560 2024-09-10T21:12:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b0c00640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ac161560 0x7f74ac0b7400 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f74a400c780 tx=0x7f74a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74a8c00640 1 -- 172.21.15.16:0/149887548 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4004b70 con 0x7f74ac161560 2024-09-10T21:12:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74a8c00640 1 -- 172.21.15.16:0/149887548 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74a4004e00 con 0x7f74ac161560 2024-09-10T21:12:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74a8c00640 1 -- 172.21.15.16:0/149887548 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a400aa30 con 0x7f74ac161560 2024-09-10T21:12:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b1600640 1 -- 172.21.15.16:0/149887548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ac1577e0 con 0x7f74ac161560 2024-09-10T21:12:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.404+0000 7f74b1600640 1 -- 172.21.15.16:0/149887548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ac1582c0 con 0x7f74ac161560 2024-09-10T21:12:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.408+0000 7f74b1600640 1 -- 172.21.15.16:0/149887548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74ac0d0740 con 0x7f74ac161560 2024-09-10T21:12:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.408+0000 7f74a8c00640 1 -- 172.21.15.16:0/149887548 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f74a400abd0 con 0x7f74ac161560 2024-09-10T21:12:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.408+0000 7f74a8c00640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7488072610 0x7f7488074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.408+0000 7f74a8c00640 1 -- 172.21.15.16:0/149887548 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f74a4093080 con 0x7f74ac161560 2024-09-10T21:12:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.408+0000 7f74abe00640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7488072610 0x7f7488074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.408+0000 7f74abe00640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7488072610 0x7f7488074ad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f749c00ab40 tx=0x7f749c00a250 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.412+0000 7f74a8c00640 1 -- 172.21.15.16:0/149887548 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f74a4020070 con 0x7f74ac161560 2024-09-10T21:12:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.648+0000 7f74b1600640 1 -- 172.21.15.16:0/149887548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) -- 0x7f74ac15f590 con 0x7f74ac161560 2024-09-10T21:12:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.856+0000 7f74a8c00640 1 -- 172.21.15.16:0/149887548 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v132) ==== 162+0+0 (secure 0 0 0) 0x7f74a405e400 con 0x7f74ac161560 2024-09-10T21:12:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.860+0000 7f74b1600640 1 -- 172.21.15.16:0/149887548 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7488072610 msgr2=0x7f7488074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.860+0000 7f74b1600640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7488072610 0x7f7488074ad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f749c00ab40 tx=0x7f749c00a250 comp rx=0 tx=0).stop 2024-09-10T21:12:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.860+0000 7f74b1600640 1 -- 172.21.15.16:0/149887548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ac161560 msgr2=0x7f74ac0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.860+0000 7f74b1600640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ac161560 0x7f74ac0b7400 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f74a400c780 tx=0x7f74a400cc40 comp rx=0 tx=0).stop 2024-09-10T21:12:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.860+0000 7f74b1600640 1 -- 172.21.15.16:0/149887548 shutdown_connections 2024-09-10T21:12:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.860+0000 7f74b1600640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7488072610 0x7f7488074ad0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.860+0000 7f74b1600640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ac161560 0x7f74ac0b7400 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.860+0000 7f74b1600640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74ac0d0c80 0x7f74ac14dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.860+0000 7f74b1600640 1 --2- 172.21.15.16:0/149887548 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f74ac0b2c20 0x7f74ac14d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.860+0000 7f74b1600640 1 -- 172.21.15.16:0/149887548 >> 172.21.15.16:0/149887548 conn(0x7f74ac0d3f00 msgr2=0x7f74ac0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.860+0000 7f74b1600640 1 -- 172.21.15.16:0/149887548 shutdown_connections 2024-09-10T21:12:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:45.860+0000 7f74b1600640 1 -- 172.21.15.16:0/149887548 wait complete. 2024-09-10T21:12:45.886 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the ... skipped 'Require FUSE client to inject client release failure' 2024-09-10T21:12:45.887 INFO:tasks.cephfs_test_runner:Starting test: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-10T21:12:45.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.028+0000 7f8735400640 1 -- 172.21.15.16:0/1140226544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300d0890 msgr2=0x7f87300d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.028+0000 7f8735400640 1 --2- 172.21.15.16:0/1140226544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300d0890 0x7f87300d01c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8728009bc0 tx=0x7f872801f350 comp rx=0 tx=0).stop 2024-09-10T21:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.028+0000 7f8735400640 1 -- 172.21.15.16:0/1140226544 shutdown_connections 2024-09-10T21:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.028+0000 7f8735400640 1 --2- 172.21.15.16:0/1140226544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300d0890 0x7f87300d01c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.028+0000 7f8735400640 1 --2- 172.21.15.16:0/1140226544 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f873015f450 0x7f87300cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.028+0000 7f8735400640 1 -- 172.21.15.16:0/1140226544 >> 172.21.15.16:0/1140226544 conn(0x7f87300d1d80 msgr2=0x7f87300d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.028+0000 7f8735400640 1 -- 172.21.15.16:0/1140226544 shutdown_connections 2024-09-10T21:12:46.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.028+0000 7f8735400640 1 -- 172.21.15.16:0/1140226544 wait complete. 2024-09-10T21:12:46.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.028+0000 7f8735400640 1 Processor -- start 2024-09-10T21:12:46.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.028+0000 7f8735400640 1 -- start start 2024-09-10T21:12:46.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.028+0000 7f8735400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87300d0890 0x7f87301562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:46.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8735400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f873015f450 0x7f87300cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:46.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8735400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300cde00 0x7f87300c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8735400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87300b2e30 con 0x7f87300d0890 2024-09-10T21:12:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8735400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f87300b2cb0 con 0x7f87300cde00 2024-09-10T21:12:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8735400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f87300b2fb0 con 0x7f873015f450 2024-09-10T21:12:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f872e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87300d0890 0x7f87301562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f872e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87300d0890 0x7f87301562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47760/0 (socket says 172.21.15.16:47760) 2024-09-10T21:12:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f872e800640 1 -- 172.21.15.16:0/3528827995 learned_addr learned my addr 172.21.15.16:0/3528827995 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f872f200640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300cde00 0x7f87300c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f872e800640 1 -- 172.21.15.16:0/3528827995 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f873015f450 msgr2=0x7f87300cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8727e00640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f873015f450 0x7f87300cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f872e800640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f873015f450 0x7f87300cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f872e800640 1 -- 172.21.15.16:0/3528827995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300cde00 msgr2=0x7f87300c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f872e800640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300cde00 0x7f87300c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f872e800640 1 -- 172.21.15.16:0/3528827995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8730156800 con 0x7f87300d0890 2024-09-10T21:12:46.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f872f200640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300cde00 0x7f87300c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:46.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8727e00640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f873015f450 0x7f87300cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:46.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f872e800640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87300d0890 0x7f87301562c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f872000e970 tx=0x7f872000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:46.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8726a00640 1 -- 172.21.15.16:0/3528827995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872000cd80 con 0x7f87300d0890 2024-09-10T21:12:46.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8726a00640 1 -- 172.21.15.16:0/3528827995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8720004020 con 0x7f87300d0890 2024-09-10T21:12:46.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8735400640 1 -- 172.21.15.16:0/3528827995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87300ce380 con 0x7f87300d0890 2024-09-10T21:12:46.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8726a00640 1 -- 172.21.15.16:0/3528827995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720004470 con 0x7f87300d0890 2024-09-10T21:12:46.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8735400640 1 -- 172.21.15.16:0/3528827995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87301f4d30 con 0x7f87300d0890 2024-09-10T21:12:46.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8735400640 1 -- 172.21.15.16:0/3528827995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87300b1e30 con 0x7f87300d0890 2024-09-10T21:12:46.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.032+0000 7f8726a00640 1 -- 172.21.15.16:0/3528827995 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8720003600 con 0x7f87300d0890 2024-09-10T21:12:46.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.036+0000 7f8726a00640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f870c072610 0x7f870c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:46.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.036+0000 7f8726a00640 1 -- 172.21.15.16:0/3528827995 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f87200939e0 con 0x7f87300d0890 2024-09-10T21:12:46.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.036+0000 7f8727e00640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f870c072610 0x7f870c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:46.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.036+0000 7f8727e00640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f870c072610 0x7f870c074ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8718001250 tx=0x7f8718009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:46.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.040+0000 7f8726a00640 1 -- 172.21.15.16:0/3528827995 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f872005f320 con 0x7f87300d0890 2024-09-10T21:12:46.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.216+0000 7f8735400640 1 -- 172.21.15.16:0/3528827995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8730072120 con 0x7f87300d0890 2024-09-10T21:12:46.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.216+0000 7f8726a00640 1 -- 172.21.15.16:0/3528827995 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+12984 (secure 0 0 0) 0x7f872005ec20 con 0x7f87300d0890 2024-09-10T21:12:46.223 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:46.223 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:44.857834+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6834/1834750661":"2024-09-11T21:12:41.662917+0000","172.21.15.52:6832/1834750661":"2024-09-11T21:12:41.662917+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:12:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.216+0000 7f8735400640 1 -- 172.21.15.16:0/3528827995 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f870c072610 msgr2=0x7f870c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.216+0000 7f8735400640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f870c072610 0x7f870c074ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8718001250 tx=0x7f8718009040 comp rx=0 tx=0).stop 2024-09-10T21:12:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.216+0000 7f8735400640 1 -- 172.21.15.16:0/3528827995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87300d0890 msgr2=0x7f87301562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.216+0000 7f8735400640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87300d0890 0x7f87301562c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f872000e970 tx=0x7f872000ee30 comp rx=0 tx=0).stop 2024-09-10T21:12:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.216+0000 7f8735400640 1 -- 172.21.15.16:0/3528827995 shutdown_connections 2024-09-10T21:12:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.216+0000 7f8735400640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f870c072610 0x7f870c074ad0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.216+0000 7f8735400640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87300cde00 0x7f87300c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.216+0000 7f8735400640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f873015f450 0x7f87300cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.216+0000 7f8735400640 1 --2- 172.21.15.16:0/3528827995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87300d0890 0x7f87301562c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.216+0000 7f8735400640 1 -- 172.21.15.16:0/3528827995 >> 172.21.15.16:0/3528827995 conn(0x7f87300d1d80 msgr2=0x7f87300b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.220+0000 7f8735400640 1 -- 172.21.15.16:0/3528827995 shutdown_connections 2024-09-10T21:12:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.220+0000 7f8735400640 1 -- 172.21.15.16:0/3528827995 wait complete. 2024-09-10T21:12:46.246 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:10:17.493770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:12:46.246 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:12:46.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.388+0000 7f1171000640 1 -- 172.21.15.16:0/1018715535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c151210 msgr2=0x7f116c0726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:46.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.388+0000 7f1171000640 1 --2- 172.21.15.16:0/1018715535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c151210 0x7f116c0726b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f115c009900 tx=0x7f115c01c8e0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.388+0000 7f1171000640 1 -- 172.21.15.16:0/1018715535 shutdown_connections 2024-09-10T21:12:46.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.388+0000 7f1171000640 1 --2- 172.21.15.16:0/1018715535 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f116c072bf0 0x7f116c15dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.388+0000 7f1171000640 1 --2- 172.21.15.16:0/1018715535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c151210 0x7f116c0726b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.388+0000 7f1171000640 1 -- 172.21.15.16:0/1018715535 >> 172.21.15.16:0/1018715535 conn(0x7f116c0b16a0 msgr2=0x7f116c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:46.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.388+0000 7f1171000640 1 -- 172.21.15.16:0/1018715535 shutdown_connections 2024-09-10T21:12:46.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.388+0000 7f1171000640 1 -- 172.21.15.16:0/1018715535 wait complete. 2024-09-10T21:12:46.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.388+0000 7f1171000640 1 Processor -- start 2024-09-10T21:12:46.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.388+0000 7f1171000640 1 -- start start 2024-09-10T21:12:46.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1171000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f116c072bf0 0x7f116c14c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:46.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1171000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c151210 0x7f116c1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:46.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1171000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f116c15e2c0 0x7f116c159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:46.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1171000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f116c15b320 con 0x7f116c151210 2024-09-10T21:12:46.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1171000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f116c15b1a0 con 0x7f116c15e2c0 2024-09-10T21:12:46.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1171000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f116c15b4a0 con 0x7f116c072bf0 2024-09-10T21:12:46.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1163e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c151210 0x7f116c1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:46.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1163e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c151210 0x7f116c1591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47784/0 (socket says 172.21.15.16:47784) 2024-09-10T21:12:46.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1163e00640 1 -- 172.21.15.16:0/1058081144 learned_addr learned my addr 172.21.15.16:0/1058081144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:46.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f116a800640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f116c15e2c0 0x7f116c159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:46.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1169e00640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f116c072bf0 0x7f116c14c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:46.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1163e00640 1 -- 172.21.15.16:0/1058081144 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f116c072bf0 msgr2=0x7f116c14c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:46.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1163e00640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f116c072bf0 0x7f116c14c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1163e00640 1 -- 172.21.15.16:0/1058081144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f116c15e2c0 msgr2=0x7f116c159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:46.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1163e00640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f116c15e2c0 0x7f116c159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1163e00640 1 -- 172.21.15.16:0/1058081144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116c1fb390 con 0x7f116c151210 2024-09-10T21:12:46.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1163e00640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c151210 0x7f116c1591d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f115400ddf0 tx=0x7f115400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:46.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1162000640 1 -- 172.21.15.16:0/1058081144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115401b070 con 0x7f116c151210 2024-09-10T21:12:46.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1162000640 1 -- 172.21.15.16:0/1058081144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f115400e540 con 0x7f116c151210 2024-09-10T21:12:46.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1162000640 1 -- 172.21.15.16:0/1058081144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115400e990 con 0x7f116c151210 2024-09-10T21:12:46.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1171000640 1 -- 172.21.15.16:0/1058081144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116c152550 con 0x7f116c151210 2024-09-10T21:12:46.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1171000640 1 -- 172.21.15.16:0/1058081144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f116c14c6e0 con 0x7f116c151210 2024-09-10T21:12:46.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1171000640 1 -- 172.21.15.16:0/1058081144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f116c0b2c50 con 0x7f116c151210 2024-09-10T21:12:46.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.392+0000 7f1162000640 1 -- 172.21.15.16:0/1058081144 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f115400b040 con 0x7f116c151210 2024-09-10T21:12:46.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.396+0000 7f1162000640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11480725e0 0x7f1148074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:46.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.396+0000 7f1162000640 1 -- 172.21.15.16:0/1058081144 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f1154093350 con 0x7f116c151210 2024-09-10T21:12:46.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.396+0000 7f1169e00640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11480725e0 0x7f1148074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:46.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.396+0000 7f1169e00640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11480725e0 0x7f1148074aa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f115c0af000 tx=0x7f115c01b120 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:46.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.400+0000 7f1162000640 1 -- 172.21.15.16:0/1058081144 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f115405e640 con 0x7f116c151210 2024-09-10T21:12:46.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1171000640 1 -- 172.21.15.16:0/1058081144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f116c0b4190 con 0x7f116c151210 2024-09-10T21:12:46.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1162000640 1 -- 172.21.15.16:0/1058081144 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) ==== 93+0+10 (secure 0 0 0) 0x7f1154063070 con 0x7f116c151210 2024-09-10T21:12:46.583 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-10T21:12:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1171000640 1 -- 172.21.15.16:0/1058081144 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11480725e0 msgr2=0x7f1148074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1171000640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11480725e0 0x7f1148074aa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f115c0af000 tx=0x7f115c01b120 comp rx=0 tx=0).stop 2024-09-10T21:12:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1171000640 1 -- 172.21.15.16:0/1058081144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c151210 msgr2=0x7f116c1591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1171000640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c151210 0x7f116c1591d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f115400ddf0 tx=0x7f115400a620 comp rx=0 tx=0).stop 2024-09-10T21:12:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1171000640 1 -- 172.21.15.16:0/1058081144 shutdown_connections 2024-09-10T21:12:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1171000640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11480725e0 0x7f1148074aa0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1171000640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f116c15e2c0 0x7f116c159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1171000640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c151210 0x7f116c1591d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1171000640 1 --2- 172.21.15.16:0/1058081144 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f116c072bf0 0x7f116c14c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1171000640 1 -- 172.21.15.16:0/1058081144 >> 172.21.15.16:0/1058081144 conn(0x7f116c0b16a0 msgr2=0x7f116c0b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:46.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1171000640 1 -- 172.21.15.16:0/1058081144 shutdown_connections 2024-09-10T21:12:46.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.576+0000 7f1171000640 1 -- 172.21.15.16:0/1058081144 wait complete. 2024-09-10T21:12:46.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2024-09-10T21:12:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.744+0000 7f489a000640 1 -- 172.21.15.16:0/2896668408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48780052f0 msgr2=0x7f487800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.744+0000 7f489a000640 1 --2- 172.21.15.16:0/2896668408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48780052f0 0x7f487800b9d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f48940709e0 tx=0x7f488c01caf0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.744+0000 7f489a000640 1 -- 172.21.15.16:0/2896668408 shutdown_connections 2024-09-10T21:12:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.744+0000 7f489a000640 1 --2- 172.21.15.16:0/2896668408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48780052f0 0x7f487800b9d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.744+0000 7f489a000640 1 -- 172.21.15.16:0/2896668408 >> 172.21.15.16:0/2896668408 conn(0x7f48940cfb20 msgr2=0x7f48940cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.744+0000 7f489a000640 1 -- 172.21.15.16:0/2896668408 shutdown_connections 2024-09-10T21:12:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.744+0000 7f489a000640 1 -- 172.21.15.16:0/2896668408 wait complete. 2024-09-10T21:12:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f489a000640 1 Processor -- start 2024-09-10T21:12:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f489a000640 1 -- start start 2024-09-10T21:12:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f489a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48780052f0 0x7f4894156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f489a000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f489414bb90 0x7f48940b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f489a000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48940b4ec0 0x7f48941f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f489a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48940ce790 con 0x7f48780052f0 2024-09-10T21:12:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f489a000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f48940ce610 con 0x7f489414bb90 2024-09-10T21:12:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f489a000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f48940ce910 con 0x7f48940b4ec0 2024-09-10T21:12:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f4893400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48780052f0 0x7f4894156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f4893400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48780052f0 0x7f4894156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47798/0 (socket says 172.21.15.16:47798) 2024-09-10T21:12:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f4893400640 1 -- 172.21.15.16:0/888741843 learned_addr learned my addr 172.21.15.16:0/888741843 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f4892a00640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f489414bb90 0x7f48940b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f4893e00640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48940b4ec0 0x7f48941f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f4893400640 1 -- 172.21.15.16:0/888741843 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48940b4ec0 msgr2=0x7f48941f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f4893400640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48940b4ec0 0x7f48941f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f4893400640 1 -- 172.21.15.16:0/888741843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f489414bb90 msgr2=0x7f48940b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f4893400640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f489414bb90 0x7f48940b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f4893400640 1 -- 172.21.15.16:0/888741843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489414b0d0 con 0x7f48780052f0 2024-09-10T21:12:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f4893400640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48780052f0 0x7f4894156c60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f48840126c0 tx=0x7f4884012ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f488b400640 1 -- 172.21.15.16:0/888741843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884013950 con 0x7f48780052f0 2024-09-10T21:12:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f488b400640 1 -- 172.21.15.16:0/888741843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4884013b10 con 0x7f48780052f0 2024-09-10T21:12:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f488b400640 1 -- 172.21.15.16:0/888741843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48840103e0 con 0x7f48780052f0 2024-09-10T21:12:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f489a000640 1 -- 172.21.15.16:0/888741843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48941f32a0 con 0x7f48780052f0 2024-09-10T21:12:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.748+0000 7f489a000640 1 -- 172.21.15.16:0/888741843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48941571a0 con 0x7f48780052f0 2024-09-10T21:12:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.752+0000 7f489a000640 1 -- 172.21.15.16:0/888741843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48940b52d0 con 0x7f48780052f0 2024-09-10T21:12:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.752+0000 7f488b400640 1 -- 172.21.15.16:0/888741843 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4884010580 con 0x7f48780052f0 2024-09-10T21:12:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.752+0000 7f488b400640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4874072610 0x7f4874074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.752+0000 7f488b400640 1 -- 172.21.15.16:0/888741843 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f4884096a30 con 0x7f48780052f0 2024-09-10T21:12:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.752+0000 7f4892a00640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4874072610 0x7f4874074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.752+0000 7f4892a00640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4874072610 0x7f4874074ad0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f48941f6ba0 tx=0x7f488000a2f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.756+0000 7f488b400640 1 -- 172.21.15.16:0/888741843 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f48940b52d0 con 0x7f48780052f0 2024-09-10T21:12:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:46.992+0000 7f489a000640 1 -- 172.21.15.16:0/888741843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) -- 0x7f489406e640 con 0x7f48780052f0 2024-09-10T21:12:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:47.864+0000 7f488b400640 1 -- 172.21.15.16:0/888741843 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v134) ==== 153+0+0 (secure 0 0 0) 0x7f4884022020 con 0x7f48780052f0 2024-09-10T21:12:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:47.868+0000 7f489a000640 1 -- 172.21.15.16:0/888741843 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4874072610 msgr2=0x7f4874074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:47.868+0000 7f489a000640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4874072610 0x7f4874074ad0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f48941f6ba0 tx=0x7f488000a2f0 comp rx=0 tx=0).stop 2024-09-10T21:12:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:47.868+0000 7f489a000640 1 -- 172.21.15.16:0/888741843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48780052f0 msgr2=0x7f4894156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:47.868+0000 7f489a000640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48780052f0 0x7f4894156c60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f48840126c0 tx=0x7f4884012ac0 comp rx=0 tx=0).stop 2024-09-10T21:12:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:47.868+0000 7f489a000640 1 -- 172.21.15.16:0/888741843 shutdown_connections 2024-09-10T21:12:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:47.868+0000 7f489a000640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4874072610 0x7f4874074ad0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:47.868+0000 7f489a000640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48940b4ec0 0x7f48941f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:47.868+0000 7f489a000640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f489414bb90 0x7f48940b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:47.868+0000 7f489a000640 1 --2- 172.21.15.16:0/888741843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48780052f0 0x7f4894156c60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:47.868+0000 7f489a000640 1 -- 172.21.15.16:0/888741843 >> 172.21.15.16:0/888741843 conn(0x7f48940cfb20 msgr2=0x7f48940b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:47.868+0000 7f489a000640 1 -- 172.21.15.16:0/888741843 shutdown_connections 2024-09-10T21:12:47.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:47.868+0000 7f489a000640 1 -- 172.21.15.16:0/888741843 wait complete. 2024-09-10T21:12:47.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:12:48.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.040+0000 7fad9b000640 1 -- 172.21.15.16:0/4180492074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad9415f400 msgr2=0x7fad94070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:48.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.040+0000 7fad9b000640 1 --2- 172.21.15.16:0/4180492074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad9415f400 0x7fad94070fe0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fad940698f0 tx=0x7fad8401c780 comp rx=0 tx=0).stop 2024-09-10T21:12:48.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.040+0000 7fad9b000640 1 -- 172.21.15.16:0/4180492074 shutdown_connections 2024-09-10T21:12:48.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.040+0000 7fad9b000640 1 --2- 172.21.15.16:0/4180492074 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fad940cfc80 0x7fad940d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.040+0000 7fad9b000640 1 --2- 172.21.15.16:0/4180492074 >> v2:172.21.15.52:3300/0 conn(0x7fad940b13c0 0x7fad94071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.040+0000 7fad9b000640 1 --2- 172.21.15.16:0/4180492074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad9415f400 0x7fad94070fe0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.040+0000 7fad9b000640 1 -- 172.21.15.16:0/4180492074 >> 172.21.15.16:0/4180492074 conn(0x7fad9409b130 msgr2=0x7fad940d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:48.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.040+0000 7fad9b000640 1 -- 172.21.15.16:0/4180492074 shutdown_connections 2024-09-10T21:12:48.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.040+0000 7fad9b000640 1 -- 172.21.15.16:0/4180492074 wait complete. 2024-09-10T21:12:48.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad9b000640 1 Processor -- start 2024-09-10T21:12:48.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad9b000640 1 -- start start 2024-09-10T21:12:48.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad9b000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad940b13c0 0x7fad94148020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:48.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad9b000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad940cfc80 0x7fad940b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:48.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad9b000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fad9415f400 0x7fad940ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:48.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad9b000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad940afd90 con 0x7fad940b13c0 2024-09-10T21:12:48.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad9b000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fad940d1260 con 0x7fad940cfc80 2024-09-10T21:12:48.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad9b000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fad940aff10 con 0x7fad9415f400 2024-09-10T21:12:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad93e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad940b13c0 0x7fad94148020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad93e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad940b13c0 0x7fad94148020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47814/0 (socket says 172.21.15.16:47814) 2024-09-10T21:12:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad93e00640 1 -- 172.21.15.16:0/3191442886 learned_addr learned my addr 172.21.15.16:0/3191442886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad99000640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fad9415f400 0x7fad940ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad8be00640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad940cfc80 0x7fad940b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad93e00640 1 -- 172.21.15.16:0/3191442886 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fad9415f400 msgr2=0x7fad940ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad93e00640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fad9415f400 0x7fad940ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad93e00640 1 -- 172.21.15.16:0/3191442886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad940cfc80 msgr2=0x7fad940b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad93e00640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad940cfc80 0x7fad940b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad93e00640 1 -- 172.21.15.16:0/3191442886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad940b5670 con 0x7fad940b13c0 2024-09-10T21:12:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad99000640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fad9415f400 0x7fad940ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad93e00640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad940b13c0 0x7fad94148020 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fad84000c10 tx=0x7fad840a6660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:48.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad90c00640 1 -- 172.21.15.16:0/3191442886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84004100 con 0x7fad940b13c0 2024-09-10T21:12:48.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad90c00640 1 -- 172.21.15.16:0/3191442886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad84004390 con 0x7fad940b13c0 2024-09-10T21:12:48.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad90c00640 1 -- 172.21.15.16:0/3191442886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84005290 con 0x7fad940b13c0 2024-09-10T21:12:48.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad9b000640 1 -- 172.21.15.16:0/3191442886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad940b5950 con 0x7fad940b13c0 2024-09-10T21:12:48.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.044+0000 7fad9b000640 1 -- 172.21.15.16:0/3191442886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad940b5f10 con 0x7fad940b13c0 2024-09-10T21:12:48.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.048+0000 7fad9b000640 1 -- 172.21.15.16:0/3191442886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad940ca2b0 con 0x7fad940b13c0 2024-09-10T21:12:48.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.048+0000 7fad90c00640 1 -- 172.21.15.16:0/3191442886 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fad84005430 con 0x7fad940b13c0 2024-09-10T21:12:48.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.048+0000 7fad90c00640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fad700725e0 0x7fad70074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:48.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.048+0000 7fad90c00640 1 -- 172.21.15.16:0/3191442886 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7fad8412d930 con 0x7fad940b13c0 2024-09-10T21:12:48.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.048+0000 7fad8be00640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fad700725e0 0x7fad70074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:48.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.048+0000 7fad8be00640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fad700725e0 0x7fad70074aa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fad941f9040 tx=0x7fad7c00b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:48.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.052+0000 7fad90c00640 1 -- 172.21.15.16:0/3191442886 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fad940ca2b0 con 0x7fad940b13c0 2024-09-10T21:12:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.228+0000 7fad9b000640 1 -- 172.21.15.16:0/3191442886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fad940b0810 con 0x7fad940b13c0 2024-09-10T21:12:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.228+0000 7fad90c00640 1 -- 172.21.15.16:0/3191442886 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+1998 (secure 0 0 0) 0x7fad840c6020 con 0x7fad940b13c0 2024-09-10T21:12:48.236 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:12:48.236 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:12:48.236 INFO:teuthology.orchestra.run.smithi016.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:12:48.236 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:12:48.236 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4226.427518 2024-09-10T21:12:48.236 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5426.620519 2024-09-10T21:12:48.236 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4255.210203 2024-09-10T21:12:48.236 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4255.734871 2024-09-10T21:12:48.236 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4192.833008 2024-09-10T21:12:48.236 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4136.940759 2024-09-10T21:12:48.236 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4217.384701 2024-09-10T21:12:48.236 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5220.337972 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-10T21:12:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:12:48.238 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:12:48.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.232+0000 7fad9b000640 1 -- 172.21.15.16:0/3191442886 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fad700725e0 msgr2=0x7fad70074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:48.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.232+0000 7fad9b000640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fad700725e0 0x7fad70074aa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fad941f9040 tx=0x7fad7c00b2e0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.232+0000 7fad9b000640 1 -- 172.21.15.16:0/3191442886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad940b13c0 msgr2=0x7fad94148020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:48.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.232+0000 7fad9b000640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad940b13c0 0x7fad94148020 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fad84000c10 tx=0x7fad840a6660 comp rx=0 tx=0).stop 2024-09-10T21:12:48.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.232+0000 7fad9b000640 1 -- 172.21.15.16:0/3191442886 shutdown_connections 2024-09-10T21:12:48.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.232+0000 7fad9b000640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fad700725e0 0x7fad70074aa0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.232+0000 7fad9b000640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fad9415f400 0x7fad940ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.232+0000 7fad9b000640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad940cfc80 0x7fad940b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.232+0000 7fad9b000640 1 --2- 172.21.15.16:0/3191442886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad940b13c0 0x7fad94148020 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.232+0000 7fad9b000640 1 -- 172.21.15.16:0/3191442886 >> 172.21.15.16:0/3191442886 conn(0x7fad9409b130 msgr2=0x7fad9415e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:48.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.232+0000 7fad9b000640 1 -- 172.21.15.16:0/3191442886 shutdown_connections 2024-09-10T21:12:48.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.232+0000 7fad9b000640 1 -- 172.21.15.16:0/3191442886 wait complete. 2024-09-10T21:12:48.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:12:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 -- 172.21.15.16:0/4233897120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6400b0eb0 msgr2=0x7fd6400aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 --2- 172.21.15.16:0/4233897120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6400b0eb0 0x7fd6400aee80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd638009c80 tx=0x7fd63801cd70 comp rx=0 tx=0).stop 2024-09-10T21:12:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 -- 172.21.15.16:0/4233897120 shutdown_connections 2024-09-10T21:12:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 --2- 172.21.15.16:0/4233897120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6400b0eb0 0x7fd6400aee80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 --2- 172.21.15.16:0/4233897120 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6400b0980 0x7fd6400ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 -- 172.21.15.16:0/4233897120 >> 172.21.15.16:0/4233897120 conn(0x7fd6400d1c70 msgr2=0x7fd6400d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 -- 172.21.15.16:0/4233897120 shutdown_connections 2024-09-10T21:12:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 -- 172.21.15.16:0/4233897120 wait complete. 2024-09-10T21:12:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 Processor -- start 2024-09-10T21:12:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 -- start start 2024-09-10T21:12:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6400b0980 0x7fd640155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6400b0eb0 0x7fd6400b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6400ce8c0 0x7fd6400718d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd640169220 con 0x7fd6400b0eb0 2024-09-10T21:12:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd6401690a0 con 0x7fd6400ce8c0 2024-09-10T21:12:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.400+0000 7fd645800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd6401693a0 con 0x7fd6400b0980 2024-09-10T21:12:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd637e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6400b0eb0 0x7fd6400b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd637e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6400b0eb0 0x7fd6400b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47828/0 (socket says 172.21.15.16:47828) 2024-09-10T21:12:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd637e00640 1 -- 172.21.15.16:0/4249162688 learned_addr learned my addr 172.21.15.16:0/4249162688 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd644e00640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6400ce8c0 0x7fd6400718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd63fe00640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6400b0980 0x7fd640155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd637e00640 1 -- 172.21.15.16:0/4249162688 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6400b0980 msgr2=0x7fd640155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd637e00640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6400b0980 0x7fd640155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd637e00640 1 -- 172.21.15.16:0/4249162688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6400ce8c0 msgr2=0x7fd6400718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd637e00640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6400ce8c0 0x7fd6400718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd637e00640 1 -- 172.21.15.16:0/4249162688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6401f7b10 con 0x7fd6400b0eb0 2024-09-10T21:12:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd644e00640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6400ce8c0 0x7fd6400718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd637e00640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6400b0eb0 0x7fd6400b4aa0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c970 tx=0x7fd62800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd63cc00640 1 -- 172.21.15.16:0/4249162688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628007c10 con 0x7fd6400b0eb0 2024-09-10T21:12:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd63cc00640 1 -- 172.21.15.16:0/4249162688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd62800f450 con 0x7fd6400b0eb0 2024-09-10T21:12:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd63cc00640 1 -- 172.21.15.16:0/4249162688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62800f8a0 con 0x7fd6400b0eb0 2024-09-10T21:12:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd645800640 1 -- 172.21.15.16:0/4249162688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6401f7df0 con 0x7fd6400b0eb0 2024-09-10T21:12:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd645800640 1 -- 172.21.15.16:0/4249162688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6401f83b0 con 0x7fd6400b0eb0 2024-09-10T21:12:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd645800640 1 -- 172.21.15.16:0/4249162688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd640155710 con 0x7fd6400b0eb0 2024-09-10T21:12:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.404+0000 7fd63cc00640 1 -- 172.21.15.16:0/4249162688 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd62800fa40 con 0x7fd6400b0eb0 2024-09-10T21:12:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.408+0000 7fd63cc00640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd61c0725e0 0x7fd61c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.408+0000 7fd63cc00640 1 -- 172.21.15.16:0/4249162688 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7fd628093770 con 0x7fd6400b0eb0 2024-09-10T21:12:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.408+0000 7fd63fe00640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd61c0725e0 0x7fd61c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:48.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.408+0000 7fd63fe00640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd61c0725e0 0x7fd61c074aa0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd630000f70 tx=0x7fd630009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.412+0000 7fd63cc00640 1 -- 172.21.15.16:0/4249162688 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd628014030 con 0x7fd6400b0eb0 2024-09-10T21:12:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.584+0000 7fd645800640 1 -- 172.21.15.16:0/4249162688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd6401f8580 con 0x7fd6400b0eb0 2024-09-10T21:12:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.588+0000 7fd63cc00640 1 -- 172.21.15.16:0/4249162688 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fd62805f140 con 0x7fd6400b0eb0 2024-09-10T21:12:48.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.588+0000 7fd645800640 1 -- 172.21.15.16:0/4249162688 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd61c0725e0 msgr2=0x7fd61c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:48.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.588+0000 7fd645800640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd61c0725e0 0x7fd61c074aa0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd630000f70 tx=0x7fd630009040 comp rx=0 tx=0).stop 2024-09-10T21:12:48.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.588+0000 7fd645800640 1 -- 172.21.15.16:0/4249162688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6400b0eb0 msgr2=0x7fd6400b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:48.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.588+0000 7fd645800640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6400b0eb0 0x7fd6400b4aa0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c970 tx=0x7fd62800ce30 comp rx=0 tx=0).stop 2024-09-10T21:12:48.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.588+0000 7fd645800640 1 -- 172.21.15.16:0/4249162688 shutdown_connections 2024-09-10T21:12:48.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.588+0000 7fd645800640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd61c0725e0 0x7fd61c074aa0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.588+0000 7fd645800640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6400ce8c0 0x7fd6400718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.588+0000 7fd645800640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6400b0eb0 0x7fd6400b4aa0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.588+0000 7fd645800640 1 --2- 172.21.15.16:0/4249162688 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6400b0980 0x7fd640155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:48.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.588+0000 7fd645800640 1 -- 172.21.15.16:0/4249162688 >> 172.21.15.16:0/4249162688 conn(0x7fd6400d1c70 msgr2=0x7fd6401584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:48.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.588+0000 7fd645800640 1 -- 172.21.15.16:0/4249162688 shutdown_connections 2024-09-10T21:12:48.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:48.588+0000 7fd645800640 1 -- 172.21.15.16:0/4249162688 wait complete. 2024-09-10T21:12:48.616 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:12:48.616 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:12:48.687 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:12:48.780 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:12:48.780 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:12:48.780 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:48.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:12:48.789 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:12:48.789 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:48.799 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:12:48.799 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:12:48.799 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:48.841 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:12:48.841 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:12:48.841 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:12:48.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:12:48.851 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:12:48.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:49.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.024+0000 7f4ccfa00640 1 -- 172.21.15.16:0/4130031536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cb0004d50 msgr2=0x7f4cb00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:49.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.024+0000 7f4ccfa00640 1 --2- 172.21.15.16:0/4130031536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cb0004d50 0x7f4cb00051d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80c6ba0 tx=0x7f4cc001e3f0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.024+0000 7f4ccda00640 1 -- 172.21.15.16:0/4130031536 reap_dead start 2024-09-10T21:12:49.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.024+0000 7f4ccfa00640 1 -- 172.21.15.16:0/4130031536 shutdown_connections 2024-09-10T21:12:49.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.024+0000 7f4ccfa00640 1 -- 172.21.15.16:0/4130031536 >> 172.21.15.16:0/4130031536 conn(0x7f4cc80cfb20 msgr2=0x7f4cc80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:49.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.024+0000 7f4ccfa00640 1 -- 172.21.15.16:0/4130031536 shutdown_connections 2024-09-10T21:12:49.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.024+0000 7f4ccfa00640 1 -- 172.21.15.16:0/4130031536 wait complete. 2024-09-10T21:12:49.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccfa00640 1 Processor -- start 2024-09-10T21:12:49.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccfa00640 1 -- start start 2024-09-10T21:12:49.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccfa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cb0004d50 0x7f4cc8150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:49.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccfa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4cc80a9440 0x7f4cc80a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:49.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccfa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc80c8dc0 0x7f4cc80c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:49.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccfa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc80ca830 con 0x7f4cb0004d50 2024-09-10T21:12:49.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccfa00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc80ca6b0 con 0x7f4cc80c8dc0 2024-09-10T21:12:49.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccfa00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc80ca9b0 con 0x7f4cc80a9440 2024-09-10T21:12:49.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cb0004d50 0x7f4cc8150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:49.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cb0004d50 0x7f4cc8150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47848/0 (socket says 172.21.15.16:47848) 2024-09-10T21:12:49.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccd000640 1 -- 172.21.15.16:0/2937036315 learned_addr learned my addr 172.21.15.16:0/2937036315 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:49.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccd000640 1 -- 172.21.15.16:0/2937036315 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4cc80a9440 msgr2=0x7f4cc80a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:49.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccd000640 1 --2- 172.21.15.16:0/2937036315 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4cc80a9440 0x7f4cc80a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccd000640 1 -- 172.21.15.16:0/2937036315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc80c8dc0 msgr2=0x7f4cc80c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:49.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccd000640 1 --2- 172.21.15.16:0/2937036315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc80c8dc0 0x7f4cc80c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccd000640 1 -- 172.21.15.16:0/2937036315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc80af250 con 0x7f4cb0004d50 2024-09-10T21:12:49.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccd000640 1 --2- 172.21.15.16:0/2937036315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cb0004d50 0x7f4cc8150020 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc012670 tx=0x7f4cbc012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4cc4c00640 1 -- 172.21.15.16:0/2937036315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc010a60 con 0x7f4cb0004d50 2024-09-10T21:12:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4cc4c00640 1 -- 172.21.15.16:0/2937036315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cbc017030 con 0x7f4cb0004d50 2024-09-10T21:12:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccfa00640 1 -- 172.21.15.16:0/2937036315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc80a9f00 con 0x7f4cb0004d50 2024-09-10T21:12:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4cc4c00640 1 -- 172.21.15.16:0/2937036315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc013450 con 0x7f4cb0004d50 2024-09-10T21:12:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.028+0000 7f4ccfa00640 1 -- 172.21.15.16:0/2937036315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc80af4f0 con 0x7f4cb0004d50 2024-09-10T21:12:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.032+0000 7f4ccfa00640 1 -- 172.21.15.16:0/2937036315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc80ca170 con 0x7f4cb0004d50 2024-09-10T21:12:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.032+0000 7f4cc4c00640 1 -- 172.21.15.16:0/2937036315 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4cbc0135f0 con 0x7f4cb0004d50 2024-09-10T21:12:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.032+0000 7f4cc4c00640 1 --2- 172.21.15.16:0/2937036315 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ca80725e0 0x7f4ca8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:49.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.032+0000 7f4cc4c00640 1 -- 172.21.15.16:0/2937036315 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f4cbc096cb0 con 0x7f4cb0004d50 2024-09-10T21:12:49.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.032+0000 7f4cc7e00640 1 --2- 172.21.15.16:0/2937036315 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ca80725e0 0x7f4ca8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:49.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.032+0000 7f4cc7e00640 1 --2- 172.21.15.16:0/2937036315 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ca80725e0 0x7f4ca8074aa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80ca060 tx=0x7f4cb4009670 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:49.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.036+0000 7f4cc4c00640 1 -- 172.21.15.16:0/2937036315 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4cbc0625f0 con 0x7f4cb0004d50 2024-09-10T21:12:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4ccfa00640 1 -- 172.21.15.16:0/2937036315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cc80b5aa0 con 0x7f4cb0004d50 2024-09-10T21:12:49.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4cc4c00640 1 -- 172.21.15.16:0/2937036315 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+3915 (secure 0 0 0) 0x7f4cbc061ef0 con 0x7f4cb0004d50 2024-09-10T21:12:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:49.251 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-10T21:12:42:665056+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3694613625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3694613625},{"type":"v1","addr":"172.21.15.16:6836","nonce":3694613625}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1264024538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1264024538},{"type":"v1","addr":"172.21.15.52:6835","nonce":1264024538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":134768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2899191048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":2899191048},{"type":"v1","addr":"172.21.15.16:6837","nonce":2899191048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":134915,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/1737118866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1737118866},{"type":"v1","addr":"172.21.15.52:6834","nonce":1737118866}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[]} 2024-09-10T21:12:49.251 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-10T21:12:49.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4ccfa00640 1 -- 172.21.15.16:0/2937036315 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ca80725e0 msgr2=0x7f4ca8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4ccfa00640 1 --2- 172.21.15.16:0/2937036315 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ca80725e0 0x7f4ca8074aa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80ca060 tx=0x7f4cb4009670 comp rx=0 tx=0).stop 2024-09-10T21:12:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4ccfa00640 1 -- 172.21.15.16:0/2937036315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cb0004d50 msgr2=0x7f4cc8150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4ccfa00640 1 --2- 172.21.15.16:0/2937036315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cb0004d50 0x7f4cc8150020 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc012670 tx=0x7f4cbc012b30 comp rx=0 tx=0).stop 2024-09-10T21:12:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4ccfa00640 1 -- 172.21.15.16:0/2937036315 shutdown_connections 2024-09-10T21:12:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4ccfa00640 1 --2- 172.21.15.16:0/2937036315 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ca80725e0 0x7f4ca8074aa0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4ccfa00640 1 --2- 172.21.15.16:0/2937036315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc80c8dc0 0x7f4cc80c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4ccfa00640 1 --2- 172.21.15.16:0/2937036315 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4cc80a9440 0x7f4cc80a98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4ccfa00640 1 --2- 172.21.15.16:0/2937036315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cb0004d50 0x7f4cc8150020 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4ccfa00640 1 -- 172.21.15.16:0/2937036315 >> 172.21.15.16:0/2937036315 conn(0x7f4cc80cfb20 msgr2=0x7f4cc815ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4ccfa00640 1 -- 172.21.15.16:0/2937036315 shutdown_connections 2024-09-10T21:12:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.244+0000 7f4ccfa00640 1 -- 172.21.15.16:0/2937036315 wait complete. 2024-09-10T21:12:49.273 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:12:49.273 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:12:49.273 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:12:49.273 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:12:49.274 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:12:49.274 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:12:49.274 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:12:49.274 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:12:49.274 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:12:49.274 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:12:49.274 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:12:49.275 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:12:49.275 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:12:49.275 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:12:49.275 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:12:49.275 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:12:49.287 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:12:49.287 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:12:49.287 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:12:49.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:12:49.290 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:12:49.290 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:12:49.291 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:12:49.291 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:12:49.331 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:12:49.332 DEBUG:teuthology.parallel:result is None 2024-09-10T21:12:49.333 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:12:49.333 DEBUG:teuthology.parallel:result is None 2024-09-10T21:12:49.340 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:12:49.340 DEBUG:teuthology.parallel:result is None 2024-09-10T21:12:49.342 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:12:49.342 DEBUG:teuthology.parallel:result is None 2024-09-10T21:12:49.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:12:49.420 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-10T21:12:49.421 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-10T21:12:49.436 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:12:49.436 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:12:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.488+0000 7f7a25000640 1 -- 172.21.15.16:0/2931290648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a200716f0 msgr2=0x7f7a200af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.488+0000 7f7a25000640 1 --2- 172.21.15.16:0/2931290648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a200716f0 0x7f7a200af580 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7a200cc360 tx=0x7f7a0c01c7a0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.488+0000 7f7a25000640 1 -- 172.21.15.16:0/2931290648 shutdown_connections 2024-09-10T21:12:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.488+0000 7f7a25000640 1 --2- 172.21.15.16:0/2931290648 >> v2:172.21.15.52:3300/0 conn(0x7f7a200b13c0 0x7f7a200afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.488+0000 7f7a25000640 1 --2- 172.21.15.16:0/2931290648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a200716f0 0x7f7a200af580 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.488+0000 7f7a25000640 1 -- 172.21.15.16:0/2931290648 >> 172.21.15.16:0/2931290648 conn(0x7f7a2009b130 msgr2=0x7f7a200d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:49.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.488+0000 7f7a25000640 1 -- 172.21.15.16:0/2931290648 shutdown_connections 2024-09-10T21:12:49.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.488+0000 7f7a25000640 1 -- 172.21.15.16:0/2931290648 wait complete. 2024-09-10T21:12:49.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a25000640 1 Processor -- start 2024-09-10T21:12:49.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a25000640 1 -- start start 2024-09-10T21:12:49.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a25000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a200716f0 0x7f7a2006e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:49.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a25000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a200b13c0 0x7f7a20166380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a25000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a2015f400 0x7f7a2015de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a25000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20168f60 con 0x7f7a200b13c0 2024-09-10T21:12:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a25000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20168de0 con 0x7f7a200716f0 2024-09-10T21:12:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a25000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7a201690e0 con 0x7f7a2015f400 2024-09-10T21:12:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a1d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a200b13c0 0x7f7a20166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a1d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a200b13c0 0x7f7a20166380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47892/0 (socket says 172.21.15.16:47892) 2024-09-10T21:12:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a1d400640 1 -- 172.21.15.16:0/467441032 learned_addr learned my addr 172.21.15.16:0/467441032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a1e800640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a2015f400 0x7f7a2015de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a1de00640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a200716f0 0x7f7a2006e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a1d400640 1 -- 172.21.15.16:0/467441032 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a2015f400 msgr2=0x7f7a2015de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a1d400640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a2015f400 0x7f7a2015de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a1d400640 1 -- 172.21.15.16:0/467441032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a200716f0 msgr2=0x7f7a2006e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a1d400640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a200716f0 0x7f7a2006e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a1d400640 1 -- 172.21.15.16:0/467441032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2006dcf0 con 0x7f7a200b13c0 2024-09-10T21:12:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a1d400640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a200b13c0 0x7f7a20166380 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0b0040 tx=0x7f7a0c0a55f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a16000640 1 -- 172.21.15.16:0/467441032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0b2040 con 0x7f7a200b13c0 2024-09-10T21:12:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a16000640 1 -- 172.21.15.16:0/467441032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a0c004490 con 0x7f7a200b13c0 2024-09-10T21:12:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a25000640 1 -- 172.21.15.16:0/467441032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a20164320 con 0x7f7a200b13c0 2024-09-10T21:12:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a16000640 1 -- 172.21.15.16:0/467441032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c005280 con 0x7f7a200b13c0 2024-09-10T21:12:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.492+0000 7f7a25000640 1 -- 172.21.15.16:0/467441032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2006df90 con 0x7f7a200b13c0 2024-09-10T21:12:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.496+0000 7f7a25000640 1 -- 172.21.15.16:0/467441032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a20070fe0 con 0x7f7a200b13c0 2024-09-10T21:12:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.496+0000 7f7a16000640 1 -- 172.21.15.16:0/467441032 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7a0c005420 con 0x7f7a200b13c0 2024-09-10T21:12:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.496+0000 7f7a16000640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a00072610 0x7f7a00074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:49.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.496+0000 7f7a16000640 1 -- 172.21.15.16:0/467441032 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f7a0c12c9d0 con 0x7f7a200b13c0 2024-09-10T21:12:49.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.496+0000 7f7a1de00640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a00072610 0x7f7a00074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:49.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.496+0000 7f7a1de00640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a00072610 0x7f7a00074ad0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a100015e0 tx=0x7f7a10009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:49.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.500+0000 7f7a16000640 1 -- 172.21.15.16:0/467441032 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7a0c0164b0 con 0x7f7a200b13c0 2024-09-10T21:12:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.688+0000 7f7a25000640 1 -- 172.21.15.16:0/467441032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7a200b73e0 con 0x7f7a200b13c0 2024-09-10T21:12:49.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.876+0000 7f7a16000640 1 -- 172.21.15.16:0/467441032 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v51) ==== 97+0+0 (secure 0 0 0) 0x7f7a0c0f8320 con 0x7f7a200b13c0 2024-09-10T21:12:49.885 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-10T21:12:49.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.880+0000 7f7a25000640 1 -- 172.21.15.16:0/467441032 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a00072610 msgr2=0x7f7a00074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:49.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.880+0000 7f7a25000640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a00072610 0x7f7a00074ad0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a100015e0 tx=0x7f7a10009040 comp rx=0 tx=0).stop 2024-09-10T21:12:49.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.880+0000 7f7a25000640 1 -- 172.21.15.16:0/467441032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a200b13c0 msgr2=0x7f7a20166380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:49.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.880+0000 7f7a25000640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a200b13c0 0x7f7a20166380 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0b0040 tx=0x7f7a0c0a55f0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.880+0000 7f7a25000640 1 -- 172.21.15.16:0/467441032 shutdown_connections 2024-09-10T21:12:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.880+0000 7f7a25000640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a00072610 0x7f7a00074ad0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.880+0000 7f7a25000640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a2015f400 0x7f7a2015de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.880+0000 7f7a25000640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a200b13c0 0x7f7a20166380 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.880+0000 7f7a25000640 1 --2- 172.21.15.16:0/467441032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a200716f0 0x7f7a2006e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.880+0000 7f7a25000640 1 -- 172.21.15.16:0/467441032 >> 172.21.15.16:0/467441032 conn(0x7f7a2009b130 msgr2=0x7f7a200af510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.880+0000 7f7a25000640 1 -- 172.21.15.16:0/467441032 shutdown_connections 2024-09-10T21:12:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:49.880+0000 7f7a25000640 1 -- 172.21.15.16:0/467441032 wait complete. 2024-09-10T21:12:49.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:12:50.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.052+0000 7f0267800640 1 -- 172.21.15.16:0/3624146972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b13c0 msgr2=0x7f02600afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:50.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.052+0000 7f0267800640 1 --2- 172.21.15.16:0/3624146972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b13c0 0x7f02600afc90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0258009c80 tx=0x7f025801cc80 comp rx=0 tx=0).stop 2024-09-10T21:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.052+0000 7f0267800640 1 -- 172.21.15.16:0/3624146972 shutdown_connections 2024-09-10T21:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.052+0000 7f0267800640 1 --2- 172.21.15.16:0/3624146972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b13c0 0x7f02600afc90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.052+0000 7f0267800640 1 -- 172.21.15.16:0/3624146972 >> 172.21.15.16:0/3624146972 conn(0x7f026009b130 msgr2=0x7f02600d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.052+0000 7f0267800640 1 -- 172.21.15.16:0/3624146972 shutdown_connections 2024-09-10T21:12:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.052+0000 7f0267800640 1 -- 172.21.15.16:0/3624146972 wait complete. 2024-09-10T21:12:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.056+0000 7f0267800640 1 Processor -- start 2024-09-10T21:12:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.056+0000 7f0267800640 1 -- start start 2024-09-10T21:12:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.056+0000 7f0267800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02600716f0 0x7f02601569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.056+0000 7f0267800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b13c0 0x7f02600b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.056+0000 7f0267800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f026015f400 0x7f02600b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.056+0000 7f0267800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f026016a060 con 0x7f02600b13c0 2024-09-10T21:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.056+0000 7f0267800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0260169ee0 con 0x7f02600716f0 2024-09-10T21:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.056+0000 7f0267800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f026016a1e0 con 0x7f026015f400 2024-09-10T21:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.056+0000 7f025fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b13c0 0x7f02600b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.056+0000 7f025fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b13c0 0x7f02600b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47904/0 (socket says 172.21.15.16:47904) 2024-09-10T21:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.056+0000 7f025fe00640 1 -- 172.21.15.16:0/795704774 learned_addr learned my addr 172.21.15.16:0/795704774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.056+0000 7f0264e00640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02600716f0 0x7f02601569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.056+0000 7f0265800640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f026015f400 0x7f02600b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.060+0000 7f025fe00640 1 -- 172.21.15.16:0/795704774 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f026015f400 msgr2=0x7f02600b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.060+0000 7f025fe00640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f026015f400 0x7f02600b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.060+0000 7f025fe00640 1 -- 172.21.15.16:0/795704774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02600716f0 msgr2=0x7f02601569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.060+0000 7f025fe00640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02600716f0 0x7f02601569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.060+0000 7f025fe00640 1 -- 172.21.15.16:0/795704774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026014dd60 con 0x7f02600b13c0 2024-09-10T21:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.060+0000 7f025fe00640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b13c0 0x7f02600b4bb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c450 tx=0x7f024c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.060+0000 7f025cc00640 1 -- 172.21.15.16:0/795704774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c012830 con 0x7f02600b13c0 2024-09-10T21:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.060+0000 7f025cc00640 1 -- 172.21.15.16:0/795704774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f024c017070 con 0x7f02600b13c0 2024-09-10T21:12:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.060+0000 7f025cc00640 1 -- 172.21.15.16:0/795704774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c012d80 con 0x7f02600b13c0 2024-09-10T21:12:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.060+0000 7f0267800640 1 -- 172.21.15.16:0/795704774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026014da80 con 0x7f02600b13c0 2024-09-10T21:12:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.060+0000 7f0267800640 1 -- 172.21.15.16:0/795704774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02601678c0 con 0x7f02600b13c0 2024-09-10T21:12:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.060+0000 7f0267800640 1 -- 172.21.15.16:0/795704774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0260070fe0 con 0x7f02600b13c0 2024-09-10T21:12:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.060+0000 7f025cc00640 1 -- 172.21.15.16:0/795704774 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f024c01c460 con 0x7f02600b13c0 2024-09-10T21:12:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.064+0000 7f025cc00640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f02400725e0 0x7f0240074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.064+0000 7f025cc00640 1 -- 172.21.15.16:0/795704774 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7f024c092d20 con 0x7f02600b13c0 2024-09-10T21:12:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.064+0000 7f0264e00640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f02400725e0 0x7f0240074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.064+0000 7f0264e00640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f02400725e0 0x7f0240074aa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f02540015e0 tx=0x7f0254009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:50.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.068+0000 7f025cc00640 1 -- 172.21.15.16:0/795704774 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f024c05e6b0 con 0x7f02600b13c0 2024-09-10T21:12:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.304+0000 7f0267800640 1 -- 172.21.15.16:0/795704774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0260157b90 con 0x7f02600b13c0 2024-09-10T21:12:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.304+0000 7f025cc00640 1 -- 172.21.15.16:0/795704774 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5304 (secure 0 0 0) 0x7f024c05dfb0 con 0x7f02600b13c0 2024-09-10T21:12:50.311 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:50.311 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:12:50.311 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-10T21:12:50.311 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.311 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-10T21:12:50.311 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmhg+lJBAAWjuYW75BGo2dmyAUi7s9+A==", 2024-09-10T21:12:50.312 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.312 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:50.312 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:12:50.312 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:12:50.312 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.312 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.312 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.312 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-10T21:12:50.312 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmQr7+MBAAOzXhWIirBOoi/7wKnrFgmg==", 2024-09-10T21:12:50.312 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.313 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:50.313 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:12:50.313 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:12:50.313 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.313 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.313 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.313 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.313 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-10T21:12:50.313 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmxxoALBAAgkmw91/VP1Uv2WMZsJKAyA==", 2024-09-10T21:12:50.313 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.314 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:50.314 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:12:50.314 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:12:50.314 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.314 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.314 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.314 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-10T21:12:50.314 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmA4tzOBAAiS+8n+Gokz6yXapVf2ecYg==", 2024-09-10T21:12:50.314 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.314 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:50.314 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:12:50.315 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:12:50.315 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.315 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.315 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.315 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-10T21:12:50.315 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHteBmRoahLRAA32PsUkqtGkeI7tMFRwas5Q==", 2024-09-10T21:12:50.315 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.315 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:50.315 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:50.316 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.316 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.316 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.316 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-10T21:12:50.316 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCLteBmGmEMBhAAr1yQEf/NkiN4+ezAQ9VR2A==", 2024-09-10T21:12:50.316 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.316 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:50.316 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:50.316 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.316 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.317 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.317 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-10T21:12:50.317 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCOteBmmMuAHBAAt+G/KApsZymjTLRSmyRbdw==", 2024-09-10T21:12:50.317 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.317 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:50.317 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:50.317 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.317 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.317 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.317 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-10T21:12:50.317 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCRteBmoQfNMBAA0vnIqlYoI13KNAAfNmJg3w==", 2024-09-10T21:12:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-10T21:12:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCWteBmc/7lDxAADlSi4UpcFJVNdp6YTkpdSw==", 2024-09-10T21:12:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.319 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.319 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.319 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-10T21:12:50.319 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZteBmBZ1LJhAAPcbZa1LQu4kNbSsJ/8Z81A==", 2024-09-10T21:12:50.319 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.319 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:50.319 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:50.319 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.319 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.319 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.320 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-10T21:12:50.320 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCcteBm7/czOxAAqREnY84+LF1N9dRXOhzz6g==", 2024-09-10T21:12:50.320 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.320 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:50.320 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:50.320 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.320 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.320 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.320 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-10T21:12:50.320 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgteBmG2J8FRAAB+cMFCnf9wJg1R3RD5SUJw==", 2024-09-10T21:12:50.320 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.320 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:12:50.321 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:12:50.321 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.321 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.321 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.321 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-10T21:12:50.321 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA==", 2024-09-10T21:12:50.321 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.321 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:50.321 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-10T21:12:50.321 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:12:50.321 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.322 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.322 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-10T21:12:50.322 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmuEm4ChAAe08QvhwIcN8ZHa3AoVcYww==", 2024-09-10T21:12:50.322 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.322 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:50.322 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:12:50.322 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:12:50.322 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:12:50.322 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.322 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-10T21:12:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBm50o1BxAAIFbVxVqU4rU8eKYpuZjTHA==", 2024-09-10T21:12:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:12:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:12:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:12:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-10T21:12:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmxtu0EBAAmN7+07lDiwjsIC/pNYUlSw==", 2024-09-10T21:12:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.324 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:12:50.324 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:12:50.324 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:12:50.324 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:12:50.324 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.324 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.324 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-10T21:12:50.324 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmfqgYCBAAm00tBe45phGieCcp1q46cw==", 2024-09-10T21:12:50.324 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.324 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:12:50.324 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:12:50.325 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:12:50.325 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.325 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.325 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.325 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-10T21:12:50.325 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmMVABIBAAoB72mPyXyUO6Y6k+fMqQQg==", 2024-09-10T21:12:50.325 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.325 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:12:50.325 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:12:50.325 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.325 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:12:50.326 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.326 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-10T21:12:50.326 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmeyYxHBAAvBkpTzjPQKAOlHzI1jQ46g==", 2024-09-10T21:12:50.326 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:12:50.326 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:12:50.326 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:12:50.326 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:12:50.326 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.326 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.326 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-10T21:12:50.326 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:12:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.308+0000 7f0267800640 1 -- 172.21.15.16:0/795704774 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f02400725e0 msgr2=0x7f0240074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.308+0000 7f0267800640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f02400725e0 0x7f0240074aa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f02540015e0 tx=0x7f0254009040 comp rx=0 tx=0).stop 2024-09-10T21:12:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.308+0000 7f0267800640 1 -- 172.21.15.16:0/795704774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b13c0 msgr2=0x7f02600b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.308+0000 7f0267800640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b13c0 0x7f02600b4bb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c450 tx=0x7f024c00c910 comp rx=0 tx=0).stop 2024-09-10T21:12:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.308+0000 7f0267800640 1 -- 172.21.15.16:0/795704774 shutdown_connections 2024-09-10T21:12:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.308+0000 7f0267800640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f02400725e0 0x7f0240074aa0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.308+0000 7f0267800640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f026015f400 0x7f02600b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.308+0000 7f0267800640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02600b13c0 0x7f02600b4bb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.308+0000 7f0267800640 1 --2- 172.21.15.16:0/795704774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02600716f0 0x7f02601569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.308+0000 7f0267800640 1 -- 172.21.15.16:0/795704774 >> 172.21.15.16:0/795704774 conn(0x7f026009b130 msgr2=0x7f026015b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.308+0000 7f0267800640 1 -- 172.21.15.16:0/795704774 shutdown_connections 2024-09-10T21:12:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.308+0000 7f0267800640 1 -- 172.21.15.16:0/795704774 wait complete. 2024-09-10T21:12:50.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:12:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 -- 172.21.15.16:0/2380571342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c8004810 msgr2=0x7f73c8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 --2- 172.21.15.16:0/2380571342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c8004810 0x7f73c8004c90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f73cc006e90 tx=0x7f73cc01cde0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e6400640 1 -- 172.21.15.16:0/2380571342 reap_dead start 2024-09-10T21:12:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 -- 172.21.15.16:0/2380571342 shutdown_connections 2024-09-10T21:12:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 -- 172.21.15.16:0/2380571342 >> 172.21.15.16:0/2380571342 conn(0x7f73e00cfb20 msgr2=0x7f73e00cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 -- 172.21.15.16:0/2380571342 shutdown_connections 2024-09-10T21:12:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 -- 172.21.15.16:0/2380571342 wait complete. 2024-09-10T21:12:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 Processor -- start 2024-09-10T21:12:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 -- start start 2024-09-10T21:12:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f73c8004810 0x7f73e0148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e00c6040 0x7f73e00c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73e00c9300 0x7f73e01f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0072ed0 con 0x7f73e00c6040 2024-09-10T21:12:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0072d50 con 0x7f73e00c9300 2024-09-10T21:12:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e8400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f73e00c7ad0 con 0x7f73c8004810 2024-09-10T21:12:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e00c6040 0x7f73e00c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e00c6040 0x7f73e00c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47926/0 (socket says 172.21.15.16:47926) 2024-09-10T21:12:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e5000640 1 -- 172.21.15.16:0/1317173426 learned_addr learned my addr 172.21.15.16:0/1317173426 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:50.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e5a00640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f73c8004810 0x7f73e0148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e6400640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73e00c9300 0x7f73e01f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e5000640 1 -- 172.21.15.16:0/1317173426 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f73c8004810 msgr2=0x7f73e0148f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e5000640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f73c8004810 0x7f73e0148f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e5000640 1 -- 172.21.15.16:0/1317173426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73e00c9300 msgr2=0x7f73e01f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e5000640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73e00c9300 0x7f73e01f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.476+0000 7f73e5000640 1 -- 172.21.15.16:0/1317173426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e0149480 con 0x7f73e00c6040 2024-09-10T21:12:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.480+0000 7f73e5000640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e00c6040 0x7f73e00c8dc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f73cc006040 tx=0x7f73cc001be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.480+0000 7f73dd600640 1 -- 172.21.15.16:0/1317173426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc0014e0 con 0x7f73e00c6040 2024-09-10T21:12:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.480+0000 7f73dd600640 1 -- 172.21.15.16:0/1317173426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73cc006280 con 0x7f73e00c6040 2024-09-10T21:12:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.480+0000 7f73dd600640 1 -- 172.21.15.16:0/1317173426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc001680 con 0x7f73e00c6040 2024-09-10T21:12:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.480+0000 7f73e8400640 1 -- 172.21.15.16:0/1317173426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e00c7d20 con 0x7f73e00c6040 2024-09-10T21:12:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.480+0000 7f73e8400640 1 -- 172.21.15.16:0/1317173426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e00c82e0 con 0x7f73e00c6040 2024-09-10T21:12:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.480+0000 7f73e8400640 1 -- 172.21.15.16:0/1317173426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e006e640 con 0x7f73e00c6040 2024-09-10T21:12:50.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.480+0000 7f73dd600640 1 -- 172.21.15.16:0/1317173426 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f73cc0c1030 con 0x7f73e00c6040 2024-09-10T21:12:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.484+0000 7f73dd600640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f73c00725e0 0x7f73c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.484+0000 7f73dd600640 1 -- 172.21.15.16:0/1317173426 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7f73cc12c740 con 0x7f73e00c6040 2024-09-10T21:12:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.484+0000 7f73e5a00640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f73c00725e0 0x7f73c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.484+0000 7f73e5a00640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f73c00725e0 0x7f73c0074aa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f73e00c8b40 tx=0x7f73d4006b20 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.488+0000 7f73dd600640 1 -- 172.21.15.16:0/1317173426 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f73cc016b90 con 0x7f73e00c6040 2024-09-10T21:12:50.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.660+0000 7f73e8400640 1 -- 172.21.15.16:0/1317173426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f73e00c96e0 con 0x7f73e00c6040 2024-09-10T21:12:50.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.660+0000 7f73dd600640 1 -- 172.21.15.16:0/1317173426 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v53) ==== 84+0+65 (secure 0 0 0) 0x7f73cc0f7a80 con 0x7f73e00c6040 2024-09-10T21:12:50.669 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:50.669 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-10T21:12:50.669 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:12:50.669 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-10T21:12:50.670 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-10T21:12:50.670 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:12:50.670 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-10T21:12:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.664+0000 7f73e8400640 1 -- 172.21.15.16:0/1317173426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f73c00725e0 msgr2=0x7f73c0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.664+0000 7f73e8400640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f73c00725e0 0x7f73c0074aa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f73e00c8b40 tx=0x7f73d4006b20 comp rx=0 tx=0).stop 2024-09-10T21:12:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.664+0000 7f73e8400640 1 -- 172.21.15.16:0/1317173426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e00c6040 msgr2=0x7f73e00c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.664+0000 7f73e8400640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e00c6040 0x7f73e00c8dc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f73cc006040 tx=0x7f73cc001be0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.664+0000 7f73e8400640 1 -- 172.21.15.16:0/1317173426 shutdown_connections 2024-09-10T21:12:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.664+0000 7f73e8400640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f73c00725e0 0x7f73c0074aa0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.664+0000 7f73e8400640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73e00c9300 0x7f73e01f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.664+0000 7f73e8400640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e00c6040 0x7f73e00c8dc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.664+0000 7f73e8400640 1 --2- 172.21.15.16:0/1317173426 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f73c8004810 0x7f73e0148f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.664+0000 7f73e8400640 1 -- 172.21.15.16:0/1317173426 >> 172.21.15.16:0/1317173426 conn(0x7f73e00cfb20 msgr2=0x7f73e015b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.664+0000 7f73e8400640 1 -- 172.21.15.16:0/1317173426 shutdown_connections 2024-09-10T21:12:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.664+0000 7f73e8400640 1 -- 172.21.15.16:0/1317173426 wait complete. 2024-09-10T21:12:50.691 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:12:50.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:12:50.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.836+0000 7f55b3e00640 1 -- 172.21.15.16:0/1577425835 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ac073bc0 msgr2=0x7f55ac073fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:50.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.836+0000 7f55b3e00640 1 --2- 172.21.15.16:0/1577425835 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ac073bc0 0x7f55ac073fa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009900 tx=0x7f55a001c790 comp rx=0 tx=0).stop 2024-09-10T21:12:50.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.836+0000 7f55b3e00640 1 -- 172.21.15.16:0/1577425835 shutdown_connections 2024-09-10T21:12:50.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.836+0000 7f55b3e00640 1 --2- 172.21.15.16:0/1577425835 >> v2:172.21.15.52:3300/0 conn(0x7f55ac0cda80 0x7f55ac15e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.836+0000 7f55b3e00640 1 --2- 172.21.15.16:0/1577425835 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ac073bc0 0x7f55ac073fa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.836+0000 7f55b3e00640 1 -- 172.21.15.16:0/1577425835 >> 172.21.15.16:0/1577425835 conn(0x7f55ac06d1e0 msgr2=0x7f55ac0b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:50.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.836+0000 7f55b3e00640 1 -- 172.21.15.16:0/1577425835 shutdown_connections 2024-09-10T21:12:50.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.836+0000 7f55b3e00640 1 -- 172.21.15.16:0/1577425835 wait complete. 2024-09-10T21:12:50.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.836+0000 7f55b3e00640 1 Processor -- start 2024-09-10T21:12:50.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.836+0000 7f55b3e00640 1 -- start start 2024-09-10T21:12:50.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55ac073bc0 0x7f55ac153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:50.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b3e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ac0b2b10 0x7f55ac139230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:50.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ac0cda80 0x7f55ac153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:50.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b3e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac167040 con 0x7f55ac0cda80 2024-09-10T21:12:50.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b3e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac15b190 con 0x7f55ac073bc0 2024-09-10T21:12:50.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b3e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f55ac1671c0 con 0x7f55ac0b2b10 2024-09-10T21:12:50.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ac0cda80 0x7f55ac153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:50.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ac0cda80 0x7f55ac153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47930/0 (socket says 172.21.15.16:47930) 2024-09-10T21:12:50.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b1e00640 1 -- 172.21.15.16:0/1584609080 learned_addr learned my addr 172.21.15.16:0/1584609080 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:50.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b1400640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55ac073bc0 0x7f55ac153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:50.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b0a00640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ac0b2b10 0x7f55ac139230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:50.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b1e00640 1 -- 172.21.15.16:0/1584609080 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ac0b2b10 msgr2=0x7f55ac139230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:50.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b1e00640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ac0b2b10 0x7f55ac139230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b1e00640 1 -- 172.21.15.16:0/1584609080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55ac073bc0 msgr2=0x7f55ac153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:50.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b1e00640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55ac073bc0 0x7f55ac153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:50.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b1e00640 1 -- 172.21.15.16:0/1584609080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ac1f9850 con 0x7f55ac0cda80 2024-09-10T21:12:50.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b1e00640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ac0cda80 0x7f55ac153f20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f55a400c7a0 tx=0x7f55a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:50.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55a9600640 1 -- 172.21.15.16:0/1584609080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4003b80 con 0x7f55ac0cda80 2024-09-10T21:12:50.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55a9600640 1 -- 172.21.15.16:0/1584609080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55a4003d40 con 0x7f55ac0cda80 2024-09-10T21:12:50.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b3e00640 1 -- 172.21.15.16:0/1584609080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ac1638e0 con 0x7f55ac0cda80 2024-09-10T21:12:50.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55a9600640 1 -- 172.21.15.16:0/1584609080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a400aab0 con 0x7f55ac0cda80 2024-09-10T21:12:50.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b3e00640 1 -- 172.21.15.16:0/1584609080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ac163ea0 con 0x7f55ac0cda80 2024-09-10T21:12:50.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.840+0000 7f55b3e00640 1 -- 172.21.15.16:0/1584609080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55ac15e500 con 0x7f55ac0cda80 2024-09-10T21:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.844+0000 7f55a9600640 1 -- 172.21.15.16:0/1584609080 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f55a400ac50 con 0x7f55ac0cda80 2024-09-10T21:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.844+0000 7f55a9600640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f558c0725e0 0x7f558c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.844+0000 7f55a9600640 1 -- 172.21.15.16:0/1584609080 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7f55a4092bd0 con 0x7f55ac0cda80 2024-09-10T21:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.844+0000 7f55b1400640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f558c0725e0 0x7f558c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.844+0000 7f55b1400640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f558c0725e0 0x7f558c074aa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55a00af000 tx=0x7f55a001afd0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:50.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:50.848+0000 7f55a9600640 1 -- 172.21.15.16:0/1584609080 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f55a405dfb0 con 0x7f55ac0cda80 2024-09-10T21:12:51.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.024+0000 7f55b3e00640 1 -- 172.21.15.16:0/1584609080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f55ac15b9b0 con 0x7f55ac0cda80 2024-09-10T21:12:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.888+0000 7f55a9600640 1 -- 172.21.15.16:0/1584609080 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v54) ==== 138+0+0 (secure 0 0 0) 0x7f55a40629e0 con 0x7f55ac0cda80 2024-09-10T21:12:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-10T21:12:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.888+0000 7f55b3e00640 1 -- 172.21.15.16:0/1584609080 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f558c0725e0 msgr2=0x7f558c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.888+0000 7f55b3e00640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f558c0725e0 0x7f558c074aa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55a00af000 tx=0x7f55a001afd0 comp rx=0 tx=0).stop 2024-09-10T21:12:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.888+0000 7f55b3e00640 1 -- 172.21.15.16:0/1584609080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ac0cda80 msgr2=0x7f55ac153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.888+0000 7f55b3e00640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ac0cda80 0x7f55ac153f20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f55a400c7a0 tx=0x7f55a400cc60 comp rx=0 tx=0).stop 2024-09-10T21:12:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.888+0000 7f55b3e00640 1 -- 172.21.15.16:0/1584609080 shutdown_connections 2024-09-10T21:12:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.888+0000 7f55b3e00640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f558c0725e0 0x7f558c074aa0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.888+0000 7f55b3e00640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ac0cda80 0x7f55ac153f20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.888+0000 7f55b3e00640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ac0b2b10 0x7f55ac139230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.888+0000 7f55b3e00640 1 --2- 172.21.15.16:0/1584609080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55ac073bc0 0x7f55ac153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.888+0000 7f55b3e00640 1 -- 172.21.15.16:0/1584609080 >> 172.21.15.16:0/1584609080 conn(0x7f55ac06d1e0 msgr2=0x7f55ac073fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.888+0000 7f55b3e00640 1 -- 172.21.15.16:0/1584609080 shutdown_connections 2024-09-10T21:12:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:51.888+0000 7f55b3e00640 1 -- 172.21.15.16:0/1584609080 wait complete. 2024-09-10T21:12:51.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:12:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 -- 172.21.15.16:0/3029015611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80cfc10 msgr2=0x7fd4f80ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 --2- 172.21.15.16:0/3029015611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80cfc10 0x7fd4f80ceac0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009900 tx=0x7fd4e001c810 comp rx=0 tx=0).stop 2024-09-10T21:12:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 -- 172.21.15.16:0/3029015611 shutdown_connections 2024-09-10T21:12:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 --2- 172.21.15.16:0/3029015611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80cfc10 0x7fd4f80ceac0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 --2- 172.21.15.16:0/3029015611 >> v2:172.21.15.52:3300/0 conn(0x7fd4f815be70 0x7fd4f80ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 -- 172.21.15.16:0/3029015611 >> 172.21.15.16:0/3029015611 conn(0x7fd4f80d1b70 msgr2=0x7fd4f80d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 -- 172.21.15.16:0/3029015611 shutdown_connections 2024-09-10T21:12:52.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 -- 172.21.15.16:0/3029015611 wait complete. 2024-09-10T21:12:52.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 Processor -- start 2024-09-10T21:12:52.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 -- start start 2024-09-10T21:12:52.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80cf000 0x7fd4f81f9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:52.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4f80cfc10 0x7fd4f815d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:52.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd4f815be70 0x7fd4f809f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:52.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8168df0 con 0x7fd4f80cf000 2024-09-10T21:12:52.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8168c70 con 0x7fd4f80cfc10 2024-09-10T21:12:52.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f8168f70 con 0x7fd4f815be70 2024-09-10T21:12:52.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80cf000 0x7fd4f81f9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:52.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80cf000 0x7fd4f81f9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44906/0 (socket says 172.21.15.16:44906) 2024-09-10T21:12:52.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f6a00640 1 -- 172.21.15.16:0/2994028536 learned_addr learned my addr 172.21.15.16:0/2994028536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:52.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4ee000640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4f80cfc10 0x7fd4f815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:52.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.076+0000 7fd4f7400640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd4f815be70 0x7fd4f809f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:52.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.080+0000 7fd4f6a00640 1 -- 172.21.15.16:0/2994028536 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd4f815be70 msgr2=0x7fd4f809f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:52.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.080+0000 7fd4f6a00640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd4f815be70 0x7fd4f809f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:52.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.080+0000 7fd4f6a00640 1 -- 172.21.15.16:0/2994028536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4f80cfc10 msgr2=0x7fd4f815d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:52.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.080+0000 7fd4f6a00640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4f80cfc10 0x7fd4f815d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:52.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.080+0000 7fd4f6a00640 1 -- 172.21.15.16:0/2994028536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f815dac0 con 0x7fd4f80cf000 2024-09-10T21:12:52.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.080+0000 7fd4f6a00640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80cf000 0x7fd4f81f9910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c9f0 tx=0x7fd4e800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:52.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.080+0000 7fd4ef400640 1 -- 172.21.15.16:0/2994028536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8007b50 con 0x7fd4f80cf000 2024-09-10T21:12:52.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.080+0000 7fd4ef400640 1 -- 172.21.15.16:0/2994028536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4e8007de0 con 0x7fd4f80cf000 2024-09-10T21:12:52.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.080+0000 7fd4ef400640 1 -- 172.21.15.16:0/2994028536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800f6b0 con 0x7fd4f80cf000 2024-09-10T21:12:52.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.080+0000 7fd4f7e00640 1 -- 172.21.15.16:0/2994028536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f815dd40 con 0x7fd4f80cf000 2024-09-10T21:12:52.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.080+0000 7fd4f7e00640 1 -- 172.21.15.16:0/2994028536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f815e300 con 0x7fd4f80cf000 2024-09-10T21:12:52.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.080+0000 7fd4f7e00640 1 -- 172.21.15.16:0/2994028536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f80cc610 con 0x7fd4f80cf000 2024-09-10T21:12:52.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.080+0000 7fd4ef400640 1 -- 172.21.15.16:0/2994028536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd4e8020030 con 0x7fd4f80cf000 2024-09-10T21:12:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.084+0000 7fd4ef400640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd4d4072610 0x7fd4d4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.084+0000 7fd4ef400640 1 -- 172.21.15.16:0/2994028536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5227+0+0 (secure 0 0 0) 0x7fd4e8093750 con 0x7fd4f80cf000 2024-09-10T21:12:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.084+0000 7fd4ee000640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd4d4072610 0x7fd4d4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.084+0000 7fd4ee000640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd4d4072610 0x7fd4d4074ad0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00af000 tx=0x7fd4e001b050 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:52.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.084+0000 7fd4ef400640 1 -- 172.21.15.16:0/2994028536 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd4e8026080 con 0x7fd4f80cf000 2024-09-10T21:12:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.260+0000 7fd4f7e00640 1 -- 172.21.15.16:0/2994028536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd4f80ceb20 con 0x7fd4f80cf000 2024-09-10T21:12:52.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.892+0000 7fd4ef400640 1 -- 172.21.15.16:0/2994028536 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v55) ==== 170+0+0 (secure 0 0 0) 0x7fd4f815dac0 con 0x7fd4f80cf000 2024-09-10T21:12:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-10T21:12:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.892+0000 7fd4f7e00640 1 -- 172.21.15.16:0/2994028536 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd4d4072610 msgr2=0x7fd4d4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.892+0000 7fd4f7e00640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd4d4072610 0x7fd4d4074ad0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00af000 tx=0x7fd4e001b050 comp rx=0 tx=0).stop 2024-09-10T21:12:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.892+0000 7fd4f7e00640 1 -- 172.21.15.16:0/2994028536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80cf000 msgr2=0x7fd4f81f9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.892+0000 7fd4f7e00640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80cf000 0x7fd4f81f9910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c9f0 tx=0x7fd4e800ceb0 comp rx=0 tx=0).stop 2024-09-10T21:12:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.892+0000 7fd4f7e00640 1 -- 172.21.15.16:0/2994028536 shutdown_connections 2024-09-10T21:12:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.892+0000 7fd4f7e00640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd4d4072610 0x7fd4d4074ad0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.892+0000 7fd4f7e00640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd4f815be70 0x7fd4f809f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.892+0000 7fd4f7e00640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4f80cfc10 0x7fd4f815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.892+0000 7fd4f7e00640 1 --2- 172.21.15.16:0/2994028536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80cf000 0x7fd4f81f9910 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.892+0000 7fd4f7e00640 1 -- 172.21.15.16:0/2994028536 >> 172.21.15.16:0/2994028536 conn(0x7fd4f80d1b70 msgr2=0x7fd4f8071c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.896+0000 7fd4f7e00640 1 -- 172.21.15.16:0/2994028536 shutdown_connections 2024-09-10T21:12:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:52.896+0000 7fd4f7e00640 1 -- 172.21.15.16:0/2994028536 wait complete. 2024-09-10T21:12:52.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:12:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.080+0000 7f4377000640 1 -- 172.21.15.16:0/3385202586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700ce040 msgr2=0x7f43700cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.080+0000 7f4377000640 1 --2- 172.21.15.16:0/3385202586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700ce040 0x7f43700cf010 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4370073b50 tx=0x7f436801f580 comp rx=0 tx=0).stop 2024-09-10T21:12:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.080+0000 7f4377000640 1 -- 172.21.15.16:0/3385202586 shutdown_connections 2024-09-10T21:12:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.080+0000 7f4377000640 1 --2- 172.21.15.16:0/3385202586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700ce040 0x7f43700cf010 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.080+0000 7f4377000640 1 --2- 172.21.15.16:0/3385202586 >> v2:172.21.15.52:3300/0 conn(0x7f43700d0210 0x7f43700cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.080+0000 7f4377000640 1 --2- 172.21.15.16:0/3385202586 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f43700cfe30 0x7f43700ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.080+0000 7f4377000640 1 -- 172.21.15.16:0/3385202586 >> 172.21.15.16:0/3385202586 conn(0x7f43700d1df0 msgr2=0x7f43700d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.080+0000 7f4377000640 1 -- 172.21.15.16:0/3385202586 shutdown_connections 2024-09-10T21:12:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.080+0000 7f4377000640 1 -- 172.21.15.16:0/3385202586 wait complete. 2024-09-10T21:12:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.080+0000 7f4377000640 1 Processor -- start 2024-09-10T21:12:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.080+0000 7f4377000640 1 -- start start 2024-09-10T21:12:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f4377000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700ce040 0x7f437015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f4377000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f43700cfe30 0x7f43700c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f4377000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43700d0210 0x7f43701594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f4377000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43700d2ad0 con 0x7f43700ce040 2024-09-10T21:12:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f4377000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43700b14f0 con 0x7f43700d0210 2024-09-10T21:12:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f4377000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f437015fee0 con 0x7f43700cfe30 2024-09-10T21:12:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f436fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700ce040 0x7f437015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f436fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700ce040 0x7f437015b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44928/0 (socket says 172.21.15.16:44928) 2024-09-10T21:12:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f436fe00640 1 -- 172.21.15.16:0/2483259351 learned_addr learned my addr 172.21.15.16:0/2483259351 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f4375000640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43700d0210 0x7f43701594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f4367400640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f43700cfe30 0x7f43700c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f436fe00640 1 -- 172.21.15.16:0/2483259351 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f43700cfe30 msgr2=0x7f43700c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f436fe00640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f43700cfe30 0x7f43700c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f436fe00640 1 -- 172.21.15.16:0/2483259351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43700d0210 msgr2=0x7f43701594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f436fe00640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43700d0210 0x7f43701594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:53.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f436fe00640 1 -- 172.21.15.16:0/2483259351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437006dd90 con 0x7f43700ce040 2024-09-10T21:12:53.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f436fe00640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700ce040 0x7f437015b0b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f436000d770 tx=0x7f436000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:53.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f436cc00640 1 -- 172.21.15.16:0/2483259351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360017070 con 0x7f43700ce040 2024-09-10T21:12:53.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f436cc00640 1 -- 172.21.15.16:0/2483259351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4360009cc0 con 0x7f43700ce040 2024-09-10T21:12:53.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f4377000640 1 -- 172.21.15.16:0/2483259351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437006e030 con 0x7f43700ce040 2024-09-10T21:12:53.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f436cc00640 1 -- 172.21.15.16:0/2483259351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43600106c0 con 0x7f43700ce040 2024-09-10T21:12:53.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f4377000640 1 -- 172.21.15.16:0/2483259351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437006e5f0 con 0x7f43700ce040 2024-09-10T21:12:53.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.084+0000 7f4377000640 1 -- 172.21.15.16:0/2483259351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4370160380 con 0x7f43700ce040 2024-09-10T21:12:53.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.088+0000 7f436cc00640 1 -- 172.21.15.16:0/2483259351 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4360010860 con 0x7f43700ce040 2024-09-10T21:12:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.088+0000 7f436cc00640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f434c0725e0 0x7f434c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.088+0000 7f436cc00640 1 -- 172.21.15.16:0/2483259351 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5609+0+0 (secure 0 0 0) 0x7f43600939a0 con 0x7f43700ce040 2024-09-10T21:12:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.088+0000 7f4367400640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f434c0725e0 0x7f434c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.088+0000 7f4367400640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f434c0725e0 0x7f434c074aa0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f43700b17f0 tx=0x7f435800b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.092+0000 7f436cc00640 1 -- 172.21.15.16:0/2483259351 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f436005eb00 con 0x7f43700ce040 2024-09-10T21:12:53.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.300+0000 7f4377000640 1 -- 172.21.15.16:0/2483259351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f43700b0910 con 0x7f43700ce040 2024-09-10T21:12:53.440 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:12:53.443 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:12:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.916+0000 7f436cc00640 1 -- 172.21.15.16:0/2483259351 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v29) ==== 172+0+0 (secure 0 0 0) 0x7f4360063530 con 0x7f43700ce040 2024-09-10T21:12:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 8 and data pool 9 2024-09-10T21:12:53.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.924+0000 7f4377000640 1 -- 172.21.15.16:0/2483259351 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f434c0725e0 msgr2=0x7f434c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:53.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.924+0000 7f4377000640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f434c0725e0 0x7f434c074aa0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f43700b17f0 tx=0x7f435800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:12:53.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.924+0000 7f4377000640 1 -- 172.21.15.16:0/2483259351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700ce040 msgr2=0x7f437015b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:53.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.924+0000 7f4377000640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700ce040 0x7f437015b0b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f436000d770 tx=0x7f436000dc30 comp rx=0 tx=0).stop 2024-09-10T21:12:53.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.924+0000 7f4377000640 1 -- 172.21.15.16:0/2483259351 shutdown_connections 2024-09-10T21:12:53.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.924+0000 7f4377000640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f434c0725e0 0x7f434c074aa0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:53.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.924+0000 7f4377000640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43700d0210 0x7f43701594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:53.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.924+0000 7f4377000640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f43700cfe30 0x7f43700c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.924+0000 7f4377000640 1 --2- 172.21.15.16:0/2483259351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700ce040 0x7f437015b0b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.924+0000 7f4377000640 1 -- 172.21.15.16:0/2483259351 >> 172.21.15.16:0/2483259351 conn(0x7f43700d1df0 msgr2=0x7f4370070960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.924+0000 7f4377000640 1 -- 172.21.15.16:0/2483259351 shutdown_connections 2024-09-10T21:12:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:53.924+0000 7f4377000640 1 -- 172.21.15.16:0/2483259351 wait complete. 2024-09-10T21:12:53.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:12:54.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.104+0000 7ff007000640 1 -- 172.21.15.16:0/1947488819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0000726b0 msgr2=0x7ff000072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:54.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.104+0000 7ff007000640 1 --2- 172.21.15.16:0/1947488819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0000726b0 0x7ff000072b40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff00006daf0 tx=0x7feff801caf0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 -- 172.21.15.16:0/1947488819 shutdown_connections 2024-09-10T21:12:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 --2- 172.21.15.16:0/1947488819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0000726b0 0x7ff000072b40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 --2- 172.21.15.16:0/1947488819 >> v2:172.21.15.52:3300/0 conn(0x7ff0001599d0 0x7ff00015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 -- 172.21.15.16:0/1947488819 >> 172.21.15.16:0/1947488819 conn(0x7ff0000cfb20 msgr2=0x7ff0000cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 -- 172.21.15.16:0/1947488819 shutdown_connections 2024-09-10T21:12:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 -- 172.21.15.16:0/1947488819 wait complete. 2024-09-10T21:12:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 Processor -- start 2024-09-10T21:12:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 -- start start 2024-09-10T21:12:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00006e640 0x7ff000146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff0000726b0 0x7ff0000cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0001599d0 0x7ff00015e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff000168eb0 con 0x7ff00006e640 2024-09-10T21:12:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff000168d30 con 0x7ff0001599d0 2024-09-10T21:12:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff000169030 con 0x7ff0000726b0 2024-09-10T21:12:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7fefffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00006e640 0x7ff000146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7fefffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00006e640 0x7ff000146eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44950/0 (socket says 172.21.15.16:44950) 2024-09-10T21:12:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7fefffe00640 1 -- 172.21.15.16:0/86992642 learned_addr learned my addr 172.21.15.16:0/86992642 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff005000640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0001599d0 0x7ff00015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7feff7e00640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff0000726b0 0x7ff0000cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7fefffe00640 1 -- 172.21.15.16:0/86992642 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff0000726b0 msgr2=0x7ff0000cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7fefffe00640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff0000726b0 0x7ff0000cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7fefffe00640 1 -- 172.21.15.16:0/86992642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0001599d0 msgr2=0x7ff00015e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7fefffe00640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0001599d0 0x7ff00015e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7fefffe00640 1 -- 172.21.15.16:0/86992642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0000c7d70 con 0x7ff00006e640 2024-09-10T21:12:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7fefffe00640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00006e640 0x7ff000146eb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feff000b410 tx=0x7feff000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7feffcc00640 1 -- 172.21.15.16:0/86992642 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0015020 con 0x7ff00006e640 2024-09-10T21:12:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7feffcc00640 1 -- 172.21.15.16:0/86992642 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feff0011920 con 0x7ff00006e640 2024-09-10T21:12:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7feffcc00640 1 -- 172.21.15.16:0/86992642 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0011d70 con 0x7ff00006e640 2024-09-10T21:12:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.108+0000 7ff007000640 1 -- 172.21.15.16:0/86992642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0000c8050 con 0x7ff00006e640 2024-09-10T21:12:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.112+0000 7ff007000640 1 -- 172.21.15.16:0/86992642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0000c8610 con 0x7ff00006e640 2024-09-10T21:12:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.112+0000 7ff007000640 1 -- 172.21.15.16:0/86992642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0000c8810 con 0x7ff00006e640 2024-09-10T21:12:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.112+0000 7feffcc00640 1 -- 172.21.15.16:0/86992642 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feff001a400 con 0x7ff00006e640 2024-09-10T21:12:54.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.112+0000 7feffcc00640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fefdc0725e0 0x7fefdc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:54.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.112+0000 7feffcc00640 1 -- 172.21.15.16:0/86992642 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7feff00934c0 con 0x7ff00006e640 2024-09-10T21:12:54.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.112+0000 7feff7e00640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fefdc0725e0 0x7fefdc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:54.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.116+0000 7feff7e00640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fefdc0725e0 0x7fefdc074aa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff0000c3580 tx=0x7fefe8009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.116+0000 7feffcc00640 1 -- 172.21.15.16:0/86992642 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff0000c8810 con 0x7ff00006e640 2024-09-10T21:12:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.292+0000 7ff007000640 1 -- 172.21.15.16:0/86992642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff000072b40 con 0x7ff00006e640 2024-09-10T21:12:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.296+0000 7feffcc00640 1 -- 172.21.15.16:0/86992642 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16690 (secure 0 0 0) 0x7feff001f020 con 0x7ff00006e640 2024-09-10T21:12:54.302 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:54.302 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":56,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:53.305031+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:12:51.039811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T21:12:52.273850+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:12:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.296+0000 7ff007000640 1 -- 172.21.15.16:0/86992642 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fefdc0725e0 msgr2=0x7fefdc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.296+0000 7ff007000640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fefdc0725e0 0x7fefdc074aa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff0000c3580 tx=0x7fefe8009040 comp rx=0 tx=0).stop 2024-09-10T21:12:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.296+0000 7ff007000640 1 -- 172.21.15.16:0/86992642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00006e640 msgr2=0x7ff000146eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.296+0000 7ff007000640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00006e640 0x7ff000146eb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feff000b410 tx=0x7feff000b8d0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.296+0000 7ff007000640 1 -- 172.21.15.16:0/86992642 shutdown_connections 2024-09-10T21:12:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.296+0000 7ff007000640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fefdc0725e0 0x7fefdc074aa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.296+0000 7ff007000640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0001599d0 0x7ff00015e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.296+0000 7ff007000640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff0000726b0 0x7ff0000cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.296+0000 7ff007000640 1 --2- 172.21.15.16:0/86992642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00006e640 0x7ff000146eb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.296+0000 7ff007000640 1 -- 172.21.15.16:0/86992642 >> 172.21.15.16:0/86992642 conn(0x7ff0000cfb20 msgr2=0x7ff0000743a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.296+0000 7ff007000640 1 -- 172.21.15.16:0/86992642 shutdown_connections 2024-09-10T21:12:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.296+0000 7ff007000640 1 -- 172.21.15.16:0/86992642 wait complete. 2024-09-10T21:12:54.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:12:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.464+0000 7fb6fa400640 1 -- 172.21.15.16:0/4037690683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f40b3200 msgr2=0x7fb6f40d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.464+0000 7fb6fa400640 1 --2- 172.21.15.16:0/4037690683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f40b3200 0x7fb6f40d01c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40b13a0 tx=0x7fb6dc01f0b0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.464+0000 7fb6fa400640 1 -- 172.21.15.16:0/4037690683 shutdown_connections 2024-09-10T21:12:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.464+0000 7fb6fa400640 1 --2- 172.21.15.16:0/4037690683 >> v2:172.21.15.52:3300/0 conn(0x7fb6f40d0890 0x7fb6f40b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.464+0000 7fb6fa400640 1 --2- 172.21.15.16:0/4037690683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f40b3200 0x7fb6f40d01c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.464+0000 7fb6fa400640 1 --2- 172.21.15.16:0/4037690683 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6f415f450 0x7fb6f40cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.464+0000 7fb6fa400640 1 -- 172.21.15.16:0/4037690683 >> 172.21.15.16:0/4037690683 conn(0x7fb6f40d1d80 msgr2=0x7fb6f40d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.464+0000 7fb6fa400640 1 -- 172.21.15.16:0/4037690683 shutdown_connections 2024-09-10T21:12:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.464+0000 7fb6fa400640 1 -- 172.21.15.16:0/4037690683 wait complete. 2024-09-10T21:12:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.464+0000 7fb6fa400640 1 Processor -- start 2024-09-10T21:12:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.464+0000 7fb6fa400640 1 -- start start 2024-09-10T21:12:54.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.464+0000 7fb6fa400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6f40d0890 0x7fb6f40c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:54.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.464+0000 7fb6fa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f415f450 0x7fb6f40cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:54.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6fa400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6f41f5820 0x7fb6f415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:54.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6fa400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f40b1950 con 0x7fb6f415f450 2024-09-10T21:12:54.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6fa400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f40b2ec0 con 0x7fb6f40d0890 2024-09-10T21:12:54.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6fa400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f40b1ad0 con 0x7fb6f41f5820 2024-09-10T21:12:54.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6eaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f415f450 0x7fb6f40cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:54.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6eaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f415f450 0x7fb6f40cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44958/0 (socket says 172.21.15.16:44958) 2024-09-10T21:12:54.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6eaa00640 1 -- 172.21.15.16:0/2973910317 learned_addr learned my addr 172.21.15.16:0/2973910317 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6f3400640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6f40d0890 0x7fb6f40c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6f3e00640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6f41f5820 0x7fb6f415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6eaa00640 1 -- 172.21.15.16:0/2973910317 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6f41f5820 msgr2=0x7fb6f415b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6eaa00640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6f41f5820 0x7fb6f415b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6eaa00640 1 -- 172.21.15.16:0/2973910317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6f40d0890 msgr2=0x7fb6f40c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6eaa00640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6f40d0890 0x7fb6f40c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6eaa00640 1 -- 172.21.15.16:0/2973910317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f40ce6e0 con 0x7fb6f415f450 2024-09-10T21:12:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6eaa00640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f415f450 0x7fb6f40cde00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0b3040 tx=0x7fb6dc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6ebe00640 1 -- 172.21.15.16:0/2973910317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0b5040 con 0x7fb6f415f450 2024-09-10T21:12:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6ebe00640 1 -- 172.21.15.16:0/2973910317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6dc005140 con 0x7fb6f415f450 2024-09-10T21:12:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6ebe00640 1 -- 172.21.15.16:0/2973910317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc005590 con 0x7fb6f415f450 2024-09-10T21:12:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6fa400640 1 -- 172.21.15.16:0/2973910317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f40c8ff0 con 0x7fb6f415f450 2024-09-10T21:12:54.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6fa400640 1 -- 172.21.15.16:0/2973910317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f40c95b0 con 0x7fb6f415f450 2024-09-10T21:12:54.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6fa400640 1 -- 172.21.15.16:0/2973910317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f40b2090 con 0x7fb6f415f450 2024-09-10T21:12:54.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.468+0000 7fb6ebe00640 1 -- 172.21.15.16:0/2973910317 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb6dc0046d0 con 0x7fb6f415f450 2024-09-10T21:12:54.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.472+0000 7fb6ebe00640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6d00725e0 0x7fb6d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:54.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.472+0000 7fb6ebe00640 1 -- 172.21.15.16:0/2973910317 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fb6dc130740 con 0x7fb6f415f450 2024-09-10T21:12:54.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.472+0000 7fb6f3400640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6d00725e0 0x7fb6d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:54.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.472+0000 7fb6f3400640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6d00725e0 0x7fb6d0074aa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40c46e0 tx=0x7fb6e40091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:54.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.476+0000 7fb6ebe00640 1 -- 172.21.15.16:0/2973910317 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb6dc018c10 con 0x7fb6f415f450 2024-09-10T21:12:54.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.648+0000 7fb6fa400640 1 -- 172.21.15.16:0/2973910317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6f415eaf0 con 0x7fb6f415f450 2024-09-10T21:12:54.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.648+0000 7fb6ebe00640 1 -- 172.21.15.16:0/2973910317 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16690 (secure 0 0 0) 0x7fb6dc0fbd90 con 0x7fb6f415f450 2024-09-10T21:12:54.657 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:54.657 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":56,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:53.305031+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:12:51.039811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T21:12:52.273850+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:12:54.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.652+0000 7fb6fa400640 1 -- 172.21.15.16:0/2973910317 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6d00725e0 msgr2=0x7fb6d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:54.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.652+0000 7fb6fa400640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6d00725e0 0x7fb6d0074aa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40c46e0 tx=0x7fb6e40091e0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.652+0000 7fb6fa400640 1 -- 172.21.15.16:0/2973910317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f415f450 msgr2=0x7fb6f40cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:54.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.652+0000 7fb6fa400640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f415f450 0x7fb6f40cde00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0b3040 tx=0x7fb6dc0045e0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.652+0000 7fb6fa400640 1 -- 172.21.15.16:0/2973910317 shutdown_connections 2024-09-10T21:12:54.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.652+0000 7fb6fa400640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6d00725e0 0x7fb6d0074aa0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.652+0000 7fb6fa400640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6f41f5820 0x7fb6f415b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.652+0000 7fb6fa400640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f415f450 0x7fb6f40cde00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.652+0000 7fb6fa400640 1 --2- 172.21.15.16:0/2973910317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6f40d0890 0x7fb6f40c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.652+0000 7fb6fa400640 1 -- 172.21.15.16:0/2973910317 >> 172.21.15.16:0/2973910317 conn(0x7fb6f40d1d80 msgr2=0x7fb6f4070b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.652+0000 7fb6fa400640 1 -- 172.21.15.16:0/2973910317 shutdown_connections 2024-09-10T21:12:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.652+0000 7fb6fa400640 1 -- 172.21.15.16:0/2973910317 wait complete. 2024-09-10T21:12:54.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:12:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.820+0000 7fb783400640 1 -- 172.21.15.16:0/1624225084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb77c06f0e0 msgr2=0x7fb77c06f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.820+0000 7fb783400640 1 --2- 172.21.15.16:0/1624225084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb77c06f0e0 0x7fb77c06f550 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb764009840 tx=0x7fb76401c740 comp rx=0 tx=0).stop 2024-09-10T21:12:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.820+0000 7fb783400640 1 -- 172.21.15.16:0/1624225084 shutdown_connections 2024-09-10T21:12:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.820+0000 7fb783400640 1 --2- 172.21.15.16:0/1624225084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb77c06f0e0 0x7fb77c06f550 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.820+0000 7fb783400640 1 -- 172.21.15.16:0/1624225084 >> 172.21.15.16:0/1624225084 conn(0x7fb77c0b16a0 msgr2=0x7fb77c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.820+0000 7fb783400640 1 -- 172.21.15.16:0/1624225084 shutdown_connections 2024-09-10T21:12:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.820+0000 7fb783400640 1 -- 172.21.15.16:0/1624225084 wait complete. 2024-09-10T21:12:54.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb783400640 1 Processor -- start 2024-09-10T21:12:54.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb783400640 1 -- start start 2024-09-10T21:12:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb783400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb77c06f0e0 0x7fb77c1399a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb783400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb77c0cbb00 0x7fb77c13d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb783400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb77c0cbee0 0x7fb77c13c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb783400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c169e20 con 0x7fb77c06f0e0 2024-09-10T21:12:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb783400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c169ca0 con 0x7fb77c0cbb00 2024-09-10T21:12:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb783400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb77c169fa0 con 0x7fb77c0cbee0 2024-09-10T21:12:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb780a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb77c06f0e0 0x7fb77c1399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb780a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb77c06f0e0 0x7fb77c1399a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44962/0 (socket says 172.21.15.16:44962) 2024-09-10T21:12:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb780a00640 1 -- 172.21.15.16:0/2389348316 learned_addr learned my addr 172.21.15.16:0/2389348316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb781400640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb77c0cbee0 0x7fb77c13c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb773e00640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb77c0cbb00 0x7fb77c13d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb780a00640 1 -- 172.21.15.16:0/2389348316 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb77c0cbee0 msgr2=0x7fb77c13c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb780a00640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb77c0cbee0 0x7fb77c13c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb780a00640 1 -- 172.21.15.16:0/2389348316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb77c0cbb00 msgr2=0x7fb77c13d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb780a00640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb77c0cbb00 0x7fb77c13d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb780a00640 1 -- 172.21.15.16:0/2389348316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c0b23d0 con 0x7fb77c06f0e0 2024-09-10T21:12:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb780a00640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb77c06f0e0 0x7fb77c1399a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb76c001230 tx=0x7fb76c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:54.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb779600640 1 -- 172.21.15.16:0/2389348316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c016020 con 0x7fb77c06f0e0 2024-09-10T21:12:54.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb779600640 1 -- 172.21.15.16:0/2389348316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb76c00a930 con 0x7fb77c06f0e0 2024-09-10T21:12:54.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb779600640 1 -- 172.21.15.16:0/2389348316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c00ad80 con 0x7fb77c06f0e0 2024-09-10T21:12:54.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb783400640 1 -- 172.21.15.16:0/2389348316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77c0b3190 con 0x7fb77c06f0e0 2024-09-10T21:12:54.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.824+0000 7fb783400640 1 -- 172.21.15.16:0/2389348316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77c0b3450 con 0x7fb77c06f0e0 2024-09-10T21:12:54.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.828+0000 7fb783400640 1 -- 172.21.15.16:0/2389348316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb77c06f550 con 0x7fb77c06f0e0 2024-09-10T21:12:54.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.828+0000 7fb779600640 1 -- 172.21.15.16:0/2389348316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb76c01b400 con 0x7fb77c06f0e0 2024-09-10T21:12:54.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.828+0000 7fb779600640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7580725e0 0x7fb758074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:54.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.828+0000 7fb779600640 1 -- 172.21.15.16:0/2389348316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fb76c093770 con 0x7fb77c06f0e0 2024-09-10T21:12:54.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.828+0000 7fb773e00640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7580725e0 0x7fb758074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:54.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.828+0000 7fb773e00640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7580725e0 0x7fb758074aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb7640af000 tx=0x7fb76401af80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:54.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:54.832+0000 7fb779600640 1 -- 172.21.15.16:0/2389348316 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb76c020020 con 0x7fb77c06f0e0 2024-09-10T21:12:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.052+0000 7fb783400640 1 -- 172.21.15.16:0/2389348316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb77c15c3a0 con 0x7fb77c06f0e0 2024-09-10T21:12:55.895 INFO:tasks.ceph.mds.d.smithi052.stderr:2024-09-10T21:12:55.890+0000 7f9026800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:12:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.928+0000 7fb779600640 1 -- 172.21.15.16:0/2389348316 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) ==== 118+0+0 (secure 0 0 0) 0x7fb76c05edb0 con 0x7fb77c06f0e0 2024-09-10T21:12:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.932+0000 7fb783400640 1 -- 172.21.15.16:0/2389348316 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7580725e0 msgr2=0x7fb758074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.932+0000 7fb783400640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7580725e0 0x7fb758074aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb7640af000 tx=0x7fb76401af80 comp rx=0 tx=0).stop 2024-09-10T21:12:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.932+0000 7fb783400640 1 -- 172.21.15.16:0/2389348316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb77c06f0e0 msgr2=0x7fb77c1399a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.932+0000 7fb783400640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb77c06f0e0 0x7fb77c1399a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb76c001230 tx=0x7fb76c00cbb0 comp rx=0 tx=0).stop 2024-09-10T21:12:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.932+0000 7fb783400640 1 -- 172.21.15.16:0/2389348316 shutdown_connections 2024-09-10T21:12:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.932+0000 7fb783400640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7580725e0 0x7fb758074aa0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.932+0000 7fb783400640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb77c0cbee0 0x7fb77c13c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.932+0000 7fb783400640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb77c0cbb00 0x7fb77c13d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.932+0000 7fb783400640 1 --2- 172.21.15.16:0/2389348316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb77c06f0e0 0x7fb77c1399a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.932+0000 7fb783400640 1 -- 172.21.15.16:0/2389348316 >> 172.21.15.16:0/2389348316 conn(0x7fb77c0b16a0 msgr2=0x7fb77c0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.932+0000 7fb783400640 1 -- 172.21.15.16:0/2389348316 shutdown_connections 2024-09-10T21:12:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:55.932+0000 7fb783400640 1 -- 172.21.15.16:0/2389348316 wait complete. 2024-09-10T21:12:55.964 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:12:55.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:12:56.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.112+0000 7f266c800640 1 -- 172.21.15.16:0/2674009827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f266406f7e0 msgr2=0x7f266406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:56.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.112+0000 7f266c800640 1 --2- 172.21.15.16:0/2674009827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f266406f7e0 0x7f266406fbc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2658009870 tx=0x7f265801c700 comp rx=0 tx=0).stop 2024-09-10T21:12:56.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.112+0000 7f266c800640 1 -- 172.21.15.16:0/2674009827 shutdown_connections 2024-09-10T21:12:56.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.112+0000 7f266c800640 1 --2- 172.21.15.16:0/2674009827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f266406f7e0 0x7f266406fbc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:56.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.112+0000 7f266c800640 1 -- 172.21.15.16:0/2674009827 >> 172.21.15.16:0/2674009827 conn(0x7f26640cd6a0 msgr2=0x7f26640cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:56.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.112+0000 7f266c800640 1 -- 172.21.15.16:0/2674009827 shutdown_connections 2024-09-10T21:12:56.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.112+0000 7f266c800640 1 -- 172.21.15.16:0/2674009827 wait complete. 2024-09-10T21:12:56.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f266c800640 1 Processor -- start 2024-09-10T21:12:56.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f266c800640 1 -- start start 2024-09-10T21:12:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f266c800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f266406f7e0 0x7f26640c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f266c800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26640747d0 0x7f26641fc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f266c800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26640cc710 0x7f2664167050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f266c800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f266416a6c0 con 0x7f26640747d0 2024-09-10T21:12:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f266c800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f266416a540 con 0x7f26640cc710 2024-09-10T21:12:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f266c800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f266416a840 con 0x7f266406f7e0 2024-09-10T21:12:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f2669400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26640747d0 0x7f26641fc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f2669400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26640747d0 0x7f26641fc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44976/0 (socket says 172.21.15.16:44976) 2024-09-10T21:12:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f2669400640 1 -- 172.21.15.16:0/2945938298 learned_addr learned my addr 172.21.15.16:0/2945938298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f2669e00640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f266406f7e0 0x7f26640c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f266a800640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26640cc710 0x7f2664167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:56.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f2669400640 1 -- 172.21.15.16:0/2945938298 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f266406f7e0 msgr2=0x7f26640c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:56.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f2669400640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f266406f7e0 0x7f26640c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:56.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f2669400640 1 -- 172.21.15.16:0/2945938298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26640cc710 msgr2=0x7f2664167050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:56.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f2669400640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26640cc710 0x7f2664167050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:56.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f2669400640 1 -- 172.21.15.16:0/2945938298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26640b66f0 con 0x7f26640747d0 2024-09-10T21:12:56.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f2669400640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26640747d0 0x7f26641fc500 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f265400c500 tx=0x7f265400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:56.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f265e000640 1 -- 172.21.15.16:0/2945938298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654017070 con 0x7f26640747d0 2024-09-10T21:12:56.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f265e000640 1 -- 172.21.15.16:0/2945938298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2654003600 con 0x7f26640747d0 2024-09-10T21:12:56.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f265e000640 1 -- 172.21.15.16:0/2945938298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654003a90 con 0x7f26640747d0 2024-09-10T21:12:56.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f266c800640 1 -- 172.21.15.16:0/2945938298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26640b3cc0 con 0x7f26640747d0 2024-09-10T21:12:56.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.116+0000 7f266c800640 1 -- 172.21.15.16:0/2945938298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26641f8d30 con 0x7f26640747d0 2024-09-10T21:12:56.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.120+0000 7f266c800640 1 -- 172.21.15.16:0/2945938298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26640b3e90 con 0x7f26640747d0 2024-09-10T21:12:56.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.120+0000 7f265e000640 1 -- 172.21.15.16:0/2945938298 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2654003c30 con 0x7f26640747d0 2024-09-10T21:12:56.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.120+0000 7f265e000640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f26480725e0 0x7f2648074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:56.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.120+0000 7f265e000640 1 -- 172.21.15.16:0/2945938298 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f26540929a0 con 0x7f26640747d0 2024-09-10T21:12:56.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.120+0000 7f2669e00640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f26480725e0 0x7f2648074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:56.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.120+0000 7f2669e00640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f26480725e0 0x7f2648074aa0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f26580af000 tx=0x7f265801af40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:56.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.124+0000 7f265e000640 1 -- 172.21.15.16:0/2945938298 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f265405dae0 con 0x7f26640747d0 2024-09-10T21:12:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.332+0000 7f266c800640 1 -- 172.21.15.16:0/2945938298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f266406fbc0 con 0x7f26640747d0 2024-09-10T21:12:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.932+0000 7f265e000640 1 -- 172.21.15.16:0/2945938298 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) ==== 122+0+0 (secure 0 0 0) 0x7f2654062510 con 0x7f26640747d0 2024-09-10T21:12:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.936+0000 7f266c800640 1 -- 172.21.15.16:0/2945938298 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f26480725e0 msgr2=0x7f2648074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.936+0000 7f266c800640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f26480725e0 0x7f2648074aa0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f26580af000 tx=0x7f265801af40 comp rx=0 tx=0).stop 2024-09-10T21:12:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.936+0000 7f266c800640 1 -- 172.21.15.16:0/2945938298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26640747d0 msgr2=0x7f26641fc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.936+0000 7f266c800640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26640747d0 0x7f26641fc500 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f265400c500 tx=0x7f265400c9c0 comp rx=0 tx=0).stop 2024-09-10T21:12:56.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.936+0000 7f266c800640 1 -- 172.21.15.16:0/2945938298 shutdown_connections 2024-09-10T21:12:56.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.936+0000 7f266c800640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f26480725e0 0x7f2648074aa0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:56.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.936+0000 7f266c800640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26640cc710 0x7f2664167050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:56.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.936+0000 7f266c800640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26640747d0 0x7f26641fc500 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:56.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.936+0000 7f266c800640 1 --2- 172.21.15.16:0/2945938298 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f266406f7e0 0x7f26640c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:56.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.936+0000 7f266c800640 1 -- 172.21.15.16:0/2945938298 >> 172.21.15.16:0/2945938298 conn(0x7f26640cd6a0 msgr2=0x7f266414c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:56.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.936+0000 7f266c800640 1 -- 172.21.15.16:0/2945938298 shutdown_connections 2024-09-10T21:12:56.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:56.936+0000 7f266c800640 1 -- 172.21.15.16:0/2945938298 wait complete. 2024-09-10T21:12:56.981 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:12:56.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:12:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.124+0000 7f9281400640 1 -- 172.21.15.16:0/1702440966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927c0d2820 msgr2=0x7f927c15fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.124+0000 7f9281400640 1 --2- 172.21.15.16:0/1702440966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927c0d2820 0x7f927c15fbe0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f927c0730f0 tx=0x7f926801c830 comp rx=0 tx=0).stop 2024-09-10T21:12:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.124+0000 7f9281400640 1 -- 172.21.15.16:0/1702440966 shutdown_connections 2024-09-10T21:12:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.124+0000 7f9281400640 1 --2- 172.21.15.16:0/1702440966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927c0d2820 0x7f927c15fbe0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.124+0000 7f9281400640 1 --2- 172.21.15.16:0/1702440966 >> v2:172.21.15.52:3300/0 conn(0x7f927c0cfd40 0x7f927c0d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.124+0000 7f9281400640 1 -- 172.21.15.16:0/1702440966 >> 172.21.15.16:0/1702440966 conn(0x7f927c0d1d00 msgr2=0x7f927c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.124+0000 7f9281400640 1 -- 172.21.15.16:0/1702440966 shutdown_connections 2024-09-10T21:12:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.124+0000 7f9281400640 1 -- 172.21.15.16:0/1702440966 wait complete. 2024-09-10T21:12:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.124+0000 7f9281400640 1 Processor -- start 2024-09-10T21:12:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.124+0000 7f9281400640 1 -- start start 2024-09-10T21:12:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f9281400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927c0cfd40 0x7f927c15b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f9281400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f927c0d2820 0x7f927c158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f9281400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f927c160120 0x7f927c0ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f9281400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f927c169240 con 0x7f927c0cfd40 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f9281400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1690c0 con 0x7f927c0d2820 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f9281400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f927c1693c0 con 0x7f927c160120 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f927be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927c0cfd40 0x7f927c15b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f927be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927c0cfd40 0x7f927c15b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44996/0 (socket says 172.21.15.16:44996) 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f927be00640 1 -- 172.21.15.16:0/2838149074 learned_addr learned my addr 172.21.15.16:0/2838149074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f927b400640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f927c0d2820 0x7f927c158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f9280a00640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f927c160120 0x7f927c0ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f927be00640 1 -- 172.21.15.16:0/2838149074 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f927c160120 msgr2=0x7f927c0ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f927be00640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f927c160120 0x7f927c0ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f927be00640 1 -- 172.21.15.16:0/2838149074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f927c0d2820 msgr2=0x7f927c158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f927be00640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f927c0d2820 0x7f927c158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f927be00640 1 -- 172.21.15.16:0/2838149074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c15b640 con 0x7f927c0cfd40 2024-09-10T21:12:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f927be00640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927c0cfd40 0x7f927c15b930 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f926c00d980 tx=0x7f926c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f9273e00640 1 -- 172.21.15.16:0/2838149074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c017070 con 0x7f927c0cfd40 2024-09-10T21:12:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f9273e00640 1 -- 172.21.15.16:0/2838149074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f926c009d20 con 0x7f927c0cfd40 2024-09-10T21:12:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f9281400640 1 -- 172.21.15.16:0/2838149074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927c2002f0 con 0x7f927c0cfd40 2024-09-10T21:12:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f9273e00640 1 -- 172.21.15.16:0/2838149074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c0106c0 con 0x7f927c0cfd40 2024-09-10T21:12:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f9281400640 1 -- 172.21.15.16:0/2838149074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927c15b360 con 0x7f927c0cfd40 2024-09-10T21:12:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.128+0000 7f9281400640 1 -- 172.21.15.16:0/2838149074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f927c0d0120 con 0x7f927c0cfd40 2024-09-10T21:12:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.132+0000 7f9273e00640 1 -- 172.21.15.16:0/2838149074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f926c010860 con 0x7f927c0cfd40 2024-09-10T21:12:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.132+0000 7f9273e00640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f925c0725e0 0x7f925c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.132+0000 7f9273e00640 1 -- 172.21.15.16:0/2838149074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f926c093680 con 0x7f927c0cfd40 2024-09-10T21:12:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.132+0000 7f927b400640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f925c0725e0 0x7f925c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.132+0000 7f927b400640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f925c0725e0 0x7f925c074aa0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f92680af000 tx=0x7f926801b070 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.136+0000 7f9273e00640 1 -- 172.21.15.16:0/2838149074 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f926c05ed50 con 0x7f927c0cfd40 2024-09-10T21:12:57.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.344+0000 7f9281400640 1 -- 172.21.15.16:0/2838149074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f927c0c8220 con 0x7f927c0cfd40 2024-09-10T21:12:57.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.344+0000 7f9273e00640 1 -- 172.21.15.16:0/2838149074 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5220 (secure 0 0 0) 0x7f926c05e650 con 0x7f927c0cfd40 2024-09-10T21:12:57.351 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"btime":"2024-09-10T21:12:56:932852+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/653119388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":653119388},{"type":"v1","addr":"172.21.15.16:6836","nonce":653119388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2947833730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2947833730},{"type":"v1","addr":"172.21.15.16:6837","nonce":2947833730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2865311562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2865311562},{"type":"v1","addr":"172.21.15.52:6834","nonce":2865311562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:12:53.304996+0000","modified":"2024-09-10T21:12:56.337640+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5592":{"gid":5592,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/2357796669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2357796669},{"type":"v1","addr":"172.21.15.52:6835","nonce":2357796669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5592,"qdb_cluster":[5592]},"id":4}]} 2024-09-10T21:12:57.352 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 33 2024-09-10T21:12:57.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.348+0000 7f9281400640 1 -- 172.21.15.16:0/2838149074 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f925c0725e0 msgr2=0x7f925c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:57.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.348+0000 7f9281400640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f925c0725e0 0x7f925c074aa0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f92680af000 tx=0x7f926801b070 comp rx=0 tx=0).stop 2024-09-10T21:12:57.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.348+0000 7f9281400640 1 -- 172.21.15.16:0/2838149074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927c0cfd40 msgr2=0x7f927c15b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:57.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.348+0000 7f9281400640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927c0cfd40 0x7f927c15b930 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f926c00d980 tx=0x7f926c00de40 comp rx=0 tx=0).stop 2024-09-10T21:12:57.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.348+0000 7f9281400640 1 -- 172.21.15.16:0/2838149074 shutdown_connections 2024-09-10T21:12:57.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.348+0000 7f9281400640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f925c0725e0 0x7f925c074aa0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.348+0000 7f9281400640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f927c160120 0x7f927c0ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.348+0000 7f9281400640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f927c0d2820 0x7f927c158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.348+0000 7f9281400640 1 --2- 172.21.15.16:0/2838149074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927c0cfd40 0x7f927c15b930 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.348+0000 7f9281400640 1 -- 172.21.15.16:0/2838149074 >> 172.21.15.16:0/2838149074 conn(0x7f927c0d1d00 msgr2=0x7f927c15e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:57.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.348+0000 7f9281400640 1 -- 172.21.15.16:0/2838149074 shutdown_connections 2024-09-10T21:12:57.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.348+0000 7f9281400640 1 -- 172.21.15.16:0/2838149074 wait complete. 2024-09-10T21:12:57.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:12:57.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.512+0000 7f9f26000640 1 -- 172.21.15.16:0/2397059737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f200cc9c0 msgr2=0x7f9f2015eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:57.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.512+0000 7f9f26000640 1 --2- 172.21.15.16:0/2397059737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f200cc9c0 0x7f9f2015eaa0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9f20075400 tx=0x7f9f0801c770 comp rx=0 tx=0).stop 2024-09-10T21:12:57.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.512+0000 7f9f26000640 1 -- 172.21.15.16:0/2397059737 shutdown_connections 2024-09-10T21:12:57.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.512+0000 7f9f26000640 1 --2- 172.21.15.16:0/2397059737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f200cc9c0 0x7f9f2015eaa0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.512+0000 7f9f26000640 1 --2- 172.21.15.16:0/2397059737 >> v2:172.21.15.52:3300/0 conn(0x7f9f2015e420 0x7f9f200cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.512+0000 7f9f26000640 1 -- 172.21.15.16:0/2397059737 >> 172.21.15.16:0/2397059737 conn(0x7f9f200d03a0 msgr2=0x7f9f200d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:57.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.512+0000 7f9f26000640 1 -- 172.21.15.16:0/2397059737 shutdown_connections 2024-09-10T21:12:57.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.512+0000 7f9f26000640 1 -- 172.21.15.16:0/2397059737 wait complete. 2024-09-10T21:12:57.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f26000640 1 Processor -- start 2024-09-10T21:12:57.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f26000640 1 -- start start 2024-09-10T21:12:57.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f26000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9f20068c10 0x7f9f200c3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:57.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f26000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f200cc9c0 0x7f9f2015d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:57.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f26000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2015e420 0x7f9f20156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:57.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f26000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20167de0 con 0x7f9f2015e420 2024-09-10T21:12:57.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f26000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9f200b3fe0 con 0x7f9f200cc9c0 2024-09-10T21:12:57.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f26000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9f20167f60 con 0x7f9f20068c10 2024-09-10T21:12:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f1fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2015e420 0x7f9f20156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f1fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2015e420 0x7f9f20156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45016/0 (socket says 172.21.15.16:45016) 2024-09-10T21:12:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f1fe00640 1 -- 172.21.15.16:0/1829192287 learned_addr learned my addr 172.21.15.16:0/1829192287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f1f400640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9f20068c10 0x7f9f200c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f1fe00640 1 -- 172.21.15.16:0/1829192287 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9f20068c10 msgr2=0x7f9f200c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f17e00640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f200cc9c0 0x7f9f2015d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f1fe00640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9f20068c10 0x7f9f200c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f1fe00640 1 -- 172.21.15.16:0/1829192287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f200cc9c0 msgr2=0x7f9f2015d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f1fe00640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f200cc9c0 0x7f9f2015d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f1fe00640 1 -- 172.21.15.16:0/1829192287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f20157930 con 0x7f9f2015e420 2024-09-10T21:12:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f1f400640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9f20068c10 0x7f9f200c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f17e00640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f200cc9c0 0x7f9f2015d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f1fe00640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2015e420 0x7f9f20156960 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800cbc0 tx=0x7f9f1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f17400640 1 -- 172.21.15.16:0/1829192287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18016020 con 0x7f9f2015e420 2024-09-10T21:12:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f17400640 1 -- 172.21.15.16:0/1829192287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f1800ab40 con 0x7f9f2015e420 2024-09-10T21:12:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f17400640 1 -- 172.21.15.16:0/1829192287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18012420 con 0x7f9f2015e420 2024-09-10T21:12:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f26000640 1 -- 172.21.15.16:0/1829192287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f201554f0 con 0x7f9f2015e420 2024-09-10T21:12:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.516+0000 7f9f26000640 1 -- 172.21.15.16:0/1829192287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f20165930 con 0x7f9f2015e420 2024-09-10T21:12:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.520+0000 7f9f26000640 1 -- 172.21.15.16:0/1829192287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f200cc480 con 0x7f9f2015e420 2024-09-10T21:12:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.520+0000 7f9f17400640 1 -- 172.21.15.16:0/1829192287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9f1801b400 con 0x7f9f2015e420 2024-09-10T21:12:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.520+0000 7f9f17400640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9efc0725e0 0x7f9efc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.520+0000 7f9f17400640 1 -- 172.21.15.16:0/1829192287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f9f18093870 con 0x7f9f2015e420 2024-09-10T21:12:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.520+0000 7f9f1f400640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9efc0725e0 0x7f9efc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.520+0000 7f9f1f400640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9efc0725e0 0x7f9efc074aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f10005fd0 tx=0x7f9f10005dc0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.524+0000 7f9f17400640 1 -- 172.21.15.16:0/1829192287 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9f18020020 con 0x7f9f2015e420 2024-09-10T21:12:57.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.700+0000 7f9f26000640 1 -- 172.21.15.16:0/1829192287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f20072a30 con 0x7f9f2015e420 2024-09-10T21:12:57.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.704+0000 7f9f17400640 1 -- 172.21.15.16:0/1829192287 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16677 (secure 0 0 0) 0x7f9f1805eeb0 con 0x7f9f2015e420 2024-09-10T21:12:57.709 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:57.710 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:54.920251+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:12:51.039811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T21:12:52.273850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:12:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.704+0000 7f9f26000640 1 -- 172.21.15.16:0/1829192287 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9efc0725e0 msgr2=0x7f9efc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.704+0000 7f9f26000640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9efc0725e0 0x7f9efc074aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f10005fd0 tx=0x7f9f10005dc0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.704+0000 7f9f26000640 1 -- 172.21.15.16:0/1829192287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2015e420 msgr2=0x7f9f20156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.704+0000 7f9f26000640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2015e420 0x7f9f20156960 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800cbc0 tx=0x7f9f1800ccb0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.704+0000 7f9f26000640 1 -- 172.21.15.16:0/1829192287 shutdown_connections 2024-09-10T21:12:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.704+0000 7f9f26000640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9efc0725e0 0x7f9efc074aa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.704+0000 7f9f26000640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2015e420 0x7f9f20156960 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.704+0000 7f9f26000640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f200cc9c0 0x7f9f2015d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.704+0000 7f9f26000640 1 --2- 172.21.15.16:0/1829192287 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9f20068c10 0x7f9f200c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.704+0000 7f9f26000640 1 -- 172.21.15.16:0/1829192287 >> 172.21.15.16:0/1829192287 conn(0x7f9f200d03a0 msgr2=0x7f9f200b4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.704+0000 7f9f26000640 1 -- 172.21.15.16:0/1829192287 shutdown_connections 2024-09-10T21:12:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.704+0000 7f9f26000640 1 -- 172.21.15.16:0/1829192287 wait complete. 2024-09-10T21:12:57.732 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:12:57.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:12:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.872+0000 7fb7f8c00640 1 -- 172.21.15.16:0/592626826 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7f40716f0 msgr2=0x7fb7f40af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.872+0000 7fb7f8c00640 1 --2- 172.21.15.16:0/592626826 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7f40716f0 0x7fb7f40af580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009870 tx=0x7fb7dc01c280 comp rx=0 tx=0).stop 2024-09-10T21:12:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.872+0000 7fb7f8c00640 1 -- 172.21.15.16:0/592626826 shutdown_connections 2024-09-10T21:12:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.872+0000 7fb7f8c00640 1 --2- 172.21.15.16:0/592626826 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7f40716f0 0x7fb7f40af580 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.872+0000 7fb7f8c00640 1 -- 172.21.15.16:0/592626826 >> 172.21.15.16:0/592626826 conn(0x7fb7f409b130 msgr2=0x7fb7f40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.872+0000 7fb7f8c00640 1 -- 172.21.15.16:0/592626826 shutdown_connections 2024-09-10T21:12:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.872+0000 7fb7f8c00640 1 -- 172.21.15.16:0/592626826 wait complete. 2024-09-10T21:12:57.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.872+0000 7fb7f8c00640 1 Processor -- start 2024-09-10T21:12:57.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.872+0000 7fb7f8c00640 1 -- start start 2024-09-10T21:12:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7f8c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7f40716f0 0x7fb7f40b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7f8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f40b13c0 0x7fb7f40b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7f8c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7f415f400 0x7fb7f40b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7f8c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f416aec0 con 0x7fb7f40b13c0 2024-09-10T21:12:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7f8c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f416ad40 con 0x7fb7f415f400 2024-09-10T21:12:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7f8c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f416b040 con 0x7fb7f40716f0 2024-09-10T21:12:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7ebe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f40b13c0 0x7fb7f40b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7ebe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f40b13c0 0x7fb7f40b55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45026/0 (socket says 172.21.15.16:45026) 2024-09-10T21:12:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7ebe00640 1 -- 172.21.15.16:0/526341111 learned_addr learned my addr 172.21.15.16:0/526341111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7f1e00640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7f40716f0 0x7fb7f40b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7f2800640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7f415f400 0x7fb7f40b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7ebe00640 1 -- 172.21.15.16:0/526341111 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7f40716f0 msgr2=0x7fb7f40b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7ebe00640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7f40716f0 0x7fb7f40b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7ebe00640 1 -- 172.21.15.16:0/526341111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7f415f400 msgr2=0x7fb7f40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7ebe00640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7f415f400 0x7fb7f40b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7ebe00640 1 -- 172.21.15.16:0/526341111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f4156ab0 con 0x7fb7f40b13c0 2024-09-10T21:12:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7ebe00640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f40b13c0 0x7fb7f40b55a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc01cb20 tx=0x7fb7dc005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7ea000640 1 -- 172.21.15.16:0/526341111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc0b1040 con 0x7fb7f40b13c0 2024-09-10T21:12:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7ea000640 1 -- 172.21.15.16:0/526341111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7dc0a4ab0 con 0x7fb7f40b13c0 2024-09-10T21:12:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7ea000640 1 -- 172.21.15.16:0/526341111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc003600 con 0x7fb7f40b13c0 2024-09-10T21:12:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7f8c00640 1 -- 172.21.15.16:0/526341111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f40b4f40 con 0x7fb7f40b13c0 2024-09-10T21:12:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7f8c00640 1 -- 172.21.15.16:0/526341111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f4156590 con 0x7fb7f40b13c0 2024-09-10T21:12:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7f8c00640 1 -- 172.21.15.16:0/526341111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f4070fe0 con 0x7fb7f40b13c0 2024-09-10T21:12:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.876+0000 7fb7ea000640 1 -- 172.21.15.16:0/526341111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb7dc0b4070 con 0x7fb7f40b13c0 2024-09-10T21:12:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.880+0000 7fb7ea000640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7d00725e0 0x7fb7d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.880+0000 7fb7ea000640 1 -- 172.21.15.16:0/526341111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fb7dc12bd90 con 0x7fb7f40b13c0 2024-09-10T21:12:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.880+0000 7fb7f1e00640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7d00725e0 0x7fb7d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.880+0000 7fb7f1e00640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7d00725e0 0x7fb7d0074aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4007fd0 tx=0x7fb7e4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:57.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:57.884+0000 7fb7ea000640 1 -- 172.21.15.16:0/526341111 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb7dc0c3020 con 0x7fb7f40b13c0 2024-09-10T21:12:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.056+0000 7fb7f8c00640 1 -- 172.21.15.16:0/526341111 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb7f415b250 con 0x7fb7d00725e0 2024-09-10T21:12:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.060+0000 7fb7ea000640 1 -- 172.21.15.16:0/526341111 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250794 (secure 0 0 0) 0x7fb7f415b250 con 0x7fb7d00725e0 2024-09-10T21:12:58.071 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:12:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.064+0000 7fb7f8c00640 1 -- 172.21.15.16:0/526341111 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7d00725e0 msgr2=0x7fb7d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.064+0000 7fb7f8c00640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7d00725e0 0x7fb7d0074aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4007fd0 tx=0x7fb7e4009040 comp rx=0 tx=0).stop 2024-09-10T21:12:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.064+0000 7fb7f8c00640 1 -- 172.21.15.16:0/526341111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f40b13c0 msgr2=0x7fb7f40b55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.064+0000 7fb7f8c00640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f40b13c0 0x7fb7f40b55a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc01cb20 tx=0x7fb7dc005ca0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.064+0000 7fb7f8c00640 1 -- 172.21.15.16:0/526341111 shutdown_connections 2024-09-10T21:12:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.064+0000 7fb7f8c00640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7d00725e0 0x7fb7d0074aa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.064+0000 7fb7f8c00640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7f415f400 0x7fb7f40b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.064+0000 7fb7f8c00640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7f40b13c0 0x7fb7f40b55a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.064+0000 7fb7f8c00640 1 --2- 172.21.15.16:0/526341111 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7f40716f0 0x7fb7f40b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.064+0000 7fb7f8c00640 1 -- 172.21.15.16:0/526341111 >> 172.21.15.16:0/526341111 conn(0x7fb7f409b130 msgr2=0x7fb7f40b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.064+0000 7fb7f8c00640 1 -- 172.21.15.16:0/526341111 shutdown_connections 2024-09-10T21:12:58.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.064+0000 7fb7f8c00640 1 -- 172.21.15.16:0/526341111 wait complete. 2024-09-10T21:12:58.092 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-10T21:12:57.489868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255812,"kb_used_data":6276,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718908,"statfs":{"total":773094113280,"available":772832161792,"internally_reserved":0,"allocated":6426624,"data_stored":3321845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.597272"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.962730+0000","last_change":"2024-09-10T21:12:53.959848+0000","last_active":"2024-09-10T21:12:53.962730+0000","last_peered":"2024-09-10T21:12:53.962730+0000","last_clean":"2024-09-10T21:12:53.962730+0000","last_became_active":"2024-09-10T21:12:53.959238+0000","last_became_peered":"2024-09-10T21:12:53.959238+0000","last_unstale":"2024-09-10T21:12:53.962730+0000","last_undegraded":"2024-09-10T21:12:53.962730+0000","last_fullsized":"2024-09-10T21:12:53.962730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:27.189161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.908750+0000","last_change":"2024-09-10T21:12:52.908750+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.908750+0000","last_undegraded":"2024-09-10T21:12:52.908750+0000","last_fullsized":"2024-09-10T21:12:52.908750+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.931243+0000","last_change":"2024-09-10T21:12:53.929639+0000","last_active":"2024-09-10T21:12:53.931243+0000","last_peered":"2024-09-10T21:12:53.931243+0000","last_clean":"2024-09-10T21:12:53.931243+0000","last_became_active":"2024-09-10T21:12:53.929333+0000","last_became_peered":"2024-09-10T21:12:53.929333+0000","last_unstale":"2024-09-10T21:12:53.931243+0000","last_undegraded":"2024-09-10T21:12:53.931243+0000","last_fullsized":"2024-09-10T21:12:53.931243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:17.409325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.936510+0000","last_change":"2024-09-10T21:12:53.932951+0000","last_active":"2024-09-10T21:12:53.936510+0000","last_peered":"2024-09-10T21:12:53.936510+0000","last_clean":"2024-09-10T21:12:53.936510+0000","last_became_active":"2024-09-10T21:12:53.930361+0000","last_became_peered":"2024-09-10T21:12:53.930361+0000","last_unstale":"2024-09-10T21:12:53.936510+0000","last_undegraded":"2024-09-10T21:12:53.936510+0000","last_fullsized":"2024-09-10T21:12:53.936510+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:01.888505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.957550+0000","last_change":"2024-09-10T21:12:53.951504+0000","last_active":"2024-09-10T21:12:53.957550+0000","last_peered":"2024-09-10T21:12:53.957550+0000","last_clean":"2024-09-10T21:12:53.957550+0000","last_became_active":"2024-09-10T21:12:53.950545+0000","last_became_peered":"2024-09-10T21:12:53.950545+0000","last_unstale":"2024-09-10T21:12:53.957550+0000","last_undegraded":"2024-09-10T21:12:53.957550+0000","last_fullsized":"2024-09-10T21:12:53.957550+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:55.157666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958761+0000","last_change":"2024-09-10T21:12:53.949235+0000","last_active":"2024-09-10T21:12:53.958761+0000","last_peered":"2024-09-10T21:12:53.958761+0000","last_clean":"2024-09-10T21:12:53.958761+0000","last_became_active":"2024-09-10T21:12:53.936152+0000","last_became_peered":"2024-09-10T21:12:53.936152+0000","last_unstale":"2024-09-10T21:12:53.958761+0000","last_undegraded":"2024-09-10T21:12:53.958761+0000","last_fullsized":"2024-09-10T21:12:53.958761+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:01.430435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.934571+0000","last_change":"2024-09-10T21:12:53.929021+0000","last_active":"2024-09-10T21:12:53.934571+0000","last_peered":"2024-09-10T21:12:53.934571+0000","last_clean":"2024-09-10T21:12:53.934571+0000","last_became_active":"2024-09-10T21:12:53.928713+0000","last_became_peered":"2024-09-10T21:12:53.928713+0000","last_unstale":"2024-09-10T21:12:53.934571+0000","last_undegraded":"2024-09-10T21:12:53.934571+0000","last_fullsized":"2024-09-10T21:12:53.934571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:57.528446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.945096+0000","last_change":"2024-09-10T21:12:53.934558+0000","last_active":"2024-09-10T21:12:53.945096+0000","last_peered":"2024-09-10T21:12:53.945096+0000","last_clean":"2024-09-10T21:12:53.945096+0000","last_became_active":"2024-09-10T21:12:53.934226+0000","last_became_peered":"2024-09-10T21:12:53.934226+0000","last_unstale":"2024-09-10T21:12:53.945096+0000","last_undegraded":"2024-09-10T21:12:53.945096+0000","last_fullsized":"2024-09-10T21:12:53.945096+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:27.657368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.905254+0000","last_change":"2024-09-10T21:12:52.905254+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.905254+0000","last_undegraded":"2024-09-10T21:12:52.905254+0000","last_fullsized":"2024-09-10T21:12:52.905254+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958633+0000","last_change":"2024-09-10T21:12:53.952820+0000","last_active":"2024-09-10T21:12:53.958633+0000","last_peered":"2024-09-10T21:12:53.958633+0000","last_clean":"2024-09-10T21:12:53.958633+0000","last_became_active":"2024-09-10T21:12:53.952356+0000","last_became_peered":"2024-09-10T21:12:53.952356+0000","last_unstale":"2024-09-10T21:12:53.958633+0000","last_undegraded":"2024-09-10T21:12:53.958633+0000","last_fullsized":"2024-09-10T21:12:53.958633+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.570424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.960202+0000","last_change":"2024-09-10T21:12:53.957907+0000","last_active":"2024-09-10T21:12:53.960202+0000","last_peered":"2024-09-10T21:12:53.960202+0000","last_clean":"2024-09-10T21:12:53.960202+0000","last_became_active":"2024-09-10T21:12:53.957595+0000","last_became_peered":"2024-09-10T21:12:53.957595+0000","last_unstale":"2024-09-10T21:12:53.960202+0000","last_undegraded":"2024-09-10T21:12:53.960202+0000","last_fullsized":"2024-09-10T21:12:53.960202+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:02.523420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.905908+0000","last_change":"2024-09-10T21:12:52.905908+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.905908+0000","last_undegraded":"2024-09-10T21:12:52.905908+0000","last_fullsized":"2024-09-10T21:12:52.905908+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.928804+0000","last_change":"2024-09-10T21:12:53.922722+0000","last_active":"2024-09-10T21:12:53.928804+0000","last_peered":"2024-09-10T21:12:53.928804+0000","last_clean":"2024-09-10T21:12:53.928804+0000","last_became_active":"2024-09-10T21:12:53.922457+0000","last_became_peered":"2024-09-10T21:12:53.922457+0000","last_unstale":"2024-09-10T21:12:53.928804+0000","last_undegraded":"2024-09-10T21:12:53.928804+0000","last_fullsized":"2024-09-10T21:12:53.928804+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:06.040633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951748+0000","last_change":"2024-09-10T21:12:53.945632+0000","last_active":"2024-09-10T21:12:53.951748+0000","last_peered":"2024-09-10T21:12:53.951748+0000","last_clean":"2024-09-10T21:12:53.951748+0000","last_became_active":"2024-09-10T21:12:53.945340+0000","last_became_peered":"2024-09-10T21:12:53.945340+0000","last_unstale":"2024-09-10T21:12:53.951748+0000","last_undegraded":"2024-09-10T21:12:53.951748+0000","last_fullsized":"2024-09-10T21:12:53.951748+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:31.129061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.937358+0000","last_change":"2024-09-10T21:12:53.934328+0000","last_active":"2024-09-10T21:12:53.937358+0000","last_peered":"2024-09-10T21:12:53.937358+0000","last_clean":"2024-09-10T21:12:53.937358+0000","last_became_active":"2024-09-10T21:12:53.934057+0000","last_became_peered":"2024-09-10T21:12:53.934057+0000","last_unstale":"2024-09-10T21:12:53.937358+0000","last_undegraded":"2024-09-10T21:12:53.937358+0000","last_fullsized":"2024-09-10T21:12:53.937358+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:25.798580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.956730+0000","last_change":"2024-09-10T21:12:53.935764+0000","last_active":"2024-09-10T21:12:53.956730+0000","last_peered":"2024-09-10T21:12:53.956730+0000","last_clean":"2024-09-10T21:12:53.956730+0000","last_became_active":"2024-09-10T21:12:53.935468+0000","last_became_peered":"2024-09-10T21:12:53.935468+0000","last_unstale":"2024-09-10T21:12:53.956730+0000","last_undegraded":"2024-09-10T21:12:53.956730+0000","last_fullsized":"2024-09-10T21:12:53.956730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:35.882232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.959571+0000","last_change":"2024-09-10T21:12:53.953515+0000","last_active":"2024-09-10T21:12:53.959571+0000","last_peered":"2024-09-10T21:12:53.959571+0000","last_clean":"2024-09-10T21:12:53.959571+0000","last_became_active":"2024-09-10T21:12:53.952358+0000","last_became_peered":"2024-09-10T21:12:53.952358+0000","last_unstale":"2024-09-10T21:12:53.959571+0000","last_undegraded":"2024-09-10T21:12:53.959571+0000","last_fullsized":"2024-09-10T21:12:53.959571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:44.453397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929660+0000","last_change":"2024-09-10T21:12:53.923168+0000","last_active":"2024-09-10T21:12:53.929660+0000","last_peered":"2024-09-10T21:12:53.929660+0000","last_clean":"2024-09-10T21:12:53.929660+0000","last_became_active":"2024-09-10T21:12:53.922878+0000","last_became_peered":"2024-09-10T21:12:53.922878+0000","last_unstale":"2024-09-10T21:12:53.929660+0000","last_undegraded":"2024-09-10T21:12:53.929660+0000","last_fullsized":"2024-09-10T21:12:53.929660+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:20.410692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.917837+0000","last_change":"2024-09-10T21:12:52.917837+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.917837+0000","last_undegraded":"2024-09-10T21:12:52.917837+0000","last_fullsized":"2024-09-10T21:12:52.917837+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.961063+0000","last_change":"2024-09-10T21:12:53.959543+0000","last_active":"2024-09-10T21:12:53.961063+0000","last_peered":"2024-09-10T21:12:53.961063+0000","last_clean":"2024-09-10T21:12:53.961063+0000","last_became_active":"2024-09-10T21:12:53.959240+0000","last_became_peered":"2024-09-10T21:12:53.959240+0000","last_unstale":"2024-09-10T21:12:53.961063+0000","last_undegraded":"2024-09-10T21:12:53.961063+0000","last_fullsized":"2024-09-10T21:12:53.961063+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:19.453855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929367+0000","last_change":"2024-09-10T21:12:53.926236+0000","last_active":"2024-09-10T21:12:53.929367+0000","last_peered":"2024-09-10T21:12:53.929367+0000","last_clean":"2024-09-10T21:12:53.929367+0000","last_became_active":"2024-09-10T21:12:53.924195+0000","last_became_peered":"2024-09-10T21:12:53.924195+0000","last_unstale":"2024-09-10T21:12:53.929367+0000","last_undegraded":"2024-09-10T21:12:53.929367+0000","last_fullsized":"2024-09-10T21:12:53.929367+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.802123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.945190+0000","last_change":"2024-09-10T21:12:53.934083+0000","last_active":"2024-09-10T21:12:53.945190+0000","last_peered":"2024-09-10T21:12:53.945190+0000","last_clean":"2024-09-10T21:12:53.945190+0000","last_became_active":"2024-09-10T21:12:53.933188+0000","last_became_peered":"2024-09-10T21:12:53.933188+0000","last_unstale":"2024-09-10T21:12:53.945190+0000","last_undegraded":"2024-09-10T21:12:53.945190+0000","last_fullsized":"2024-09-10T21:12:53.945190+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:00.887008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.922965+0000","last_change":"2024-09-10T21:12:53.920537+0000","last_active":"2024-09-10T21:12:53.922965+0000","last_peered":"2024-09-10T21:12:53.922965+0000","last_clean":"2024-09-10T21:12:53.922965+0000","last_became_active":"2024-09-10T21:12:53.920252+0000","last_became_peered":"2024-09-10T21:12:53.920252+0000","last_unstale":"2024-09-10T21:12:53.922965+0000","last_undegraded":"2024-09-10T21:12:53.922965+0000","last_fullsized":"2024-09-10T21:12:53.922965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:51.239584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.905107+0000","last_change":"2024-09-10T21:12:52.905107+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.905107+0000","last_undegraded":"2024-09-10T21:12:52.905107+0000","last_fullsized":"2024-09-10T21:12:52.905107+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.909354+0000","last_change":"2024-09-10T21:12:52.909354+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.909354+0000","last_undegraded":"2024-09-10T21:12:52.909354+0000","last_fullsized":"2024-09-10T21:12:52.909354+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929245+0000","last_change":"2024-09-10T21:12:53.923199+0000","last_active":"2024-09-10T21:12:53.929245+0000","last_peered":"2024-09-10T21:12:53.929245+0000","last_clean":"2024-09-10T21:12:53.929245+0000","last_became_active":"2024-09-10T21:12:53.922779+0000","last_became_peered":"2024-09-10T21:12:53.922779+0000","last_unstale":"2024-09-10T21:12:53.929245+0000","last_undegraded":"2024-09-10T21:12:53.929245+0000","last_fullsized":"2024-09-10T21:12:53.929245+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:20.410692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.914654+0000","last_change":"2024-09-10T21:12:52.914654+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.914654+0000","last_undegraded":"2024-09-10T21:12:52.914654+0000","last_fullsized":"2024-09-10T21:12:52.914654+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951229+0000","last_change":"2024-09-10T21:12:53.939591+0000","last_active":"2024-09-10T21:12:53.951229+0000","last_peered":"2024-09-10T21:12:53.951229+0000","last_clean":"2024-09-10T21:12:53.951229+0000","last_became_active":"2024-09-10T21:12:53.939314+0000","last_became_peered":"2024-09-10T21:12:53.939314+0000","last_unstale":"2024-09-10T21:12:53.951229+0000","last_undegraded":"2024-09-10T21:12:53.951229+0000","last_fullsized":"2024-09-10T21:12:53.951229+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:02.044073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.946748+0000","last_change":"2024-09-10T21:12:53.937058+0000","last_active":"2024-09-10T21:12:53.946748+0000","last_peered":"2024-09-10T21:12:53.946748+0000","last_clean":"2024-09-10T21:12:53.946748+0000","last_became_active":"2024-09-10T21:12:53.936798+0000","last_became_peered":"2024-09-10T21:12:53.936798+0000","last_unstale":"2024-09-10T21:12:53.946748+0000","last_undegraded":"2024-09-10T21:12:53.946748+0000","last_fullsized":"2024-09-10T21:12:53.946748+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:53.754974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929094+0000","last_change":"2024-09-10T21:12:53.925270+0000","last_active":"2024-09-10T21:12:53.929094+0000","last_peered":"2024-09-10T21:12:53.929094+0000","last_clean":"2024-09-10T21:12:53.929094+0000","last_became_active":"2024-09-10T21:12:53.923493+0000","last_became_peered":"2024-09-10T21:12:53.923493+0000","last_unstale":"2024-09-10T21:12:53.929094+0000","last_undegraded":"2024-09-10T21:12:53.929094+0000","last_fullsized":"2024-09-10T21:12:53.929094+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:01.637614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.914057+0000","last_change":"2024-09-10T21:12:52.914057+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.914057+0000","last_undegraded":"2024-09-10T21:12:52.914057+0000","last_fullsized":"2024-09-10T21:12:52.914057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.945266+0000","last_change":"2024-09-10T21:12:53.939129+0000","last_active":"2024-09-10T21:12:53.945266+0000","last_peered":"2024-09-10T21:12:53.945266+0000","last_clean":"2024-09-10T21:12:53.945266+0000","last_became_active":"2024-09-10T21:12:53.938807+0000","last_became_peered":"2024-09-10T21:12:53.938807+0000","last_unstale":"2024-09-10T21:12:53.945266+0000","last_undegraded":"2024-09-10T21:12:53.945266+0000","last_fullsized":"2024-09-10T21:12:53.945266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:30.631397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.956016+0000","last_change":"2024-09-10T21:12:53.938689+0000","last_active":"2024-09-10T21:12:53.956016+0000","last_peered":"2024-09-10T21:12:53.956016+0000","last_clean":"2024-09-10T21:12:53.956016+0000","last_became_active":"2024-09-10T21:12:53.935691+0000","last_became_peered":"2024-09-10T21:12:53.935691+0000","last_unstale":"2024-09-10T21:12:53.956016+0000","last_undegraded":"2024-09-10T21:12:53.956016+0000","last_fullsized":"2024-09-10T21:12:53.956016+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:10.944201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":17,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.957007+0000","last_change":"2024-09-10T21:12:52.916629+0000","last_active":"2024-09-10T21:12:53.957007+0000","last_peered":"2024-09-10T21:12:53.957007+0000","last_clean":"2024-09-10T21:12:53.957007+0000","last_became_active":"2024-09-10T21:12:52.916091+0000","last_became_peered":"2024-09-10T21:12:52.916091+0000","last_unstale":"2024-09-10T21:12:53.957007+0000","last_undegraded":"2024-09-10T21:12:53.957007+0000","last_fullsized":"2024-09-10T21:12:53.957007+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:27.028802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.959594+0000","last_change":"2024-09-10T21:12:53.953827+0000","last_active":"2024-09-10T21:12:53.959594+0000","last_peered":"2024-09-10T21:12:53.959594+0000","last_clean":"2024-09-10T21:12:53.959594+0000","last_became_active":"2024-09-10T21:12:53.953262+0000","last_became_peered":"2024-09-10T21:12:53.953262+0000","last_unstale":"2024-09-10T21:12:53.959594+0000","last_undegraded":"2024-09-10T21:12:53.959594+0000","last_fullsized":"2024-09-10T21:12:53.959594+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:20.378633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.928971+0000","last_change":"2024-09-10T21:12:52.926961+0000","last_active":"2024-09-10T21:12:52.928971+0000","last_peered":"2024-09-10T21:12:52.928971+0000","last_clean":"2024-09-10T21:12:52.928971+0000","last_became_active":"2024-09-10T21:12:52.926723+0000","last_became_peered":"2024-09-10T21:12:52.926723+0000","last_unstale":"2024-09-10T21:12:52.928971+0000","last_undegraded":"2024-09-10T21:12:52.928971+0000","last_fullsized":"2024-09-10T21:12:52.928971+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:40.512719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951499+0000","last_change":"2024-09-10T21:12:53.944452+0000","last_active":"2024-09-10T21:12:53.951499+0000","last_peered":"2024-09-10T21:12:53.951499+0000","last_clean":"2024-09-10T21:12:53.951499+0000","last_became_active":"2024-09-10T21:12:53.944142+0000","last_became_peered":"2024-09-10T21:12:53.944142+0000","last_unstale":"2024-09-10T21:12:53.951499+0000","last_undegraded":"2024-09-10T21:12:53.951499+0000","last_fullsized":"2024-09-10T21:12:53.951499+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.507264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.963349+0000","last_change":"2024-09-10T21:12:52.924953+0000","last_active":"2024-09-10T21:12:53.963349+0000","last_peered":"2024-09-10T21:12:53.963349+0000","last_clean":"2024-09-10T21:12:53.963349+0000","last_became_active":"2024-09-10T21:12:52.924601+0000","last_became_peered":"2024-09-10T21:12:52.924601+0000","last_unstale":"2024-09-10T21:12:53.963349+0000","last_undegraded":"2024-09-10T21:12:53.963349+0000","last_fullsized":"2024-09-10T21:12:53.963349+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:44.842888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958253+0000","last_change":"2024-09-10T21:12:53.950957+0000","last_active":"2024-09-10T21:12:53.958253+0000","last_peered":"2024-09-10T21:12:53.958253+0000","last_clean":"2024-09-10T21:12:53.958253+0000","last_became_active":"2024-09-10T21:12:53.950673+0000","last_became_peered":"2024-09-10T21:12:53.950673+0000","last_unstale":"2024-09-10T21:12:53.958253+0000","last_undegraded":"2024-09-10T21:12:53.958253+0000","last_fullsized":"2024-09-10T21:12:53.958253+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:16.326966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.954185+0000","last_change":"2024-09-10T21:12:52.920769+0000","last_active":"2024-09-10T21:12:53.954185+0000","last_peered":"2024-09-10T21:12:53.954185+0000","last_clean":"2024-09-10T21:12:53.954185+0000","last_became_active":"2024-09-10T21:12:52.920460+0000","last_became_peered":"2024-09-10T21:12:52.920460+0000","last_unstale":"2024-09-10T21:12:53.954185+0000","last_undegraded":"2024-09-10T21:12:53.954185+0000","last_fullsized":"2024-09-10T21:12:53.954185+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:53.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951781+0000","last_change":"2024-09-10T21:12:53.946296+0000","last_active":"2024-09-10T21:12:53.951781+0000","last_peered":"2024-09-10T21:12:53.951781+0000","last_clean":"2024-09-10T21:12:53.951781+0000","last_became_active":"2024-09-10T21:12:53.945868+0000","last_became_peered":"2024-09-10T21:12:53.945868+0000","last_unstale":"2024-09-10T21:12:53.951781+0000","last_undegraded":"2024-09-10T21:12:53.951781+0000","last_fullsized":"2024-09-10T21:12:53.951781+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:56.605281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.955272+0000","last_change":"2024-09-10T21:12:52.917429+0000","last_active":"2024-09-10T21:12:53.955272+0000","last_peered":"2024-09-10T21:12:53.955272+0000","last_clean":"2024-09-10T21:12:53.955272+0000","last_became_active":"2024-09-10T21:12:52.917148+0000","last_became_peered":"2024-09-10T21:12:52.917148+0000","last_unstale":"2024-09-10T21:12:53.955272+0000","last_undegraded":"2024-09-10T21:12:53.955272+0000","last_fullsized":"2024-09-10T21:12:53.955272+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:05.034728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.950682+0000","last_change":"2024-09-10T21:12:53.947773+0000","last_active":"2024-09-10T21:12:53.950682+0000","last_peered":"2024-09-10T21:12:53.950682+0000","last_clean":"2024-09-10T21:12:53.950682+0000","last_became_active":"2024-09-10T21:12:53.947473+0000","last_became_peered":"2024-09-10T21:12:53.947473+0000","last_unstale":"2024-09-10T21:12:53.950682+0000","last_undegraded":"2024-09-10T21:12:53.950682+0000","last_fullsized":"2024-09-10T21:12:53.950682+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:45.835056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.961946+0000","last_change":"2024-09-10T21:12:52.907684+0000","last_active":"2024-09-10T21:12:53.961946+0000","last_peered":"2024-09-10T21:12:53.961946+0000","last_clean":"2024-09-10T21:12:53.961946+0000","last_became_active":"2024-09-10T21:12:52.907395+0000","last_became_peered":"2024-09-10T21:12:52.907395+0000","last_unstale":"2024-09-10T21:12:53.961946+0000","last_undegraded":"2024-09-10T21:12:53.961946+0000","last_fullsized":"2024-09-10T21:12:53.961946+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:19.885587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.936138+0000","last_change":"2024-09-10T21:12:53.934723+0000","last_active":"2024-09-10T21:12:53.936138+0000","last_peered":"2024-09-10T21:12:53.936138+0000","last_clean":"2024-09-10T21:12:53.936138+0000","last_became_active":"2024-09-10T21:12:53.932327+0000","last_became_peered":"2024-09-10T21:12:53.932327+0000","last_unstale":"2024-09-10T21:12:53.936138+0000","last_undegraded":"2024-09-10T21:12:53.936138+0000","last_fullsized":"2024-09-10T21:12:53.936138+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:43.720965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.925403+0000","last_change":"2024-09-10T21:12:52.922523+0000","last_active":"2024-09-10T21:12:52.925403+0000","last_peered":"2024-09-10T21:12:52.925403+0000","last_clean":"2024-09-10T21:12:52.925403+0000","last_became_active":"2024-09-10T21:12:52.922266+0000","last_became_peered":"2024-09-10T21:12:52.922266+0000","last_unstale":"2024-09-10T21:12:52.925403+0000","last_undegraded":"2024-09-10T21:12:52.925403+0000","last_fullsized":"2024-09-10T21:12:52.925403+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:16.245436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.959105+0000","last_change":"2024-09-10T21:12:53.951739+0000","last_active":"2024-09-10T21:12:53.959105+0000","last_peered":"2024-09-10T21:12:53.959105+0000","last_clean":"2024-09-10T21:12:53.959105+0000","last_became_active":"2024-09-10T21:12:53.943111+0000","last_became_peered":"2024-09-10T21:12:53.943111+0000","last_unstale":"2024-09-10T21:12:53.959105+0000","last_undegraded":"2024-09-10T21:12:53.959105+0000","last_fullsized":"2024-09-10T21:12:53.959105+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:02.237320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.916370+0000","last_change":"2024-09-10T21:12:52.925890+0000","last_active":"2024-09-10T21:12:53.916370+0000","last_peered":"2024-09-10T21:12:53.916370+0000","last_clean":"2024-09-10T21:12:53.916370+0000","last_became_active":"2024-09-10T21:12:52.925061+0000","last_became_peered":"2024-09-10T21:12:52.925061+0000","last_unstale":"2024-09-10T21:12:53.916370+0000","last_undegraded":"2024-09-10T21:12:53.916370+0000","last_fullsized":"2024-09-10T21:12:53.916370+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:53.791545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.953214+0000","last_change":"2024-09-10T21:12:53.947582+0000","last_active":"2024-09-10T21:12:53.953214+0000","last_peered":"2024-09-10T21:12:53.953214+0000","last_clean":"2024-09-10T21:12:53.953214+0000","last_became_active":"2024-09-10T21:12:53.947316+0000","last_became_peered":"2024-09-10T21:12:53.947316+0000","last_unstale":"2024-09-10T21:12:53.953214+0000","last_undegraded":"2024-09-10T21:12:53.953214+0000","last_fullsized":"2024-09-10T21:12:53.953214+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:42.777708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.922253+0000","last_change":"2024-09-10T21:12:52.920456+0000","last_active":"2024-09-10T21:12:52.922253+0000","last_peered":"2024-09-10T21:12:52.922253+0000","last_clean":"2024-09-10T21:12:52.922253+0000","last_became_active":"2024-09-10T21:12:52.920172+0000","last_became_peered":"2024-09-10T21:12:52.920172+0000","last_unstale":"2024-09-10T21:12:52.922253+0000","last_undegraded":"2024-09-10T21:12:52.922253+0000","last_fullsized":"2024-09-10T21:12:52.922253+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:09.727174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951060+0000","last_change":"2024-09-10T21:12:53.945665+0000","last_active":"2024-09-10T21:12:53.951060+0000","last_peered":"2024-09-10T21:12:53.951060+0000","last_clean":"2024-09-10T21:12:53.951060+0000","last_became_active":"2024-09-10T21:12:53.945387+0000","last_became_peered":"2024-09-10T21:12:53.945387+0000","last_unstale":"2024-09-10T21:12:53.951060+0000","last_undegraded":"2024-09-10T21:12:53.951060+0000","last_fullsized":"2024-09-10T21:12:53.951060+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:48.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.925467+0000","last_change":"2024-09-10T21:12:52.921420+0000","last_active":"2024-09-10T21:12:53.925467+0000","last_peered":"2024-09-10T21:12:53.925467+0000","last_clean":"2024-09-10T21:12:53.925467+0000","last_became_active":"2024-09-10T21:12:52.920194+0000","last_became_peered":"2024-09-10T21:12:52.920194+0000","last_unstale":"2024-09-10T21:12:53.925467+0000","last_undegraded":"2024-09-10T21:12:53.925467+0000","last_fullsized":"2024-09-10T21:12:53.925467+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:02.042065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.961558+0000","last_change":"2024-09-10T21:12:53.959498+0000","last_active":"2024-09-10T21:12:53.961558+0000","last_peered":"2024-09-10T21:12:53.961558+0000","last_clean":"2024-09-10T21:12:53.961558+0000","last_became_active":"2024-09-10T21:12:53.959157+0000","last_became_peered":"2024-09-10T21:12:53.959157+0000","last_unstale":"2024-09-10T21:12:53.961558+0000","last_undegraded":"2024-09-10T21:12:53.961558+0000","last_fullsized":"2024-09-10T21:12:53.961558+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:06.297558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.954331+0000","last_change":"2024-09-10T21:12:52.920836+0000","last_active":"2024-09-10T21:12:53.954331+0000","last_peered":"2024-09-10T21:12:53.954331+0000","last_clean":"2024-09-10T21:12:53.954331+0000","last_became_active":"2024-09-10T21:12:52.920547+0000","last_became_peered":"2024-09-10T21:12:52.920547+0000","last_unstale":"2024-09-10T21:12:53.954331+0000","last_undegraded":"2024-09-10T21:12:53.954331+0000","last_fullsized":"2024-09-10T21:12:53.954331+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:22.866900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.937227+0000","last_change":"2024-09-10T21:12:53.936019+0000","last_active":"2024-09-10T21:12:53.937227+0000","last_peered":"2024-09-10T21:12:53.937227+0000","last_clean":"2024-09-10T21:12:53.937227+0000","last_became_active":"2024-09-10T21:12:53.935728+0000","last_became_peered":"2024-09-10T21:12:53.935728+0000","last_unstale":"2024-09-10T21:12:53.937227+0000","last_undegraded":"2024-09-10T21:12:53.937227+0000","last_fullsized":"2024-09-10T21:12:53.937227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:28.139924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":19,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.957568+0000","last_change":"2024-09-10T21:12:52.924581+0000","last_active":"2024-09-10T21:12:53.957568+0000","last_peered":"2024-09-10T21:12:53.957568+0000","last_clean":"2024-09-10T21:12:53.957568+0000","last_became_active":"2024-09-10T21:12:52.924327+0000","last_became_peered":"2024-09-10T21:12:52.924327+0000","last_unstale":"2024-09-10T21:12:53.957568+0000","last_undegraded":"2024-09-10T21:12:53.957568+0000","last_fullsized":"2024-09-10T21:12:53.957568+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:10.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.905486+0000","last_change":"2024-09-10T21:12:52.905486+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.905486+0000","last_undegraded":"2024-09-10T21:12:52.905486+0000","last_fullsized":"2024-09-10T21:12:52.905486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.956965+0000","last_change":"2024-09-10T21:12:52.919371+0000","last_active":"2024-09-10T21:12:53.956965+0000","last_peered":"2024-09-10T21:12:53.956965+0000","last_clean":"2024-09-10T21:12:53.956965+0000","last_became_active":"2024-09-10T21:12:52.918995+0000","last_became_peered":"2024-09-10T21:12:52.918995+0000","last_unstale":"2024-09-10T21:12:53.956965+0000","last_undegraded":"2024-09-10T21:12:53.956965+0000","last_fullsized":"2024-09-10T21:12:53.956965+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:33.805170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.932517+0000","last_change":"2024-09-10T21:12:53.930651+0000","last_active":"2024-09-10T21:12:53.932517+0000","last_peered":"2024-09-10T21:12:53.932517+0000","last_clean":"2024-09-10T21:12:53.932517+0000","last_became_active":"2024-09-10T21:12:53.930387+0000","last_became_peered":"2024-09-10T21:12:53.930387+0000","last_unstale":"2024-09-10T21:12:53.932517+0000","last_undegraded":"2024-09-10T21:12:53.932517+0000","last_fullsized":"2024-09-10T21:12:53.932517+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:42.777708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.913794+0000","last_change":"2024-09-10T21:12:52.923255+0000","last_active":"2024-09-10T21:12:53.913794+0000","last_peered":"2024-09-10T21:12:53.913794+0000","last_clean":"2024-09-10T21:12:53.913794+0000","last_became_active":"2024-09-10T21:12:52.919965+0000","last_became_peered":"2024-09-10T21:12:52.919965+0000","last_unstale":"2024-09-10T21:12:53.913794+0000","last_undegraded":"2024-09-10T21:12:53.913794+0000","last_fullsized":"2024-09-10T21:12:53.913794+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:15.274251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.904539+0000","last_change":"2024-09-10T21:12:52.904539+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.904539+0000","last_undegraded":"2024-09-10T21:12:52.904539+0000","last_fullsized":"2024-09-10T21:12:52.904539+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.914460+0000","last_change":"2024-09-10T21:12:52.923218+0000","last_active":"2024-09-10T21:12:53.914460+0000","last_peered":"2024-09-10T21:12:53.914460+0000","last_clean":"2024-09-10T21:12:53.914460+0000","last_became_active":"2024-09-10T21:12:52.919955+0000","last_became_peered":"2024-09-10T21:12:52.919955+0000","last_unstale":"2024-09-10T21:12:53.914460+0000","last_undegraded":"2024-09-10T21:12:53.914460+0000","last_fullsized":"2024-09-10T21:12:53.914460+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:53.792649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.925605+0000","last_change":"2024-09-10T21:12:53.922279+0000","last_active":"2024-09-10T21:12:53.925605+0000","last_peered":"2024-09-10T21:12:53.925605+0000","last_clean":"2024-09-10T21:12:53.925605+0000","last_became_active":"2024-09-10T21:12:53.921900+0000","last_became_peered":"2024-09-10T21:12:53.921900+0000","last_unstale":"2024-09-10T21:12:53.925605+0000","last_undegraded":"2024-09-10T21:12:53.925605+0000","last_fullsized":"2024-09-10T21:12:53.925605+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:43.504550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":21,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.954508+0000","last_change":"2024-09-10T21:12:52.920975+0000","last_active":"2024-09-10T21:12:53.954508+0000","last_peered":"2024-09-10T21:12:53.954508+0000","last_clean":"2024-09-10T21:12:53.954508+0000","last_became_active":"2024-09-10T21:12:52.919980+0000","last_became_peered":"2024-09-10T21:12:52.919980+0000","last_unstale":"2024-09-10T21:12:53.954508+0000","last_undegraded":"2024-09-10T21:12:53.954508+0000","last_fullsized":"2024-09-10T21:12:53.954508+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:28.166375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.952338+0000","last_change":"2024-09-10T21:12:53.949707+0000","last_active":"2024-09-10T21:12:53.952338+0000","last_peered":"2024-09-10T21:12:53.952338+0000","last_clean":"2024-09-10T21:12:53.952338+0000","last_became_active":"2024-09-10T21:12:53.949438+0000","last_became_peered":"2024-09-10T21:12:53.949438+0000","last_unstale":"2024-09-10T21:12:53.952338+0000","last_undegraded":"2024-09-10T21:12:53.952338+0000","last_fullsized":"2024-09-10T21:12:53.952338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:31.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.929524+0000","last_change":"2024-09-10T21:12:52.928329+0000","last_active":"2024-09-10T21:12:52.929524+0000","last_peered":"2024-09-10T21:12:52.929524+0000","last_clean":"2024-09-10T21:12:52.929524+0000","last_became_active":"2024-09-10T21:12:52.928026+0000","last_became_peered":"2024-09-10T21:12:52.928026+0000","last_unstale":"2024-09-10T21:12:52.929524+0000","last_undegraded":"2024-09-10T21:12:52.929524+0000","last_fullsized":"2024-09-10T21:12:52.929524+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:55.890316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.952364+0000","last_change":"2024-09-10T21:12:53.946213+0000","last_active":"2024-09-10T21:12:53.952364+0000","last_peered":"2024-09-10T21:12:53.952364+0000","last_clean":"2024-09-10T21:12:53.952364+0000","last_became_active":"2024-09-10T21:12:53.945887+0000","last_became_peered":"2024-09-10T21:12:53.945887+0000","last_unstale":"2024-09-10T21:12:53.952364+0000","last_undegraded":"2024-09-10T21:12:53.952364+0000","last_fullsized":"2024-09-10T21:12:53.952364+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:28.847293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958052+0000","last_change":"2024-09-10T21:12:52.920779+0000","last_active":"2024-09-10T21:12:53.958052+0000","last_peered":"2024-09-10T21:12:53.958052+0000","last_clean":"2024-09-10T21:12:53.958052+0000","last_became_active":"2024-09-10T21:12:52.920513+0000","last_became_peered":"2024-09-10T21:12:52.920513+0000","last_unstale":"2024-09-10T21:12:53.958052+0000","last_undegraded":"2024-09-10T21:12:53.958052+0000","last_fullsized":"2024-09-10T21:12:53.958052+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:18.447950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.910688+0000","last_change":"2024-09-10T21:12:52.910688+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.910688+0000","last_undegraded":"2024-09-10T21:12:52.910688+0000","last_fullsized":"2024-09-10T21:12:52.910688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.953684+0000","last_change":"2024-09-10T21:12:52.925824+0000","last_active":"2024-09-10T21:12:53.953684+0000","last_peered":"2024-09-10T21:12:53.953684+0000","last_clean":"2024-09-10T21:12:53.953684+0000","last_became_active":"2024-09-10T21:12:52.925229+0000","last_became_peered":"2024-09-10T21:12:52.925229+0000","last_unstale":"2024-09-10T21:12:53.953684+0000","last_undegraded":"2024-09-10T21:12:53.953684+0000","last_fullsized":"2024-09-10T21:12:53.953684+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:10.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.934630+0000","last_change":"2024-09-10T21:12:53.930750+0000","last_active":"2024-09-10T21:12:53.934630+0000","last_peered":"2024-09-10T21:12:53.934630+0000","last_clean":"2024-09-10T21:12:53.934630+0000","last_became_active":"2024-09-10T21:12:53.930443+0000","last_became_peered":"2024-09-10T21:12:53.930443+0000","last_unstale":"2024-09-10T21:12:53.934630+0000","last_undegraded":"2024-09-10T21:12:53.934630+0000","last_fullsized":"2024-09-10T21:12:53.934630+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:02.992141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929696+0000","last_change":"2024-09-10T21:12:52.913454+0000","last_active":"2024-09-10T21:12:53.929696+0000","last_peered":"2024-09-10T21:12:53.929696+0000","last_clean":"2024-09-10T21:12:53.929696+0000","last_became_active":"2024-09-10T21:12:52.913160+0000","last_became_peered":"2024-09-10T21:12:52.913160+0000","last_unstale":"2024-09-10T21:12:53.929696+0000","last_undegraded":"2024-09-10T21:12:53.929696+0000","last_fullsized":"2024-09-10T21:12:53.929696+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:35.717760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.904827+0000","last_change":"2024-09-10T21:12:52.904827+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.904827+0000","last_undegraded":"2024-09-10T21:12:52.904827+0000","last_fullsized":"2024-09-10T21:12:52.904827+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.955822+0000","last_change":"2024-09-10T21:12:52.919813+0000","last_active":"2024-09-10T21:12:53.955822+0000","last_peered":"2024-09-10T21:12:53.955822+0000","last_clean":"2024-09-10T21:12:53.955822+0000","last_became_active":"2024-09-10T21:12:52.919373+0000","last_became_peered":"2024-09-10T21:12:52.919373+0000","last_unstale":"2024-09-10T21:12:53.955822+0000","last_undegraded":"2024-09-10T21:12:53.955822+0000","last_fullsized":"2024-09-10T21:12:53.955822+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:25.283722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.946775+0000","last_change":"2024-09-10T21:12:53.937502+0000","last_active":"2024-09-10T21:12:53.946775+0000","last_peered":"2024-09-10T21:12:53.946775+0000","last_clean":"2024-09-10T21:12:53.946775+0000","last_became_active":"2024-09-10T21:12:53.937201+0000","last_became_peered":"2024-09-10T21:12:53.937201+0000","last_unstale":"2024-09-10T21:12:53.946775+0000","last_undegraded":"2024-09-10T21:12:53.946775+0000","last_fullsized":"2024-09-10T21:12:53.946775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:33.689313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.926985+0000","last_change":"2024-09-10T21:12:52.920031+0000","last_active":"2024-09-10T21:12:53.926985+0000","last_peered":"2024-09-10T21:12:53.926985+0000","last_clean":"2024-09-10T21:12:53.926985+0000","last_became_active":"2024-09-10T21:12:52.919769+0000","last_became_peered":"2024-09-10T21:12:52.919769+0000","last_unstale":"2024-09-10T21:12:53.926985+0000","last_undegraded":"2024-09-10T21:12:53.926985+0000","last_fullsized":"2024-09-10T21:12:53.926985+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:08.609135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.940861+0000","last_change":"2024-09-10T21:12:53.928673+0000","last_active":"2024-09-10T21:12:53.940861+0000","last_peered":"2024-09-10T21:12:53.940861+0000","last_clean":"2024-09-10T21:12:53.940861+0000","last_became_active":"2024-09-10T21:12:53.928390+0000","last_became_peered":"2024-09-10T21:12:53.928390+0000","last_unstale":"2024-09-10T21:12:53.940861+0000","last_undegraded":"2024-09-10T21:12:53.940861+0000","last_fullsized":"2024-09-10T21:12:53.940861+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:51.847731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.916361+0000","last_change":"2024-09-10T21:12:52.910741+0000","last_active":"2024-09-10T21:12:52.916361+0000","last_peered":"2024-09-10T21:12:52.916361+0000","last_clean":"2024-09-10T21:12:52.916361+0000","last_became_active":"2024-09-10T21:12:52.910465+0000","last_became_peered":"2024-09-10T21:12:52.910465+0000","last_unstale":"2024-09-10T21:12:52.916361+0000","last_undegraded":"2024-09-10T21:12:52.916361+0000","last_fullsized":"2024-09-10T21:12:52.916361+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:06.500073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929686+0000","last_change":"2024-09-10T21:12:53.926389+0000","last_active":"2024-09-10T21:12:53.929686+0000","last_peered":"2024-09-10T21:12:53.929686+0000","last_clean":"2024-09-10T21:12:53.929686+0000","last_became_active":"2024-09-10T21:12:53.926136+0000","last_became_peered":"2024-09-10T21:12:53.926136+0000","last_unstale":"2024-09-10T21:12:53.929686+0000","last_undegraded":"2024-09-10T21:12:53.929686+0000","last_fullsized":"2024-09-10T21:12:53.929686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:25.798580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.929496+0000","last_change":"2024-09-10T21:12:52.927712+0000","last_active":"2024-09-10T21:12:52.929496+0000","last_peered":"2024-09-10T21:12:52.929496+0000","last_clean":"2024-09-10T21:12:52.929496+0000","last_became_active":"2024-09-10T21:12:52.927386+0000","last_became_peered":"2024-09-10T21:12:52.927386+0000","last_unstale":"2024-09-10T21:12:52.929496+0000","last_undegraded":"2024-09-10T21:12:52.929496+0000","last_fullsized":"2024-09-10T21:12:52.929496+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:45.588719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.931285+0000","last_change":"2024-09-10T21:12:53.925268+0000","last_active":"2024-09-10T21:12:53.931285+0000","last_peered":"2024-09-10T21:12:53.931285+0000","last_clean":"2024-09-10T21:12:53.931285+0000","last_became_active":"2024-09-10T21:12:53.924980+0000","last_became_peered":"2024-09-10T21:12:53.924980+0000","last_unstale":"2024-09-10T21:12:53.931285+0000","last_undegraded":"2024-09-10T21:12:53.931285+0000","last_fullsized":"2024-09-10T21:12:53.931285+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.802123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.943098+0000","last_change":"2024-09-10T21:12:52.908936+0000","last_active":"2024-09-10T21:12:53.943098+0000","last_peered":"2024-09-10T21:12:53.943098+0000","last_clean":"2024-09-10T21:12:53.943098+0000","last_became_active":"2024-09-10T21:12:52.908661+0000","last_became_peered":"2024-09-10T21:12:52.908661+0000","last_unstale":"2024-09-10T21:12:53.943098+0000","last_undegraded":"2024-09-10T21:12:53.943098+0000","last_fullsized":"2024-09-10T21:12:53.943098+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:07.860598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.907015+0000","last_change":"2024-09-10T21:12:52.907015+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.907015+0000","last_undegraded":"2024-09-10T21:12:52.907015+0000","last_fullsized":"2024-09-10T21:12:52.907015+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.916877+0000","last_change":"2024-09-10T21:12:52.911674+0000","last_active":"2024-09-10T21:12:52.916877+0000","last_peered":"2024-09-10T21:12:52.916877+0000","last_clean":"2024-09-10T21:12:52.916877+0000","last_became_active":"2024-09-10T21:12:52.911271+0000","last_became_peered":"2024-09-10T21:12:52.911271+0000","last_unstale":"2024-09-10T21:12:52.916877+0000","last_undegraded":"2024-09-10T21:12:52.916877+0000","last_fullsized":"2024-09-10T21:12:52.916877+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:46.241035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.940925+0000","last_change":"2024-09-10T21:12:53.933870+0000","last_active":"2024-09-10T21:12:53.940925+0000","last_peered":"2024-09-10T21:12:53.940925+0000","last_clean":"2024-09-10T21:12:53.940925+0000","last_became_active":"2024-09-10T21:12:53.933440+0000","last_became_peered":"2024-09-10T21:12:53.933440+0000","last_unstale":"2024-09-10T21:12:53.940925+0000","last_undegraded":"2024-09-10T21:12:53.940925+0000","last_fullsized":"2024-09-10T21:12:53.940925+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:25.088706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.917741+0000","last_change":"2024-09-10T21:12:52.920768+0000","last_active":"2024-09-10T21:12:53.917741+0000","last_peered":"2024-09-10T21:12:53.917741+0000","last_clean":"2024-09-10T21:12:53.917741+0000","last_became_active":"2024-09-10T21:12:52.918131+0000","last_became_peered":"2024-09-10T21:12:52.918131+0000","last_unstale":"2024-09-10T21:12:53.917741+0000","last_undegraded":"2024-09-10T21:12:53.917741+0000","last_fullsized":"2024-09-10T21:12:53.917741+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:48.097204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958785+0000","last_change":"2024-09-10T21:12:53.949638+0000","last_active":"2024-09-10T21:12:53.958785+0000","last_peered":"2024-09-10T21:12:53.958785+0000","last_clean":"2024-09-10T21:12:53.958785+0000","last_became_active":"2024-09-10T21:12:53.949177+0000","last_became_peered":"2024-09-10T21:12:53.949177+0000","last_unstale":"2024-09-10T21:12:53.958785+0000","last_undegraded":"2024-09-10T21:12:53.958785+0000","last_fullsized":"2024-09-10T21:12:53.958785+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:38.456803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.926904+0000","last_change":"2024-09-10T21:12:52.925485+0000","last_active":"2024-09-10T21:12:52.926904+0000","last_peered":"2024-09-10T21:12:52.926904+0000","last_clean":"2024-09-10T21:12:52.926904+0000","last_became_active":"2024-09-10T21:12:52.925196+0000","last_became_peered":"2024-09-10T21:12:52.925196+0000","last_unstale":"2024-09-10T21:12:52.926904+0000","last_undegraded":"2024-09-10T21:12:52.926904+0000","last_fullsized":"2024-09-10T21:12:52.926904+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:26.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.952972+0000","last_change":"2024-09-10T21:12:53.946489+0000","last_active":"2024-09-10T21:12:53.952972+0000","last_peered":"2024-09-10T21:12:53.952972+0000","last_clean":"2024-09-10T21:12:53.952972+0000","last_became_active":"2024-09-10T21:12:53.945218+0000","last_became_peered":"2024-09-10T21:12:53.945218+0000","last_unstale":"2024-09-10T21:12:53.952972+0000","last_undegraded":"2024-09-10T21:12:53.952972+0000","last_fullsized":"2024-09-10T21:12:53.952972+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:17.409325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.917083+0000","last_change":"2024-09-10T21:12:52.926307+0000","last_active":"2024-09-10T21:12:53.917083+0000","last_peered":"2024-09-10T21:12:53.917083+0000","last_clean":"2024-09-10T21:12:53.917083+0000","last_became_active":"2024-09-10T21:12:52.926021+0000","last_became_peered":"2024-09-10T21:12:52.926021+0000","last_unstale":"2024-09-10T21:12:53.917083+0000","last_undegraded":"2024-09-10T21:12:53.917083+0000","last_fullsized":"2024-09-10T21:12:53.917083+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:53.791545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951502+0000","last_change":"2024-09-10T21:12:53.944529+0000","last_active":"2024-09-10T21:12:53.951502+0000","last_peered":"2024-09-10T21:12:53.951502+0000","last_clean":"2024-09-10T21:12:53.951502+0000","last_became_active":"2024-09-10T21:12:53.944191+0000","last_became_peered":"2024-09-10T21:12:53.944191+0000","last_unstale":"2024-09-10T21:12:53.951502+0000","last_undegraded":"2024-09-10T21:12:53.951502+0000","last_fullsized":"2024-09-10T21:12:53.951502+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:18.729522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.921710+0000","last_change":"2024-09-10T21:12:52.916297+0000","last_active":"2024-09-10T21:12:53.921710+0000","last_peered":"2024-09-10T21:12:53.921710+0000","last_clean":"2024-09-10T21:12:53.921710+0000","last_became_active":"2024-09-10T21:12:52.914668+0000","last_became_peered":"2024-09-10T21:12:52.914668+0000","last_unstale":"2024-09-10T21:12:53.921710+0000","last_undegraded":"2024-09-10T21:12:53.921710+0000","last_fullsized":"2024-09-10T21:12:53.921710+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:50.985945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.906196+0000","last_change":"2024-09-10T21:12:52.906196+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.906196+0000","last_undegraded":"2024-09-10T21:12:52.906196+0000","last_fullsized":"2024-09-10T21:12:52.906196+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.927648+0000","last_change":"2024-09-10T21:12:52.914672+0000","last_active":"2024-09-10T21:12:53.927648+0000","last_peered":"2024-09-10T21:12:53.927648+0000","last_clean":"2024-09-10T21:12:53.927648+0000","last_became_active":"2024-09-10T21:12:52.914355+0000","last_became_peered":"2024-09-10T21:12:52.914355+0000","last_unstale":"2024-09-10T21:12:53.927648+0000","last_undegraded":"2024-09-10T21:12:53.927648+0000","last_fullsized":"2024-09-10T21:12:53.927648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:44.022375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.950021+0000","last_change":"2024-09-10T21:12:53.938346+0000","last_active":"2024-09-10T21:12:53.950021+0000","last_peered":"2024-09-10T21:12:53.950021+0000","last_clean":"2024-09-10T21:12:53.950021+0000","last_became_active":"2024-09-10T21:12:53.938178+0000","last_became_peered":"2024-09-10T21:12:53.938178+0000","last_unstale":"2024-09-10T21:12:53.950021+0000","last_undegraded":"2024-09-10T21:12:53.950021+0000","last_fullsized":"2024-09-10T21:12:53.950021+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:30.402157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.916982+0000","last_change":"2024-09-10T21:12:52.922364+0000","last_active":"2024-09-10T21:12:53.916982+0000","last_peered":"2024-09-10T21:12:53.916982+0000","last_clean":"2024-09-10T21:12:53.916982+0000","last_became_active":"2024-09-10T21:12:52.921965+0000","last_became_peered":"2024-09-10T21:12:52.921965+0000","last_unstale":"2024-09-10T21:12:53.916982+0000","last_undegraded":"2024-09-10T21:12:53.916982+0000","last_fullsized":"2024-09-10T21:12:53.916982+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:48.097204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":169,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.907968+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:12:52.907968+0000","last_peered":"2024-09-10T21:12:52.907968+0000","last_clean":"2024-09-10T21:12:52.907968+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:12:52.907968+0000","last_undegraded":"2024-09-10T21:12:52.907968+0000","last_fullsized":"2024-09-10T21:12:52.907968+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509474,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":667648,"data_stored":273322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1245184,"data_stored":856893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":651264,"data_stored":266525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":651264,"data_stored":266525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":651264,"data_stored":266525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509474,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":675840,"data_stored":274912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":638976,"data_stored":260250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1245184,"data_stored":856893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:12:58.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:12:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.272+0000 7f88ca800640 1 -- 172.21.15.16:0/2830206992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c415bc70 msgr2=0x7f88c406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.272+0000 7f88ca800640 1 --2- 172.21.15.16:0/2830206992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c415bc70 0x7f88c406ffe0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f88b0009900 tx=0x7f88b001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.272+0000 7f88ca800640 1 -- 172.21.15.16:0/2830206992 shutdown_connections 2024-09-10T21:12:58.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.272+0000 7f88ca800640 1 --2- 172.21.15.16:0/2830206992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c415bc70 0x7f88c406ffe0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.272+0000 7f88ca800640 1 --2- 172.21.15.16:0/2830206992 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f88c4154f90 0x7f88c4072d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.272+0000 7f88ca800640 1 -- 172.21.15.16:0/2830206992 >> 172.21.15.16:0/2830206992 conn(0x7f88c40cfb20 msgr2=0x7f88c40cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:58.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.272+0000 7f88ca800640 1 -- 172.21.15.16:0/2830206992 shutdown_connections 2024-09-10T21:12:58.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.272+0000 7f88ca800640 1 -- 172.21.15.16:0/2830206992 wait complete. 2024-09-10T21:12:58.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.272+0000 7f88ca800640 1 Processor -- start 2024-09-10T21:12:58.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.272+0000 7f88ca800640 1 -- start start 2024-09-10T21:12:58.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88ca800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88c4070520 0x7f88c40c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88ca800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f88c4154f90 0x7f88c40c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88ca800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c415bc70 0x7f88c4159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88ca800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88c4167bc0 con 0x7f88c415bc70 2024-09-10T21:12:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88ca800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f88c4167a40 con 0x7f88c4070520 2024-09-10T21:12:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88ca800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f88c4167d40 con 0x7f88c4154f90 2024-09-10T21:12:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c415bc70 0x7f88c4159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c415bc70 0x7f88c4159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45042/0 (socket says 172.21.15.16:45042) 2024-09-10T21:12:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88c3e00640 1 -- 172.21.15.16:0/3248590015 learned_addr learned my addr 172.21.15.16:0/3248590015 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88c2a00640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f88c4154f90 0x7f88c40c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88c3400640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88c4070520 0x7f88c40c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88c3e00640 1 -- 172.21.15.16:0/3248590015 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f88c4154f90 msgr2=0x7f88c40c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88c3e00640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f88c4154f90 0x7f88c40c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88c3e00640 1 -- 172.21.15.16:0/3248590015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88c4070520 msgr2=0x7f88c40c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88c3e00640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88c4070520 0x7f88c40c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88c3e00640 1 -- 172.21.15.16:0/3248590015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c40b2950 con 0x7f88c415bc70 2024-09-10T21:12:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88c3e00640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c415bc70 0x7f88c4159d10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00c780 tx=0x7f88bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88bb400640 1 -- 172.21.15.16:0/3248590015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc004b70 con 0x7f88c415bc70 2024-09-10T21:12:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88bb400640 1 -- 172.21.15.16:0/3248590015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88bc004e00 con 0x7f88c415bc70 2024-09-10T21:12:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88bb400640 1 -- 172.21.15.16:0/3248590015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc00aa90 con 0x7f88c415bc70 2024-09-10T21:12:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88ca800640 1 -- 172.21.15.16:0/3248590015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c40b2c10 con 0x7f88c415bc70 2024-09-10T21:12:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88ca800640 1 -- 172.21.15.16:0/3248590015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c40b5060 con 0x7f88c415bc70 2024-09-10T21:12:58.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.276+0000 7f88ca800640 1 -- 172.21.15.16:0/3248590015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c4072d40 con 0x7f88c415bc70 2024-09-10T21:12:58.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.280+0000 7f88bb400640 1 -- 172.21.15.16:0/3248590015 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f88bc00ac30 con 0x7f88c415bc70 2024-09-10T21:12:58.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.280+0000 7f88bb400640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88a40725e0 0x7f88a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:58.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.280+0000 7f88bb400640 1 -- 172.21.15.16:0/3248590015 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f88bc093450 con 0x7f88c415bc70 2024-09-10T21:12:58.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.280+0000 7f88c3400640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88a40725e0 0x7f88a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:58.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.280+0000 7f88c3400640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88a40725e0 0x7f88a4074aa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f88b4004630 tx=0x7f88b400b390 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.284+0000 7f88bb400640 1 -- 172.21.15.16:0/3248590015 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f88bc020070 con 0x7f88c415bc70 2024-09-10T21:12:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.456+0000 7f88ca800640 1 -- 172.21.15.16:0/3248590015 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f88c4152530 con 0x7f88a40725e0 2024-09-10T21:12:58.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.460+0000 7f88bb400640 1 -- 172.21.15.16:0/3248590015 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250794 (secure 0 0 0) 0x7f88c4152530 con 0x7f88a40725e0 2024-09-10T21:12:58.470 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:12:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.464+0000 7f88ca800640 1 -- 172.21.15.16:0/3248590015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88a40725e0 msgr2=0x7f88a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.464+0000 7f88ca800640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88a40725e0 0x7f88a4074aa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f88b4004630 tx=0x7f88b400b390 comp rx=0 tx=0).stop 2024-09-10T21:12:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.464+0000 7f88ca800640 1 -- 172.21.15.16:0/3248590015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c415bc70 msgr2=0x7f88c4159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.464+0000 7f88ca800640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c415bc70 0x7f88c4159d10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00c780 tx=0x7f88bc00cc40 comp rx=0 tx=0).stop 2024-09-10T21:12:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.464+0000 7f88ca800640 1 -- 172.21.15.16:0/3248590015 shutdown_connections 2024-09-10T21:12:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.464+0000 7f88ca800640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88a40725e0 0x7f88a4074aa0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.464+0000 7f88ca800640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88c415bc70 0x7f88c4159d10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.464+0000 7f88ca800640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f88c4154f90 0x7f88c40c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.464+0000 7f88ca800640 1 --2- 172.21.15.16:0/3248590015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88c4070520 0x7f88c40c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.464+0000 7f88ca800640 1 -- 172.21.15.16:0/3248590015 >> 172.21.15.16:0/3248590015 conn(0x7f88c40cfb20 msgr2=0x7f88c40ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.464+0000 7f88ca800640 1 -- 172.21.15.16:0/3248590015 shutdown_connections 2024-09-10T21:12:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.464+0000 7f88ca800640 1 -- 172.21.15.16:0/3248590015 wait complete. 2024-09-10T21:12:58.491 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-10T21:12:57.489868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255812,"kb_used_data":6276,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718908,"statfs":{"total":773094113280,"available":772832161792,"internally_reserved":0,"allocated":6426624,"data_stored":3321845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.597272"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.962730+0000","last_change":"2024-09-10T21:12:53.959848+0000","last_active":"2024-09-10T21:12:53.962730+0000","last_peered":"2024-09-10T21:12:53.962730+0000","last_clean":"2024-09-10T21:12:53.962730+0000","last_became_active":"2024-09-10T21:12:53.959238+0000","last_became_peered":"2024-09-10T21:12:53.959238+0000","last_unstale":"2024-09-10T21:12:53.962730+0000","last_undegraded":"2024-09-10T21:12:53.962730+0000","last_fullsized":"2024-09-10T21:12:53.962730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:27.189161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.908750+0000","last_change":"2024-09-10T21:12:52.908750+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.908750+0000","last_undegraded":"2024-09-10T21:12:52.908750+0000","last_fullsized":"2024-09-10T21:12:52.908750+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.931243+0000","last_change":"2024-09-10T21:12:53.929639+0000","last_active":"2024-09-10T21:12:53.931243+0000","last_peered":"2024-09-10T21:12:53.931243+0000","last_clean":"2024-09-10T21:12:53.931243+0000","last_became_active":"2024-09-10T21:12:53.929333+0000","last_became_peered":"2024-09-10T21:12:53.929333+0000","last_unstale":"2024-09-10T21:12:53.931243+0000","last_undegraded":"2024-09-10T21:12:53.931243+0000","last_fullsized":"2024-09-10T21:12:53.931243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:17.409325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.936510+0000","last_change":"2024-09-10T21:12:53.932951+0000","last_active":"2024-09-10T21:12:53.936510+0000","last_peered":"2024-09-10T21:12:53.936510+0000","last_clean":"2024-09-10T21:12:53.936510+0000","last_became_active":"2024-09-10T21:12:53.930361+0000","last_became_peered":"2024-09-10T21:12:53.930361+0000","last_unstale":"2024-09-10T21:12:53.936510+0000","last_undegraded":"2024-09-10T21:12:53.936510+0000","last_fullsized":"2024-09-10T21:12:53.936510+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:01.888505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.957550+0000","last_change":"2024-09-10T21:12:53.951504+0000","last_active":"2024-09-10T21:12:53.957550+0000","last_peered":"2024-09-10T21:12:53.957550+0000","last_clean":"2024-09-10T21:12:53.957550+0000","last_became_active":"2024-09-10T21:12:53.950545+0000","last_became_peered":"2024-09-10T21:12:53.950545+0000","last_unstale":"2024-09-10T21:12:53.957550+0000","last_undegraded":"2024-09-10T21:12:53.957550+0000","last_fullsized":"2024-09-10T21:12:53.957550+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:55.157666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958761+0000","last_change":"2024-09-10T21:12:53.949235+0000","last_active":"2024-09-10T21:12:53.958761+0000","last_peered":"2024-09-10T21:12:53.958761+0000","last_clean":"2024-09-10T21:12:53.958761+0000","last_became_active":"2024-09-10T21:12:53.936152+0000","last_became_peered":"2024-09-10T21:12:53.936152+0000","last_unstale":"2024-09-10T21:12:53.958761+0000","last_undegraded":"2024-09-10T21:12:53.958761+0000","last_fullsized":"2024-09-10T21:12:53.958761+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:01.430435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.934571+0000","last_change":"2024-09-10T21:12:53.929021+0000","last_active":"2024-09-10T21:12:53.934571+0000","last_peered":"2024-09-10T21:12:53.934571+0000","last_clean":"2024-09-10T21:12:53.934571+0000","last_became_active":"2024-09-10T21:12:53.928713+0000","last_became_peered":"2024-09-10T21:12:53.928713+0000","last_unstale":"2024-09-10T21:12:53.934571+0000","last_undegraded":"2024-09-10T21:12:53.934571+0000","last_fullsized":"2024-09-10T21:12:53.934571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:57.528446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.945096+0000","last_change":"2024-09-10T21:12:53.934558+0000","last_active":"2024-09-10T21:12:53.945096+0000","last_peered":"2024-09-10T21:12:53.945096+0000","last_clean":"2024-09-10T21:12:53.945096+0000","last_became_active":"2024-09-10T21:12:53.934226+0000","last_became_peered":"2024-09-10T21:12:53.934226+0000","last_unstale":"2024-09-10T21:12:53.945096+0000","last_undegraded":"2024-09-10T21:12:53.945096+0000","last_fullsized":"2024-09-10T21:12:53.945096+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:27.657368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.905254+0000","last_change":"2024-09-10T21:12:52.905254+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.905254+0000","last_undegraded":"2024-09-10T21:12:52.905254+0000","last_fullsized":"2024-09-10T21:12:52.905254+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958633+0000","last_change":"2024-09-10T21:12:53.952820+0000","last_active":"2024-09-10T21:12:53.958633+0000","last_peered":"2024-09-10T21:12:53.958633+0000","last_clean":"2024-09-10T21:12:53.958633+0000","last_became_active":"2024-09-10T21:12:53.952356+0000","last_became_peered":"2024-09-10T21:12:53.952356+0000","last_unstale":"2024-09-10T21:12:53.958633+0000","last_undegraded":"2024-09-10T21:12:53.958633+0000","last_fullsized":"2024-09-10T21:12:53.958633+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.570424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.960202+0000","last_change":"2024-09-10T21:12:53.957907+0000","last_active":"2024-09-10T21:12:53.960202+0000","last_peered":"2024-09-10T21:12:53.960202+0000","last_clean":"2024-09-10T21:12:53.960202+0000","last_became_active":"2024-09-10T21:12:53.957595+0000","last_became_peered":"2024-09-10T21:12:53.957595+0000","last_unstale":"2024-09-10T21:12:53.960202+0000","last_undegraded":"2024-09-10T21:12:53.960202+0000","last_fullsized":"2024-09-10T21:12:53.960202+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:02.523420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.905908+0000","last_change":"2024-09-10T21:12:52.905908+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.905908+0000","last_undegraded":"2024-09-10T21:12:52.905908+0000","last_fullsized":"2024-09-10T21:12:52.905908+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.928804+0000","last_change":"2024-09-10T21:12:53.922722+0000","last_active":"2024-09-10T21:12:53.928804+0000","last_peered":"2024-09-10T21:12:53.928804+0000","last_clean":"2024-09-10T21:12:53.928804+0000","last_became_active":"2024-09-10T21:12:53.922457+0000","last_became_peered":"2024-09-10T21:12:53.922457+0000","last_unstale":"2024-09-10T21:12:53.928804+0000","last_undegraded":"2024-09-10T21:12:53.928804+0000","last_fullsized":"2024-09-10T21:12:53.928804+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:06.040633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951748+0000","last_change":"2024-09-10T21:12:53.945632+0000","last_active":"2024-09-10T21:12:53.951748+0000","last_peered":"2024-09-10T21:12:53.951748+0000","last_clean":"2024-09-10T21:12:53.951748+0000","last_became_active":"2024-09-10T21:12:53.945340+0000","last_became_peered":"2024-09-10T21:12:53.945340+0000","last_unstale":"2024-09-10T21:12:53.951748+0000","last_undegraded":"2024-09-10T21:12:53.951748+0000","last_fullsized":"2024-09-10T21:12:53.951748+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:31.129061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.937358+0000","last_change":"2024-09-10T21:12:53.934328+0000","last_active":"2024-09-10T21:12:53.937358+0000","last_peered":"2024-09-10T21:12:53.937358+0000","last_clean":"2024-09-10T21:12:53.937358+0000","last_became_active":"2024-09-10T21:12:53.934057+0000","last_became_peered":"2024-09-10T21:12:53.934057+0000","last_unstale":"2024-09-10T21:12:53.937358+0000","last_undegraded":"2024-09-10T21:12:53.937358+0000","last_fullsized":"2024-09-10T21:12:53.937358+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:25.798580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.956730+0000","last_change":"2024-09-10T21:12:53.935764+0000","last_active":"2024-09-10T21:12:53.956730+0000","last_peered":"2024-09-10T21:12:53.956730+0000","last_clean":"2024-09-10T21:12:53.956730+0000","last_became_active":"2024-09-10T21:12:53.935468+0000","last_became_peered":"2024-09-10T21:12:53.935468+0000","last_unstale":"2024-09-10T21:12:53.956730+0000","last_undegraded":"2024-09-10T21:12:53.956730+0000","last_fullsized":"2024-09-10T21:12:53.956730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:35.882232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.959571+0000","last_change":"2024-09-10T21:12:53.953515+0000","last_active":"2024-09-10T21:12:53.959571+0000","last_peered":"2024-09-10T21:12:53.959571+0000","last_clean":"2024-09-10T21:12:53.959571+0000","last_became_active":"2024-09-10T21:12:53.952358+0000","last_became_peered":"2024-09-10T21:12:53.952358+0000","last_unstale":"2024-09-10T21:12:53.959571+0000","last_undegraded":"2024-09-10T21:12:53.959571+0000","last_fullsized":"2024-09-10T21:12:53.959571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:44.453397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929660+0000","last_change":"2024-09-10T21:12:53.923168+0000","last_active":"2024-09-10T21:12:53.929660+0000","last_peered":"2024-09-10T21:12:53.929660+0000","last_clean":"2024-09-10T21:12:53.929660+0000","last_became_active":"2024-09-10T21:12:53.922878+0000","last_became_peered":"2024-09-10T21:12:53.922878+0000","last_unstale":"2024-09-10T21:12:53.929660+0000","last_undegraded":"2024-09-10T21:12:53.929660+0000","last_fullsized":"2024-09-10T21:12:53.929660+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:20.410692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.917837+0000","last_change":"2024-09-10T21:12:52.917837+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.917837+0000","last_undegraded":"2024-09-10T21:12:52.917837+0000","last_fullsized":"2024-09-10T21:12:52.917837+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.961063+0000","last_change":"2024-09-10T21:12:53.959543+0000","last_active":"2024-09-10T21:12:53.961063+0000","last_peered":"2024-09-10T21:12:53.961063+0000","last_clean":"2024-09-10T21:12:53.961063+0000","last_became_active":"2024-09-10T21:12:53.959240+0000","last_became_peered":"2024-09-10T21:12:53.959240+0000","last_unstale":"2024-09-10T21:12:53.961063+0000","last_undegraded":"2024-09-10T21:12:53.961063+0000","last_fullsized":"2024-09-10T21:12:53.961063+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:19.453855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929367+0000","last_change":"2024-09-10T21:12:53.926236+0000","last_active":"2024-09-10T21:12:53.929367+0000","last_peered":"2024-09-10T21:12:53.929367+0000","last_clean":"2024-09-10T21:12:53.929367+0000","last_became_active":"2024-09-10T21:12:53.924195+0000","last_became_peered":"2024-09-10T21:12:53.924195+0000","last_unstale":"2024-09-10T21:12:53.929367+0000","last_undegraded":"2024-09-10T21:12:53.929367+0000","last_fullsized":"2024-09-10T21:12:53.929367+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.802123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.945190+0000","last_change":"2024-09-10T21:12:53.934083+0000","last_active":"2024-09-10T21:12:53.945190+0000","last_peered":"2024-09-10T21:12:53.945190+0000","last_clean":"2024-09-10T21:12:53.945190+0000","last_became_active":"2024-09-10T21:12:53.933188+0000","last_became_peered":"2024-09-10T21:12:53.933188+0000","last_unstale":"2024-09-10T21:12:53.945190+0000","last_undegraded":"2024-09-10T21:12:53.945190+0000","last_fullsized":"2024-09-10T21:12:53.945190+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:00.887008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.922965+0000","last_change":"2024-09-10T21:12:53.920537+0000","last_active":"2024-09-10T21:12:53.922965+0000","last_peered":"2024-09-10T21:12:53.922965+0000","last_clean":"2024-09-10T21:12:53.922965+0000","last_became_active":"2024-09-10T21:12:53.920252+0000","last_became_peered":"2024-09-10T21:12:53.920252+0000","last_unstale":"2024-09-10T21:12:53.922965+0000","last_undegraded":"2024-09-10T21:12:53.922965+0000","last_fullsized":"2024-09-10T21:12:53.922965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:51.239584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.905107+0000","last_change":"2024-09-10T21:12:52.905107+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.905107+0000","last_undegraded":"2024-09-10T21:12:52.905107+0000","last_fullsized":"2024-09-10T21:12:52.905107+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.909354+0000","last_change":"2024-09-10T21:12:52.909354+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.909354+0000","last_undegraded":"2024-09-10T21:12:52.909354+0000","last_fullsized":"2024-09-10T21:12:52.909354+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929245+0000","last_change":"2024-09-10T21:12:53.923199+0000","last_active":"2024-09-10T21:12:53.929245+0000","last_peered":"2024-09-10T21:12:53.929245+0000","last_clean":"2024-09-10T21:12:53.929245+0000","last_became_active":"2024-09-10T21:12:53.922779+0000","last_became_peered":"2024-09-10T21:12:53.922779+0000","last_unstale":"2024-09-10T21:12:53.929245+0000","last_undegraded":"2024-09-10T21:12:53.929245+0000","last_fullsized":"2024-09-10T21:12:53.929245+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:20.410692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.914654+0000","last_change":"2024-09-10T21:12:52.914654+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.914654+0000","last_undegraded":"2024-09-10T21:12:52.914654+0000","last_fullsized":"2024-09-10T21:12:52.914654+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951229+0000","last_change":"2024-09-10T21:12:53.939591+0000","last_active":"2024-09-10T21:12:53.951229+0000","last_peered":"2024-09-10T21:12:53.951229+0000","last_clean":"2024-09-10T21:12:53.951229+0000","last_became_active":"2024-09-10T21:12:53.939314+0000","last_became_peered":"2024-09-10T21:12:53.939314+0000","last_unstale":"2024-09-10T21:12:53.951229+0000","last_undegraded":"2024-09-10T21:12:53.951229+0000","last_fullsized":"2024-09-10T21:12:53.951229+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:02.044073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.946748+0000","last_change":"2024-09-10T21:12:53.937058+0000","last_active":"2024-09-10T21:12:53.946748+0000","last_peered":"2024-09-10T21:12:53.946748+0000","last_clean":"2024-09-10T21:12:53.946748+0000","last_became_active":"2024-09-10T21:12:53.936798+0000","last_became_peered":"2024-09-10T21:12:53.936798+0000","last_unstale":"2024-09-10T21:12:53.946748+0000","last_undegraded":"2024-09-10T21:12:53.946748+0000","last_fullsized":"2024-09-10T21:12:53.946748+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:53.754974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929094+0000","last_change":"2024-09-10T21:12:53.925270+0000","last_active":"2024-09-10T21:12:53.929094+0000","last_peered":"2024-09-10T21:12:53.929094+0000","last_clean":"2024-09-10T21:12:53.929094+0000","last_became_active":"2024-09-10T21:12:53.923493+0000","last_became_peered":"2024-09-10T21:12:53.923493+0000","last_unstale":"2024-09-10T21:12:53.929094+0000","last_undegraded":"2024-09-10T21:12:53.929094+0000","last_fullsized":"2024-09-10T21:12:53.929094+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:01.637614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.914057+0000","last_change":"2024-09-10T21:12:52.914057+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.914057+0000","last_undegraded":"2024-09-10T21:12:52.914057+0000","last_fullsized":"2024-09-10T21:12:52.914057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.945266+0000","last_change":"2024-09-10T21:12:53.939129+0000","last_active":"2024-09-10T21:12:53.945266+0000","last_peered":"2024-09-10T21:12:53.945266+0000","last_clean":"2024-09-10T21:12:53.945266+0000","last_became_active":"2024-09-10T21:12:53.938807+0000","last_became_peered":"2024-09-10T21:12:53.938807+0000","last_unstale":"2024-09-10T21:12:53.945266+0000","last_undegraded":"2024-09-10T21:12:53.945266+0000","last_fullsized":"2024-09-10T21:12:53.945266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:30.631397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.956016+0000","last_change":"2024-09-10T21:12:53.938689+0000","last_active":"2024-09-10T21:12:53.956016+0000","last_peered":"2024-09-10T21:12:53.956016+0000","last_clean":"2024-09-10T21:12:53.956016+0000","last_became_active":"2024-09-10T21:12:53.935691+0000","last_became_peered":"2024-09-10T21:12:53.935691+0000","last_unstale":"2024-09-10T21:12:53.956016+0000","last_undegraded":"2024-09-10T21:12:53.956016+0000","last_fullsized":"2024-09-10T21:12:53.956016+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:10.944201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":17,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.957007+0000","last_change":"2024-09-10T21:12:52.916629+0000","last_active":"2024-09-10T21:12:53.957007+0000","last_peered":"2024-09-10T21:12:53.957007+0000","last_clean":"2024-09-10T21:12:53.957007+0000","last_became_active":"2024-09-10T21:12:52.916091+0000","last_became_peered":"2024-09-10T21:12:52.916091+0000","last_unstale":"2024-09-10T21:12:53.957007+0000","last_undegraded":"2024-09-10T21:12:53.957007+0000","last_fullsized":"2024-09-10T21:12:53.957007+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:27.028802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.959594+0000","last_change":"2024-09-10T21:12:53.953827+0000","last_active":"2024-09-10T21:12:53.959594+0000","last_peered":"2024-09-10T21:12:53.959594+0000","last_clean":"2024-09-10T21:12:53.959594+0000","last_became_active":"2024-09-10T21:12:53.953262+0000","last_became_peered":"2024-09-10T21:12:53.953262+0000","last_unstale":"2024-09-10T21:12:53.959594+0000","last_undegraded":"2024-09-10T21:12:53.959594+0000","last_fullsized":"2024-09-10T21:12:53.959594+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:20.378633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.928971+0000","last_change":"2024-09-10T21:12:52.926961+0000","last_active":"2024-09-10T21:12:52.928971+0000","last_peered":"2024-09-10T21:12:52.928971+0000","last_clean":"2024-09-10T21:12:52.928971+0000","last_became_active":"2024-09-10T21:12:52.926723+0000","last_became_peered":"2024-09-10T21:12:52.926723+0000","last_unstale":"2024-09-10T21:12:52.928971+0000","last_undegraded":"2024-09-10T21:12:52.928971+0000","last_fullsized":"2024-09-10T21:12:52.928971+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:40.512719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951499+0000","last_change":"2024-09-10T21:12:53.944452+0000","last_active":"2024-09-10T21:12:53.951499+0000","last_peered":"2024-09-10T21:12:53.951499+0000","last_clean":"2024-09-10T21:12:53.951499+0000","last_became_active":"2024-09-10T21:12:53.944142+0000","last_became_peered":"2024-09-10T21:12:53.944142+0000","last_unstale":"2024-09-10T21:12:53.951499+0000","last_undegraded":"2024-09-10T21:12:53.951499+0000","last_fullsized":"2024-09-10T21:12:53.951499+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.507264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.963349+0000","last_change":"2024-09-10T21:12:52.924953+0000","last_active":"2024-09-10T21:12:53.963349+0000","last_peered":"2024-09-10T21:12:53.963349+0000","last_clean":"2024-09-10T21:12:53.963349+0000","last_became_active":"2024-09-10T21:12:52.924601+0000","last_became_peered":"2024-09-10T21:12:52.924601+0000","last_unstale":"2024-09-10T21:12:53.963349+0000","last_undegraded":"2024-09-10T21:12:53.963349+0000","last_fullsized":"2024-09-10T21:12:53.963349+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:44.842888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958253+0000","last_change":"2024-09-10T21:12:53.950957+0000","last_active":"2024-09-10T21:12:53.958253+0000","last_peered":"2024-09-10T21:12:53.958253+0000","last_clean":"2024-09-10T21:12:53.958253+0000","last_became_active":"2024-09-10T21:12:53.950673+0000","last_became_peered":"2024-09-10T21:12:53.950673+0000","last_unstale":"2024-09-10T21:12:53.958253+0000","last_undegraded":"2024-09-10T21:12:53.958253+0000","last_fullsized":"2024-09-10T21:12:53.958253+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:16.326966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.954185+0000","last_change":"2024-09-10T21:12:52.920769+0000","last_active":"2024-09-10T21:12:53.954185+0000","last_peered":"2024-09-10T21:12:53.954185+0000","last_clean":"2024-09-10T21:12:53.954185+0000","last_became_active":"2024-09-10T21:12:52.920460+0000","last_became_peered":"2024-09-10T21:12:52.920460+0000","last_unstale":"2024-09-10T21:12:53.954185+0000","last_undegraded":"2024-09-10T21:12:53.954185+0000","last_fullsized":"2024-09-10T21:12:53.954185+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:53.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951781+0000","last_change":"2024-09-10T21:12:53.946296+0000","last_active":"2024-09-10T21:12:53.951781+0000","last_peered":"2024-09-10T21:12:53.951781+0000","last_clean":"2024-09-10T21:12:53.951781+0000","last_became_active":"2024-09-10T21:12:53.945868+0000","last_became_peered":"2024-09-10T21:12:53.945868+0000","last_unstale":"2024-09-10T21:12:53.951781+0000","last_undegraded":"2024-09-10T21:12:53.951781+0000","last_fullsized":"2024-09-10T21:12:53.951781+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:56.605281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.955272+0000","last_change":"2024-09-10T21:12:52.917429+0000","last_active":"2024-09-10T21:12:53.955272+0000","last_peered":"2024-09-10T21:12:53.955272+0000","last_clean":"2024-09-10T21:12:53.955272+0000","last_became_active":"2024-09-10T21:12:52.917148+0000","last_became_peered":"2024-09-10T21:12:52.917148+0000","last_unstale":"2024-09-10T21:12:53.955272+0000","last_undegraded":"2024-09-10T21:12:53.955272+0000","last_fullsized":"2024-09-10T21:12:53.955272+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:05.034728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.950682+0000","last_change":"2024-09-10T21:12:53.947773+0000","last_active":"2024-09-10T21:12:53.950682+0000","last_peered":"2024-09-10T21:12:53.950682+0000","last_clean":"2024-09-10T21:12:53.950682+0000","last_became_active":"2024-09-10T21:12:53.947473+0000","last_became_peered":"2024-09-10T21:12:53.947473+0000","last_unstale":"2024-09-10T21:12:53.950682+0000","last_undegraded":"2024-09-10T21:12:53.950682+0000","last_fullsized":"2024-09-10T21:12:53.950682+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:45.835056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.961946+0000","last_change":"2024-09-10T21:12:52.907684+0000","last_active":"2024-09-10T21:12:53.961946+0000","last_peered":"2024-09-10T21:12:53.961946+0000","last_clean":"2024-09-10T21:12:53.961946+0000","last_became_active":"2024-09-10T21:12:52.907395+0000","last_became_peered":"2024-09-10T21:12:52.907395+0000","last_unstale":"2024-09-10T21:12:53.961946+0000","last_undegraded":"2024-09-10T21:12:53.961946+0000","last_fullsized":"2024-09-10T21:12:53.961946+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:19.885587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.936138+0000","last_change":"2024-09-10T21:12:53.934723+0000","last_active":"2024-09-10T21:12:53.936138+0000","last_peered":"2024-09-10T21:12:53.936138+0000","last_clean":"2024-09-10T21:12:53.936138+0000","last_became_active":"2024-09-10T21:12:53.932327+0000","last_became_peered":"2024-09-10T21:12:53.932327+0000","last_unstale":"2024-09-10T21:12:53.936138+0000","last_undegraded":"2024-09-10T21:12:53.936138+0000","last_fullsized":"2024-09-10T21:12:53.936138+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:43.720965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.925403+0000","last_change":"2024-09-10T21:12:52.922523+0000","last_active":"2024-09-10T21:12:52.925403+0000","last_peered":"2024-09-10T21:12:52.925403+0000","last_clean":"2024-09-10T21:12:52.925403+0000","last_became_active":"2024-09-10T21:12:52.922266+0000","last_became_peered":"2024-09-10T21:12:52.922266+0000","last_unstale":"2024-09-10T21:12:52.925403+0000","last_undegraded":"2024-09-10T21:12:52.925403+0000","last_fullsized":"2024-09-10T21:12:52.925403+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:16.245436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.959105+0000","last_change":"2024-09-10T21:12:53.951739+0000","last_active":"2024-09-10T21:12:53.959105+0000","last_peered":"2024-09-10T21:12:53.959105+0000","last_clean":"2024-09-10T21:12:53.959105+0000","last_became_active":"2024-09-10T21:12:53.943111+0000","last_became_peered":"2024-09-10T21:12:53.943111+0000","last_unstale":"2024-09-10T21:12:53.959105+0000","last_undegraded":"2024-09-10T21:12:53.959105+0000","last_fullsized":"2024-09-10T21:12:53.959105+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:02.237320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.916370+0000","last_change":"2024-09-10T21:12:52.925890+0000","last_active":"2024-09-10T21:12:53.916370+0000","last_peered":"2024-09-10T21:12:53.916370+0000","last_clean":"2024-09-10T21:12:53.916370+0000","last_became_active":"2024-09-10T21:12:52.925061+0000","last_became_peered":"2024-09-10T21:12:52.925061+0000","last_unstale":"2024-09-10T21:12:53.916370+0000","last_undegraded":"2024-09-10T21:12:53.916370+0000","last_fullsized":"2024-09-10T21:12:53.916370+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:53.791545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.953214+0000","last_change":"2024-09-10T21:12:53.947582+0000","last_active":"2024-09-10T21:12:53.953214+0000","last_peered":"2024-09-10T21:12:53.953214+0000","last_clean":"2024-09-10T21:12:53.953214+0000","last_became_active":"2024-09-10T21:12:53.947316+0000","last_became_peered":"2024-09-10T21:12:53.947316+0000","last_unstale":"2024-09-10T21:12:53.953214+0000","last_undegraded":"2024-09-10T21:12:53.953214+0000","last_fullsized":"2024-09-10T21:12:53.953214+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:42.777708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.922253+0000","last_change":"2024-09-10T21:12:52.920456+0000","last_active":"2024-09-10T21:12:52.922253+0000","last_peered":"2024-09-10T21:12:52.922253+0000","last_clean":"2024-09-10T21:12:52.922253+0000","last_became_active":"2024-09-10T21:12:52.920172+0000","last_became_peered":"2024-09-10T21:12:52.920172+0000","last_unstale":"2024-09-10T21:12:52.922253+0000","last_undegraded":"2024-09-10T21:12:52.922253+0000","last_fullsized":"2024-09-10T21:12:52.922253+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:09.727174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951060+0000","last_change":"2024-09-10T21:12:53.945665+0000","last_active":"2024-09-10T21:12:53.951060+0000","last_peered":"2024-09-10T21:12:53.951060+0000","last_clean":"2024-09-10T21:12:53.951060+0000","last_became_active":"2024-09-10T21:12:53.945387+0000","last_became_peered":"2024-09-10T21:12:53.945387+0000","last_unstale":"2024-09-10T21:12:53.951060+0000","last_undegraded":"2024-09-10T21:12:53.951060+0000","last_fullsized":"2024-09-10T21:12:53.951060+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:48.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.925467+0000","last_change":"2024-09-10T21:12:52.921420+0000","last_active":"2024-09-10T21:12:53.925467+0000","last_peered":"2024-09-10T21:12:53.925467+0000","last_clean":"2024-09-10T21:12:53.925467+0000","last_became_active":"2024-09-10T21:12:52.920194+0000","last_became_peered":"2024-09-10T21:12:52.920194+0000","last_unstale":"2024-09-10T21:12:53.925467+0000","last_undegraded":"2024-09-10T21:12:53.925467+0000","last_fullsized":"2024-09-10T21:12:53.925467+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:02.042065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.961558+0000","last_change":"2024-09-10T21:12:53.959498+0000","last_active":"2024-09-10T21:12:53.961558+0000","last_peered":"2024-09-10T21:12:53.961558+0000","last_clean":"2024-09-10T21:12:53.961558+0000","last_became_active":"2024-09-10T21:12:53.959157+0000","last_became_peered":"2024-09-10T21:12:53.959157+0000","last_unstale":"2024-09-10T21:12:53.961558+0000","last_undegraded":"2024-09-10T21:12:53.961558+0000","last_fullsized":"2024-09-10T21:12:53.961558+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:06.297558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.954331+0000","last_change":"2024-09-10T21:12:52.920836+0000","last_active":"2024-09-10T21:12:53.954331+0000","last_peered":"2024-09-10T21:12:53.954331+0000","last_clean":"2024-09-10T21:12:53.954331+0000","last_became_active":"2024-09-10T21:12:52.920547+0000","last_became_peered":"2024-09-10T21:12:52.920547+0000","last_unstale":"2024-09-10T21:12:53.954331+0000","last_undegraded":"2024-09-10T21:12:53.954331+0000","last_fullsized":"2024-09-10T21:12:53.954331+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:22.866900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.937227+0000","last_change":"2024-09-10T21:12:53.936019+0000","last_active":"2024-09-10T21:12:53.937227+0000","last_peered":"2024-09-10T21:12:53.937227+0000","last_clean":"2024-09-10T21:12:53.937227+0000","last_became_active":"2024-09-10T21:12:53.935728+0000","last_became_peered":"2024-09-10T21:12:53.935728+0000","last_unstale":"2024-09-10T21:12:53.937227+0000","last_undegraded":"2024-09-10T21:12:53.937227+0000","last_fullsized":"2024-09-10T21:12:53.937227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:28.139924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":19,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.957568+0000","last_change":"2024-09-10T21:12:52.924581+0000","last_active":"2024-09-10T21:12:53.957568+0000","last_peered":"2024-09-10T21:12:53.957568+0000","last_clean":"2024-09-10T21:12:53.957568+0000","last_became_active":"2024-09-10T21:12:52.924327+0000","last_became_peered":"2024-09-10T21:12:52.924327+0000","last_unstale":"2024-09-10T21:12:53.957568+0000","last_undegraded":"2024-09-10T21:12:53.957568+0000","last_fullsized":"2024-09-10T21:12:53.957568+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:10.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.905486+0000","last_change":"2024-09-10T21:12:52.905486+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.905486+0000","last_undegraded":"2024-09-10T21:12:52.905486+0000","last_fullsized":"2024-09-10T21:12:52.905486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.956965+0000","last_change":"2024-09-10T21:12:52.919371+0000","last_active":"2024-09-10T21:12:53.956965+0000","last_peered":"2024-09-10T21:12:53.956965+0000","last_clean":"2024-09-10T21:12:53.956965+0000","last_became_active":"2024-09-10T21:12:52.918995+0000","last_became_peered":"2024-09-10T21:12:52.918995+0000","last_unstale":"2024-09-10T21:12:53.956965+0000","last_undegraded":"2024-09-10T21:12:53.956965+0000","last_fullsized":"2024-09-10T21:12:53.956965+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:33.805170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.932517+0000","last_change":"2024-09-10T21:12:53.930651+0000","last_active":"2024-09-10T21:12:53.932517+0000","last_peered":"2024-09-10T21:12:53.932517+0000","last_clean":"2024-09-10T21:12:53.932517+0000","last_became_active":"2024-09-10T21:12:53.930387+0000","last_became_peered":"2024-09-10T21:12:53.930387+0000","last_unstale":"2024-09-10T21:12:53.932517+0000","last_undegraded":"2024-09-10T21:12:53.932517+0000","last_fullsized":"2024-09-10T21:12:53.932517+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:42.777708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.913794+0000","last_change":"2024-09-10T21:12:52.923255+0000","last_active":"2024-09-10T21:12:53.913794+0000","last_peered":"2024-09-10T21:12:53.913794+0000","last_clean":"2024-09-10T21:12:53.913794+0000","last_became_active":"2024-09-10T21:12:52.919965+0000","last_became_peered":"2024-09-10T21:12:52.919965+0000","last_unstale":"2024-09-10T21:12:53.913794+0000","last_undegraded":"2024-09-10T21:12:53.913794+0000","last_fullsized":"2024-09-10T21:12:53.913794+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:15.274251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.904539+0000","last_change":"2024-09-10T21:12:52.904539+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.904539+0000","last_undegraded":"2024-09-10T21:12:52.904539+0000","last_fullsized":"2024-09-10T21:12:52.904539+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.914460+0000","last_change":"2024-09-10T21:12:52.923218+0000","last_active":"2024-09-10T21:12:53.914460+0000","last_peered":"2024-09-10T21:12:53.914460+0000","last_clean":"2024-09-10T21:12:53.914460+0000","last_became_active":"2024-09-10T21:12:52.919955+0000","last_became_peered":"2024-09-10T21:12:52.919955+0000","last_unstale":"2024-09-10T21:12:53.914460+0000","last_undegraded":"2024-09-10T21:12:53.914460+0000","last_fullsized":"2024-09-10T21:12:53.914460+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:53.792649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.925605+0000","last_change":"2024-09-10T21:12:53.922279+0000","last_active":"2024-09-10T21:12:53.925605+0000","last_peered":"2024-09-10T21:12:53.925605+0000","last_clean":"2024-09-10T21:12:53.925605+0000","last_became_active":"2024-09-10T21:12:53.921900+0000","last_became_peered":"2024-09-10T21:12:53.921900+0000","last_unstale":"2024-09-10T21:12:53.925605+0000","last_undegraded":"2024-09-10T21:12:53.925605+0000","last_fullsized":"2024-09-10T21:12:53.925605+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:43.504550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":21,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.954508+0000","last_change":"2024-09-10T21:12:52.920975+0000","last_active":"2024-09-10T21:12:53.954508+0000","last_peered":"2024-09-10T21:12:53.954508+0000","last_clean":"2024-09-10T21:12:53.954508+0000","last_became_active":"2024-09-10T21:12:52.919980+0000","last_became_peered":"2024-09-10T21:12:52.919980+0000","last_unstale":"2024-09-10T21:12:53.954508+0000","last_undegraded":"2024-09-10T21:12:53.954508+0000","last_fullsized":"2024-09-10T21:12:53.954508+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:28.166375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.952338+0000","last_change":"2024-09-10T21:12:53.949707+0000","last_active":"2024-09-10T21:12:53.952338+0000","last_peered":"2024-09-10T21:12:53.952338+0000","last_clean":"2024-09-10T21:12:53.952338+0000","last_became_active":"2024-09-10T21:12:53.949438+0000","last_became_peered":"2024-09-10T21:12:53.949438+0000","last_unstale":"2024-09-10T21:12:53.952338+0000","last_undegraded":"2024-09-10T21:12:53.952338+0000","last_fullsized":"2024-09-10T21:12:53.952338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:31.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.929524+0000","last_change":"2024-09-10T21:12:52.928329+0000","last_active":"2024-09-10T21:12:52.929524+0000","last_peered":"2024-09-10T21:12:52.929524+0000","last_clean":"2024-09-10T21:12:52.929524+0000","last_became_active":"2024-09-10T21:12:52.928026+0000","last_became_peered":"2024-09-10T21:12:52.928026+0000","last_unstale":"2024-09-10T21:12:52.929524+0000","last_undegraded":"2024-09-10T21:12:52.929524+0000","last_fullsized":"2024-09-10T21:12:52.929524+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:55.890316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.952364+0000","last_change":"2024-09-10T21:12:53.946213+0000","last_active":"2024-09-10T21:12:53.952364+0000","last_peered":"2024-09-10T21:12:53.952364+0000","last_clean":"2024-09-10T21:12:53.952364+0000","last_became_active":"2024-09-10T21:12:53.945887+0000","last_became_peered":"2024-09-10T21:12:53.945887+0000","last_unstale":"2024-09-10T21:12:53.952364+0000","last_undegraded":"2024-09-10T21:12:53.952364+0000","last_fullsized":"2024-09-10T21:12:53.952364+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:28.847293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958052+0000","last_change":"2024-09-10T21:12:52.920779+0000","last_active":"2024-09-10T21:12:53.958052+0000","last_peered":"2024-09-10T21:12:53.958052+0000","last_clean":"2024-09-10T21:12:53.958052+0000","last_became_active":"2024-09-10T21:12:52.920513+0000","last_became_peered":"2024-09-10T21:12:52.920513+0000","last_unstale":"2024-09-10T21:12:53.958052+0000","last_undegraded":"2024-09-10T21:12:53.958052+0000","last_fullsized":"2024-09-10T21:12:53.958052+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:18.447950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.910688+0000","last_change":"2024-09-10T21:12:52.910688+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.910688+0000","last_undegraded":"2024-09-10T21:12:52.910688+0000","last_fullsized":"2024-09-10T21:12:52.910688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.953684+0000","last_change":"2024-09-10T21:12:52.925824+0000","last_active":"2024-09-10T21:12:53.953684+0000","last_peered":"2024-09-10T21:12:53.953684+0000","last_clean":"2024-09-10T21:12:53.953684+0000","last_became_active":"2024-09-10T21:12:52.925229+0000","last_became_peered":"2024-09-10T21:12:52.925229+0000","last_unstale":"2024-09-10T21:12:53.953684+0000","last_undegraded":"2024-09-10T21:12:53.953684+0000","last_fullsized":"2024-09-10T21:12:53.953684+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:10.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.934630+0000","last_change":"2024-09-10T21:12:53.930750+0000","last_active":"2024-09-10T21:12:53.934630+0000","last_peered":"2024-09-10T21:12:53.934630+0000","last_clean":"2024-09-10T21:12:53.934630+0000","last_became_active":"2024-09-10T21:12:53.930443+0000","last_became_peered":"2024-09-10T21:12:53.930443+0000","last_unstale":"2024-09-10T21:12:53.934630+0000","last_undegraded":"2024-09-10T21:12:53.934630+0000","last_fullsized":"2024-09-10T21:12:53.934630+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:02.992141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929696+0000","last_change":"2024-09-10T21:12:52.913454+0000","last_active":"2024-09-10T21:12:53.929696+0000","last_peered":"2024-09-10T21:12:53.929696+0000","last_clean":"2024-09-10T21:12:53.929696+0000","last_became_active":"2024-09-10T21:12:52.913160+0000","last_became_peered":"2024-09-10T21:12:52.913160+0000","last_unstale":"2024-09-10T21:12:53.929696+0000","last_undegraded":"2024-09-10T21:12:53.929696+0000","last_fullsized":"2024-09-10T21:12:53.929696+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:35.717760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.904827+0000","last_change":"2024-09-10T21:12:52.904827+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.904827+0000","last_undegraded":"2024-09-10T21:12:52.904827+0000","last_fullsized":"2024-09-10T21:12:52.904827+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.955822+0000","last_change":"2024-09-10T21:12:52.919813+0000","last_active":"2024-09-10T21:12:53.955822+0000","last_peered":"2024-09-10T21:12:53.955822+0000","last_clean":"2024-09-10T21:12:53.955822+0000","last_became_active":"2024-09-10T21:12:52.919373+0000","last_became_peered":"2024-09-10T21:12:52.919373+0000","last_unstale":"2024-09-10T21:12:53.955822+0000","last_undegraded":"2024-09-10T21:12:53.955822+0000","last_fullsized":"2024-09-10T21:12:53.955822+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:25.283722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.946775+0000","last_change":"2024-09-10T21:12:53.937502+0000","last_active":"2024-09-10T21:12:53.946775+0000","last_peered":"2024-09-10T21:12:53.946775+0000","last_clean":"2024-09-10T21:12:53.946775+0000","last_became_active":"2024-09-10T21:12:53.937201+0000","last_became_peered":"2024-09-10T21:12:53.937201+0000","last_unstale":"2024-09-10T21:12:53.946775+0000","last_undegraded":"2024-09-10T21:12:53.946775+0000","last_fullsized":"2024-09-10T21:12:53.946775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:33.689313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.926985+0000","last_change":"2024-09-10T21:12:52.920031+0000","last_active":"2024-09-10T21:12:53.926985+0000","last_peered":"2024-09-10T21:12:53.926985+0000","last_clean":"2024-09-10T21:12:53.926985+0000","last_became_active":"2024-09-10T21:12:52.919769+0000","last_became_peered":"2024-09-10T21:12:52.919769+0000","last_unstale":"2024-09-10T21:12:53.926985+0000","last_undegraded":"2024-09-10T21:12:53.926985+0000","last_fullsized":"2024-09-10T21:12:53.926985+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:08.609135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.940861+0000","last_change":"2024-09-10T21:12:53.928673+0000","last_active":"2024-09-10T21:12:53.940861+0000","last_peered":"2024-09-10T21:12:53.940861+0000","last_clean":"2024-09-10T21:12:53.940861+0000","last_became_active":"2024-09-10T21:12:53.928390+0000","last_became_peered":"2024-09-10T21:12:53.928390+0000","last_unstale":"2024-09-10T21:12:53.940861+0000","last_undegraded":"2024-09-10T21:12:53.940861+0000","last_fullsized":"2024-09-10T21:12:53.940861+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:51.847731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.916361+0000","last_change":"2024-09-10T21:12:52.910741+0000","last_active":"2024-09-10T21:12:52.916361+0000","last_peered":"2024-09-10T21:12:52.916361+0000","last_clean":"2024-09-10T21:12:52.916361+0000","last_became_active":"2024-09-10T21:12:52.910465+0000","last_became_peered":"2024-09-10T21:12:52.910465+0000","last_unstale":"2024-09-10T21:12:52.916361+0000","last_undegraded":"2024-09-10T21:12:52.916361+0000","last_fullsized":"2024-09-10T21:12:52.916361+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:06.500073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929686+0000","last_change":"2024-09-10T21:12:53.926389+0000","last_active":"2024-09-10T21:12:53.929686+0000","last_peered":"2024-09-10T21:12:53.929686+0000","last_clean":"2024-09-10T21:12:53.929686+0000","last_became_active":"2024-09-10T21:12:53.926136+0000","last_became_peered":"2024-09-10T21:12:53.926136+0000","last_unstale":"2024-09-10T21:12:53.929686+0000","last_undegraded":"2024-09-10T21:12:53.929686+0000","last_fullsized":"2024-09-10T21:12:53.929686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:25.798580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.929496+0000","last_change":"2024-09-10T21:12:52.927712+0000","last_active":"2024-09-10T21:12:52.929496+0000","last_peered":"2024-09-10T21:12:52.929496+0000","last_clean":"2024-09-10T21:12:52.929496+0000","last_became_active":"2024-09-10T21:12:52.927386+0000","last_became_peered":"2024-09-10T21:12:52.927386+0000","last_unstale":"2024-09-10T21:12:52.929496+0000","last_undegraded":"2024-09-10T21:12:52.929496+0000","last_fullsized":"2024-09-10T21:12:52.929496+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:45.588719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.931285+0000","last_change":"2024-09-10T21:12:53.925268+0000","last_active":"2024-09-10T21:12:53.931285+0000","last_peered":"2024-09-10T21:12:53.931285+0000","last_clean":"2024-09-10T21:12:53.931285+0000","last_became_active":"2024-09-10T21:12:53.924980+0000","last_became_peered":"2024-09-10T21:12:53.924980+0000","last_unstale":"2024-09-10T21:12:53.931285+0000","last_undegraded":"2024-09-10T21:12:53.931285+0000","last_fullsized":"2024-09-10T21:12:53.931285+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.802123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.943098+0000","last_change":"2024-09-10T21:12:52.908936+0000","last_active":"2024-09-10T21:12:53.943098+0000","last_peered":"2024-09-10T21:12:53.943098+0000","last_clean":"2024-09-10T21:12:53.943098+0000","last_became_active":"2024-09-10T21:12:52.908661+0000","last_became_peered":"2024-09-10T21:12:52.908661+0000","last_unstale":"2024-09-10T21:12:53.943098+0000","last_undegraded":"2024-09-10T21:12:53.943098+0000","last_fullsized":"2024-09-10T21:12:53.943098+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:07.860598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.907015+0000","last_change":"2024-09-10T21:12:52.907015+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.907015+0000","last_undegraded":"2024-09-10T21:12:52.907015+0000","last_fullsized":"2024-09-10T21:12:52.907015+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.916877+0000","last_change":"2024-09-10T21:12:52.911674+0000","last_active":"2024-09-10T21:12:52.916877+0000","last_peered":"2024-09-10T21:12:52.916877+0000","last_clean":"2024-09-10T21:12:52.916877+0000","last_became_active":"2024-09-10T21:12:52.911271+0000","last_became_peered":"2024-09-10T21:12:52.911271+0000","last_unstale":"2024-09-10T21:12:52.916877+0000","last_undegraded":"2024-09-10T21:12:52.916877+0000","last_fullsized":"2024-09-10T21:12:52.916877+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:46.241035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.940925+0000","last_change":"2024-09-10T21:12:53.933870+0000","last_active":"2024-09-10T21:12:53.940925+0000","last_peered":"2024-09-10T21:12:53.940925+0000","last_clean":"2024-09-10T21:12:53.940925+0000","last_became_active":"2024-09-10T21:12:53.933440+0000","last_became_peered":"2024-09-10T21:12:53.933440+0000","last_unstale":"2024-09-10T21:12:53.940925+0000","last_undegraded":"2024-09-10T21:12:53.940925+0000","last_fullsized":"2024-09-10T21:12:53.940925+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:25.088706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.917741+0000","last_change":"2024-09-10T21:12:52.920768+0000","last_active":"2024-09-10T21:12:53.917741+0000","last_peered":"2024-09-10T21:12:53.917741+0000","last_clean":"2024-09-10T21:12:53.917741+0000","last_became_active":"2024-09-10T21:12:52.918131+0000","last_became_peered":"2024-09-10T21:12:52.918131+0000","last_unstale":"2024-09-10T21:12:53.917741+0000","last_undegraded":"2024-09-10T21:12:53.917741+0000","last_fullsized":"2024-09-10T21:12:53.917741+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:48.097204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958785+0000","last_change":"2024-09-10T21:12:53.949638+0000","last_active":"2024-09-10T21:12:53.958785+0000","last_peered":"2024-09-10T21:12:53.958785+0000","last_clean":"2024-09-10T21:12:53.958785+0000","last_became_active":"2024-09-10T21:12:53.949177+0000","last_became_peered":"2024-09-10T21:12:53.949177+0000","last_unstale":"2024-09-10T21:12:53.958785+0000","last_undegraded":"2024-09-10T21:12:53.958785+0000","last_fullsized":"2024-09-10T21:12:53.958785+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:38.456803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.926904+0000","last_change":"2024-09-10T21:12:52.925485+0000","last_active":"2024-09-10T21:12:52.926904+0000","last_peered":"2024-09-10T21:12:52.926904+0000","last_clean":"2024-09-10T21:12:52.926904+0000","last_became_active":"2024-09-10T21:12:52.925196+0000","last_became_peered":"2024-09-10T21:12:52.925196+0000","last_unstale":"2024-09-10T21:12:52.926904+0000","last_undegraded":"2024-09-10T21:12:52.926904+0000","last_fullsized":"2024-09-10T21:12:52.926904+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:26.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.952972+0000","last_change":"2024-09-10T21:12:53.946489+0000","last_active":"2024-09-10T21:12:53.952972+0000","last_peered":"2024-09-10T21:12:53.952972+0000","last_clean":"2024-09-10T21:12:53.952972+0000","last_became_active":"2024-09-10T21:12:53.945218+0000","last_became_peered":"2024-09-10T21:12:53.945218+0000","last_unstale":"2024-09-10T21:12:53.952972+0000","last_undegraded":"2024-09-10T21:12:53.952972+0000","last_fullsized":"2024-09-10T21:12:53.952972+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:17.409325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.917083+0000","last_change":"2024-09-10T21:12:52.926307+0000","last_active":"2024-09-10T21:12:53.917083+0000","last_peered":"2024-09-10T21:12:53.917083+0000","last_clean":"2024-09-10T21:12:53.917083+0000","last_became_active":"2024-09-10T21:12:52.926021+0000","last_became_peered":"2024-09-10T21:12:52.926021+0000","last_unstale":"2024-09-10T21:12:53.917083+0000","last_undegraded":"2024-09-10T21:12:53.917083+0000","last_fullsized":"2024-09-10T21:12:53.917083+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:53.791545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951502+0000","last_change":"2024-09-10T21:12:53.944529+0000","last_active":"2024-09-10T21:12:53.951502+0000","last_peered":"2024-09-10T21:12:53.951502+0000","last_clean":"2024-09-10T21:12:53.951502+0000","last_became_active":"2024-09-10T21:12:53.944191+0000","last_became_peered":"2024-09-10T21:12:53.944191+0000","last_unstale":"2024-09-10T21:12:53.951502+0000","last_undegraded":"2024-09-10T21:12:53.951502+0000","last_fullsized":"2024-09-10T21:12:53.951502+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:18.729522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.921710+0000","last_change":"2024-09-10T21:12:52.916297+0000","last_active":"2024-09-10T21:12:53.921710+0000","last_peered":"2024-09-10T21:12:53.921710+0000","last_clean":"2024-09-10T21:12:53.921710+0000","last_became_active":"2024-09-10T21:12:52.914668+0000","last_became_peered":"2024-09-10T21:12:52.914668+0000","last_unstale":"2024-09-10T21:12:53.921710+0000","last_undegraded":"2024-09-10T21:12:53.921710+0000","last_fullsized":"2024-09-10T21:12:53.921710+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:50.985945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.906196+0000","last_change":"2024-09-10T21:12:52.906196+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.906196+0000","last_undegraded":"2024-09-10T21:12:52.906196+0000","last_fullsized":"2024-09-10T21:12:52.906196+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.927648+0000","last_change":"2024-09-10T21:12:52.914672+0000","last_active":"2024-09-10T21:12:53.927648+0000","last_peered":"2024-09-10T21:12:53.927648+0000","last_clean":"2024-09-10T21:12:53.927648+0000","last_became_active":"2024-09-10T21:12:52.914355+0000","last_became_peered":"2024-09-10T21:12:52.914355+0000","last_unstale":"2024-09-10T21:12:53.927648+0000","last_undegraded":"2024-09-10T21:12:53.927648+0000","last_fullsized":"2024-09-10T21:12:53.927648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:44.022375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.950021+0000","last_change":"2024-09-10T21:12:53.938346+0000","last_active":"2024-09-10T21:12:53.950021+0000","last_peered":"2024-09-10T21:12:53.950021+0000","last_clean":"2024-09-10T21:12:53.950021+0000","last_became_active":"2024-09-10T21:12:53.938178+0000","last_became_peered":"2024-09-10T21:12:53.938178+0000","last_unstale":"2024-09-10T21:12:53.950021+0000","last_undegraded":"2024-09-10T21:12:53.950021+0000","last_fullsized":"2024-09-10T21:12:53.950021+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:30.402157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.916982+0000","last_change":"2024-09-10T21:12:52.922364+0000","last_active":"2024-09-10T21:12:53.916982+0000","last_peered":"2024-09-10T21:12:53.916982+0000","last_clean":"2024-09-10T21:12:53.916982+0000","last_became_active":"2024-09-10T21:12:52.921965+0000","last_became_peered":"2024-09-10T21:12:52.921965+0000","last_unstale":"2024-09-10T21:12:53.916982+0000","last_undegraded":"2024-09-10T21:12:53.916982+0000","last_fullsized":"2024-09-10T21:12:53.916982+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:48.097204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":169,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.907968+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:12:52.907968+0000","last_peered":"2024-09-10T21:12:52.907968+0000","last_clean":"2024-09-10T21:12:52.907968+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:12:52.907968+0000","last_undegraded":"2024-09-10T21:12:52.907968+0000","last_fullsized":"2024-09-10T21:12:52.907968+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509474,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":667648,"data_stored":273322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1245184,"data_stored":856893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":651264,"data_stored":266525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":651264,"data_stored":266525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":651264,"data_stored":266525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509474,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":675840,"data_stored":274912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":638976,"data_stored":260250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1245184,"data_stored":856893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:12:58.495 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:12:58.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.672+0000 7ff310600640 1 -- 172.21.15.16:0/1124940622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30815f400 msgr2=0x7ff308070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:58.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.672+0000 7ff310600640 1 --2- 172.21.15.16:0/1124940622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30815f400 0x7ff308070fe0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc005fd0 tx=0x7ff2fc01c700 comp rx=0 tx=0).stop 2024-09-10T21:12:58.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.672+0000 7ff310600640 1 -- 172.21.15.16:0/1124940622 shutdown_connections 2024-09-10T21:12:58.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.672+0000 7ff310600640 1 --2- 172.21.15.16:0/1124940622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30815f400 0x7ff308070fe0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.672+0000 7ff310600640 1 -- 172.21.15.16:0/1124940622 >> 172.21.15.16:0/1124940622 conn(0x7ff30809b130 msgr2=0x7ff3080d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:58.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.672+0000 7ff310600640 1 -- 172.21.15.16:0/1124940622 shutdown_connections 2024-09-10T21:12:58.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.672+0000 7ff310600640 1 -- 172.21.15.16:0/1124940622 wait complete. 2024-09-10T21:12:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.672+0000 7ff310600640 1 Processor -- start 2024-09-10T21:12:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff310600640 1 -- start start 2024-09-10T21:12:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff310600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3080b0590 0x7ff30814db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff310600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3080b13c0 0x7ff308156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff310600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30815f400 0x7ff3080b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff310600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff30816a800 con 0x7ff30815f400 2024-09-10T21:12:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff310600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff3080d1260 con 0x7ff3080b13c0 2024-09-10T21:12:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff310600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff30816a980 con 0x7ff3080b0590 2024-09-10T21:12:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff30e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30815f400 0x7ff3080b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff30e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30815f400 0x7ff3080b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45054/0 (socket says 172.21.15.16:45054) 2024-09-10T21:12:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff30e600640 1 -- 172.21.15.16:0/1938963157 learned_addr learned my addr 172.21.15.16:0/1938963157 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff30dc00640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3080b0590 0x7ff30814db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff30d200640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3080b13c0 0x7ff308156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff30e600640 1 -- 172.21.15.16:0/1938963157 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3080b0590 msgr2=0x7ff30814db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff30e600640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3080b0590 0x7ff30814db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff30e600640 1 -- 172.21.15.16:0/1938963157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3080b13c0 msgr2=0x7ff308156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff30e600640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3080b13c0 0x7ff308156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff30e600640 1 -- 172.21.15.16:0/1938963157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30814e090 con 0x7ff30815f400 2024-09-10T21:12:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff30dc00640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3080b0590 0x7ff30814db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff30d200640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3080b13c0 0x7ff308156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff30e600640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30815f400 0x7ff3080b1e80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff30000c780 tx=0x7ff30000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff305600640 1 -- 172.21.15.16:0/1938963157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300017070 con 0x7ff30815f400 2024-09-10T21:12:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff305600640 1 -- 172.21.15.16:0/1938963157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff300005070 con 0x7ff30815f400 2024-09-10T21:12:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff310600640 1 -- 172.21.15.16:0/1938963157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff308153460 con 0x7ff30815f400 2024-09-10T21:12:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff305600640 1 -- 172.21.15.16:0/1938963157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300005500 con 0x7ff30815f400 2024-09-10T21:12:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff310600640 1 -- 172.21.15.16:0/1938963157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff308153a20 con 0x7ff30815f400 2024-09-10T21:12:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.676+0000 7ff310600640 1 -- 172.21.15.16:0/1938963157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff308070fe0 con 0x7ff30815f400 2024-09-10T21:12:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.680+0000 7ff305600640 1 -- 172.21.15.16:0/1938963157 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff30000a5e0 con 0x7ff30815f400 2024-09-10T21:12:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.680+0000 7ff305600640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff2e8072610 0x7ff2e8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.680+0000 7ff305600640 1 -- 172.21.15.16:0/1938963157 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff300092be0 con 0x7ff30815f400 2024-09-10T21:12:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.680+0000 7ff30dc00640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff2e8072610 0x7ff2e8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.680+0000 7ff30dc00640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff2e8072610 0x7ff2e8074ad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0ae000 tx=0x7ff2fc0aa040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.684+0000 7ff305600640 1 -- 172.21.15.16:0/1938963157 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff30814e090 con 0x7ff30815f400 2024-09-10T21:12:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.852+0000 7ff310600640 1 -- 172.21.15.16:0/1938963157 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff308157440 con 0x7ff2e8072610 2024-09-10T21:12:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.856+0000 7ff305600640 1 -- 172.21.15.16:0/1938963157 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250794 (secure 0 0 0) 0x7ff2fc0fd3c0 con 0x7ff2e8072610 2024-09-10T21:12:58.867 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:12:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.860+0000 7ff310600640 1 -- 172.21.15.16:0/1938963157 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff2e8072610 msgr2=0x7ff2e8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.860+0000 7ff310600640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff2e8072610 0x7ff2e8074ad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0ae000 tx=0x7ff2fc0aa040 comp rx=0 tx=0).stop 2024-09-10T21:12:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.860+0000 7ff310600640 1 -- 172.21.15.16:0/1938963157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30815f400 msgr2=0x7ff3080b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.860+0000 7ff310600640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30815f400 0x7ff3080b1e80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff30000c780 tx=0x7ff30000cc40 comp rx=0 tx=0).stop 2024-09-10T21:12:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.860+0000 7ff310600640 1 -- 172.21.15.16:0/1938963157 shutdown_connections 2024-09-10T21:12:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.860+0000 7ff310600640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff2e8072610 0x7ff2e8074ad0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.860+0000 7ff310600640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff30815f400 0x7ff3080b1e80 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.860+0000 7ff310600640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3080b13c0 0x7ff308156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.860+0000 7ff310600640 1 --2- 172.21.15.16:0/1938963157 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3080b0590 0x7ff30814db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.860+0000 7ff310600640 1 -- 172.21.15.16:0/1938963157 >> 172.21.15.16:0/1938963157 conn(0x7ff30809b130 msgr2=0x7ff30815a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.860+0000 7ff310600640 1 -- 172.21.15.16:0/1938963157 shutdown_connections 2024-09-10T21:12:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:58.860+0000 7ff310600640 1 -- 172.21.15.16:0/1938963157 wait complete. 2024-09-10T21:12:58.888 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-10T21:12:57.489868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255812,"kb_used_data":6276,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718908,"statfs":{"total":773094113280,"available":772832161792,"internally_reserved":0,"allocated":6426624,"data_stored":3321845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.597272"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.962730+0000","last_change":"2024-09-10T21:12:53.959848+0000","last_active":"2024-09-10T21:12:53.962730+0000","last_peered":"2024-09-10T21:12:53.962730+0000","last_clean":"2024-09-10T21:12:53.962730+0000","last_became_active":"2024-09-10T21:12:53.959238+0000","last_became_peered":"2024-09-10T21:12:53.959238+0000","last_unstale":"2024-09-10T21:12:53.962730+0000","last_undegraded":"2024-09-10T21:12:53.962730+0000","last_fullsized":"2024-09-10T21:12:53.962730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:27.189161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.908750+0000","last_change":"2024-09-10T21:12:52.908750+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.908750+0000","last_undegraded":"2024-09-10T21:12:52.908750+0000","last_fullsized":"2024-09-10T21:12:52.908750+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.931243+0000","last_change":"2024-09-10T21:12:53.929639+0000","last_active":"2024-09-10T21:12:53.931243+0000","last_peered":"2024-09-10T21:12:53.931243+0000","last_clean":"2024-09-10T21:12:53.931243+0000","last_became_active":"2024-09-10T21:12:53.929333+0000","last_became_peered":"2024-09-10T21:12:53.929333+0000","last_unstale":"2024-09-10T21:12:53.931243+0000","last_undegraded":"2024-09-10T21:12:53.931243+0000","last_fullsized":"2024-09-10T21:12:53.931243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:17.409325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.936510+0000","last_change":"2024-09-10T21:12:53.932951+0000","last_active":"2024-09-10T21:12:53.936510+0000","last_peered":"2024-09-10T21:12:53.936510+0000","last_clean":"2024-09-10T21:12:53.936510+0000","last_became_active":"2024-09-10T21:12:53.930361+0000","last_became_peered":"2024-09-10T21:12:53.930361+0000","last_unstale":"2024-09-10T21:12:53.936510+0000","last_undegraded":"2024-09-10T21:12:53.936510+0000","last_fullsized":"2024-09-10T21:12:53.936510+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:01.888505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.957550+0000","last_change":"2024-09-10T21:12:53.951504+0000","last_active":"2024-09-10T21:12:53.957550+0000","last_peered":"2024-09-10T21:12:53.957550+0000","last_clean":"2024-09-10T21:12:53.957550+0000","last_became_active":"2024-09-10T21:12:53.950545+0000","last_became_peered":"2024-09-10T21:12:53.950545+0000","last_unstale":"2024-09-10T21:12:53.957550+0000","last_undegraded":"2024-09-10T21:12:53.957550+0000","last_fullsized":"2024-09-10T21:12:53.957550+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:55.157666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958761+0000","last_change":"2024-09-10T21:12:53.949235+0000","last_active":"2024-09-10T21:12:53.958761+0000","last_peered":"2024-09-10T21:12:53.958761+0000","last_clean":"2024-09-10T21:12:53.958761+0000","last_became_active":"2024-09-10T21:12:53.936152+0000","last_became_peered":"2024-09-10T21:12:53.936152+0000","last_unstale":"2024-09-10T21:12:53.958761+0000","last_undegraded":"2024-09-10T21:12:53.958761+0000","last_fullsized":"2024-09-10T21:12:53.958761+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:01.430435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.934571+0000","last_change":"2024-09-10T21:12:53.929021+0000","last_active":"2024-09-10T21:12:53.934571+0000","last_peered":"2024-09-10T21:12:53.934571+0000","last_clean":"2024-09-10T21:12:53.934571+0000","last_became_active":"2024-09-10T21:12:53.928713+0000","last_became_peered":"2024-09-10T21:12:53.928713+0000","last_unstale":"2024-09-10T21:12:53.934571+0000","last_undegraded":"2024-09-10T21:12:53.934571+0000","last_fullsized":"2024-09-10T21:12:53.934571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:57.528446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.945096+0000","last_change":"2024-09-10T21:12:53.934558+0000","last_active":"2024-09-10T21:12:53.945096+0000","last_peered":"2024-09-10T21:12:53.945096+0000","last_clean":"2024-09-10T21:12:53.945096+0000","last_became_active":"2024-09-10T21:12:53.934226+0000","last_became_peered":"2024-09-10T21:12:53.934226+0000","last_unstale":"2024-09-10T21:12:53.945096+0000","last_undegraded":"2024-09-10T21:12:53.945096+0000","last_fullsized":"2024-09-10T21:12:53.945096+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:27.657368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.905254+0000","last_change":"2024-09-10T21:12:52.905254+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.905254+0000","last_undegraded":"2024-09-10T21:12:52.905254+0000","last_fullsized":"2024-09-10T21:12:52.905254+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958633+0000","last_change":"2024-09-10T21:12:53.952820+0000","last_active":"2024-09-10T21:12:53.958633+0000","last_peered":"2024-09-10T21:12:53.958633+0000","last_clean":"2024-09-10T21:12:53.958633+0000","last_became_active":"2024-09-10T21:12:53.952356+0000","last_became_peered":"2024-09-10T21:12:53.952356+0000","last_unstale":"2024-09-10T21:12:53.958633+0000","last_undegraded":"2024-09-10T21:12:53.958633+0000","last_fullsized":"2024-09-10T21:12:53.958633+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:00.570424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.960202+0000","last_change":"2024-09-10T21:12:53.957907+0000","last_active":"2024-09-10T21:12:53.960202+0000","last_peered":"2024-09-10T21:12:53.960202+0000","last_clean":"2024-09-10T21:12:53.960202+0000","last_became_active":"2024-09-10T21:12:53.957595+0000","last_became_peered":"2024-09-10T21:12:53.957595+0000","last_unstale":"2024-09-10T21:12:53.960202+0000","last_undegraded":"2024-09-10T21:12:53.960202+0000","last_fullsized":"2024-09-10T21:12:53.960202+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:02.523420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.905908+0000","last_change":"2024-09-10T21:12:52.905908+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.905908+0000","last_undegraded":"2024-09-10T21:12:52.905908+0000","last_fullsized":"2024-09-10T21:12:52.905908+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.928804+0000","last_change":"2024-09-10T21:12:53.922722+0000","last_active":"2024-09-10T21:12:53.928804+0000","last_peered":"2024-09-10T21:12:53.928804+0000","last_clean":"2024-09-10T21:12:53.928804+0000","last_became_active":"2024-09-10T21:12:53.922457+0000","last_became_peered":"2024-09-10T21:12:53.922457+0000","last_unstale":"2024-09-10T21:12:53.928804+0000","last_undegraded":"2024-09-10T21:12:53.928804+0000","last_fullsized":"2024-09-10T21:12:53.928804+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:06.040633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951748+0000","last_change":"2024-09-10T21:12:53.945632+0000","last_active":"2024-09-10T21:12:53.951748+0000","last_peered":"2024-09-10T21:12:53.951748+0000","last_clean":"2024-09-10T21:12:53.951748+0000","last_became_active":"2024-09-10T21:12:53.945340+0000","last_became_peered":"2024-09-10T21:12:53.945340+0000","last_unstale":"2024-09-10T21:12:53.951748+0000","last_undegraded":"2024-09-10T21:12:53.951748+0000","last_fullsized":"2024-09-10T21:12:53.951748+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:31.129061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.937358+0000","last_change":"2024-09-10T21:12:53.934328+0000","last_active":"2024-09-10T21:12:53.937358+0000","last_peered":"2024-09-10T21:12:53.937358+0000","last_clean":"2024-09-10T21:12:53.937358+0000","last_became_active":"2024-09-10T21:12:53.934057+0000","last_became_peered":"2024-09-10T21:12:53.934057+0000","last_unstale":"2024-09-10T21:12:53.937358+0000","last_undegraded":"2024-09-10T21:12:53.937358+0000","last_fullsized":"2024-09-10T21:12:53.937358+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:25.798580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.956730+0000","last_change":"2024-09-10T21:12:53.935764+0000","last_active":"2024-09-10T21:12:53.956730+0000","last_peered":"2024-09-10T21:12:53.956730+0000","last_clean":"2024-09-10T21:12:53.956730+0000","last_became_active":"2024-09-10T21:12:53.935468+0000","last_became_peered":"2024-09-10T21:12:53.935468+0000","last_unstale":"2024-09-10T21:12:53.956730+0000","last_undegraded":"2024-09-10T21:12:53.956730+0000","last_fullsized":"2024-09-10T21:12:53.956730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:35.882232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.959571+0000","last_change":"2024-09-10T21:12:53.953515+0000","last_active":"2024-09-10T21:12:53.959571+0000","last_peered":"2024-09-10T21:12:53.959571+0000","last_clean":"2024-09-10T21:12:53.959571+0000","last_became_active":"2024-09-10T21:12:53.952358+0000","last_became_peered":"2024-09-10T21:12:53.952358+0000","last_unstale":"2024-09-10T21:12:53.959571+0000","last_undegraded":"2024-09-10T21:12:53.959571+0000","last_fullsized":"2024-09-10T21:12:53.959571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:44.453397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929660+0000","last_change":"2024-09-10T21:12:53.923168+0000","last_active":"2024-09-10T21:12:53.929660+0000","last_peered":"2024-09-10T21:12:53.929660+0000","last_clean":"2024-09-10T21:12:53.929660+0000","last_became_active":"2024-09-10T21:12:53.922878+0000","last_became_peered":"2024-09-10T21:12:53.922878+0000","last_unstale":"2024-09-10T21:12:53.929660+0000","last_undegraded":"2024-09-10T21:12:53.929660+0000","last_fullsized":"2024-09-10T21:12:53.929660+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:20.410692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.917837+0000","last_change":"2024-09-10T21:12:52.917837+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.917837+0000","last_undegraded":"2024-09-10T21:12:52.917837+0000","last_fullsized":"2024-09-10T21:12:52.917837+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.961063+0000","last_change":"2024-09-10T21:12:53.959543+0000","last_active":"2024-09-10T21:12:53.961063+0000","last_peered":"2024-09-10T21:12:53.961063+0000","last_clean":"2024-09-10T21:12:53.961063+0000","last_became_active":"2024-09-10T21:12:53.959240+0000","last_became_peered":"2024-09-10T21:12:53.959240+0000","last_unstale":"2024-09-10T21:12:53.961063+0000","last_undegraded":"2024-09-10T21:12:53.961063+0000","last_fullsized":"2024-09-10T21:12:53.961063+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:19.453855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929367+0000","last_change":"2024-09-10T21:12:53.926236+0000","last_active":"2024-09-10T21:12:53.929367+0000","last_peered":"2024-09-10T21:12:53.929367+0000","last_clean":"2024-09-10T21:12:53.929367+0000","last_became_active":"2024-09-10T21:12:53.924195+0000","last_became_peered":"2024-09-10T21:12:53.924195+0000","last_unstale":"2024-09-10T21:12:53.929367+0000","last_undegraded":"2024-09-10T21:12:53.929367+0000","last_fullsized":"2024-09-10T21:12:53.929367+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.802123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.945190+0000","last_change":"2024-09-10T21:12:53.934083+0000","last_active":"2024-09-10T21:12:53.945190+0000","last_peered":"2024-09-10T21:12:53.945190+0000","last_clean":"2024-09-10T21:12:53.945190+0000","last_became_active":"2024-09-10T21:12:53.933188+0000","last_became_peered":"2024-09-10T21:12:53.933188+0000","last_unstale":"2024-09-10T21:12:53.945190+0000","last_undegraded":"2024-09-10T21:12:53.945190+0000","last_fullsized":"2024-09-10T21:12:53.945190+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:00.887008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.922965+0000","last_change":"2024-09-10T21:12:53.920537+0000","last_active":"2024-09-10T21:12:53.922965+0000","last_peered":"2024-09-10T21:12:53.922965+0000","last_clean":"2024-09-10T21:12:53.922965+0000","last_became_active":"2024-09-10T21:12:53.920252+0000","last_became_peered":"2024-09-10T21:12:53.920252+0000","last_unstale":"2024-09-10T21:12:53.922965+0000","last_undegraded":"2024-09-10T21:12:53.922965+0000","last_fullsized":"2024-09-10T21:12:53.922965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:51.239584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.905107+0000","last_change":"2024-09-10T21:12:52.905107+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.905107+0000","last_undegraded":"2024-09-10T21:12:52.905107+0000","last_fullsized":"2024-09-10T21:12:52.905107+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.909354+0000","last_change":"2024-09-10T21:12:52.909354+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.909354+0000","last_undegraded":"2024-09-10T21:12:52.909354+0000","last_fullsized":"2024-09-10T21:12:52.909354+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929245+0000","last_change":"2024-09-10T21:12:53.923199+0000","last_active":"2024-09-10T21:12:53.929245+0000","last_peered":"2024-09-10T21:12:53.929245+0000","last_clean":"2024-09-10T21:12:53.929245+0000","last_became_active":"2024-09-10T21:12:53.922779+0000","last_became_peered":"2024-09-10T21:12:53.922779+0000","last_unstale":"2024-09-10T21:12:53.929245+0000","last_undegraded":"2024-09-10T21:12:53.929245+0000","last_fullsized":"2024-09-10T21:12:53.929245+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:20.410692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.914654+0000","last_change":"2024-09-10T21:12:52.914654+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.914654+0000","last_undegraded":"2024-09-10T21:12:52.914654+0000","last_fullsized":"2024-09-10T21:12:52.914654+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951229+0000","last_change":"2024-09-10T21:12:53.939591+0000","last_active":"2024-09-10T21:12:53.951229+0000","last_peered":"2024-09-10T21:12:53.951229+0000","last_clean":"2024-09-10T21:12:53.951229+0000","last_became_active":"2024-09-10T21:12:53.939314+0000","last_became_peered":"2024-09-10T21:12:53.939314+0000","last_unstale":"2024-09-10T21:12:53.951229+0000","last_undegraded":"2024-09-10T21:12:53.951229+0000","last_fullsized":"2024-09-10T21:12:53.951229+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:02.044073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.946748+0000","last_change":"2024-09-10T21:12:53.937058+0000","last_active":"2024-09-10T21:12:53.946748+0000","last_peered":"2024-09-10T21:12:53.946748+0000","last_clean":"2024-09-10T21:12:53.946748+0000","last_became_active":"2024-09-10T21:12:53.936798+0000","last_became_peered":"2024-09-10T21:12:53.936798+0000","last_unstale":"2024-09-10T21:12:53.946748+0000","last_undegraded":"2024-09-10T21:12:53.946748+0000","last_fullsized":"2024-09-10T21:12:53.946748+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:53.754974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929094+0000","last_change":"2024-09-10T21:12:53.925270+0000","last_active":"2024-09-10T21:12:53.929094+0000","last_peered":"2024-09-10T21:12:53.929094+0000","last_clean":"2024-09-10T21:12:53.929094+0000","last_became_active":"2024-09-10T21:12:53.923493+0000","last_became_peered":"2024-09-10T21:12:53.923493+0000","last_unstale":"2024-09-10T21:12:53.929094+0000","last_undegraded":"2024-09-10T21:12:53.929094+0000","last_fullsized":"2024-09-10T21:12:53.929094+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:01.637614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.914057+0000","last_change":"2024-09-10T21:12:52.914057+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.914057+0000","last_undegraded":"2024-09-10T21:12:52.914057+0000","last_fullsized":"2024-09-10T21:12:52.914057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.945266+0000","last_change":"2024-09-10T21:12:53.939129+0000","last_active":"2024-09-10T21:12:53.945266+0000","last_peered":"2024-09-10T21:12:53.945266+0000","last_clean":"2024-09-10T21:12:53.945266+0000","last_became_active":"2024-09-10T21:12:53.938807+0000","last_became_peered":"2024-09-10T21:12:53.938807+0000","last_unstale":"2024-09-10T21:12:53.945266+0000","last_undegraded":"2024-09-10T21:12:53.945266+0000","last_fullsized":"2024-09-10T21:12:53.945266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:30.631397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.956016+0000","last_change":"2024-09-10T21:12:53.938689+0000","last_active":"2024-09-10T21:12:53.956016+0000","last_peered":"2024-09-10T21:12:53.956016+0000","last_clean":"2024-09-10T21:12:53.956016+0000","last_became_active":"2024-09-10T21:12:53.935691+0000","last_became_peered":"2024-09-10T21:12:53.935691+0000","last_unstale":"2024-09-10T21:12:53.956016+0000","last_undegraded":"2024-09-10T21:12:53.956016+0000","last_fullsized":"2024-09-10T21:12:53.956016+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:10.944201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":17,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.957007+0000","last_change":"2024-09-10T21:12:52.916629+0000","last_active":"2024-09-10T21:12:53.957007+0000","last_peered":"2024-09-10T21:12:53.957007+0000","last_clean":"2024-09-10T21:12:53.957007+0000","last_became_active":"2024-09-10T21:12:52.916091+0000","last_became_peered":"2024-09-10T21:12:52.916091+0000","last_unstale":"2024-09-10T21:12:53.957007+0000","last_undegraded":"2024-09-10T21:12:53.957007+0000","last_fullsized":"2024-09-10T21:12:53.957007+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:27.028802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.959594+0000","last_change":"2024-09-10T21:12:53.953827+0000","last_active":"2024-09-10T21:12:53.959594+0000","last_peered":"2024-09-10T21:12:53.959594+0000","last_clean":"2024-09-10T21:12:53.959594+0000","last_became_active":"2024-09-10T21:12:53.953262+0000","last_became_peered":"2024-09-10T21:12:53.953262+0000","last_unstale":"2024-09-10T21:12:53.959594+0000","last_undegraded":"2024-09-10T21:12:53.959594+0000","last_fullsized":"2024-09-10T21:12:53.959594+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:20.378633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.928971+0000","last_change":"2024-09-10T21:12:52.926961+0000","last_active":"2024-09-10T21:12:52.928971+0000","last_peered":"2024-09-10T21:12:52.928971+0000","last_clean":"2024-09-10T21:12:52.928971+0000","last_became_active":"2024-09-10T21:12:52.926723+0000","last_became_peered":"2024-09-10T21:12:52.926723+0000","last_unstale":"2024-09-10T21:12:52.928971+0000","last_undegraded":"2024-09-10T21:12:52.928971+0000","last_fullsized":"2024-09-10T21:12:52.928971+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:40.512719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951499+0000","last_change":"2024-09-10T21:12:53.944452+0000","last_active":"2024-09-10T21:12:53.951499+0000","last_peered":"2024-09-10T21:12:53.951499+0000","last_clean":"2024-09-10T21:12:53.951499+0000","last_became_active":"2024-09-10T21:12:53.944142+0000","last_became_peered":"2024-09-10T21:12:53.944142+0000","last_unstale":"2024-09-10T21:12:53.951499+0000","last_undegraded":"2024-09-10T21:12:53.951499+0000","last_fullsized":"2024-09-10T21:12:53.951499+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.507264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.963349+0000","last_change":"2024-09-10T21:12:52.924953+0000","last_active":"2024-09-10T21:12:53.963349+0000","last_peered":"2024-09-10T21:12:53.963349+0000","last_clean":"2024-09-10T21:12:53.963349+0000","last_became_active":"2024-09-10T21:12:52.924601+0000","last_became_peered":"2024-09-10T21:12:52.924601+0000","last_unstale":"2024-09-10T21:12:53.963349+0000","last_undegraded":"2024-09-10T21:12:53.963349+0000","last_fullsized":"2024-09-10T21:12:53.963349+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:44.842888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958253+0000","last_change":"2024-09-10T21:12:53.950957+0000","last_active":"2024-09-10T21:12:53.958253+0000","last_peered":"2024-09-10T21:12:53.958253+0000","last_clean":"2024-09-10T21:12:53.958253+0000","last_became_active":"2024-09-10T21:12:53.950673+0000","last_became_peered":"2024-09-10T21:12:53.950673+0000","last_unstale":"2024-09-10T21:12:53.958253+0000","last_undegraded":"2024-09-10T21:12:53.958253+0000","last_fullsized":"2024-09-10T21:12:53.958253+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:16.326966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.954185+0000","last_change":"2024-09-10T21:12:52.920769+0000","last_active":"2024-09-10T21:12:53.954185+0000","last_peered":"2024-09-10T21:12:53.954185+0000","last_clean":"2024-09-10T21:12:53.954185+0000","last_became_active":"2024-09-10T21:12:52.920460+0000","last_became_peered":"2024-09-10T21:12:52.920460+0000","last_unstale":"2024-09-10T21:12:53.954185+0000","last_undegraded":"2024-09-10T21:12:53.954185+0000","last_fullsized":"2024-09-10T21:12:53.954185+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:53.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951781+0000","last_change":"2024-09-10T21:12:53.946296+0000","last_active":"2024-09-10T21:12:53.951781+0000","last_peered":"2024-09-10T21:12:53.951781+0000","last_clean":"2024-09-10T21:12:53.951781+0000","last_became_active":"2024-09-10T21:12:53.945868+0000","last_became_peered":"2024-09-10T21:12:53.945868+0000","last_unstale":"2024-09-10T21:12:53.951781+0000","last_undegraded":"2024-09-10T21:12:53.951781+0000","last_fullsized":"2024-09-10T21:12:53.951781+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:56.605281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.955272+0000","last_change":"2024-09-10T21:12:52.917429+0000","last_active":"2024-09-10T21:12:53.955272+0000","last_peered":"2024-09-10T21:12:53.955272+0000","last_clean":"2024-09-10T21:12:53.955272+0000","last_became_active":"2024-09-10T21:12:52.917148+0000","last_became_peered":"2024-09-10T21:12:52.917148+0000","last_unstale":"2024-09-10T21:12:53.955272+0000","last_undegraded":"2024-09-10T21:12:53.955272+0000","last_fullsized":"2024-09-10T21:12:53.955272+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:05.034728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.950682+0000","last_change":"2024-09-10T21:12:53.947773+0000","last_active":"2024-09-10T21:12:53.950682+0000","last_peered":"2024-09-10T21:12:53.950682+0000","last_clean":"2024-09-10T21:12:53.950682+0000","last_became_active":"2024-09-10T21:12:53.947473+0000","last_became_peered":"2024-09-10T21:12:53.947473+0000","last_unstale":"2024-09-10T21:12:53.950682+0000","last_undegraded":"2024-09-10T21:12:53.950682+0000","last_fullsized":"2024-09-10T21:12:53.950682+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:45.835056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.961946+0000","last_change":"2024-09-10T21:12:52.907684+0000","last_active":"2024-09-10T21:12:53.961946+0000","last_peered":"2024-09-10T21:12:53.961946+0000","last_clean":"2024-09-10T21:12:53.961946+0000","last_became_active":"2024-09-10T21:12:52.907395+0000","last_became_peered":"2024-09-10T21:12:52.907395+0000","last_unstale":"2024-09-10T21:12:53.961946+0000","last_undegraded":"2024-09-10T21:12:53.961946+0000","last_fullsized":"2024-09-10T21:12:53.961946+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:19.885587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.936138+0000","last_change":"2024-09-10T21:12:53.934723+0000","last_active":"2024-09-10T21:12:53.936138+0000","last_peered":"2024-09-10T21:12:53.936138+0000","last_clean":"2024-09-10T21:12:53.936138+0000","last_became_active":"2024-09-10T21:12:53.932327+0000","last_became_peered":"2024-09-10T21:12:53.932327+0000","last_unstale":"2024-09-10T21:12:53.936138+0000","last_undegraded":"2024-09-10T21:12:53.936138+0000","last_fullsized":"2024-09-10T21:12:53.936138+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:43.720965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.925403+0000","last_change":"2024-09-10T21:12:52.922523+0000","last_active":"2024-09-10T21:12:52.925403+0000","last_peered":"2024-09-10T21:12:52.925403+0000","last_clean":"2024-09-10T21:12:52.925403+0000","last_became_active":"2024-09-10T21:12:52.922266+0000","last_became_peered":"2024-09-10T21:12:52.922266+0000","last_unstale":"2024-09-10T21:12:52.925403+0000","last_undegraded":"2024-09-10T21:12:52.925403+0000","last_fullsized":"2024-09-10T21:12:52.925403+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:16.245436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.959105+0000","last_change":"2024-09-10T21:12:53.951739+0000","last_active":"2024-09-10T21:12:53.959105+0000","last_peered":"2024-09-10T21:12:53.959105+0000","last_clean":"2024-09-10T21:12:53.959105+0000","last_became_active":"2024-09-10T21:12:53.943111+0000","last_became_peered":"2024-09-10T21:12:53.943111+0000","last_unstale":"2024-09-10T21:12:53.959105+0000","last_undegraded":"2024-09-10T21:12:53.959105+0000","last_fullsized":"2024-09-10T21:12:53.959105+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:02.237320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.916370+0000","last_change":"2024-09-10T21:12:52.925890+0000","last_active":"2024-09-10T21:12:53.916370+0000","last_peered":"2024-09-10T21:12:53.916370+0000","last_clean":"2024-09-10T21:12:53.916370+0000","last_became_active":"2024-09-10T21:12:52.925061+0000","last_became_peered":"2024-09-10T21:12:52.925061+0000","last_unstale":"2024-09-10T21:12:53.916370+0000","last_undegraded":"2024-09-10T21:12:53.916370+0000","last_fullsized":"2024-09-10T21:12:53.916370+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:53.791545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.953214+0000","last_change":"2024-09-10T21:12:53.947582+0000","last_active":"2024-09-10T21:12:53.953214+0000","last_peered":"2024-09-10T21:12:53.953214+0000","last_clean":"2024-09-10T21:12:53.953214+0000","last_became_active":"2024-09-10T21:12:53.947316+0000","last_became_peered":"2024-09-10T21:12:53.947316+0000","last_unstale":"2024-09-10T21:12:53.953214+0000","last_undegraded":"2024-09-10T21:12:53.953214+0000","last_fullsized":"2024-09-10T21:12:53.953214+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:42.777708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.922253+0000","last_change":"2024-09-10T21:12:52.920456+0000","last_active":"2024-09-10T21:12:52.922253+0000","last_peered":"2024-09-10T21:12:52.922253+0000","last_clean":"2024-09-10T21:12:52.922253+0000","last_became_active":"2024-09-10T21:12:52.920172+0000","last_became_peered":"2024-09-10T21:12:52.920172+0000","last_unstale":"2024-09-10T21:12:52.922253+0000","last_undegraded":"2024-09-10T21:12:52.922253+0000","last_fullsized":"2024-09-10T21:12:52.922253+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:09.727174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951060+0000","last_change":"2024-09-10T21:12:53.945665+0000","last_active":"2024-09-10T21:12:53.951060+0000","last_peered":"2024-09-10T21:12:53.951060+0000","last_clean":"2024-09-10T21:12:53.951060+0000","last_became_active":"2024-09-10T21:12:53.945387+0000","last_became_peered":"2024-09-10T21:12:53.945387+0000","last_unstale":"2024-09-10T21:12:53.951060+0000","last_undegraded":"2024-09-10T21:12:53.951060+0000","last_fullsized":"2024-09-10T21:12:53.951060+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:48.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.925467+0000","last_change":"2024-09-10T21:12:52.921420+0000","last_active":"2024-09-10T21:12:53.925467+0000","last_peered":"2024-09-10T21:12:53.925467+0000","last_clean":"2024-09-10T21:12:53.925467+0000","last_became_active":"2024-09-10T21:12:52.920194+0000","last_became_peered":"2024-09-10T21:12:52.920194+0000","last_unstale":"2024-09-10T21:12:53.925467+0000","last_undegraded":"2024-09-10T21:12:53.925467+0000","last_fullsized":"2024-09-10T21:12:53.925467+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:02.042065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.961558+0000","last_change":"2024-09-10T21:12:53.959498+0000","last_active":"2024-09-10T21:12:53.961558+0000","last_peered":"2024-09-10T21:12:53.961558+0000","last_clean":"2024-09-10T21:12:53.961558+0000","last_became_active":"2024-09-10T21:12:53.959157+0000","last_became_peered":"2024-09-10T21:12:53.959157+0000","last_unstale":"2024-09-10T21:12:53.961558+0000","last_undegraded":"2024-09-10T21:12:53.961558+0000","last_fullsized":"2024-09-10T21:12:53.961558+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:06.297558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.954331+0000","last_change":"2024-09-10T21:12:52.920836+0000","last_active":"2024-09-10T21:12:53.954331+0000","last_peered":"2024-09-10T21:12:53.954331+0000","last_clean":"2024-09-10T21:12:53.954331+0000","last_became_active":"2024-09-10T21:12:52.920547+0000","last_became_peered":"2024-09-10T21:12:52.920547+0000","last_unstale":"2024-09-10T21:12:53.954331+0000","last_undegraded":"2024-09-10T21:12:53.954331+0000","last_fullsized":"2024-09-10T21:12:53.954331+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:22.866900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.937227+0000","last_change":"2024-09-10T21:12:53.936019+0000","last_active":"2024-09-10T21:12:53.937227+0000","last_peered":"2024-09-10T21:12:53.937227+0000","last_clean":"2024-09-10T21:12:53.937227+0000","last_became_active":"2024-09-10T21:12:53.935728+0000","last_became_peered":"2024-09-10T21:12:53.935728+0000","last_unstale":"2024-09-10T21:12:53.937227+0000","last_undegraded":"2024-09-10T21:12:53.937227+0000","last_fullsized":"2024-09-10T21:12:53.937227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:28.139924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":19,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.957568+0000","last_change":"2024-09-10T21:12:52.924581+0000","last_active":"2024-09-10T21:12:53.957568+0000","last_peered":"2024-09-10T21:12:53.957568+0000","last_clean":"2024-09-10T21:12:53.957568+0000","last_became_active":"2024-09-10T21:12:52.924327+0000","last_became_peered":"2024-09-10T21:12:52.924327+0000","last_unstale":"2024-09-10T21:12:53.957568+0000","last_undegraded":"2024-09-10T21:12:53.957568+0000","last_fullsized":"2024-09-10T21:12:53.957568+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:10.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.905486+0000","last_change":"2024-09-10T21:12:52.905486+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.905486+0000","last_undegraded":"2024-09-10T21:12:52.905486+0000","last_fullsized":"2024-09-10T21:12:52.905486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.956965+0000","last_change":"2024-09-10T21:12:52.919371+0000","last_active":"2024-09-10T21:12:53.956965+0000","last_peered":"2024-09-10T21:12:53.956965+0000","last_clean":"2024-09-10T21:12:53.956965+0000","last_became_active":"2024-09-10T21:12:52.918995+0000","last_became_peered":"2024-09-10T21:12:52.918995+0000","last_unstale":"2024-09-10T21:12:53.956965+0000","last_undegraded":"2024-09-10T21:12:53.956965+0000","last_fullsized":"2024-09-10T21:12:53.956965+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:33.805170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.932517+0000","last_change":"2024-09-10T21:12:53.930651+0000","last_active":"2024-09-10T21:12:53.932517+0000","last_peered":"2024-09-10T21:12:53.932517+0000","last_clean":"2024-09-10T21:12:53.932517+0000","last_became_active":"2024-09-10T21:12:53.930387+0000","last_became_peered":"2024-09-10T21:12:53.930387+0000","last_unstale":"2024-09-10T21:12:53.932517+0000","last_undegraded":"2024-09-10T21:12:53.932517+0000","last_fullsized":"2024-09-10T21:12:53.932517+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:42.777708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.913794+0000","last_change":"2024-09-10T21:12:52.923255+0000","last_active":"2024-09-10T21:12:53.913794+0000","last_peered":"2024-09-10T21:12:53.913794+0000","last_clean":"2024-09-10T21:12:53.913794+0000","last_became_active":"2024-09-10T21:12:52.919965+0000","last_became_peered":"2024-09-10T21:12:52.919965+0000","last_unstale":"2024-09-10T21:12:53.913794+0000","last_undegraded":"2024-09-10T21:12:53.913794+0000","last_fullsized":"2024-09-10T21:12:53.913794+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:15.274251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.904539+0000","last_change":"2024-09-10T21:12:52.904539+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.904539+0000","last_undegraded":"2024-09-10T21:12:52.904539+0000","last_fullsized":"2024-09-10T21:12:52.904539+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.914460+0000","last_change":"2024-09-10T21:12:52.923218+0000","last_active":"2024-09-10T21:12:53.914460+0000","last_peered":"2024-09-10T21:12:53.914460+0000","last_clean":"2024-09-10T21:12:53.914460+0000","last_became_active":"2024-09-10T21:12:52.919955+0000","last_became_peered":"2024-09-10T21:12:52.919955+0000","last_unstale":"2024-09-10T21:12:53.914460+0000","last_undegraded":"2024-09-10T21:12:53.914460+0000","last_fullsized":"2024-09-10T21:12:53.914460+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:53.792649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.925605+0000","last_change":"2024-09-10T21:12:53.922279+0000","last_active":"2024-09-10T21:12:53.925605+0000","last_peered":"2024-09-10T21:12:53.925605+0000","last_clean":"2024-09-10T21:12:53.925605+0000","last_became_active":"2024-09-10T21:12:53.921900+0000","last_became_peered":"2024-09-10T21:12:53.921900+0000","last_unstale":"2024-09-10T21:12:53.925605+0000","last_undegraded":"2024-09-10T21:12:53.925605+0000","last_fullsized":"2024-09-10T21:12:53.925605+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:43.504550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":21,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.954508+0000","last_change":"2024-09-10T21:12:52.920975+0000","last_active":"2024-09-10T21:12:53.954508+0000","last_peered":"2024-09-10T21:12:53.954508+0000","last_clean":"2024-09-10T21:12:53.954508+0000","last_became_active":"2024-09-10T21:12:52.919980+0000","last_became_peered":"2024-09-10T21:12:52.919980+0000","last_unstale":"2024-09-10T21:12:53.954508+0000","last_undegraded":"2024-09-10T21:12:53.954508+0000","last_fullsized":"2024-09-10T21:12:53.954508+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:28.166375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.952338+0000","last_change":"2024-09-10T21:12:53.949707+0000","last_active":"2024-09-10T21:12:53.952338+0000","last_peered":"2024-09-10T21:12:53.952338+0000","last_clean":"2024-09-10T21:12:53.952338+0000","last_became_active":"2024-09-10T21:12:53.949438+0000","last_became_peered":"2024-09-10T21:12:53.949438+0000","last_unstale":"2024-09-10T21:12:53.952338+0000","last_undegraded":"2024-09-10T21:12:53.952338+0000","last_fullsized":"2024-09-10T21:12:53.952338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:31.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.929524+0000","last_change":"2024-09-10T21:12:52.928329+0000","last_active":"2024-09-10T21:12:52.929524+0000","last_peered":"2024-09-10T21:12:52.929524+0000","last_clean":"2024-09-10T21:12:52.929524+0000","last_became_active":"2024-09-10T21:12:52.928026+0000","last_became_peered":"2024-09-10T21:12:52.928026+0000","last_unstale":"2024-09-10T21:12:52.929524+0000","last_undegraded":"2024-09-10T21:12:52.929524+0000","last_fullsized":"2024-09-10T21:12:52.929524+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:55.890316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.952364+0000","last_change":"2024-09-10T21:12:53.946213+0000","last_active":"2024-09-10T21:12:53.952364+0000","last_peered":"2024-09-10T21:12:53.952364+0000","last_clean":"2024-09-10T21:12:53.952364+0000","last_became_active":"2024-09-10T21:12:53.945887+0000","last_became_peered":"2024-09-10T21:12:53.945887+0000","last_unstale":"2024-09-10T21:12:53.952364+0000","last_undegraded":"2024-09-10T21:12:53.952364+0000","last_fullsized":"2024-09-10T21:12:53.952364+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:28.847293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958052+0000","last_change":"2024-09-10T21:12:52.920779+0000","last_active":"2024-09-10T21:12:53.958052+0000","last_peered":"2024-09-10T21:12:53.958052+0000","last_clean":"2024-09-10T21:12:53.958052+0000","last_became_active":"2024-09-10T21:12:52.920513+0000","last_became_peered":"2024-09-10T21:12:52.920513+0000","last_unstale":"2024-09-10T21:12:53.958052+0000","last_undegraded":"2024-09-10T21:12:53.958052+0000","last_fullsized":"2024-09-10T21:12:53.958052+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:18.447950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.910688+0000","last_change":"2024-09-10T21:12:52.910688+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.910688+0000","last_undegraded":"2024-09-10T21:12:52.910688+0000","last_fullsized":"2024-09-10T21:12:52.910688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.953684+0000","last_change":"2024-09-10T21:12:52.925824+0000","last_active":"2024-09-10T21:12:53.953684+0000","last_peered":"2024-09-10T21:12:53.953684+0000","last_clean":"2024-09-10T21:12:53.953684+0000","last_became_active":"2024-09-10T21:12:52.925229+0000","last_became_peered":"2024-09-10T21:12:52.925229+0000","last_unstale":"2024-09-10T21:12:53.953684+0000","last_undegraded":"2024-09-10T21:12:53.953684+0000","last_fullsized":"2024-09-10T21:12:53.953684+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:10.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.934630+0000","last_change":"2024-09-10T21:12:53.930750+0000","last_active":"2024-09-10T21:12:53.934630+0000","last_peered":"2024-09-10T21:12:53.934630+0000","last_clean":"2024-09-10T21:12:53.934630+0000","last_became_active":"2024-09-10T21:12:53.930443+0000","last_became_peered":"2024-09-10T21:12:53.930443+0000","last_unstale":"2024-09-10T21:12:53.934630+0000","last_undegraded":"2024-09-10T21:12:53.934630+0000","last_fullsized":"2024-09-10T21:12:53.934630+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:02.992141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929696+0000","last_change":"2024-09-10T21:12:52.913454+0000","last_active":"2024-09-10T21:12:53.929696+0000","last_peered":"2024-09-10T21:12:53.929696+0000","last_clean":"2024-09-10T21:12:53.929696+0000","last_became_active":"2024-09-10T21:12:52.913160+0000","last_became_peered":"2024-09-10T21:12:52.913160+0000","last_unstale":"2024-09-10T21:12:53.929696+0000","last_undegraded":"2024-09-10T21:12:53.929696+0000","last_fullsized":"2024-09-10T21:12:53.929696+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:35.717760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.904827+0000","last_change":"2024-09-10T21:12:52.904827+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.904827+0000","last_undegraded":"2024-09-10T21:12:52.904827+0000","last_fullsized":"2024-09-10T21:12:52.904827+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.955822+0000","last_change":"2024-09-10T21:12:52.919813+0000","last_active":"2024-09-10T21:12:53.955822+0000","last_peered":"2024-09-10T21:12:53.955822+0000","last_clean":"2024-09-10T21:12:53.955822+0000","last_became_active":"2024-09-10T21:12:52.919373+0000","last_became_peered":"2024-09-10T21:12:52.919373+0000","last_unstale":"2024-09-10T21:12:53.955822+0000","last_undegraded":"2024-09-10T21:12:53.955822+0000","last_fullsized":"2024-09-10T21:12:53.955822+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:25.283722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.946775+0000","last_change":"2024-09-10T21:12:53.937502+0000","last_active":"2024-09-10T21:12:53.946775+0000","last_peered":"2024-09-10T21:12:53.946775+0000","last_clean":"2024-09-10T21:12:53.946775+0000","last_became_active":"2024-09-10T21:12:53.937201+0000","last_became_peered":"2024-09-10T21:12:53.937201+0000","last_unstale":"2024-09-10T21:12:53.946775+0000","last_undegraded":"2024-09-10T21:12:53.946775+0000","last_fullsized":"2024-09-10T21:12:53.946775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:33.689313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.926985+0000","last_change":"2024-09-10T21:12:52.920031+0000","last_active":"2024-09-10T21:12:53.926985+0000","last_peered":"2024-09-10T21:12:53.926985+0000","last_clean":"2024-09-10T21:12:53.926985+0000","last_became_active":"2024-09-10T21:12:52.919769+0000","last_became_peered":"2024-09-10T21:12:52.919769+0000","last_unstale":"2024-09-10T21:12:53.926985+0000","last_undegraded":"2024-09-10T21:12:53.926985+0000","last_fullsized":"2024-09-10T21:12:53.926985+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:08.609135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.940861+0000","last_change":"2024-09-10T21:12:53.928673+0000","last_active":"2024-09-10T21:12:53.940861+0000","last_peered":"2024-09-10T21:12:53.940861+0000","last_clean":"2024-09-10T21:12:53.940861+0000","last_became_active":"2024-09-10T21:12:53.928390+0000","last_became_peered":"2024-09-10T21:12:53.928390+0000","last_unstale":"2024-09-10T21:12:53.940861+0000","last_undegraded":"2024-09-10T21:12:53.940861+0000","last_fullsized":"2024-09-10T21:12:53.940861+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:51.847731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.916361+0000","last_change":"2024-09-10T21:12:52.910741+0000","last_active":"2024-09-10T21:12:52.916361+0000","last_peered":"2024-09-10T21:12:52.916361+0000","last_clean":"2024-09-10T21:12:52.916361+0000","last_became_active":"2024-09-10T21:12:52.910465+0000","last_became_peered":"2024-09-10T21:12:52.910465+0000","last_unstale":"2024-09-10T21:12:52.916361+0000","last_undegraded":"2024-09-10T21:12:52.916361+0000","last_fullsized":"2024-09-10T21:12:52.916361+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:06.500073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.929686+0000","last_change":"2024-09-10T21:12:53.926389+0000","last_active":"2024-09-10T21:12:53.929686+0000","last_peered":"2024-09-10T21:12:53.929686+0000","last_clean":"2024-09-10T21:12:53.929686+0000","last_became_active":"2024-09-10T21:12:53.926136+0000","last_became_peered":"2024-09-10T21:12:53.926136+0000","last_unstale":"2024-09-10T21:12:53.929686+0000","last_undegraded":"2024-09-10T21:12:53.929686+0000","last_fullsized":"2024-09-10T21:12:53.929686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:25.798580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.929496+0000","last_change":"2024-09-10T21:12:52.927712+0000","last_active":"2024-09-10T21:12:52.929496+0000","last_peered":"2024-09-10T21:12:52.929496+0000","last_clean":"2024-09-10T21:12:52.929496+0000","last_became_active":"2024-09-10T21:12:52.927386+0000","last_became_peered":"2024-09-10T21:12:52.927386+0000","last_unstale":"2024-09-10T21:12:52.929496+0000","last_undegraded":"2024-09-10T21:12:52.929496+0000","last_fullsized":"2024-09-10T21:12:52.929496+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:45.588719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.931285+0000","last_change":"2024-09-10T21:12:53.925268+0000","last_active":"2024-09-10T21:12:53.931285+0000","last_peered":"2024-09-10T21:12:53.931285+0000","last_clean":"2024-09-10T21:12:53.931285+0000","last_became_active":"2024-09-10T21:12:53.924980+0000","last_became_peered":"2024-09-10T21:12:53.924980+0000","last_unstale":"2024-09-10T21:12:53.931285+0000","last_undegraded":"2024-09-10T21:12:53.931285+0000","last_fullsized":"2024-09-10T21:12:53.931285+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:44.802123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.943098+0000","last_change":"2024-09-10T21:12:52.908936+0000","last_active":"2024-09-10T21:12:53.943098+0000","last_peered":"2024-09-10T21:12:53.943098+0000","last_clean":"2024-09-10T21:12:53.943098+0000","last_became_active":"2024-09-10T21:12:52.908661+0000","last_became_peered":"2024-09-10T21:12:52.908661+0000","last_unstale":"2024-09-10T21:12:53.943098+0000","last_undegraded":"2024-09-10T21:12:53.943098+0000","last_fullsized":"2024-09-10T21:12:53.943098+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:07.860598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.907015+0000","last_change":"2024-09-10T21:12:52.907015+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.907015+0000","last_undegraded":"2024-09-10T21:12:52.907015+0000","last_fullsized":"2024-09-10T21:12:52.907015+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.916877+0000","last_change":"2024-09-10T21:12:52.911674+0000","last_active":"2024-09-10T21:12:52.916877+0000","last_peered":"2024-09-10T21:12:52.916877+0000","last_clean":"2024-09-10T21:12:52.916877+0000","last_became_active":"2024-09-10T21:12:52.911271+0000","last_became_peered":"2024-09-10T21:12:52.911271+0000","last_unstale":"2024-09-10T21:12:52.916877+0000","last_undegraded":"2024-09-10T21:12:52.916877+0000","last_fullsized":"2024-09-10T21:12:52.916877+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:46.241035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.940925+0000","last_change":"2024-09-10T21:12:53.933870+0000","last_active":"2024-09-10T21:12:53.940925+0000","last_peered":"2024-09-10T21:12:53.940925+0000","last_clean":"2024-09-10T21:12:53.940925+0000","last_became_active":"2024-09-10T21:12:53.933440+0000","last_became_peered":"2024-09-10T21:12:53.933440+0000","last_unstale":"2024-09-10T21:12:53.940925+0000","last_undegraded":"2024-09-10T21:12:53.940925+0000","last_fullsized":"2024-09-10T21:12:53.940925+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:25.088706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.917741+0000","last_change":"2024-09-10T21:12:52.920768+0000","last_active":"2024-09-10T21:12:53.917741+0000","last_peered":"2024-09-10T21:12:53.917741+0000","last_clean":"2024-09-10T21:12:53.917741+0000","last_became_active":"2024-09-10T21:12:52.918131+0000","last_became_peered":"2024-09-10T21:12:52.918131+0000","last_unstale":"2024-09-10T21:12:53.917741+0000","last_undegraded":"2024-09-10T21:12:53.917741+0000","last_fullsized":"2024-09-10T21:12:53.917741+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:48.097204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.958785+0000","last_change":"2024-09-10T21:12:53.949638+0000","last_active":"2024-09-10T21:12:53.958785+0000","last_peered":"2024-09-10T21:12:53.958785+0000","last_clean":"2024-09-10T21:12:53.958785+0000","last_became_active":"2024-09-10T21:12:53.949177+0000","last_became_peered":"2024-09-10T21:12:53.949177+0000","last_unstale":"2024-09-10T21:12:53.958785+0000","last_undegraded":"2024-09-10T21:12:53.958785+0000","last_fullsized":"2024-09-10T21:12:53.958785+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:38.456803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.926904+0000","last_change":"2024-09-10T21:12:52.925485+0000","last_active":"2024-09-10T21:12:52.926904+0000","last_peered":"2024-09-10T21:12:52.926904+0000","last_clean":"2024-09-10T21:12:52.926904+0000","last_became_active":"2024-09-10T21:12:52.925196+0000","last_became_peered":"2024-09-10T21:12:52.925196+0000","last_unstale":"2024-09-10T21:12:52.926904+0000","last_undegraded":"2024-09-10T21:12:52.926904+0000","last_fullsized":"2024-09-10T21:12:52.926904+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:26.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.952972+0000","last_change":"2024-09-10T21:12:53.946489+0000","last_active":"2024-09-10T21:12:53.952972+0000","last_peered":"2024-09-10T21:12:53.952972+0000","last_clean":"2024-09-10T21:12:53.952972+0000","last_became_active":"2024-09-10T21:12:53.945218+0000","last_became_peered":"2024-09-10T21:12:53.945218+0000","last_unstale":"2024-09-10T21:12:53.952972+0000","last_undegraded":"2024-09-10T21:12:53.952972+0000","last_fullsized":"2024-09-10T21:12:53.952972+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:17.409325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.917083+0000","last_change":"2024-09-10T21:12:52.926307+0000","last_active":"2024-09-10T21:12:53.917083+0000","last_peered":"2024-09-10T21:12:53.917083+0000","last_clean":"2024-09-10T21:12:53.917083+0000","last_became_active":"2024-09-10T21:12:52.926021+0000","last_became_peered":"2024-09-10T21:12:52.926021+0000","last_unstale":"2024-09-10T21:12:53.917083+0000","last_undegraded":"2024-09-10T21:12:53.917083+0000","last_fullsized":"2024-09-10T21:12:53.917083+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:53.791545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.951502+0000","last_change":"2024-09-10T21:12:53.944529+0000","last_active":"2024-09-10T21:12:53.951502+0000","last_peered":"2024-09-10T21:12:53.951502+0000","last_clean":"2024-09-10T21:12:53.951502+0000","last_became_active":"2024-09-10T21:12:53.944191+0000","last_became_peered":"2024-09-10T21:12:53.944191+0000","last_unstale":"2024-09-10T21:12:53.951502+0000","last_undegraded":"2024-09-10T21:12:53.951502+0000","last_fullsized":"2024-09-10T21:12:53.951502+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:18.729522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.921710+0000","last_change":"2024-09-10T21:12:52.916297+0000","last_active":"2024-09-10T21:12:53.921710+0000","last_peered":"2024-09-10T21:12:53.921710+0000","last_clean":"2024-09-10T21:12:53.921710+0000","last_became_active":"2024-09-10T21:12:52.914668+0000","last_became_peered":"2024-09-10T21:12:52.914668+0000","last_unstale":"2024-09-10T21:12:53.921710+0000","last_undegraded":"2024-09-10T21:12:53.921710+0000","last_fullsized":"2024-09-10T21:12:53.921710+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:50.985945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-10T21:12:52.906196+0000","last_change":"2024-09-10T21:12:52.906196+0000","last_active":"2024-09-10T21:12:52.891384+0000","last_peered":"2024-09-10T21:12:52.891384+0000","last_clean":"2024-09-10T21:12:52.891384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:12:52.906196+0000","last_undegraded":"2024-09-10T21:12:52.906196+0000","last_fullsized":"2024-09-10T21:12:52.906196+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.927648+0000","last_change":"2024-09-10T21:12:52.914672+0000","last_active":"2024-09-10T21:12:53.927648+0000","last_peered":"2024-09-10T21:12:53.927648+0000","last_clean":"2024-09-10T21:12:53.927648+0000","last_became_active":"2024-09-10T21:12:52.914355+0000","last_became_peered":"2024-09-10T21:12:52.914355+0000","last_unstale":"2024-09-10T21:12:53.927648+0000","last_undegraded":"2024-09-10T21:12:53.927648+0000","last_fullsized":"2024-09-10T21:12:53.927648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:44.022375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.950021+0000","last_change":"2024-09-10T21:12:53.938346+0000","last_active":"2024-09-10T21:12:53.950021+0000","last_peered":"2024-09-10T21:12:53.950021+0000","last_clean":"2024-09-10T21:12:53.950021+0000","last_became_active":"2024-09-10T21:12:53.938178+0000","last_became_peered":"2024-09-10T21:12:53.938178+0000","last_unstale":"2024-09-10T21:12:53.950021+0000","last_undegraded":"2024-09-10T21:12:53.950021+0000","last_fullsized":"2024-09-10T21:12:53.950021+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:30.402157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T21:12:53.916982+0000","last_change":"2024-09-10T21:12:52.922364+0000","last_active":"2024-09-10T21:12:53.916982+0000","last_peered":"2024-09-10T21:12:53.916982+0000","last_clean":"2024-09-10T21:12:53.916982+0000","last_became_active":"2024-09-10T21:12:52.921965+0000","last_became_peered":"2024-09-10T21:12:52.921965+0000","last_unstale":"2024-09-10T21:12:53.916982+0000","last_undegraded":"2024-09-10T21:12:53.916982+0000","last_fullsized":"2024-09-10T21:12:53.916982+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:48.097204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":169,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T21:12:52.907968+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:12:52.907968+0000","last_peered":"2024-09-10T21:12:52.907968+0000","last_clean":"2024-09-10T21:12:52.907968+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:12:52.907968+0000","last_undegraded":"2024-09-10T21:12:52.907968+0000","last_fullsized":"2024-09-10T21:12:52.907968+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509474,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":667648,"data_stored":273322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1245184,"data_stored":856893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":651264,"data_stored":266525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509473,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":651264,"data_stored":266525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":651264,"data_stored":266525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509474,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":675840,"data_stored":274912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":638976,"data_stored":260250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1245184,"data_stored":856893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:12:58.892 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3e is not active+clean 2024-09-10T21:12:58.892 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.908750+0000', 'last_change': '2024-09-10T21:12:52.908750+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.908750+0000', 'last_undegraded': '2024-09-10T21:12:52.908750+0000', 'last_fullsized': '2024-09-10T21:12:52.908750+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:12:58.892 INFO:tasks.ceph.ceph_manager.ceph:PG 9.37 is not active+clean 2024-09-10T21:12:58.892 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.905254+0000', 'last_change': '2024-09-10T21:12:52.905254+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.905254+0000', 'last_undegraded': '2024-09-10T21:12:52.905254+0000', 'last_fullsized': '2024-09-10T21:12:52.905254+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:12:58.893 INFO:tasks.ceph.ceph_manager.ceph:PG 9.34 is not active+clean 2024-09-10T21:12:58.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.905908+0000', 'last_change': '2024-09-10T21:12:52.905908+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.905908+0000', 'last_undegraded': '2024-09-10T21:12:52.905908+0000', 'last_fullsized': '2024-09-10T21:12:52.905908+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:12:58.893 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2d is not active+clean 2024-09-10T21:12:58.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.917837+0000', 'last_change': '2024-09-10T21:12:52.917837+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.917837+0000', 'last_undegraded': '2024-09-10T21:12:52.917837+0000', 'last_fullsized': '2024-09-10T21:12:52.917837+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:12:58.893 INFO:tasks.ceph.ceph_manager.ceph:PG 9.28 is not active+clean 2024-09-10T21:12:58.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.905107+0000', 'last_change': '2024-09-10T21:12:52.905107+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.905107+0000', 'last_undegraded': '2024-09-10T21:12:52.905107+0000', 'last_fullsized': '2024-09-10T21:12:52.905107+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:12:58.893 INFO:tasks.ceph.ceph_manager.ceph:PG 9.27 is not active+clean 2024-09-10T21:12:58.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.909354+0000', 'last_change': '2024-09-10T21:12:52.909354+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.909354+0000', 'last_undegraded': '2024-09-10T21:12:52.909354+0000', 'last_fullsized': '2024-09-10T21:12:52.909354+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:12:58.894 INFO:tasks.ceph.ceph_manager.ceph:PG 9.25 is not active+clean 2024-09-10T21:12:58.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.914654+0000', 'last_change': '2024-09-10T21:12:52.914654+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.914654+0000', 'last_undegraded': '2024-09-10T21:12:52.914654+0000', 'last_fullsized': '2024-09-10T21:12:52.914654+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:12:58.894 INFO:tasks.ceph.ceph_manager.ceph:PG 9.21 is not active+clean 2024-09-10T21:12:58.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.914057+0000', 'last_change': '2024-09-10T21:12:52.914057+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.914057+0000', 'last_undegraded': '2024-09-10T21:12:52.914057+0000', 'last_fullsized': '2024-09-10T21:12:52.914057+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:12:58.894 INFO:tasks.ceph.ceph_manager.ceph:PG 9.12 is not active+clean 2024-09-10T21:12:58.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.905486+0000', 'last_change': '2024-09-10T21:12:52.905486+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.905486+0000', 'last_undegraded': '2024-09-10T21:12:52.905486+0000', 'last_fullsized': '2024-09-10T21:12:52.905486+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:12:58.894 INFO:tasks.ceph.ceph_manager.ceph:PG 9.10 is not active+clean 2024-09-10T21:12:58.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.904539+0000', 'last_change': '2024-09-10T21:12:52.904539+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.904539+0000', 'last_undegraded': '2024-09-10T21:12:52.904539+0000', 'last_fullsized': '2024-09-10T21:12:52.904539+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:12:58.895 INFO:tasks.ceph.ceph_manager.ceph:PG 9.c is not active+clean 2024-09-10T21:12:58.895 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.910688+0000', 'last_change': '2024-09-10T21:12:52.910688+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.910688+0000', 'last_undegraded': '2024-09-10T21:12:52.910688+0000', 'last_fullsized': '2024-09-10T21:12:52.910688+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:12:58.895 INFO:tasks.ceph.ceph_manager.ceph:PG 9.a is not active+clean 2024-09-10T21:12:58.895 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.904827+0000', 'last_change': '2024-09-10T21:12:52.904827+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.904827+0000', 'last_undegraded': '2024-09-10T21:12:52.904827+0000', 'last_fullsized': '2024-09-10T21:12:52.904827+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:12:58.895 INFO:tasks.ceph.ceph_manager.ceph:PG 9.4 is not active+clean 2024-09-10T21:12:58.896 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.907015+0000', 'last_change': '2024-09-10T21:12:52.907015+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.907015+0000', 'last_undegraded': '2024-09-10T21:12:52.907015+0000', 'last_fullsized': '2024-09-10T21:12:52.907015+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:12:58.896 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1 is not active+clean 2024-09-10T21:12:58.896 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:12:52.906196+0000', 'last_change': '2024-09-10T21:12:52.906196+0000', 'last_active': '2024-09-10T21:12:52.891384+0000', 'last_peered': '2024-09-10T21:12:52.891384+0000', 'last_clean': '2024-09-10T21:12:52.891384+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:12:52.906196+0000', 'last_undegraded': '2024-09-10T21:12:52.906196+0000', 'last_fullsized': '2024-09-10T21:12:52.906196+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'last_clean_scrub_stamp': '2024-09-10T21:12:52.891384+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:12:58.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T21:12:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 -- 172.21.15.16:0/231250840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049015f400 msgr2=0x7f0490070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 --2- 172.21.15.16:0/231250840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049015f400 0x7f0490070fe0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f04900cc360 tx=0x7f048001c780 comp rx=0 tx=0).stop 2024-09-10T21:12:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 -- 172.21.15.16:0/231250840 shutdown_connections 2024-09-10T21:12:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 --2- 172.21.15.16:0/231250840 >> v2:172.21.15.52:3300/0 conn(0x7f04900b0590 0x7f0490071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 --2- 172.21.15.16:0/231250840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049015f400 0x7f0490070fe0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 -- 172.21.15.16:0/231250840 >> 172.21.15.16:0/231250840 conn(0x7f049009b130 msgr2=0x7f04900d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 -- 172.21.15.16:0/231250840 shutdown_connections 2024-09-10T21:12:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 -- 172.21.15.16:0/231250840 wait complete. 2024-09-10T21:12:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 Processor -- start 2024-09-10T21:12:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 -- start start 2024-09-10T21:12:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04900b0590 0x7f04901574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f04900b13c0 0x7f0490164f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049015f400 0x7f04901654b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0490168f60 con 0x7f049015f400 2024-09-10T21:12:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0490168de0 con 0x7f04900b0590 2024-09-10T21:12:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.072+0000 7f0495e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f04901690e0 con 0x7f04900b13c0 2024-09-10T21:12:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f048f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049015f400 0x7f04901654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f048f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049015f400 0x7f04901654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45074/0 (socket says 172.21.15.16:45074) 2024-09-10T21:12:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f048f400640 1 -- 172.21.15.16:0/1525715750 learned_addr learned my addr 172.21.15.16:0/1525715750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f048ea00640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04900b0590 0x7f04901574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f0487e00640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f04900b13c0 0x7f0490164f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f048f400640 1 -- 172.21.15.16:0/1525715750 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f04900b13c0 msgr2=0x7f0490164f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f048f400640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f04900b13c0 0x7f0490164f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f048f400640 1 -- 172.21.15.16:0/1525715750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04900b0590 msgr2=0x7f04901574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f048f400640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04900b0590 0x7f04901574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f048f400640 1 -- 172.21.15.16:0/1525715750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049015cb80 con 0x7f049015f400 2024-09-10T21:12:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f0487e00640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f04900b13c0 0x7f0490164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f048f400640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049015f400 0x7f04901654b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f048800ea50 tx=0x7f048800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f0487400640 1 -- 172.21.15.16:0/1525715750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048800cd60 con 0x7f049015f400 2024-09-10T21:12:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f0487400640 1 -- 172.21.15.16:0/1525715750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0488004b30 con 0x7f049015f400 2024-09-10T21:12:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f0487400640 1 -- 172.21.15.16:0/1525715750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488005020 con 0x7f049015f400 2024-09-10T21:12:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f0495e00640 1 -- 172.21.15.16:0/1525715750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f049015ba10 con 0x7f049015f400 2024-09-10T21:12:59.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f0495e00640 1 -- 172.21.15.16:0/1525715750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f049015d120 con 0x7f049015f400 2024-09-10T21:12:59.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f0495e00640 1 -- 172.21.15.16:0/1525715750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04900cfc80 con 0x7f049015f400 2024-09-10T21:12:59.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.076+0000 7f0487400640 1 -- 172.21.15.16:0/1525715750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f048800f040 con 0x7f049015f400 2024-09-10T21:12:59.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.080+0000 7f0487400640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f046c072610 0x7f046c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.080+0000 7f0487400640 1 -- 172.21.15.16:0/1525715750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f04880937c0 con 0x7f049015f400 2024-09-10T21:12:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.080+0000 7f048ea00640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f046c072610 0x7f046c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.080+0000 7f048ea00640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f046c072610 0x7f046c074ad0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04800ae000 tx=0x7f04800b3040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:59.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.084+0000 7f0487400640 1 -- 172.21.15.16:0/1525715750 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0488016030 con 0x7f049015f400 2024-09-10T21:12:59.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0495e00640 1 -- 172.21.15.16:0/1525715750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f04901590e0 con 0x7f049015f400 2024-09-10T21:12:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0487400640 1 -- 172.21.15.16:0/1525715750 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1141 (secure 0 0 0) 0x7f048805e7c0 con 0x7f049015f400 2024-09-10T21:12:59.326 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:59.326 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":174,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":8,"num_up_osds":8,"osd_up_since":1726002618,"num_in_osds":8,"osd_in_since":1726002608,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":83},{"state_name":"creating+peering","count":14}],"num_pgs":97,"num_pools":3,"num_objects":21,"data_bytes":592756,"bytes_used":261951488,"bytes_avail":772832161792,"bytes_total":773094113280,"inactive_pgs_ratio":0.14432989060878754,"write_bytes_sec":2378,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":33,"btime":"2024-09-10T21:12:56:932852+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":5592}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-10T21:12:13.480543+0000","services":{}},"progress_events":{}} 2024-09-10T21:12:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0495e00640 1 -- 172.21.15.16:0/1525715750 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f046c072610 msgr2=0x7f046c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0495e00640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f046c072610 0x7f046c074ad0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04800ae000 tx=0x7f04800b3040 comp rx=0 tx=0).stop 2024-09-10T21:12:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0495e00640 1 -- 172.21.15.16:0/1525715750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049015f400 msgr2=0x7f04901654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0495e00640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049015f400 0x7f04901654b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f048800ea50 tx=0x7f048800ef10 comp rx=0 tx=0).stop 2024-09-10T21:12:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0495e00640 1 -- 172.21.15.16:0/1525715750 shutdown_connections 2024-09-10T21:12:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0495e00640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f046c072610 0x7f046c074ad0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0495e00640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049015f400 0x7f04901654b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0495e00640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f04900b13c0 0x7f0490164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0495e00640 1 --2- 172.21.15.16:0/1525715750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04900b0590 0x7f04901574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0495e00640 1 -- 172.21.15.16:0/1525715750 >> 172.21.15.16:0/1525715750 conn(0x7f049009b130 msgr2=0x7f04900cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0495e00640 1 -- 172.21.15.16:0/1525715750 shutdown_connections 2024-09-10T21:12:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.320+0000 7f0495e00640 1 -- 172.21.15.16:0/1525715750 wait complete. 2024-09-10T21:12:59.347 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T21:12:59.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:12:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 -- 172.21.15.16:0/4136034982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe78815a270 msgr2=0x7fe7880ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 --2- 172.21.15.16:0/4136034982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe78815a270 0x7fe7880ce610 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe78806f4f0 tx=0x7fe77801da10 comp rx=0 tx=0).stop 2024-09-10T21:12:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 -- 172.21.15.16:0/4136034982 shutdown_connections 2024-09-10T21:12:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 --2- 172.21.15.16:0/4136034982 >> v2:172.21.15.52:3300/0 conn(0x7fe7880705e0 0x7fe7880ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 --2- 172.21.15.16:0/4136034982 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe7880d06b0 0x7fe7880700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 --2- 172.21.15.16:0/4136034982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe78815a270 0x7fe7880ce610 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 -- 172.21.15.16:0/4136034982 >> 172.21.15.16:0/4136034982 conn(0x7fe7880cfb20 msgr2=0x7fe7880cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 -- 172.21.15.16:0/4136034982 shutdown_connections 2024-09-10T21:12:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 -- 172.21.15.16:0/4136034982 wait complete. 2024-09-10T21:12:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 Processor -- start 2024-09-10T21:12:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 -- start start 2024-09-10T21:12:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7880705e0 0x7fe788155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7880d06b0 0x7fe7881561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe78815a270 0x7fe78815e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe788072e50 con 0x7fe7880d06b0 2024-09-10T21:12:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe788072cd0 con 0x7fe7880705e0 2024-09-10T21:12:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe78d400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe788072fd0 con 0x7fe78815a270 2024-09-10T21:12:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe77fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7880d06b0 0x7fe7881561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe77fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7880d06b0 0x7fe7881561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45092/0 (socket says 172.21.15.16:45092) 2024-09-10T21:12:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe77fe00640 1 -- 172.21.15.16:0/1969720295 learned_addr learned my addr 172.21.15.16:0/1969720295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:12:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe787200640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe78815a270 0x7fe78815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.488+0000 7fe786800640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7880705e0 0x7fe788155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.492+0000 7fe77fe00640 1 -- 172.21.15.16:0/1969720295 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe78815a270 msgr2=0x7fe78815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.492+0000 7fe77fe00640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe78815a270 0x7fe78815e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.492+0000 7fe77fe00640 1 -- 172.21.15.16:0/1969720295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7880705e0 msgr2=0x7fe788155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.492+0000 7fe77fe00640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7880705e0 0x7fe788155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.492+0000 7fe77fe00640 1 -- 172.21.15.16:0/1969720295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe788160c60 con 0x7fe7880d06b0 2024-09-10T21:12:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.492+0000 7fe77fe00640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7880d06b0 0x7fe7881561d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe77000d980 tx=0x7fe77000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.492+0000 7fe77ea00640 1 -- 172.21.15.16:0/1969720295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770017070 con 0x7fe7880d06b0 2024-09-10T21:12:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.492+0000 7fe77ea00640 1 -- 172.21.15.16:0/1969720295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe770009d60 con 0x7fe7880d06b0 2024-09-10T21:12:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.492+0000 7fe77ea00640 1 -- 172.21.15.16:0/1969720295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770005280 con 0x7fe7880d06b0 2024-09-10T21:12:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.492+0000 7fe78d400640 1 -- 172.21.15.16:0/1969720295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe788160f40 con 0x7fe7880d06b0 2024-09-10T21:12:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.492+0000 7fe78d400640 1 -- 172.21.15.16:0/1969720295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe788161a20 con 0x7fe7880d06b0 2024-09-10T21:12:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.492+0000 7fe78d400640 1 -- 172.21.15.16:0/1969720295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7880700a0 con 0x7fe7880d06b0 2024-09-10T21:12:59.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.492+0000 7fe77ea00640 1 -- 172.21.15.16:0/1969720295 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe770005420 con 0x7fe7880d06b0 2024-09-10T21:12:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.496+0000 7fe77ea00640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe7640725e0 0x7fe764074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.496+0000 7fe77ea00640 1 -- 172.21.15.16:0/1969720295 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fe770093440 con 0x7fe7880d06b0 2024-09-10T21:12:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.496+0000 7fe786800640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe7640725e0 0x7fe764074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.496+0000 7fe786800640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe7640725e0 0x7fe764074aa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe7780b0000 tx=0x7fe7780a8040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.500+0000 7fe77ea00640 1 -- 172.21.15.16:0/1969720295 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe77005e430 con 0x7fe7880d06b0 2024-09-10T21:12:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.668+0000 7fe78d400640 1 -- 172.21.15.16:0/1969720295 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7880b5aa0 con 0x7fe7640725e0 2024-09-10T21:12:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.676+0000 7fe77ea00640 1 -- 172.21.15.16:0/1969720295 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7fe7880b5aa0 con 0x7fe7640725e0 2024-09-10T21:12:59.683 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:12:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:12:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.676+0000 7fe78d400640 1 -- 172.21.15.16:0/1969720295 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe7640725e0 msgr2=0x7fe764074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.676+0000 7fe78d400640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe7640725e0 0x7fe764074aa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe7780b0000 tx=0x7fe7780a8040 comp rx=0 tx=0).stop 2024-09-10T21:12:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.676+0000 7fe78d400640 1 -- 172.21.15.16:0/1969720295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7880d06b0 msgr2=0x7fe7881561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.676+0000 7fe78d400640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7880d06b0 0x7fe7881561d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe77000d980 tx=0x7fe77000de40 comp rx=0 tx=0).stop 2024-09-10T21:12:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.676+0000 7fe78d400640 1 -- 172.21.15.16:0/1969720295 shutdown_connections 2024-09-10T21:12:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.676+0000 7fe78d400640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe7640725e0 0x7fe764074aa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.676+0000 7fe78d400640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe78815a270 0x7fe78815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.676+0000 7fe78d400640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7880d06b0 0x7fe7881561d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.676+0000 7fe78d400640 1 --2- 172.21.15.16:0/1969720295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7880705e0 0x7fe788155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.676+0000 7fe78d400640 1 -- 172.21.15.16:0/1969720295 >> 172.21.15.16:0/1969720295 conn(0x7fe7880cfb20 msgr2=0x7fe78815d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.680+0000 7fe78d400640 1 -- 172.21.15.16:0/1969720295 shutdown_connections 2024-09-10T21:12:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:12:59.680+0000 7fe78d400640 1 -- 172.21.15.16:0/1969720295 wait complete. 2024-09-10T21:12:59.704 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-10T21:12:59.490187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256060,"kb_used_data":6504,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718660,"statfs":{"total":773094113280,"available":772831907840,"internally_reserved":0,"allocated":6660096,"data_stored":3422356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.593577"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929963+0000","last_change":"2024-09-10T21:12:53.959848+0000","last_active":"2024-09-10T21:12:54.929963+0000","last_peered":"2024-09-10T21:12:54.929963+0000","last_clean":"2024-09-10T21:12:54.929963+0000","last_became_active":"2024-09-10T21:12:53.959238+0000","last_became_peered":"2024-09-10T21:12:53.959238+0000","last_unstale":"2024-09-10T21:12:54.929963+0000","last_undegraded":"2024-09-10T21:12:54.929963+0000","last_fullsized":"2024-09-10T21:12:54.929963+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:21.592698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958252+0000","last_change":"2024-09-10T21:12:53.928126+0000","last_active":"2024-09-10T21:12:54.958252+0000","last_peered":"2024-09-10T21:12:54.958252+0000","last_clean":"2024-09-10T21:12:54.958252+0000","last_became_active":"2024-09-10T21:12:53.927596+0000","last_became_peered":"2024-09-10T21:12:53.927596+0000","last_unstale":"2024-09-10T21:12:54.958252+0000","last_undegraded":"2024-09-10T21:12:54.958252+0000","last_fullsized":"2024-09-10T21:12:54.958252+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:05.525627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929071+0000","last_change":"2024-09-10T21:12:53.929639+0000","last_active":"2024-09-10T21:12:54.929071+0000","last_peered":"2024-09-10T21:12:54.929071+0000","last_clean":"2024-09-10T21:12:54.929071+0000","last_became_active":"2024-09-10T21:12:53.929333+0000","last_became_peered":"2024-09-10T21:12:53.929333+0000","last_unstale":"2024-09-10T21:12:54.929071+0000","last_undegraded":"2024-09-10T21:12:54.929071+0000","last_fullsized":"2024-09-10T21:12:54.929071+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:35.276062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959451+0000","last_change":"2024-09-10T21:12:53.932951+0000","last_active":"2024-09-10T21:12:54.959451+0000","last_peered":"2024-09-10T21:12:54.959451+0000","last_clean":"2024-09-10T21:12:54.959451+0000","last_became_active":"2024-09-10T21:12:53.930361+0000","last_became_peered":"2024-09-10T21:12:53.930361+0000","last_unstale":"2024-09-10T21:12:54.959451+0000","last_undegraded":"2024-09-10T21:12:54.959451+0000","last_fullsized":"2024-09-10T21:12:54.959451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:25.811630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929510+0000","last_change":"2024-09-10T21:12:53.951504+0000","last_active":"2024-09-10T21:12:54.929510+0000","last_peered":"2024-09-10T21:12:54.929510+0000","last_clean":"2024-09-10T21:12:54.929510+0000","last_became_active":"2024-09-10T21:12:53.950545+0000","last_became_peered":"2024-09-10T21:12:53.950545+0000","last_unstale":"2024-09-10T21:12:54.929510+0000","last_undegraded":"2024-09-10T21:12:54.929510+0000","last_fullsized":"2024-09-10T21:12:54.929510+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:03.478755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929053+0000","last_change":"2024-09-10T21:12:53.949235+0000","last_active":"2024-09-10T21:12:54.929053+0000","last_peered":"2024-09-10T21:12:54.929053+0000","last_clean":"2024-09-10T21:12:54.929053+0000","last_became_active":"2024-09-10T21:12:53.936152+0000","last_became_peered":"2024-09-10T21:12:53.936152+0000","last_unstale":"2024-09-10T21:12:54.929053+0000","last_undegraded":"2024-09-10T21:12:54.929053+0000","last_fullsized":"2024-09-10T21:12:54.929053+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:09.566010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958816+0000","last_change":"2024-09-10T21:12:53.929021+0000","last_active":"2024-09-10T21:12:54.958816+0000","last_peered":"2024-09-10T21:12:54.958816+0000","last_clean":"2024-09-10T21:12:54.958816+0000","last_became_active":"2024-09-10T21:12:53.928713+0000","last_became_peered":"2024-09-10T21:12:53.928713+0000","last_unstale":"2024-09-10T21:12:54.958816+0000","last_undegraded":"2024-09-10T21:12:54.958816+0000","last_fullsized":"2024-09-10T21:12:54.958816+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:32.138010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928873+0000","last_change":"2024-09-10T21:12:53.934558+0000","last_active":"2024-09-10T21:12:54.928873+0000","last_peered":"2024-09-10T21:12:54.928873+0000","last_clean":"2024-09-10T21:12:54.928873+0000","last_became_active":"2024-09-10T21:12:53.934226+0000","last_became_peered":"2024-09-10T21:12:53.934226+0000","last_unstale":"2024-09-10T21:12:54.928873+0000","last_undegraded":"2024-09-10T21:12:54.928873+0000","last_fullsized":"2024-09-10T21:12:54.928873+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:13.993307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957958+0000","last_change":"2024-09-10T21:12:53.960070+0000","last_active":"2024-09-10T21:12:54.957958+0000","last_peered":"2024-09-10T21:12:54.957958+0000","last_clean":"2024-09-10T21:12:54.957958+0000","last_became_active":"2024-09-10T21:12:53.959848+0000","last_became_peered":"2024-09-10T21:12:53.959848+0000","last_unstale":"2024-09-10T21:12:54.957958+0000","last_undegraded":"2024-09-10T21:12:54.957958+0000","last_fullsized":"2024-09-10T21:12:54.957958+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:46.189946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930056+0000","last_change":"2024-09-10T21:12:53.952820+0000","last_active":"2024-09-10T21:12:54.930056+0000","last_peered":"2024-09-10T21:12:54.930056+0000","last_clean":"2024-09-10T21:12:54.930056+0000","last_became_active":"2024-09-10T21:12:53.952356+0000","last_became_peered":"2024-09-10T21:12:53.952356+0000","last_unstale":"2024-09-10T21:12:54.930056+0000","last_undegraded":"2024-09-10T21:12:54.930056+0000","last_fullsized":"2024-09-10T21:12:54.930056+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:32.544813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957210+0000","last_change":"2024-09-10T21:12:53.957907+0000","last_active":"2024-09-10T21:12:54.957210+0000","last_peered":"2024-09-10T21:12:54.957210+0000","last_clean":"2024-09-10T21:12:54.957210+0000","last_became_active":"2024-09-10T21:12:53.957595+0000","last_became_peered":"2024-09-10T21:12:53.957595+0000","last_unstale":"2024-09-10T21:12:54.957210+0000","last_undegraded":"2024-09-10T21:12:54.957210+0000","last_fullsized":"2024-09-10T21:12:54.957210+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:51.847731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957615+0000","last_change":"2024-09-10T21:12:53.936808+0000","last_active":"2024-09-10T21:12:54.957615+0000","last_peered":"2024-09-10T21:12:54.957615+0000","last_clean":"2024-09-10T21:12:54.957615+0000","last_became_active":"2024-09-10T21:12:53.936441+0000","last_became_peered":"2024-09-10T21:12:53.936441+0000","last_unstale":"2024-09-10T21:12:54.957615+0000","last_undegraded":"2024-09-10T21:12:54.957615+0000","last_fullsized":"2024-09-10T21:12:54.957615+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:30.866160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958673+0000","last_change":"2024-09-10T21:12:53.922722+0000","last_active":"2024-09-10T21:12:54.958673+0000","last_peered":"2024-09-10T21:12:54.958673+0000","last_clean":"2024-09-10T21:12:54.958673+0000","last_became_active":"2024-09-10T21:12:53.922457+0000","last_became_peered":"2024-09-10T21:12:53.922457+0000","last_unstale":"2024-09-10T21:12:54.958673+0000","last_undegraded":"2024-09-10T21:12:54.958673+0000","last_fullsized":"2024-09-10T21:12:54.958673+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:31.129061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928842+0000","last_change":"2024-09-10T21:12:53.945632+0000","last_active":"2024-09-10T21:12:54.928842+0000","last_peered":"2024-09-10T21:12:54.928842+0000","last_clean":"2024-09-10T21:12:54.928842+0000","last_became_active":"2024-09-10T21:12:53.945340+0000","last_became_peered":"2024-09-10T21:12:53.945340+0000","last_unstale":"2024-09-10T21:12:54.928842+0000","last_undegraded":"2024-09-10T21:12:54.928842+0000","last_fullsized":"2024-09-10T21:12:54.928842+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:56.966298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958445+0000","last_change":"2024-09-10T21:12:53.934328+0000","last_active":"2024-09-10T21:12:54.958445+0000","last_peered":"2024-09-10T21:12:54.958445+0000","last_clean":"2024-09-10T21:12:54.958445+0000","last_became_active":"2024-09-10T21:12:53.934057+0000","last_became_peered":"2024-09-10T21:12:53.934057+0000","last_unstale":"2024-09-10T21:12:54.958445+0000","last_undegraded":"2024-09-10T21:12:54.958445+0000","last_fullsized":"2024-09-10T21:12:54.958445+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:43.720965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928963+0000","last_change":"2024-09-10T21:12:53.935764+0000","last_active":"2024-09-10T21:12:54.928963+0000","last_peered":"2024-09-10T21:12:54.928963+0000","last_clean":"2024-09-10T21:12:54.928963+0000","last_became_active":"2024-09-10T21:12:53.935468+0000","last_became_peered":"2024-09-10T21:12:53.935468+0000","last_unstale":"2024-09-10T21:12:54.928963+0000","last_undegraded":"2024-09-10T21:12:54.928963+0000","last_fullsized":"2024-09-10T21:12:54.928963+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:47.640549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930205+0000","last_change":"2024-09-10T21:12:53.953515+0000","last_active":"2024-09-10T21:12:54.930205+0000","last_peered":"2024-09-10T21:12:54.930205+0000","last_clean":"2024-09-10T21:12:54.930205+0000","last_became_active":"2024-09-10T21:12:53.952358+0000","last_became_peered":"2024-09-10T21:12:53.952358+0000","last_unstale":"2024-09-10T21:12:54.930205+0000","last_undegraded":"2024-09-10T21:12:54.930205+0000","last_fullsized":"2024-09-10T21:12:54.930205+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:42.501794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929232+0000","last_change":"2024-09-10T21:12:53.923168+0000","last_active":"2024-09-10T21:12:54.929232+0000","last_peered":"2024-09-10T21:12:54.929232+0000","last_clean":"2024-09-10T21:12:54.929232+0000","last_became_active":"2024-09-10T21:12:53.922878+0000","last_became_peered":"2024-09-10T21:12:53.922878+0000","last_unstale":"2024-09-10T21:12:54.929232+0000","last_undegraded":"2024-09-10T21:12:54.929232+0000","last_fullsized":"2024-09-10T21:12:54.929232+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:55.156529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958104+0000","last_change":"2024-09-10T21:12:53.937960+0000","last_active":"2024-09-10T21:12:54.958104+0000","last_peered":"2024-09-10T21:12:54.958104+0000","last_clean":"2024-09-10T21:12:54.958104+0000","last_became_active":"2024-09-10T21:12:53.936967+0000","last_became_peered":"2024-09-10T21:12:53.936967+0000","last_unstale":"2024-09-10T21:12:54.958104+0000","last_undegraded":"2024-09-10T21:12:54.958104+0000","last_fullsized":"2024-09-10T21:12:54.958104+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:46.189946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957142+0000","last_change":"2024-09-10T21:12:53.959543+0000","last_active":"2024-09-10T21:12:54.957142+0000","last_peered":"2024-09-10T21:12:54.957142+0000","last_clean":"2024-09-10T21:12:54.957142+0000","last_became_active":"2024-09-10T21:12:53.959240+0000","last_became_peered":"2024-09-10T21:12:53.959240+0000","last_unstale":"2024-09-10T21:12:54.957142+0000","last_undegraded":"2024-09-10T21:12:54.957142+0000","last_fullsized":"2024-09-10T21:12:54.957142+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:34.811075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958858+0000","last_change":"2024-09-10T21:12:53.926236+0000","last_active":"2024-09-10T21:12:54.958858+0000","last_peered":"2024-09-10T21:12:54.958858+0000","last_clean":"2024-09-10T21:12:54.958858+0000","last_became_active":"2024-09-10T21:12:53.924195+0000","last_became_peered":"2024-09-10T21:12:53.924195+0000","last_unstale":"2024-09-10T21:12:54.958858+0000","last_undegraded":"2024-09-10T21:12:54.958858+0000","last_fullsized":"2024-09-10T21:12:54.958858+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:28.847293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957079+0000","last_change":"2024-09-10T21:12:53.934083+0000","last_active":"2024-09-10T21:12:54.957079+0000","last_peered":"2024-09-10T21:12:54.957079+0000","last_clean":"2024-09-10T21:12:54.957079+0000","last_became_active":"2024-09-10T21:12:53.933188+0000","last_became_peered":"2024-09-10T21:12:53.933188+0000","last_unstale":"2024-09-10T21:12:54.957079+0000","last_undegraded":"2024-09-10T21:12:54.957079+0000","last_fullsized":"2024-09-10T21:12:54.957079+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:45.028281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958876+0000","last_change":"2024-09-10T21:12:53.920537+0000","last_active":"2024-09-10T21:12:54.958876+0000","last_peered":"2024-09-10T21:12:54.958876+0000","last_clean":"2024-09-10T21:12:54.958876+0000","last_became_active":"2024-09-10T21:12:53.920252+0000","last_became_peered":"2024-09-10T21:12:53.920252+0000","last_unstale":"2024-09-10T21:12:54.958876+0000","last_undegraded":"2024-09-10T21:12:54.958876+0000","last_fullsized":"2024-09-10T21:12:54.958876+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:04.923623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957439+0000","last_change":"2024-09-10T21:12:53.946022+0000","last_active":"2024-09-10T21:12:54.957439+0000","last_peered":"2024-09-10T21:12:54.957439+0000","last_clean":"2024-09-10T21:12:54.957439+0000","last_became_active":"2024-09-10T21:12:53.945751+0000","last_became_peered":"2024-09-10T21:12:53.945751+0000","last_unstale":"2024-09-10T21:12:54.957439+0000","last_undegraded":"2024-09-10T21:12:54.957439+0000","last_fullsized":"2024-09-10T21:12:54.957439+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:24.397204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958359+0000","last_change":"2024-09-10T21:12:53.940509+0000","last_active":"2024-09-10T21:12:54.958359+0000","last_peered":"2024-09-10T21:12:54.958359+0000","last_clean":"2024-09-10T21:12:54.958359+0000","last_became_active":"2024-09-10T21:12:53.940110+0000","last_became_peered":"2024-09-10T21:12:53.940110+0000","last_unstale":"2024-09-10T21:12:54.958359+0000","last_undegraded":"2024-09-10T21:12:54.958359+0000","last_fullsized":"2024-09-10T21:12:54.958359+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:00.280645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958833+0000","last_change":"2024-09-10T21:12:53.923199+0000","last_active":"2024-09-10T21:12:54.958833+0000","last_peered":"2024-09-10T21:12:54.958833+0000","last_clean":"2024-09-10T21:12:54.958833+0000","last_became_active":"2024-09-10T21:12:53.922779+0000","last_became_peered":"2024-09-10T21:12:53.922779+0000","last_unstale":"2024-09-10T21:12:54.958833+0000","last_undegraded":"2024-09-10T21:12:54.958833+0000","last_fullsized":"2024-09-10T21:12:54.958833+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:48.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958350+0000","last_change":"2024-09-10T21:12:53.927601+0000","last_active":"2024-09-10T21:12:54.958350+0000","last_peered":"2024-09-10T21:12:54.958350+0000","last_clean":"2024-09-10T21:12:54.958350+0000","last_became_active":"2024-09-10T21:12:53.927129+0000","last_became_peered":"2024-09-10T21:12:53.927129+0000","last_unstale":"2024-09-10T21:12:54.958350+0000","last_undegraded":"2024-09-10T21:12:54.958350+0000","last_fullsized":"2024-09-10T21:12:54.958350+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:23.222712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930225+0000","last_change":"2024-09-10T21:12:53.939591+0000","last_active":"2024-09-10T21:12:54.930225+0000","last_peered":"2024-09-10T21:12:54.930225+0000","last_clean":"2024-09-10T21:12:54.930225+0000","last_became_active":"2024-09-10T21:12:53.939314+0000","last_became_peered":"2024-09-10T21:12:53.939314+0000","last_unstale":"2024-09-10T21:12:54.930225+0000","last_undegraded":"2024-09-10T21:12:54.930225+0000","last_fullsized":"2024-09-10T21:12:54.930225+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:11.587953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930610+0000","last_change":"2024-09-10T21:12:53.937058+0000","last_active":"2024-09-10T21:12:54.930610+0000","last_peered":"2024-09-10T21:12:54.930610+0000","last_clean":"2024-09-10T21:12:54.930610+0000","last_became_active":"2024-09-10T21:12:53.936798+0000","last_became_peered":"2024-09-10T21:12:53.936798+0000","last_unstale":"2024-09-10T21:12:54.930610+0000","last_undegraded":"2024-09-10T21:12:54.930610+0000","last_fullsized":"2024-09-10T21:12:54.930610+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:14.822315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959093+0000","last_change":"2024-09-10T21:12:53.925270+0000","last_active":"2024-09-10T21:12:54.959093+0000","last_peered":"2024-09-10T21:12:54.959093+0000","last_clean":"2024-09-10T21:12:54.959093+0000","last_became_active":"2024-09-10T21:12:53.923493+0000","last_became_peered":"2024-09-10T21:12:53.923493+0000","last_unstale":"2024-09-10T21:12:54.959093+0000","last_undegraded":"2024-09-10T21:12:54.959093+0000","last_fullsized":"2024-09-10T21:12:54.959093+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:40.051608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957686+0000","last_change":"2024-09-10T21:12:53.932068+0000","last_active":"2024-09-10T21:12:54.957686+0000","last_peered":"2024-09-10T21:12:54.957686+0000","last_clean":"2024-09-10T21:12:54.957686+0000","last_became_active":"2024-09-10T21:12:53.931714+0000","last_became_peered":"2024-09-10T21:12:53.931714+0000","last_unstale":"2024-09-10T21:12:54.957686+0000","last_undegraded":"2024-09-10T21:12:54.957686+0000","last_fullsized":"2024-09-10T21:12:54.957686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:13.024445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.956938+0000","last_change":"2024-09-10T21:12:53.939129+0000","last_active":"2024-09-10T21:12:54.956938+0000","last_peered":"2024-09-10T21:12:54.956938+0000","last_clean":"2024-09-10T21:12:54.956938+0000","last_became_active":"2024-09-10T21:12:53.938807+0000","last_became_peered":"2024-09-10T21:12:53.938807+0000","last_unstale":"2024-09-10T21:12:54.956938+0000","last_undegraded":"2024-09-10T21:12:54.956938+0000","last_fullsized":"2024-09-10T21:12:54.956938+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:03.047971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929664+0000","last_change":"2024-09-10T21:12:53.938689+0000","last_active":"2024-09-10T21:12:54.929664+0000","last_peered":"2024-09-10T21:12:54.929664+0000","last_clean":"2024-09-10T21:12:54.929664+0000","last_became_active":"2024-09-10T21:12:53.935691+0000","last_became_peered":"2024-09-10T21:12:53.935691+0000","last_unstale":"2024-09-10T21:12:54.929664+0000","last_undegraded":"2024-09-10T21:12:54.929664+0000","last_fullsized":"2024-09-10T21:12:54.929664+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:19.453855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":19,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957411+0000","last_change":"2024-09-10T21:12:52.916629+0000","last_active":"2024-09-10T21:12:54.957411+0000","last_peered":"2024-09-10T21:12:54.957411+0000","last_clean":"2024-09-10T21:12:54.957411+0000","last_became_active":"2024-09-10T21:12:52.916091+0000","last_became_peered":"2024-09-10T21:12:52.916091+0000","last_unstale":"2024-09-10T21:12:54.957411+0000","last_undegraded":"2024-09-10T21:12:54.957411+0000","last_fullsized":"2024-09-10T21:12:54.957411+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:27.028802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930407+0000","last_change":"2024-09-10T21:12:53.953827+0000","last_active":"2024-09-10T21:12:54.930407+0000","last_peered":"2024-09-10T21:12:54.930407+0000","last_clean":"2024-09-10T21:12:54.930407+0000","last_became_active":"2024-09-10T21:12:53.953262+0000","last_became_peered":"2024-09-10T21:12:53.953262+0000","last_unstale":"2024-09-10T21:12:54.930407+0000","last_undegraded":"2024-09-10T21:12:54.930407+0000","last_fullsized":"2024-09-10T21:12:54.930407+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:29.044625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958380+0000","last_change":"2024-09-10T21:12:52.926961+0000","last_active":"2024-09-10T21:12:54.958380+0000","last_peered":"2024-09-10T21:12:54.958380+0000","last_clean":"2024-09-10T21:12:54.958380+0000","last_became_active":"2024-09-10T21:12:52.926723+0000","last_became_peered":"2024-09-10T21:12:52.926723+0000","last_unstale":"2024-09-10T21:12:54.958380+0000","last_undegraded":"2024-09-10T21:12:54.958380+0000","last_fullsized":"2024-09-10T21:12:54.958380+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:44.000606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930976+0000","last_change":"2024-09-10T21:12:53.944452+0000","last_active":"2024-09-10T21:12:54.930976+0000","last_peered":"2024-09-10T21:12:54.930976+0000","last_clean":"2024-09-10T21:12:54.930976+0000","last_became_active":"2024-09-10T21:12:53.944142+0000","last_became_peered":"2024-09-10T21:12:53.944142+0000","last_unstale":"2024-09-10T21:12:54.930976+0000","last_undegraded":"2024-09-10T21:12:54.930976+0000","last_fullsized":"2024-09-10T21:12:54.930976+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:27.806946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929380+0000","last_change":"2024-09-10T21:12:52.924953+0000","last_active":"2024-09-10T21:12:54.929380+0000","last_peered":"2024-09-10T21:12:54.929380+0000","last_clean":"2024-09-10T21:12:54.929380+0000","last_became_active":"2024-09-10T21:12:52.924601+0000","last_became_peered":"2024-09-10T21:12:52.924601+0000","last_unstale":"2024-09-10T21:12:54.929380+0000","last_undegraded":"2024-09-10T21:12:54.929380+0000","last_fullsized":"2024-09-10T21:12:54.929380+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:44.842888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929019+0000","last_change":"2024-09-10T21:12:53.950957+0000","last_active":"2024-09-10T21:12:54.929019+0000","last_peered":"2024-09-10T21:12:54.929019+0000","last_clean":"2024-09-10T21:12:54.929019+0000","last_became_active":"2024-09-10T21:12:53.950673+0000","last_became_peered":"2024-09-10T21:12:53.950673+0000","last_unstale":"2024-09-10T21:12:54.929019+0000","last_undegraded":"2024-09-10T21:12:54.929019+0000","last_fullsized":"2024-09-10T21:12:54.929019+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:27.782259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959533+0000","last_change":"2024-09-10T21:12:52.920769+0000","last_active":"2024-09-10T21:12:54.959533+0000","last_peered":"2024-09-10T21:12:54.959533+0000","last_clean":"2024-09-10T21:12:54.959533+0000","last_became_active":"2024-09-10T21:12:52.920460+0000","last_became_peered":"2024-09-10T21:12:52.920460+0000","last_unstale":"2024-09-10T21:12:54.959533+0000","last_undegraded":"2024-09-10T21:12:54.959533+0000","last_fullsized":"2024-09-10T21:12:54.959533+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:53.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930430+0000","last_change":"2024-09-10T21:12:53.946296+0000","last_active":"2024-09-10T21:12:54.930430+0000","last_peered":"2024-09-10T21:12:54.930430+0000","last_clean":"2024-09-10T21:12:54.930430+0000","last_became_active":"2024-09-10T21:12:53.945868+0000","last_became_peered":"2024-09-10T21:12:53.945868+0000","last_unstale":"2024-09-10T21:12:54.930430+0000","last_undegraded":"2024-09-10T21:12:54.930430+0000","last_fullsized":"2024-09-10T21:12:54.930430+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:52.681542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929006+0000","last_change":"2024-09-10T21:12:52.917429+0000","last_active":"2024-09-10T21:12:54.929006+0000","last_peered":"2024-09-10T21:12:54.929006+0000","last_clean":"2024-09-10T21:12:54.929006+0000","last_became_active":"2024-09-10T21:12:52.917148+0000","last_became_peered":"2024-09-10T21:12:52.917148+0000","last_unstale":"2024-09-10T21:12:54.929006+0000","last_undegraded":"2024-09-10T21:12:54.929006+0000","last_fullsized":"2024-09-10T21:12:54.929006+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:05.034728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957049+0000","last_change":"2024-09-10T21:12:53.947773+0000","last_active":"2024-09-10T21:12:54.957049+0000","last_peered":"2024-09-10T21:12:54.957049+0000","last_clean":"2024-09-10T21:12:54.957049+0000","last_became_active":"2024-09-10T21:12:53.947473+0000","last_became_peered":"2024-09-10T21:12:53.947473+0000","last_unstale":"2024-09-10T21:12:54.957049+0000","last_undegraded":"2024-09-10T21:12:54.957049+0000","last_fullsized":"2024-09-10T21:12:54.957049+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:09.615040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929858+0000","last_change":"2024-09-10T21:12:52.907684+0000","last_active":"2024-09-10T21:12:54.929858+0000","last_peered":"2024-09-10T21:12:54.929858+0000","last_clean":"2024-09-10T21:12:54.929858+0000","last_became_active":"2024-09-10T21:12:52.907395+0000","last_became_peered":"2024-09-10T21:12:52.907395+0000","last_unstale":"2024-09-10T21:12:54.929858+0000","last_undegraded":"2024-09-10T21:12:54.929858+0000","last_fullsized":"2024-09-10T21:12:54.929858+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:19.885587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928722+0000","last_change":"2024-09-10T21:12:53.934723+0000","last_active":"2024-09-10T21:12:54.928722+0000","last_peered":"2024-09-10T21:12:54.928722+0000","last_clean":"2024-09-10T21:12:54.928722+0000","last_became_active":"2024-09-10T21:12:53.932327+0000","last_became_peered":"2024-09-10T21:12:53.932327+0000","last_unstale":"2024-09-10T21:12:54.928722+0000","last_undegraded":"2024-09-10T21:12:54.928722+0000","last_fullsized":"2024-09-10T21:12:54.928722+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:17.710007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957534+0000","last_change":"2024-09-10T21:12:52.922523+0000","last_active":"2024-09-10T21:12:54.957534+0000","last_peered":"2024-09-10T21:12:54.957534+0000","last_clean":"2024-09-10T21:12:54.957534+0000","last_became_active":"2024-09-10T21:12:52.922266+0000","last_became_peered":"2024-09-10T21:12:52.922266+0000","last_unstale":"2024-09-10T21:12:54.957534+0000","last_undegraded":"2024-09-10T21:12:54.957534+0000","last_fullsized":"2024-09-10T21:12:54.957534+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:01.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928915+0000","last_change":"2024-09-10T21:12:53.951739+0000","last_active":"2024-09-10T21:12:54.928915+0000","last_peered":"2024-09-10T21:12:54.928915+0000","last_clean":"2024-09-10T21:12:54.928915+0000","last_became_active":"2024-09-10T21:12:53.943111+0000","last_became_peered":"2024-09-10T21:12:53.943111+0000","last_unstale":"2024-09-10T21:12:54.928915+0000","last_undegraded":"2024-09-10T21:12:54.928915+0000","last_fullsized":"2024-09-10T21:12:54.928915+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:00.034619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958276+0000","last_change":"2024-09-10T21:12:52.925890+0000","last_active":"2024-09-10T21:12:54.958276+0000","last_peered":"2024-09-10T21:12:54.958276+0000","last_clean":"2024-09-10T21:12:54.958276+0000","last_became_active":"2024-09-10T21:12:52.925061+0000","last_became_peered":"2024-09-10T21:12:52.925061+0000","last_unstale":"2024-09-10T21:12:54.958276+0000","last_undegraded":"2024-09-10T21:12:54.958276+0000","last_fullsized":"2024-09-10T21:12:54.958276+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:53.791545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959064+0000","last_change":"2024-09-10T21:12:53.947582+0000","last_active":"2024-09-10T21:12:54.959064+0000","last_peered":"2024-09-10T21:12:54.959064+0000","last_clean":"2024-09-10T21:12:54.959064+0000","last_became_active":"2024-09-10T21:12:53.947316+0000","last_became_peered":"2024-09-10T21:12:53.947316+0000","last_unstale":"2024-09-10T21:12:54.959064+0000","last_undegraded":"2024-09-10T21:12:54.959064+0000","last_fullsized":"2024-09-10T21:12:54.959064+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:52.259847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957862+0000","last_change":"2024-09-10T21:12:52.920456+0000","last_active":"2024-09-10T21:12:54.957862+0000","last_peered":"2024-09-10T21:12:54.957862+0000","last_clean":"2024-09-10T21:12:54.957862+0000","last_became_active":"2024-09-10T21:12:52.920172+0000","last_became_peered":"2024-09-10T21:12:52.920172+0000","last_unstale":"2024-09-10T21:12:54.957862+0000","last_undegraded":"2024-09-10T21:12:54.957862+0000","last_fullsized":"2024-09-10T21:12:54.957862+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:51.616323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929355+0000","last_change":"2024-09-10T21:12:53.945665+0000","last_active":"2024-09-10T21:12:54.929355+0000","last_peered":"2024-09-10T21:12:54.929355+0000","last_clean":"2024-09-10T21:12:54.929355+0000","last_became_active":"2024-09-10T21:12:53.945387+0000","last_became_peered":"2024-09-10T21:12:53.945387+0000","last_unstale":"2024-09-10T21:12:54.929355+0000","last_undegraded":"2024-09-10T21:12:54.929355+0000","last_fullsized":"2024-09-10T21:12:54.929355+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:31.090679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930717+0000","last_change":"2024-09-10T21:12:52.921420+0000","last_active":"2024-09-10T21:12:54.930717+0000","last_peered":"2024-09-10T21:12:54.930717+0000","last_clean":"2024-09-10T21:12:54.930717+0000","last_became_active":"2024-09-10T21:12:52.920194+0000","last_became_peered":"2024-09-10T21:12:52.920194+0000","last_unstale":"2024-09-10T21:12:54.930717+0000","last_undegraded":"2024-09-10T21:12:54.930717+0000","last_fullsized":"2024-09-10T21:12:54.930717+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:02.042065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.931041+0000","last_change":"2024-09-10T21:12:53.959498+0000","last_active":"2024-09-10T21:12:54.931041+0000","last_peered":"2024-09-10T21:12:54.931041+0000","last_clean":"2024-09-10T21:12:54.931041+0000","last_became_active":"2024-09-10T21:12:53.959157+0000","last_became_peered":"2024-09-10T21:12:53.959157+0000","last_unstale":"2024-09-10T21:12:54.931041+0000","last_undegraded":"2024-09-10T21:12:54.931041+0000","last_fullsized":"2024-09-10T21:12:54.931041+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:11.561518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959620+0000","last_change":"2024-09-10T21:12:52.920836+0000","last_active":"2024-09-10T21:12:54.959620+0000","last_peered":"2024-09-10T21:12:54.959620+0000","last_clean":"2024-09-10T21:12:54.959620+0000","last_became_active":"2024-09-10T21:12:52.920547+0000","last_became_peered":"2024-09-10T21:12:52.920547+0000","last_unstale":"2024-09-10T21:12:54.959620+0000","last_undegraded":"2024-09-10T21:12:54.959620+0000","last_fullsized":"2024-09-10T21:12:54.959620+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:22.866900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959452+0000","last_change":"2024-09-10T21:12:53.936019+0000","last_active":"2024-09-10T21:12:54.959452+0000","last_peered":"2024-09-10T21:12:54.959452+0000","last_clean":"2024-09-10T21:12:54.959452+0000","last_became_active":"2024-09-10T21:12:53.935728+0000","last_became_peered":"2024-09-10T21:12:53.935728+0000","last_unstale":"2024-09-10T21:12:54.959452+0000","last_undegraded":"2024-09-10T21:12:54.959452+0000","last_fullsized":"2024-09-10T21:12:54.959452+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:32.220970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":21,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958482+0000","last_change":"2024-09-10T21:12:52.924581+0000","last_active":"2024-09-10T21:12:54.958482+0000","last_peered":"2024-09-10T21:12:54.958482+0000","last_clean":"2024-09-10T21:12:54.958482+0000","last_became_active":"2024-09-10T21:12:52.924327+0000","last_became_peered":"2024-09-10T21:12:52.924327+0000","last_unstale":"2024-09-10T21:12:54.958482+0000","last_undegraded":"2024-09-10T21:12:54.958482+0000","last_fullsized":"2024-09-10T21:12:54.958482+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:10.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957565+0000","last_change":"2024-09-10T21:12:53.935986+0000","last_active":"2024-09-10T21:12:54.957565+0000","last_peered":"2024-09-10T21:12:54.957565+0000","last_clean":"2024-09-10T21:12:54.957565+0000","last_became_active":"2024-09-10T21:12:53.935361+0000","last_became_peered":"2024-09-10T21:12:53.935361+0000","last_unstale":"2024-09-10T21:12:54.957565+0000","last_undegraded":"2024-09-10T21:12:54.957565+0000","last_fullsized":"2024-09-10T21:12:54.957565+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:42.454264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930802+0000","last_change":"2024-09-10T21:12:52.919371+0000","last_active":"2024-09-10T21:12:54.930802+0000","last_peered":"2024-09-10T21:12:54.930802+0000","last_clean":"2024-09-10T21:12:54.930802+0000","last_became_active":"2024-09-10T21:12:52.918995+0000","last_became_peered":"2024-09-10T21:12:52.918995+0000","last_unstale":"2024-09-10T21:12:54.930802+0000","last_undegraded":"2024-09-10T21:12:54.930802+0000","last_fullsized":"2024-09-10T21:12:54.930802+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:33.805170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928901+0000","last_change":"2024-09-10T21:12:53.930651+0000","last_active":"2024-09-10T21:12:54.928901+0000","last_peered":"2024-09-10T21:12:54.928901+0000","last_clean":"2024-09-10T21:12:54.928901+0000","last_became_active":"2024-09-10T21:12:53.930387+0000","last_became_peered":"2024-09-10T21:12:53.930387+0000","last_unstale":"2024-09-10T21:12:54.928901+0000","last_undegraded":"2024-09-10T21:12:54.928901+0000","last_fullsized":"2024-09-10T21:12:54.928901+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:32.429787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959240+0000","last_change":"2024-09-10T21:12:52.923255+0000","last_active":"2024-09-10T21:12:54.959240+0000","last_peered":"2024-09-10T21:12:54.959240+0000","last_clean":"2024-09-10T21:12:54.959240+0000","last_became_active":"2024-09-10T21:12:52.919965+0000","last_became_peered":"2024-09-10T21:12:52.919965+0000","last_unstale":"2024-09-10T21:12:54.959240+0000","last_undegraded":"2024-09-10T21:12:54.959240+0000","last_fullsized":"2024-09-10T21:12:54.959240+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:15.274251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957533+0000","last_change":"2024-09-10T21:12:53.933395+0000","last_active":"2024-09-10T21:12:54.957533+0000","last_peered":"2024-09-10T21:12:54.957533+0000","last_clean":"2024-09-10T21:12:54.957533+0000","last_became_active":"2024-09-10T21:12:53.933116+0000","last_became_peered":"2024-09-10T21:12:53.933116+0000","last_unstale":"2024-09-10T21:12:54.957533+0000","last_undegraded":"2024-09-10T21:12:54.957533+0000","last_fullsized":"2024-09-10T21:12:54.957533+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:25.230192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959046+0000","last_change":"2024-09-10T21:12:52.923218+0000","last_active":"2024-09-10T21:12:54.959046+0000","last_peered":"2024-09-10T21:12:54.959046+0000","last_clean":"2024-09-10T21:12:54.959046+0000","last_became_active":"2024-09-10T21:12:52.919955+0000","last_became_peered":"2024-09-10T21:12:52.919955+0000","last_unstale":"2024-09-10T21:12:54.959046+0000","last_undegraded":"2024-09-10T21:12:54.959046+0000","last_fullsized":"2024-09-10T21:12:54.959046+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:53.792649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959072+0000","last_change":"2024-09-10T21:12:53.922279+0000","last_active":"2024-09-10T21:12:54.959072+0000","last_peered":"2024-09-10T21:12:54.959072+0000","last_clean":"2024-09-10T21:12:54.959072+0000","last_became_active":"2024-09-10T21:12:53.921900+0000","last_became_peered":"2024-09-10T21:12:53.921900+0000","last_unstale":"2024-09-10T21:12:54.959072+0000","last_undegraded":"2024-09-10T21:12:54.959072+0000","last_fullsized":"2024-09-10T21:12:54.959072+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:56.649446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":23,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958890+0000","last_change":"2024-09-10T21:12:52.920975+0000","last_active":"2024-09-10T21:12:54.958890+0000","last_peered":"2024-09-10T21:12:54.958890+0000","last_clean":"2024-09-10T21:12:54.958890+0000","last_became_active":"2024-09-10T21:12:52.919980+0000","last_became_peered":"2024-09-10T21:12:52.919980+0000","last_unstale":"2024-09-10T21:12:54.958890+0000","last_undegraded":"2024-09-10T21:12:54.958890+0000","last_fullsized":"2024-09-10T21:12:54.958890+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:28.166375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929206+0000","last_change":"2024-09-10T21:12:53.949707+0000","last_active":"2024-09-10T21:12:54.929206+0000","last_peered":"2024-09-10T21:12:54.929206+0000","last_clean":"2024-09-10T21:12:54.929206+0000","last_became_active":"2024-09-10T21:12:53.949438+0000","last_became_peered":"2024-09-10T21:12:53.949438+0000","last_unstale":"2024-09-10T21:12:54.929206+0000","last_undegraded":"2024-09-10T21:12:54.929206+0000","last_fullsized":"2024-09-10T21:12:54.929206+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:06.481982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958468+0000","last_change":"2024-09-10T21:12:52.928329+0000","last_active":"2024-09-10T21:12:54.958468+0000","last_peered":"2024-09-10T21:12:54.958468+0000","last_clean":"2024-09-10T21:12:54.958468+0000","last_became_active":"2024-09-10T21:12:52.928026+0000","last_became_peered":"2024-09-10T21:12:52.928026+0000","last_unstale":"2024-09-10T21:12:54.958468+0000","last_undegraded":"2024-09-10T21:12:54.958468+0000","last_fullsized":"2024-09-10T21:12:54.958468+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:11.221179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929423+0000","last_change":"2024-09-10T21:12:53.946213+0000","last_active":"2024-09-10T21:12:54.929423+0000","last_peered":"2024-09-10T21:12:54.929423+0000","last_clean":"2024-09-10T21:12:54.929423+0000","last_became_active":"2024-09-10T21:12:53.945887+0000","last_became_peered":"2024-09-10T21:12:53.945887+0000","last_unstale":"2024-09-10T21:12:54.929423+0000","last_undegraded":"2024-09-10T21:12:54.929423+0000","last_fullsized":"2024-09-10T21:12:54.929423+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:29.514818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930850+0000","last_change":"2024-09-10T21:12:52.920779+0000","last_active":"2024-09-10T21:12:54.930850+0000","last_peered":"2024-09-10T21:12:54.930850+0000","last_clean":"2024-09-10T21:12:54.930850+0000","last_became_active":"2024-09-10T21:12:52.920513+0000","last_became_peered":"2024-09-10T21:12:52.920513+0000","last_unstale":"2024-09-10T21:12:54.930850+0000","last_undegraded":"2024-09-10T21:12:54.930850+0000","last_fullsized":"2024-09-10T21:12:54.930850+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:18.447950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958095+0000","last_change":"2024-09-10T21:12:53.926597+0000","last_active":"2024-09-10T21:12:54.958095+0000","last_peered":"2024-09-10T21:12:54.958095+0000","last_clean":"2024-09-10T21:12:54.958095+0000","last_became_active":"2024-09-10T21:12:53.925894+0000","last_became_peered":"2024-09-10T21:12:53.925894+0000","last_unstale":"2024-09-10T21:12:54.958095+0000","last_undegraded":"2024-09-10T21:12:54.958095+0000","last_fullsized":"2024-09-10T21:12:54.958095+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:46.459101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929285+0000","last_change":"2024-09-10T21:12:52.925824+0000","last_active":"2024-09-10T21:12:54.929285+0000","last_peered":"2024-09-10T21:12:54.929285+0000","last_clean":"2024-09-10T21:12:54.929285+0000","last_became_active":"2024-09-10T21:12:52.925229+0000","last_became_peered":"2024-09-10T21:12:52.925229+0000","last_unstale":"2024-09-10T21:12:54.929285+0000","last_undegraded":"2024-09-10T21:12:54.929285+0000","last_fullsized":"2024-09-10T21:12:54.929285+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:10.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959532+0000","last_change":"2024-09-10T21:12:53.930750+0000","last_active":"2024-09-10T21:12:54.959532+0000","last_peered":"2024-09-10T21:12:54.959532+0000","last_clean":"2024-09-10T21:12:54.959532+0000","last_became_active":"2024-09-10T21:12:53.930443+0000","last_became_peered":"2024-09-10T21:12:53.930443+0000","last_unstale":"2024-09-10T21:12:54.959532+0000","last_undegraded":"2024-09-10T21:12:54.959532+0000","last_fullsized":"2024-09-10T21:12:54.959532+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:06.203980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.931151+0000","last_change":"2024-09-10T21:12:52.913454+0000","last_active":"2024-09-10T21:12:54.931151+0000","last_peered":"2024-09-10T21:12:54.931151+0000","last_clean":"2024-09-10T21:12:54.931151+0000","last_became_active":"2024-09-10T21:12:52.913160+0000","last_became_peered":"2024-09-10T21:12:52.913160+0000","last_unstale":"2024-09-10T21:12:54.931151+0000","last_undegraded":"2024-09-10T21:12:54.931151+0000","last_fullsized":"2024-09-10T21:12:54.931151+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:35.717760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957784+0000","last_change":"2024-09-10T21:12:53.960736+0000","last_active":"2024-09-10T21:12:54.957784+0000","last_peered":"2024-09-10T21:12:54.957784+0000","last_clean":"2024-09-10T21:12:54.957784+0000","last_became_active":"2024-09-10T21:12:53.960493+0000","last_became_peered":"2024-09-10T21:12:53.960493+0000","last_unstale":"2024-09-10T21:12:54.957784+0000","last_undegraded":"2024-09-10T21:12:54.957784+0000","last_fullsized":"2024-09-10T21:12:54.957784+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:43.293392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957171+0000","last_change":"2024-09-10T21:12:52.919813+0000","last_active":"2024-09-10T21:12:54.957171+0000","last_peered":"2024-09-10T21:12:54.957171+0000","last_clean":"2024-09-10T21:12:54.957171+0000","last_became_active":"2024-09-10T21:12:52.919373+0000","last_became_peered":"2024-09-10T21:12:52.919373+0000","last_unstale":"2024-09-10T21:12:54.957171+0000","last_undegraded":"2024-09-10T21:12:54.957171+0000","last_fullsized":"2024-09-10T21:12:54.957171+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:25.283722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930686+0000","last_change":"2024-09-10T21:12:53.937502+0000","last_active":"2024-09-10T21:12:54.930686+0000","last_peered":"2024-09-10T21:12:54.930686+0000","last_clean":"2024-09-10T21:12:54.930686+0000","last_became_active":"2024-09-10T21:12:53.937201+0000","last_became_peered":"2024-09-10T21:12:53.937201+0000","last_unstale":"2024-09-10T21:12:54.930686+0000","last_undegraded":"2024-09-10T21:12:54.930686+0000","last_fullsized":"2024-09-10T21:12:54.930686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:36.394239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930603+0000","last_change":"2024-09-10T21:12:52.920031+0000","last_active":"2024-09-10T21:12:54.930603+0000","last_peered":"2024-09-10T21:12:54.930603+0000","last_clean":"2024-09-10T21:12:54.930603+0000","last_became_active":"2024-09-10T21:12:52.919769+0000","last_became_peered":"2024-09-10T21:12:52.919769+0000","last_unstale":"2024-09-10T21:12:54.930603+0000","last_undegraded":"2024-09-10T21:12:54.930603+0000","last_fullsized":"2024-09-10T21:12:54.930603+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:08.609135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929907+0000","last_change":"2024-09-10T21:12:53.928673+0000","last_active":"2024-09-10T21:12:54.929907+0000","last_peered":"2024-09-10T21:12:54.929907+0000","last_clean":"2024-09-10T21:12:54.929907+0000","last_became_active":"2024-09-10T21:12:53.928390+0000","last_became_peered":"2024-09-10T21:12:53.928390+0000","last_unstale":"2024-09-10T21:12:54.929907+0000","last_undegraded":"2024-09-10T21:12:54.929907+0000","last_fullsized":"2024-09-10T21:12:54.929907+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:12.631625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957618+0000","last_change":"2024-09-10T21:12:52.910741+0000","last_active":"2024-09-10T21:12:54.957618+0000","last_peered":"2024-09-10T21:12:54.957618+0000","last_clean":"2024-09-10T21:12:54.957618+0000","last_became_active":"2024-09-10T21:12:52.910465+0000","last_became_peered":"2024-09-10T21:12:52.910465+0000","last_unstale":"2024-09-10T21:12:54.957618+0000","last_undegraded":"2024-09-10T21:12:54.957618+0000","last_fullsized":"2024-09-10T21:12:54.957618+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:05.464342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929412+0000","last_change":"2024-09-10T21:12:53.926389+0000","last_active":"2024-09-10T21:12:54.929412+0000","last_peered":"2024-09-10T21:12:54.929412+0000","last_clean":"2024-09-10T21:12:54.929412+0000","last_became_active":"2024-09-10T21:12:53.926136+0000","last_became_peered":"2024-09-10T21:12:53.926136+0000","last_unstale":"2024-09-10T21:12:54.929412+0000","last_undegraded":"2024-09-10T21:12:54.929412+0000","last_fullsized":"2024-09-10T21:12:54.929412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:01.092653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958487+0000","last_change":"2024-09-10T21:12:52.927712+0000","last_active":"2024-09-10T21:12:54.958487+0000","last_peered":"2024-09-10T21:12:54.958487+0000","last_clean":"2024-09-10T21:12:54.958487+0000","last_became_active":"2024-09-10T21:12:52.927386+0000","last_became_peered":"2024-09-10T21:12:52.927386+0000","last_unstale":"2024-09-10T21:12:54.958487+0000","last_undegraded":"2024-09-10T21:12:54.958487+0000","last_fullsized":"2024-09-10T21:12:54.958487+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:34.638336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929562+0000","last_change":"2024-09-10T21:12:53.925268+0000","last_active":"2024-09-10T21:12:54.929562+0000","last_peered":"2024-09-10T21:12:54.929562+0000","last_clean":"2024-09-10T21:12:54.929562+0000","last_became_active":"2024-09-10T21:12:53.924980+0000","last_became_peered":"2024-09-10T21:12:53.924980+0000","last_unstale":"2024-09-10T21:12:54.929562+0000","last_undegraded":"2024-09-10T21:12:54.929562+0000","last_fullsized":"2024-09-10T21:12:54.929562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:07.220398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"57'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930494+0000","last_change":"2024-09-10T21:12:52.908936+0000","last_active":"2024-09-10T21:12:54.930494+0000","last_peered":"2024-09-10T21:12:54.930494+0000","last_clean":"2024-09-10T21:12:54.930494+0000","last_became_active":"2024-09-10T21:12:52.908661+0000","last_became_peered":"2024-09-10T21:12:52.908661+0000","last_unstale":"2024-09-10T21:12:54.930494+0000","last_undegraded":"2024-09-10T21:12:54.930494+0000","last_fullsized":"2024-09-10T21:12:54.930494+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:07.860598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958110+0000","last_change":"2024-09-10T21:12:53.926960+0000","last_active":"2024-09-10T21:12:54.958110+0000","last_peered":"2024-09-10T21:12:54.958110+0000","last_clean":"2024-09-10T21:12:54.958110+0000","last_became_active":"2024-09-10T21:12:53.926305+0000","last_became_peered":"2024-09-10T21:12:53.926305+0000","last_unstale":"2024-09-10T21:12:54.958110+0000","last_undegraded":"2024-09-10T21:12:54.958110+0000","last_fullsized":"2024-09-10T21:12:54.958110+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:47.334567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957765+0000","last_change":"2024-09-10T21:12:52.911674+0000","last_active":"2024-09-10T21:12:54.957765+0000","last_peered":"2024-09-10T21:12:54.957765+0000","last_clean":"2024-09-10T21:12:54.957765+0000","last_became_active":"2024-09-10T21:12:52.911271+0000","last_became_peered":"2024-09-10T21:12:52.911271+0000","last_unstale":"2024-09-10T21:12:54.957765+0000","last_undegraded":"2024-09-10T21:12:54.957765+0000","last_fullsized":"2024-09-10T21:12:54.957765+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:22.078699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930233+0000","last_change":"2024-09-10T21:12:53.933870+0000","last_active":"2024-09-10T21:12:54.930233+0000","last_peered":"2024-09-10T21:12:54.930233+0000","last_clean":"2024-09-10T21:12:54.930233+0000","last_became_active":"2024-09-10T21:12:53.933440+0000","last_became_peered":"2024-09-10T21:12:53.933440+0000","last_unstale":"2024-09-10T21:12:54.930233+0000","last_undegraded":"2024-09-10T21:12:54.930233+0000","last_fullsized":"2024-09-10T21:12:54.930233+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:08.970278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928860+0000","last_change":"2024-09-10T21:12:52.920768+0000","last_active":"2024-09-10T21:12:54.928860+0000","last_peered":"2024-09-10T21:12:54.928860+0000","last_clean":"2024-09-10T21:12:54.928860+0000","last_became_active":"2024-09-10T21:12:52.918131+0000","last_became_peered":"2024-09-10T21:12:52.918131+0000","last_unstale":"2024-09-10T21:12:54.928860+0000","last_undegraded":"2024-09-10T21:12:54.928860+0000","last_fullsized":"2024-09-10T21:12:54.928860+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:48.097204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929649+0000","last_change":"2024-09-10T21:12:53.949638+0000","last_active":"2024-09-10T21:12:54.929649+0000","last_peered":"2024-09-10T21:12:54.929649+0000","last_clean":"2024-09-10T21:12:54.929649+0000","last_became_active":"2024-09-10T21:12:53.949177+0000","last_became_peered":"2024-09-10T21:12:53.949177+0000","last_unstale":"2024-09-10T21:12:54.929649+0000","last_undegraded":"2024-09-10T21:12:54.929649+0000","last_fullsized":"2024-09-10T21:12:54.929649+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:54.508939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957828+0000","last_change":"2024-09-10T21:12:52.925485+0000","last_active":"2024-09-10T21:12:54.957828+0000","last_peered":"2024-09-10T21:12:54.957828+0000","last_clean":"2024-09-10T21:12:54.957828+0000","last_became_active":"2024-09-10T21:12:52.925196+0000","last_became_peered":"2024-09-10T21:12:52.925196+0000","last_unstale":"2024-09-10T21:12:54.957828+0000","last_undegraded":"2024-09-10T21:12:54.957828+0000","last_fullsized":"2024-09-10T21:12:54.957828+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:08.353885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959065+0000","last_change":"2024-09-10T21:12:53.946489+0000","last_active":"2024-09-10T21:12:54.959065+0000","last_peered":"2024-09-10T21:12:54.959065+0000","last_clean":"2024-09-10T21:12:54.959065+0000","last_became_active":"2024-09-10T21:12:53.945218+0000","last_became_peered":"2024-09-10T21:12:53.945218+0000","last_unstale":"2024-09-10T21:12:54.959065+0000","last_undegraded":"2024-09-10T21:12:54.959065+0000","last_fullsized":"2024-09-10T21:12:54.959065+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:31.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928933+0000","last_change":"2024-09-10T21:12:52.926307+0000","last_active":"2024-09-10T21:12:54.928933+0000","last_peered":"2024-09-10T21:12:54.928933+0000","last_clean":"2024-09-10T21:12:54.928933+0000","last_became_active":"2024-09-10T21:12:52.926021+0000","last_became_peered":"2024-09-10T21:12:52.926021+0000","last_unstale":"2024-09-10T21:12:54.928933+0000","last_undegraded":"2024-09-10T21:12:54.928933+0000","last_fullsized":"2024-09-10T21:12:54.928933+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:53.791545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929471+0000","last_change":"2024-09-10T21:12:53.944529+0000","last_active":"2024-09-10T21:12:54.929471+0000","last_peered":"2024-09-10T21:12:54.929471+0000","last_clean":"2024-09-10T21:12:54.929471+0000","last_became_active":"2024-09-10T21:12:53.944191+0000","last_became_peered":"2024-09-10T21:12:53.944191+0000","last_unstale":"2024-09-10T21:12:54.929471+0000","last_undegraded":"2024-09-10T21:12:54.929471+0000","last_fullsized":"2024-09-10T21:12:54.929471+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:36:46.415277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929112+0000","last_change":"2024-09-10T21:12:52.916297+0000","last_active":"2024-09-10T21:12:54.929112+0000","last_peered":"2024-09-10T21:12:54.929112+0000","last_clean":"2024-09-10T21:12:54.929112+0000","last_became_active":"2024-09-10T21:12:52.914668+0000","last_became_peered":"2024-09-10T21:12:52.914668+0000","last_unstale":"2024-09-10T21:12:54.929112+0000","last_undegraded":"2024-09-10T21:12:54.929112+0000","last_fullsized":"2024-09-10T21:12:54.929112+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:50.985945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957729+0000","last_change":"2024-09-10T21:12:53.942067+0000","last_active":"2024-09-10T21:12:54.957729+0000","last_peered":"2024-09-10T21:12:54.957729+0000","last_clean":"2024-09-10T21:12:54.957729+0000","last_became_active":"2024-09-10T21:12:53.941717+0000","last_became_peered":"2024-09-10T21:12:53.941717+0000","last_unstale":"2024-09-10T21:12:54.957729+0000","last_undegraded":"2024-09-10T21:12:54.957729+0000","last_fullsized":"2024-09-10T21:12:54.957729+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:18.933434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930024+0000","last_change":"2024-09-10T21:12:52.914672+0000","last_active":"2024-09-10T21:12:54.930024+0000","last_peered":"2024-09-10T21:12:54.930024+0000","last_clean":"2024-09-10T21:12:54.930024+0000","last_became_active":"2024-09-10T21:12:52.914355+0000","last_became_peered":"2024-09-10T21:12:52.914355+0000","last_unstale":"2024-09-10T21:12:54.930024+0000","last_undegraded":"2024-09-10T21:12:54.930024+0000","last_fullsized":"2024-09-10T21:12:54.930024+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:44.022375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930226+0000","last_change":"2024-09-10T21:12:53.938346+0000","last_active":"2024-09-10T21:12:54.930226+0000","last_peered":"2024-09-10T21:12:54.930226+0000","last_clean":"2024-09-10T21:12:54.930226+0000","last_became_active":"2024-09-10T21:12:53.938178+0000","last_became_peered":"2024-09-10T21:12:53.938178+0000","last_unstale":"2024-09-10T21:12:54.930226+0000","last_undegraded":"2024-09-10T21:12:54.930226+0000","last_fullsized":"2024-09-10T21:12:54.930226+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:42.283446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958516+0000","last_change":"2024-09-10T21:12:52.922364+0000","last_active":"2024-09-10T21:12:54.958516+0000","last_peered":"2024-09-10T21:12:54.958516+0000","last_clean":"2024-09-10T21:12:54.958516+0000","last_became_active":"2024-09-10T21:12:52.921965+0000","last_became_peered":"2024-09-10T21:12:52.921965+0000","last_unstale":"2024-09-10T21:12:54.958516+0000","last_undegraded":"2024-09-10T21:12:54.958516+0000","last_fullsized":"2024-09-10T21:12:54.958516+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:48.097204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957707+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:12:54.957707+0000","last_peered":"2024-09-10T21:12:54.957707+0000","last_clean":"2024-09-10T21:12:54.957707+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:12:54.957707+0000","last_undegraded":"2024-09-10T21:12:54.957707+0000","last_fullsized":"2024-09-10T21:12:54.957707+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":679936,"data_stored":279622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509477,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1273856,"data_stored":870002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509475,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":684032,"data_stored":280216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509474,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":679936,"data_stored":279690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509476,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":696320,"data_stored":281372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":688128,"data_stored":281212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":684032,"data_stored":279692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1273856,"data_stored":870550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:13:02.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:13:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.860+0000 7fbdcb600640 1 -- 172.21.15.16:0/3421665511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc40d16a0 msgr2=0x7fbdc40715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.860+0000 7fbdcb600640 1 --2- 172.21.15.16:0/3421665511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc40d16a0 0x7fbdc40715b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40b48e0 tx=0x7fbdb001c770 comp rx=0 tx=0).stop 2024-09-10T21:13:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.860+0000 7fbdcb600640 1 -- 172.21.15.16:0/3421665511 shutdown_connections 2024-09-10T21:13:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.860+0000 7fbdcb600640 1 --2- 172.21.15.16:0/3421665511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc40d16a0 0x7fbdc40715b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.860+0000 7fbdcb600640 1 --2- 172.21.15.16:0/3421665511 >> v2:172.21.15.52:3300/0 conn(0x7fbdc415f2e0 0x7fbdc4071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.860+0000 7fbdcb600640 1 -- 172.21.15.16:0/3421665511 >> 172.21.15.16:0/3421665511 conn(0x7fbdc409b130 msgr2=0x7fbdc40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.860+0000 7fbdcb600640 1 -- 172.21.15.16:0/3421665511 shutdown_connections 2024-09-10T21:13:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.860+0000 7fbdcb600640 1 -- 172.21.15.16:0/3421665511 wait complete. 2024-09-10T21:13:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.860+0000 7fbdcb600640 1 Processor -- start 2024-09-10T21:13:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.860+0000 7fbdcb600640 1 -- start start 2024-09-10T21:13:02.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdcb600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc415f2e0 0x7fbdc414eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:02.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdcb600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc415f6c0 0x7fbdc415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:02.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdcb600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbdc415b730 0x7fbdc4153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdcb600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc4169530 con 0x7fbdc415f2e0 2024-09-10T21:13:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdcb600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41693b0 con 0x7fbdc415f6c0 2024-09-10T21:13:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdcb600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc41696b0 con 0x7fbdc415b730 2024-09-10T21:13:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc415f2e0 0x7fbdc414eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc415f2e0 0x7fbdc414eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36956/0 (socket says 172.21.15.16:36956) 2024-09-10T21:13:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc8c00640 1 -- 172.21.15.16:0/3373974839 learned_addr learned my addr 172.21.15.16:0/3373974839 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc8c00640 1 -- 172.21.15.16:0/3373974839 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbdc415b730 msgr2=0x7fbdc4153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc8c00640 1 --2- 172.21.15.16:0/3373974839 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbdc415b730 0x7fbdc4153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc3e00640 1 --2- 172.21.15.16:0/3373974839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc415f6c0 0x7fbdc415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc8c00640 1 -- 172.21.15.16:0/3373974839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc415f6c0 msgr2=0x7fbdc415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc8c00640 1 --2- 172.21.15.16:0/3373974839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc415f6c0 0x7fbdc415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc8c00640 1 -- 172.21.15.16:0/3373974839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc4151410 con 0x7fbdc415f2e0 2024-09-10T21:13:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc8c00640 1 --2- 172.21.15.16:0/3373974839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc415f2e0 0x7fbdc414eb60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800d980 tx=0x7fbdb800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc0c00640 1 -- 172.21.15.16:0/3373974839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8017070 con 0x7fbdc415f2e0 2024-09-10T21:13:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc0c00640 1 -- 172.21.15.16:0/3373974839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdb8009d20 con 0x7fbdc415f2e0 2024-09-10T21:13:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc0c00640 1 -- 172.21.15.16:0/3373974839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80106c0 con 0x7fbdc415f2e0 2024-09-10T21:13:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdcb600640 1 -- 172.21.15.16:0/3373974839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc41f95e0 con 0x7fbdc415f2e0 2024-09-10T21:13:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdcb600640 1 -- 172.21.15.16:0/3373974839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc41f9ba0 con 0x7fbdc415f2e0 2024-09-10T21:13:02.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdcb600640 1 -- 172.21.15.16:0/3373974839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc40cfc80 con 0x7fbdc415f2e0 2024-09-10T21:13:02.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.864+0000 7fbdc0c00640 1 -- 172.21.15.16:0/3373974839 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbdb8010860 con 0x7fbdc415f2e0 2024-09-10T21:13:02.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.868+0000 7fbdc0c00640 1 --2- 172.21.15.16:0/3373974839 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbda40725e0 0x7fbda4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.868+0000 7fbdc0c00640 1 -- 172.21.15.16:0/3373974839 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fbdb80935e0 con 0x7fbdc415f2e0 2024-09-10T21:13:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.868+0000 7fbdc3e00640 1 --2- 172.21.15.16:0/3373974839 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbda40725e0 0x7fbda4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.868+0000 7fbdc3e00640 1 --2- 172.21.15.16:0/3373974839 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbda40725e0 0x7fbda4074aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbdb00af000 tx=0x7fbdb001afb0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:02.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:02.872+0000 7fbdc0c00640 1 -- 172.21.15.16:0/3373974839 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbdb805ed50 con 0x7fbdc415f2e0 2024-09-10T21:13:03.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.040+0000 7fbdcb600640 1 -- 172.21.15.16:0/3373974839 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbdc415b020 con 0x7fbda40725e0 2024-09-10T21:13:03.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.044+0000 7fbdc0c00640 1 -- 172.21.15.16:0/3373974839 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7fbdc415b020 con 0x7fbda40725e0 2024-09-10T21:13:03.055 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:13:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.048+0000 7fbdcb600640 1 -- 172.21.15.16:0/3373974839 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbda40725e0 msgr2=0x7fbda4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.048+0000 7fbdcb600640 1 --2- 172.21.15.16:0/3373974839 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbda40725e0 0x7fbda4074aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbdb00af000 tx=0x7fbdb001afb0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.048+0000 7fbdcb600640 1 -- 172.21.15.16:0/3373974839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc415f2e0 msgr2=0x7fbdc414eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.048+0000 7fbdcb600640 1 --2- 172.21.15.16:0/3373974839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc415f2e0 0x7fbdc414eb60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800d980 tx=0x7fbdb800de40 comp rx=0 tx=0).stop 2024-09-10T21:13:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.048+0000 7fbdcb600640 1 -- 172.21.15.16:0/3373974839 shutdown_connections 2024-09-10T21:13:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.048+0000 7fbdcb600640 1 --2- 172.21.15.16:0/3373974839 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbda40725e0 0x7fbda4074aa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.048+0000 7fbdcb600640 1 --2- 172.21.15.16:0/3373974839 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbdc415b730 0x7fbdc4153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.048+0000 7fbdcb600640 1 --2- 172.21.15.16:0/3373974839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdc415f6c0 0x7fbdc415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.048+0000 7fbdcb600640 1 --2- 172.21.15.16:0/3373974839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc415f2e0 0x7fbdc414eb60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.048+0000 7fbdcb600640 1 -- 172.21.15.16:0/3373974839 >> 172.21.15.16:0/3373974839 conn(0x7fbdc409b130 msgr2=0x7fbdc40b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.048+0000 7fbdcb600640 1 -- 172.21.15.16:0/3373974839 shutdown_connections 2024-09-10T21:13:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.048+0000 7fbdcb600640 1 -- 172.21.15.16:0/3373974839 wait complete. 2024-09-10T21:13:03.077 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-09-10T21:13:01.490799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256060,"kb_used_data":6504,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718660,"statfs":{"total":773094113280,"available":772831907840,"internally_reserved":0,"allocated":6660096,"data_stored":3422356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002013"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929963+0000","last_change":"2024-09-10T21:12:53.959848+0000","last_active":"2024-09-10T21:12:54.929963+0000","last_peered":"2024-09-10T21:12:54.929963+0000","last_clean":"2024-09-10T21:12:54.929963+0000","last_became_active":"2024-09-10T21:12:53.959238+0000","last_became_peered":"2024-09-10T21:12:53.959238+0000","last_unstale":"2024-09-10T21:12:54.929963+0000","last_undegraded":"2024-09-10T21:12:54.929963+0000","last_fullsized":"2024-09-10T21:12:54.929963+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:21.592698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958252+0000","last_change":"2024-09-10T21:12:53.928126+0000","last_active":"2024-09-10T21:12:54.958252+0000","last_peered":"2024-09-10T21:12:54.958252+0000","last_clean":"2024-09-10T21:12:54.958252+0000","last_became_active":"2024-09-10T21:12:53.927596+0000","last_became_peered":"2024-09-10T21:12:53.927596+0000","last_unstale":"2024-09-10T21:12:54.958252+0000","last_undegraded":"2024-09-10T21:12:54.958252+0000","last_fullsized":"2024-09-10T21:12:54.958252+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:05.525627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929071+0000","last_change":"2024-09-10T21:12:53.929639+0000","last_active":"2024-09-10T21:12:54.929071+0000","last_peered":"2024-09-10T21:12:54.929071+0000","last_clean":"2024-09-10T21:12:54.929071+0000","last_became_active":"2024-09-10T21:12:53.929333+0000","last_became_peered":"2024-09-10T21:12:53.929333+0000","last_unstale":"2024-09-10T21:12:54.929071+0000","last_undegraded":"2024-09-10T21:12:54.929071+0000","last_fullsized":"2024-09-10T21:12:54.929071+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:35.276062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959451+0000","last_change":"2024-09-10T21:12:53.932951+0000","last_active":"2024-09-10T21:12:54.959451+0000","last_peered":"2024-09-10T21:12:54.959451+0000","last_clean":"2024-09-10T21:12:54.959451+0000","last_became_active":"2024-09-10T21:12:53.930361+0000","last_became_peered":"2024-09-10T21:12:53.930361+0000","last_unstale":"2024-09-10T21:12:54.959451+0000","last_undegraded":"2024-09-10T21:12:54.959451+0000","last_fullsized":"2024-09-10T21:12:54.959451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:25.811630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929510+0000","last_change":"2024-09-10T21:12:53.951504+0000","last_active":"2024-09-10T21:12:54.929510+0000","last_peered":"2024-09-10T21:12:54.929510+0000","last_clean":"2024-09-10T21:12:54.929510+0000","last_became_active":"2024-09-10T21:12:53.950545+0000","last_became_peered":"2024-09-10T21:12:53.950545+0000","last_unstale":"2024-09-10T21:12:54.929510+0000","last_undegraded":"2024-09-10T21:12:54.929510+0000","last_fullsized":"2024-09-10T21:12:54.929510+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:03.478755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929053+0000","last_change":"2024-09-10T21:12:53.949235+0000","last_active":"2024-09-10T21:12:54.929053+0000","last_peered":"2024-09-10T21:12:54.929053+0000","last_clean":"2024-09-10T21:12:54.929053+0000","last_became_active":"2024-09-10T21:12:53.936152+0000","last_became_peered":"2024-09-10T21:12:53.936152+0000","last_unstale":"2024-09-10T21:12:54.929053+0000","last_undegraded":"2024-09-10T21:12:54.929053+0000","last_fullsized":"2024-09-10T21:12:54.929053+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:09.566010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958816+0000","last_change":"2024-09-10T21:12:53.929021+0000","last_active":"2024-09-10T21:12:54.958816+0000","last_peered":"2024-09-10T21:12:54.958816+0000","last_clean":"2024-09-10T21:12:54.958816+0000","last_became_active":"2024-09-10T21:12:53.928713+0000","last_became_peered":"2024-09-10T21:12:53.928713+0000","last_unstale":"2024-09-10T21:12:54.958816+0000","last_undegraded":"2024-09-10T21:12:54.958816+0000","last_fullsized":"2024-09-10T21:12:54.958816+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:32.138010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928873+0000","last_change":"2024-09-10T21:12:53.934558+0000","last_active":"2024-09-10T21:12:54.928873+0000","last_peered":"2024-09-10T21:12:54.928873+0000","last_clean":"2024-09-10T21:12:54.928873+0000","last_became_active":"2024-09-10T21:12:53.934226+0000","last_became_peered":"2024-09-10T21:12:53.934226+0000","last_unstale":"2024-09-10T21:12:54.928873+0000","last_undegraded":"2024-09-10T21:12:54.928873+0000","last_fullsized":"2024-09-10T21:12:54.928873+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:13.993307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957958+0000","last_change":"2024-09-10T21:12:53.960070+0000","last_active":"2024-09-10T21:12:54.957958+0000","last_peered":"2024-09-10T21:12:54.957958+0000","last_clean":"2024-09-10T21:12:54.957958+0000","last_became_active":"2024-09-10T21:12:53.959848+0000","last_became_peered":"2024-09-10T21:12:53.959848+0000","last_unstale":"2024-09-10T21:12:54.957958+0000","last_undegraded":"2024-09-10T21:12:54.957958+0000","last_fullsized":"2024-09-10T21:12:54.957958+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:46.189946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930056+0000","last_change":"2024-09-10T21:12:53.952820+0000","last_active":"2024-09-10T21:12:54.930056+0000","last_peered":"2024-09-10T21:12:54.930056+0000","last_clean":"2024-09-10T21:12:54.930056+0000","last_became_active":"2024-09-10T21:12:53.952356+0000","last_became_peered":"2024-09-10T21:12:53.952356+0000","last_unstale":"2024-09-10T21:12:54.930056+0000","last_undegraded":"2024-09-10T21:12:54.930056+0000","last_fullsized":"2024-09-10T21:12:54.930056+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:32.544813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957210+0000","last_change":"2024-09-10T21:12:53.957907+0000","last_active":"2024-09-10T21:12:54.957210+0000","last_peered":"2024-09-10T21:12:54.957210+0000","last_clean":"2024-09-10T21:12:54.957210+0000","last_became_active":"2024-09-10T21:12:53.957595+0000","last_became_peered":"2024-09-10T21:12:53.957595+0000","last_unstale":"2024-09-10T21:12:54.957210+0000","last_undegraded":"2024-09-10T21:12:54.957210+0000","last_fullsized":"2024-09-10T21:12:54.957210+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:51.847731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957615+0000","last_change":"2024-09-10T21:12:53.936808+0000","last_active":"2024-09-10T21:12:54.957615+0000","last_peered":"2024-09-10T21:12:54.957615+0000","last_clean":"2024-09-10T21:12:54.957615+0000","last_became_active":"2024-09-10T21:12:53.936441+0000","last_became_peered":"2024-09-10T21:12:53.936441+0000","last_unstale":"2024-09-10T21:12:54.957615+0000","last_undegraded":"2024-09-10T21:12:54.957615+0000","last_fullsized":"2024-09-10T21:12:54.957615+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:30.866160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958673+0000","last_change":"2024-09-10T21:12:53.922722+0000","last_active":"2024-09-10T21:12:54.958673+0000","last_peered":"2024-09-10T21:12:54.958673+0000","last_clean":"2024-09-10T21:12:54.958673+0000","last_became_active":"2024-09-10T21:12:53.922457+0000","last_became_peered":"2024-09-10T21:12:53.922457+0000","last_unstale":"2024-09-10T21:12:54.958673+0000","last_undegraded":"2024-09-10T21:12:54.958673+0000","last_fullsized":"2024-09-10T21:12:54.958673+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:31.129061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928842+0000","last_change":"2024-09-10T21:12:53.945632+0000","last_active":"2024-09-10T21:12:54.928842+0000","last_peered":"2024-09-10T21:12:54.928842+0000","last_clean":"2024-09-10T21:12:54.928842+0000","last_became_active":"2024-09-10T21:12:53.945340+0000","last_became_peered":"2024-09-10T21:12:53.945340+0000","last_unstale":"2024-09-10T21:12:54.928842+0000","last_undegraded":"2024-09-10T21:12:54.928842+0000","last_fullsized":"2024-09-10T21:12:54.928842+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:56.966298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958445+0000","last_change":"2024-09-10T21:12:53.934328+0000","last_active":"2024-09-10T21:12:54.958445+0000","last_peered":"2024-09-10T21:12:54.958445+0000","last_clean":"2024-09-10T21:12:54.958445+0000","last_became_active":"2024-09-10T21:12:53.934057+0000","last_became_peered":"2024-09-10T21:12:53.934057+0000","last_unstale":"2024-09-10T21:12:54.958445+0000","last_undegraded":"2024-09-10T21:12:54.958445+0000","last_fullsized":"2024-09-10T21:12:54.958445+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:43.720965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928963+0000","last_change":"2024-09-10T21:12:53.935764+0000","last_active":"2024-09-10T21:12:54.928963+0000","last_peered":"2024-09-10T21:12:54.928963+0000","last_clean":"2024-09-10T21:12:54.928963+0000","last_became_active":"2024-09-10T21:12:53.935468+0000","last_became_peered":"2024-09-10T21:12:53.935468+0000","last_unstale":"2024-09-10T21:12:54.928963+0000","last_undegraded":"2024-09-10T21:12:54.928963+0000","last_fullsized":"2024-09-10T21:12:54.928963+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:47.640549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930205+0000","last_change":"2024-09-10T21:12:53.953515+0000","last_active":"2024-09-10T21:12:54.930205+0000","last_peered":"2024-09-10T21:12:54.930205+0000","last_clean":"2024-09-10T21:12:54.930205+0000","last_became_active":"2024-09-10T21:12:53.952358+0000","last_became_peered":"2024-09-10T21:12:53.952358+0000","last_unstale":"2024-09-10T21:12:54.930205+0000","last_undegraded":"2024-09-10T21:12:54.930205+0000","last_fullsized":"2024-09-10T21:12:54.930205+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:42.501794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929232+0000","last_change":"2024-09-10T21:12:53.923168+0000","last_active":"2024-09-10T21:12:54.929232+0000","last_peered":"2024-09-10T21:12:54.929232+0000","last_clean":"2024-09-10T21:12:54.929232+0000","last_became_active":"2024-09-10T21:12:53.922878+0000","last_became_peered":"2024-09-10T21:12:53.922878+0000","last_unstale":"2024-09-10T21:12:54.929232+0000","last_undegraded":"2024-09-10T21:12:54.929232+0000","last_fullsized":"2024-09-10T21:12:54.929232+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:55.156529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958104+0000","last_change":"2024-09-10T21:12:53.937960+0000","last_active":"2024-09-10T21:12:54.958104+0000","last_peered":"2024-09-10T21:12:54.958104+0000","last_clean":"2024-09-10T21:12:54.958104+0000","last_became_active":"2024-09-10T21:12:53.936967+0000","last_became_peered":"2024-09-10T21:12:53.936967+0000","last_unstale":"2024-09-10T21:12:54.958104+0000","last_undegraded":"2024-09-10T21:12:54.958104+0000","last_fullsized":"2024-09-10T21:12:54.958104+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:46.189946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957142+0000","last_change":"2024-09-10T21:12:53.959543+0000","last_active":"2024-09-10T21:12:54.957142+0000","last_peered":"2024-09-10T21:12:54.957142+0000","last_clean":"2024-09-10T21:12:54.957142+0000","last_became_active":"2024-09-10T21:12:53.959240+0000","last_became_peered":"2024-09-10T21:12:53.959240+0000","last_unstale":"2024-09-10T21:12:54.957142+0000","last_undegraded":"2024-09-10T21:12:54.957142+0000","last_fullsized":"2024-09-10T21:12:54.957142+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:34.811075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958858+0000","last_change":"2024-09-10T21:12:53.926236+0000","last_active":"2024-09-10T21:12:54.958858+0000","last_peered":"2024-09-10T21:12:54.958858+0000","last_clean":"2024-09-10T21:12:54.958858+0000","last_became_active":"2024-09-10T21:12:53.924195+0000","last_became_peered":"2024-09-10T21:12:53.924195+0000","last_unstale":"2024-09-10T21:12:54.958858+0000","last_undegraded":"2024-09-10T21:12:54.958858+0000","last_fullsized":"2024-09-10T21:12:54.958858+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:28.847293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957079+0000","last_change":"2024-09-10T21:12:53.934083+0000","last_active":"2024-09-10T21:12:54.957079+0000","last_peered":"2024-09-10T21:12:54.957079+0000","last_clean":"2024-09-10T21:12:54.957079+0000","last_became_active":"2024-09-10T21:12:53.933188+0000","last_became_peered":"2024-09-10T21:12:53.933188+0000","last_unstale":"2024-09-10T21:12:54.957079+0000","last_undegraded":"2024-09-10T21:12:54.957079+0000","last_fullsized":"2024-09-10T21:12:54.957079+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:45.028281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958876+0000","last_change":"2024-09-10T21:12:53.920537+0000","last_active":"2024-09-10T21:12:54.958876+0000","last_peered":"2024-09-10T21:12:54.958876+0000","last_clean":"2024-09-10T21:12:54.958876+0000","last_became_active":"2024-09-10T21:12:53.920252+0000","last_became_peered":"2024-09-10T21:12:53.920252+0000","last_unstale":"2024-09-10T21:12:54.958876+0000","last_undegraded":"2024-09-10T21:12:54.958876+0000","last_fullsized":"2024-09-10T21:12:54.958876+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:04.923623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957439+0000","last_change":"2024-09-10T21:12:53.946022+0000","last_active":"2024-09-10T21:12:54.957439+0000","last_peered":"2024-09-10T21:12:54.957439+0000","last_clean":"2024-09-10T21:12:54.957439+0000","last_became_active":"2024-09-10T21:12:53.945751+0000","last_became_peered":"2024-09-10T21:12:53.945751+0000","last_unstale":"2024-09-10T21:12:54.957439+0000","last_undegraded":"2024-09-10T21:12:54.957439+0000","last_fullsized":"2024-09-10T21:12:54.957439+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:24.397204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958359+0000","last_change":"2024-09-10T21:12:53.940509+0000","last_active":"2024-09-10T21:12:54.958359+0000","last_peered":"2024-09-10T21:12:54.958359+0000","last_clean":"2024-09-10T21:12:54.958359+0000","last_became_active":"2024-09-10T21:12:53.940110+0000","last_became_peered":"2024-09-10T21:12:53.940110+0000","last_unstale":"2024-09-10T21:12:54.958359+0000","last_undegraded":"2024-09-10T21:12:54.958359+0000","last_fullsized":"2024-09-10T21:12:54.958359+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:00.280645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958833+0000","last_change":"2024-09-10T21:12:53.923199+0000","last_active":"2024-09-10T21:12:54.958833+0000","last_peered":"2024-09-10T21:12:54.958833+0000","last_clean":"2024-09-10T21:12:54.958833+0000","last_became_active":"2024-09-10T21:12:53.922779+0000","last_became_peered":"2024-09-10T21:12:53.922779+0000","last_unstale":"2024-09-10T21:12:54.958833+0000","last_undegraded":"2024-09-10T21:12:54.958833+0000","last_fullsized":"2024-09-10T21:12:54.958833+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:48.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958350+0000","last_change":"2024-09-10T21:12:53.927601+0000","last_active":"2024-09-10T21:12:54.958350+0000","last_peered":"2024-09-10T21:12:54.958350+0000","last_clean":"2024-09-10T21:12:54.958350+0000","last_became_active":"2024-09-10T21:12:53.927129+0000","last_became_peered":"2024-09-10T21:12:53.927129+0000","last_unstale":"2024-09-10T21:12:54.958350+0000","last_undegraded":"2024-09-10T21:12:54.958350+0000","last_fullsized":"2024-09-10T21:12:54.958350+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:23.222712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930225+0000","last_change":"2024-09-10T21:12:53.939591+0000","last_active":"2024-09-10T21:12:54.930225+0000","last_peered":"2024-09-10T21:12:54.930225+0000","last_clean":"2024-09-10T21:12:54.930225+0000","last_became_active":"2024-09-10T21:12:53.939314+0000","last_became_peered":"2024-09-10T21:12:53.939314+0000","last_unstale":"2024-09-10T21:12:54.930225+0000","last_undegraded":"2024-09-10T21:12:54.930225+0000","last_fullsized":"2024-09-10T21:12:54.930225+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:11.587953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930610+0000","last_change":"2024-09-10T21:12:53.937058+0000","last_active":"2024-09-10T21:12:54.930610+0000","last_peered":"2024-09-10T21:12:54.930610+0000","last_clean":"2024-09-10T21:12:54.930610+0000","last_became_active":"2024-09-10T21:12:53.936798+0000","last_became_peered":"2024-09-10T21:12:53.936798+0000","last_unstale":"2024-09-10T21:12:54.930610+0000","last_undegraded":"2024-09-10T21:12:54.930610+0000","last_fullsized":"2024-09-10T21:12:54.930610+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:14.822315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959093+0000","last_change":"2024-09-10T21:12:53.925270+0000","last_active":"2024-09-10T21:12:54.959093+0000","last_peered":"2024-09-10T21:12:54.959093+0000","last_clean":"2024-09-10T21:12:54.959093+0000","last_became_active":"2024-09-10T21:12:53.923493+0000","last_became_peered":"2024-09-10T21:12:53.923493+0000","last_unstale":"2024-09-10T21:12:54.959093+0000","last_undegraded":"2024-09-10T21:12:54.959093+0000","last_fullsized":"2024-09-10T21:12:54.959093+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:40.051608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957686+0000","last_change":"2024-09-10T21:12:53.932068+0000","last_active":"2024-09-10T21:12:54.957686+0000","last_peered":"2024-09-10T21:12:54.957686+0000","last_clean":"2024-09-10T21:12:54.957686+0000","last_became_active":"2024-09-10T21:12:53.931714+0000","last_became_peered":"2024-09-10T21:12:53.931714+0000","last_unstale":"2024-09-10T21:12:54.957686+0000","last_undegraded":"2024-09-10T21:12:54.957686+0000","last_fullsized":"2024-09-10T21:12:54.957686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:13.024445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.956938+0000","last_change":"2024-09-10T21:12:53.939129+0000","last_active":"2024-09-10T21:12:54.956938+0000","last_peered":"2024-09-10T21:12:54.956938+0000","last_clean":"2024-09-10T21:12:54.956938+0000","last_became_active":"2024-09-10T21:12:53.938807+0000","last_became_peered":"2024-09-10T21:12:53.938807+0000","last_unstale":"2024-09-10T21:12:54.956938+0000","last_undegraded":"2024-09-10T21:12:54.956938+0000","last_fullsized":"2024-09-10T21:12:54.956938+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:03.047971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929664+0000","last_change":"2024-09-10T21:12:53.938689+0000","last_active":"2024-09-10T21:12:54.929664+0000","last_peered":"2024-09-10T21:12:54.929664+0000","last_clean":"2024-09-10T21:12:54.929664+0000","last_became_active":"2024-09-10T21:12:53.935691+0000","last_became_peered":"2024-09-10T21:12:53.935691+0000","last_unstale":"2024-09-10T21:12:54.929664+0000","last_undegraded":"2024-09-10T21:12:54.929664+0000","last_fullsized":"2024-09-10T21:12:54.929664+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:19.453855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":19,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957411+0000","last_change":"2024-09-10T21:12:52.916629+0000","last_active":"2024-09-10T21:12:54.957411+0000","last_peered":"2024-09-10T21:12:54.957411+0000","last_clean":"2024-09-10T21:12:54.957411+0000","last_became_active":"2024-09-10T21:12:52.916091+0000","last_became_peered":"2024-09-10T21:12:52.916091+0000","last_unstale":"2024-09-10T21:12:54.957411+0000","last_undegraded":"2024-09-10T21:12:54.957411+0000","last_fullsized":"2024-09-10T21:12:54.957411+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:27.028802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930407+0000","last_change":"2024-09-10T21:12:53.953827+0000","last_active":"2024-09-10T21:12:54.930407+0000","last_peered":"2024-09-10T21:12:54.930407+0000","last_clean":"2024-09-10T21:12:54.930407+0000","last_became_active":"2024-09-10T21:12:53.953262+0000","last_became_peered":"2024-09-10T21:12:53.953262+0000","last_unstale":"2024-09-10T21:12:54.930407+0000","last_undegraded":"2024-09-10T21:12:54.930407+0000","last_fullsized":"2024-09-10T21:12:54.930407+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:29.044625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958380+0000","last_change":"2024-09-10T21:12:52.926961+0000","last_active":"2024-09-10T21:12:54.958380+0000","last_peered":"2024-09-10T21:12:54.958380+0000","last_clean":"2024-09-10T21:12:54.958380+0000","last_became_active":"2024-09-10T21:12:52.926723+0000","last_became_peered":"2024-09-10T21:12:52.926723+0000","last_unstale":"2024-09-10T21:12:54.958380+0000","last_undegraded":"2024-09-10T21:12:54.958380+0000","last_fullsized":"2024-09-10T21:12:54.958380+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:44.000606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930976+0000","last_change":"2024-09-10T21:12:53.944452+0000","last_active":"2024-09-10T21:12:54.930976+0000","last_peered":"2024-09-10T21:12:54.930976+0000","last_clean":"2024-09-10T21:12:54.930976+0000","last_became_active":"2024-09-10T21:12:53.944142+0000","last_became_peered":"2024-09-10T21:12:53.944142+0000","last_unstale":"2024-09-10T21:12:54.930976+0000","last_undegraded":"2024-09-10T21:12:54.930976+0000","last_fullsized":"2024-09-10T21:12:54.930976+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:27.806946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929380+0000","last_change":"2024-09-10T21:12:52.924953+0000","last_active":"2024-09-10T21:12:54.929380+0000","last_peered":"2024-09-10T21:12:54.929380+0000","last_clean":"2024-09-10T21:12:54.929380+0000","last_became_active":"2024-09-10T21:12:52.924601+0000","last_became_peered":"2024-09-10T21:12:52.924601+0000","last_unstale":"2024-09-10T21:12:54.929380+0000","last_undegraded":"2024-09-10T21:12:54.929380+0000","last_fullsized":"2024-09-10T21:12:54.929380+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:44.842888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929019+0000","last_change":"2024-09-10T21:12:53.950957+0000","last_active":"2024-09-10T21:12:54.929019+0000","last_peered":"2024-09-10T21:12:54.929019+0000","last_clean":"2024-09-10T21:12:54.929019+0000","last_became_active":"2024-09-10T21:12:53.950673+0000","last_became_peered":"2024-09-10T21:12:53.950673+0000","last_unstale":"2024-09-10T21:12:54.929019+0000","last_undegraded":"2024-09-10T21:12:54.929019+0000","last_fullsized":"2024-09-10T21:12:54.929019+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:27.782259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959533+0000","last_change":"2024-09-10T21:12:52.920769+0000","last_active":"2024-09-10T21:12:54.959533+0000","last_peered":"2024-09-10T21:12:54.959533+0000","last_clean":"2024-09-10T21:12:54.959533+0000","last_became_active":"2024-09-10T21:12:52.920460+0000","last_became_peered":"2024-09-10T21:12:52.920460+0000","last_unstale":"2024-09-10T21:12:54.959533+0000","last_undegraded":"2024-09-10T21:12:54.959533+0000","last_fullsized":"2024-09-10T21:12:54.959533+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:53.192838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930430+0000","last_change":"2024-09-10T21:12:53.946296+0000","last_active":"2024-09-10T21:12:54.930430+0000","last_peered":"2024-09-10T21:12:54.930430+0000","last_clean":"2024-09-10T21:12:54.930430+0000","last_became_active":"2024-09-10T21:12:53.945868+0000","last_became_peered":"2024-09-10T21:12:53.945868+0000","last_unstale":"2024-09-10T21:12:54.930430+0000","last_undegraded":"2024-09-10T21:12:54.930430+0000","last_fullsized":"2024-09-10T21:12:54.930430+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:52.681542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929006+0000","last_change":"2024-09-10T21:12:52.917429+0000","last_active":"2024-09-10T21:12:54.929006+0000","last_peered":"2024-09-10T21:12:54.929006+0000","last_clean":"2024-09-10T21:12:54.929006+0000","last_became_active":"2024-09-10T21:12:52.917148+0000","last_became_peered":"2024-09-10T21:12:52.917148+0000","last_unstale":"2024-09-10T21:12:54.929006+0000","last_undegraded":"2024-09-10T21:12:54.929006+0000","last_fullsized":"2024-09-10T21:12:54.929006+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:05.034728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957049+0000","last_change":"2024-09-10T21:12:53.947773+0000","last_active":"2024-09-10T21:12:54.957049+0000","last_peered":"2024-09-10T21:12:54.957049+0000","last_clean":"2024-09-10T21:12:54.957049+0000","last_became_active":"2024-09-10T21:12:53.947473+0000","last_became_peered":"2024-09-10T21:12:53.947473+0000","last_unstale":"2024-09-10T21:12:54.957049+0000","last_undegraded":"2024-09-10T21:12:54.957049+0000","last_fullsized":"2024-09-10T21:12:54.957049+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:09.615040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929858+0000","last_change":"2024-09-10T21:12:52.907684+0000","last_active":"2024-09-10T21:12:54.929858+0000","last_peered":"2024-09-10T21:12:54.929858+0000","last_clean":"2024-09-10T21:12:54.929858+0000","last_became_active":"2024-09-10T21:12:52.907395+0000","last_became_peered":"2024-09-10T21:12:52.907395+0000","last_unstale":"2024-09-10T21:12:54.929858+0000","last_undegraded":"2024-09-10T21:12:54.929858+0000","last_fullsized":"2024-09-10T21:12:54.929858+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:19.885587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928722+0000","last_change":"2024-09-10T21:12:53.934723+0000","last_active":"2024-09-10T21:12:54.928722+0000","last_peered":"2024-09-10T21:12:54.928722+0000","last_clean":"2024-09-10T21:12:54.928722+0000","last_became_active":"2024-09-10T21:12:53.932327+0000","last_became_peered":"2024-09-10T21:12:53.932327+0000","last_unstale":"2024-09-10T21:12:54.928722+0000","last_undegraded":"2024-09-10T21:12:54.928722+0000","last_fullsized":"2024-09-10T21:12:54.928722+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:17.710007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957534+0000","last_change":"2024-09-10T21:12:52.922523+0000","last_active":"2024-09-10T21:12:54.957534+0000","last_peered":"2024-09-10T21:12:54.957534+0000","last_clean":"2024-09-10T21:12:54.957534+0000","last_became_active":"2024-09-10T21:12:52.922266+0000","last_became_peered":"2024-09-10T21:12:52.922266+0000","last_unstale":"2024-09-10T21:12:54.957534+0000","last_undegraded":"2024-09-10T21:12:54.957534+0000","last_fullsized":"2024-09-10T21:12:54.957534+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:01.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928915+0000","last_change":"2024-09-10T21:12:53.951739+0000","last_active":"2024-09-10T21:12:54.928915+0000","last_peered":"2024-09-10T21:12:54.928915+0000","last_clean":"2024-09-10T21:12:54.928915+0000","last_became_active":"2024-09-10T21:12:53.943111+0000","last_became_peered":"2024-09-10T21:12:53.943111+0000","last_unstale":"2024-09-10T21:12:54.928915+0000","last_undegraded":"2024-09-10T21:12:54.928915+0000","last_fullsized":"2024-09-10T21:12:54.928915+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:00.034619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958276+0000","last_change":"2024-09-10T21:12:52.925890+0000","last_active":"2024-09-10T21:12:54.958276+0000","last_peered":"2024-09-10T21:12:54.958276+0000","last_clean":"2024-09-10T21:12:54.958276+0000","last_became_active":"2024-09-10T21:12:52.925061+0000","last_became_peered":"2024-09-10T21:12:52.925061+0000","last_unstale":"2024-09-10T21:12:54.958276+0000","last_undegraded":"2024-09-10T21:12:54.958276+0000","last_fullsized":"2024-09-10T21:12:54.958276+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:53.791545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959064+0000","last_change":"2024-09-10T21:12:53.947582+0000","last_active":"2024-09-10T21:12:54.959064+0000","last_peered":"2024-09-10T21:12:54.959064+0000","last_clean":"2024-09-10T21:12:54.959064+0000","last_became_active":"2024-09-10T21:12:53.947316+0000","last_became_peered":"2024-09-10T21:12:53.947316+0000","last_unstale":"2024-09-10T21:12:54.959064+0000","last_undegraded":"2024-09-10T21:12:54.959064+0000","last_fullsized":"2024-09-10T21:12:54.959064+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:52.259847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957862+0000","last_change":"2024-09-10T21:12:52.920456+0000","last_active":"2024-09-10T21:12:54.957862+0000","last_peered":"2024-09-10T21:12:54.957862+0000","last_clean":"2024-09-10T21:12:54.957862+0000","last_became_active":"2024-09-10T21:12:52.920172+0000","last_became_peered":"2024-09-10T21:12:52.920172+0000","last_unstale":"2024-09-10T21:12:54.957862+0000","last_undegraded":"2024-09-10T21:12:54.957862+0000","last_fullsized":"2024-09-10T21:12:54.957862+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:51.616323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929355+0000","last_change":"2024-09-10T21:12:53.945665+0000","last_active":"2024-09-10T21:12:54.929355+0000","last_peered":"2024-09-10T21:12:54.929355+0000","last_clean":"2024-09-10T21:12:54.929355+0000","last_became_active":"2024-09-10T21:12:53.945387+0000","last_became_peered":"2024-09-10T21:12:53.945387+0000","last_unstale":"2024-09-10T21:12:54.929355+0000","last_undegraded":"2024-09-10T21:12:54.929355+0000","last_fullsized":"2024-09-10T21:12:54.929355+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:31.090679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930717+0000","last_change":"2024-09-10T21:12:52.921420+0000","last_active":"2024-09-10T21:12:54.930717+0000","last_peered":"2024-09-10T21:12:54.930717+0000","last_clean":"2024-09-10T21:12:54.930717+0000","last_became_active":"2024-09-10T21:12:52.920194+0000","last_became_peered":"2024-09-10T21:12:52.920194+0000","last_unstale":"2024-09-10T21:12:54.930717+0000","last_undegraded":"2024-09-10T21:12:54.930717+0000","last_fullsized":"2024-09-10T21:12:54.930717+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:02.042065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.931041+0000","last_change":"2024-09-10T21:12:53.959498+0000","last_active":"2024-09-10T21:12:54.931041+0000","last_peered":"2024-09-10T21:12:54.931041+0000","last_clean":"2024-09-10T21:12:54.931041+0000","last_became_active":"2024-09-10T21:12:53.959157+0000","last_became_peered":"2024-09-10T21:12:53.959157+0000","last_unstale":"2024-09-10T21:12:54.931041+0000","last_undegraded":"2024-09-10T21:12:54.931041+0000","last_fullsized":"2024-09-10T21:12:54.931041+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:11.561518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959620+0000","last_change":"2024-09-10T21:12:52.920836+0000","last_active":"2024-09-10T21:12:54.959620+0000","last_peered":"2024-09-10T21:12:54.959620+0000","last_clean":"2024-09-10T21:12:54.959620+0000","last_became_active":"2024-09-10T21:12:52.920547+0000","last_became_peered":"2024-09-10T21:12:52.920547+0000","last_unstale":"2024-09-10T21:12:54.959620+0000","last_undegraded":"2024-09-10T21:12:54.959620+0000","last_fullsized":"2024-09-10T21:12:54.959620+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:22.866900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959452+0000","last_change":"2024-09-10T21:12:53.936019+0000","last_active":"2024-09-10T21:12:54.959452+0000","last_peered":"2024-09-10T21:12:54.959452+0000","last_clean":"2024-09-10T21:12:54.959452+0000","last_became_active":"2024-09-10T21:12:53.935728+0000","last_became_peered":"2024-09-10T21:12:53.935728+0000","last_unstale":"2024-09-10T21:12:54.959452+0000","last_undegraded":"2024-09-10T21:12:54.959452+0000","last_fullsized":"2024-09-10T21:12:54.959452+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:32.220970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":21,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958482+0000","last_change":"2024-09-10T21:12:52.924581+0000","last_active":"2024-09-10T21:12:54.958482+0000","last_peered":"2024-09-10T21:12:54.958482+0000","last_clean":"2024-09-10T21:12:54.958482+0000","last_became_active":"2024-09-10T21:12:52.924327+0000","last_became_peered":"2024-09-10T21:12:52.924327+0000","last_unstale":"2024-09-10T21:12:54.958482+0000","last_undegraded":"2024-09-10T21:12:54.958482+0000","last_fullsized":"2024-09-10T21:12:54.958482+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:10.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957565+0000","last_change":"2024-09-10T21:12:53.935986+0000","last_active":"2024-09-10T21:12:54.957565+0000","last_peered":"2024-09-10T21:12:54.957565+0000","last_clean":"2024-09-10T21:12:54.957565+0000","last_became_active":"2024-09-10T21:12:53.935361+0000","last_became_peered":"2024-09-10T21:12:53.935361+0000","last_unstale":"2024-09-10T21:12:54.957565+0000","last_undegraded":"2024-09-10T21:12:54.957565+0000","last_fullsized":"2024-09-10T21:12:54.957565+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:42.454264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930802+0000","last_change":"2024-09-10T21:12:52.919371+0000","last_active":"2024-09-10T21:12:54.930802+0000","last_peered":"2024-09-10T21:12:54.930802+0000","last_clean":"2024-09-10T21:12:54.930802+0000","last_became_active":"2024-09-10T21:12:52.918995+0000","last_became_peered":"2024-09-10T21:12:52.918995+0000","last_unstale":"2024-09-10T21:12:54.930802+0000","last_undegraded":"2024-09-10T21:12:54.930802+0000","last_fullsized":"2024-09-10T21:12:54.930802+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:33.805170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928901+0000","last_change":"2024-09-10T21:12:53.930651+0000","last_active":"2024-09-10T21:12:54.928901+0000","last_peered":"2024-09-10T21:12:54.928901+0000","last_clean":"2024-09-10T21:12:54.928901+0000","last_became_active":"2024-09-10T21:12:53.930387+0000","last_became_peered":"2024-09-10T21:12:53.930387+0000","last_unstale":"2024-09-10T21:12:54.928901+0000","last_undegraded":"2024-09-10T21:12:54.928901+0000","last_fullsized":"2024-09-10T21:12:54.928901+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:32.429787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959240+0000","last_change":"2024-09-10T21:12:52.923255+0000","last_active":"2024-09-10T21:12:54.959240+0000","last_peered":"2024-09-10T21:12:54.959240+0000","last_clean":"2024-09-10T21:12:54.959240+0000","last_became_active":"2024-09-10T21:12:52.919965+0000","last_became_peered":"2024-09-10T21:12:52.919965+0000","last_unstale":"2024-09-10T21:12:54.959240+0000","last_undegraded":"2024-09-10T21:12:54.959240+0000","last_fullsized":"2024-09-10T21:12:54.959240+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:15.274251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957533+0000","last_change":"2024-09-10T21:12:53.933395+0000","last_active":"2024-09-10T21:12:54.957533+0000","last_peered":"2024-09-10T21:12:54.957533+0000","last_clean":"2024-09-10T21:12:54.957533+0000","last_became_active":"2024-09-10T21:12:53.933116+0000","last_became_peered":"2024-09-10T21:12:53.933116+0000","last_unstale":"2024-09-10T21:12:54.957533+0000","last_undegraded":"2024-09-10T21:12:54.957533+0000","last_fullsized":"2024-09-10T21:12:54.957533+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:25.230192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959046+0000","last_change":"2024-09-10T21:12:52.923218+0000","last_active":"2024-09-10T21:12:54.959046+0000","last_peered":"2024-09-10T21:12:54.959046+0000","last_clean":"2024-09-10T21:12:54.959046+0000","last_became_active":"2024-09-10T21:12:52.919955+0000","last_became_peered":"2024-09-10T21:12:52.919955+0000","last_unstale":"2024-09-10T21:12:54.959046+0000","last_undegraded":"2024-09-10T21:12:54.959046+0000","last_fullsized":"2024-09-10T21:12:54.959046+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:53.792649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959072+0000","last_change":"2024-09-10T21:12:53.922279+0000","last_active":"2024-09-10T21:12:54.959072+0000","last_peered":"2024-09-10T21:12:54.959072+0000","last_clean":"2024-09-10T21:12:54.959072+0000","last_became_active":"2024-09-10T21:12:53.921900+0000","last_became_peered":"2024-09-10T21:12:53.921900+0000","last_unstale":"2024-09-10T21:12:54.959072+0000","last_undegraded":"2024-09-10T21:12:54.959072+0000","last_fullsized":"2024-09-10T21:12:54.959072+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:56.649446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":23,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958890+0000","last_change":"2024-09-10T21:12:52.920975+0000","last_active":"2024-09-10T21:12:54.958890+0000","last_peered":"2024-09-10T21:12:54.958890+0000","last_clean":"2024-09-10T21:12:54.958890+0000","last_became_active":"2024-09-10T21:12:52.919980+0000","last_became_peered":"2024-09-10T21:12:52.919980+0000","last_unstale":"2024-09-10T21:12:54.958890+0000","last_undegraded":"2024-09-10T21:12:54.958890+0000","last_fullsized":"2024-09-10T21:12:54.958890+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:28.166375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929206+0000","last_change":"2024-09-10T21:12:53.949707+0000","last_active":"2024-09-10T21:12:54.929206+0000","last_peered":"2024-09-10T21:12:54.929206+0000","last_clean":"2024-09-10T21:12:54.929206+0000","last_became_active":"2024-09-10T21:12:53.949438+0000","last_became_peered":"2024-09-10T21:12:53.949438+0000","last_unstale":"2024-09-10T21:12:54.929206+0000","last_undegraded":"2024-09-10T21:12:54.929206+0000","last_fullsized":"2024-09-10T21:12:54.929206+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:06.481982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958468+0000","last_change":"2024-09-10T21:12:52.928329+0000","last_active":"2024-09-10T21:12:54.958468+0000","last_peered":"2024-09-10T21:12:54.958468+0000","last_clean":"2024-09-10T21:12:54.958468+0000","last_became_active":"2024-09-10T21:12:52.928026+0000","last_became_peered":"2024-09-10T21:12:52.928026+0000","last_unstale":"2024-09-10T21:12:54.958468+0000","last_undegraded":"2024-09-10T21:12:54.958468+0000","last_fullsized":"2024-09-10T21:12:54.958468+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:11.221179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929423+0000","last_change":"2024-09-10T21:12:53.946213+0000","last_active":"2024-09-10T21:12:54.929423+0000","last_peered":"2024-09-10T21:12:54.929423+0000","last_clean":"2024-09-10T21:12:54.929423+0000","last_became_active":"2024-09-10T21:12:53.945887+0000","last_became_peered":"2024-09-10T21:12:53.945887+0000","last_unstale":"2024-09-10T21:12:54.929423+0000","last_undegraded":"2024-09-10T21:12:54.929423+0000","last_fullsized":"2024-09-10T21:12:54.929423+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:29.514818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930850+0000","last_change":"2024-09-10T21:12:52.920779+0000","last_active":"2024-09-10T21:12:54.930850+0000","last_peered":"2024-09-10T21:12:54.930850+0000","last_clean":"2024-09-10T21:12:54.930850+0000","last_became_active":"2024-09-10T21:12:52.920513+0000","last_became_peered":"2024-09-10T21:12:52.920513+0000","last_unstale":"2024-09-10T21:12:54.930850+0000","last_undegraded":"2024-09-10T21:12:54.930850+0000","last_fullsized":"2024-09-10T21:12:54.930850+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:18.447950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958095+0000","last_change":"2024-09-10T21:12:53.926597+0000","last_active":"2024-09-10T21:12:54.958095+0000","last_peered":"2024-09-10T21:12:54.958095+0000","last_clean":"2024-09-10T21:12:54.958095+0000","last_became_active":"2024-09-10T21:12:53.925894+0000","last_became_peered":"2024-09-10T21:12:53.925894+0000","last_unstale":"2024-09-10T21:12:54.958095+0000","last_undegraded":"2024-09-10T21:12:54.958095+0000","last_fullsized":"2024-09-10T21:12:54.958095+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:46.459101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929285+0000","last_change":"2024-09-10T21:12:52.925824+0000","last_active":"2024-09-10T21:12:54.929285+0000","last_peered":"2024-09-10T21:12:54.929285+0000","last_clean":"2024-09-10T21:12:54.929285+0000","last_became_active":"2024-09-10T21:12:52.925229+0000","last_became_peered":"2024-09-10T21:12:52.925229+0000","last_unstale":"2024-09-10T21:12:54.929285+0000","last_undegraded":"2024-09-10T21:12:54.929285+0000","last_fullsized":"2024-09-10T21:12:54.929285+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:10.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959532+0000","last_change":"2024-09-10T21:12:53.930750+0000","last_active":"2024-09-10T21:12:54.959532+0000","last_peered":"2024-09-10T21:12:54.959532+0000","last_clean":"2024-09-10T21:12:54.959532+0000","last_became_active":"2024-09-10T21:12:53.930443+0000","last_became_peered":"2024-09-10T21:12:53.930443+0000","last_unstale":"2024-09-10T21:12:54.959532+0000","last_undegraded":"2024-09-10T21:12:54.959532+0000","last_fullsized":"2024-09-10T21:12:54.959532+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:06.203980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.931151+0000","last_change":"2024-09-10T21:12:52.913454+0000","last_active":"2024-09-10T21:12:54.931151+0000","last_peered":"2024-09-10T21:12:54.931151+0000","last_clean":"2024-09-10T21:12:54.931151+0000","last_became_active":"2024-09-10T21:12:52.913160+0000","last_became_peered":"2024-09-10T21:12:52.913160+0000","last_unstale":"2024-09-10T21:12:54.931151+0000","last_undegraded":"2024-09-10T21:12:54.931151+0000","last_fullsized":"2024-09-10T21:12:54.931151+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:35.717760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957784+0000","last_change":"2024-09-10T21:12:53.960736+0000","last_active":"2024-09-10T21:12:54.957784+0000","last_peered":"2024-09-10T21:12:54.957784+0000","last_clean":"2024-09-10T21:12:54.957784+0000","last_became_active":"2024-09-10T21:12:53.960493+0000","last_became_peered":"2024-09-10T21:12:53.960493+0000","last_unstale":"2024-09-10T21:12:54.957784+0000","last_undegraded":"2024-09-10T21:12:54.957784+0000","last_fullsized":"2024-09-10T21:12:54.957784+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:43.293392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957171+0000","last_change":"2024-09-10T21:12:52.919813+0000","last_active":"2024-09-10T21:12:54.957171+0000","last_peered":"2024-09-10T21:12:54.957171+0000","last_clean":"2024-09-10T21:12:54.957171+0000","last_became_active":"2024-09-10T21:12:52.919373+0000","last_became_peered":"2024-09-10T21:12:52.919373+0000","last_unstale":"2024-09-10T21:12:54.957171+0000","last_undegraded":"2024-09-10T21:12:54.957171+0000","last_fullsized":"2024-09-10T21:12:54.957171+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:25.283722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930686+0000","last_change":"2024-09-10T21:12:53.937502+0000","last_active":"2024-09-10T21:12:54.930686+0000","last_peered":"2024-09-10T21:12:54.930686+0000","last_clean":"2024-09-10T21:12:54.930686+0000","last_became_active":"2024-09-10T21:12:53.937201+0000","last_became_peered":"2024-09-10T21:12:53.937201+0000","last_unstale":"2024-09-10T21:12:54.930686+0000","last_undegraded":"2024-09-10T21:12:54.930686+0000","last_fullsized":"2024-09-10T21:12:54.930686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:36.394239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930603+0000","last_change":"2024-09-10T21:12:52.920031+0000","last_active":"2024-09-10T21:12:54.930603+0000","last_peered":"2024-09-10T21:12:54.930603+0000","last_clean":"2024-09-10T21:12:54.930603+0000","last_became_active":"2024-09-10T21:12:52.919769+0000","last_became_peered":"2024-09-10T21:12:52.919769+0000","last_unstale":"2024-09-10T21:12:54.930603+0000","last_undegraded":"2024-09-10T21:12:54.930603+0000","last_fullsized":"2024-09-10T21:12:54.930603+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:08.609135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929907+0000","last_change":"2024-09-10T21:12:53.928673+0000","last_active":"2024-09-10T21:12:54.929907+0000","last_peered":"2024-09-10T21:12:54.929907+0000","last_clean":"2024-09-10T21:12:54.929907+0000","last_became_active":"2024-09-10T21:12:53.928390+0000","last_became_peered":"2024-09-10T21:12:53.928390+0000","last_unstale":"2024-09-10T21:12:54.929907+0000","last_undegraded":"2024-09-10T21:12:54.929907+0000","last_fullsized":"2024-09-10T21:12:54.929907+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:12.631625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957618+0000","last_change":"2024-09-10T21:12:52.910741+0000","last_active":"2024-09-10T21:12:54.957618+0000","last_peered":"2024-09-10T21:12:54.957618+0000","last_clean":"2024-09-10T21:12:54.957618+0000","last_became_active":"2024-09-10T21:12:52.910465+0000","last_became_peered":"2024-09-10T21:12:52.910465+0000","last_unstale":"2024-09-10T21:12:54.957618+0000","last_undegraded":"2024-09-10T21:12:54.957618+0000","last_fullsized":"2024-09-10T21:12:54.957618+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:05.464342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929412+0000","last_change":"2024-09-10T21:12:53.926389+0000","last_active":"2024-09-10T21:12:54.929412+0000","last_peered":"2024-09-10T21:12:54.929412+0000","last_clean":"2024-09-10T21:12:54.929412+0000","last_became_active":"2024-09-10T21:12:53.926136+0000","last_became_peered":"2024-09-10T21:12:53.926136+0000","last_unstale":"2024-09-10T21:12:54.929412+0000","last_undegraded":"2024-09-10T21:12:54.929412+0000","last_fullsized":"2024-09-10T21:12:54.929412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:01.092653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958487+0000","last_change":"2024-09-10T21:12:52.927712+0000","last_active":"2024-09-10T21:12:54.958487+0000","last_peered":"2024-09-10T21:12:54.958487+0000","last_clean":"2024-09-10T21:12:54.958487+0000","last_became_active":"2024-09-10T21:12:52.927386+0000","last_became_peered":"2024-09-10T21:12:52.927386+0000","last_unstale":"2024-09-10T21:12:54.958487+0000","last_undegraded":"2024-09-10T21:12:54.958487+0000","last_fullsized":"2024-09-10T21:12:54.958487+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:34.638336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929562+0000","last_change":"2024-09-10T21:12:53.925268+0000","last_active":"2024-09-10T21:12:54.929562+0000","last_peered":"2024-09-10T21:12:54.929562+0000","last_clean":"2024-09-10T21:12:54.929562+0000","last_became_active":"2024-09-10T21:12:53.924980+0000","last_became_peered":"2024-09-10T21:12:53.924980+0000","last_unstale":"2024-09-10T21:12:54.929562+0000","last_undegraded":"2024-09-10T21:12:54.929562+0000","last_fullsized":"2024-09-10T21:12:54.929562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:07.220398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"57'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930494+0000","last_change":"2024-09-10T21:12:52.908936+0000","last_active":"2024-09-10T21:12:54.930494+0000","last_peered":"2024-09-10T21:12:54.930494+0000","last_clean":"2024-09-10T21:12:54.930494+0000","last_became_active":"2024-09-10T21:12:52.908661+0000","last_became_peered":"2024-09-10T21:12:52.908661+0000","last_unstale":"2024-09-10T21:12:54.930494+0000","last_undegraded":"2024-09-10T21:12:54.930494+0000","last_fullsized":"2024-09-10T21:12:54.930494+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:07.860598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958110+0000","last_change":"2024-09-10T21:12:53.926960+0000","last_active":"2024-09-10T21:12:54.958110+0000","last_peered":"2024-09-10T21:12:54.958110+0000","last_clean":"2024-09-10T21:12:54.958110+0000","last_became_active":"2024-09-10T21:12:53.926305+0000","last_became_peered":"2024-09-10T21:12:53.926305+0000","last_unstale":"2024-09-10T21:12:54.958110+0000","last_undegraded":"2024-09-10T21:12:54.958110+0000","last_fullsized":"2024-09-10T21:12:54.958110+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:47.334567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957765+0000","last_change":"2024-09-10T21:12:52.911674+0000","last_active":"2024-09-10T21:12:54.957765+0000","last_peered":"2024-09-10T21:12:54.957765+0000","last_clean":"2024-09-10T21:12:54.957765+0000","last_became_active":"2024-09-10T21:12:52.911271+0000","last_became_peered":"2024-09-10T21:12:52.911271+0000","last_unstale":"2024-09-10T21:12:54.957765+0000","last_undegraded":"2024-09-10T21:12:54.957765+0000","last_fullsized":"2024-09-10T21:12:54.957765+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:22.078699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930233+0000","last_change":"2024-09-10T21:12:53.933870+0000","last_active":"2024-09-10T21:12:54.930233+0000","last_peered":"2024-09-10T21:12:54.930233+0000","last_clean":"2024-09-10T21:12:54.930233+0000","last_became_active":"2024-09-10T21:12:53.933440+0000","last_became_peered":"2024-09-10T21:12:53.933440+0000","last_unstale":"2024-09-10T21:12:54.930233+0000","last_undegraded":"2024-09-10T21:12:54.930233+0000","last_fullsized":"2024-09-10T21:12:54.930233+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:08.970278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928860+0000","last_change":"2024-09-10T21:12:52.920768+0000","last_active":"2024-09-10T21:12:54.928860+0000","last_peered":"2024-09-10T21:12:54.928860+0000","last_clean":"2024-09-10T21:12:54.928860+0000","last_became_active":"2024-09-10T21:12:52.918131+0000","last_became_peered":"2024-09-10T21:12:52.918131+0000","last_unstale":"2024-09-10T21:12:54.928860+0000","last_undegraded":"2024-09-10T21:12:54.928860+0000","last_fullsized":"2024-09-10T21:12:54.928860+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:48.097204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929649+0000","last_change":"2024-09-10T21:12:53.949638+0000","last_active":"2024-09-10T21:12:54.929649+0000","last_peered":"2024-09-10T21:12:54.929649+0000","last_clean":"2024-09-10T21:12:54.929649+0000","last_became_active":"2024-09-10T21:12:53.949177+0000","last_became_peered":"2024-09-10T21:12:53.949177+0000","last_unstale":"2024-09-10T21:12:54.929649+0000","last_undegraded":"2024-09-10T21:12:54.929649+0000","last_fullsized":"2024-09-10T21:12:54.929649+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:54.508939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957828+0000","last_change":"2024-09-10T21:12:52.925485+0000","last_active":"2024-09-10T21:12:54.957828+0000","last_peered":"2024-09-10T21:12:54.957828+0000","last_clean":"2024-09-10T21:12:54.957828+0000","last_became_active":"2024-09-10T21:12:52.925196+0000","last_became_peered":"2024-09-10T21:12:52.925196+0000","last_unstale":"2024-09-10T21:12:54.957828+0000","last_undegraded":"2024-09-10T21:12:54.957828+0000","last_fullsized":"2024-09-10T21:12:54.957828+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:08.353885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.959065+0000","last_change":"2024-09-10T21:12:53.946489+0000","last_active":"2024-09-10T21:12:54.959065+0000","last_peered":"2024-09-10T21:12:54.959065+0000","last_clean":"2024-09-10T21:12:54.959065+0000","last_became_active":"2024-09-10T21:12:53.945218+0000","last_became_peered":"2024-09-10T21:12:53.945218+0000","last_unstale":"2024-09-10T21:12:54.959065+0000","last_undegraded":"2024-09-10T21:12:54.959065+0000","last_fullsized":"2024-09-10T21:12:54.959065+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:31.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.928933+0000","last_change":"2024-09-10T21:12:52.926307+0000","last_active":"2024-09-10T21:12:54.928933+0000","last_peered":"2024-09-10T21:12:54.928933+0000","last_clean":"2024-09-10T21:12:54.928933+0000","last_became_active":"2024-09-10T21:12:52.926021+0000","last_became_peered":"2024-09-10T21:12:52.926021+0000","last_unstale":"2024-09-10T21:12:54.928933+0000","last_undegraded":"2024-09-10T21:12:54.928933+0000","last_fullsized":"2024-09-10T21:12:54.928933+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:53.791545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929471+0000","last_change":"2024-09-10T21:12:53.944529+0000","last_active":"2024-09-10T21:12:54.929471+0000","last_peered":"2024-09-10T21:12:54.929471+0000","last_clean":"2024-09-10T21:12:54.929471+0000","last_became_active":"2024-09-10T21:12:53.944191+0000","last_became_peered":"2024-09-10T21:12:53.944191+0000","last_unstale":"2024-09-10T21:12:54.929471+0000","last_undegraded":"2024-09-10T21:12:54.929471+0000","last_fullsized":"2024-09-10T21:12:54.929471+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:36:46.415277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.929112+0000","last_change":"2024-09-10T21:12:52.916297+0000","last_active":"2024-09-10T21:12:54.929112+0000","last_peered":"2024-09-10T21:12:54.929112+0000","last_clean":"2024-09-10T21:12:54.929112+0000","last_became_active":"2024-09-10T21:12:52.914668+0000","last_became_peered":"2024-09-10T21:12:52.914668+0000","last_unstale":"2024-09-10T21:12:54.929112+0000","last_undegraded":"2024-09-10T21:12:54.929112+0000","last_fullsized":"2024-09-10T21:12:54.929112+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:50.985945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957729+0000","last_change":"2024-09-10T21:12:53.942067+0000","last_active":"2024-09-10T21:12:54.957729+0000","last_peered":"2024-09-10T21:12:54.957729+0000","last_clean":"2024-09-10T21:12:54.957729+0000","last_became_active":"2024-09-10T21:12:53.941717+0000","last_became_peered":"2024-09-10T21:12:53.941717+0000","last_unstale":"2024-09-10T21:12:54.957729+0000","last_undegraded":"2024-09-10T21:12:54.957729+0000","last_fullsized":"2024-09-10T21:12:54.957729+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:18.933434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930024+0000","last_change":"2024-09-10T21:12:52.914672+0000","last_active":"2024-09-10T21:12:54.930024+0000","last_peered":"2024-09-10T21:12:54.930024+0000","last_clean":"2024-09-10T21:12:54.930024+0000","last_became_active":"2024-09-10T21:12:52.914355+0000","last_became_peered":"2024-09-10T21:12:52.914355+0000","last_unstale":"2024-09-10T21:12:54.930024+0000","last_undegraded":"2024-09-10T21:12:54.930024+0000","last_fullsized":"2024-09-10T21:12:54.930024+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:44.022375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.930226+0000","last_change":"2024-09-10T21:12:53.938346+0000","last_active":"2024-09-10T21:12:54.930226+0000","last_peered":"2024-09-10T21:12:54.930226+0000","last_clean":"2024-09-10T21:12:54.930226+0000","last_became_active":"2024-09-10T21:12:53.938178+0000","last_became_peered":"2024-09-10T21:12:53.938178+0000","last_unstale":"2024-09-10T21:12:54.930226+0000","last_undegraded":"2024-09-10T21:12:54.930226+0000","last_fullsized":"2024-09-10T21:12:54.930226+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:52.891384+0000","last_clean_scrub_stamp":"2024-09-10T21:12:52.891384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:42.283446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.958516+0000","last_change":"2024-09-10T21:12:52.922364+0000","last_active":"2024-09-10T21:12:54.958516+0000","last_peered":"2024-09-10T21:12:54.958516+0000","last_clean":"2024-09-10T21:12:54.958516+0000","last_became_active":"2024-09-10T21:12:52.921965+0000","last_became_peered":"2024-09-10T21:12:52.921965+0000","last_unstale":"2024-09-10T21:12:54.958516+0000","last_undegraded":"2024-09-10T21:12:54.958516+0000","last_fullsized":"2024-09-10T21:12:54.958516+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:12:51.885478+0000","last_clean_scrub_stamp":"2024-09-10T21:12:51.885478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:48.097204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:12:54.957707+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:12:54.957707+0000","last_peered":"2024-09-10T21:12:54.957707+0000","last_clean":"2024-09-10T21:12:54.957707+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:12:54.957707+0000","last_undegraded":"2024-09-10T21:12:54.957707+0000","last_fullsized":"2024-09-10T21:12:54.957707+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":679936,"data_stored":279622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509477,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1273856,"data_stored":870002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509475,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":684032,"data_stored":280216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509474,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":679936,"data_stored":279690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509476,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":696320,"data_stored":281372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":688128,"data_stored":281212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":684032,"data_stored":279692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1273856,"data_stored":870550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:13:03.084 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:13:03.084 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:13:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.268+0000 7f59cfa00640 1 -- 172.21.15.16:0/1497161314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59b0003c70 msgr2=0x7f59b0004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.268+0000 7f59cfa00640 1 --2- 172.21.15.16:0/1497161314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59b0003c70 0x7f59b0004070 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f59c8074d90 tx=0x7f59bc01d010 comp rx=0 tx=0).stop 2024-09-10T21:13:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.268+0000 7f59cda00640 1 -- 172.21.15.16:0/1497161314 reap_dead start 2024-09-10T21:13:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.268+0000 7f59cfa00640 1 -- 172.21.15.16:0/1497161314 shutdown_connections 2024-09-10T21:13:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.268+0000 7f59cfa00640 1 -- 172.21.15.16:0/1497161314 >> 172.21.15.16:0/1497161314 conn(0x7f59c80d1df0 msgr2=0x7f59c80d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.268+0000 7f59cfa00640 1 -- 172.21.15.16:0/1497161314 shutdown_connections 2024-09-10T21:13:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.268+0000 7f59cfa00640 1 -- 172.21.15.16:0/1497161314 wait complete. 2024-09-10T21:13:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.268+0000 7f59cfa00640 1 Processor -- start 2024-09-10T21:13:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.268+0000 7f59cfa00640 1 -- start start 2024-09-10T21:13:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cfa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f59b0003c70 0x7f59c81f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cfa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c81e7000 0x7f59c81553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cfa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59c80b6cf0 0x7f59c80b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cfa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80cf1e0 con 0x7f59c80b6cf0 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cfa00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80b14f0 con 0x7f59c81e7000 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cfa00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f59c80cf360 con 0x7f59b0003c70 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59c80b6cf0 0x7f59c80b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59c80b6cf0 0x7f59c80b7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36974/0 (socket says 172.21.15.16:36974) 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cda00640 1 -- 172.21.15.16:0/3814131420 learned_addr learned my addr 172.21.15.16:0/3814131420 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59c7e00640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c81e7000 0x7f59c81553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cd000640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f59b0003c70 0x7f59c81f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cda00640 1 -- 172.21.15.16:0/3814131420 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f59b0003c70 msgr2=0x7f59c81f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cda00640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f59b0003c70 0x7f59c81f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cda00640 1 -- 172.21.15.16:0/3814131420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c81e7000 msgr2=0x7f59c81553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cda00640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c81e7000 0x7f59c81553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cda00640 1 -- 172.21.15.16:0/3814131420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c81e7480 con 0x7f59c80b6cf0 2024-09-10T21:13:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59c7e00640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c81e7000 0x7f59c81553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cda00640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59c80b6cf0 0x7f59c80b7170 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f59c8074850 tx=0x7f59c0006700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59c4c00640 1 -- 172.21.15.16:0/3814131420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0003d90 con 0x7f59c80b6cf0 2024-09-10T21:13:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59c4c00640 1 -- 172.21.15.16:0/3814131420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59c0006830 con 0x7f59c80b6cf0 2024-09-10T21:13:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cfa00640 1 -- 172.21.15.16:0/3814131420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c81558f0 con 0x7f59c80b6cf0 2024-09-10T21:13:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59c4c00640 1 -- 172.21.15.16:0/3814131420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0011ae0 con 0x7f59c80b6cf0 2024-09-10T21:13:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cfa00640 1 -- 172.21.15.16:0/3814131420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c80b6210 con 0x7f59c80b6cf0 2024-09-10T21:13:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59cfa00640 1 -- 172.21.15.16:0/3814131420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c80ce7c0 con 0x7f59c80b6cf0 2024-09-10T21:13:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.272+0000 7f59c4c00640 1 -- 172.21.15.16:0/3814131420 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f59c0010040 con 0x7f59c80b6cf0 2024-09-10T21:13:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.276+0000 7f59c4c00640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59a8072610 0x7f59a8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.276+0000 7f59c4c00640 1 -- 172.21.15.16:0/3814131420 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f59c0096a70 con 0x7f59c80b6cf0 2024-09-10T21:13:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.276+0000 7f59cd000640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59a8072610 0x7f59a8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.276+0000 7f59cd000640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59a8072610 0x7f59a8074ad0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0ae000 tx=0x7f59bc019db0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:03.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.280+0000 7f59c4c00640 1 -- 172.21.15.16:0/3814131420 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f59c81e7480 con 0x7f59c80b6cf0 2024-09-10T21:13:03.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.512+0000 7f59cfa00640 1 -- 172.21.15.16:0/3814131420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f59c8070ce0 con 0x7f59c80b6cf0 2024-09-10T21:13:03.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.516+0000 7f59c4c00640 1 -- 172.21.15.16:0/3814131420 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f59c00620b0 con 0x7f59c80b6cf0 2024-09-10T21:13:03.522 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-10T21:13:03.522 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA== 2024-09-10T21:13:03.522 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-10T21:13:03.522 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-10T21:13:03.522 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:13:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-10T21:13:03.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.516+0000 7f59cfa00640 1 -- 172.21.15.16:0/3814131420 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59a8072610 msgr2=0x7f59a8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.516+0000 7f59cfa00640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59a8072610 0x7f59a8074ad0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0ae000 tx=0x7f59bc019db0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.516+0000 7f59cfa00640 1 -- 172.21.15.16:0/3814131420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59c80b6cf0 msgr2=0x7f59c80b7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.516+0000 7f59cfa00640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59c80b6cf0 0x7f59c80b7170 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f59c8074850 tx=0x7f59c0006700 comp rx=0 tx=0).stop 2024-09-10T21:13:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.516+0000 7f59cfa00640 1 -- 172.21.15.16:0/3814131420 shutdown_connections 2024-09-10T21:13:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.516+0000 7f59cfa00640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59a8072610 0x7f59a8074ad0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.516+0000 7f59cfa00640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59c80b6cf0 0x7f59c80b7170 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.516+0000 7f59cfa00640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c81e7000 0x7f59c81553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.516+0000 7f59cfa00640 1 --2- 172.21.15.16:0/3814131420 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f59b0003c70 0x7f59c81f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.516+0000 7f59cfa00640 1 -- 172.21.15.16:0/3814131420 >> 172.21.15.16:0/3814131420 conn(0x7f59c80d1df0 msgr2=0x7f59c80aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.516+0000 7f59cfa00640 1 -- 172.21.15.16:0/3814131420 shutdown_connections 2024-09-10T21:13:03.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.516+0000 7f59cfa00640 1 -- 172.21.15.16:0/3814131420 wait complete. 2024-09-10T21:13:03.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:03.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.684+0000 7f9fd2600640 1 -- 172.21.15.16:0/3245652889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fcc15f830 msgr2=0x7f9fcc0ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:03.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.684+0000 7f9fd2600640 1 --2- 172.21.15.16:0/3245652889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fcc15f830 0x7f9fcc0ca170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009840 tx=0x7f9fb401c740 comp rx=0 tx=0).stop 2024-09-10T21:13:03.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.684+0000 7f9fd2600640 1 -- 172.21.15.16:0/3245652889 shutdown_connections 2024-09-10T21:13:03.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.684+0000 7f9fd2600640 1 --2- 172.21.15.16:0/3245652889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fcc15f830 0x7f9fcc0ca170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.684+0000 7f9fd2600640 1 -- 172.21.15.16:0/3245652889 >> 172.21.15.16:0/3245652889 conn(0x7f9fcc0cfb20 msgr2=0x7f9fcc0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:03.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.684+0000 7f9fd2600640 1 -- 172.21.15.16:0/3245652889 shutdown_connections 2024-09-10T21:13:03.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.684+0000 7f9fd2600640 1 -- 172.21.15.16:0/3245652889 wait complete. 2024-09-10T21:13:03.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.684+0000 7f9fd2600640 1 Processor -- start 2024-09-10T21:13:03.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.684+0000 7f9fd2600640 1 -- start start 2024-09-10T21:13:03.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fd2600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc0ca6b0 0x7f9fcc0c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:03.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fd2600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fcc1599d0 0x7f9fcc0c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:03.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fd2600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fcc15f830 0x7f9fcc0c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:03.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fd2600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc169d60 con 0x7f9fcc0ca6b0 2024-09-10T21:13:03.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fd2600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc169be0 con 0x7f9fcc1599d0 2024-09-10T21:13:03.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fd2600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9fcc169ee0 con 0x7f9fcc15f830 2024-09-10T21:13:03.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fcb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc0ca6b0 0x7f9fcc0c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:03.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fcb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc0ca6b0 0x7f9fcc0c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36986/0 (socket says 172.21.15.16:36986) 2024-09-10T21:13:03.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fcb400640 1 -- 172.21.15.16:0/3038420447 learned_addr learned my addr 172.21.15.16:0/3038420447 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:03.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fc3e00640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fcc1599d0 0x7f9fcc0c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:03.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fcbe00640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fcc15f830 0x7f9fcc0c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:03.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fcb400640 1 -- 172.21.15.16:0/3038420447 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fcc15f830 msgr2=0x7f9fcc0c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:03.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fcb400640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fcc15f830 0x7f9fcc0c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fcb400640 1 -- 172.21.15.16:0/3038420447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fcc1599d0 msgr2=0x7f9fcc0c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:03.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fcb400640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fcc1599d0 0x7f9fcc0c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fcb400640 1 -- 172.21.15.16:0/3038420447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc1587d0 con 0x7f9fcc0ca6b0 2024-09-10T21:13:03.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fcb400640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc0ca6b0 0x7f9fcc0c9890 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc001ec0 tx=0x7f9fbc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:03.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fc3400640 1 -- 172.21.15.16:0/3038420447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc016020 con 0x7f9fcc0ca6b0 2024-09-10T21:13:03.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fd2600640 1 -- 172.21.15.16:0/3038420447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc158ab0 con 0x7f9fcc0ca6b0 2024-09-10T21:13:03.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fc3400640 1 -- 172.21.15.16:0/3038420447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fbc00a950 con 0x7f9fcc0ca6b0 2024-09-10T21:13:03.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fc3400640 1 -- 172.21.15.16:0/3038420447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc00ada0 con 0x7f9fcc0ca6b0 2024-09-10T21:13:03.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fd2600640 1 -- 172.21.15.16:0/3038420447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc159070 con 0x7f9fcc0ca6b0 2024-09-10T21:13:03.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.688+0000 7f9fd2600640 1 -- 172.21.15.16:0/3038420447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fcc0ca170 con 0x7f9fcc0ca6b0 2024-09-10T21:13:03.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.692+0000 7f9fc3400640 1 -- 172.21.15.16:0/3038420447 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9fbc01b400 con 0x7f9fcc0ca6b0 2024-09-10T21:13:03.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.692+0000 7f9fc3400640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fa80725e0 0x7f9fa8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:03.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.692+0000 7f9fc3400640 1 -- 172.21.15.16:0/3038420447 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f9fbc093670 con 0x7f9fcc0ca6b0 2024-09-10T21:13:03.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.692+0000 7f9fc3e00640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fa80725e0 0x7f9fa8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:03.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.692+0000 7f9fc3e00640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fa80725e0 0x7f9fa8074aa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40af000 tx=0x7f9fb40b2040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:03.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.696+0000 7f9fc3400640 1 -- 172.21.15.16:0/3038420447 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9fbc020020 con 0x7f9fcc0ca6b0 2024-09-10T21:13:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.904+0000 7f9fd2600640 1 -- 172.21.15.16:0/3038420447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fcc0b34e0 con 0x7f9fcc0ca6b0 2024-09-10T21:13:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.904+0000 7f9fc3400640 1 -- 172.21.15.16:0/3038420447 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5220 (secure 0 0 0) 0x7f9fbc05e5e0 con 0x7f9fcc0ca6b0 2024-09-10T21:13:03.912 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:03.912 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"btime":"2024-09-10T21:12:56:932852+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/653119388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":653119388},{"type":"v1","addr":"172.21.15.16:6836","nonce":653119388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2947833730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2947833730},{"type":"v1","addr":"172.21.15.16:6837","nonce":2947833730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2865311562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2865311562},{"type":"v1","addr":"172.21.15.52:6834","nonce":2865311562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:12:53.304996+0000","modified":"2024-09-10T21:12:56.337640+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5592":{"gid":5592,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/2357796669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2357796669},{"type":"v1","addr":"172.21.15.52:6835","nonce":2357796669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5592,"qdb_cluster":[5592]},"id":4}]} 2024-09-10T21:13:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 33 2024-09-10T21:13:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.908+0000 7f9fd2600640 1 -- 172.21.15.16:0/3038420447 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fa80725e0 msgr2=0x7f9fa8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.908+0000 7f9fd2600640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fa80725e0 0x7f9fa8074aa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40af000 tx=0x7f9fb40b2040 comp rx=0 tx=0).stop 2024-09-10T21:13:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.908+0000 7f9fd2600640 1 -- 172.21.15.16:0/3038420447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc0ca6b0 msgr2=0x7f9fcc0c9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.908+0000 7f9fd2600640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc0ca6b0 0x7f9fcc0c9890 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc001ec0 tx=0x7f9fbc00cb90 comp rx=0 tx=0).stop 2024-09-10T21:13:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.908+0000 7f9fd2600640 1 -- 172.21.15.16:0/3038420447 shutdown_connections 2024-09-10T21:13:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.908+0000 7f9fd2600640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fa80725e0 0x7f9fa8074aa0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.908+0000 7f9fd2600640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fcc15f830 0x7f9fcc0c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.908+0000 7f9fd2600640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fcc1599d0 0x7f9fcc0c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.908+0000 7f9fd2600640 1 --2- 172.21.15.16:0/3038420447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc0ca6b0 0x7f9fcc0c9890 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.908+0000 7f9fd2600640 1 -- 172.21.15.16:0/3038420447 >> 172.21.15.16:0/3038420447 conn(0x7f9fcc0cfb20 msgr2=0x7f9fcc159e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.908+0000 7f9fd2600640 1 -- 172.21.15.16:0/3038420447 shutdown_connections 2024-09-10T21:13:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:03.908+0000 7f9fd2600640 1 -- 172.21.15.16:0/3038420447 wait complete. 2024-09-10T21:13:04.336 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:12:53.304996+0000', 'modified': '2024-09-10T21:12:56.337640+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5592}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5592': {'gid': 5592, 'name': 'd', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.52:6835/2357796669', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6833', 'nonce': 2357796669}, {'type': 'v1', 'addr': '172.21.15.52:6835', 'nonce': 2357796669}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5592, 'qdb_cluster': [5592]} 2024-09-10T21:13:04.336 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:13:04.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T21:13:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.480+0000 7ff941400640 1 -- 172.21.15.16:0/3926806088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93c0d03c0 msgr2=0x7ff93c15fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.480+0000 7ff941400640 1 --2- 172.21.15.16:0/3926806088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93c0d03c0 0x7ff93c15fe00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0708f0 tx=0x7ff92401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.480+0000 7ff941400640 1 -- 172.21.15.16:0/3926806088 shutdown_connections 2024-09-10T21:13:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.480+0000 7ff941400640 1 --2- 172.21.15.16:0/3926806088 >> v2:172.21.15.52:3300/0 conn(0x7ff93c0b2860 0x7ff93c160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.480+0000 7ff941400640 1 --2- 172.21.15.16:0/3926806088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93c0d03c0 0x7ff93c15fe00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.480+0000 7ff941400640 1 --2- 172.21.15.16:0/3926806088 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff93c0cffe0 0x7ff93c0ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.480+0000 7ff941400640 1 -- 172.21.15.16:0/3926806088 >> 172.21.15.16:0/3926806088 conn(0x7ff93c0d1d80 msgr2=0x7ff93c0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.480+0000 7ff941400640 1 -- 172.21.15.16:0/3926806088 shutdown_connections 2024-09-10T21:13:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.480+0000 7ff941400640 1 -- 172.21.15.16:0/3926806088 wait complete. 2024-09-10T21:13:04.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.480+0000 7ff941400640 1 Processor -- start 2024-09-10T21:13:04.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.480+0000 7ff941400640 1 -- start start 2024-09-10T21:13:04.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff941400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93c0b2860 0x7ff93c156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff941400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff93c0cffe0 0x7ff93c15a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff941400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff93c0d03c0 0x7ff93c165aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff941400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c0cca10 con 0x7ff93c0b2860 2024-09-10T21:13:04.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff941400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c0cc890 con 0x7ff93c0d03c0 2024-09-10T21:13:04.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff941400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff93c0ccb90 con 0x7ff93c0cffe0 2024-09-10T21:13:04.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff93a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93c0b2860 0x7ff93c156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff93a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93c0b2860 0x7ff93c156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36996/0 (socket says 172.21.15.16:36996) 2024-09-10T21:13:04.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff93a800640 1 -- 172.21.15.16:0/1641976266 learned_addr learned my addr 172.21.15.16:0/1641976266 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:04.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff93b200640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff93c0d03c0 0x7ff93c165aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff933e00640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff93c0cffe0 0x7ff93c15a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff93a800640 1 -- 172.21.15.16:0/1641976266 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff93c0cffe0 msgr2=0x7ff93c15a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff93a800640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff93c0cffe0 0x7ff93c15a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff93a800640 1 -- 172.21.15.16:0/1641976266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff93c0d03c0 msgr2=0x7ff93c165aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff93a800640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff93c0d03c0 0x7ff93c165aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff93a800640 1 -- 172.21.15.16:0/1641976266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93c1fb0f0 con 0x7ff93c0b2860 2024-09-10T21:13:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff93a800640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93c0b2860 0x7ff93c156ae0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00b580 tx=0x7ff92c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff932a00640 1 -- 172.21.15.16:0/1641976266 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c014070 con 0x7ff93c0b2860 2024-09-10T21:13:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff932a00640 1 -- 172.21.15.16:0/1641976266 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff92c010980 con 0x7ff93c0b2860 2024-09-10T21:13:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff932a00640 1 -- 172.21.15.16:0/1641976266 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c010dd0 con 0x7ff93c0b2860 2024-09-10T21:13:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff941400640 1 -- 172.21.15.16:0/1641976266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93c1fb3d0 con 0x7ff93c0b2860 2024-09-10T21:13:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff941400640 1 -- 172.21.15.16:0/1641976266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff93c1fb990 con 0x7ff93c0b2860 2024-09-10T21:13:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.484+0000 7ff932a00640 1 -- 172.21.15.16:0/1641976266 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff92c00f3c0 con 0x7ff93c0b2860 2024-09-10T21:13:04.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff932a00640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff9180725e0 0x7ff918074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff932a00640 1 -- 172.21.15.16:0/1641976266 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff92c093340 con 0x7ff93c0b2860 2024-09-10T21:13:04.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff933e00640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff9180725e0 0x7ff918074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff9447a4000 1 Processor -- start 2024-09-10T21:13:04.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff9447a4000 1 -- start start 2024-09-10T21:13:04.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff933e00640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff9180725e0 0x7ff918074aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff9240af000 tx=0x7ff924019660 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:04.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff9447a4000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 0x5629e7881db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff9447a4000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78822f0 0x5629e78358d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff9447a4000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e7835e10 0x5629e78361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff9447a4000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5629e77563f0 con 0x5629e78819d0 2024-09-10T21:13:04.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff9447a4000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5629e7693b20 con 0x5629e7835e10 2024-09-10T21:13:04.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff9447a4000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x5629e7682f50 con 0x5629e78822f0 2024-09-10T21:13:04.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff93a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 0x5629e7881db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff93a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 0x5629e7881db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36998/0 (socket says 172.21.15.16:36998) 2024-09-10T21:13:04.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff93a800640 1 -- 172.21.15.16:0/2821779956 learned_addr learned my addr 172.21.15.16:0/2821779956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:04.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff933e00640 1 --2- 172.21.15.16:0/2821779956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78822f0 0x5629e78358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff93b200640 1 --2- 172.21.15.16:0/2821779956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e7835e10 0x5629e78361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff93a800640 1 -- 172.21.15.16:0/2821779956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78822f0 msgr2=0x5629e78358d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff93a800640 1 --2- 172.21.15.16:0/2821779956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78822f0 0x5629e78358d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff93a800640 1 -- 172.21.15.16:0/2821779956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e7835e10 msgr2=0x5629e78361f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff93a800640 1 --2- 172.21.15.16:0/2821779956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e7835e10 0x5629e78361f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff93a800640 1 -- 172.21.15.16:0/2821779956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629e764a590 con 0x5629e78819d0 2024-09-10T21:13:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff933e00640 1 --2- 172.21.15.16:0/2821779956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78822f0 0x5629e78358d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff93a800640 1 --2- 172.21.15.16:0/2821779956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 0x5629e7881db0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff92c05e2d0 tx=0x7ff92c05db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976c50863286813f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff92b400640 1 -- 172.21.15.16:0/2821779956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c01e070 con 0x5629e78819d0 2024-09-10T21:13:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff92b400640 1 -- 172.21.15.16:0/2821779956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff92c067790 con 0x5629e78819d0 2024-09-10T21:13:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.488+0000 7ff92b400640 1 -- 172.21.15.16:0/2821779956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c07de60 con 0x5629e78819d0 2024-09-10T21:13:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 -- 172.21.15.16:0/2821779956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 msgr2=0x5629e7881db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 --2- 172.21.15.16:0/2821779956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 0x5629e7881db0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff92c05e2d0 tx=0x7ff92c05db80 comp rx=0 tx=0).stop 2024-09-10T21:13:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 -- 172.21.15.16:0/2821779956 shutdown_connections 2024-09-10T21:13:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 --2- 172.21.15.16:0/2821779956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e7835e10 0x5629e78361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 --2- 172.21.15.16:0/2821779956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78822f0 0x5629e78358d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 --2- 172.21.15.16:0/2821779956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 0x5629e7881db0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 -- 172.21.15.16:0/2821779956 >> 172.21.15.16:0/2821779956 conn(0x5629e771f130 msgr2=0x5629e7864e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 -- 172.21.15.16:0/2821779956 shutdown_connections 2024-09-10T21:13:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 -- 172.21.15.16:0/2821779956 wait complete. 2024-09-10T21:13:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 -- start start 2024-09-10T21:13:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 Processor -- start 2024-09-10T21:13:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e7835e10 0x5629e78626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e78819d0 0x5629e7862e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78822f0 0x5629e78633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5629e761d5d0 con 0x5629e7835e10 2024-09-10T21:13:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5629e7686d60 con 0x5629e78819d0 2024-09-10T21:13:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x5629e7629500 con 0x5629e78822f0 2024-09-10T21:13:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 10 client.0 _ll_register_callbacks cb 0x5629e762d810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff93a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e7835e10 0x5629e78626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff93a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e7835e10 0x5629e78626a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37010/0 (socket says 172.21.15.16:37010) 2024-09-10T21:13:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff93a800640 1 -- 172.21.15.16:0/2659426941 learned_addr learned my addr 172.21.15.16:0/2659426941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff933e00640 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e78819d0 0x5629e7862e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff93b200640 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78822f0 0x5629e78633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff93a800640 1 -- 172.21.15.16:0/2659426941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78822f0 msgr2=0x5629e78633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff93a800640 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78822f0 0x5629e78633b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff93a800640 1 -- 172.21.15.16:0/2659426941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e78819d0 msgr2=0x5629e7862e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff93a800640 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e78819d0 0x5629e7862e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff93a800640 1 -- 172.21.15.16:0/2659426941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629e7621170 con 0x5629e7835e10 2024-09-10T21:13:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff93b200640 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78822f0 0x5629e78633b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff93a800640 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e7835e10 0x5629e78626a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00bf40 tx=0x7ff92c067790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8a7c9d09cfcd0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff91fe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:13:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff91fe00640 1 -- 172.21.15.16:0/2659426941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c083070 con 0x5629e7835e10 2024-09-10T21:13:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff9447a4000 1 -- 172.21.15.16:0/2659426941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5629e761b0f0 con 0x5629e7835e10 2024-09-10T21:13:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.492+0000 7ff91fe00640 1 -- 172.21.15.16:0/2659426941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff92c083210 con 0x5629e7835e10 2024-09-10T21:13:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff91fe00640 1 -- 172.21.15.16:0/2659426941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff92c01e070 con 0x5629e7835e10 2024-09-10T21:13:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 10 client.5745 fetch_fsmap learned FSMap version 33 2024-09-10T21:13:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 -- 172.21.15.16:0/2659426941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x5629e7707ef0 con 0x5629e7835e10 2024-09-10T21:13:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff91fe00640 1 -- 172.21.15.16:0/2659426941 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5629e761b0f0 con 0x5629e7835e10 2024-09-10T21:13:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff91fe00640 1 -- 172.21.15.16:0/2659426941 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 33) ==== 3502+0+0 (secure 0 0 0) 0x7ff92c067300 con 0x5629e7835e10 2024-09-10T21:13:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 10 client.5745 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T21:13:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 10 client.5745 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:13:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 20 client.5745 populate_metadata read hostname 'smithi016' 2024-09-10T21:13:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.52:6833/2357796669,v1:172.21.15.52:6835/2357796669] conn(0x5629e78640f0 0x5629e7893c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 4 client.5745 mds_command: new command op to 5592 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:13:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 -- 172.21.15.16:0/2659426941 --> [v2:172.21.15.52:6833/2357796669,v1:172.21.15.52:6835/2357796669] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629e7695a70 con 0x5629e78640f0 2024-09-10T21:13:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff933e00640 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.52:6833/2357796669,v1:172.21.15.52:6835/2357796669] conn(0x5629e78640f0 0x5629e7893c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff91fe00640 10 client.5745 ms_handle_connect on v2:172.21.15.52:6833/2357796669 2024-09-10T21:13:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff933e00640 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.52:6833/2357796669,v1:172.21.15.52:6835/2357796669] conn(0x5629e78640f0 0x5629e7893c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=400582675353af0a server_cookie=419d6d198656dfc6 in_seq=0 out_seq=0 2024-09-10T21:13:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff91fe00640 1 -- 172.21.15.16:0/2659426941 <== mds.0 v2:172.21.15.52:6833/2357796669 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5629e7695a70 con 0x5629e78640f0 2024-09-10T21:13:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff91fe00640 10 client.5745 handle_command_reply: tid=0 2024-09-10T21:13:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 client.5745 shutdown 2024-09-10T21:13:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 -- 172.21.15.16:0/2659426941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e7835e10 msgr2=0x5629e78626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e7835e10 0x5629e78626a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00bf40 tx=0x7ff92c067790 comp rx=0 tx=0).stop 2024-09-10T21:13:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 -- 172.21.15.16:0/2659426941 shutdown_connections 2024-09-10T21:13:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.52:6833/2357796669,v1:172.21.15.52:6835/2357796669] conn(0x5629e78640f0 0x5629e7893c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78822f0 0x5629e78633b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e78819d0 0x5629e7862e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 --2- 172.21.15.16:0/2659426941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e7835e10 0x5629e78626a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 -- 172.21.15.16:0/2659426941 >> 172.21.15.16:0/2659426941 conn(0x5629e771f130 msgr2=0x5629e7834fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff91fe00640 0 client.5745 ms_handle_reset on v2:172.21.15.52:6833/2357796669 2024-09-10T21:13:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 -- 172.21.15.16:0/2659426941 shutdown_connections 2024-09-10T21:13:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 1 -- 172.21.15.16:0/2659426941 wait complete. 2024-09-10T21:13:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.496+0000 7ff9447a4000 20 client.5745 trim_cache size 0 max 16384 2024-09-10T21:13:04.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff9447a4000 1 Processor -- start 2024-09-10T21:13:04.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff9447a4000 1 -- start start 2024-09-10T21:13:04.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff9447a4000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e7835e10 0x5629e78f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff9447a4000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 0x5629e78f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff9447a4000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e78822f0 0x5629e78f4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff9447a4000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c2041e0 con 0x5629e78819d0 2024-09-10T21:13:04.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff9447a4000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c203280 con 0x5629e78822f0 2024-09-10T21:13:04.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff9447a4000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff93c0af850 con 0x5629e7835e10 2024-09-10T21:13:04.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff933e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 0x5629e78f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff933e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 0x5629e78f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37012/0 (socket says 172.21.15.16:37012) 2024-09-10T21:13:04.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff933e00640 1 -- 172.21.15.16:0/208834146 learned_addr learned my addr 172.21.15.16:0/208834146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:04.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff93b200640 1 --2- 172.21.15.16:0/208834146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e78822f0 0x5629e78f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff93a800640 1 --2- 172.21.15.16:0/208834146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e7835e10 0x5629e78f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff933e00640 1 -- 172.21.15.16:0/208834146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e7835e10 msgr2=0x5629e78f3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:04.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff933e00640 1 --2- 172.21.15.16:0/208834146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e7835e10 0x5629e78f3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff933e00640 1 -- 172.21.15.16:0/208834146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e78822f0 msgr2=0x5629e78f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:04.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff933e00640 1 --2- 172.21.15.16:0/208834146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e78822f0 0x5629e78f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff933e00640 1 -- 172.21.15.16:0/208834146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629e76ded90 con 0x5629e78819d0 2024-09-10T21:13:04.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff93a800640 1 --2- 172.21.15.16:0/208834146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e7835e10 0x5629e78f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:04.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.516+0000 7ff933e00640 1 --2- 172.21.15.16:0/208834146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 0x5629e78f4310 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff92401d160 tx=0x7ff9240b4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd5ee37f8c0e7d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:04.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff92aa00640 1 -- 172.21.15.16:0/208834146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9240b2040 con 0x5629e78819d0 2024-09-10T21:13:04.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff92aa00640 1 -- 172.21.15.16:0/208834146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff924005580 con 0x5629e78819d0 2024-09-10T21:13:04.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff92aa00640 1 -- 172.21.15.16:0/208834146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924004930 con 0x5629e78819d0 2024-09-10T21:13:04.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 -- 172.21.15.16:0/208834146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 msgr2=0x5629e78f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 --2- 172.21.15.16:0/208834146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 0x5629e78f4310 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff92401d160 tx=0x7ff9240b4000 comp rx=0 tx=0).stop 2024-09-10T21:13:04.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 -- 172.21.15.16:0/208834146 shutdown_connections 2024-09-10T21:13:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 --2- 172.21.15.16:0/208834146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e78822f0 0x5629e78f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 --2- 172.21.15.16:0/208834146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78819d0 0x5629e78f4310 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 --2- 172.21.15.16:0/208834146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e7835e10 0x5629e78f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 -- 172.21.15.16:0/208834146 >> 172.21.15.16:0/208834146 conn(0x5629e771f130 msgr2=0x5629e783dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 -- 172.21.15.16:0/208834146 shutdown_connections 2024-09-10T21:13:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 -- 172.21.15.16:0/208834146 wait complete. 2024-09-10T21:13:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 -- start start 2024-09-10T21:13:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 Processor -- start 2024-09-10T21:13:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e7835e10 0x5629e78f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78819d0 0x5629e78f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78822f0 0x5629e78f8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5629e7686d60 con 0x5629e78822f0 2024-09-10T21:13:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c155d60 con 0x5629e7835e10 2024-09-10T21:13:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x5629e761d5d0 con 0x5629e78819d0 2024-09-10T21:13:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 10 client.0 _ll_register_callbacks cb 0x7ff92c062070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff93a800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e7835e10 0x5629e78f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff93b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78822f0 0x5629e78f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff933e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78819d0 0x5629e78f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff93a800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e7835e10 0x5629e78f79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:35174/0 (socket says 172.21.15.16:35174) 2024-09-10T21:13:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff93a800640 1 -- 172.21.15.16:0/1422587596 learned_addr learned my addr 172.21.15.16:0/1422587596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff93b200640 1 -- 172.21.15.16:0/1422587596 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78819d0 msgr2=0x5629e78f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff93b200640 1 --2- 172.21.15.16:0/1422587596 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78819d0 0x5629e78f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff93b200640 1 -- 172.21.15.16:0/1422587596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e7835e10 msgr2=0x5629e78f79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff93b200640 1 --2- 172.21.15.16:0/1422587596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e7835e10 0x5629e78f79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff93b200640 1 -- 172.21.15.16:0/1422587596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629e7649a90 con 0x5629e78822f0 2024-09-10T21:13:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff93a800640 1 --2- 172.21.15.16:0/1422587596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e7835e10 0x5629e78f79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff93b200640 1 --2- 172.21.15.16:0/1422587596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78822f0 0x5629e78f8470 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff93400afd0 tx=0x7ff934010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45c278b0ae2f97f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff91fe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:13:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff91fe00640 1 -- 172.21.15.16:0/1422587596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93401b410 con 0x5629e78822f0 2024-09-10T21:13:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff91fe00640 1 -- 172.21.15.16:0/1422587596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff93401b5b0 con 0x5629e78822f0 2024-09-10T21:13:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.520+0000 7ff9447a4000 1 -- 172.21.15.16:0/1422587596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff924005580 con 0x5629e78822f0 2024-09-10T21:13:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff91fe00640 1 -- 172.21.15.16:0/1422587596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff934028050 con 0x5629e78822f0 2024-09-10T21:13:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 10 client.5751 fetch_fsmap learned FSMap version 33 2024-09-10T21:13:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 -- 172.21.15.16:0/1422587596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x5629e7628050 con 0x5629e78822f0 2024-09-10T21:13:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff91fe00640 1 -- 172.21.15.16:0/1422587596 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff924005580 con 0x5629e78822f0 2024-09-10T21:13:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff91fe00640 1 -- 172.21.15.16:0/1422587596 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 33) ==== 3502+0+0 (secure 0 0 0) 0x7ff93402b020 con 0x5629e78822f0 2024-09-10T21:13:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 10 client.5751 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T21:13:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 10 client.5751 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:13:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 20 client.5751 populate_metadata read hostname 'smithi016' 2024-09-10T21:13:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 --2- 172.21.15.16:0/1422587596 >> [v2:172.21.15.52:6833/2357796669,v1:172.21.15.52:6835/2357796669] conn(0x5629e78974e0 0x5629e7896670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 4 client.5751 mds_command: new command op to 5592 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:13:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 -- 172.21.15.16:0/1422587596 --> [v2:172.21.15.52:6833/2357796669,v1:172.21.15.52:6835/2357796669] -- command(tid 0: {"prefix": "status"}) -- 0x5629e7691520 con 0x5629e78974e0 2024-09-10T21:13:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff93a800640 1 --2- 172.21.15.16:0/1422587596 >> [v2:172.21.15.52:6833/2357796669,v1:172.21.15.52:6835/2357796669] conn(0x5629e78974e0 0x5629e7896670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff93a800640 1 --2- 172.21.15.16:0/1422587596 >> [v2:172.21.15.52:6833/2357796669,v1:172.21.15.52:6835/2357796669] conn(0x5629e78974e0 0x5629e7896670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7df951d77ef4b841 server_cookie=a3fcafcc3502ea5e in_seq=0 out_seq=0 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff91fe00640 10 client.5751 ms_handle_connect on v2:172.21.15.52:6833/2357796669 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff91fe00640 1 -- 172.21.15.16:0/1422587596 <== mds.0 v2:172.21.15.52:6833/2357796669 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5629e7691520 con 0x5629e78974e0 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff91fe00640 10 client.5751 handle_command_reply: tid=0 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 client.5751 shutdown 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 -- 172.21.15.16:0/1422587596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78822f0 msgr2=0x5629e78f8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 --2- 172.21.15.16:0/1422587596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78822f0 0x5629e78f8470 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff93400afd0 tx=0x7ff934010eb0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 -- 172.21.15.16:0/1422587596 shutdown_connections 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 --2- 172.21.15.16:0/1422587596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e78822f0 0x5629e78f8470 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 --2- 172.21.15.16:0/1422587596 >> [v2:172.21.15.52:6833/2357796669,v1:172.21.15.52:6835/2357796669] conn(0x5629e78974e0 0x5629e7896670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 --2- 172.21.15.16:0/1422587596 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5629e78819d0 0x5629e78f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 --2- 172.21.15.16:0/1422587596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629e7835e10 0x5629e78f79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff91fe00640 0 client.5751 ms_handle_reset on v2:172.21.15.52:6833/2357796669 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 -- 172.21.15.16:0/1422587596 >> 172.21.15.16:0/1422587596 conn(0x5629e771f130 msgr2=0x5629e78805b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 -- 172.21.15.16:0/1422587596 shutdown_connections 2024-09-10T21:13:04.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 1 -- 172.21.15.16:0/1422587596 wait complete. 2024-09-10T21:13:04.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff9447a4000 20 client.5751 trim_cache size 0 max 16384 2024-09-10T21:13:04.536 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:13:04.536 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "eb19be60-6d8f-4816-b778-3d7d6698304d", 2024-09-10T21:13:04.536 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-10T21:13:04.536 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5592, 2024-09-10T21:13:04.536 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-10T21:13:04.536 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-10T21:13:04.536 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-10T21:13:04.536 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 10.603524012999999, 2024-09-10T21:13:04.537 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 33, 2024-09-10T21:13:04.537 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 57, 2024-09-10T21:13:04.537 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 57, 2024-09-10T21:13:04.537 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 15.091526729 2024-09-10T21:13:04.537 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:13:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff941400640 1 -- 172.21.15.16:0/1641976266 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff9180725e0 msgr2=0x7ff918074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff941400640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff9180725e0 0x7ff918074aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff9240af000 tx=0x7ff924019660 comp rx=0 tx=0).stop 2024-09-10T21:13:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff941400640 1 -- 172.21.15.16:0/1641976266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93c0b2860 msgr2=0x7ff93c156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff941400640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93c0b2860 0x7ff93c156ae0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00b580 tx=0x7ff92c00ba40 comp rx=0 tx=0).stop 2024-09-10T21:13:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff941400640 1 -- 172.21.15.16:0/1641976266 shutdown_connections 2024-09-10T21:13:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff941400640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff9180725e0 0x7ff918074aa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff941400640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff93c0d03c0 0x7ff93c165aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff941400640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff93c0cffe0 0x7ff93c15a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff941400640 1 --2- 172.21.15.16:0/1641976266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93c0b2860 0x7ff93c156ae0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff941400640 1 -- 172.21.15.16:0/1641976266 >> 172.21.15.16:0/1641976266 conn(0x7ff93c0d1d80 msgr2=0x7ff93c072270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff941400640 1 -- 172.21.15.16:0/1641976266 shutdown_connections 2024-09-10T21:13:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.524+0000 7ff941400640 1 -- 172.21.15.16:0/1641976266 wait complete. 2024-09-10T21:13:04.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:13:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.692+0000 7f1575600640 1 -- 172.21.15.16:0/939268091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1554004810 msgr2=0x7f1554004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.692+0000 7f1575600640 1 --2- 172.21.15.16:0/939268091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1554004810 0x7f1554004c90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f15580016c0 tx=0x7f155801c770 comp rx=0 tx=0).stop 2024-09-10T21:13:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.692+0000 7f1575600640 1 -- 172.21.15.16:0/939268091 shutdown_connections 2024-09-10T21:13:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.692+0000 7f1575600640 1 --2- 172.21.15.16:0/939268091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1554004810 0x7f1554004c90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.692+0000 7f1575600640 1 -- 172.21.15.16:0/939268091 >> 172.21.15.16:0/939268091 conn(0x7f15700d1d80 msgr2=0x7f15700d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.692+0000 7f1575600640 1 -- 172.21.15.16:0/939268091 shutdown_connections 2024-09-10T21:13:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.692+0000 7f1575600640 1 -- 172.21.15.16:0/939268091 wait complete. 2024-09-10T21:13:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1575600640 1 Processor -- start 2024-09-10T21:13:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1575600640 1 -- start start 2024-09-10T21:13:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1575600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1554004810 0x7f1570152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1575600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1570153430 0x7f15700ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1575600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15700b0860 0x7f15700b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1575600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15700cfc80 con 0x7f15700b0860 2024-09-10T21:13:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1575600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f15700b2fd0 con 0x7f1554004810 2024-09-10T21:13:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1575600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f15700cfe00 con 0x7f1570153430 2024-09-10T21:13:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f156f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15700b0860 0x7f15700b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f156f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15700b0860 0x7f15700b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37048/0 (socket says 172.21.15.16:37048) 2024-09-10T21:13:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f156f400640 1 -- 172.21.15.16:0/2038164457 learned_addr learned my addr 172.21.15.16:0/2038164457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f156ea00640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1554004810 0x7f1570152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1567e00640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1570153430 0x7f15700ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f156f400640 1 -- 172.21.15.16:0/2038164457 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1570153430 msgr2=0x7f15700ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f156f400640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1570153430 0x7f15700ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f156f400640 1 -- 172.21.15.16:0/2038164457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1554004810 msgr2=0x7f1570152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f156f400640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1554004810 0x7f1570152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f156f400640 1 -- 172.21.15.16:0/2038164457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15700ca270 con 0x7f15700b0860 2024-09-10T21:13:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f156f400640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15700b0860 0x7f15700b0d10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f15700b13a0 tx=0x7f1568010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1566a00640 1 -- 172.21.15.16:0/2038164457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156800ec00 con 0x7f15700b0860 2024-09-10T21:13:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1566a00640 1 -- 172.21.15.16:0/2038164457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f156800ee90 con 0x7f15700b0860 2024-09-10T21:13:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1566a00640 1 -- 172.21.15.16:0/2038164457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568012690 con 0x7f15700b0860 2024-09-10T21:13:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1575600640 1 -- 172.21.15.16:0/2038164457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1570153180 con 0x7f15700b0860 2024-09-10T21:13:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1575600640 1 -- 172.21.15.16:0/2038164457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15700ca7f0 con 0x7f15700b0860 2024-09-10T21:13:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.696+0000 7f1575600640 1 -- 172.21.15.16:0/2038164457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15700d02f0 con 0x7f15700b0860 2024-09-10T21:13:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.700+0000 7f1566a00640 1 -- 172.21.15.16:0/2038164457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f156800f040 con 0x7f15700b0860 2024-09-10T21:13:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.700+0000 7f1566a00640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f154c072610 0x7f154c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.700+0000 7f1566a00640 1 -- 172.21.15.16:0/2038164457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f1568093440 con 0x7f15700b0860 2024-09-10T21:13:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.700+0000 7f156ea00640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f154c072610 0x7f154c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.700+0000 7f156ea00640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f154c072610 0x7f154c074ad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f15700cc380 tx=0x7f15600011d0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.704+0000 7f1566a00640 1 -- 172.21.15.16:0/2038164457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1568023070 con 0x7f15700b0860 2024-09-10T21:13:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1575600640 1 -- 172.21.15.16:0/2038164457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1570074ba0 con 0x7f15700b0860 2024-09-10T21:13:04.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1566a00640 1 -- 172.21.15.16:0/2038164457 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7f156805e430 con 0x7f15700b0860 2024-09-10T21:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:13:04.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1575600640 1 -- 172.21.15.16:0/2038164457 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f154c072610 msgr2=0x7f154c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1575600640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f154c072610 0x7f154c074ad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f15700cc380 tx=0x7f15600011d0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1575600640 1 -- 172.21.15.16:0/2038164457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15700b0860 msgr2=0x7f15700b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:04.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1575600640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15700b0860 0x7f15700b0d10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f15700b13a0 tx=0x7f1568010da0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1575600640 1 -- 172.21.15.16:0/2038164457 shutdown_connections 2024-09-10T21:13:04.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1575600640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f154c072610 0x7f154c074ad0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1575600640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15700b0860 0x7f15700b0d10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1575600640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1570153430 0x7f15700ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1575600640 1 --2- 172.21.15.16:0/2038164457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1554004810 0x7f1570152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1575600640 1 -- 172.21.15.16:0/2038164457 >> 172.21.15.16:0/2038164457 conn(0x7f15700d1d80 msgr2=0x7f157006e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1575600640 1 -- 172.21.15.16:0/2038164457 shutdown_connections 2024-09-10T21:13:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:04.912+0000 7f1575600640 1 -- 172.21.15.16:0/2038164457 wait complete. 2024-09-10T21:13:04.939 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:13:04.940 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:13:04.940 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-10T21:13:04.940 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:13:04.940 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:13:04.940 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:13:04.940 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:13:04.940 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:13:04.940 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:13:04.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:04.941 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-10T21:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-10T21:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-10T21:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-10T21:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42264sec preferred_lft 42264sec 2024-09-10T21:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-10T21:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-10T21:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:30:fb:74:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:04.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:04.954 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-10T21:13:05.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:05.016 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-10T21:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-10T21:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:13:05.067 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:13:05.067 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:13:05.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:13:05.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:13:05.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:13:05.067 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:13:05.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:13:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:13:05.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:05.205 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:13:05.262 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:13:05.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:05.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-10T21:13:05.324 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:13:05.325 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:13:05.381 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:13:05.381 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:13:05.432 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:13:05.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:05.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:13:05.539 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:13:05.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:05.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:13:05.560 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:13:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:13:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout:client135071 2024-09-10T21:13:05.676 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:05.676 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:05.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:13:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.868+0000 7f8613c00640 1 -- 172.21.15.16:0/3575186760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f4005920 msgr2=0x7f85f4005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.868+0000 7f8613c00640 1 --2- 172.21.15.16:0/3575186760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f4005920 0x7f85f4005da0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f85f8006fa0 tx=0x7f85f801ce50 comp rx=0 tx=0).stop 2024-09-10T21:13:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.868+0000 7f8613c00640 1 -- 172.21.15.16:0/3575186760 shutdown_connections 2024-09-10T21:13:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.868+0000 7f8613c00640 1 --2- 172.21.15.16:0/3575186760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f4005920 0x7f85f4005da0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.868+0000 7f8613c00640 1 -- 172.21.15.16:0/3575186760 >> 172.21.15.16:0/3575186760 conn(0x7f860c0cfb20 msgr2=0x7f860c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.868+0000 7f8613c00640 1 -- 172.21.15.16:0/3575186760 shutdown_connections 2024-09-10T21:13:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.868+0000 7f8613c00640 1 -- 172.21.15.16:0/3575186760 wait complete. 2024-09-10T21:13:05.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.868+0000 7f8613c00640 1 Processor -- start 2024-09-10T21:13:05.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.868+0000 7f8613c00640 1 -- start start 2024-09-10T21:13:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8613c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f4005920 0x7f860c0c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8613c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860c1f8f30 0x7f860c1fb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8613c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f860c0c9530 0x7f860c1fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8613c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f860c0700a0 con 0x7f85f4005920 2024-09-10T21:13:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8613c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f860c06ec60 con 0x7f860c1f8f30 2024-09-10T21:13:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8613c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f860c070220 con 0x7f860c0c9530 2024-09-10T21:13:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8611200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f4005920 0x7f860c0c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8611200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f4005920 0x7f860c0c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37064/0 (socket says 172.21.15.16:37064) 2024-09-10T21:13:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8611200640 1 -- 172.21.15.16:0/3548446920 learned_addr learned my addr 172.21.15.16:0/3548446920 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f860be00640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860c1f8f30 0x7f860c1fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8611c00640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f860c0c9530 0x7f860c1fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8611200640 1 -- 172.21.15.16:0/3548446920 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f860c0c9530 msgr2=0x7f860c1fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8611200640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f860c0c9530 0x7f860c1fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8611200640 1 -- 172.21.15.16:0/3548446920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860c1f8f30 msgr2=0x7f860c1fb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8611200640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860c1f8f30 0x7f860c1fb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8611200640 1 -- 172.21.15.16:0/3548446920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860c072500 con 0x7f85f4005920 2024-09-10T21:13:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8611c00640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f860c0c9530 0x7f860c1fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8611200640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f4005920 0x7f860c0c8ff0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8600004d70 tx=0x7f8600001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8608c00640 1 -- 172.21.15.16:0/3548446920 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86000105d0 con 0x7f85f4005920 2024-09-10T21:13:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8608c00640 1 -- 172.21.15.16:0/3548446920 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8600016070 con 0x7f85f4005920 2024-09-10T21:13:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8613c00640 1 -- 172.21.15.16:0/3548446920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860c0c9ab0 con 0x7f85f4005920 2024-09-10T21:13:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8608c00640 1 -- 172.21.15.16:0/3548446920 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600010ae0 con 0x7f85f4005920 2024-09-10T21:13:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8613c00640 1 -- 172.21.15.16:0/3548446920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f860c0727c0 con 0x7f85f4005920 2024-09-10T21:13:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.872+0000 7f8613c00640 1 -- 172.21.15.16:0/3548446920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f860c070780 con 0x7f85f4005920 2024-09-10T21:13:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.876+0000 7f8608c00640 1 -- 172.21.15.16:0/3548446920 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8600010c80 con 0x7f85f4005920 2024-09-10T21:13:05.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.876+0000 7f8608c00640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f85ec072610 0x7f85ec074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:05.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.876+0000 7f8608c00640 1 -- 172.21.15.16:0/3548446920 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f8600093a50 con 0x7f85f4005920 2024-09-10T21:13:05.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.876+0000 7f860be00640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f85ec072610 0x7f85ec074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:05.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.876+0000 7f860be00640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f85ec072610 0x7f85ec074ad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f85f80ab000 tx=0x7f85f8019c90 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:05.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:05.880+0000 7f8608c00640 1 -- 172.21.15.16:0/3548446920 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f860005f090 con 0x7f85f4005920 2024-09-10T21:13:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8613c00640 1 -- 172.21.15.16:0/3548446920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f860c0b5aa0 con 0x7f85f4005920 2024-09-10T21:13:06.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8608c00640 1 -- 172.21.15.16:0/3548446920 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7f860005e990 con 0x7f85f4005920 2024-09-10T21:13:06.098 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:13:06.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8613c00640 1 -- 172.21.15.16:0/3548446920 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f85ec072610 msgr2=0x7f85ec074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:06.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8613c00640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f85ec072610 0x7f85ec074ad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f85f80ab000 tx=0x7f85f8019c90 comp rx=0 tx=0).stop 2024-09-10T21:13:06.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8613c00640 1 -- 172.21.15.16:0/3548446920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f4005920 msgr2=0x7f860c0c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:06.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8613c00640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f4005920 0x7f860c0c8ff0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8600004d70 tx=0x7f8600001640 comp rx=0 tx=0).stop 2024-09-10T21:13:06.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8613c00640 1 -- 172.21.15.16:0/3548446920 shutdown_connections 2024-09-10T21:13:06.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8613c00640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f85ec072610 0x7f85ec074ad0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:06.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8613c00640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f860c0c9530 0x7f860c1fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:06.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8613c00640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860c1f8f30 0x7f860c1fb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:06.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8613c00640 1 --2- 172.21.15.16:0/3548446920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f4005920 0x7f860c0c8ff0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:06.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8613c00640 1 -- 172.21.15.16:0/3548446920 >> 172.21.15.16:0/3548446920 conn(0x7f860c0cfb20 msgr2=0x7f860c0b5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:06.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8613c00640 1 -- 172.21.15.16:0/3548446920 shutdown_connections 2024-09-10T21:13:06.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:06.092+0000 7f8613c00640 1 -- 172.21.15.16:0/3548446920 wait complete. 2024-09-10T21:13:06.119 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:13:06.120 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:13:06.120 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-09-10T21:13:06.120 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T21:13:06.120 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:13:06.120 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:13:06.120 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:13:06.120 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:13:06.120 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:13:06.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:06.120 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-09-10T21:13:06.131 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:13:06.131 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:13:06.131 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:13:06.131 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:06.131 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:13:06.131 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:06.131 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:13:06.132 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:06.132 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-09-10T21:13:06.132 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:13:06.132 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:06.132 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-09-10T21:13:06.132 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:13:06.132 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:06.132 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-09-10T21:13:06.132 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:13:06.132 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42264sec preferred_lft 42264sec 2024-09-10T21:13:06.132 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-09-10T21:13:06.132 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:06.132 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:13:06.133 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:06.133 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-09-10T21:13:06.133 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:13:06.133 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:7e:b4:71:a2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:06.133 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:13:06.133 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:06.133 INFO:teuthology.orchestra.run.smithi052.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:13:06.133 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:06.133 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:13:06.133 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:06.133 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:13:06.134 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:06.134 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:13:06.134 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:13:06.134 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:13:06.134 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:06.134 INFO:teuthology.orchestra.run.smithi052.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:13:06.134 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:13:06.134 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:13:06.134 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:06.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:06.135 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:13:06.201 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-09-10T21:13:06.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:06.203 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-09-10T21:13:06.259 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-09-10T21:13:06.259 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:13:06.259 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:13:06.259 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:13:06.260 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:13:06.260 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:13:06.260 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:13:06.260 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:13:06.260 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:13:06.260 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:13:06.260 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:13:06.260 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:13:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:13:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:13:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:13:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:13:06.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:06.407 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:13:06.416 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:13:06.416 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:13:06.417 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:13:06.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:06.417 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-09-10T21:13:06.481 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:13:06.481 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T21:13:06.533 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:13:06.533 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:13:06.533 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:13:06.589 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T21:13:06.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:06.589 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:13:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:13:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:13:06.700 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:13:06.700 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:13:06.700 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:13:06.700 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 1@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:13:06.700 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:13:06.704 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:13:06.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:06.704 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:13:06.724 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:13:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:13:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:13:06.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:13:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:13:06.830 INFO:teuthology.orchestra.run.smithi052.stdout:client135074 2024-09-10T21:13:06.831 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:13:06.832 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:06.882 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:13:06.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:13:06.968 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:13:07.043 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:13:07.043 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:13:07.044 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:07.044 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:07.054 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:07.054 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:07.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:13:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.300+0000 7f9d82400640 1 -- 172.21.15.16:0/2691932247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d60003a60 msgr2=0x7f9d60003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.300+0000 7f9d82400640 1 --2- 172.21.15.16:0/2691932247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d60003a60 0x7f9d60003e80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0016c0 tx=0x7f9d6c01c710 comp rx=0 tx=0).stop 2024-09-10T21:13:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.300+0000 7f9d82400640 1 -- 172.21.15.16:0/2691932247 shutdown_connections 2024-09-10T21:13:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.300+0000 7f9d82400640 1 --2- 172.21.15.16:0/2691932247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d60003a60 0x7f9d60003e80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.300+0000 7f9d82400640 1 -- 172.21.15.16:0/2691932247 >> 172.21.15.16:0/2691932247 conn(0x7f9d7c0cfb20 msgr2=0x7f9d7c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.300+0000 7f9d82400640 1 -- 172.21.15.16:0/2691932247 shutdown_connections 2024-09-10T21:13:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.300+0000 7f9d82400640 1 -- 172.21.15.16:0/2691932247 wait complete. 2024-09-10T21:13:07.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d82400640 1 Processor -- start 2024-09-10T21:13:07.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d82400640 1 -- start start 2024-09-10T21:13:07.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d82400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d60003a60 0x7f9d7c14c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d82400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d7c1e5ef0 0x7f9d7c0b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d82400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c14bfe0 0x7f9d7c0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d82400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c070430 con 0x7f9d7c14bfe0 2024-09-10T21:13:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d82400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c0702b0 con 0x7f9d7c1e5ef0 2024-09-10T21:13:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d82400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7c0705b0 con 0x7f9d60003a60 2024-09-10T21:13:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c14bfe0 0x7f9d7c0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c14bfe0 0x7f9d7c0b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37082/0 (socket says 172.21.15.16:37082) 2024-09-10T21:13:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d7be00640 1 -- 172.21.15.16:0/570332033 learned_addr learned my addr 172.21.15.16:0/570332033 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d7b400640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d60003a60 0x7f9d7c14c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d72a00640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d7c1e5ef0 0x7f9d7c0b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d7be00640 1 -- 172.21.15.16:0/570332033 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d60003a60 msgr2=0x7f9d7c14c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d7be00640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d60003a60 0x7f9d7c14c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d7be00640 1 -- 172.21.15.16:0/570332033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d7c1e5ef0 msgr2=0x7f9d7c0b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d7be00640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d7c1e5ef0 0x7f9d7c0b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d7be00640 1 -- 172.21.15.16:0/570332033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7c0b11a0 con 0x7f9d7c14bfe0 2024-09-10T21:13:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d7b400640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d60003a60 0x7f9d7c14c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d7be00640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c14bfe0 0x7f9d7c0b3e20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c074b60 tx=0x7f9d74006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d73e00640 1 -- 172.21.15.16:0/570332033 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74015070 con 0x7f9d7c14bfe0 2024-09-10T21:13:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d73e00640 1 -- 172.21.15.16:0/570332033 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d7400fc90 con 0x7f9d7c14bfe0 2024-09-10T21:13:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d73e00640 1 -- 172.21.15.16:0/570332033 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74006910 con 0x7f9d7c14bfe0 2024-09-10T21:13:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d82400640 1 -- 172.21.15.16:0/570332033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7c157890 con 0x7f9d7c14bfe0 2024-09-10T21:13:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d82400640 1 -- 172.21.15.16:0/570332033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d7c157e50 con 0x7f9d7c14bfe0 2024-09-10T21:13:07.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.304+0000 7f9d82400640 1 -- 172.21.15.16:0/570332033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d7c072740 con 0x7f9d7c14bfe0 2024-09-10T21:13:07.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.308+0000 7f9d73e00640 1 -- 172.21.15.16:0/570332033 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9d74011690 con 0x7f9d7c14bfe0 2024-09-10T21:13:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.308+0000 7f9d73e00640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d58072610 0x7f9d58074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.308+0000 7f9d73e00640 1 -- 172.21.15.16:0/570332033 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f9d74093f20 con 0x7f9d7c14bfe0 2024-09-10T21:13:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.308+0000 7f9d7b400640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d58072610 0x7f9d58074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.308+0000 7f9d7b400640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d58072610 0x7f9d58074ad0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9d600057f0 tx=0x7f9d6c019500 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.312+0000 7f9d73e00640 1 -- 172.21.15.16:0/570332033 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9d7405f690 con 0x7f9d7c14bfe0 2024-09-10T21:13:07.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.496+0000 7f9d82400640 1 -- 172.21.15.16:0/570332033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9d7c0b5aa0 con 0x7f9d7c14bfe0 2024-09-10T21:13:07.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.496+0000 7f9d73e00640 1 -- 172.21.15.16:0/570332033 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) ==== 80+0+0 (secure 0 0 0) 0x7f9d7405ef90 con 0x7f9d7c14bfe0 2024-09-10T21:13:07.504 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:13:07.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.500+0000 7f9d82400640 1 -- 172.21.15.16:0/570332033 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d58072610 msgr2=0x7f9d58074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.500+0000 7f9d82400640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d58072610 0x7f9d58074ad0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9d600057f0 tx=0x7f9d6c019500 comp rx=0 tx=0).stop 2024-09-10T21:13:07.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.500+0000 7f9d82400640 1 -- 172.21.15.16:0/570332033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c14bfe0 msgr2=0x7f9d7c0b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.500+0000 7f9d82400640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c14bfe0 0x7f9d7c0b3e20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c074b60 tx=0x7f9d74006720 comp rx=0 tx=0).stop 2024-09-10T21:13:07.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.500+0000 7f9d82400640 1 -- 172.21.15.16:0/570332033 shutdown_connections 2024-09-10T21:13:07.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.500+0000 7f9d82400640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d58072610 0x7f9d58074ad0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.500+0000 7f9d82400640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d7c14bfe0 0x7f9d7c0b3e20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.500+0000 7f9d82400640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d7c1e5ef0 0x7f9d7c0b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.500+0000 7f9d82400640 1 --2- 172.21.15.16:0/570332033 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d60003a60 0x7f9d7c14c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.500+0000 7f9d82400640 1 -- 172.21.15.16:0/570332033 >> 172.21.15.16:0/570332033 conn(0x7f9d7c0cfb20 msgr2=0x7f9d7c158860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:07.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.500+0000 7f9d82400640 1 -- 172.21.15.16:0/570332033 shutdown_connections 2024-09-10T21:13:07.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.500+0000 7f9d82400640 1 -- 172.21.15.16:0/570332033 wait complete. 2024-09-10T21:13:07.525 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:13:07.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:07.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:13:07.552 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:13:07.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:07.553 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:13:07.606 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:13:07.606 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:13:07.606 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:07.615 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:13:07.615 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:07.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:13:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.804+0000 7fa0d9600640 1 -- 172.21.15.16:0/3430601132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0d40cffe0 msgr2=0x7fa0d40ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.804+0000 7fa0d9600640 1 --2- 172.21.15.16:0/3430601132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0d40cffe0 0x7fa0d40ce2b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40b4930 tx=0x7fa0c401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.804+0000 7fa0d9600640 1 -- 172.21.15.16:0/3430601132 shutdown_connections 2024-09-10T21:13:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.804+0000 7fa0d9600640 1 --2- 172.21.15.16:0/3430601132 >> v2:172.21.15.52:3300/0 conn(0x7fa0d40b2860 0x7fa0d4160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.804+0000 7fa0d9600640 1 --2- 172.21.15.16:0/3430601132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0d40cffe0 0x7fa0d40ce2b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.804+0000 7fa0d9600640 1 -- 172.21.15.16:0/3430601132 >> 172.21.15.16:0/3430601132 conn(0x7fa0d40d1d80 msgr2=0x7fa0d40d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.804+0000 7fa0d9600640 1 -- 172.21.15.16:0/3430601132 shutdown_connections 2024-09-10T21:13:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.804+0000 7fa0d9600640 1 -- 172.21.15.16:0/3430601132 wait complete. 2024-09-10T21:13:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d9600640 1 Processor -- start 2024-09-10T21:13:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d9600640 1 -- start start 2024-09-10T21:13:07.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d9600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0d40b2860 0x7fa0d41fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d9600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d40cffe0 0x7fa0d40b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0d40d03c0 0x7fa0d406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d9600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d40cd930 con 0x7fa0d40d03c0 2024-09-10T21:13:07.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d9600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d40cd7b0 con 0x7fa0d40cffe0 2024-09-10T21:13:07.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d9600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d40cdab0 con 0x7fa0d40b2860 2024-09-10T21:13:07.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0d40d03c0 0x7fa0d406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d2a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0d40b2860 0x7fa0d41fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d2a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0d40b2860 0x7fa0d41fb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:57046/0 (socket says 172.21.15.16:57046) 2024-09-10T21:13:07.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d2a00640 1 -- 172.21.15.16:0/62985304 learned_addr learned my addr 172.21.15.16:0/62985304 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:07.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0d40d03c0 0x7fa0d406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37098/0 (socket says 172.21.15.16:37098) 2024-09-10T21:13:07.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0cbe00640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d40cffe0 0x7fa0d40b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d2a00640 1 -- 172.21.15.16:0/62985304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d40cffe0 msgr2=0x7fa0d40b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d2a00640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d40cffe0 0x7fa0d40b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d2a00640 1 -- 172.21.15.16:0/62985304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0d40d03c0 msgr2=0x7fa0d406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d2a00640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0d40d03c0 0x7fa0d406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d2a00640 1 -- 172.21.15.16:0/62985304 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d406e7b0 con 0x7fa0d40b2860 2024-09-10T21:13:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d3400640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0d40d03c0 0x7fa0d406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d2a00640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0d40b2860 0x7fa0d41fb9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0c401cda0 tx=0x7fa0c40a5650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0caa00640 1 -- 172.21.15.16:0/62985304 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40bb070 con 0x7fa0d40b2860 2024-09-10T21:13:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0caa00640 1 -- 172.21.15.16:0/62985304 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0c40a6d80 con 0x7fa0d40b2860 2024-09-10T21:13:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0caa00640 1 -- 172.21.15.16:0/62985304 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40b2040 con 0x7fa0d40b2860 2024-09-10T21:13:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d9600640 1 -- 172.21.15.16:0/62985304 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d41faec0 con 0x7fa0d40b2860 2024-09-10T21:13:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d9600640 1 -- 172.21.15.16:0/62985304 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d406ea90 con 0x7fa0d40b2860 2024-09-10T21:13:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.808+0000 7fa0d9600640 1 -- 172.21.15.16:0/62985304 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d40ce2b0 con 0x7fa0d40b2860 2024-09-10T21:13:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.812+0000 7fa0caa00640 1 -- 172.21.15.16:0/62985304 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa0c4005290 con 0x7fa0d40b2860 2024-09-10T21:13:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.812+0000 7fa0caa00640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0b0072610 0x7fa0b0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.812+0000 7fa0caa00640 1 -- 172.21.15.16:0/62985304 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fa0c412cc90 con 0x7fa0d40b2860 2024-09-10T21:13:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.812+0000 7fa0cbe00640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0b0072610 0x7fa0b0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.812+0000 7fa0cbe00640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0b0072610 0x7fa0b0074ad0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4159bc0 tx=0x7fa0bc009060 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:07.816+0000 7fa0caa00640 1 -- 172.21.15.16:0/62985304 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa0c40164b0 con 0x7fa0d40b2860 2024-09-10T21:13:08.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.000+0000 7fa0d9600640 1 -- 172.21.15.16:0/62985304 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa0d40cf950 con 0x7fa0d40b2860 2024-09-10T21:13:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.000+0000 7fa0caa00640 1 -- 172.21.15.16:0/62985304 <== mon.2 v2:172.21.15.52:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) ==== 80+0+0 (secure 0 0 0) 0x7fa0c40f7d90 con 0x7fa0d40b2860 2024-09-10T21:13:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:13:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.000+0000 7fa0d9600640 1 -- 172.21.15.16:0/62985304 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0b0072610 msgr2=0x7fa0b0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.000+0000 7fa0d9600640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0b0072610 0x7fa0b0074ad0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4159bc0 tx=0x7fa0bc009060 comp rx=0 tx=0).stop 2024-09-10T21:13:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.000+0000 7fa0d9600640 1 -- 172.21.15.16:0/62985304 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0d40b2860 msgr2=0x7fa0d41fb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.000+0000 7fa0d9600640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0d40b2860 0x7fa0d41fb9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0c401cda0 tx=0x7fa0c40a5650 comp rx=0 tx=0).stop 2024-09-10T21:13:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.000+0000 7fa0d9600640 1 -- 172.21.15.16:0/62985304 shutdown_connections 2024-09-10T21:13:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.000+0000 7fa0d9600640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0b0072610 0x7fa0b0074ad0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.000+0000 7fa0d9600640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0d40d03c0 0x7fa0d406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.000+0000 7fa0d9600640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0d40cffe0 0x7fa0d40b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.000+0000 7fa0d9600640 1 --2- 172.21.15.16:0/62985304 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0d40b2860 0x7fa0d41fb9a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.004+0000 7fa0d9600640 1 -- 172.21.15.16:0/62985304 >> 172.21.15.16:0/62985304 conn(0x7fa0d40d1d80 msgr2=0x7fa0d40b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:08.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.004+0000 7fa0d9600640 1 -- 172.21.15.16:0/62985304 shutdown_connections 2024-09-10T21:13:08.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.004+0000 7fa0d9600640 1 -- 172.21.15.16:0/62985304 wait complete. 2024-09-10T21:13:08.028 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T21:13:08.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:08.028 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T21:13:08.057 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:13:08.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:08.057 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T21:13:08.114 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:13:08.114 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:08.114 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:08.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:13:08.123 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:13:08.123 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:13:08.124 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:08.165 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:13:08.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.304+0000 7fb28a200640 1 -- 172.21.15.16:0/1757221561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb284070520 msgr2=0x7fb28415e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.304+0000 7fb28a200640 1 --2- 172.21.15.16:0/1757221561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb284070520 0x7fb28415e5e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb28406db90 tx=0x7fb27c01cac0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.304+0000 7fb28a200640 1 -- 172.21.15.16:0/1757221561 shutdown_connections 2024-09-10T21:13:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.304+0000 7fb28a200640 1 --2- 172.21.15.16:0/1757221561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb284070520 0x7fb28415e5e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.304+0000 7fb28a200640 1 --2- 172.21.15.16:0/1757221561 >> v2:172.21.15.52:3300/0 conn(0x7fb28415bc70 0x7fb28406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.304+0000 7fb28a200640 1 --2- 172.21.15.16:0/1757221561 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb284154f90 0x7fb284072d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.304+0000 7fb28a200640 1 -- 172.21.15.16:0/1757221561 >> 172.21.15.16:0/1757221561 conn(0x7fb2840cfb20 msgr2=0x7fb2840cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.304+0000 7fb28a200640 1 -- 172.21.15.16:0/1757221561 shutdown_connections 2024-09-10T21:13:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.304+0000 7fb28a200640 1 -- 172.21.15.16:0/1757221561 wait complete. 2024-09-10T21:13:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb28a200640 1 Processor -- start 2024-09-10T21:13:08.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb28a200640 1 -- start start 2024-09-10T21:13:08.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb28a200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb284070520 0x7fb284145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:08.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb28a200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb284154f90 0x7fb284151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb28a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb28415bc70 0x7fb28414ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb28a200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2840cac00 con 0x7fb28415bc70 2024-09-10T21:13:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb28a200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb2840caa80 con 0x7fb284154f90 2024-09-10T21:13:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb28a200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb2840cad80 con 0x7fb284070520 2024-09-10T21:13:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb283e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb28415bc70 0x7fb28414ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb283e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb28415bc70 0x7fb28414ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37128/0 (socket says 172.21.15.16:37128) 2024-09-10T21:13:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb283e00640 1 -- 172.21.15.16:0/3174468271 learned_addr learned my addr 172.21.15.16:0/3174468271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb283400640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb284070520 0x7fb284145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb27be00640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb284154f90 0x7fb284151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb283e00640 1 -- 172.21.15.16:0/3174468271 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb284070520 msgr2=0x7fb284145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb283e00640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb284070520 0x7fb284145ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb283e00640 1 -- 172.21.15.16:0/3174468271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb284154f90 msgr2=0x7fb284151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb283e00640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb284154f90 0x7fb284151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb283e00640 1 -- 172.21.15.16:0/3174468271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28414db30 con 0x7fb28415bc70 2024-09-10T21:13:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb283400640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb284070520 0x7fb284145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb27be00640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb284154f90 0x7fb284151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb283e00640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb28415bc70 0x7fb28414ebd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009bd0 tx=0x7fb27c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb27b400640 1 -- 172.21.15.16:0/3174468271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0bb070 con 0x7fb28415bc70 2024-09-10T21:13:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb27b400640 1 -- 172.21.15.16:0/3174468271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb27c004e00 con 0x7fb28415bc70 2024-09-10T21:13:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb27b400640 1 -- 172.21.15.16:0/3174468271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0b2040 con 0x7fb28415bc70 2024-09-10T21:13:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb28a200640 1 -- 172.21.15.16:0/3174468271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28414de10 con 0x7fb28415bc70 2024-09-10T21:13:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb28a200640 1 -- 172.21.15.16:0/3174468271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28414e3d0 con 0x7fb28415bc70 2024-09-10T21:13:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.308+0000 7fb28a200640 1 -- 172.21.15.16:0/3174468271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb284072d40 con 0x7fb28415bc70 2024-09-10T21:13:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.312+0000 7fb27b400640 1 -- 172.21.15.16:0/3174468271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb27c0b5030 con 0x7fb28415bc70 2024-09-10T21:13:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.312+0000 7fb27b400640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb2600725e0 0x7fb260074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.312+0000 7fb27b400640 1 -- 172.21.15.16:0/3174468271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fb27c12ce00 con 0x7fb28415bc70 2024-09-10T21:13:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.312+0000 7fb283400640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb2600725e0 0x7fb260074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.312+0000 7fb283400640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb2600725e0 0x7fb260074aa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb2740058d0 tx=0x7fb274005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.316+0000 7fb27b400640 1 -- 172.21.15.16:0/3174468271 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb27c016970 con 0x7fb28415bc70 2024-09-10T21:13:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.528+0000 7fb28a200640 1 -- 172.21.15.16:0/3174468271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb28415aa30 con 0x7fb28415bc70 2024-09-10T21:13:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.528+0000 7fb27b400640 1 -- 172.21.15.16:0/3174468271 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5220 (secure 0 0 0) 0x7fb27c0f84d0 con 0x7fb28415bc70 2024-09-10T21:13:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:08.536 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"btime":"2024-09-10T21:12:56:932852+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/653119388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":653119388},{"type":"v1","addr":"172.21.15.16:6836","nonce":653119388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2947833730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2947833730},{"type":"v1","addr":"172.21.15.16:6837","nonce":2947833730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2865311562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2865311562},{"type":"v1","addr":"172.21.15.52:6834","nonce":2865311562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:12:53.304996+0000","modified":"2024-09-10T21:12:56.337640+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5592":{"gid":5592,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/2357796669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2357796669},{"type":"v1","addr":"172.21.15.52:6835","nonce":2357796669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5592,"qdb_cluster":[5592]},"id":4}]} 2024-09-10T21:13:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 33 2024-09-10T21:13:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.532+0000 7fb28a200640 1 -- 172.21.15.16:0/3174468271 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb2600725e0 msgr2=0x7fb260074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.532+0000 7fb28a200640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb2600725e0 0x7fb260074aa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb2740058d0 tx=0x7fb274005860 comp rx=0 tx=0).stop 2024-09-10T21:13:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.532+0000 7fb28a200640 1 -- 172.21.15.16:0/3174468271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb28415bc70 msgr2=0x7fb28414ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.532+0000 7fb28a200640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb28415bc70 0x7fb28414ebd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009bd0 tx=0x7fb27c003f60 comp rx=0 tx=0).stop 2024-09-10T21:13:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.532+0000 7fb28a200640 1 -- 172.21.15.16:0/3174468271 shutdown_connections 2024-09-10T21:13:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.532+0000 7fb28a200640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb2600725e0 0x7fb260074aa0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.532+0000 7fb28a200640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb28415bc70 0x7fb28414ebd0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.532+0000 7fb28a200640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb284154f90 0x7fb284151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.532+0000 7fb28a200640 1 --2- 172.21.15.16:0/3174468271 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb284070520 0x7fb284145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.532+0000 7fb28a200640 1 -- 172.21.15.16:0/3174468271 >> 172.21.15.16:0/3174468271 conn(0x7fb2840cfb20 msgr2=0x7fb2840cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.532+0000 7fb28a200640 1 -- 172.21.15.16:0/3174468271 shutdown_connections 2024-09-10T21:13:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.532+0000 7fb28a200640 1 -- 172.21.15.16:0/3174468271 wait complete. 2024-09-10T21:13:08.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:08.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.696+0000 7f5b66000640 1 -- 172.21.15.16:0/4292227088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b600d0890 msgr2=0x7f5b600d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:08.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.696+0000 7f5b66000640 1 --2- 172.21.15.16:0/4292227088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b600d0890 0x7f5b600d01c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5b600b3810 tx=0x7f5b5801f370 comp rx=0 tx=0).stop 2024-09-10T21:13:08.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.696+0000 7f5b66000640 1 -- 172.21.15.16:0/4292227088 shutdown_connections 2024-09-10T21:13:08.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.696+0000 7f5b66000640 1 --2- 172.21.15.16:0/4292227088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b600d0890 0x7f5b600d01c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.696+0000 7f5b66000640 1 --2- 172.21.15.16:0/4292227088 >> v2:172.21.15.52:3300/0 conn(0x7f5b6015f450 0x7f5b600cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.696+0000 7f5b66000640 1 -- 172.21.15.16:0/4292227088 >> 172.21.15.16:0/4292227088 conn(0x7f5b600d1d80 msgr2=0x7f5b600d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:08.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.696+0000 7f5b66000640 1 -- 172.21.15.16:0/4292227088 shutdown_connections 2024-09-10T21:13:08.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.696+0000 7f5b66000640 1 -- 172.21.15.16:0/4292227088 wait complete. 2024-09-10T21:13:08.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b66000640 1 Processor -- start 2024-09-10T21:13:08.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b66000640 1 -- start start 2024-09-10T21:13:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b66000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b600d0890 0x7f5b60153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b66000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6015f450 0x7f5b600b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b66000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b600c86c0 0x7f5b600c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b66000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b600b2e30 con 0x7f5b6015f450 2024-09-10T21:13:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b66000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b600b2cb0 con 0x7f5b600d0890 2024-09-10T21:13:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b66000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5b600b2fb0 con 0x7f5b600c86c0 2024-09-10T21:13:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b57e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6015f450 0x7f5b600b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:08.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b57e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6015f450 0x7f5b600b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37132/0 (socket says 172.21.15.16:37132) 2024-09-10T21:13:08.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b57e00640 1 -- 172.21.15.16:0/3773776990 learned_addr learned my addr 172.21.15.16:0/3773776990 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:08.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b5f400640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b600d0890 0x7f5b60153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:08.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b5fe00640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b600c86c0 0x7f5b600c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:08.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b57e00640 1 -- 172.21.15.16:0/3773776990 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b600c86c0 msgr2=0x7f5b600c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:08.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b57e00640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b600c86c0 0x7f5b600c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b57e00640 1 -- 172.21.15.16:0/3773776990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b600d0890 msgr2=0x7f5b60153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:08.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b57e00640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b600d0890 0x7f5b60153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b57e00640 1 -- 172.21.15.16:0/3773776990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b600b5a50 con 0x7f5b6015f450 2024-09-10T21:13:08.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b57e00640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6015f450 0x7f5b600b5510 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c430 tx=0x7f5b4800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:08.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b57400640 1 -- 172.21.15.16:0/3773776990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b480068a0 con 0x7f5b6015f450 2024-09-10T21:13:08.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b57400640 1 -- 172.21.15.16:0/3773776990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b48006b30 con 0x7f5b6015f450 2024-09-10T21:13:08.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b57400640 1 -- 172.21.15.16:0/3773776990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48007020 con 0x7f5b6015f450 2024-09-10T21:13:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b66000640 1 -- 172.21.15.16:0/3773776990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b600b60f0 con 0x7f5b6015f450 2024-09-10T21:13:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.700+0000 7f5b66000640 1 -- 172.21.15.16:0/3773776990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b600b66b0 con 0x7f5b6015f450 2024-09-10T21:13:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.704+0000 7f5b66000640 1 -- 172.21.15.16:0/3773776990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b600b1e30 con 0x7f5b6015f450 2024-09-10T21:13:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.704+0000 7f5b57400640 1 -- 172.21.15.16:0/3773776990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5b480071c0 con 0x7f5b6015f450 2024-09-10T21:13:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.704+0000 7f5b57400640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b3c072610 0x7f5b3c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.704+0000 7f5b57400640 1 -- 172.21.15.16:0/3773776990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f5b48092b50 con 0x7f5b6015f450 2024-09-10T21:13:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.704+0000 7f5b5f400640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b3c072610 0x7f5b3c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.704+0000 7f5b5f400640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b3c072610 0x7f5b3c074ad0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000bb50 tx=0x7f5b5000b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.708+0000 7f5b57400640 1 -- 172.21.15.16:0/3773776990 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5b4805dbd0 con 0x7f5b6015f450 2024-09-10T21:13:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.916+0000 7f5b66000640 1 -- 172.21.15.16:0/3773776990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b600d22a0 con 0x7f5b6015f450 2024-09-10T21:13:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.916+0000 7f5b57400640 1 -- 172.21.15.16:0/3773776990 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5220 (secure 0 0 0) 0x7f5b48062600 con 0x7f5b6015f450 2024-09-10T21:13:08.925 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:08.925 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"btime":"2024-09-10T21:12:56:932852+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/653119388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":653119388},{"type":"v1","addr":"172.21.15.16:6836","nonce":653119388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2947833730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2947833730},{"type":"v1","addr":"172.21.15.16:6837","nonce":2947833730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2865311562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2865311562},{"type":"v1","addr":"172.21.15.52:6834","nonce":2865311562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:12:53.304996+0000","modified":"2024-09-10T21:12:56.337640+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5592":{"gid":5592,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/2357796669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2357796669},{"type":"v1","addr":"172.21.15.52:6835","nonce":2357796669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5592,"qdb_cluster":[5592]},"id":4}]} 2024-09-10T21:13:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 33 2024-09-10T21:13:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.920+0000 7f5b66000640 1 -- 172.21.15.16:0/3773776990 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b3c072610 msgr2=0x7f5b3c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.920+0000 7f5b66000640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b3c072610 0x7f5b3c074ad0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000bb50 tx=0x7f5b5000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.920+0000 7f5b66000640 1 -- 172.21.15.16:0/3773776990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6015f450 msgr2=0x7f5b600b5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.920+0000 7f5b66000640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6015f450 0x7f5b600b5510 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c430 tx=0x7f5b4800c8f0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.920+0000 7f5b66000640 1 -- 172.21.15.16:0/3773776990 shutdown_connections 2024-09-10T21:13:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.920+0000 7f5b66000640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b3c072610 0x7f5b3c074ad0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.920+0000 7f5b66000640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b600c86c0 0x7f5b600c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.920+0000 7f5b66000640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6015f450 0x7f5b600b5510 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.920+0000 7f5b66000640 1 --2- 172.21.15.16:0/3773776990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b600d0890 0x7f5b60153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.920+0000 7f5b66000640 1 -- 172.21.15.16:0/3773776990 >> 172.21.15.16:0/3773776990 conn(0x7f5b600d1d80 msgr2=0x7f5b600cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.920+0000 7f5b66000640 1 -- 172.21.15.16:0/3773776990 shutdown_connections 2024-09-10T21:13:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:08.920+0000 7f5b66000640 1 -- 172.21.15.16:0/3773776990 wait complete. 2024-09-10T21:13:08.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:13:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.088+0000 7f4091000640 1 -- 172.21.15.16:0/2997311879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f408c0726b0 msgr2=0x7f408c072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.088+0000 7f4091000640 1 --2- 172.21.15.16:0/2997311879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f408c0726b0 0x7f408c072b40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f408c06daf0 tx=0x7f408401ccf0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.088+0000 7f4091000640 1 -- 172.21.15.16:0/2997311879 shutdown_connections 2024-09-10T21:13:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.088+0000 7f4091000640 1 --2- 172.21.15.16:0/2997311879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f408c0726b0 0x7f408c072b40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.088+0000 7f4091000640 1 --2- 172.21.15.16:0/2997311879 >> v2:172.21.15.52:3300/0 conn(0x7f408c1599d0 0x7f408c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.088+0000 7f4091000640 1 -- 172.21.15.16:0/2997311879 >> 172.21.15.16:0/2997311879 conn(0x7f408c0cfb20 msgr2=0x7f408c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.088+0000 7f4091000640 1 -- 172.21.15.16:0/2997311879 shutdown_connections 2024-09-10T21:13:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.088+0000 7f4091000640 1 -- 172.21.15.16:0/2997311879 wait complete. 2024-09-10T21:13:09.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.088+0000 7f4091000640 1 Processor -- start 2024-09-10T21:13:09.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.088+0000 7f4091000640 1 -- start start 2024-09-10T21:13:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4091000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f408c06e640 0x7f408c0b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4091000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f408c0726b0 0x7f408c147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4091000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f408c1599d0 0x7f408c1480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4091000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f408c168f10 con 0x7f408c1599d0 2024-09-10T21:13:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4091000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f408c168d90 con 0x7f408c0726b0 2024-09-10T21:13:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4091000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f408c169090 con 0x7f408c06e640 2024-09-10T21:13:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f408a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f408c1599d0 0x7f408c1480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f408a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f408c1599d0 0x7f408c1480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37150/0 (socket says 172.21.15.16:37150) 2024-09-10T21:13:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f408a800640 1 -- 172.21.15.16:0/3831066771 learned_addr learned my addr 172.21.15.16:0/3831066771 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4089e00640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f408c06e640 0x7f408c0b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4083e00640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f408c0726b0 0x7f408c147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f408a800640 1 -- 172.21.15.16:0/3831066771 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f408c06e640 msgr2=0x7f408c0b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f408a800640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f408c06e640 0x7f408c0b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f408a800640 1 -- 172.21.15.16:0/3831066771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f408c0726b0 msgr2=0x7f408c147b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f408a800640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f408c0726b0 0x7f408c147b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f408a800640 1 -- 172.21.15.16:0/3831066771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c0c95b0 con 0x7f408c1599d0 2024-09-10T21:13:09.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4083e00640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f408c0726b0 0x7f408c147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:09.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f408a800640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f408c1599d0 0x7f408c1480a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4084009bd0 tx=0x7f40840a5b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:09.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4082000640 1 -- 172.21.15.16:0/3831066771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40840bb070 con 0x7f408c1599d0 2024-09-10T21:13:09.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4082000640 1 -- 172.21.15.16:0/3831066771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4084004d40 con 0x7f408c1599d0 2024-09-10T21:13:09.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4082000640 1 -- 172.21.15.16:0/3831066771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40840b2040 con 0x7f408c1599d0 2024-09-10T21:13:09.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4091000640 1 -- 172.21.15.16:0/3831066771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408c0c8a30 con 0x7f408c1599d0 2024-09-10T21:13:09.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4091000640 1 -- 172.21.15.16:0/3831066771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408c0c8ff0 con 0x7f408c1599d0 2024-09-10T21:13:09.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4091000640 1 -- 172.21.15.16:0/3831066771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f408c072b40 con 0x7f408c1599d0 2024-09-10T21:13:09.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.092+0000 7f4082000640 1 -- 172.21.15.16:0/3831066771 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f40840b5030 con 0x7f408c1599d0 2024-09-10T21:13:09.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.096+0000 7f4082000640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40680725e0 0x7f4068074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.096+0000 7f4082000640 1 -- 172.21.15.16:0/3831066771 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f408412ce30 con 0x7f408c1599d0 2024-09-10T21:13:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.096+0000 7f4089e00640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40680725e0 0x7f4068074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.096+0000 7f4089e00640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40680725e0 0x7f4068074aa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f407c00ab40 tx=0x7f407c00a250 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.100+0000 7f4082000640 1 -- 172.21.15.16:0/3831066771 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4084016970 con 0x7f408c1599d0 2024-09-10T21:13:09.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.272+0000 7f4091000640 1 -- 172.21.15.16:0/3831066771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f408c166200 con 0x7f408c1599d0 2024-09-10T21:13:09.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.272+0000 7f4082000640 1 -- 172.21.15.16:0/3831066771 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16677 (secure 0 0 0) 0x7f40840f7e20 con 0x7f408c1599d0 2024-09-10T21:13:09.280 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:09.280 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:54.920251+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:12:51.039811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T21:12:52.273850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:13:09.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.276+0000 7f4091000640 1 -- 172.21.15.16:0/3831066771 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40680725e0 msgr2=0x7f4068074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.276+0000 7f4091000640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40680725e0 0x7f4068074aa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f407c00ab40 tx=0x7f407c00a250 comp rx=0 tx=0).stop 2024-09-10T21:13:09.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.276+0000 7f4091000640 1 -- 172.21.15.16:0/3831066771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f408c1599d0 msgr2=0x7f408c1480a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.276+0000 7f4091000640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f408c1599d0 0x7f408c1480a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4084009bd0 tx=0x7f40840a5b20 comp rx=0 tx=0).stop 2024-09-10T21:13:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.276+0000 7f4091000640 1 -- 172.21.15.16:0/3831066771 shutdown_connections 2024-09-10T21:13:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.276+0000 7f4091000640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40680725e0 0x7f4068074aa0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.276+0000 7f4091000640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f408c1599d0 0x7f408c1480a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.276+0000 7f4091000640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f408c0726b0 0x7f408c147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.276+0000 7f4091000640 1 --2- 172.21.15.16:0/3831066771 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f408c06e640 0x7f408c0b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.276+0000 7f4091000640 1 -- 172.21.15.16:0/3831066771 >> 172.21.15.16:0/3831066771 conn(0x7f408c0cfb20 msgr2=0x7f408c0b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.276+0000 7f4091000640 1 -- 172.21.15.16:0/3831066771 shutdown_connections 2024-09-10T21:13:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.276+0000 7f4091000640 1 -- 172.21.15.16:0/3831066771 wait complete. 2024-09-10T21:13:09.303 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:13:09.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:09.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.444+0000 7efcfde00640 1 -- 172.21.15.16:0/826101545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcdc004010 msgr2=0x7efcdc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.444+0000 7efcfde00640 1 --2- 172.21.15.16:0/826101545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcdc004010 0x7efcdc004810 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efce00016f0 tx=0x7efce001c7b0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.444+0000 7efcf7400640 1 -- 172.21.15.16:0/826101545 reap_dead start 2024-09-10T21:13:09.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.444+0000 7efcfde00640 1 -- 172.21.15.16:0/826101545 shutdown_connections 2024-09-10T21:13:09.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.444+0000 7efcfde00640 1 -- 172.21.15.16:0/826101545 >> 172.21.15.16:0/826101545 conn(0x7efcf80cd6b0 msgr2=0x7efcf80cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.444+0000 7efcfde00640 1 -- 172.21.15.16:0/826101545 shutdown_connections 2024-09-10T21:13:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.444+0000 7efcfde00640 1 -- 172.21.15.16:0/826101545 wait complete. 2024-09-10T21:13:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.444+0000 7efcfde00640 1 Processor -- start 2024-09-10T21:13:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.444+0000 7efcfde00640 1 -- start start 2024-09-10T21:13:09.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcfde00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcdc004010 0x7efcf81f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcfde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf8070a20 0x7efcf80b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcfde00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efcf81f45c0 0x7efcf80c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcfde00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efcf80d0530 con 0x7efcf8070a20 2024-09-10T21:13:09.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcfde00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efcf80d03b0 con 0x7efcdc004010 2024-09-10T21:13:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcfde00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7efcf80d06b0 con 0x7efcf81f45c0 2024-09-10T21:13:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcefe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf8070a20 0x7efcf80b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcefe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf8070a20 0x7efcf80b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37172/0 (socket says 172.21.15.16:37172) 2024-09-10T21:13:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcefe00640 1 -- 172.21.15.16:0/1115588806 learned_addr learned my addr 172.21.15.16:0/1115588806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcf7400640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efcf81f45c0 0x7efcf80c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcf6a00640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcdc004010 0x7efcf81f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcefe00640 1 -- 172.21.15.16:0/1115588806 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efcf81f45c0 msgr2=0x7efcf80c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcefe00640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efcf81f45c0 0x7efcf80c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcefe00640 1 -- 172.21.15.16:0/1115588806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcdc004010 msgr2=0x7efcf81f1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcefe00640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcdc004010 0x7efcf81f1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcefe00640 1 -- 172.21.15.16:0/1115588806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf815c4d0 con 0x7efcf8070a20 2024-09-10T21:13:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcf6a00640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcdc004010 0x7efcf81f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:09.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcf7400640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efcf81f45c0 0x7efcf80c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:09.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcefe00640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf8070a20 0x7efcf80b1c00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efce001b080 tx=0x7efce0001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:09.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcef400640 1 -- 172.21.15.16:0/1115588806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce00a5820 con 0x7efcf8070a20 2024-09-10T21:13:09.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcef400640 1 -- 172.21.15.16:0/1115588806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efce00a5ab0 con 0x7efcf8070a20 2024-09-10T21:13:09.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcfde00640 1 -- 172.21.15.16:0/1115588806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf815c7b0 con 0x7efcf8070a20 2024-09-10T21:13:09.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcef400640 1 -- 172.21.15.16:0/1115588806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce0003600 con 0x7efcf8070a20 2024-09-10T21:13:09.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcfde00640 1 -- 172.21.15.16:0/1115588806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf815cd70 con 0x7efcf8070a20 2024-09-10T21:13:09.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.448+0000 7efcfde00640 1 -- 172.21.15.16:0/1115588806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf80b2a90 con 0x7efcf8070a20 2024-09-10T21:13:09.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.452+0000 7efcef400640 1 -- 172.21.15.16:0/1115588806 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efce00c0030 con 0x7efcf8070a20 2024-09-10T21:13:09.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.452+0000 7efcef400640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efcd40725e0 0x7efcd4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.452+0000 7efcef400640 1 -- 172.21.15.16:0/1115588806 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7efce012cd30 con 0x7efcf8070a20 2024-09-10T21:13:09.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.452+0000 7efcf6a00640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efcd40725e0 0x7efcd4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.452+0000 7efcf6a00640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efcd40725e0 0x7efcd4074aa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efcf80a2200 tx=0x7efce80091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:09.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.456+0000 7efcef400640 1 -- 172.21.15.16:0/1115588806 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efce0016510 con 0x7efcf8070a20 2024-09-10T21:13:09.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcfde00640 1 -- 172.21.15.16:0/1115588806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcf80d18b0 con 0x7efcf8070a20 2024-09-10T21:13:09.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcef400640 1 -- 172.21.15.16:0/1115588806 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5220 (secure 0 0 0) 0x7efce00f8370 con 0x7efcf8070a20 2024-09-10T21:13:09.674 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:09.674 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"btime":"2024-09-10T21:12:56:932852+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/653119388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":653119388},{"type":"v1","addr":"172.21.15.16:6836","nonce":653119388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2947833730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2947833730},{"type":"v1","addr":"172.21.15.16:6837","nonce":2947833730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2865311562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2865311562},{"type":"v1","addr":"172.21.15.52:6834","nonce":2865311562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:12:53.304996+0000","modified":"2024-09-10T21:12:56.337640+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5592":{"gid":5592,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/2357796669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2357796669},{"type":"v1","addr":"172.21.15.52:6835","nonce":2357796669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5592,"qdb_cluster":[5592]},"id":4}]} 2024-09-10T21:13:09.675 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 33 2024-09-10T21:13:09.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcfde00640 1 -- 172.21.15.16:0/1115588806 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efcd40725e0 msgr2=0x7efcd4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcfde00640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efcd40725e0 0x7efcd4074aa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efcf80a2200 tx=0x7efce80091e0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcfde00640 1 -- 172.21.15.16:0/1115588806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf8070a20 msgr2=0x7efcf80b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcfde00640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf8070a20 0x7efcf80b1c00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efce001b080 tx=0x7efce0001310 comp rx=0 tx=0).stop 2024-09-10T21:13:09.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcfde00640 1 -- 172.21.15.16:0/1115588806 shutdown_connections 2024-09-10T21:13:09.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcfde00640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efcd40725e0 0x7efcd4074aa0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcfde00640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efcf81f45c0 0x7efcf80c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcfde00640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf8070a20 0x7efcf80b1c00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcfde00640 1 --2- 172.21.15.16:0/1115588806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcdc004010 0x7efcf81f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcfde00640 1 -- 172.21.15.16:0/1115588806 >> 172.21.15.16:0/1115588806 conn(0x7efcf80cd6b0 msgr2=0x7efcf8071a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:09.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcfde00640 1 -- 172.21.15.16:0/1115588806 shutdown_connections 2024-09-10T21:13:09.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.668+0000 7efcfde00640 1 -- 172.21.15.16:0/1115588806 wait complete. 2024-09-10T21:13:09.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 -- 172.21.15.16:0/3033459136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b0161560 msgr2=0x7f97b00d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 --2- 172.21.15.16:0/3033459136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b0161560 0x7f97b00d4a20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f97b00b9f80 tx=0x7f97a001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 -- 172.21.15.16:0/3033459136 shutdown_connections 2024-09-10T21:13:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 --2- 172.21.15.16:0/3033459136 >> v2:172.21.15.52:3300/0 conn(0x7f97b00d0e50 0x7f97b00b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 --2- 172.21.15.16:0/3033459136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b0161560 0x7f97b00d4a20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 -- 172.21.15.16:0/3033459136 >> 172.21.15.16:0/3033459136 conn(0x7f97b00d3f00 msgr2=0x7f97b00d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 -- 172.21.15.16:0/3033459136 shutdown_connections 2024-09-10T21:13:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 -- 172.21.15.16:0/3033459136 wait complete. 2024-09-10T21:13:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 Processor -- start 2024-09-10T21:13:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 -- start start 2024-09-10T21:13:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b00b2c20 0x7f97b0154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b00d0e50 0x7f97b0158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97b0161560 0x7f97b00b8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97b016ad30 con 0x7f97b00d0e50 2024-09-10T21:13:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97b016abb0 con 0x7f97b00b2c20 2024-09-10T21:13:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.836+0000 7f97b6000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f97b016aeb0 con 0x7f97b0161560 2024-09-10T21:13:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b00d0e50 0x7f97b0158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b00d0e50 0x7f97b0158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37194/0 (socket says 172.21.15.16:37194) 2024-09-10T21:13:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97a7e00640 1 -- 172.21.15.16:0/1193853067 learned_addr learned my addr 172.21.15.16:0/1193853067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97b4c00640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b00b2c20 0x7f97b0154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97a7e00640 1 -- 172.21.15.16:0/1193853067 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97b0161560 msgr2=0x7f97b00b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97a7e00640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97b0161560 0x7f97b00b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97a7e00640 1 -- 172.21.15.16:0/1193853067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b00b2c20 msgr2=0x7f97b0154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97a7e00640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b00b2c20 0x7f97b0154520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97a7e00640 1 -- 172.21.15.16:0/1193853067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b00b8b90 con 0x7f97b00d0e50 2024-09-10T21:13:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97b4c00640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b00b2c20 0x7f97b0154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97a7e00640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b00d0e50 0x7f97b0158500 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f979800c480 tx=0x7f979800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97ad600640 1 -- 172.21.15.16:0/1193853067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798017070 con 0x7f97b00d0e50 2024-09-10T21:13:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97ad600640 1 -- 172.21.15.16:0/1193853067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9798003600 con 0x7f97b00d0e50 2024-09-10T21:13:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97b6000640 1 -- 172.21.15.16:0/1193853067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b015bbb0 con 0x7f97b00d0e50 2024-09-10T21:13:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97ad600640 1 -- 172.21.15.16:0/1193853067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798003a50 con 0x7f97b00d0e50 2024-09-10T21:13:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97b6000640 1 -- 172.21.15.16:0/1193853067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b0158a40 con 0x7f97b00d0e50 2024-09-10T21:13:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97b6000640 1 -- 172.21.15.16:0/1193853067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b00b0d00 con 0x7f97b00d0e50 2024-09-10T21:13:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.840+0000 7f97ad600640 1 -- 172.21.15.16:0/1193853067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9798003bf0 con 0x7f97b00d0e50 2024-09-10T21:13:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.844+0000 7f97ad600640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f978c0725e0 0x7f978c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.844+0000 7f97ad600640 1 -- 172.21.15.16:0/1193853067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f9798092910 con 0x7f97b00d0e50 2024-09-10T21:13:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.844+0000 7f97b4c00640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f978c0725e0 0x7f978c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.844+0000 7f97b4c00640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f978c0725e0 0x7f978c074aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f97a00af000 tx=0x7f97a001b100 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:09.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:09.848+0000 7f97ad600640 1 -- 172.21.15.16:0/1193853067 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f979805e0a0 con 0x7f97b00d0e50 2024-09-10T21:13:10.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97b6000640 1 -- 172.21.15.16:0/1193853067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97b00d0310 con 0x7f97b00d0e50 2024-09-10T21:13:10.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97ad600640 1 -- 172.21.15.16:0/1193853067 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5220 (secure 0 0 0) 0x7f979805d9a0 con 0x7f97b00d0e50 2024-09-10T21:13:10.070 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:10.070 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"btime":"2024-09-10T21:12:56:932852+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/653119388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":653119388},{"type":"v1","addr":"172.21.15.16:6836","nonce":653119388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2947833730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2947833730},{"type":"v1","addr":"172.21.15.16:6837","nonce":2947833730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2865311562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2865311562},{"type":"v1","addr":"172.21.15.52:6834","nonce":2865311562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:12:53.304996+0000","modified":"2024-09-10T21:12:56.337640+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5592":{"gid":5592,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/2357796669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2357796669},{"type":"v1","addr":"172.21.15.52:6835","nonce":2357796669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5592,"qdb_cluster":[5592]},"id":4}]} 2024-09-10T21:13:10.071 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 33 2024-09-10T21:13:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97b6000640 1 -- 172.21.15.16:0/1193853067 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f978c0725e0 msgr2=0x7f978c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97b6000640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f978c0725e0 0x7f978c074aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f97a00af000 tx=0x7f97a001b100 comp rx=0 tx=0).stop 2024-09-10T21:13:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97b6000640 1 -- 172.21.15.16:0/1193853067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b00d0e50 msgr2=0x7f97b0158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97b6000640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b00d0e50 0x7f97b0158500 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f979800c480 tx=0x7f979800c940 comp rx=0 tx=0).stop 2024-09-10T21:13:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97b6000640 1 -- 172.21.15.16:0/1193853067 shutdown_connections 2024-09-10T21:13:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97b6000640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f978c0725e0 0x7f978c074aa0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97b6000640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97b0161560 0x7f97b00b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97b6000640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97b00d0e50 0x7f97b0158500 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97b6000640 1 --2- 172.21.15.16:0/1193853067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b00b2c20 0x7f97b0154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97b6000640 1 -- 172.21.15.16:0/1193853067 >> 172.21.15.16:0/1193853067 conn(0x7f97b00d3f00 msgr2=0x7f97b00d1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:10.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97b6000640 1 -- 172.21.15.16:0/1193853067 shutdown_connections 2024-09-10T21:13:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.064+0000 7f97b6000640 1 -- 172.21.15.16:0/1193853067 wait complete. 2024-09-10T21:13:10.093 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:13:10.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.232+0000 7f4173a00640 1 -- 172.21.15.16:0/824342419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f416c0b13c0 msgr2=0x7f416c071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.232+0000 7f4173a00640 1 --2- 172.21.15.16:0/824342419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f416c0b13c0 0x7f416c071520 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f416c073290 tx=0x7f415801c7a0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.232+0000 7f4173a00640 1 -- 172.21.15.16:0/824342419 shutdown_connections 2024-09-10T21:13:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.232+0000 7f4173a00640 1 --2- 172.21.15.16:0/824342419 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f416c0cfc80 0x7f416c0d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.232+0000 7f4173a00640 1 --2- 172.21.15.16:0/824342419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f416c0b13c0 0x7f416c071520 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.232+0000 7f4173a00640 1 --2- 172.21.15.16:0/824342419 >> v2:172.21.15.52:3300/0 conn(0x7f416c15f400 0x7f416c070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.232+0000 7f4173a00640 1 -- 172.21.15.16:0/824342419 >> 172.21.15.16:0/824342419 conn(0x7f416c09b130 msgr2=0x7f416c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.232+0000 7f4173a00640 1 -- 172.21.15.16:0/824342419 shutdown_connections 2024-09-10T21:13:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.232+0000 7f4173a00640 1 -- 172.21.15.16:0/824342419 wait complete. 2024-09-10T21:13:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.232+0000 7f4173a00640 1 Processor -- start 2024-09-10T21:13:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.232+0000 7f4173a00640 1 -- start start 2024-09-10T21:13:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4173a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f416c0b13c0 0x7f416c15de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4173a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f416c0cfc80 0x7f416c154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4173a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f416c15f400 0x7f416c1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4173a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f416c0afd90 con 0x7f416c0b13c0 2024-09-10T21:13:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4173a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f416c0d1260 con 0x7f416c0cfc80 2024-09-10T21:13:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4173a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f416c0aff10 con 0x7f416c15f400 2024-09-10T21:13:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4171000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f416c0b13c0 0x7f416c15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4171000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f416c0b13c0 0x7f416c15de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37204/0 (socket says 172.21.15.16:37204) 2024-09-10T21:13:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4171000640 1 -- 172.21.15.16:0/2548548564 learned_addr learned my addr 172.21.15.16:0/2548548564 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f416be00640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f416c0cfc80 0x7f416c154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4171a00640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f416c15f400 0x7f416c1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4171000640 1 -- 172.21.15.16:0/2548548564 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f416c15f400 msgr2=0x7f416c1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4171000640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f416c15f400 0x7f416c1546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4171000640 1 -- 172.21.15.16:0/2548548564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f416c0cfc80 msgr2=0x7f416c154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4171000640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f416c0cfc80 0x7f416c154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4171000640 1 -- 172.21.15.16:0/2548548564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1f9330 con 0x7f416c0b13c0 2024-09-10T21:13:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4171000640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f416c0b13c0 0x7f416c15de70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f416000d700 tx=0x7f416000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4168c00640 1 -- 172.21.15.16:0/2548548564 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160017070 con 0x7f416c0b13c0 2024-09-10T21:13:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4168c00640 1 -- 172.21.15.16:0/2548548564 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4160009d20 con 0x7f416c0b13c0 2024-09-10T21:13:10.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4168c00640 1 -- 172.21.15.16:0/2548548564 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41600106c0 con 0x7f416c0b13c0 2024-09-10T21:13:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4173a00640 1 -- 172.21.15.16:0/2548548564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c1f9610 con 0x7f416c0b13c0 2024-09-10T21:13:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4173a00640 1 -- 172.21.15.16:0/2548548564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c1f9bd0 con 0x7f416c0b13c0 2024-09-10T21:13:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.236+0000 7f4173a00640 1 -- 172.21.15.16:0/2548548564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c0d0110 con 0x7f416c0b13c0 2024-09-10T21:13:10.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.240+0000 7f4168c00640 1 -- 172.21.15.16:0/2548548564 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4160010860 con 0x7f416c0b13c0 2024-09-10T21:13:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.240+0000 7f4168c00640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f414c0725e0 0x7f414c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.240+0000 7f4168c00640 1 -- 172.21.15.16:0/2548548564 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f4160093740 con 0x7f416c0b13c0 2024-09-10T21:13:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.240+0000 7f416be00640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f414c0725e0 0x7f414c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.240+0000 7f416be00640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f414c0725e0 0x7f414c074aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f41580af000 tx=0x7f415801afe0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.244+0000 7f4168c00640 1 -- 172.21.15.16:0/2548548564 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f416005e730 con 0x7f416c0b13c0 2024-09-10T21:13:10.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.416+0000 7f4173a00640 1 -- 172.21.15.16:0/2548548564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f416c159500 con 0x7f416c0b13c0 2024-09-10T21:13:10.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.420+0000 7f4168c00640 1 -- 172.21.15.16:0/2548548564 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16677 (secure 0 0 0) 0x7f4160063160 con 0x7f416c0b13c0 2024-09-10T21:13:10.426 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:10.426 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:12:54.920251+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:12:51.039811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T21:12:52.273850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:13:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.420+0000 7f4173a00640 1 -- 172.21.15.16:0/2548548564 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f414c0725e0 msgr2=0x7f414c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.420+0000 7f4173a00640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f414c0725e0 0x7f414c074aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f41580af000 tx=0x7f415801afe0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.420+0000 7f4173a00640 1 -- 172.21.15.16:0/2548548564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f416c0b13c0 msgr2=0x7f416c15de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.420+0000 7f4173a00640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f416c0b13c0 0x7f416c15de70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f416000d700 tx=0x7f416000dbc0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.420+0000 7f4173a00640 1 -- 172.21.15.16:0/2548548564 shutdown_connections 2024-09-10T21:13:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.420+0000 7f4173a00640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f414c0725e0 0x7f414c074aa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.420+0000 7f4173a00640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f416c15f400 0x7f416c1546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.420+0000 7f4173a00640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f416c0cfc80 0x7f416c154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.420+0000 7f4173a00640 1 --2- 172.21.15.16:0/2548548564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f416c0b13c0 0x7f416c15de70 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.420+0000 7f4173a00640 1 -- 172.21.15.16:0/2548548564 >> 172.21.15.16:0/2548548564 conn(0x7f416c09b130 msgr2=0x7f416c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.420+0000 7f4173a00640 1 -- 172.21.15.16:0/2548548564 shutdown_connections 2024-09-10T21:13:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.420+0000 7f4173a00640 1 -- 172.21.15.16:0/2548548564 wait complete. 2024-09-10T21:13:10.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:13:10.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 -- 172.21.15.16:0/2106683742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3f00716f0 msgr2=0x7ff3f00af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 --2- 172.21.15.16:0/2106683742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3f00716f0 0x7ff3f00af580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009840 tx=0x7ff3dc01c700 comp rx=0 tx=0).stop 2024-09-10T21:13:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 -- 172.21.15.16:0/2106683742 shutdown_connections 2024-09-10T21:13:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 --2- 172.21.15.16:0/2106683742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3f00716f0 0x7ff3f00af580 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 --2- 172.21.15.16:0/2106683742 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3f00b13c0 0x7ff3f00afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 -- 172.21.15.16:0/2106683742 >> 172.21.15.16:0/2106683742 conn(0x7ff3f009b130 msgr2=0x7ff3f00d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 -- 172.21.15.16:0/2106683742 shutdown_connections 2024-09-10T21:13:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 -- 172.21.15.16:0/2106683742 wait complete. 2024-09-10T21:13:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 Processor -- start 2024-09-10T21:13:10.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 -- start start 2024-09-10T21:13:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3f00716f0 0x7ff3f01648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f00b13c0 0x7ff3f015e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3f015f400 0x7ff3f014dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f0168fb0 con 0x7ff3f00b13c0 2024-09-10T21:13:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f0168e30 con 0x7ff3f015f400 2024-09-10T21:13:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3f5600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f0169130 con 0x7ff3f00716f0 2024-09-10T21:13:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3ee000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f00b13c0 0x7ff3f015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3ee000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f00b13c0 0x7ff3f015e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37208/0 (socket says 172.21.15.16:37208) 2024-09-10T21:13:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3ee000640 1 -- 172.21.15.16:0/650404990 learned_addr learned my addr 172.21.15.16:0/650404990 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3eea00640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3f00716f0 0x7ff3f01648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3ef400640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3f015f400 0x7ff3f014dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3ee000640 1 -- 172.21.15.16:0/650404990 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3f00716f0 msgr2=0x7ff3f01648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3ee000640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3f00716f0 0x7ff3f01648f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3ee000640 1 -- 172.21.15.16:0/650404990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3f015f400 msgr2=0x7ff3f014dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3ee000640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3f015f400 0x7ff3f014dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3ee000640 1 -- 172.21.15.16:0/650404990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f00b4e00 con 0x7ff3f00b13c0 2024-09-10T21:13:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.592+0000 7ff3eea00640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3f00716f0 0x7ff3f01648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.596+0000 7ff3ee000640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f00b13c0 0x7ff3f015e400 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc01adb0 tx=0x7ff3dc0a65f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.596+0000 7ff3e6a00640 1 -- 172.21.15.16:0/650404990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0bc030 con 0x7ff3f00b13c0 2024-09-10T21:13:10.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.596+0000 7ff3e6a00640 1 -- 172.21.15.16:0/650404990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3dc004120 con 0x7ff3f00b13c0 2024-09-10T21:13:10.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.596+0000 7ff3e6a00640 1 -- 172.21.15.16:0/650404990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc005020 con 0x7ff3f00b13c0 2024-09-10T21:13:10.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.596+0000 7ff3f5600640 1 -- 172.21.15.16:0/650404990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f00b50e0 con 0x7ff3f00b13c0 2024-09-10T21:13:10.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.596+0000 7ff3f5600640 1 -- 172.21.15.16:0/650404990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f00b56a0 con 0x7ff3f00b13c0 2024-09-10T21:13:10.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.596+0000 7ff3f5600640 1 -- 172.21.15.16:0/650404990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f00b1a40 con 0x7ff3f00b13c0 2024-09-10T21:13:10.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.596+0000 7ff3e6a00640 1 -- 172.21.15.16:0/650404990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff3dc0c9060 con 0x7ff3f00b13c0 2024-09-10T21:13:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.596+0000 7ff3e6a00640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff3d00725e0 0x7ff3d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.596+0000 7ff3e6a00640 1 -- 172.21.15.16:0/650404990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5767+0+0 (secure 0 0 0) 0x7ff3dc12de70 con 0x7ff3f00b13c0 2024-09-10T21:13:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.596+0000 7ff3eea00640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff3d00725e0 0x7ff3d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.600+0000 7ff3eea00640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff3d00725e0 0x7ff3d0074aa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0007fd0 tx=0x7ff3e0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:10.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.600+0000 7ff3e6a00640 1 -- 172.21.15.16:0/650404990 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff3dc016410 con 0x7ff3f00b13c0 2024-09-10T21:13:10.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.808+0000 7ff3f5600640 1 -- 172.21.15.16:0/650404990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff3f00761e0 con 0x7ff3f00b13c0 2024-09-10T21:13:10.817 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-10T21:13:10.812+0000 7ff429400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:13:10.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.812+0000 7ff3e6a00640 1 -- 172.21.15.16:0/650404990 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) ==== 161+0+0 (secure 0 0 0) 0x7ff3dc0f9620 con 0x7ff3f00b13c0 2024-09-10T21:13:10.819 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:13:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.816+0000 7ff3f5600640 1 -- 172.21.15.16:0/650404990 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff3d00725e0 msgr2=0x7ff3d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.816+0000 7ff3f5600640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff3d00725e0 0x7ff3d0074aa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0007fd0 tx=0x7ff3e0009040 comp rx=0 tx=0).stop 2024-09-10T21:13:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.816+0000 7ff3f5600640 1 -- 172.21.15.16:0/650404990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f00b13c0 msgr2=0x7ff3f015e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.816+0000 7ff3f5600640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f00b13c0 0x7ff3f015e400 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc01adb0 tx=0x7ff3dc0a65f0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.816+0000 7ff3f5600640 1 -- 172.21.15.16:0/650404990 shutdown_connections 2024-09-10T21:13:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.816+0000 7ff3f5600640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff3d00725e0 0x7ff3d0074aa0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.816+0000 7ff3f5600640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3f015f400 0x7ff3f014dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.816+0000 7ff3f5600640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f00b13c0 0x7ff3f015e400 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.816+0000 7ff3f5600640 1 --2- 172.21.15.16:0/650404990 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3f00716f0 0x7ff3f01648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.816+0000 7ff3f5600640 1 -- 172.21.15.16:0/650404990 >> 172.21.15.16:0/650404990 conn(0x7ff3f009b130 msgr2=0x7ff3f006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.816+0000 7ff3f5600640 1 -- 172.21.15.16:0/650404990 shutdown_connections 2024-09-10T21:13:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.816+0000 7ff3f5600640 1 -- 172.21.15.16:0/650404990 wait complete. 2024-09-10T21:13:10.830 INFO:tasks.ceph.mds.d.smithi052.stderr: -2291> 2024-09-10T21:12:55.890+0000 7f9026800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:13:10.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:13:10.879 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:13:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.984+0000 7f4270a00640 1 -- 172.21.15.16:0/3920906519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c15ad80 msgr2=0x7f426c0b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.984+0000 7f4270a00640 1 --2- 172.21.15.16:0/3920906519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c15ad80 0x7f426c0b3f80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4254009900 tx=0x7f425401c850 comp rx=0 tx=0).stop 2024-09-10T21:13:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.984+0000 7f4270a00640 1 -- 172.21.15.16:0/3920906519 shutdown_connections 2024-09-10T21:13:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.984+0000 7f4270a00640 1 --2- 172.21.15.16:0/3920906519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c15ad80 0x7f426c0b3f80 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.984+0000 7f4270a00640 1 --2- 172.21.15.16:0/3920906519 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f426c0b33b0 0x7f426c070070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.984+0000 7f4270a00640 1 -- 172.21.15.16:0/3920906519 >> 172.21.15.16:0/3920906519 conn(0x7f426c0b16a0 msgr2=0x7f426c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:10.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.984+0000 7f4270a00640 1 -- 172.21.15.16:0/3920906519 shutdown_connections 2024-09-10T21:13:10.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.984+0000 7f4270a00640 1 -- 172.21.15.16:0/3920906519 wait complete. 2024-09-10T21:13:10.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4270a00640 1 Processor -- start 2024-09-10T21:13:10.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4270a00640 1 -- start start 2024-09-10T21:13:10.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4270a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0b33b0 0x7f426c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:10.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4270a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f426c15a9a0 0x7f426c0b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:10.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4270a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f426c15ad80 0x7f426c0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:10.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4270a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f426c167e70 con 0x7f426c0b33b0 2024-09-10T21:13:10.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4270a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f426c167cf0 con 0x7f426c15ad80 2024-09-10T21:13:10.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4270a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f426c167ff0 con 0x7f426c15a9a0 2024-09-10T21:13:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4269e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0b33b0 0x7f426c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4269e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0b33b0 0x7f426c15b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37232/0 (socket says 172.21.15.16:37232) 2024-09-10T21:13:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4269e00640 1 -- 172.21.15.16:0/3636897808 learned_addr learned my addr 172.21.15.16:0/3636897808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f426a800640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f426c15ad80 0x7f426c0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4263e00640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f426c15a9a0 0x7f426c0b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4269e00640 1 -- 172.21.15.16:0/3636897808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f426c15a9a0 msgr2=0x7f426c0b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4269e00640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f426c15a9a0 0x7f426c0b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4269e00640 1 -- 172.21.15.16:0/3636897808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f426c15ad80 msgr2=0x7f426c0b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4269e00640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f426c15ad80 0x7f426c0b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4269e00640 1 -- 172.21.15.16:0/3636897808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426c0ae850 con 0x7f426c0b33b0 2024-09-10T21:13:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4269e00640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0b33b0 0x7f426c15b6c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c500 tx=0x7f425c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:10.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4262000640 1 -- 172.21.15.16:0/3636897808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c017070 con 0x7f426c0b33b0 2024-09-10T21:13:10.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4262000640 1 -- 172.21.15.16:0/3636897808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f425c0053c0 con 0x7f426c0b33b0 2024-09-10T21:13:10.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4270a00640 1 -- 172.21.15.16:0/3636897808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426c0aeb30 con 0x7f426c0b33b0 2024-09-10T21:13:10.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4262000640 1 -- 172.21.15.16:0/3636897808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c012890 con 0x7f426c0b33b0 2024-09-10T21:13:10.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.988+0000 7f4270a00640 1 -- 172.21.15.16:0/3636897808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f426c0af0f0 con 0x7f426c0b33b0 2024-09-10T21:13:11.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.992+0000 7f4270a00640 1 -- 172.21.15.16:0/3636897808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426c070070 con 0x7f426c0b33b0 2024-09-10T21:13:11.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.992+0000 7f4262000640 1 -- 172.21.15.16:0/3636897808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f425c003600 con 0x7f426c0b33b0 2024-09-10T21:13:11.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.992+0000 7f4262000640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42480725e0 0x7f4248074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.992+0000 7f4262000640 1 -- 172.21.15.16:0/3636897808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5853+0+0 (secure 0 0 0) 0x7f425c0928b0 con 0x7f426c0b33b0 2024-09-10T21:13:11.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.992+0000 7f4263e00640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42480725e0 0x7f4248074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.992+0000 7f4263e00640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42480725e0 0x7f4248074aa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f42540af000 tx=0x7f425401b090 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:11.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:10.996+0000 7f4262000640 1 -- 172.21.15.16:0/3636897808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f425c05d820 con 0x7f426c0b33b0 2024-09-10T21:13:11.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.208+0000 7f4270a00640 1 -- 172.21.15.16:0/3636897808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f426c15e560 con 0x7f426c0b33b0 2024-09-10T21:13:11.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.816+0000 7f4262000640 1 -- 172.21.15.16:0/3636897808 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) ==== 104+0+0 (secure 0 0 0) 0x7f425c062250 con 0x7f426c0b33b0 2024-09-10T21:13:11.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.820+0000 7f4270a00640 1 -- 172.21.15.16:0/3636897808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42480725e0 msgr2=0x7f4248074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.820+0000 7f4270a00640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42480725e0 0x7f4248074aa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f42540af000 tx=0x7f425401b090 comp rx=0 tx=0).stop 2024-09-10T21:13:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.820+0000 7f4270a00640 1 -- 172.21.15.16:0/3636897808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0b33b0 msgr2=0x7f426c15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.820+0000 7f4270a00640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0b33b0 0x7f426c15b6c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c500 tx=0x7f425c00c9c0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.820+0000 7f4270a00640 1 -- 172.21.15.16:0/3636897808 shutdown_connections 2024-09-10T21:13:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.820+0000 7f4270a00640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42480725e0 0x7f4248074aa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.820+0000 7f4270a00640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f426c15ad80 0x7f426c0b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.820+0000 7f4270a00640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f426c15a9a0 0x7f426c0b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.820+0000 7f4270a00640 1 --2- 172.21.15.16:0/3636897808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0b33b0 0x7f426c15b6c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.820+0000 7f4270a00640 1 -- 172.21.15.16:0/3636897808 >> 172.21.15.16:0/3636897808 conn(0x7f426c0b16a0 msgr2=0x7f426c072990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.820+0000 7f4270a00640 1 -- 172.21.15.16:0/3636897808 shutdown_connections 2024-09-10T21:13:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.820+0000 7f4270a00640 1 -- 172.21.15.16:0/3636897808 wait complete. 2024-09-10T21:13:11.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:13:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.996+0000 7f466be00640 1 -- 172.21.15.16:0/4033716057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46640d28b0 msgr2=0x7f46640ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.996+0000 7f466be00640 1 --2- 172.21.15.16:0/4033716057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46640d28b0 0x7f46640ceeb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4664073310 tx=0x7f465001c720 comp rx=0 tx=0).stop 2024-09-10T21:13:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.996+0000 7f466be00640 1 -- 172.21.15.16:0/4033716057 shutdown_connections 2024-09-10T21:13:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.996+0000 7f466be00640 1 --2- 172.21.15.16:0/4033716057 >> v2:172.21.15.52:3300/0 conn(0x7f46640b1e60 0x7f46640b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.996+0000 7f466be00640 1 --2- 172.21.15.16:0/4033716057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46640d28b0 0x7f46640ceeb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.996+0000 7f466be00640 1 -- 172.21.15.16:0/4033716057 >> 172.21.15.16:0/4033716057 conn(0x7f46640d1d90 msgr2=0x7f46640d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.996+0000 7f466be00640 1 -- 172.21.15.16:0/4033716057 shutdown_connections 2024-09-10T21:13:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:11.996+0000 7f466be00640 1 -- 172.21.15.16:0/4033716057 wait complete. 2024-09-10T21:13:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f466be00640 1 Processor -- start 2024-09-10T21:13:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f466be00640 1 -- start start 2024-09-10T21:13:12.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f466be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46640b1e60 0x7f46641f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f466be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46640d28b0 0x7f466415ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f466be00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f466415bf00 0x7f466415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f466be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4664168ea0 con 0x7f46640b1e60 2024-09-10T21:13:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f466be00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4664168d20 con 0x7f46640d28b0 2024-09-10T21:13:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f466be00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4664169020 con 0x7f466415bf00 2024-09-10T21:13:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4669400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46640b1e60 0x7f46641f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4669400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46640b1e60 0x7f46641f7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37252/0 (socket says 172.21.15.16:37252) 2024-09-10T21:13:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4669400640 1 -- 172.21.15.16:0/2918713926 learned_addr learned my addr 172.21.15.16:0/2918713926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4668a00640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46640d28b0 0x7f466415ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4669e00640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f466415bf00 0x7f466415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4669400640 1 -- 172.21.15.16:0/2918713926 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f466415bf00 msgr2=0x7f466415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4669400640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f466415bf00 0x7f466415b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4669400640 1 -- 172.21.15.16:0/2918713926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46640d28b0 msgr2=0x7f466415ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4669400640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46640d28b0 0x7f466415ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4669400640 1 -- 172.21.15.16:0/2918713926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4664157ec0 con 0x7f46640b1e60 2024-09-10T21:13:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4669400640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46640b1e60 0x7f46641f7f70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f465800c560 tx=0x7f465800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4661600640 1 -- 172.21.15.16:0/2918713926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658017070 con 0x7f46640b1e60 2024-09-10T21:13:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4661600640 1 -- 172.21.15.16:0/2918713926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4658003600 con 0x7f46640b1e60 2024-09-10T21:13:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f466be00640 1 -- 172.21.15.16:0/2918713926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4664157900 con 0x7f46640b1e60 2024-09-10T21:13:12.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f4661600640 1 -- 172.21.15.16:0/2918713926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658003a50 con 0x7f46640b1e60 2024-09-10T21:13:12.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.000+0000 7f466be00640 1 -- 172.21.15.16:0/2918713926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46641f7770 con 0x7f46640b1e60 2024-09-10T21:13:12.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.004+0000 7f466be00640 1 -- 172.21.15.16:0/2918713926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46640b22b0 con 0x7f46640b1e60 2024-09-10T21:13:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.004+0000 7f4661600640 1 -- 172.21.15.16:0/2918713926 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4658003bf0 con 0x7f46640b1e60 2024-09-10T21:13:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.004+0000 7f4661600640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46440725e0 0x7f4644074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.004+0000 7f4661600640 1 -- 172.21.15.16:0/2918713926 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5853+0+0 (secure 0 0 0) 0x7f4658092a60 con 0x7f46640b1e60 2024-09-10T21:13:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.004+0000 7f4668a00640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46440725e0 0x7f4644074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.004+0000 7f4668a00640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46440725e0 0x7f4644074aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f46500af000 tx=0x7f465001af60 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.008+0000 7f4661600640 1 -- 172.21.15.16:0/2918713926 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f465805d9c0 con 0x7f46640b1e60 2024-09-10T21:13:12.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.184+0000 7f466be00640 1 -- 172.21.15.16:0/2918713926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4664156050 con 0x7f46640b1e60 2024-09-10T21:13:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.820+0000 7f4661600640 1 -- 172.21.15.16:0/2918713926 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v61) ==== 181+0+0 (secure 0 0 0) 0x7f46580623f0 con 0x7f46640b1e60 2024-09-10T21:13:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:13:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.824+0000 7f466be00640 1 -- 172.21.15.16:0/2918713926 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46440725e0 msgr2=0x7f4644074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.824+0000 7f466be00640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46440725e0 0x7f4644074aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f46500af000 tx=0x7f465001af60 comp rx=0 tx=0).stop 2024-09-10T21:13:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.824+0000 7f466be00640 1 -- 172.21.15.16:0/2918713926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46640b1e60 msgr2=0x7f46641f7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.824+0000 7f466be00640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46640b1e60 0x7f46641f7f70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f465800c560 tx=0x7f465800ca20 comp rx=0 tx=0).stop 2024-09-10T21:13:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.824+0000 7f466be00640 1 -- 172.21.15.16:0/2918713926 shutdown_connections 2024-09-10T21:13:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.824+0000 7f466be00640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46440725e0 0x7f4644074aa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.824+0000 7f466be00640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f466415bf00 0x7f466415b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.824+0000 7f466be00640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46640d28b0 0x7f466415ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.824+0000 7f466be00640 1 --2- 172.21.15.16:0/2918713926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46640b1e60 0x7f46641f7f70 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.824+0000 7f466be00640 1 -- 172.21.15.16:0/2918713926 >> 172.21.15.16:0/2918713926 conn(0x7f46640d1d90 msgr2=0x7f46640d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.824+0000 7f466be00640 1 -- 172.21.15.16:0/2918713926 shutdown_connections 2024-09-10T21:13:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:12.824+0000 7f466be00640 1 -- 172.21.15.16:0/2918713926 wait complete. 2024-09-10T21:13:12.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:13:13.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.000+0000 7fb118a00640 1 -- 172.21.15.16:0/422444617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb114074400 msgr2=0x7fb1140cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:13.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.000+0000 7fb118a00640 1 --2- 172.21.15.16:0/422444617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb114074400 0x7fb1140cf460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb114074180 tx=0x7fb10001c770 comp rx=0 tx=0).stop 2024-09-10T21:13:13.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.000+0000 7fb118a00640 1 -- 172.21.15.16:0/422444617 shutdown_connections 2024-09-10T21:13:13.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.000+0000 7fb118a00640 1 --2- 172.21.15.16:0/422444617 >> v2:172.21.15.52:3300/0 conn(0x7fb11406ec70 0x7fb11415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.000+0000 7fb118a00640 1 --2- 172.21.15.16:0/422444617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb114074400 0x7fb1140cf460 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.000+0000 7fb118a00640 1 -- 172.21.15.16:0/422444617 >> 172.21.15.16:0/422444617 conn(0x7fb1140b1630 msgr2=0x7fb1140b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.000+0000 7fb118a00640 1 -- 172.21.15.16:0/422444617 shutdown_connections 2024-09-10T21:13:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.000+0000 7fb118a00640 1 -- 172.21.15.16:0/422444617 wait complete. 2024-09-10T21:13:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.000+0000 7fb118a00640 1 Processor -- start 2024-09-10T21:13:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.000+0000 7fb118a00640 1 -- start start 2024-09-10T21:13:13.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb118a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb11406ec70 0x7fb11415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:13.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb118a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb114074400 0x7fb114155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:13.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb118a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb11415baf0 0x7fb11414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb118a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb114168b80 con 0x7fb114074400 2024-09-10T21:13:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb118a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb114168a00 con 0x7fb11415baf0 2024-09-10T21:13:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb118a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb114168d00 con 0x7fb11406ec70 2024-09-10T21:13:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb112a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb114074400 0x7fb114155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb112a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb114074400 0x7fb114155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36960/0 (socket says 172.21.15.16:36960) 2024-09-10T21:13:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb112a00640 1 -- 172.21.15.16:0/916184756 learned_addr learned my addr 172.21.15.16:0/916184756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb113400640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb11406ec70 0x7fb11415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb113e00640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb11415baf0 0x7fb11414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb112a00640 1 -- 172.21.15.16:0/916184756 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb11406ec70 msgr2=0x7fb11415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb112a00640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb11406ec70 0x7fb11415c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb112a00640 1 -- 172.21.15.16:0/916184756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb11415baf0 msgr2=0x7fb11414cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:13.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb112a00640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb11415baf0 0x7fb11414cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:13.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb112a00640 1 -- 172.21.15.16:0/916184756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11414da60 con 0x7fb114074400 2024-09-10T21:13:13.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb112a00640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb114074400 0x7fb114155e00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb1000b0040 tx=0x7fb1000a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:13.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb10b400640 1 -- 172.21.15.16:0/916184756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1000b2040 con 0x7fb114074400 2024-09-10T21:13:13.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb10b400640 1 -- 172.21.15.16:0/916184756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb100004400 con 0x7fb114074400 2024-09-10T21:13:13.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb118a00640 1 -- 172.21.15.16:0/916184756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1141f79a0 con 0x7fb114074400 2024-09-10T21:13:13.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb10b400640 1 -- 172.21.15.16:0/916184756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100005290 con 0x7fb114074400 2024-09-10T21:13:13.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb118a00640 1 -- 172.21.15.16:0/916184756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1141f7f60 con 0x7fb114074400 2024-09-10T21:13:13.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.004+0000 7fb118a00640 1 -- 172.21.15.16:0/916184756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb11415f630 con 0x7fb114074400 2024-09-10T21:13:13.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.008+0000 7fb10b400640 1 -- 172.21.15.16:0/916184756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb100005430 con 0x7fb114074400 2024-09-10T21:13:13.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.008+0000 7fb10b400640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb0f40725e0 0x7fb0f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:13.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.008+0000 7fb10b400640 1 -- 172.21.15.16:0/916184756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5415+0+0 (secure 0 0 0) 0x7fb10012cc20 con 0x7fb114074400 2024-09-10T21:13:13.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.008+0000 7fb113400640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb0f40725e0 0x7fb0f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:13.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.008+0000 7fb113400640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb0f40725e0 0x7fb0f4074aa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb1040098e0 tx=0x7fb104009060 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:13.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.012+0000 7fb10b400640 1 -- 172.21.15.16:0/916184756 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb1000164b0 con 0x7fb114074400 2024-09-10T21:13:13.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.188+0000 7fb118a00640 1 -- 172.21.15.16:0/916184756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb114153bb0 con 0x7fb114074400 2024-09-10T21:13:13.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.824+0000 7fb10b400640 1 -- 172.21.15.16:0/916184756 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v62) ==== 169+0+0 (secure 0 0 0) 0x7fb1000f7cb0 con 0x7fb114074400 2024-09-10T21:13:13.832 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-10T21:13:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.828+0000 7fb118a00640 1 -- 172.21.15.16:0/916184756 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb0f40725e0 msgr2=0x7fb0f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.828+0000 7fb118a00640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb0f40725e0 0x7fb0f4074aa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb1040098e0 tx=0x7fb104009060 comp rx=0 tx=0).stop 2024-09-10T21:13:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.828+0000 7fb118a00640 1 -- 172.21.15.16:0/916184756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb114074400 msgr2=0x7fb114155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.828+0000 7fb118a00640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb114074400 0x7fb114155e00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb1000b0040 tx=0x7fb1000a5700 comp rx=0 tx=0).stop 2024-09-10T21:13:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.828+0000 7fb118a00640 1 -- 172.21.15.16:0/916184756 shutdown_connections 2024-09-10T21:13:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.828+0000 7fb118a00640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb0f40725e0 0x7fb0f4074aa0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.828+0000 7fb118a00640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb11415baf0 0x7fb11414cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.828+0000 7fb118a00640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb114074400 0x7fb114155e00 secure :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb1000b0040 tx=0x7fb1000a5700 comp rx=0 tx=0).stop 2024-09-10T21:13:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.828+0000 7fb118a00640 1 --2- 172.21.15.16:0/916184756 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb11406ec70 0x7fb11415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.828+0000 7fb118a00640 1 -- 172.21.15.16:0/916184756 >> 172.21.15.16:0/916184756 conn(0x7fb1140b1630 msgr2=0x7fb1140b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.832+0000 7fb118a00640 1 -- 172.21.15.16:0/916184756 shutdown_connections 2024-09-10T21:13:13.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:13.832+0000 7fb118a00640 1 -- 172.21.15.16:0/916184756 wait complete. 2024-09-10T21:13:13.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:13:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.008+0000 7f5b0d600640 1 -- 172.21.15.16:0/2056629563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b080ca170 msgr2=0x7f5b080ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.008+0000 7f5b0d600640 1 --2- 172.21.15.16:0/2056629563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b080ca170 0x7f5b080ca600 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5b08074ee0 tx=0x7f5b0001dc00 comp rx=0 tx=0).stop 2024-09-10T21:13:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.008+0000 7f5b0d600640 1 -- 172.21.15.16:0/2056629563 shutdown_connections 2024-09-10T21:13:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.008+0000 7f5b0d600640 1 --2- 172.21.15.16:0/2056629563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b080ca170 0x7f5b080ca600 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.008+0000 7f5b0d600640 1 --2- 172.21.15.16:0/2056629563 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b0815a270 0x7f5b080700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.008+0000 7f5b0d600640 1 -- 172.21.15.16:0/2056629563 >> 172.21.15.16:0/2056629563 conn(0x7f5b080cfb20 msgr2=0x7f5b080cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.008+0000 7f5b0d600640 1 -- 172.21.15.16:0/2056629563 shutdown_connections 2024-09-10T21:13:14.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.008+0000 7f5b0d600640 1 -- 172.21.15.16:0/2056629563 wait complete. 2024-09-10T21:13:14.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5b0d600640 1 Processor -- start 2024-09-10T21:13:14.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5b0d600640 1 -- start start 2024-09-10T21:13:14.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5b0d600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b080705e0 0x7f5b080c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5b0d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b080ca170 0x7f5b080c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5b0d600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b0815a270 0x7f5b08155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5b0d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08072c30 con 0x7f5b080ca170 2024-09-10T21:13:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5b0d600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08072ab0 con 0x7f5b0815a270 2024-09-10T21:13:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5b0d600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5b08072db0 con 0x7f5b080705e0 2024-09-10T21:13:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5afe000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b080ca170 0x7f5b080c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5afe000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b080ca170 0x7f5b080c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36978/0 (socket says 172.21.15.16:36978) 2024-09-10T21:13:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5afe000640 1 -- 172.21.15.16:0/590100253 learned_addr learned my addr 172.21.15.16:0/590100253 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5b06a00640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b080705e0 0x7f5b080c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5b07400640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b0815a270 0x7f5b08155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5afe000640 1 -- 172.21.15.16:0/590100253 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b080705e0 msgr2=0x7f5b080c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5afe000640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b080705e0 0x7f5b080c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5afe000640 1 -- 172.21.15.16:0/590100253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b0815a270 msgr2=0x7f5b08155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5afe000640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b0815a270 0x7f5b08155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5afe000640 1 -- 172.21.15.16:0/590100253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b080c77a0 con 0x7f5b080ca170 2024-09-10T21:13:14.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5afe000640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b080ca170 0x7f5b080c8c00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5af000c400 tx=0x7f5af000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:14.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5aff400640 1 -- 172.21.15.16:0/590100253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0007020 con 0x7f5b080ca170 2024-09-10T21:13:14.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5aff400640 1 -- 172.21.15.16:0/590100253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5af00072b0 con 0x7f5b080ca170 2024-09-10T21:13:14.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5aff400640 1 -- 172.21.15.16:0/590100253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af00067e0 con 0x7f5b080ca170 2024-09-10T21:13:14.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5b0d600640 1 -- 172.21.15.16:0/590100253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b080c7a40 con 0x7f5b080ca170 2024-09-10T21:13:14.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.012+0000 7f5b0d600640 1 -- 172.21.15.16:0/590100253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b080c8000 con 0x7f5b080ca170 2024-09-10T21:13:14.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.016+0000 7f5b0d600640 1 -- 172.21.15.16:0/590100253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b080ca600 con 0x7f5b080ca170 2024-09-10T21:13:14.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.016+0000 7f5aff400640 1 -- 172.21.15.16:0/590100253 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5af0006980 con 0x7f5b080ca170 2024-09-10T21:13:14.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.016+0000 7f5aff400640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5ae40725e0 0x7f5ae4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.016+0000 7f5aff400640 1 -- 172.21.15.16:0/590100253 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f5af0092910 con 0x7f5b080ca170 2024-09-10T21:13:14.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.016+0000 7f5b06a00640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5ae40725e0 0x7f5ae4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.016+0000 7f5b06a00640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5ae40725e0 0x7f5ae4074aa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5af8004630 tx=0x7f5af800b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:14.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.020+0000 7f5aff400640 1 -- 172.21.15.16:0/590100253 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5af005dc90 con 0x7f5b080ca170 2024-09-10T21:13:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.196+0000 7f5b0d600640 1 -- 172.21.15.16:0/590100253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f5b080c81d0 con 0x7f5b080ca170 2024-09-10T21:13:14.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.200+0000 7f5aff400640 1 -- 172.21.15.16:0/590100253 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7f5af00626c0 con 0x7f5b080ca170 2024-09-10T21:13:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.200+0000 7f5b0d600640 1 -- 172.21.15.16:0/590100253 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5ae40725e0 msgr2=0x7f5ae4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.200+0000 7f5b0d600640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5ae40725e0 0x7f5ae4074aa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5af8004630 tx=0x7f5af800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.200+0000 7f5b0d600640 1 -- 172.21.15.16:0/590100253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b080ca170 msgr2=0x7f5b080c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.200+0000 7f5b0d600640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b080ca170 0x7f5b080c8c00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5af000c400 tx=0x7f5af000c8c0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.200+0000 7f5b0d600640 1 -- 172.21.15.16:0/590100253 shutdown_connections 2024-09-10T21:13:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.200+0000 7f5b0d600640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5ae40725e0 0x7f5ae4074aa0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.200+0000 7f5b0d600640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b0815a270 0x7f5b08155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.200+0000 7f5b0d600640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b080ca170 0x7f5b080c8c00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.200+0000 7f5b0d600640 1 --2- 172.21.15.16:0/590100253 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b080705e0 0x7f5b080c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.200+0000 7f5b0d600640 1 -- 172.21.15.16:0/590100253 >> 172.21.15.16:0/590100253 conn(0x7f5b080cfb20 msgr2=0x7f5b080731f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:14.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.200+0000 7f5b0d600640 1 -- 172.21.15.16:0/590100253 shutdown_connections 2024-09-10T21:13:14.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.200+0000 7f5b0d600640 1 -- 172.21.15.16:0/590100253 wait complete. 2024-09-10T21:13:14.228 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2024-09-10T21:13:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.368+0000 7f0842a00640 1 -- 172.21.15.16:0/2669058352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c15f350 msgr2=0x7f083c0cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.368+0000 7f0842a00640 1 --2- 172.21.15.16:0/2669058352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c15f350 0x7f083c0cd130 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f083c0b7e30 tx=0x7f082c01c800 comp rx=0 tx=0).stop 2024-09-10T21:13:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.368+0000 7f0842a00640 1 -- 172.21.15.16:0/2669058352 shutdown_connections 2024-09-10T21:13:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.368+0000 7f0842a00640 1 --2- 172.21.15.16:0/2669058352 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f083c0d0530 0x7f083c0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.368+0000 7f0842a00640 1 --2- 172.21.15.16:0/2669058352 >> v2:172.21.15.52:3300/0 conn(0x7f083c0cfb60 0x7f083c0cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.368+0000 7f0842a00640 1 --2- 172.21.15.16:0/2669058352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c15f350 0x7f083c0cd130 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.368+0000 7f0842a00640 1 -- 172.21.15.16:0/2669058352 >> 172.21.15.16:0/2669058352 conn(0x7f083c0d1d80 msgr2=0x7f083c0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.368+0000 7f0842a00640 1 -- 172.21.15.16:0/2669058352 shutdown_connections 2024-09-10T21:13:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.368+0000 7f0842a00640 1 -- 172.21.15.16:0/2669058352 wait complete. 2024-09-10T21:13:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.368+0000 7f0842a00640 1 Processor -- start 2024-09-10T21:13:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.368+0000 7f0842a00640 1 -- start start 2024-09-10T21:13:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0842a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f083c0cfb60 0x7f083c0b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0842a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0d0530 0x7f083c0c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0842a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f083c15f350 0x7f083c156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0842a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f083c0af200 con 0x7f083c0d0530 2024-09-10T21:13:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0842a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f083c0af080 con 0x7f083c15f350 2024-09-10T21:13:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0842a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f083c0af380 con 0x7f083c0cfb60 2024-09-10T21:13:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0833e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0d0530 0x7f083c0c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0833e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0d0530 0x7f083c0c4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37000/0 (socket says 172.21.15.16:37000) 2024-09-10T21:13:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0833e00640 1 -- 172.21.15.16:0/2637487459 learned_addr learned my addr 172.21.15.16:0/2637487459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0840a00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f083c15f350 0x7f083c156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f083be00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f083c0cfb60 0x7f083c0b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0833e00640 1 -- 172.21.15.16:0/2637487459 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f083c0cfb60 msgr2=0x7f083c0b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0833e00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f083c0cfb60 0x7f083c0b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0833e00640 1 -- 172.21.15.16:0/2637487459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f083c15f350 msgr2=0x7f083c156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0833e00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f083c15f350 0x7f083c156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0833e00640 1 -- 172.21.15.16:0/2637487459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c0b7880 con 0x7f083c0d0530 2024-09-10T21:13:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0840a00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f083c15f350 0x7f083c156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0833e00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0d0530 0x7f083c0c4ef0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f082400d770 tx=0x7f082400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0838c00640 1 -- 172.21.15.16:0/2637487459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824017070 con 0x7f083c0d0530 2024-09-10T21:13:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0838c00640 1 -- 172.21.15.16:0/2637487459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0824009d20 con 0x7f083c0d0530 2024-09-10T21:13:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0838c00640 1 -- 172.21.15.16:0/2637487459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08240106c0 con 0x7f083c0d0530 2024-09-10T21:13:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0842a00640 1 -- 172.21.15.16:0/2637487459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c157d10 con 0x7f083c0d0530 2024-09-10T21:13:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0842a00640 1 -- 172.21.15.16:0/2637487459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c0c5430 con 0x7f083c0d0530 2024-09-10T21:13:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0842a00640 1 -- 172.21.15.16:0/2637487459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083c0ce500 con 0x7f083c0d0530 2024-09-10T21:13:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.372+0000 7f0838c00640 1 -- 172.21.15.16:0/2637487459 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0824010860 con 0x7f083c0d0530 2024-09-10T21:13:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.376+0000 7f0838c00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f08180725e0 0x7f0818074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.376+0000 7f0838c00640 1 -- 172.21.15.16:0/2637487459 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f0824093430 con 0x7f083c0d0530 2024-09-10T21:13:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.376+0000 7f083be00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f08180725e0 0x7f0818074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.376+0000 7f083be00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f08180725e0 0x7f0818074aa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f082c0af000 tx=0x7f082c01b040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:14.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.380+0000 7f0838c00640 1 -- 172.21.15.16:0/2637487459 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f082405ed70 con 0x7f083c0d0530 2024-09-10T21:13:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.616+0000 7f0842a00640 1 -- 172.21.15.16:0/2637487459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) -- 0x7f083c071450 con 0x7f083c0d0530 2024-09-10T21:13:14.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.828+0000 7f0838c00640 1 -- 172.21.15.16:0/2637487459 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v159) ==== 150+0+0 (secure 0 0 0) 0x7f082405e670 con 0x7f083c0d0530 2024-09-10T21:13:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.828+0000 7f0842a00640 1 -- 172.21.15.16:0/2637487459 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f08180725e0 msgr2=0x7f0818074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.828+0000 7f0842a00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f08180725e0 0x7f0818074aa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f082c0af000 tx=0x7f082c01b040 comp rx=0 tx=0).stop 2024-09-10T21:13:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.828+0000 7f0842a00640 1 -- 172.21.15.16:0/2637487459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0d0530 msgr2=0x7f083c0c4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.828+0000 7f0842a00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0d0530 0x7f083c0c4ef0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f082400d770 tx=0x7f082400dc30 comp rx=0 tx=0).stop 2024-09-10T21:13:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.828+0000 7f0842a00640 1 -- 172.21.15.16:0/2637487459 shutdown_connections 2024-09-10T21:13:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.828+0000 7f0842a00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f08180725e0 0x7f0818074aa0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.828+0000 7f0842a00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f083c15f350 0x7f083c156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.828+0000 7f0842a00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0d0530 0x7f083c0c4ef0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.828+0000 7f0842a00640 1 --2- 172.21.15.16:0/2637487459 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f083c0cfb60 0x7f083c0b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.828+0000 7f0842a00640 1 -- 172.21.15.16:0/2637487459 >> 172.21.15.16:0/2637487459 conn(0x7f083c0d1d80 msgr2=0x7f083c15b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.828+0000 7f0842a00640 1 -- 172.21.15.16:0/2637487459 shutdown_connections 2024-09-10T21:13:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.828+0000 7f0842a00640 1 -- 172.21.15.16:0/2637487459 wait complete. 2024-09-10T21:13:14.856 INFO:tasks.cephfs_test_runner:test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) ... skipped 'Require FUSE client to inject client release failure' 2024-09-10T21:13:14.856 INFO:tasks.cephfs_test_runner:Starting test: test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) That the MDS will not let a client sit above mds_max_caps_per_client caps. 2024-09-10T21:13:14.856 INFO:tasks.cephfs_test_runner:test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-10T21:13:14.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:13:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.996+0000 7f0115200640 1 -- 172.21.15.16:0/2507608863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01100b13c0 msgr2=0x7f0110071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.996+0000 7f0115200640 1 --2- 172.21.15.16:0/2507608863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01100b13c0 0x7f0110071520 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009840 tx=0x7f00f801c700 comp rx=0 tx=0).stop 2024-09-10T21:13:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.996+0000 7f0115200640 1 -- 172.21.15.16:0/2507608863 shutdown_connections 2024-09-10T21:13:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.996+0000 7f0115200640 1 --2- 172.21.15.16:0/2507608863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01100b13c0 0x7f0110071520 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.996+0000 7f0115200640 1 --2- 172.21.15.16:0/2507608863 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f011015f400 0x7f0110070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.996+0000 7f0115200640 1 -- 172.21.15.16:0/2507608863 >> 172.21.15.16:0/2507608863 conn(0x7f011009b130 msgr2=0x7f01100d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.996+0000 7f0115200640 1 -- 172.21.15.16:0/2507608863 shutdown_connections 2024-09-10T21:13:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:14.996+0000 7f0115200640 1 -- 172.21.15.16:0/2507608863 wait complete. 2024-09-10T21:13:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0115200640 1 Processor -- start 2024-09-10T21:13:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0115200640 1 -- start start 2024-09-10T21:13:15.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0115200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01100b13c0 0x7f011014e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0115200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01100cfc80 0x7f011014cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0115200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f011015f400 0x7f0110156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0115200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0110169770 con 0x7f01100cfc80 2024-09-10T21:13:15.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0115200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f01101695f0 con 0x7f011015f400 2024-09-10T21:13:15.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0115200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f01101698f0 con 0x7f01100b13c0 2024-09-10T21:13:15.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0105400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01100cfc80 0x7f011014cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0105400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01100cfc80 0x7f011014cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37018/0 (socket says 172.21.15.16:37018) 2024-09-10T21:13:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0105400640 1 -- 172.21.15.16:0/1287244581 learned_addr learned my addr 172.21.15.16:0/1287244581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f010de00640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01100b13c0 0x7f011014e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f010e800640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f011015f400 0x7f0110156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0105400640 1 -- 172.21.15.16:0/1287244581 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01100b13c0 msgr2=0x7f011014e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0105400640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01100b13c0 0x7f011014e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0105400640 1 -- 172.21.15.16:0/1287244581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f011015f400 msgr2=0x7f0110156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0105400640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f011015f400 0x7f0110156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0105400640 1 -- 172.21.15.16:0/1287244581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011014d7f0 con 0x7f01100cfc80 2024-09-10T21:13:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0105400640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01100cfc80 0x7f011014cd50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f00f801adb0 tx=0x7f00f80a66c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0106a00640 1 -- 172.21.15.16:0/1287244581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f80be030 con 0x7f01100cfc80 2024-09-10T21:13:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0106a00640 1 -- 172.21.15.16:0/1287244581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00f80041b0 con 0x7f01100cfc80 2024-09-10T21:13:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0106a00640 1 -- 172.21.15.16:0/1287244581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8005020 con 0x7f01100cfc80 2024-09-10T21:13:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0115200640 1 -- 172.21.15.16:0/1287244581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011014dad0 con 0x7f01100cfc80 2024-09-10T21:13:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.000+0000 7f0115200640 1 -- 172.21.15.16:0/1287244581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011014e090 con 0x7f01100cfc80 2024-09-10T21:13:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.004+0000 7f0115200640 1 -- 172.21.15.16:0/1287244581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01100d00d0 con 0x7f01100cfc80 2024-09-10T21:13:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.004+0000 7f0106a00640 1 -- 172.21.15.16:0/1287244581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f00f80051c0 con 0x7f01100cfc80 2024-09-10T21:13:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.004+0000 7f0106a00640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f00ec072610 0x7f00ec074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.004+0000 7f0106a00640 1 -- 172.21.15.16:0/1287244581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f00f812db40 con 0x7f01100cfc80 2024-09-10T21:13:15.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.004+0000 7f010de00640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f00ec072610 0x7f00ec074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.004+0000 7f010de00640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f00ec072610 0x7f00ec074ad0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f01100ced90 tx=0x7f010000b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:15.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.008+0000 7f0106a00640 1 -- 172.21.15.16:0/1287244581 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f00f80c9020 con 0x7f01100cfc80 2024-09-10T21:13:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.184+0000 7f0115200640 1 -- 172.21.15.16:0/1287244581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01100b4480 con 0x7f01100cfc80 2024-09-10T21:13:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.184+0000 7f0106a00640 1 -- 172.21.15.16:0/1287244581 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+12984 (secure 0 0 0) 0x7f00f8016410 con 0x7f01100cfc80 2024-09-10T21:13:15.191 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:15.191 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":62,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:13:13.824436+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6835/2357796669":"2024-09-11T21:13:10.814969+0000","172.21.15.52:6833/2357796669":"2024-09-11T21:13:10.814969+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:13:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.188+0000 7f0115200640 1 -- 172.21.15.16:0/1287244581 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f00ec072610 msgr2=0x7f00ec074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.188+0000 7f0115200640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f00ec072610 0x7f00ec074ad0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f01100ced90 tx=0x7f010000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.188+0000 7f0115200640 1 -- 172.21.15.16:0/1287244581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01100cfc80 msgr2=0x7f011014cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.188+0000 7f0115200640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01100cfc80 0x7f011014cd50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f00f801adb0 tx=0x7f00f80a66c0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.188+0000 7f0115200640 1 -- 172.21.15.16:0/1287244581 shutdown_connections 2024-09-10T21:13:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.188+0000 7f0115200640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f00ec072610 0x7f00ec074ad0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.188+0000 7f0115200640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f011015f400 0x7f0110156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.188+0000 7f0115200640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01100cfc80 0x7f011014cd50 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.188+0000 7f0115200640 1 --2- 172.21.15.16:0/1287244581 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f01100b13c0 0x7f011014e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.188+0000 7f0115200640 1 -- 172.21.15.16:0/1287244581 >> 172.21.15.16:0/1287244581 conn(0x7f011009b130 msgr2=0x7f0110158770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.188+0000 7f0115200640 1 -- 172.21.15.16:0/1287244581 shutdown_connections 2024-09-10T21:13:15.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.188+0000 7f0115200640 1 -- 172.21.15.16:0/1287244581 wait complete. 2024-09-10T21:13:15.214 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:10:17.493770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:13:15.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:13:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.356+0000 7f97e9600640 1 -- 172.21.15.16:0/1448125616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e40cfe30 msgr2=0x7f97e40ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.356+0000 7f97e9600640 1 --2- 172.21.15.16:0/1448125616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e40cfe30 0x7f97e40ce590 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009900 tx=0x7f97d401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.356+0000 7f97e9600640 1 -- 172.21.15.16:0/1448125616 shutdown_connections 2024-09-10T21:13:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.356+0000 7f97e9600640 1 --2- 172.21.15.16:0/1448125616 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97e40d0210 0x7f97e40cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.356+0000 7f97e9600640 1 --2- 172.21.15.16:0/1448125616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e40cfe30 0x7f97e40ce590 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.356+0000 7f97e9600640 1 -- 172.21.15.16:0/1448125616 >> 172.21.15.16:0/1448125616 conn(0x7f97e40d1df0 msgr2=0x7f97e40d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.356+0000 7f97e9600640 1 -- 172.21.15.16:0/1448125616 shutdown_connections 2024-09-10T21:13:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.356+0000 7f97e9600640 1 -- 172.21.15.16:0/1448125616 wait complete. 2024-09-10T21:13:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e9600640 1 Processor -- start 2024-09-10T21:13:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e9600640 1 -- start start 2024-09-10T21:13:15.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e40ce040 0x7f97e415c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e9600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97e40cfe30 0x7f97e41f8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e9600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97e40d0210 0x7f97e41f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e9600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97e40b12e0 con 0x7f97e40ce040 2024-09-10T21:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e9600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97e40d2b20 con 0x7f97e40cfe30 2024-09-10T21:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e9600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f97e40b1460 con 0x7f97e40d0210 2024-09-10T21:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e40ce040 0x7f97e415c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e40ce040 0x7f97e415c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37036/0 (socket says 172.21.15.16:37036) 2024-09-10T21:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e2a00640 1 -- 172.21.15.16:0/3013723750 learned_addr learned my addr 172.21.15.16:0/3013723750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e2a00640 1 -- 172.21.15.16:0/3013723750 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97e40d0210 msgr2=0x7f97e41f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e2a00640 1 --2- 172.21.15.16:0/3013723750 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97e40d0210 0x7f97e41f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e2a00640 1 -- 172.21.15.16:0/3013723750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97e40cfe30 msgr2=0x7f97e41f8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e2a00640 1 --2- 172.21.15.16:0/3013723750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97e40cfe30 0x7f97e41f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e2a00640 1 -- 172.21.15.16:0/3013723750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e414fda0 con 0x7f97e40ce040 2024-09-10T21:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e2a00640 1 --2- 172.21.15.16:0/3013723750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e40ce040 0x7f97e415c350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009850 tx=0x7f97d40a8d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97daa00640 1 -- 172.21.15.16:0/3013723750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d40b5040 con 0x7f97e40ce040 2024-09-10T21:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97daa00640 1 -- 172.21.15.16:0/3013723750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97d4003990 con 0x7f97e40ce040 2024-09-10T21:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97daa00640 1 -- 172.21.15.16:0/3013723750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4003de0 con 0x7f97e40ce040 2024-09-10T21:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e9600640 1 -- 172.21.15.16:0/3013723750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e40b6800 con 0x7f97e40ce040 2024-09-10T21:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e9600640 1 -- 172.21.15.16:0/3013723750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e40b6ac0 con 0x7f97e40ce040 2024-09-10T21:13:15.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.360+0000 7f97e9600640 1 -- 172.21.15.16:0/3013723750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e40cf2b0 con 0x7f97e40ce040 2024-09-10T21:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.364+0000 7f97daa00640 1 -- 172.21.15.16:0/3013723750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f97d40b8070 con 0x7f97e40ce040 2024-09-10T21:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.364+0000 7f97daa00640 1 --2- 172.21.15.16:0/3013723750 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f97c00725e0 0x7f97c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.364+0000 7f97daa00640 1 -- 172.21.15.16:0/3013723750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f97d412ffe0 con 0x7f97e40ce040 2024-09-10T21:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.364+0000 7f97dbe00640 1 --2- 172.21.15.16:0/3013723750 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f97c00725e0 0x7f97c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.364+0000 7f97dbe00640 1 --2- 172.21.15.16:0/3013723750 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f97c00725e0 0x7f97c0074aa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97e40c8fe0 tx=0x7f97cc005be0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:15.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.368+0000 7f97daa00640 1 -- 172.21.15.16:0/3013723750 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f97e414fda0 con 0x7f97e40ce040 2024-09-10T21:13:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.540+0000 7f97e9600640 1 -- 172.21.15.16:0/3013723750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97e40b44e0 con 0x7f97e40ce040 2024-09-10T21:13:15.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.544+0000 7f97daa00640 1 -- 172.21.15.16:0/3013723750 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) ==== 93+0+10 (secure 0 0 0) 0x7f97d4018c30 con 0x7f97e40ce040 2024-09-10T21:13:15.549 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-10T21:13:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.544+0000 7f97e9600640 1 -- 172.21.15.16:0/3013723750 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f97c00725e0 msgr2=0x7f97c0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.544+0000 7f97e9600640 1 --2- 172.21.15.16:0/3013723750 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f97c00725e0 0x7f97c0074aa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97e40c8fe0 tx=0x7f97cc005be0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.544+0000 7f97e9600640 1 -- 172.21.15.16:0/3013723750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e40ce040 msgr2=0x7f97e415c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.544+0000 7f97e9600640 1 --2- 172.21.15.16:0/3013723750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e40ce040 0x7f97e415c350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009850 tx=0x7f97d40a8d60 comp rx=0 tx=0).stop 2024-09-10T21:13:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.544+0000 7f97e9600640 1 -- 172.21.15.16:0/3013723750 shutdown_connections 2024-09-10T21:13:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.544+0000 7f97e9600640 1 --2- 172.21.15.16:0/3013723750 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f97c00725e0 0x7f97c0074aa0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.544+0000 7f97e9600640 1 --2- 172.21.15.16:0/3013723750 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97e40d0210 0x7f97e41f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.544+0000 7f97e9600640 1 --2- 172.21.15.16:0/3013723750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97e40cfe30 0x7f97e41f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.544+0000 7f97e9600640 1 --2- 172.21.15.16:0/3013723750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e40ce040 0x7f97e415c350 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.544+0000 7f97e9600640 1 -- 172.21.15.16:0/3013723750 >> 172.21.15.16:0/3013723750 conn(0x7f97e40d1df0 msgr2=0x7f97e41581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.544+0000 7f97e9600640 1 -- 172.21.15.16:0/3013723750 shutdown_connections 2024-09-10T21:13:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.544+0000 7f97e9600640 1 -- 172.21.15.16:0/3013723750 wait complete. 2024-09-10T21:13:15.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2024-09-10T21:13:15.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.712+0000 7f3769c00640 1 -- 172.21.15.16:0/2082243410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3748005220 msgr2=0x7f374800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.712+0000 7f3769c00640 1 --2- 172.21.15.16:0/2082243410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3748005220 0x7f374800b9d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f375c009eb0 tx=0x7f375c01d5d0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.712+0000 7f3769c00640 1 -- 172.21.15.16:0/2082243410 shutdown_connections 2024-09-10T21:13:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.712+0000 7f3769c00640 1 --2- 172.21.15.16:0/2082243410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3748005220 0x7f374800b9d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.712+0000 7f3769c00640 1 -- 172.21.15.16:0/2082243410 >> 172.21.15.16:0/2082243410 conn(0x7f37640d1d80 msgr2=0x7f37640d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.712+0000 7f3769c00640 1 -- 172.21.15.16:0/2082243410 shutdown_connections 2024-09-10T21:13:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.712+0000 7f3769c00640 1 -- 172.21.15.16:0/2082243410 wait complete. 2024-09-10T21:13:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.712+0000 7f3769c00640 1 Processor -- start 2024-09-10T21:13:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.712+0000 7f3769c00640 1 -- start start 2024-09-10T21:13:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f3769c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3748005220 0x7f37641f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f3769c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37641f4930 0x7f37640cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f3769c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37640cb310 0x7f37640c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f3769c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37640cfc80 con 0x7f37641f4930 2024-09-10T21:13:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f3769c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f37640b2ec0 con 0x7f3748005220 2024-09-10T21:13:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f3769c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f37640cfe00 con 0x7f37640cb310 2024-09-10T21:13:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f375be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37641f4930 0x7f37640cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f375be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37641f4930 0x7f37640cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37062/0 (socket says 172.21.15.16:37062) 2024-09-10T21:13:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f375be00640 1 -- 172.21.15.16:0/3032885621 learned_addr learned my addr 172.21.15.16:0/3032885621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f3763400640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37640cb310 0x7f37640c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f3762a00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3748005220 0x7f37641f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f375be00640 1 -- 172.21.15.16:0/3032885621 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37640cb310 msgr2=0x7f37640c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f375be00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37640cb310 0x7f37640c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f375be00640 1 -- 172.21.15.16:0/3032885621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3748005220 msgr2=0x7f37641f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f375be00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3748005220 0x7f37641f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f375be00640 1 -- 172.21.15.16:0/3032885621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37641f5770 con 0x7f37641f4930 2024-09-10T21:13:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f3763400640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37640cb310 0x7f37640c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f3762a00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3748005220 0x7f37641f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f375be00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37641f4930 0x7f37640cadd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f374c000fe0 tx=0x7f374c00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f375aa00640 1 -- 172.21.15.16:0/3032885621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c015070 con 0x7f37641f4930 2024-09-10T21:13:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f375aa00640 1 -- 172.21.15.16:0/3032885621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f374c0109a0 con 0x7f37641f4930 2024-09-10T21:13:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f3769c00640 1 -- 172.21.15.16:0/3032885621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37640cb7b0 con 0x7f37641f4930 2024-09-10T21:13:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f375aa00640 1 -- 172.21.15.16:0/3032885621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c010df0 con 0x7f37641f4930 2024-09-10T21:13:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f3769c00640 1 -- 172.21.15.16:0/3032885621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37640cbd70 con 0x7f37641f4930 2024-09-10T21:13:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.716+0000 7f3769c00640 1 -- 172.21.15.16:0/3032885621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37640d02f0 con 0x7f37641f4930 2024-09-10T21:13:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.720+0000 7f375aa00640 1 -- 172.21.15.16:0/3032885621 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f374c011460 con 0x7f37641f4930 2024-09-10T21:13:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.720+0000 7f375aa00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f37400725e0 0x7f3740074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.720+0000 7f375aa00640 1 -- 172.21.15.16:0/3032885621 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f374c093650 con 0x7f37641f4930 2024-09-10T21:13:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.720+0000 7f3762a00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f37400725e0 0x7f3740074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.720+0000 7f3762a00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f37400725e0 0x7f3740074aa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f37640c53e0 tx=0x7f3754006b80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.724+0000 7f375aa00640 1 -- 172.21.15.16:0/3032885621 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f374c05ef90 con 0x7f37641f4930 2024-09-10T21:13:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:15.960+0000 7f3769c00640 1 -- 172.21.15.16:0/3032885621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) -- 0x7f376415eb70 con 0x7f37641f4930 2024-09-10T21:13:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:16.836+0000 7f375aa00640 1 -- 172.21.15.16:0/3032885621 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v161) ==== 151+0+0 (secure 0 0 0) 0x7f374c05e890 con 0x7f37641f4930 2024-09-10T21:13:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:16.836+0000 7f3769c00640 1 -- 172.21.15.16:0/3032885621 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f37400725e0 msgr2=0x7f3740074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:16.836+0000 7f3769c00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f37400725e0 0x7f3740074aa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f37640c53e0 tx=0x7f3754006b80 comp rx=0 tx=0).stop 2024-09-10T21:13:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:16.836+0000 7f3769c00640 1 -- 172.21.15.16:0/3032885621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37641f4930 msgr2=0x7f37640cadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:16.836+0000 7f3769c00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37641f4930 0x7f37640cadd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f374c000fe0 tx=0x7f374c00ea20 comp rx=0 tx=0).stop 2024-09-10T21:13:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:16.836+0000 7f3769c00640 1 -- 172.21.15.16:0/3032885621 shutdown_connections 2024-09-10T21:13:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:16.836+0000 7f3769c00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f37400725e0 0x7f3740074aa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:16.836+0000 7f3769c00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37640cb310 0x7f37640c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:16.836+0000 7f3769c00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37641f4930 0x7f37640cadd0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:16.836+0000 7f3769c00640 1 --2- 172.21.15.16:0/3032885621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3748005220 0x7f37641f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:16.836+0000 7f3769c00640 1 -- 172.21.15.16:0/3032885621 >> 172.21.15.16:0/3032885621 conn(0x7f37640d1d80 msgr2=0x7f376415cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:16.840+0000 7f3769c00640 1 -- 172.21.15.16:0/3032885621 shutdown_connections 2024-09-10T21:13:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:16.840+0000 7f3769c00640 1 -- 172.21.15.16:0/3032885621 wait complete. 2024-09-10T21:13:16.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:13:17.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.016+0000 7f6218e00640 1 -- 172.21.15.16:0/2636555606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f8004080 msgr2=0x7f61f8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.016+0000 7f6218e00640 1 --2- 172.21.15.16:0/2636555606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f8004080 0x7f61f8004810 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00c2e0 tx=0x7f61fc01cf20 comp rx=0 tx=0).stop 2024-09-10T21:13:17.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.016+0000 7f6213e00640 1 -- 172.21.15.16:0/2636555606 reap_dead start 2024-09-10T21:13:17.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.016+0000 7f6218e00640 1 -- 172.21.15.16:0/2636555606 shutdown_connections 2024-09-10T21:13:17.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.016+0000 7f6218e00640 1 -- 172.21.15.16:0/2636555606 >> 172.21.15.16:0/2636555606 conn(0x7f62140d1d00 msgr2=0x7f62140d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:17.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.016+0000 7f6218e00640 1 -- 172.21.15.16:0/2636555606 shutdown_connections 2024-09-10T21:13:17.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.016+0000 7f6218e00640 1 -- 172.21.15.16:0/2636555606 wait complete. 2024-09-10T21:13:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6218e00640 1 Processor -- start 2024-09-10T21:13:17.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6218e00640 1 -- start start 2024-09-10T21:13:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6218e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f8004080 0x7f62141f7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6218e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62140b3e40 0x7f62140c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6218e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f621415aba0 0x7f6214163400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6218e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6214167230 con 0x7f61f8004080 2024-09-10T21:13:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6218e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62141670b0 con 0x7f621415aba0 2024-09-10T21:13:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6218e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f62141673b0 con 0x7f62140b3e40 2024-09-10T21:13:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6213400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f8004080 0x7f62141f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6213400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f8004080 0x7f62141f7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37074/0 (socket says 172.21.15.16:37074) 2024-09-10T21:13:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6213400640 1 -- 172.21.15.16:0/3914156713 learned_addr learned my addr 172.21.15.16:0/3914156713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6213400640 1 -- 172.21.15.16:0/3914156713 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62140b3e40 msgr2=0x7f62140c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6213400640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62140b3e40 0x7f62140c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6213400640 1 -- 172.21.15.16:0/3914156713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f621415aba0 msgr2=0x7f6214163400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6213e00640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f621415aba0 0x7f6214163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6213400640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f621415aba0 0x7f6214163400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6213400640 1 -- 172.21.15.16:0/3914156713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62141f7da0 con 0x7f61f8004080 2024-09-10T21:13:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6213e00640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f621415aba0 0x7f6214163400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6213400640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f8004080 0x7f62141f7860 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6204003ef0 tx=0x7f6204001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f620b400640 1 -- 172.21.15.16:0/3914156713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204011640 con 0x7f61f8004080 2024-09-10T21:13:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f620b400640 1 -- 172.21.15.16:0/3914156713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6204016020 con 0x7f61f8004080 2024-09-10T21:13:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6218e00640 1 -- 172.21.15.16:0/3914156713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62140c9640 con 0x7f61f8004080 2024-09-10T21:13:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f620b400640 1 -- 172.21.15.16:0/3914156713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204011a80 con 0x7f61f8004080 2024-09-10T21:13:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6218e00640 1 -- 172.21.15.16:0/3914156713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62141f7060 con 0x7f61f8004080 2024-09-10T21:13:17.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.020+0000 7f6218e00640 1 -- 172.21.15.16:0/3914156713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62140d0120 con 0x7f61f8004080 2024-09-10T21:13:17.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.024+0000 7f620b400640 1 -- 172.21.15.16:0/3914156713 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6204011c20 con 0x7f61f8004080 2024-09-10T21:13:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.024+0000 7f620b400640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f61f00725e0 0x7f61f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.024+0000 7f620b400640 1 -- 172.21.15.16:0/3914156713 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f6204093c00 con 0x7f61f8004080 2024-09-10T21:13:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.024+0000 7f620be00640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f61f00725e0 0x7f61f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.024+0000 7f620be00640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f61f00725e0 0x7f61f0074aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0ab000 tx=0x7f61f8005840 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:17.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.028+0000 7f620b400640 1 -- 172.21.15.16:0/3914156713 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6204020020 con 0x7f61f8004080 2024-09-10T21:13:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f6218e00640 1 -- 172.21.15.16:0/3914156713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f62140008e0 con 0x7f61f8004080 2024-09-10T21:13:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f620b400640 1 -- 172.21.15.16:0/3914156713 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v35) ==== 59+0+1998 (secure 0 0 0) 0x7f620405f5d0 con 0x7f61f8004080 2024-09-10T21:13:17.210 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:13:17.210 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:13:17.210 INFO:teuthology.orchestra.run.smithi016.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:13:17.210 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:13:17.210 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4226.427518 2024-09-10T21:13:17.210 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5426.620519 2024-09-10T21:13:17.210 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4255.210203 2024-09-10T21:13:17.211 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4255.734871 2024-09-10T21:13:17.211 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4192.833008 2024-09-10T21:13:17.211 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4136.940759 2024-09-10T21:13:17.211 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4217.384701 2024-09-10T21:13:17.211 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5220.337972 2024-09-10T21:13:17.211 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-10T21:13:17.211 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:13:17.211 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-10T21:13:17.211 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-10T21:13:17.211 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:13:17.211 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:13:17.211 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-10T21:13:17.212 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:13:17.212 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:13:17.212 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:13:17.212 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-10T21:13:17.212 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-10T21:13:17.212 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-10T21:13:17.212 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:13:17.212 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:13:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f6218e00640 1 -- 172.21.15.16:0/3914156713 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f61f00725e0 msgr2=0x7f61f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f6218e00640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f61f00725e0 0x7f61f0074aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0ab000 tx=0x7f61f8005840 comp rx=0 tx=0).stop 2024-09-10T21:13:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f6218e00640 1 -- 172.21.15.16:0/3914156713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f8004080 msgr2=0x7f62141f7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f6218e00640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f8004080 0x7f62141f7860 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6204003ef0 tx=0x7f6204001640 comp rx=0 tx=0).stop 2024-09-10T21:13:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f6218e00640 1 -- 172.21.15.16:0/3914156713 shutdown_connections 2024-09-10T21:13:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f6218e00640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f61f00725e0 0x7f61f0074aa0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f6218e00640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f621415aba0 0x7f6214163400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f6218e00640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62140b3e40 0x7f62140c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f6218e00640 1 --2- 172.21.15.16:0/3914156713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f8004080 0x7f62141f7860 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f6218e00640 1 -- 172.21.15.16:0/3914156713 >> 172.21.15.16:0/3914156713 conn(0x7f62140d1d00 msgr2=0x7f62140ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f6218e00640 1 -- 172.21.15.16:0/3914156713 shutdown_connections 2024-09-10T21:13:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.204+0000 7f6218e00640 1 -- 172.21.15.16:0/3914156713 wait complete. 2024-09-10T21:13:17.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:13:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.372+0000 7ff09fc00640 1 -- 172.21.15.16:0/1545058424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0980cfe30 msgr2=0x7ff0980ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.372+0000 7ff09fc00640 1 --2- 172.21.15.16:0/1545058424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0980cfe30 0x7ff0980ce590 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009930 tx=0x7ff08c01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.372+0000 7ff09fc00640 1 -- 172.21.15.16:0/1545058424 shutdown_connections 2024-09-10T21:13:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.372+0000 7ff09fc00640 1 --2- 172.21.15.16:0/1545058424 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff0980ce040 0x7ff0980b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.372+0000 7ff09fc00640 1 --2- 172.21.15.16:0/1545058424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0980cfe30 0x7ff0980ce590 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.372+0000 7ff09fc00640 1 -- 172.21.15.16:0/1545058424 >> 172.21.15.16:0/1545058424 conn(0x7ff0980d1df0 msgr2=0x7ff0980d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.372+0000 7ff09fc00640 1 -- 172.21.15.16:0/1545058424 shutdown_connections 2024-09-10T21:13:17.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.372+0000 7ff09fc00640 1 -- 172.21.15.16:0/1545058424 wait complete. 2024-09-10T21:13:17.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff09fc00640 1 Processor -- start 2024-09-10T21:13:17.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff09fc00640 1 -- start start 2024-09-10T21:13:17.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff09fc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0980ce040 0x7ff0981f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff09fc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0980cfe30 0x7ff0980b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff09fc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff0980d0210 0x7ff0980b7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff09fc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0980d2b00 con 0x7ff0980cfe30 2024-09-10T21:13:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff09fc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff0980b14f0 con 0x7ff0980ce040 2024-09-10T21:13:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff09fc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff0980cf1e0 con 0x7ff0980d0210 2024-09-10T21:13:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff097e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0980cfe30 0x7ff0980b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff097e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0980cfe30 0x7ff0980b6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37084/0 (socket says 172.21.15.16:37084) 2024-09-10T21:13:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff097e00640 1 -- 172.21.15.16:0/4267979890 learned_addr learned my addr 172.21.15.16:0/4267979890 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff09d200640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0980ce040 0x7ff0981f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff09dc00640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff0980d0210 0x7ff0980b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff097e00640 1 -- 172.21.15.16:0/4267979890 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff0980d0210 msgr2=0x7ff0980b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff097e00640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff0980d0210 0x7ff0980b7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff097e00640 1 -- 172.21.15.16:0/4267979890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0980ce040 msgr2=0x7ff0981f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff097e00640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0980ce040 0x7ff0981f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff097e00640 1 -- 172.21.15.16:0/4267979890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09806fb80 con 0x7ff0980cfe30 2024-09-10T21:13:17.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff09d200640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0980ce040 0x7ff0981f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:17.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff097e00640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0980cfe30 0x7ff0980b6c50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff08400b410 tx=0x7ff08400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:17.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff094c00640 1 -- 172.21.15.16:0/4267979890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084015030 con 0x7ff0980cfe30 2024-09-10T21:13:17.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff094c00640 1 -- 172.21.15.16:0/4267979890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff084004020 con 0x7ff0980cfe30 2024-09-10T21:13:17.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff094c00640 1 -- 172.21.15.16:0/4267979890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0840044b0 con 0x7ff0980cfe30 2024-09-10T21:13:17.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff09fc00640 1 -- 172.21.15.16:0/4267979890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0981f8410 con 0x7ff0980cfe30 2024-09-10T21:13:17.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.376+0000 7ff09fc00640 1 -- 172.21.15.16:0/4267979890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0981f4a00 con 0x7ff0980cfe30 2024-09-10T21:13:17.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.380+0000 7ff09fc00640 1 -- 172.21.15.16:0/4267979890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0981f8a20 con 0x7ff0980cfe30 2024-09-10T21:13:17.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.380+0000 7ff094c00640 1 -- 172.21.15.16:0/4267979890 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff084022060 con 0x7ff0980cfe30 2024-09-10T21:13:17.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.380+0000 7ff094c00640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff0780725e0 0x7ff078074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.380+0000 7ff094c00640 1 -- 172.21.15.16:0/4267979890 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7ff084092af0 con 0x7ff0980cfe30 2024-09-10T21:13:17.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.380+0000 7ff09d200640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff0780725e0 0x7ff078074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.380+0000 7ff09d200640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff0780725e0 0x7ff078074aa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0b2000 tx=0x7ff08c01d910 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.384+0000 7ff094c00640 1 -- 172.21.15.16:0/4267979890 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff08405e4c0 con 0x7ff0980cfe30 2024-09-10T21:13:17.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.560+0000 7ff09fc00640 1 -- 172.21.15.16:0/4267979890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff0980b1710 con 0x7ff0980cfe30 2024-09-10T21:13:17.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.564+0000 7ff094c00640 1 -- 172.21.15.16:0/4267979890 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7ff08405ddc0 con 0x7ff0980cfe30 2024-09-10T21:13:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.568+0000 7ff09fc00640 1 -- 172.21.15.16:0/4267979890 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff0780725e0 msgr2=0x7ff078074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.568+0000 7ff09fc00640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff0780725e0 0x7ff078074aa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0b2000 tx=0x7ff08c01d910 comp rx=0 tx=0).stop 2024-09-10T21:13:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.568+0000 7ff09fc00640 1 -- 172.21.15.16:0/4267979890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0980cfe30 msgr2=0x7ff0980b6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.568+0000 7ff09fc00640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0980cfe30 0x7ff0980b6c50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff08400b410 tx=0x7ff08400b8d0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.568+0000 7ff09fc00640 1 -- 172.21.15.16:0/4267979890 shutdown_connections 2024-09-10T21:13:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.568+0000 7ff09fc00640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff0780725e0 0x7ff078074aa0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.568+0000 7ff09fc00640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff0980d0210 0x7ff0980b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.568+0000 7ff09fc00640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0980cfe30 0x7ff0980b6c50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.568+0000 7ff09fc00640 1 --2- 172.21.15.16:0/4267979890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0980ce040 0x7ff0981f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.568+0000 7ff09fc00640 1 -- 172.21.15.16:0/4267979890 >> 172.21.15.16:0/4267979890 conn(0x7ff0980d1df0 msgr2=0x7ff0980d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:17.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.568+0000 7ff09fc00640 1 -- 172.21.15.16:0/4267979890 shutdown_connections 2024-09-10T21:13:17.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:17.568+0000 7ff09fc00640 1 -- 172.21.15.16:0/4267979890 wait complete. 2024-09-10T21:13:17.593 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:13:17.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:13:17.679 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:13:17.796 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:13:17.796 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:17.796 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:17.805 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:13:17.805 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:13:17.805 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:17.815 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:13:17.815 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:17.815 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:17.857 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:13:17.857 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:13:17.857 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:17.867 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:13:17.867 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:13:17.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:18.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.044+0000 7f5494800640 1 -- 172.21.15.16:0/1706832384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c071230 msgr2=0x7f548c0d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:18.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.044+0000 7f5494800640 1 --2- 172.21.15.16:0/1706832384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c071230 0x7f548c0d0590 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f548c073f60 tx=0x7f547c01c890 comp rx=0 tx=0).stop 2024-09-10T21:13:18.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.044+0000 7f5494800640 1 -- 172.21.15.16:0/1706832384 shutdown_connections 2024-09-10T21:13:18.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.044+0000 7f5494800640 1 --2- 172.21.15.16:0/1706832384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c071230 0x7f548c0d0590 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.044+0000 7f5494800640 1 --2- 172.21.15.16:0/1706832384 >> v2:172.21.15.52:3300/0 conn(0x7f548c161620 0x7f548c070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.044+0000 7f5494800640 1 -- 172.21.15.16:0/1706832384 >> 172.21.15.16:0/1706832384 conn(0x7f548c09b130 msgr2=0x7f548c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:18.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.044+0000 7f5494800640 1 -- 172.21.15.16:0/1706832384 shutdown_connections 2024-09-10T21:13:18.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.044+0000 7f5494800640 1 -- 172.21.15.16:0/1706832384 wait complete. 2024-09-10T21:13:18.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.044+0000 7f5494800640 1 Processor -- start 2024-09-10T21:13:18.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5494800640 1 -- start start 2024-09-10T21:13:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5494800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f548c071230 0x7f548c1fc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5494800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b0b40 0x7f548c142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5494800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f548c161620 0x7f548c142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5494800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f548c16b310 con 0x7f548c0b0b40 2024-09-10T21:13:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5494800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f548c16b190 con 0x7f548c161620 2024-09-10T21:13:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5494800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f548c16b490 con 0x7f548c071230 2024-09-10T21:13:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5491400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b0b40 0x7f548c142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5491400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b0b40 0x7f548c142940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37116/0 (socket says 172.21.15.16:37116) 2024-09-10T21:13:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5491400640 1 -- 172.21.15.16:0/2005805631 learned_addr learned my addr 172.21.15.16:0/2005805631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5492800640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f548c161620 0x7f548c142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5491e00640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f548c071230 0x7f548c1fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5491400640 1 -- 172.21.15.16:0/2005805631 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f548c071230 msgr2=0x7f548c1fc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5491400640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f548c071230 0x7f548c1fc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5491400640 1 -- 172.21.15.16:0/2005805631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f548c161620 msgr2=0x7f548c142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5491400640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f548c161620 0x7f548c142e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5491400640 1 -- 172.21.15.16:0/2005805631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c0c8290 con 0x7f548c0b0b40 2024-09-10T21:13:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5492800640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f548c161620 0x7f548c142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5491e00640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f548c071230 0x7f548c1fc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5491400640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b0b40 0x7f548c142940 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f547c0b0040 tx=0x7f547c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:18.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5486000640 1 -- 172.21.15.16:0/2005805631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c0b2040 con 0x7f548c0b0b40 2024-09-10T21:13:18.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5486000640 1 -- 172.21.15.16:0/2005805631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f547c004450 con 0x7f548c0b0b40 2024-09-10T21:13:18.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5494800640 1 -- 172.21.15.16:0/2005805631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548c0c8570 con 0x7f548c0b0b40 2024-09-10T21:13:18.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5486000640 1 -- 172.21.15.16:0/2005805631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c005290 con 0x7f548c0b0b40 2024-09-10T21:13:18.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5494800640 1 -- 172.21.15.16:0/2005805631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f548c1fb8b0 con 0x7f548c0b0b40 2024-09-10T21:13:18.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.048+0000 7f5494800640 1 -- 172.21.15.16:0/2005805631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f548c0d0590 con 0x7f548c0b0b40 2024-09-10T21:13:18.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.052+0000 7f5486000640 1 -- 172.21.15.16:0/2005805631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f547c005430 con 0x7f548c0b0b40 2024-09-10T21:13:18.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.052+0000 7f5486000640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54700725e0 0x7f5470074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.052+0000 7f5486000640 1 -- 172.21.15.16:0/2005805631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f547c12cd20 con 0x7f548c0b0b40 2024-09-10T21:13:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.052+0000 7f5491e00640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54700725e0 0x7f5470074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.052+0000 7f5491e00640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54700725e0 0x7f5470074aa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f548000ba60 tx=0x7f5480005cf0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.056+0000 7f5486000640 1 -- 172.21.15.16:0/2005805631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f547c0164b0 con 0x7f548c0b0b40 2024-09-10T21:13:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5494800640 1 -- 172.21.15.16:0/2005805631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f548c15c6a0 con 0x7f548c0b0b40 2024-09-10T21:13:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5486000640 1 -- 172.21.15.16:0/2005805631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+3910 (secure 0 0 0) 0x7f547c0f86f0 con 0x7f548c0b0b40 2024-09-10T21:13:18.274 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:18.274 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"btime":"2024-09-10T21:13:11:817696+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/653119388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":653119388},{"type":"v1","addr":"172.21.15.16:6836","nonce":653119388}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2947833730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2947833730},{"type":"v1","addr":"172.21.15.16:6837","nonce":2947833730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2865311562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2865311562},{"type":"v1","addr":"172.21.15.52:6834","nonce":2865311562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":135116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1279598872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1279598872},{"type":"v1","addr":"172.21.15.52:6835","nonce":1279598872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[]} 2024-09-10T21:13:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 35 2024-09-10T21:13:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5494800640 1 -- 172.21.15.16:0/2005805631 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54700725e0 msgr2=0x7f5470074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5494800640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54700725e0 0x7f5470074aa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f548000ba60 tx=0x7f5480005cf0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5494800640 1 -- 172.21.15.16:0/2005805631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b0b40 msgr2=0x7f548c142940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5494800640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b0b40 0x7f548c142940 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f547c0b0040 tx=0x7f547c004040 comp rx=0 tx=0).stop 2024-09-10T21:13:18.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5494800640 1 -- 172.21.15.16:0/2005805631 shutdown_connections 2024-09-10T21:13:18.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5494800640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54700725e0 0x7f5470074aa0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5494800640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f548c161620 0x7f548c142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5494800640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f548c0b0b40 0x7f548c142940 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5494800640 1 --2- 172.21.15.16:0/2005805631 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f548c071230 0x7f548c1fc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5494800640 1 -- 172.21.15.16:0/2005805631 >> 172.21.15.16:0/2005805631 conn(0x7f548c09b130 msgr2=0x7f548c160860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:18.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5494800640 1 -- 172.21.15.16:0/2005805631 shutdown_connections 2024-09-10T21:13:18.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.268+0000 7f5494800640 1 -- 172.21.15.16:0/2005805631 wait complete. 2024-09-10T21:13:18.295 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:13:18.295 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:13:18.295 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:13:18.296 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:13:18.296 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:13:18.296 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:13:18.296 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:13:18.296 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:13:18.296 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:13:18.296 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:13:18.297 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:13:18.297 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:13:18.297 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:13:18.297 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:13:18.297 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:13:18.297 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:13:18.310 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:13:18.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:13:18.311 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:13:18.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:13:18.312 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:13:18.312 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:13:18.313 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:13:18.313 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:13:18.359 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:13:18.359 DEBUG:teuthology.parallel:result is None 2024-09-10T21:13:18.360 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:13:18.360 DEBUG:teuthology.parallel:result is None 2024-09-10T21:13:18.361 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:13:18.361 DEBUG:teuthology.parallel:result is None 2024-09-10T21:13:18.362 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:13:18.362 DEBUG:teuthology.parallel:result is None 2024-09-10T21:13:18.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:13:18.447 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-10T21:13:18.448 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-10T21:13:18.458 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:13:18.458 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:13:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.508+0000 7f2830e00640 1 -- 172.21.15.16:0/2702761398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c15d9c0 msgr2=0x7f282c0c73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.508+0000 7f2830e00640 1 --2- 172.21.15.16:0/2702761398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c15d9c0 0x7f282c0c73d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f282c073b40 tx=0x7f281c01c7d0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.508+0000 7f2830e00640 1 -- 172.21.15.16:0/2702761398 shutdown_connections 2024-09-10T21:13:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.508+0000 7f2830e00640 1 --2- 172.21.15.16:0/2702761398 >> v2:172.21.15.52:3300/0 conn(0x7f282c0cfb20 0x7f282c0b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.508+0000 7f2830e00640 1 --2- 172.21.15.16:0/2702761398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c15d9c0 0x7f282c0c73d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.508+0000 7f2830e00640 1 -- 172.21.15.16:0/2702761398 >> 172.21.15.16:0/2702761398 conn(0x7f282c0d0300 msgr2=0x7f282c0d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.508+0000 7f2830e00640 1 -- 172.21.15.16:0/2702761398 shutdown_connections 2024-09-10T21:13:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.508+0000 7f2830e00640 1 -- 172.21.15.16:0/2702761398 wait complete. 2024-09-10T21:13:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.508+0000 7f2830e00640 1 Processor -- start 2024-09-10T21:13:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.508+0000 7f2830e00640 1 -- start start 2024-09-10T21:13:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f2830e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f282c0b1800 0x7f282c1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f2830e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c0cfb20 0x7f282c0c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f2830e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f282c15d9c0 0x7f282c0c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f2830e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f282c0cff00 con 0x7f282c0cfb20 2024-09-10T21:13:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f2830e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f282c0c3be0 con 0x7f282c0b1800 2024-09-10T21:13:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f2830e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f282c0d0080 con 0x7f282c15d9c0 2024-09-10T21:13:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f282aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c0cfb20 0x7f282c0c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f282aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c0cfb20 0x7f282c0c3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37182/0 (socket says 172.21.15.16:37182) 2024-09-10T21:13:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f282aa00640 1 -- 172.21.15.16:0/850479152 learned_addr learned my addr 172.21.15.16:0/850479152 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f282b400640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f282c0b1800 0x7f282c1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f282be00640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f282c15d9c0 0x7f282c0c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f282aa00640 1 -- 172.21.15.16:0/850479152 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f282c15d9c0 msgr2=0x7f282c0c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f282aa00640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f282c15d9c0 0x7f282c0c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f282aa00640 1 -- 172.21.15.16:0/850479152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f282c0b1800 msgr2=0x7f282c1550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f282aa00640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f282c0b1800 0x7f282c1550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f282aa00640 1 -- 172.21.15.16:0/850479152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282c1f78f0 con 0x7f282c0cfb20 2024-09-10T21:13:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f282b400640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f282c0b1800 0x7f282c1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f282aa00640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c0cfb20 0x7f282c0c3fe0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f281800e9f0 tx=0x7f281800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f2823400640 1 -- 172.21.15.16:0/850479152 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281800cd80 con 0x7f282c0cfb20 2024-09-10T21:13:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f2823400640 1 -- 172.21.15.16:0/850479152 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2818010430 con 0x7f282c0cfb20 2024-09-10T21:13:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f2830e00640 1 -- 172.21.15.16:0/850479152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282c1f7b60 con 0x7f282c0cfb20 2024-09-10T21:13:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f2823400640 1 -- 172.21.15.16:0/850479152 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818010860 con 0x7f282c0cfb20 2024-09-10T21:13:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f2830e00640 1 -- 172.21.15.16:0/850479152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282c166d00 con 0x7f282c0cfb20 2024-09-10T21:13:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.512+0000 7f2830e00640 1 -- 172.21.15.16:0/850479152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f282c0b12c0 con 0x7f282c0cfb20 2024-09-10T21:13:18.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.516+0000 7f2823400640 1 -- 172.21.15.16:0/850479152 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2818010a00 con 0x7f282c0cfb20 2024-09-10T21:13:18.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.516+0000 7f2823400640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f280c0725e0 0x7f280c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:18.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.516+0000 7f2823400640 1 -- 172.21.15.16:0/850479152 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f2818093b30 con 0x7f282c0cfb20 2024-09-10T21:13:18.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.516+0000 7f282b400640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f280c0725e0 0x7f280c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:18.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.516+0000 7f282b400640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f280c0725e0 0x7f280c074aa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f281c0af000 tx=0x7f281c01b010 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:18.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.520+0000 7f2823400640 1 -- 172.21.15.16:0/850479152 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f281805f5c0 con 0x7f282c0cfb20 2024-09-10T21:13:18.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.700+0000 7f2830e00640 1 -- 172.21.15.16:0/850479152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f282c0b3c70 con 0x7f282c0cfb20 2024-09-10T21:13:18.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.852+0000 7f2823400640 1 -- 172.21.15.16:0/850479152 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v64) ==== 97+0+0 (secure 0 0 0) 0x7f281805eec0 con 0x7f282c0cfb20 2024-09-10T21:13:18.860 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-10T21:13:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.856+0000 7f2830e00640 1 -- 172.21.15.16:0/850479152 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f280c0725e0 msgr2=0x7f280c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.856+0000 7f2830e00640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f280c0725e0 0x7f280c074aa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f281c0af000 tx=0x7f281c01b010 comp rx=0 tx=0).stop 2024-09-10T21:13:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.856+0000 7f2830e00640 1 -- 172.21.15.16:0/850479152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c0cfb20 msgr2=0x7f282c0c3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.856+0000 7f2830e00640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c0cfb20 0x7f282c0c3fe0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f281800e9f0 tx=0x7f281800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.856+0000 7f2830e00640 1 -- 172.21.15.16:0/850479152 shutdown_connections 2024-09-10T21:13:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.856+0000 7f2830e00640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f280c0725e0 0x7f280c074aa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.856+0000 7f2830e00640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f282c15d9c0 0x7f282c0c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.856+0000 7f2830e00640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c0cfb20 0x7f282c0c3fe0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.856+0000 7f2830e00640 1 --2- 172.21.15.16:0/850479152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f282c0b1800 0x7f282c1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.856+0000 7f2830e00640 1 -- 172.21.15.16:0/850479152 >> 172.21.15.16:0/850479152 conn(0x7f282c0d0300 msgr2=0x7f282c070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.856+0000 7f2830e00640 1 -- 172.21.15.16:0/850479152 shutdown_connections 2024-09-10T21:13:18.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:18.856+0000 7f2830e00640 1 -- 172.21.15.16:0/850479152 wait complete. 2024-09-10T21:13:18.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:13:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.028+0000 7f184f400640 1 -- 172.21.15.16:0/904755287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18480705e0 msgr2=0x7f18480ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.028+0000 7f184f400640 1 --2- 172.21.15.16:0/904755287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18480705e0 0x7f18480ca170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1840009bc0 tx=0x7f184001cb10 comp rx=0 tx=0).stop 2024-09-10T21:13:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.028+0000 7f184f400640 1 -- 172.21.15.16:0/904755287 shutdown_connections 2024-09-10T21:13:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.028+0000 7f184f400640 1 --2- 172.21.15.16:0/904755287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18480705e0 0x7f18480ca170 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.028+0000 7f184f400640 1 --2- 172.21.15.16:0/904755287 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18480d06b0 0x7f18480700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.028+0000 7f184f400640 1 -- 172.21.15.16:0/904755287 >> 172.21.15.16:0/904755287 conn(0x7f18480cfb20 msgr2=0x7f18480cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.028+0000 7f184f400640 1 -- 172.21.15.16:0/904755287 shutdown_connections 2024-09-10T21:13:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.028+0000 7f184f400640 1 -- 172.21.15.16:0/904755287 wait complete. 2024-09-10T21:13:19.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.028+0000 7f184f400640 1 Processor -- start 2024-09-10T21:13:19.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f184f400640 1 -- start start 2024-09-10T21:13:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f184f400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18480705e0 0x7f1848160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f184f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480d06b0 0x7f184815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f184f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f184815a270 0x7f184815fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f184f400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18481671f0 con 0x7f18480d06b0 2024-09-10T21:13:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f184f400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1848072fd0 con 0x7f184815a270 2024-09-10T21:13:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f184f400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f1848167370 con 0x7f18480705e0 2024-09-10T21:13:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f1847e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480d06b0 0x7f184815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f1847e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480d06b0 0x7f184815b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37198/0 (socket says 172.21.15.16:37198) 2024-09-10T21:13:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f1847e00640 1 -- 172.21.15.16:0/1829779554 learned_addr learned my addr 172.21.15.16:0/1829779554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f184ca00640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18480705e0 0x7f1848160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f184d400640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f184815a270 0x7f184815fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f1847e00640 1 -- 172.21.15.16:0/1829779554 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18480705e0 msgr2=0x7f1848160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f1847e00640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18480705e0 0x7f1848160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f1847e00640 1 -- 172.21.15.16:0/1829779554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f184815a270 msgr2=0x7f184815fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f1847e00640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f184815a270 0x7f184815fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f1847e00640 1 -- 172.21.15.16:0/1829779554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184815e7c0 con 0x7f18480d06b0 2024-09-10T21:13:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f1847e00640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480d06b0 0x7f184815b900 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f183400ea90 tx=0x7f183400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f1844c00640 1 -- 172.21.15.16:0/1829779554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183401b070 con 0x7f18480d06b0 2024-09-10T21:13:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f184f400640 1 -- 172.21.15.16:0/1829779554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184815e4e0 con 0x7f18480d06b0 2024-09-10T21:13:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f1844c00640 1 -- 172.21.15.16:0/1829779554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f183400ce30 con 0x7f18480d06b0 2024-09-10T21:13:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f1844c00640 1 -- 172.21.15.16:0/1829779554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834016650 con 0x7f18480d06b0 2024-09-10T21:13:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f184f400640 1 -- 172.21.15.16:0/1829779554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184815f5a0 con 0x7f18480d06b0 2024-09-10T21:13:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.032+0000 7f184f400640 1 -- 172.21.15.16:0/1829779554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18480ca170 con 0x7f18480d06b0 2024-09-10T21:13:19.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.036+0000 7f1844c00640 1 -- 172.21.15.16:0/1829779554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f183400b040 con 0x7f18480d06b0 2024-09-10T21:13:19.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.036+0000 7f1844c00640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18280725e0 0x7f1828074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.036+0000 7f1844c00640 1 -- 172.21.15.16:0/1829779554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7f1834093220 con 0x7f18480d06b0 2024-09-10T21:13:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.036+0000 7f184ca00640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18280725e0 0x7f1828074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.036+0000 7f184ca00640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18280725e0 0x7f1828074aa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f183c001870 tx=0x7f183c001380 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:19.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.040+0000 7f1844c00640 1 -- 172.21.15.16:0/1829779554 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f183405e570 con 0x7f18480d06b0 2024-09-10T21:13:19.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.272+0000 7f184f400640 1 -- 172.21.15.16:0/1829779554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f18480b5aa0 con 0x7f18480d06b0 2024-09-10T21:13:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.272+0000 7f1844c00640 1 -- 172.21.15.16:0/1829779554 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) ==== 80+0+5304 (secure 0 0 0) 0x7f1834091e20 con 0x7f18480d06b0 2024-09-10T21:13:19.280 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:19.280 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:13:19.281 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-10T21:13:19.281 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.281 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-10T21:13:19.281 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmhg+lJBAAWjuYW75BGo2dmyAUi7s9+A==", 2024-09-10T21:13:19.281 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.281 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:19.281 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:13:19.281 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:13:19.281 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.281 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.281 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.281 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-10T21:13:19.282 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmQr7+MBAAOzXhWIirBOoi/7wKnrFgmg==", 2024-09-10T21:13:19.282 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.282 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:19.282 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:13:19.282 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:13:19.282 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.282 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.282 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.282 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-10T21:13:19.282 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmxxoALBAAgkmw91/VP1Uv2WMZsJKAyA==", 2024-09-10T21:13:19.282 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.283 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:19.283 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:13:19.283 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:13:19.283 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.283 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.283 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.283 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-10T21:13:19.283 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmA4tzOBAAiS+8n+Gokz6yXapVf2ecYg==", 2024-09-10T21:13:19.283 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.283 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:19.283 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:13:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:13:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-10T21:13:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHteBmRoahLRAA32PsUkqtGkeI7tMFRwas5Q==", 2024-09-10T21:13:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-10T21:13:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCLteBmGmEMBhAAr1yQEf/NkiN4+ezAQ9VR2A==", 2024-09-10T21:13:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.285 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-10T21:13:19.286 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCOteBmmMuAHBAAt+G/KApsZymjTLRSmyRbdw==", 2024-09-10T21:13:19.286 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.286 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:19.286 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:19.286 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.286 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.286 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.286 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-10T21:13:19.286 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCRteBmoQfNMBAA0vnIqlYoI13KNAAfNmJg3w==", 2024-09-10T21:13:19.286 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.286 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:19.286 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:19.287 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.287 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.287 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.287 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-10T21:13:19.287 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCWteBmc/7lDxAADlSi4UpcFJVNdp6YTkpdSw==", 2024-09-10T21:13:19.287 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.287 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:19.287 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:19.287 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.287 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.288 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.288 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-10T21:13:19.288 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZteBmBZ1LJhAAPcbZa1LQu4kNbSsJ/8Z81A==", 2024-09-10T21:13:19.288 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.288 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:19.288 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:19.288 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.288 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.288 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.288 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-10T21:13:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCcteBm7/czOxAAqREnY84+LF1N9dRXOhzz6g==", 2024-09-10T21:13:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-10T21:13:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgteBmG2J8FRAAB+cMFCnf9wJg1R3RD5SUJw==", 2024-09-10T21:13:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:19.289 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-10T21:13:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA==", 2024-09-10T21:13:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-10T21:13:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:13:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.290 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.291 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-10T21:13:19.291 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmuEm4ChAAe08QvhwIcN8ZHa3AoVcYww==", 2024-09-10T21:13:19.291 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.291 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:19.291 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:13:19.291 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:13:19.291 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:13:19.291 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.291 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.291 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-10T21:13:19.291 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBm50o1BxAAIFbVxVqU4rU8eKYpuZjTHA==", 2024-09-10T21:13:19.291 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:19.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:13:19.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:13:19.292 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:13:19.292 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.292 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.292 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-10T21:13:19.292 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmxtu0EBAAmN7+07lDiwjsIC/pNYUlSw==", 2024-09-10T21:13:19.292 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:19.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:13:19.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:13:19.293 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:13:19.293 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.293 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.293 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-10T21:13:19.293 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmfqgYCBAAm00tBe45phGieCcp1q46cw==", 2024-09-10T21:13:19.293 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:13:19.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:13:19.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:13:19.293 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.294 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.294 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.294 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-10T21:13:19.294 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmMVABIBAAoB72mPyXyUO6Y6k+fMqQQg==", 2024-09-10T21:13:19.294 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.294 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:13:19.294 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:13:19.294 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.294 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:19.294 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.294 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-10T21:13:19.295 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmeyYxHBAAvBkpTzjPQKAOlHzI1jQ46g==", 2024-09-10T21:13:19.295 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:19.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:13:19.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:13:19.295 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:19.295 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.295 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.295 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-10T21:13:19.295 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:13:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.276+0000 7f184f400640 1 -- 172.21.15.16:0/1829779554 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18280725e0 msgr2=0x7f1828074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.276+0000 7f184f400640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18280725e0 0x7f1828074aa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f183c001870 tx=0x7f183c001380 comp rx=0 tx=0).stop 2024-09-10T21:13:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.276+0000 7f184f400640 1 -- 172.21.15.16:0/1829779554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480d06b0 msgr2=0x7f184815b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.276+0000 7f184f400640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480d06b0 0x7f184815b900 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f183400ea90 tx=0x7f183400ef50 comp rx=0 tx=0).stop 2024-09-10T21:13:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.276+0000 7f184f400640 1 -- 172.21.15.16:0/1829779554 shutdown_connections 2024-09-10T21:13:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.276+0000 7f184f400640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18280725e0 0x7f1828074aa0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.276+0000 7f184f400640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f184815a270 0x7f184815fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.276+0000 7f184f400640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480d06b0 0x7f184815b900 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.276+0000 7f184f400640 1 --2- 172.21.15.16:0/1829779554 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18480705e0 0x7f1848160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.276+0000 7f184f400640 1 -- 172.21.15.16:0/1829779554 >> 172.21.15.16:0/1829779554 conn(0x7f18480cfb20 msgr2=0x7f18480b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.276+0000 7f184f400640 1 -- 172.21.15.16:0/1829779554 shutdown_connections 2024-09-10T21:13:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.276+0000 7f184f400640 1 -- 172.21.15.16:0/1829779554 wait complete. 2024-09-10T21:13:19.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:13:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.440+0000 7f8843200640 1 -- 172.21.15.16:0/1726847406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883c1599d0 msgr2=0x7f883c15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.440+0000 7f8843200640 1 --2- 172.21.15.16:0/1726847406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883c1599d0 0x7f883c15f830 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f882c009840 tx=0x7f882c01c790 comp rx=0 tx=0).stop 2024-09-10T21:13:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.440+0000 7f8843200640 1 -- 172.21.15.16:0/1726847406 shutdown_connections 2024-09-10T21:13:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.440+0000 7f8843200640 1 --2- 172.21.15.16:0/1726847406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883c1599d0 0x7f883c15f830 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.440+0000 7f8843200640 1 -- 172.21.15.16:0/1726847406 >> 172.21.15.16:0/1726847406 conn(0x7f883c0cfb20 msgr2=0x7f883c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.440+0000 7f8843200640 1 -- 172.21.15.16:0/1726847406 shutdown_connections 2024-09-10T21:13:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.440+0000 7f8843200640 1 -- 172.21.15.16:0/1726847406 wait complete. 2024-09-10T21:13:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8843200640 1 Processor -- start 2024-09-10T21:13:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8843200640 1 -- start start 2024-09-10T21:13:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8843200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f883c06e640 0x7f883c1fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8843200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883c0726b0 0x7f883c147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8843200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f883c1599d0 0x7f883c1f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8843200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f883c16a570 con 0x7f883c0726b0 2024-09-10T21:13:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8843200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f883c16a3f0 con 0x7f883c06e640 2024-09-10T21:13:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8843200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f883c16a6f0 con 0x7f883c1599d0 2024-09-10T21:13:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8833400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883c0726b0 0x7f883c147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8833400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883c0726b0 0x7f883c147ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37210/0 (socket says 172.21.15.16:37210) 2024-09-10T21:13:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8833400640 1 -- 172.21.15.16:0/3153041264 learned_addr learned my addr 172.21.15.16:0/3153041264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f883be00640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f883c06e640 0x7f883c1fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8841200640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f883c1599d0 0x7f883c1f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8833400640 1 -- 172.21.15.16:0/3153041264 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f883c1599d0 msgr2=0x7f883c1f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8833400640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f883c1599d0 0x7f883c1f7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8833400640 1 -- 172.21.15.16:0/3153041264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f883c06e640 msgr2=0x7f883c1fbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8833400640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f883c06e640 0x7f883c1fbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8833400640 1 -- 172.21.15.16:0/3153041264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c148400 con 0x7f883c0726b0 2024-09-10T21:13:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f883be00640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f883c06e640 0x7f883c1fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8841200640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f883c1599d0 0x7f883c1f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8833400640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883c0726b0 0x7f883c147ec0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f882400c4e0 tx=0x7f882400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8838c00640 1 -- 172.21.15.16:0/3153041264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824016020 con 0x7f883c0726b0 2024-09-10T21:13:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8838c00640 1 -- 172.21.15.16:0/3153041264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8824012920 con 0x7f883c0726b0 2024-09-10T21:13:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8838c00640 1 -- 172.21.15.16:0/3153041264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824012d50 con 0x7f883c0726b0 2024-09-10T21:13:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8843200640 1 -- 172.21.15.16:0/3153041264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883c1fb230 con 0x7f883c0726b0 2024-09-10T21:13:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.444+0000 7f8843200640 1 -- 172.21.15.16:0/3153041264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883c1f8a50 con 0x7f883c0726b0 2024-09-10T21:13:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.448+0000 7f8843200640 1 -- 172.21.15.16:0/3153041264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f883c15f830 con 0x7f883c0726b0 2024-09-10T21:13:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.448+0000 7f8838c00640 1 -- 172.21.15.16:0/3153041264 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f882401a3b0 con 0x7f883c0726b0 2024-09-10T21:13:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.448+0000 7f8838c00640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88180725e0 0x7f8818074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.448+0000 7f8838c00640 1 -- 172.21.15.16:0/3153041264 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7f8824092ff0 con 0x7f883c0726b0 2024-09-10T21:13:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.448+0000 7f883be00640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88180725e0 0x7f8818074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.448+0000 7f883be00640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88180725e0 0x7f8818074aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f882c0af000 tx=0x7f882c01afd0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.452+0000 7f8838c00640 1 -- 172.21.15.16:0/3153041264 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8824020030 con 0x7f883c0726b0 2024-09-10T21:13:19.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8843200640 1 -- 172.21.15.16:0/3153041264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f883c0d02d0 con 0x7f883c0726b0 2024-09-10T21:13:19.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8838c00640 1 -- 172.21.15.16:0/3153041264 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v66) ==== 84+0+65 (secure 0 0 0) 0x7f882405e980 con 0x7f883c0726b0 2024-09-10T21:13:19.638 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:19.638 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-10T21:13:19.638 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:19.639 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-10T21:13:19.639 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-10T21:13:19.639 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:19.639 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-10T21:13:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8843200640 1 -- 172.21.15.16:0/3153041264 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88180725e0 msgr2=0x7f8818074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8843200640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88180725e0 0x7f8818074aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f882c0af000 tx=0x7f882c01afd0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8843200640 1 -- 172.21.15.16:0/3153041264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883c0726b0 msgr2=0x7f883c147ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8843200640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883c0726b0 0x7f883c147ec0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f882400c4e0 tx=0x7f882400c9a0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8843200640 1 -- 172.21.15.16:0/3153041264 shutdown_connections 2024-09-10T21:13:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8843200640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88180725e0 0x7f8818074aa0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8843200640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f883c1599d0 0x7f883c1f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8843200640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883c0726b0 0x7f883c147ec0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8843200640 1 --2- 172.21.15.16:0/3153041264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f883c06e640 0x7f883c1fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8843200640 1 -- 172.21.15.16:0/3153041264 >> 172.21.15.16:0/3153041264 conn(0x7f883c0cfb20 msgr2=0x7f883c0731e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8843200640 1 -- 172.21.15.16:0/3153041264 shutdown_connections 2024-09-10T21:13:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.632+0000 7f8843200640 1 -- 172.21.15.16:0/3153041264 wait complete. 2024-09-10T21:13:19.661 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:13:19.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:13:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.804+0000 7fc4f8400640 1 -- 172.21.15.16:0/2001697891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f00ce7c0 msgr2=0x7fc4f00ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.804+0000 7fc4f8400640 1 --2- 172.21.15.16:0/2001697891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f00ce7c0 0x7fc4f00ceba0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009840 tx=0x7fc4e401c7c0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.804+0000 7fc4f8400640 1 -- 172.21.15.16:0/2001697891 shutdown_connections 2024-09-10T21:13:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.804+0000 7fc4f8400640 1 --2- 172.21.15.16:0/2001697891 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f00cfc40 0x7fc4f015ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.804+0000 7fc4f8400640 1 --2- 172.21.15.16:0/2001697891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f00ce7c0 0x7fc4f00ceba0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.804+0000 7fc4f8400640 1 -- 172.21.15.16:0/2001697891 >> 172.21.15.16:0/2001697891 conn(0x7fc4f00d1e60 msgr2=0x7fc4f00d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.804+0000 7fc4f8400640 1 -- 172.21.15.16:0/2001697891 shutdown_connections 2024-09-10T21:13:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.804+0000 7fc4f8400640 1 -- 172.21.15.16:0/2001697891 wait complete. 2024-09-10T21:13:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f8400640 1 Processor -- start 2024-09-10T21:13:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f8400640 1 -- start start 2024-09-10T21:13:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f8400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f00ce7c0 0x7fc4f0150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f8400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f00cfc40 0x7fc4f00b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f8400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f0160490 0x7fc4f015f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f8400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01693a0 con 0x7fc4f00cfc40 2024-09-10T21:13:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f8400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0169220 con 0x7fc4f00ce7c0 2024-09-10T21:13:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f8400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f0169520 con 0x7fc4f0160490 2024-09-10T21:13:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f00cfc40 0x7fc4f00b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f00cfc40 0x7fc4f00b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37236/0 (socket says 172.21.15.16:37236) 2024-09-10T21:13:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f5000640 1 -- 172.21.15.16:0/2909760593 learned_addr learned my addr 172.21.15.16:0/2909760593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f5a00640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f00ce7c0 0x7fc4f0150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f6400640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f0160490 0x7fc4f015f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f5000640 1 -- 172.21.15.16:0/2909760593 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f0160490 msgr2=0x7fc4f015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f5000640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f0160490 0x7fc4f015f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f5000640 1 -- 172.21.15.16:0/2909760593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f00ce7c0 msgr2=0x7fc4f0150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f5000640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f00ce7c0 0x7fc4f0150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f5000640 1 -- 172.21.15.16:0/2909760593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f00b60b0 con 0x7fc4f00cfc40 2024-09-10T21:13:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f5000640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f00cfc40 0x7fc4f00b56a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00e9f0 tx=0x7fc4dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4ed600640 1 -- 172.21.15.16:0/2909760593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc00cd80 con 0x7fc4f00cfc40 2024-09-10T21:13:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4ed600640 1 -- 172.21.15.16:0/2909760593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4dc010430 con 0x7fc4f00cfc40 2024-09-10T21:13:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f8400640 1 -- 172.21.15.16:0/2909760593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f00b6390 con 0x7fc4f00cfc40 2024-09-10T21:13:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4ed600640 1 -- 172.21.15.16:0/2909760593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc010860 con 0x7fc4f00cfc40 2024-09-10T21:13:19.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f8400640 1 -- 172.21.15.16:0/2909760593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f00b6c30 con 0x7fc4f00cfc40 2024-09-10T21:13:19.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.808+0000 7fc4f8400640 1 -- 172.21.15.16:0/2909760593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f00ceba0 con 0x7fc4f00cfc40 2024-09-10T21:13:19.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.812+0000 7fc4ed600640 1 -- 172.21.15.16:0/2909760593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc4dc010a00 con 0x7fc4f00cfc40 2024-09-10T21:13:19.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.812+0000 7fc4ed600640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc4d0072610 0x7fc4d0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.812+0000 7fc4ed600640 1 -- 172.21.15.16:0/2909760593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7fc4dc093a80 con 0x7fc4f00cfc40 2024-09-10T21:13:19.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.812+0000 7fc4f5a00640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc4d0072610 0x7fc4d0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.812+0000 7fc4f5a00640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc4d0072610 0x7fc4d0074ad0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40af000 tx=0x7fc4e401b060 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:19.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.816+0000 7fc4ed600640 1 -- 172.21.15.16:0/2909760593 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc4dc05ee80 con 0x7fc4f00cfc40 2024-09-10T21:13:20.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:19.996+0000 7fc4f8400640 1 -- 172.21.15.16:0/2909760593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc4f006d070 con 0x7fc4f00cfc40 2024-09-10T21:13:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:20.868+0000 7fc4ed600640 1 -- 172.21.15.16:0/2909760593 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v67) ==== 138+0+0 (secure 0 0 0) 0x7fc4dc0638b0 con 0x7fc4f00cfc40 2024-09-10T21:13:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-10T21:13:20.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:20.872+0000 7fc4f8400640 1 -- 172.21.15.16:0/2909760593 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc4d0072610 msgr2=0x7fc4d0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:20.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:20.872+0000 7fc4f8400640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc4d0072610 0x7fc4d0074ad0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40af000 tx=0x7fc4e401b060 comp rx=0 tx=0).stop 2024-09-10T21:13:20.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:20.872+0000 7fc4f8400640 1 -- 172.21.15.16:0/2909760593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f00cfc40 msgr2=0x7fc4f00b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:20.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:20.872+0000 7fc4f8400640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f00cfc40 0x7fc4f00b56a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00e9f0 tx=0x7fc4dc00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:13:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:20.872+0000 7fc4f8400640 1 -- 172.21.15.16:0/2909760593 shutdown_connections 2024-09-10T21:13:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:20.872+0000 7fc4f8400640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc4d0072610 0x7fc4d0074ad0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:20.872+0000 7fc4f8400640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f0160490 0x7fc4f015f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:20.872+0000 7fc4f8400640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f00cfc40 0x7fc4f00b56a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:20.872+0000 7fc4f8400640 1 --2- 172.21.15.16:0/2909760593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f00ce7c0 0x7fc4f0150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:20.872+0000 7fc4f8400640 1 -- 172.21.15.16:0/2909760593 >> 172.21.15.16:0/2909760593 conn(0x7fc4f00d1e60 msgr2=0x7fc4f00b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:20.872+0000 7fc4f8400640 1 -- 172.21.15.16:0/2909760593 shutdown_connections 2024-09-10T21:13:20.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:20.872+0000 7fc4f8400640 1 -- 172.21.15.16:0/2909760593 wait complete. 2024-09-10T21:13:20.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:13:21.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.052+0000 7f6b81600640 1 -- 172.21.15.16:0/1948051839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c0ce040 msgr2=0x7f6b7c1671a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.052+0000 7f6b81600640 1 --2- 172.21.15.16:0/1948051839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c0ce040 0x7f6b7c1671a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0cda10 tx=0x7f6b7401f580 comp rx=0 tx=0).stop 2024-09-10T21:13:21.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.052+0000 7f6b81600640 1 -- 172.21.15.16:0/1948051839 shutdown_connections 2024-09-10T21:13:21.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.052+0000 7f6b81600640 1 --2- 172.21.15.16:0/1948051839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c0ce040 0x7f6b7c1671a0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.052+0000 7f6b81600640 1 -- 172.21.15.16:0/1948051839 >> 172.21.15.16:0/1948051839 conn(0x7f6b7c0d1df0 msgr2=0x7f6b7c0d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:21.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b81600640 1 -- 172.21.15.16:0/1948051839 shutdown_connections 2024-09-10T21:13:21.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b81600640 1 -- 172.21.15.16:0/1948051839 wait complete. 2024-09-10T21:13:21.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b81600640 1 Processor -- start 2024-09-10T21:13:21.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b81600640 1 -- start start 2024-09-10T21:13:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b81600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b7c0ce040 0x7f6b7c1f8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b81600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c0cfe30 0x7f6b7c0c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b81600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6b7c0d0210 0x7f6b7c0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b81600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c0d2b20 con 0x7f6b7c0cfe30 2024-09-10T21:13:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b81600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c0b12e0 con 0x7f6b7c0ce040 2024-09-10T21:13:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b81600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7c1f8b70 con 0x7f6b7c0d0210 2024-09-10T21:13:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b73e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c0cfe30 0x7f6b7c0c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b73e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c0cfe30 0x7f6b7c0c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37254/0 (socket says 172.21.15.16:37254) 2024-09-10T21:13:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b73e00640 1 -- 172.21.15.16:0/2562819936 learned_addr learned my addr 172.21.15.16:0/2562819936 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b7aa00640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b7c0ce040 0x7f6b7c1f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b7b400640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6b7c0d0210 0x7f6b7c0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b73e00640 1 -- 172.21.15.16:0/2562819936 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6b7c0d0210 msgr2=0x7f6b7c0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b73e00640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6b7c0d0210 0x7f6b7c0c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b73e00640 1 -- 172.21.15.16:0/2562819936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b7c0ce040 msgr2=0x7f6b7c1f8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b73e00640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b7c0ce040 0x7f6b7c1f8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b73e00640 1 -- 172.21.15.16:0/2562819936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c0cb130 con 0x7f6b7c0cfe30 2024-09-10T21:13:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b7aa00640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b7c0ce040 0x7f6b7c1f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b73e00640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c0cfe30 0x7f6b7c0c8ec0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c420 tx=0x7f6b6400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b72a00640 1 -- 172.21.15.16:0/2562819936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64017070 con 0x7f6b7c0cfe30 2024-09-10T21:13:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b72a00640 1 -- 172.21.15.16:0/2562819936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b640053c0 con 0x7f6b7c0cfe30 2024-09-10T21:13:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b72a00640 1 -- 172.21.15.16:0/2562819936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64003860 con 0x7f6b7c0cfe30 2024-09-10T21:13:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.056+0000 7f6b81600640 1 -- 172.21.15.16:0/2562819936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c0cb410 con 0x7f6b7c0cfe30 2024-09-10T21:13:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.060+0000 7f6b81600640 1 -- 172.21.15.16:0/2562819936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c0cb9d0 con 0x7f6b7c0cfe30 2024-09-10T21:13:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.060+0000 7f6b81600640 1 -- 172.21.15.16:0/2562819936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b7c0ce590 con 0x7f6b7c0cfe30 2024-09-10T21:13:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.060+0000 7f6b72a00640 1 -- 172.21.15.16:0/2562819936 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6b64003a00 con 0x7f6b7c0cfe30 2024-09-10T21:13:21.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.064+0000 7f6b72a00640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b58072610 0x7f6b58074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.064+0000 7f6b72a00640 1 -- 172.21.15.16:0/2562819936 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5227+0+0 (secure 0 0 0) 0x7f6b64092ad0 con 0x7f6b7c0cfe30 2024-09-10T21:13:21.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.064+0000 7f6b7aa00640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b58072610 0x7f6b58074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.064+0000 7f6b7aa00640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b58072610 0x7f6b58074ad0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c000f70 tx=0x7f6b6c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.064+0000 7f6b72a00640 1 -- 172.21.15.16:0/2562819936 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6b64021020 con 0x7f6b7c0cfe30 2024-09-10T21:13:21.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.240+0000 7f6b81600640 1 -- 172.21.15.16:0/2562819936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6b7c0afd80 con 0x7f6b7c0cfe30 2024-09-10T21:13:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.872+0000 7f6b72a00640 1 -- 172.21.15.16:0/2562819936 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v68) ==== 170+0+0 (secure 0 0 0) 0x7f6b6405e400 con 0x7f6b7c0cfe30 2024-09-10T21:13:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-10T21:13:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.876+0000 7f6b81600640 1 -- 172.21.15.16:0/2562819936 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b58072610 msgr2=0x7f6b58074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.876+0000 7f6b81600640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b58072610 0x7f6b58074ad0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c000f70 tx=0x7f6b6c009040 comp rx=0 tx=0).stop 2024-09-10T21:13:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.876+0000 7f6b81600640 1 -- 172.21.15.16:0/2562819936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c0cfe30 msgr2=0x7f6b7c0c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.876+0000 7f6b81600640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c0cfe30 0x7f6b7c0c8ec0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c420 tx=0x7f6b6400c8e0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.876+0000 7f6b81600640 1 -- 172.21.15.16:0/2562819936 shutdown_connections 2024-09-10T21:13:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.876+0000 7f6b81600640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6b58072610 0x7f6b58074ad0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.876+0000 7f6b81600640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6b7c0d0210 0x7f6b7c0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.876+0000 7f6b81600640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c0cfe30 0x7f6b7c0c8ec0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.876+0000 7f6b81600640 1 --2- 172.21.15.16:0/2562819936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b7c0ce040 0x7f6b7c1f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.876+0000 7f6b81600640 1 -- 172.21.15.16:0/2562819936 >> 172.21.15.16:0/2562819936 conn(0x7f6b7c0d1df0 msgr2=0x7f6b7c0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.876+0000 7f6b81600640 1 -- 172.21.15.16:0/2562819936 shutdown_connections 2024-09-10T21:13:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:21.880+0000 7f6b81600640 1 -- 172.21.15.16:0/2562819936 wait complete. 2024-09-10T21:13:21.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:13:22.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.068+0000 7fa3ee800640 1 -- 172.21.15.16:0/2695635487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e80d0210 msgr2=0x7fa3e80cead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:22.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.068+0000 7fa3ee800640 1 --2- 172.21.15.16:0/2695635487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e80d0210 0x7fa3e80cead0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8074170 tx=0x7fa3d001f0b0 comp rx=0 tx=0).stop 2024-09-10T21:13:22.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.068+0000 7fa3ee800640 1 -- 172.21.15.16:0/2695635487 shutdown_connections 2024-09-10T21:13:22.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.068+0000 7fa3ee800640 1 --2- 172.21.15.16:0/2695635487 >> v2:172.21.15.52:3300/0 conn(0x7fa3e80ce040 0x7fa3e80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:22.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.068+0000 7fa3ee800640 1 --2- 172.21.15.16:0/2695635487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e80d0210 0x7fa3e80cead0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:22.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.068+0000 7fa3ee800640 1 --2- 172.21.15.16:0/2695635487 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa3e80cfe30 0x7fa3e80ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.068+0000 7fa3ee800640 1 -- 172.21.15.16:0/2695635487 >> 172.21.15.16:0/2695635487 conn(0x7fa3e80d1df0 msgr2=0x7fa3e80d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.068+0000 7fa3ee800640 1 -- 172.21.15.16:0/2695635487 shutdown_connections 2024-09-10T21:13:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.068+0000 7fa3ee800640 1 -- 172.21.15.16:0/2695635487 wait complete. 2024-09-10T21:13:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3ee800640 1 Processor -- start 2024-09-10T21:13:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3ee800640 1 -- start start 2024-09-10T21:13:22.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3ee800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3e80ce040 0x7fa3e8072270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:22.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3ee800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa3e80cfe30 0x7fa3e81f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:22.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3ee800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e80d0210 0x7fa3e81f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:22.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3ee800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e80b1460 con 0x7fa3e80d0210 2024-09-10T21:13:22.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3ee800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e80b12e0 con 0x7fa3e80ce040 2024-09-10T21:13:22.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3ee800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e815fee0 con 0x7fa3e80cfe30 2024-09-10T21:13:22.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e80d0210 0x7fa3e81f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:22.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3e80ce040 0x7fa3e8072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:22.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e80d0210 0x7fa3e81f1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41772/0 (socket says 172.21.15.16:41772) 2024-09-10T21:13:22.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3e7e00640 1 -- 172.21.15.16:0/2354771220 learned_addr learned my addr 172.21.15.16:0/2354771220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:22.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3e80ce040 0x7fa3e8072270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:54064/0 (socket says 172.21.15.16:54064) 2024-09-10T21:13:22.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3dfe00640 1 --2- 172.21.15.16:0/2354771220 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa3e80cfe30 0x7fa3e81f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:22.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3e7e00640 1 -- 172.21.15.16:0/2354771220 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa3e80cfe30 msgr2=0x7fa3e81f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:22.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3e7e00640 1 --2- 172.21.15.16:0/2354771220 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa3e80cfe30 0x7fa3e81f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:22.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3e7e00640 1 -- 172.21.15.16:0/2354771220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3e80ce040 msgr2=0x7fa3e8072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:22.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3e7e00640 1 --2- 172.21.15.16:0/2354771220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3e80ce040 0x7fa3e8072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:22.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3e7e00640 1 -- 172.21.15.16:0/2354771220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e80c7ad0 con 0x7fa3e80d0210 2024-09-10T21:13:22.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3e7e00640 1 --2- 172.21.15.16:0/2354771220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e80d0210 0x7fa3e81f1800 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000e990 tx=0x7fa3e000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:22.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3df400640 1 -- 172.21.15.16:0/2354771220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0004d30 con 0x7fa3e80d0210 2024-09-10T21:13:22.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3df400640 1 -- 172.21.15.16:0/2354771220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e000caa0 con 0x7fa3e80d0210 2024-09-10T21:13:22.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3df400640 1 -- 172.21.15.16:0/2354771220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0005020 con 0x7fa3e80d0210 2024-09-10T21:13:22.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.072+0000 7fa3ee800640 1 -- 172.21.15.16:0/2354771220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e81f4c00 con 0x7fa3e80d0210 2024-09-10T21:13:22.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.076+0000 7fa3ee800640 1 -- 172.21.15.16:0/2354771220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e81f51c0 con 0x7fa3e80d0210 2024-09-10T21:13:22.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.076+0000 7fa3ee800640 1 -- 172.21.15.16:0/2354771220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e8160390 con 0x7fa3e80d0210 2024-09-10T21:13:22.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.076+0000 7fa3df400640 1 -- 172.21.15.16:0/2354771220 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa3e000f040 con 0x7fa3e80d0210 2024-09-10T21:13:22.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.076+0000 7fa3df400640 1 --2- 172.21.15.16:0/2354771220 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa3c40725e0 0x7fa3c4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:22.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.076+0000 7fa3df400640 1 -- 172.21.15.16:0/2354771220 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5609+0+0 (secure 0 0 0) 0x7fa3e0093700 con 0x7fa3e80d0210 2024-09-10T21:13:22.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.076+0000 7fa3e7400640 1 --2- 172.21.15.16:0/2354771220 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa3c40725e0 0x7fa3c4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:22.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.080+0000 7fa3e7400640 1 --2- 172.21.15.16:0/2354771220 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa3c40725e0 0x7fa3c4074aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8004630 tx=0x7fa3d800b390 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:22.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.080+0000 7fa3df400640 1 -- 172.21.15.16:0/2354771220 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa3e80c7ad0 con 0x7fa3e80d0210 2024-09-10T21:13:22.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.288+0000 7fa3ee800640 1 -- 172.21.15.16:0/2354771220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa3e8071680 con 0x7fa3e80d0210 2024-09-10T21:13:22.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.876+0000 7fa3df400640 1 -- 172.21.15.16:0/2354771220 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v38) ==== 174+0+0 (secure 0 0 0) 0x7fa3e0016030 con 0x7fa3e80d0210 2024-09-10T21:13:22.882 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 10 and data pool 11 2024-09-10T21:13:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.880+0000 7fa3ee800640 1 -- 172.21.15.16:0/2354771220 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa3c40725e0 msgr2=0x7fa3c4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.880+0000 7fa3ee800640 1 --2- 172.21.15.16:0/2354771220 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa3c40725e0 0x7fa3c4074aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8004630 tx=0x7fa3d800b390 comp rx=0 tx=0).stop 2024-09-10T21:13:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.880+0000 7fa3ee800640 1 -- 172.21.15.16:0/2354771220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e80d0210 msgr2=0x7fa3e81f1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.880+0000 7fa3ee800640 1 --2- 172.21.15.16:0/2354771220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e80d0210 0x7fa3e81f1800 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000e990 tx=0x7fa3e000ee50 comp rx=0 tx=0).stop 2024-09-10T21:13:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.880+0000 7fa3ee800640 1 -- 172.21.15.16:0/2354771220 shutdown_connections 2024-09-10T21:13:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.880+0000 7fa3ee800640 1 --2- 172.21.15.16:0/2354771220 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa3c40725e0 0x7fa3c4074aa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.880+0000 7fa3ee800640 1 --2- 172.21.15.16:0/2354771220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e80d0210 0x7fa3e81f1800 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.880+0000 7fa3ee800640 1 --2- 172.21.15.16:0/2354771220 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa3e80cfe30 0x7fa3e81f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.880+0000 7fa3ee800640 1 --2- 172.21.15.16:0/2354771220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3e80ce040 0x7fa3e8072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.880+0000 7fa3ee800640 1 -- 172.21.15.16:0/2354771220 >> 172.21.15.16:0/2354771220 conn(0x7fa3e80d1df0 msgr2=0x7fa3e80b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.880+0000 7fa3ee800640 1 -- 172.21.15.16:0/2354771220 shutdown_connections 2024-09-10T21:13:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:22.880+0000 7fa3ee800640 1 -- 172.21.15.16:0/2354771220 wait complete. 2024-09-10T21:13:22.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:13:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.072+0000 7f67dda00640 1 -- 172.21.15.16:0/1124626991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67bc004b20 msgr2=0x7f67bc004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.072+0000 7f67dda00640 1 --2- 172.21.15.16:0/1124626991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67bc004b20 0x7f67bc004f40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009b60 tx=0x7f67c801d280 comp rx=0 tx=0).stop 2024-09-10T21:13:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.072+0000 7f67d7400640 1 -- 172.21.15.16:0/1124626991 reap_dead start 2024-09-10T21:13:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.072+0000 7f67dda00640 1 -- 172.21.15.16:0/1124626991 shutdown_connections 2024-09-10T21:13:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.072+0000 7f67dda00640 1 -- 172.21.15.16:0/1124626991 >> 172.21.15.16:0/1124626991 conn(0x7f67d80cfb20 msgr2=0x7f67d80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.072+0000 7f67dda00640 1 -- 172.21.15.16:0/1124626991 shutdown_connections 2024-09-10T21:13:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.072+0000 7f67dda00640 1 -- 172.21.15.16:0/1124626991 wait complete. 2024-09-10T21:13:23.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67dda00640 1 Processor -- start 2024-09-10T21:13:23.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67dda00640 1 -- start start 2024-09-10T21:13:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67dda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67bc004b20 0x7f67d80b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67dda00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67d80c6bf0 0x7f67d80b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67dda00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67d81fb020 0x7f67d806e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67dda00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8157ef0 con 0x7f67bc004b20 2024-09-10T21:13:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67dda00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8157d70 con 0x7f67d81fb020 2024-09-10T21:13:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67dda00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f67d80ce850 con 0x7f67d80c6bf0 2024-09-10T21:13:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67d6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67bc004b20 0x7f67d80b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67d6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67bc004b20 0x7f67d80b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41784/0 (socket says 172.21.15.16:41784) 2024-09-10T21:13:23.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67d6a00640 1 -- 172.21.15.16:0/1228633790 learned_addr learned my addr 172.21.15.16:0/1228633790 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67d6a00640 1 -- 172.21.15.16:0/1228633790 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67d80c6bf0 msgr2=0x7f67d80b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67d6a00640 1 --2- 172.21.15.16:0/1228633790 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67d80c6bf0 0x7f67d80b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67d6a00640 1 -- 172.21.15.16:0/1228633790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67d81fb020 msgr2=0x7f67d806e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67d6a00640 1 --2- 172.21.15.16:0/1228633790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67d81fb020 0x7f67d806e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67d6a00640 1 -- 172.21.15.16:0/1228633790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d81f7cf0 con 0x7f67bc004b20 2024-09-10T21:13:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67d6a00640 1 --2- 172.21.15.16:0/1228633790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67bc004b20 0x7f67d80b0a10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f67c801d0b0 tx=0x7f67c80055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67cea00640 1 -- 172.21.15.16:0/1228633790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c80b1040 con 0x7f67bc004b20 2024-09-10T21:13:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67cea00640 1 -- 172.21.15.16:0/1228633790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67c80ac400 con 0x7f67bc004b20 2024-09-10T21:13:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67cea00640 1 -- 172.21.15.16:0/1228633790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8004290 con 0x7f67bc004b20 2024-09-10T21:13:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67dda00640 1 -- 172.21.15.16:0/1228633790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d81f7fd0 con 0x7f67bc004b20 2024-09-10T21:13:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67dda00640 1 -- 172.21.15.16:0/1228633790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d81f8590 con 0x7f67bc004b20 2024-09-10T21:13:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.076+0000 7f67dda00640 1 -- 172.21.15.16:0/1228633790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d80c83d0 con 0x7f67bc004b20 2024-09-10T21:13:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.080+0000 7f67cea00640 1 -- 172.21.15.16:0/1228633790 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f67d80ca980 con 0x7f67bc004b20 2024-09-10T21:13:23.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.080+0000 7f67cea00640 1 --2- 172.21.15.16:0/1228633790 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67b40725e0 0x7f67b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:23.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.080+0000 7f67cea00640 1 -- 172.21.15.16:0/1228633790 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f67c812d8d0 con 0x7f67bc004b20 2024-09-10T21:13:23.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.080+0000 7f67cfe00640 1 --2- 172.21.15.16:0/1228633790 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67b40725e0 0x7f67b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:23.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.080+0000 7f67cfe00640 1 --2- 172.21.15.16:0/1228633790 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67b40725e0 0x7f67b4074aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f67d81fc090 tx=0x7f67c0000f00 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.084+0000 7f67cea00640 1 -- 172.21.15.16:0/1228633790 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f67c80c4020 con 0x7f67bc004b20 2024-09-10T21:13:23.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.256+0000 7f67dda00640 1 -- 172.21.15.16:0/1228633790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67d80b5aa0 con 0x7f67bc004b20 2024-09-10T21:13:23.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.260+0000 7f67cea00640 1 -- 172.21.15.16:0/1228633790 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16633 (secure 0 0 0) 0x7f67c8016fc0 con 0x7f67bc004b20 2024-09-10T21:13:23.266 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:23.266 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":69,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:13:22.294525+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:13:20.010491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T21:13:21.255527+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:13:23.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.260+0000 7f67dda00640 1 -- 172.21.15.16:0/1228633790 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67b40725e0 msgr2=0x7f67b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:23.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.260+0000 7f67dda00640 1 --2- 172.21.15.16:0/1228633790 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67b40725e0 0x7f67b4074aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f67d81fc090 tx=0x7f67c0000f00 comp rx=0 tx=0).stop 2024-09-10T21:13:23.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.260+0000 7f67dda00640 1 -- 172.21.15.16:0/1228633790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67bc004b20 msgr2=0x7f67d80b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:23.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.260+0000 7f67dda00640 1 --2- 172.21.15.16:0/1228633790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67bc004b20 0x7f67d80b0a10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f67c801d0b0 tx=0x7f67c80055d0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.260+0000 7f67dda00640 1 -- 172.21.15.16:0/1228633790 shutdown_connections 2024-09-10T21:13:23.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.260+0000 7f67dda00640 1 --2- 172.21.15.16:0/1228633790 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67b40725e0 0x7f67b4074aa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.260+0000 7f67dda00640 1 --2- 172.21.15.16:0/1228633790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67d81fb020 0x7f67d806e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.260+0000 7f67dda00640 1 --2- 172.21.15.16:0/1228633790 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67d80c6bf0 0x7f67d80b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.260+0000 7f67dda00640 1 --2- 172.21.15.16:0/1228633790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67bc004b20 0x7f67d80b0a10 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.260+0000 7f67dda00640 1 -- 172.21.15.16:0/1228633790 >> 172.21.15.16:0/1228633790 conn(0x7f67d80cfb20 msgr2=0x7f67d815df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:23.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.260+0000 7f67dda00640 1 -- 172.21.15.16:0/1228633790 shutdown_connections 2024-09-10T21:13:23.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.260+0000 7f67dda00640 1 -- 172.21.15.16:0/1228633790 wait complete. 2024-09-10T21:13:23.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:13:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.428+0000 7f7ac6000640 1 -- 172.21.15.16:0/1996754752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac015f830 msgr2=0x7f7ac00ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.428+0000 7f7ac6000640 1 --2- 172.21.15.16:0/1996754752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac015f830 0x7f7ac00ca170 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009840 tx=0x7f7aa801c820 comp rx=0 tx=0).stop 2024-09-10T21:13:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.428+0000 7f7ac6000640 1 -- 172.21.15.16:0/1996754752 shutdown_connections 2024-09-10T21:13:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.428+0000 7f7ac6000640 1 --2- 172.21.15.16:0/1996754752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac015f830 0x7f7ac00ca170 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.428+0000 7f7ac6000640 1 -- 172.21.15.16:0/1996754752 >> 172.21.15.16:0/1996754752 conn(0x7f7ac00cfb20 msgr2=0x7f7ac00cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.428+0000 7f7ac6000640 1 -- 172.21.15.16:0/1996754752 shutdown_connections 2024-09-10T21:13:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.428+0000 7f7ac6000640 1 -- 172.21.15.16:0/1996754752 wait complete. 2024-09-10T21:13:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.428+0000 7f7ac6000640 1 Processor -- start 2024-09-10T21:13:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.428+0000 7f7ac6000640 1 -- start start 2024-09-10T21:13:23.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ac6000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ac00ca6b0 0x7f7ac00c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:23.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ac6000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ac01599d0 0x7f7ac01669c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:23.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ac6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac015f830 0x7f7ac0163980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:23.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ac6000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac0169d60 con 0x7f7ac015f830 2024-09-10T21:13:23.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ac6000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac0169be0 con 0x7f7ac00ca6b0 2024-09-10T21:13:23.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ac6000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac0169ee0 con 0x7f7ac01599d0 2024-09-10T21:13:23.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7abfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac015f830 0x7f7ac0163980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:23.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7abf400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ac00ca6b0 0x7f7ac00c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:23.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7abfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac015f830 0x7f7ac0163980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41802/0 (socket says 172.21.15.16:41802) 2024-09-10T21:13:23.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7abfe00640 1 -- 172.21.15.16:0/2471644495 learned_addr learned my addr 172.21.15.16:0/2471644495 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:23.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ab7e00640 1 --2- 172.21.15.16:0/2471644495 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ac01599d0 0x7f7ac01669c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7abfe00640 1 -- 172.21.15.16:0/2471644495 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ac01599d0 msgr2=0x7f7ac01669c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7abfe00640 1 --2- 172.21.15.16:0/2471644495 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ac01599d0 0x7f7ac01669c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7abfe00640 1 -- 172.21.15.16:0/2471644495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ac00ca6b0 msgr2=0x7f7ac00c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7abfe00640 1 --2- 172.21.15.16:0/2471644495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ac00ca6b0 0x7f7ac00c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7abfe00640 1 -- 172.21.15.16:0/2471644495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac0165c00 con 0x7f7ac015f830 2024-09-10T21:13:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7abfe00640 1 --2- 172.21.15.16:0/2471644495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac015f830 0x7f7ac0163980 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800c700 tx=0x7f7ab800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ab7400640 1 -- 172.21.15.16:0/2471644495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8003db0 con 0x7f7ac015f830 2024-09-10T21:13:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ab7400640 1 -- 172.21.15.16:0/2471644495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ab800ce10 con 0x7f7ac015f830 2024-09-10T21:13:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ab7400640 1 -- 172.21.15.16:0/2471644495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8005290 con 0x7f7ac015f830 2024-09-10T21:13:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ac6000640 1 -- 172.21.15.16:0/2471644495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac0165ee0 con 0x7f7ac015f830 2024-09-10T21:13:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ac6000640 1 -- 172.21.15.16:0/2471644495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac01664a0 con 0x7f7ac015f830 2024-09-10T21:13:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ac6000640 1 -- 172.21.15.16:0/2471644495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac00ca170 con 0x7f7ac015f830 2024-09-10T21:13:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.432+0000 7f7ab7400640 1 -- 172.21.15.16:0/2471644495 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7ab8005430 con 0x7f7ac015f830 2024-09-10T21:13:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.436+0000 7f7ab7400640 1 --2- 172.21.15.16:0/2471644495 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a9c0725e0 0x7f7a9c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.436+0000 7f7ab7400640 1 -- 172.21.15.16:0/2471644495 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f7ab8092fe0 con 0x7f7ac015f830 2024-09-10T21:13:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.436+0000 7f7abf400640 1 --2- 172.21.15.16:0/2471644495 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a9c0725e0 0x7f7a9c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.436+0000 7f7abf400640 1 --2- 172.21.15.16:0/2471644495 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a9c0725e0 0x7f7a9c074aa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0007fd0 tx=0x7f7ab0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.440+0000 7f7ab7400640 1 -- 172.21.15.16:0/2471644495 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7ab805dfd0 con 0x7f7ac015f830 2024-09-10T21:13:23.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:13:23.503 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:13:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.620+0000 7f7ac6000640 1 -- 172.21.15.16:0/2471644495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ac015e3b0 con 0x7f7ac015f830 2024-09-10T21:13:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.620+0000 7f7ab7400640 1 -- 172.21.15.16:0/2471644495 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16633 (secure 0 0 0) 0x7f7ab8062a00 con 0x7f7ac015f830 2024-09-10T21:13:23.627 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:23.627 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":69,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:13:22.294525+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:13:20.010491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T21:13:21.255527+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:13:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.624+0000 7f7ac6000640 1 -- 172.21.15.16:0/2471644495 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a9c0725e0 msgr2=0x7f7a9c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.624+0000 7f7ac6000640 1 --2- 172.21.15.16:0/2471644495 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a9c0725e0 0x7f7a9c074aa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0007fd0 tx=0x7f7ab0009040 comp rx=0 tx=0).stop 2024-09-10T21:13:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.624+0000 7f7ac6000640 1 -- 172.21.15.16:0/2471644495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac015f830 msgr2=0x7f7ac0163980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.624+0000 7f7ac6000640 1 --2- 172.21.15.16:0/2471644495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac015f830 0x7f7ac0163980 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800c700 tx=0x7f7ab800cbc0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.624+0000 7f7ac6000640 1 -- 172.21.15.16:0/2471644495 shutdown_connections 2024-09-10T21:13:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.624+0000 7f7ac6000640 1 --2- 172.21.15.16:0/2471644495 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a9c0725e0 0x7f7a9c074aa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.624+0000 7f7ac6000640 1 --2- 172.21.15.16:0/2471644495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac015f830 0x7f7ac0163980 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.624+0000 7f7ac6000640 1 --2- 172.21.15.16:0/2471644495 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ac01599d0 0x7f7ac01669c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.624+0000 7f7ac6000640 1 --2- 172.21.15.16:0/2471644495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ac00ca6b0 0x7f7ac00c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.624+0000 7f7ac6000640 1 -- 172.21.15.16:0/2471644495 >> 172.21.15.16:0/2471644495 conn(0x7f7ac00cfb20 msgr2=0x7f7ac00b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.624+0000 7f7ac6000640 1 -- 172.21.15.16:0/2471644495 shutdown_connections 2024-09-10T21:13:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.624+0000 7f7ac6000640 1 -- 172.21.15.16:0/2471644495 wait complete. 2024-09-10T21:13:23.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:13:23.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.792+0000 7fa1d0200640 1 -- 172.21.15.16:0/3572870073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b0003a40 msgr2=0x7fa1b0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:23.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.792+0000 7fa1d0200640 1 --2- 172.21.15.16:0/3572870073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b0003a40 0x7fa1b0003e20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009b60 tx=0x7fa1bc01d1b0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.792+0000 7fa1ce200640 1 -- 172.21.15.16:0/3572870073 reap_dead start 2024-09-10T21:13:23.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.792+0000 7fa1d0200640 1 -- 172.21.15.16:0/3572870073 shutdown_connections 2024-09-10T21:13:23.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.792+0000 7fa1d0200640 1 -- 172.21.15.16:0/3572870073 >> 172.21.15.16:0/3572870073 conn(0x7fa1c80d1df0 msgr2=0x7fa1c80d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:23.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.792+0000 7fa1d0200640 1 -- 172.21.15.16:0/3572870073 shutdown_connections 2024-09-10T21:13:23.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.792+0000 7fa1d0200640 1 -- 172.21.15.16:0/3572870073 wait complete. 2024-09-10T21:13:23.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1d0200640 1 Processor -- start 2024-09-10T21:13:23.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1d0200640 1 -- start start 2024-09-10T21:13:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1d0200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b0003a40 0x7fa1c8158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1d0200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa1c80c98d0 0x7fa1c80cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1d0200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1c80cb890 0x7fa1c815b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1d0200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c80cf1e0 con 0x7fa1b0003a40 2024-09-10T21:13:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1d0200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c80b14f0 con 0x7fa1c80cb890 2024-09-10T21:13:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1d0200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c80cf360 con 0x7fa1c80c98d0 2024-09-10T21:13:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1cd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b0003a40 0x7fa1c8158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1cd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b0003a40 0x7fa1c8158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41806/0 (socket says 172.21.15.16:41806) 2024-09-10T21:13:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1cd800640 1 -- 172.21.15.16:0/769869342 learned_addr learned my addr 172.21.15.16:0/769869342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1cd800640 1 -- 172.21.15.16:0/769869342 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa1c80c98d0 msgr2=0x7fa1c80cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1cd800640 1 --2- 172.21.15.16:0/769869342 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa1c80c98d0 0x7fa1c80cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1cd800640 1 -- 172.21.15.16:0/769869342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1c80cb890 msgr2=0x7fa1c815b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1cd800640 1 --2- 172.21.15.16:0/769869342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1c80cb890 0x7fa1c815b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1cd800640 1 -- 172.21.15.16:0/769869342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c80cbd50 con 0x7fa1b0003a40 2024-09-10T21:13:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1cd800640 1 --2- 172.21.15.16:0/769869342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b0003a40 0x7fa1c8158230 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0af040 tx=0x7fa1bc005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1c5600640 1 -- 172.21.15.16:0/769869342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0b1040 con 0x7fa1b0003a40 2024-09-10T21:13:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1c5600640 1 -- 172.21.15.16:0/769869342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1bc0ac8a0 con 0x7fa1b0003a40 2024-09-10T21:13:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1d0200640 1 -- 172.21.15.16:0/769869342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c80cc030 con 0x7fa1b0003a40 2024-09-10T21:13:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1c5600640 1 -- 172.21.15.16:0/769869342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0accf0 con 0x7fa1b0003a40 2024-09-10T21:13:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.796+0000 7fa1d0200640 1 -- 172.21.15.16:0/769869342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c80c7050 con 0x7fa1b0003a40 2024-09-10T21:13:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.800+0000 7fa1d0200640 1 -- 172.21.15.16:0/769869342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c80ce880 con 0x7fa1b0003a40 2024-09-10T21:13:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.800+0000 7fa1c5600640 1 -- 172.21.15.16:0/769869342 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa1bc0046d0 con 0x7fa1b0003a40 2024-09-10T21:13:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.800+0000 7fa1c5600640 1 --2- 172.21.15.16:0/769869342 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa1a80725e0 0x7fa1a8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.800+0000 7fa1c5600640 1 -- 172.21.15.16:0/769869342 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fa1bc12d680 con 0x7fa1b0003a40 2024-09-10T21:13:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.800+0000 7fa1cce00640 1 --2- 172.21.15.16:0/769869342 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa1a80725e0 0x7fa1a8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.800+0000 7fa1cce00640 1 --2- 172.21.15.16:0/769869342 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa1a80725e0 0x7fa1a8074aa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80c9d10 tx=0x7fa1b4001d40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:23.804+0000 7fa1c5600640 1 -- 172.21.15.16:0/769869342 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa1bc0a5e20 con 0x7fa1b0003a40 2024-09-10T21:13:24.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.020+0000 7fa1d0200640 1 -- 172.21.15.16:0/769869342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa1c80cc200 con 0x7fa1b0003a40 2024-09-10T21:13:24.872 INFO:tasks.ceph.mds.b.smithi052.stderr:2024-09-10T21:13:24.862+0000 7f4cb1400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:13:24.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.896+0000 7fa1c5600640 1 -- 172.21.15.16:0/769869342 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) ==== 118+0+0 (secure 0 0 0) 0x7fa1bc016ec0 con 0x7fa1b0003a40 2024-09-10T21:13:24.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.900+0000 7fa1d0200640 1 -- 172.21.15.16:0/769869342 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa1a80725e0 msgr2=0x7fa1a8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:24.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.900+0000 7fa1d0200640 1 --2- 172.21.15.16:0/769869342 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa1a80725e0 0x7fa1a8074aa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80c9d10 tx=0x7fa1b4001d40 comp rx=0 tx=0).stop 2024-09-10T21:13:24.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.900+0000 7fa1d0200640 1 -- 172.21.15.16:0/769869342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b0003a40 msgr2=0x7fa1c8158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:24.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.900+0000 7fa1d0200640 1 --2- 172.21.15.16:0/769869342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b0003a40 0x7fa1c8158230 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0af040 tx=0x7fa1bc005620 comp rx=0 tx=0).stop 2024-09-10T21:13:24.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.900+0000 7fa1d0200640 1 -- 172.21.15.16:0/769869342 shutdown_connections 2024-09-10T21:13:24.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.900+0000 7fa1d0200640 1 --2- 172.21.15.16:0/769869342 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa1a80725e0 0x7fa1a8074aa0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:24.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.900+0000 7fa1d0200640 1 --2- 172.21.15.16:0/769869342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1c80cb890 0x7fa1c815b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:24.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.900+0000 7fa1d0200640 1 --2- 172.21.15.16:0/769869342 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa1c80c98d0 0x7fa1c80cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:24.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.900+0000 7fa1d0200640 1 --2- 172.21.15.16:0/769869342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1b0003a40 0x7fa1c8158230 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:24.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.900+0000 7fa1d0200640 1 -- 172.21.15.16:0/769869342 >> 172.21.15.16:0/769869342 conn(0x7fa1c80d1df0 msgr2=0x7fa1c81e9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:24.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.900+0000 7fa1d0200640 1 -- 172.21.15.16:0/769869342 shutdown_connections 2024-09-10T21:13:24.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:24.900+0000 7fa1d0200640 1 -- 172.21.15.16:0/769869342 wait complete. 2024-09-10T21:13:24.926 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:13:24.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:13:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 -- 172.21.15.16:0/807294455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46c161430 msgr2=0x7fb46c0d1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 --2- 172.21.15.16:0/807294455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46c161430 0x7fb46c0d1aa0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009840 tx=0x7fb45c01c820 comp rx=0 tx=0).stop 2024-09-10T21:13:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 -- 172.21.15.16:0/807294455 shutdown_connections 2024-09-10T21:13:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 --2- 172.21.15.16:0/807294455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46c161430 0x7fb46c0d1aa0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 -- 172.21.15.16:0/807294455 >> 172.21.15.16:0/807294455 conn(0x7fb46c0d3ed0 msgr2=0x7fb46c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 -- 172.21.15.16:0/807294455 shutdown_connections 2024-09-10T21:13:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 -- 172.21.15.16:0/807294455 wait complete. 2024-09-10T21:13:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 Processor -- start 2024-09-10T21:13:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 -- start start 2024-09-10T21:13:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46c0cf480 0x7fb46c150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb46c161430 0x7fb46c156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb46c161810 0x7fb46c151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c16c0a0 con 0x7fb46c0cf480 2024-09-10T21:13:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c16bf20 con 0x7fb46c161810 2024-09-10T21:13:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb472a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb46c16c220 con 0x7fb46c161430 2024-09-10T21:13:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb46be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46c0cf480 0x7fb46c150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb46be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46c0cf480 0x7fb46c150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41816/0 (socket says 172.21.15.16:41816) 2024-09-10T21:13:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb46be00640 1 -- 172.21.15.16:0/2717271427 learned_addr learned my addr 172.21.15.16:0/2717271427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb463e00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb46c161430 0x7fb46c156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb470a00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb46c161810 0x7fb46c151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb46be00640 1 -- 172.21.15.16:0/2717271427 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb46c161430 msgr2=0x7fb46c156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb46be00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb46c161430 0x7fb46c156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb46be00640 1 -- 172.21.15.16:0/2717271427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb46c161810 msgr2=0x7fb46c151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb46be00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb46c161810 0x7fb46c151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.072+0000 7fb46be00640 1 -- 172.21.15.16:0/2717271427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c1576b0 con 0x7fb46c0cf480 2024-09-10T21:13:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.076+0000 7fb46be00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46c0cf480 0x7fb46c150cb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0b0040 tx=0x7fb45c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.076+0000 7fb468c00640 1 -- 172.21.15.16:0/2717271427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0042b0 con 0x7fb46c0cf480 2024-09-10T21:13:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.076+0000 7fb468c00640 1 -- 172.21.15.16:0/2717271427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb45c003600 con 0x7fb46c0cf480 2024-09-10T21:13:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.076+0000 7fb468c00640 1 -- 172.21.15.16:0/2717271427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c003a50 con 0x7fb46c0cf480 2024-09-10T21:13:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.076+0000 7fb472a00640 1 -- 172.21.15.16:0/2717271427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c157990 con 0x7fb46c0cf480 2024-09-10T21:13:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.076+0000 7fb472a00640 1 -- 172.21.15.16:0/2717271427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c157f50 con 0x7fb46c0cf480 2024-09-10T21:13:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.076+0000 7fb472a00640 1 -- 172.21.15.16:0/2717271427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb46c0b40f0 con 0x7fb46c0cf480 2024-09-10T21:13:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.076+0000 7fb468c00640 1 -- 172.21.15.16:0/2717271427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb45c003bf0 con 0x7fb46c0cf480 2024-09-10T21:13:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.076+0000 7fb468c00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4480725e0 0x7fb448074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.080+0000 7fb468c00640 1 -- 172.21.15.16:0/2717271427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fb45c12e1d0 con 0x7fb46c0cf480 2024-09-10T21:13:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.080+0000 7fb463e00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4480725e0 0x7fb448074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.080+0000 7fb463e00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4480725e0 0x7fb448074aa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0b4640 tx=0x7fb454009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.084+0000 7fb468c00640 1 -- 172.21.15.16:0/2717271427 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb45c0c9020 con 0x7fb46c0cf480 2024-09-10T21:13:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.292+0000 7fb472a00640 1 -- 172.21.15.16:0/2717271427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb46c074800 con 0x7fb46c0cf480 2024-09-10T21:13:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.904+0000 7fb468c00640 1 -- 172.21.15.16:0/2717271427 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7fb45c016410 con 0x7fb46c0cf480 2024-09-10T21:13:25.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.908+0000 7fb472a00640 1 -- 172.21.15.16:0/2717271427 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4480725e0 msgr2=0x7fb448074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:25.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.908+0000 7fb472a00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4480725e0 0x7fb448074aa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0b4640 tx=0x7fb454009040 comp rx=0 tx=0).stop 2024-09-10T21:13:25.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.908+0000 7fb472a00640 1 -- 172.21.15.16:0/2717271427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46c0cf480 msgr2=0x7fb46c150cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:25.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.908+0000 7fb472a00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46c0cf480 0x7fb46c150cb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0b0040 tx=0x7fb45c0040c0 comp rx=0 tx=0).stop 2024-09-10T21:13:25.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.908+0000 7fb472a00640 1 -- 172.21.15.16:0/2717271427 shutdown_connections 2024-09-10T21:13:25.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.908+0000 7fb472a00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4480725e0 0x7fb448074aa0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:25.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.908+0000 7fb472a00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb46c161810 0x7fb46c151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:25.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.908+0000 7fb472a00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb46c161430 0x7fb46c156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:25.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.908+0000 7fb472a00640 1 --2- 172.21.15.16:0/2717271427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb46c0cf480 0x7fb46c150cb0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:25.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.908+0000 7fb472a00640 1 -- 172.21.15.16:0/2717271427 >> 172.21.15.16:0/2717271427 conn(0x7fb46c0d3ed0 msgr2=0x7fb46c0b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:25.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.908+0000 7fb472a00640 1 -- 172.21.15.16:0/2717271427 shutdown_connections 2024-09-10T21:13:25.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:25.908+0000 7fb472a00640 1 -- 172.21.15.16:0/2717271427 wait complete. 2024-09-10T21:13:25.935 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:13:25.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.076+0000 7f5b56a00640 1 -- 172.21.15.16:0/1718210442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b500b13c0 msgr2=0x7f5b500afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.076+0000 7f5b56a00640 1 --2- 172.21.15.16:0/1718210442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b500b13c0 0x7f5b500afac0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009bc0 tx=0x7f5b4801ca80 comp rx=0 tx=0).stop 2024-09-10T21:13:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.076+0000 7f5b56a00640 1 -- 172.21.15.16:0/1718210442 shutdown_connections 2024-09-10T21:13:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.076+0000 7f5b56a00640 1 --2- 172.21.15.16:0/1718210442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b500b13c0 0x7f5b500afac0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.076+0000 7f5b56a00640 1 --2- 172.21.15.16:0/1718210442 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b500716f0 0x7f5b500af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.076+0000 7f5b56a00640 1 -- 172.21.15.16:0/1718210442 >> 172.21.15.16:0/1718210442 conn(0x7f5b5009b130 msgr2=0x7f5b500d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.076+0000 7f5b56a00640 1 -- 172.21.15.16:0/1718210442 shutdown_connections 2024-09-10T21:13:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.076+0000 7f5b56a00640 1 -- 172.21.15.16:0/1718210442 wait complete. 2024-09-10T21:13:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.076+0000 7f5b56a00640 1 Processor -- start 2024-09-10T21:13:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b56a00640 1 -- start start 2024-09-10T21:13:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b56a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b500716f0 0x7f5b501fb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b56a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b500b13c0 0x7f5b5015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b56a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b5015f400 0x7f5b500c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b56a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b50168f60 con 0x7f5b500716f0 2024-09-10T21:13:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b56a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b50168de0 con 0x7f5b500b13c0 2024-09-10T21:13:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b56a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5b501690e0 con 0x7f5b5015f400 2024-09-10T21:13:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b4fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b500716f0 0x7f5b501fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b4fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b500716f0 0x7f5b501fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41836/0 (socket says 172.21.15.16:41836) 2024-09-10T21:13:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b4fe00640 1 -- 172.21.15.16:0/2774979712 learned_addr learned my addr 172.21.15.16:0/2774979712 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b54a00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b5015f400 0x7f5b500c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b47e00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b500b13c0 0x7f5b5015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b4fe00640 1 -- 172.21.15.16:0/2774979712 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b5015f400 msgr2=0x7f5b500c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b4fe00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b5015f400 0x7f5b500c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b4fe00640 1 -- 172.21.15.16:0/2774979712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b500b13c0 msgr2=0x7f5b5015ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b4fe00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b500b13c0 0x7f5b5015ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b4fe00640 1 -- 172.21.15.16:0/2774979712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b500c6bb0 con 0x7f5b500716f0 2024-09-10T21:13:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b54a00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b5015f400 0x7f5b500c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b4fe00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b500716f0 0x7f5b501fb760 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000c400 tx=0x7f5b4000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b4cc00640 1 -- 172.21.15.16:0/2774979712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40017070 con 0x7f5b500716f0 2024-09-10T21:13:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b4cc00640 1 -- 172.21.15.16:0/2774979712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b40003600 con 0x7f5b500716f0 2024-09-10T21:13:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b4cc00640 1 -- 172.21.15.16:0/2774979712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40003a30 con 0x7f5b500716f0 2024-09-10T21:13:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b56a00640 1 -- 172.21.15.16:0/2774979712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b500c6e90 con 0x7f5b500716f0 2024-09-10T21:13:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b56a00640 1 -- 172.21.15.16:0/2774979712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b500c7450 con 0x7f5b500716f0 2024-09-10T21:13:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.080+0000 7f5b56a00640 1 -- 172.21.15.16:0/2774979712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b50070fe0 con 0x7f5b500716f0 2024-09-10T21:13:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.084+0000 7f5b4cc00640 1 -- 172.21.15.16:0/2774979712 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5b40003bd0 con 0x7f5b500716f0 2024-09-10T21:13:26.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.084+0000 7f5b4cc00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b2c072610 0x7f5b2c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:26.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.084+0000 7f5b4cc00640 1 -- 172.21.15.16:0/2774979712 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f5b40092870 con 0x7f5b500716f0 2024-09-10T21:13:26.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.084+0000 7f5b47e00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b2c072610 0x7f5b2c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.084+0000 7f5b47e00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b2c072610 0x7f5b2c074ad0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5b380058d0 tx=0x7f5b38005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.088+0000 7f5b4cc00640 1 -- 172.21.15.16:0/2774979712 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5b4005deb0 con 0x7f5b500716f0 2024-09-10T21:13:26.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.296+0000 7f5b56a00640 1 -- 172.21.15.16:0/2774979712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b500af6a0 con 0x7f5b500716f0 2024-09-10T21:13:26.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.296+0000 7f5b4cc00640 1 -- 172.21.15.16:0/2774979712 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5234 (secure 0 0 0) 0x7f5b4005d7b0 con 0x7f5b500716f0 2024-09-10T21:13:26.305 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:26.305 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-10T21:13:25:905095+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4029516332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4029516332},{"type":"v1","addr":"172.21.15.16:6836","nonce":4029516332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/806881470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":806881470},{"type":"v1","addr":"172.21.15.16:6837","nonce":806881470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":135167,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2949376643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2949376643},{"type":"v1","addr":"172.21.15.52:6834","nonce":2949376643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:22.294493+0000","modified":"2024-09-10T21:13:25.298310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135170},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135170":{"gid":135170,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/1574608409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1574608409},{"type":"v1","addr":"172.21.15.52:6835","nonce":1574608409}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135170,"qdb_cluster":[135170]},"id":5}]} 2024-09-10T21:13:26.305 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-10T21:13:26.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.300+0000 7f5b56a00640 1 -- 172.21.15.16:0/2774979712 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b2c072610 msgr2=0x7f5b2c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:26.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.300+0000 7f5b56a00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b2c072610 0x7f5b2c074ad0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5b380058d0 tx=0x7f5b38005860 comp rx=0 tx=0).stop 2024-09-10T21:13:26.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.300+0000 7f5b56a00640 1 -- 172.21.15.16:0/2774979712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b500716f0 msgr2=0x7f5b501fb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:26.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.300+0000 7f5b56a00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b500716f0 0x7f5b501fb760 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000c400 tx=0x7f5b4000c8c0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.300+0000 7f5b56a00640 1 -- 172.21.15.16:0/2774979712 shutdown_connections 2024-09-10T21:13:26.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.300+0000 7f5b56a00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5b2c072610 0x7f5b2c074ad0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.300+0000 7f5b56a00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5b5015f400 0x7f5b500c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.300+0000 7f5b56a00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b500b13c0 0x7f5b5015ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.300+0000 7f5b56a00640 1 --2- 172.21.15.16:0/2774979712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b500716f0 0x7f5b501fb760 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.300+0000 7f5b56a00640 1 -- 172.21.15.16:0/2774979712 >> 172.21.15.16:0/2774979712 conn(0x7f5b5009b130 msgr2=0x7f5b500ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:26.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.300+0000 7f5b56a00640 1 -- 172.21.15.16:0/2774979712 shutdown_connections 2024-09-10T21:13:26.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.300+0000 7f5b56a00640 1 -- 172.21.15.16:0/2774979712 wait complete. 2024-09-10T21:13:26.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:13:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.468+0000 7f4ee9c00640 1 -- 172.21.15.16:0/2088430849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec8004d50 msgr2=0x7f4ec80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.468+0000 7f4ee9c00640 1 --2- 172.21.15.16:0/2088430849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec8004d50 0x7f4ec80051d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009e00 tx=0x7f4edc01d130 comp rx=0 tx=0).stop 2024-09-10T21:13:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.468+0000 7f4ee3400640 1 -- 172.21.15.16:0/2088430849 reap_dead start 2024-09-10T21:13:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.468+0000 7f4ee9c00640 1 -- 172.21.15.16:0/2088430849 shutdown_connections 2024-09-10T21:13:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.468+0000 7f4ee9c00640 1 -- 172.21.15.16:0/2088430849 >> 172.21.15.16:0/2088430849 conn(0x7f4ee409b130 msgr2=0x7f4ee40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.468+0000 7f4ee9c00640 1 -- 172.21.15.16:0/2088430849 shutdown_connections 2024-09-10T21:13:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.468+0000 7f4ee9c00640 1 -- 172.21.15.16:0/2088430849 wait complete. 2024-09-10T21:13:26.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee9c00640 1 Processor -- start 2024-09-10T21:13:26.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee9c00640 1 -- start start 2024-09-10T21:13:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee9c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4ec8004d50 0x7f4ee41f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee41f4740 0x7f4ee415a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee9c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee40c7250 0x7f4ee40afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee9c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee40aef00 con 0x7f4ee41f4740 2024-09-10T21:13:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee9c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee40aed80 con 0x7f4ee40c7250 2024-09-10T21:13:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee9c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee40af080 con 0x7f4ec8004d50 2024-09-10T21:13:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4edbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee41f4740 0x7f4ee415a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4edbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee41f4740 0x7f4ee415a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41846/0 (socket says 172.21.15.16:41846) 2024-09-10T21:13:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4edbe00640 1 -- 172.21.15.16:0/443695796 learned_addr learned my addr 172.21.15.16:0/443695796 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee3400640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee40c7250 0x7f4ee40afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee2a00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4ec8004d50 0x7f4ee41f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4edbe00640 1 -- 172.21.15.16:0/443695796 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4ec8004d50 msgr2=0x7f4ee41f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4edbe00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4ec8004d50 0x7f4ee41f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4edbe00640 1 -- 172.21.15.16:0/443695796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee40c7250 msgr2=0x7f4ee40afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4edbe00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee40c7250 0x7f4ee40afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4edbe00640 1 -- 172.21.15.16:0/443695796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee40b74e0 con 0x7f4ee41f4740 2024-09-10T21:13:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee3400640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee40c7250 0x7f4ee40afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee2a00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4ec8004d50 0x7f4ee41f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4edbe00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee41f4740 0x7f4ee415a200 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00fa00 tx=0x7f4ecc00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4edaa00640 1 -- 172.21.15.16:0/443695796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc00bcb0 con 0x7f4ee41f4740 2024-09-10T21:13:26.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4edaa00640 1 -- 172.21.15.16:0/443695796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ecc011440 con 0x7f4ee41f4740 2024-09-10T21:13:26.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee9c00640 1 -- 172.21.15.16:0/443695796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee41f4c90 con 0x7f4ee41f4740 2024-09-10T21:13:26.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4edaa00640 1 -- 172.21.15.16:0/443695796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc011830 con 0x7f4ee41f4740 2024-09-10T21:13:26.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee9c00640 1 -- 172.21.15.16:0/443695796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41f5250 con 0x7f4ee41f4740 2024-09-10T21:13:26.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.472+0000 7f4ee9c00640 1 -- 172.21.15.16:0/443695796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee4074b30 con 0x7f4ee41f4740 2024-09-10T21:13:26.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.476+0000 7f4edaa00640 1 -- 172.21.15.16:0/443695796 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4ecc023030 con 0x7f4ee41f4740 2024-09-10T21:13:26.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.476+0000 7f4edaa00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ec0072610 0x7f4ec0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:26.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.476+0000 7f4edaa00640 1 -- 172.21.15.16:0/443695796 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f4ecc093cc0 con 0x7f4ee41f4740 2024-09-10T21:13:26.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.476+0000 7f4ee2a00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ec0072610 0x7f4ec0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:26.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.476+0000 7f4ee2a00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ec0072610 0x7f4ec0074ad0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4000fc0 tx=0x7f4ed40015d0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:26.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.480+0000 7f4edaa00640 1 -- 172.21.15.16:0/443695796 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4ecc09a080 con 0x7f4ee41f4740 2024-09-10T21:13:26.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.660+0000 7f4ee9c00640 1 -- 172.21.15.16:0/443695796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ee415ff50 con 0x7f4ee41f4740 2024-09-10T21:13:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.664+0000 7f4edaa00640 1 -- 172.21.15.16:0/443695796 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16620 (secure 0 0 0) 0x7f4ecc05f300 con 0x7f4ee41f4740 2024-09-10T21:13:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:26.669 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:13:23.893029+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:13:20.010491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T21:13:21.255527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:13:26.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.664+0000 7f4ee9c00640 1 -- 172.21.15.16:0/443695796 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ec0072610 msgr2=0x7f4ec0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:26.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.664+0000 7f4ee9c00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ec0072610 0x7f4ec0074ad0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4000fc0 tx=0x7f4ed40015d0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.664+0000 7f4ee9c00640 1 -- 172.21.15.16:0/443695796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee41f4740 msgr2=0x7f4ee415a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:26.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.664+0000 7f4ee9c00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee41f4740 0x7f4ee415a200 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00fa00 tx=0x7f4ecc00fec0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.664+0000 7f4ee9c00640 1 -- 172.21.15.16:0/443695796 shutdown_connections 2024-09-10T21:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.664+0000 7f4ee9c00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ec0072610 0x7f4ec0074ad0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.664+0000 7f4ee9c00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ee40c7250 0x7f4ee40afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.664+0000 7f4ee9c00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee41f4740 0x7f4ee415a200 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.664+0000 7f4ee9c00640 1 --2- 172.21.15.16:0/443695796 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4ec8004d50 0x7f4ee41f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.664+0000 7f4ee9c00640 1 -- 172.21.15.16:0/443695796 >> 172.21.15.16:0/443695796 conn(0x7f4ee409b130 msgr2=0x7f4ee415f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.664+0000 7f4ee9c00640 1 -- 172.21.15.16:0/443695796 shutdown_connections 2024-09-10T21:13:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.664+0000 7f4ee9c00640 1 -- 172.21.15.16:0/443695796 wait complete. 2024-09-10T21:13:26.691 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:13:26.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:13:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.832+0000 7f077de00640 1 -- 172.21.15.16:0/1838975285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f075c003ab0 msgr2=0x7f075c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.832+0000 7f077de00640 1 --2- 172.21.15.16:0/1838975285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f075c003ab0 0x7f075c003ed0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0768006fa0 tx=0x7f076801ce50 comp rx=0 tx=0).stop 2024-09-10T21:13:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.832+0000 7f077de00640 1 -- 172.21.15.16:0/1838975285 shutdown_connections 2024-09-10T21:13:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.832+0000 7f077de00640 1 --2- 172.21.15.16:0/1838975285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f075c003ab0 0x7f075c003ed0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.832+0000 7f077de00640 1 -- 172.21.15.16:0/1838975285 >> 172.21.15.16:0/1838975285 conn(0x7f07780d1d80 msgr2=0x7f07780d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.832+0000 7f077de00640 1 -- 172.21.15.16:0/1838975285 shutdown_connections 2024-09-10T21:13:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.832+0000 7f077de00640 1 -- 172.21.15.16:0/1838975285 wait complete. 2024-09-10T21:13:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.832+0000 7f077de00640 1 Processor -- start 2024-09-10T21:13:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.832+0000 7f077de00640 1 -- start start 2024-09-10T21:13:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f077de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f075c003ab0 0x7f07781fb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f077de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07780b53a0 0x7f07781572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f077de00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07780b3ae0 0x7f07780b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f077de00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07780d0700 con 0x7f075c003ab0 2024-09-10T21:13:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f077de00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07780d0580 con 0x7f07780b53a0 2024-09-10T21:13:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f077de00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f07780ce500 con 0x7f07780b3ae0 2024-09-10T21:13:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f0776a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f075c003ab0 0x7f07781fb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f0776a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f075c003ab0 0x7f07781fb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41866/0 (socket says 172.21.15.16:41866) 2024-09-10T21:13:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f0776a00640 1 -- 172.21.15.16:0/793745176 learned_addr learned my addr 172.21.15.16:0/793745176 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f076fe00640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07780b53a0 0x7f07781572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f0777400640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07780b3ae0 0x7f07780b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f0776a00640 1 -- 172.21.15.16:0/793745176 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07780b3ae0 msgr2=0x7f07780b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f0776a00640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07780b3ae0 0x7f07780b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f0776a00640 1 -- 172.21.15.16:0/793745176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07780b53a0 msgr2=0x7f07781572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f0776a00640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07780b53a0 0x7f07781572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f0776a00640 1 -- 172.21.15.16:0/793745176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07781fba80 con 0x7f075c003ab0 2024-09-10T21:13:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f0776a00640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f075c003ab0 0x7f07781fb540 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f076801b550 tx=0x7f0768001c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f076f400640 1 -- 172.21.15.16:0/793745176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768001500 con 0x7f075c003ab0 2024-09-10T21:13:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f076f400640 1 -- 172.21.15.16:0/793745176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07680016a0 con 0x7f075c003ab0 2024-09-10T21:13:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f076f400640 1 -- 172.21.15.16:0/793745176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768006330 con 0x7f075c003ab0 2024-09-10T21:13:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f077de00640 1 -- 172.21.15.16:0/793745176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07780b5940 con 0x7f075c003ab0 2024-09-10T21:13:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f077de00640 1 -- 172.21.15.16:0/793745176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07780b5f00 con 0x7f075c003ab0 2024-09-10T21:13:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f077de00640 1 -- 172.21.15.16:0/793745176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07780d0880 con 0x7f075c003ab0 2024-09-10T21:13:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.836+0000 7f076f400640 1 -- 172.21.15.16:0/793745176 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0768003600 con 0x7f075c003ab0 2024-09-10T21:13:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.840+0000 7f076f400640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0754072610 0x7f0754074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.840+0000 7f076f400640 1 -- 172.21.15.16:0/793745176 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f076812c780 con 0x7f075c003ab0 2024-09-10T21:13:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.840+0000 7f076fe00640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0754072610 0x7f0754074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.840+0000 7f076fe00640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0754072610 0x7f0754074ad0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f07781faae0 tx=0x7f0760006b80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:26.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:26.844+0000 7f076f400640 1 -- 172.21.15.16:0/793745176 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0768016c30 con 0x7f075c003ab0 2024-09-10T21:13:27.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.016+0000 7f077de00640 1 -- 172.21.15.16:0/793745176 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0778158c90 con 0x7f0754072610 2024-09-10T21:13:27.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.020+0000 7f076f400640 1 -- 172.21.15.16:0/793745176 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252056 (secure 0 0 0) 0x7f0778158c90 con 0x7f0754072610 2024-09-10T21:13:27.028 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:27.029 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:13:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.024+0000 7f077de00640 1 -- 172.21.15.16:0/793745176 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0754072610 msgr2=0x7f0754074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.024+0000 7f077de00640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0754072610 0x7f0754074ad0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f07781faae0 tx=0x7f0760006b80 comp rx=0 tx=0).stop 2024-09-10T21:13:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.024+0000 7f077de00640 1 -- 172.21.15.16:0/793745176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f075c003ab0 msgr2=0x7f07781fb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.024+0000 7f077de00640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f075c003ab0 0x7f07781fb540 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f076801b550 tx=0x7f0768001c40 comp rx=0 tx=0).stop 2024-09-10T21:13:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.024+0000 7f077de00640 1 -- 172.21.15.16:0/793745176 shutdown_connections 2024-09-10T21:13:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.024+0000 7f077de00640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0754072610 0x7f0754074ad0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.024+0000 7f077de00640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07780b3ae0 0x7f07780b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.024+0000 7f077de00640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07780b53a0 0x7f07781572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.024+0000 7f077de00640 1 --2- 172.21.15.16:0/793745176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f075c003ab0 0x7f07781fb540 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.024+0000 7f077de00640 1 -- 172.21.15.16:0/793745176 >> 172.21.15.16:0/793745176 conn(0x7f07780d1d80 msgr2=0x7f07780b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.024+0000 7f077de00640 1 -- 172.21.15.16:0/793745176 shutdown_connections 2024-09-10T21:13:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.024+0000 7f077de00640 1 -- 172.21.15.16:0/793745176 wait complete. 2024-09-10T21:13:27.049 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-10T21:13:25.495345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265556,"kb_used_data":7696,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754709164,"statfs":{"total":773094113280,"available":772822183936,"internally_reserved":0,"allocated":7880704,"data_stored":3967057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.623087"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.912119+0000","last_change":"2024-09-10T21:13:22.903796+0000","last_active":"2024-09-10T21:13:22.912119+0000","last_peered":"2024-09-10T21:13:22.912119+0000","last_clean":"2024-09-10T21:13:22.912119+0000","last_became_active":"2024-09-10T21:13:22.903508+0000","last_became_peered":"2024-09-10T21:13:22.903508+0000","last_unstale":"2024-09-10T21:13:22.912119+0000","last_undegraded":"2024-09-10T21:13:22.912119+0000","last_fullsized":"2024-09-10T21:13:22.912119+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:35.266008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922798+0000","last_change":"2024-09-10T21:13:22.920023+0000","last_active":"2024-09-10T21:13:22.922798+0000","last_peered":"2024-09-10T21:13:22.922798+0000","last_clean":"2024-09-10T21:13:22.922798+0000","last_became_active":"2024-09-10T21:13:22.919291+0000","last_became_peered":"2024-09-10T21:13:22.919291+0000","last_unstale":"2024-09-10T21:13:22.922798+0000","last_undegraded":"2024-09-10T21:13:22.922798+0000","last_fullsized":"2024-09-10T21:13:22.922798+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:29.476609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922222+0000","last_change":"2024-09-10T21:13:22.911628+0000","last_active":"2024-09-10T21:13:22.922222+0000","last_peered":"2024-09-10T21:13:22.922222+0000","last_clean":"2024-09-10T21:13:22.922222+0000","last_became_active":"2024-09-10T21:13:22.911281+0000","last_became_peered":"2024-09-10T21:13:22.911281+0000","last_unstale":"2024-09-10T21:13:22.922222+0000","last_undegraded":"2024-09-10T21:13:22.922222+0000","last_fullsized":"2024-09-10T21:13:22.922222+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:07.351230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922257+0000","last_change":"2024-09-10T21:13:22.913645+0000","last_active":"2024-09-10T21:13:22.922257+0000","last_peered":"2024-09-10T21:13:22.922257+0000","last_clean":"2024-09-10T21:13:22.922257+0000","last_became_active":"2024-09-10T21:13:22.913364+0000","last_became_peered":"2024-09-10T21:13:22.913364+0000","last_unstale":"2024-09-10T21:13:22.922257+0000","last_undegraded":"2024-09-10T21:13:22.922257+0000","last_fullsized":"2024-09-10T21:13:22.922257+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:04.798092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901562+0000","last_change":"2024-09-10T21:13:22.913846+0000","last_active":"2024-09-10T21:13:23.901562+0000","last_peered":"2024-09-10T21:13:23.901562+0000","last_clean":"2024-09-10T21:13:23.901562+0000","last_became_active":"2024-09-10T21:13:22.913585+0000","last_became_peered":"2024-09-10T21:13:22.913585+0000","last_unstale":"2024-09-10T21:13:23.901562+0000","last_undegraded":"2024-09-10T21:13:23.901562+0000","last_fullsized":"2024-09-10T21:13:23.901562+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:42.541215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.900917+0000","last_change":"2024-09-10T21:13:22.894948+0000","last_active":"2024-09-10T21:13:23.900917+0000","last_peered":"2024-09-10T21:13:23.900917+0000","last_clean":"2024-09-10T21:13:23.900917+0000","last_became_active":"2024-09-10T21:13:22.894625+0000","last_became_peered":"2024-09-10T21:13:22.894625+0000","last_unstale":"2024-09-10T21:13:23.900917+0000","last_undegraded":"2024-09-10T21:13:23.900917+0000","last_fullsized":"2024-09-10T21:13:23.900917+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:12.944060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918505+0000","last_change":"2024-09-10T21:13:22.912267+0000","last_active":"2024-09-10T21:13:22.918505+0000","last_peered":"2024-09-10T21:13:22.918505+0000","last_clean":"2024-09-10T21:13:22.918505+0000","last_became_active":"2024-09-10T21:13:22.908721+0000","last_became_peered":"2024-09-10T21:13:22.908721+0000","last_unstale":"2024-09-10T21:13:22.918505+0000","last_undegraded":"2024-09-10T21:13:22.918505+0000","last_fullsized":"2024-09-10T21:13:22.918505+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:06.697602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.911388+0000","last_change":"2024-09-10T21:13:22.906168+0000","last_active":"2024-09-10T21:13:22.911388+0000","last_peered":"2024-09-10T21:13:22.911388+0000","last_clean":"2024-09-10T21:13:22.911388+0000","last_became_active":"2024-09-10T21:13:22.905916+0000","last_became_peered":"2024-09-10T21:13:22.905916+0000","last_unstale":"2024-09-10T21:13:22.911388+0000","last_undegraded":"2024-09-10T21:13:22.911388+0000","last_fullsized":"2024-09-10T21:13:22.911388+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:28.915207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922412+0000","last_change":"2024-09-10T21:13:22.917607+0000","last_active":"2024-09-10T21:13:22.922412+0000","last_peered":"2024-09-10T21:13:22.922412+0000","last_clean":"2024-09-10T21:13:22.922412+0000","last_became_active":"2024-09-10T21:13:22.917214+0000","last_became_peered":"2024-09-10T21:13:22.917214+0000","last_unstale":"2024-09-10T21:13:22.922412+0000","last_undegraded":"2024-09-10T21:13:22.922412+0000","last_fullsized":"2024-09-10T21:13:22.922412+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:40.270334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918561+0000","last_change":"2024-09-10T21:13:22.912745+0000","last_active":"2024-09-10T21:13:22.918561+0000","last_peered":"2024-09-10T21:13:22.918561+0000","last_clean":"2024-09-10T21:13:22.918561+0000","last_became_active":"2024-09-10T21:13:22.910798+0000","last_became_peered":"2024-09-10T21:13:22.910798+0000","last_unstale":"2024-09-10T21:13:22.918561+0000","last_undegraded":"2024-09-10T21:13:22.918561+0000","last_fullsized":"2024-09-10T21:13:22.918561+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:14.832743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.905362+0000","last_change":"2024-09-10T21:13:22.899726+0000","last_active":"2024-09-10T21:13:22.905362+0000","last_peered":"2024-09-10T21:13:22.905362+0000","last_clean":"2024-09-10T21:13:22.905362+0000","last_became_active":"2024-09-10T21:13:22.899476+0000","last_became_peered":"2024-09-10T21:13:22.899476+0000","last_unstale":"2024-09-10T21:13:22.905362+0000","last_undegraded":"2024-09-10T21:13:22.905362+0000","last_fullsized":"2024-09-10T21:13:22.905362+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:46.578364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902993+0000","last_change":"2024-09-10T21:13:22.907292+0000","last_active":"2024-09-10T21:13:23.902993+0000","last_peered":"2024-09-10T21:13:23.902993+0000","last_clean":"2024-09-10T21:13:23.902993+0000","last_became_active":"2024-09-10T21:13:22.898081+0000","last_became_peered":"2024-09-10T21:13:22.898081+0000","last_unstale":"2024-09-10T21:13:23.902993+0000","last_undegraded":"2024-09-10T21:13:23.902993+0000","last_fullsized":"2024-09-10T21:13:23.902993+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:16.374896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922347+0000","last_change":"2024-09-10T21:13:22.917376+0000","last_active":"2024-09-10T21:13:22.922347+0000","last_peered":"2024-09-10T21:13:22.922347+0000","last_clean":"2024-09-10T21:13:22.922347+0000","last_became_active":"2024-09-10T21:13:22.917053+0000","last_became_peered":"2024-09-10T21:13:22.917053+0000","last_unstale":"2024-09-10T21:13:22.922347+0000","last_undegraded":"2024-09-10T21:13:22.922347+0000","last_fullsized":"2024-09-10T21:13:22.922347+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:42.649794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.903127+0000","last_change":"2024-09-10T21:13:22.900730+0000","last_active":"2024-09-10T21:13:23.903127+0000","last_peered":"2024-09-10T21:13:23.903127+0000","last_clean":"2024-09-10T21:13:23.903127+0000","last_became_active":"2024-09-10T21:13:22.900385+0000","last_became_peered":"2024-09-10T21:13:22.900385+0000","last_unstale":"2024-09-10T21:13:23.903127+0000","last_undegraded":"2024-09-10T21:13:23.903127+0000","last_fullsized":"2024-09-10T21:13:23.903127+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:25.620749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901431+0000","last_change":"2024-09-10T21:13:22.908969+0000","last_active":"2024-09-10T21:13:23.901431+0000","last_peered":"2024-09-10T21:13:23.901431+0000","last_clean":"2024-09-10T21:13:23.901431+0000","last_became_active":"2024-09-10T21:13:22.908612+0000","last_became_peered":"2024-09-10T21:13:22.908612+0000","last_unstale":"2024-09-10T21:13:23.901431+0000","last_undegraded":"2024-09-10T21:13:23.901431+0000","last_fullsized":"2024-09-10T21:13:23.901431+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:58.136168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922158+0000","last_change":"2024-09-10T21:13:22.920560+0000","last_active":"2024-09-10T21:13:22.922158+0000","last_peered":"2024-09-10T21:13:22.922158+0000","last_clean":"2024-09-10T21:13:22.922158+0000","last_became_active":"2024-09-10T21:13:22.920155+0000","last_became_peered":"2024-09-10T21:13:22.920155+0000","last_unstale":"2024-09-10T21:13:22.922158+0000","last_undegraded":"2024-09-10T21:13:22.922158+0000","last_fullsized":"2024-09-10T21:13:22.922158+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:47.643798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902272+0000","last_change":"2024-09-10T21:13:22.902540+0000","last_active":"2024-09-10T21:13:23.902272+0000","last_peered":"2024-09-10T21:13:23.902272+0000","last_clean":"2024-09-10T21:13:23.902272+0000","last_became_active":"2024-09-10T21:13:22.902169+0000","last_became_peered":"2024-09-10T21:13:22.902169+0000","last_unstale":"2024-09-10T21:13:23.902272+0000","last_undegraded":"2024-09-10T21:13:23.902272+0000","last_fullsized":"2024-09-10T21:13:23.902272+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:36:09.142008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902256+0000","last_change":"2024-09-10T21:13:22.912041+0000","last_active":"2024-09-10T21:13:23.902256+0000","last_peered":"2024-09-10T21:13:23.902256+0000","last_clean":"2024-09-10T21:13:23.902256+0000","last_became_active":"2024-09-10T21:13:22.911678+0000","last_became_peered":"2024-09-10T21:13:22.911678+0000","last_unstale":"2024-09-10T21:13:23.902256+0000","last_undegraded":"2024-09-10T21:13:23.902256+0000","last_fullsized":"2024-09-10T21:13:23.902256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:58.781293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.911673+0000","last_change":"2024-09-10T21:13:22.903186+0000","last_active":"2024-09-10T21:13:22.911673+0000","last_peered":"2024-09-10T21:13:22.911673+0000","last_clean":"2024-09-10T21:13:22.911673+0000","last_became_active":"2024-09-10T21:13:22.902897+0000","last_became_peered":"2024-09-10T21:13:22.902897+0000","last_unstale":"2024-09-10T21:13:22.911673+0000","last_undegraded":"2024-09-10T21:13:22.911673+0000","last_fullsized":"2024-09-10T21:13:22.911673+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:47.545186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901857+0000","last_change":"2024-09-10T21:13:22.893957+0000","last_active":"2024-09-10T21:13:23.901857+0000","last_peered":"2024-09-10T21:13:23.901857+0000","last_clean":"2024-09-10T21:13:23.901857+0000","last_became_active":"2024-09-10T21:13:22.893678+0000","last_became_peered":"2024-09-10T21:13:22.893678+0000","last_unstale":"2024-09-10T21:13:23.901857+0000","last_undegraded":"2024-09-10T21:13:23.901857+0000","last_fullsized":"2024-09-10T21:13:23.901857+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:15.626682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.925059+0000","last_change":"2024-09-10T21:13:22.924024+0000","last_active":"2024-09-10T21:13:22.925059+0000","last_peered":"2024-09-10T21:13:22.925059+0000","last_clean":"2024-09-10T21:13:22.925059+0000","last_became_active":"2024-09-10T21:13:22.923663+0000","last_became_peered":"2024-09-10T21:13:22.923663+0000","last_unstale":"2024-09-10T21:13:22.925059+0000","last_undegraded":"2024-09-10T21:13:22.925059+0000","last_fullsized":"2024-09-10T21:13:22.925059+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:26.813935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.900996+0000","last_change":"2024-09-10T21:13:22.903097+0000","last_active":"2024-09-10T21:13:23.900996+0000","last_peered":"2024-09-10T21:13:23.900996+0000","last_clean":"2024-09-10T21:13:23.900996+0000","last_became_active":"2024-09-10T21:13:22.902716+0000","last_became_peered":"2024-09-10T21:13:22.902716+0000","last_unstale":"2024-09-10T21:13:23.900996+0000","last_undegraded":"2024-09-10T21:13:23.900996+0000","last_fullsized":"2024-09-10T21:13:23.900996+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:28.965753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918526+0000","last_change":"2024-09-10T21:13:22.915676+0000","last_active":"2024-09-10T21:13:22.918526+0000","last_peered":"2024-09-10T21:13:22.918526+0000","last_clean":"2024-09-10T21:13:22.918526+0000","last_became_active":"2024-09-10T21:13:22.915330+0000","last_became_peered":"2024-09-10T21:13:22.915330+0000","last_unstale":"2024-09-10T21:13:22.918526+0000","last_undegraded":"2024-09-10T21:13:22.918526+0000","last_fullsized":"2024-09-10T21:13:22.918526+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:52.872636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.897629+0000","last_change":"2024-09-10T21:13:22.893235+0000","last_active":"2024-09-10T21:13:22.897629+0000","last_peered":"2024-09-10T21:13:22.897629+0000","last_clean":"2024-09-10T21:13:22.897629+0000","last_became_active":"2024-09-10T21:13:22.892180+0000","last_became_peered":"2024-09-10T21:13:22.892180+0000","last_unstale":"2024-09-10T21:13:22.897629+0000","last_undegraded":"2024-09-10T21:13:22.897629+0000","last_fullsized":"2024-09-10T21:13:22.897629+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:20.849337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.923422+0000","last_change":"2024-09-10T21:13:22.921263+0000","last_active":"2024-09-10T21:13:22.923422+0000","last_peered":"2024-09-10T21:13:22.923422+0000","last_clean":"2024-09-10T21:13:22.923422+0000","last_became_active":"2024-09-10T21:13:22.920957+0000","last_became_peered":"2024-09-10T21:13:22.920957+0000","last_unstale":"2024-09-10T21:13:22.923422+0000","last_undegraded":"2024-09-10T21:13:22.923422+0000","last_fullsized":"2024-09-10T21:13:22.923422+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:35.263152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.902365+0000","last_change":"2024-09-10T21:13:22.897818+0000","last_active":"2024-09-10T21:13:22.902365+0000","last_peered":"2024-09-10T21:13:22.902365+0000","last_clean":"2024-09-10T21:13:22.902365+0000","last_became_active":"2024-09-10T21:13:22.897548+0000","last_became_peered":"2024-09-10T21:13:22.897548+0000","last_unstale":"2024-09-10T21:13:22.902365+0000","last_undegraded":"2024-09-10T21:13:22.902365+0000","last_fullsized":"2024-09-10T21:13:22.902365+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:29.463517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901931+0000","last_change":"2024-09-10T21:13:22.909907+0000","last_active":"2024-09-10T21:13:23.901931+0000","last_peered":"2024-09-10T21:13:23.901931+0000","last_clean":"2024-09-10T21:13:23.901931+0000","last_became_active":"2024-09-10T21:13:22.909634+0000","last_became_peered":"2024-09-10T21:13:22.909634+0000","last_unstale":"2024-09-10T21:13:23.901931+0000","last_undegraded":"2024-09-10T21:13:23.901931+0000","last_fullsized":"2024-09-10T21:13:23.901931+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:08.408152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922097+0000","last_change":"2024-09-10T21:13:22.918658+0000","last_active":"2024-09-10T21:13:22.922097+0000","last_peered":"2024-09-10T21:13:22.922097+0000","last_clean":"2024-09-10T21:13:22.922097+0000","last_became_active":"2024-09-10T21:13:22.918275+0000","last_became_peered":"2024-09-10T21:13:22.918275+0000","last_unstale":"2024-09-10T21:13:22.922097+0000","last_undegraded":"2024-09-10T21:13:22.922097+0000","last_fullsized":"2024-09-10T21:13:22.922097+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:26.044069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.921633+0000","last_change":"2024-09-10T21:13:22.908668+0000","last_active":"2024-09-10T21:13:22.921633+0000","last_peered":"2024-09-10T21:13:22.921633+0000","last_clean":"2024-09-10T21:13:22.921633+0000","last_became_active":"2024-09-10T21:13:22.907915+0000","last_became_peered":"2024-09-10T21:13:22.907915+0000","last_unstale":"2024-09-10T21:13:22.921633+0000","last_undegraded":"2024-09-10T21:13:22.921633+0000","last_fullsized":"2024-09-10T21:13:22.921633+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:21.667515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.911191+0000","last_change":"2024-09-10T21:13:22.903576+0000","last_active":"2024-09-10T21:13:22.911191+0000","last_peered":"2024-09-10T21:13:22.911191+0000","last_clean":"2024-09-10T21:13:22.911191+0000","last_became_active":"2024-09-10T21:13:22.903261+0000","last_became_peered":"2024-09-10T21:13:22.903261+0000","last_unstale":"2024-09-10T21:13:22.911191+0000","last_undegraded":"2024-09-10T21:13:22.911191+0000","last_fullsized":"2024-09-10T21:13:22.911191+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:39.122262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922023+0000","last_change":"2024-09-10T21:13:22.918311+0000","last_active":"2024-09-10T21:13:22.922023+0000","last_peered":"2024-09-10T21:13:22.922023+0000","last_clean":"2024-09-10T21:13:22.922023+0000","last_became_active":"2024-09-10T21:13:22.918038+0000","last_became_peered":"2024-09-10T21:13:22.918038+0000","last_unstale":"2024-09-10T21:13:22.922023+0000","last_undegraded":"2024-09-10T21:13:22.922023+0000","last_fullsized":"2024-09-10T21:13:22.922023+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:38:27.645205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922123+0000","last_change":"2024-09-10T21:13:22.918559+0000","last_active":"2024-09-10T21:13:22.922123+0000","last_peered":"2024-09-10T21:13:22.922123+0000","last_clean":"2024-09-10T21:13:22.922123+0000","last_became_active":"2024-09-10T21:13:22.918289+0000","last_became_peered":"2024-09-10T21:13:22.918289+0000","last_unstale":"2024-09-10T21:13:22.922123+0000","last_undegraded":"2024-09-10T21:13:22.922123+0000","last_fullsized":"2024-09-10T21:13:22.922123+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:11.743269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918068+0000","last_change":"2024-09-10T21:13:22.916148+0000","last_active":"2024-09-10T21:13:22.918068+0000","last_peered":"2024-09-10T21:13:22.918068+0000","last_clean":"2024-09-10T21:13:22.918068+0000","last_became_active":"2024-09-10T21:13:22.912870+0000","last_became_peered":"2024-09-10T21:13:22.912870+0000","last_unstale":"2024-09-10T21:13:22.918068+0000","last_undegraded":"2024-09-10T21:13:22.918068+0000","last_fullsized":"2024-09-10T21:13:22.918068+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:23:44.907538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901208+0000","last_change":"2024-09-10T21:13:21.891223+0000","last_active":"2024-09-10T21:13:23.901208+0000","last_peered":"2024-09-10T21:13:23.901208+0000","last_clean":"2024-09-10T21:13:23.901208+0000","last_became_active":"2024-09-10T21:13:21.890880+0000","last_became_peered":"2024-09-10T21:13:21.890880+0000","last_unstale":"2024-09-10T21:13:23.901208+0000","last_undegraded":"2024-09-10T21:13:23.901208+0000","last_fullsized":"2024-09-10T21:13:23.901208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:28.473861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.920982+0000","last_change":"2024-09-10T21:13:22.910860+0000","last_active":"2024-09-10T21:13:22.920982+0000","last_peered":"2024-09-10T21:13:22.920982+0000","last_clean":"2024-09-10T21:13:22.920982+0000","last_became_active":"2024-09-10T21:13:22.910569+0000","last_became_peered":"2024-09-10T21:13:22.910569+0000","last_unstale":"2024-09-10T21:13:22.920982+0000","last_undegraded":"2024-09-10T21:13:22.920982+0000","last_fullsized":"2024-09-10T21:13:22.920982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:10.231854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902611+0000","last_change":"2024-09-10T21:13:21.890910+0000","last_active":"2024-09-10T21:13:23.902611+0000","last_peered":"2024-09-10T21:13:23.902611+0000","last_clean":"2024-09-10T21:13:23.902611+0000","last_became_active":"2024-09-10T21:13:21.890621+0000","last_became_peered":"2024-09-10T21:13:21.890621+0000","last_unstale":"2024-09-10T21:13:23.902611+0000","last_undegraded":"2024-09-10T21:13:23.902611+0000","last_fullsized":"2024-09-10T21:13:23.902611+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:57.940612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901137+0000","last_change":"2024-09-10T21:13:22.894314+0000","last_active":"2024-09-10T21:13:23.901137+0000","last_peered":"2024-09-10T21:13:23.901137+0000","last_clean":"2024-09-10T21:13:23.901137+0000","last_became_active":"2024-09-10T21:13:22.894039+0000","last_became_peered":"2024-09-10T21:13:22.894039+0000","last_unstale":"2024-09-10T21:13:23.901137+0000","last_undegraded":"2024-09-10T21:13:23.901137+0000","last_fullsized":"2024-09-10T21:13:23.901137+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:33.037742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.912016+0000","last_change":"2024-09-10T21:13:21.911067+0000","last_active":"2024-09-10T21:13:22.912016+0000","last_peered":"2024-09-10T21:13:22.912016+0000","last_clean":"2024-09-10T21:13:22.912016+0000","last_became_active":"2024-09-10T21:13:21.910626+0000","last_became_peered":"2024-09-10T21:13:21.910626+0000","last_unstale":"2024-09-10T21:13:22.912016+0000","last_undegraded":"2024-09-10T21:13:22.912016+0000","last_fullsized":"2024-09-10T21:13:22.912016+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:37.367655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918469+0000","last_change":"2024-09-10T21:13:22.911184+0000","last_active":"2024-09-10T21:13:22.918469+0000","last_peered":"2024-09-10T21:13:22.918469+0000","last_clean":"2024-09-10T21:13:22.918469+0000","last_became_active":"2024-09-10T21:13:22.910849+0000","last_became_peered":"2024-09-10T21:13:22.910849+0000","last_unstale":"2024-09-10T21:13:22.918469+0000","last_undegraded":"2024-09-10T21:13:22.918469+0000","last_fullsized":"2024-09-10T21:13:22.918469+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:14.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901073+0000","last_change":"2024-09-10T21:13:21.903374+0000","last_active":"2024-09-10T21:13:23.901073+0000","last_peered":"2024-09-10T21:13:23.901073+0000","last_clean":"2024-09-10T21:13:23.901073+0000","last_became_active":"2024-09-10T21:13:21.903062+0000","last_became_peered":"2024-09-10T21:13:21.903062+0000","last_unstale":"2024-09-10T21:13:23.901073+0000","last_undegraded":"2024-09-10T21:13:23.901073+0000","last_fullsized":"2024-09-10T21:13:23.901073+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:25.041321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.911347+0000","last_change":"2024-09-10T21:13:22.906151+0000","last_active":"2024-09-10T21:13:22.911347+0000","last_peered":"2024-09-10T21:13:22.911347+0000","last_clean":"2024-09-10T21:13:22.911347+0000","last_became_active":"2024-09-10T21:13:22.905888+0000","last_became_peered":"2024-09-10T21:13:22.905888+0000","last_unstale":"2024-09-10T21:13:22.911347+0000","last_undegraded":"2024-09-10T21:13:22.911347+0000","last_fullsized":"2024-09-10T21:13:22.911347+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:16.225082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901882+0000","last_change":"2024-09-10T21:13:21.895415+0000","last_active":"2024-09-10T21:13:23.901882+0000","last_peered":"2024-09-10T21:13:23.901882+0000","last_clean":"2024-09-10T21:13:23.901882+0000","last_became_active":"2024-09-10T21:13:21.894307+0000","last_became_peered":"2024-09-10T21:13:21.894307+0000","last_unstale":"2024-09-10T21:13:23.901882+0000","last_undegraded":"2024-09-10T21:13:23.901882+0000","last_fullsized":"2024-09-10T21:13:23.901882+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:14.392919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901290+0000","last_change":"2024-09-10T21:13:22.894260+0000","last_active":"2024-09-10T21:13:23.901290+0000","last_peered":"2024-09-10T21:13:23.901290+0000","last_clean":"2024-09-10T21:13:23.901290+0000","last_became_active":"2024-09-10T21:13:22.893955+0000","last_became_peered":"2024-09-10T21:13:22.893955+0000","last_unstale":"2024-09-10T21:13:23.901290+0000","last_undegraded":"2024-09-10T21:13:23.901290+0000","last_fullsized":"2024-09-10T21:13:23.901290+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:49.018588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.916134+0000","last_change":"2024-09-10T21:13:21.908654+0000","last_active":"2024-09-10T21:13:22.916134+0000","last_peered":"2024-09-10T21:13:22.916134+0000","last_clean":"2024-09-10T21:13:22.916134+0000","last_became_active":"2024-09-10T21:13:21.908331+0000","last_became_peered":"2024-09-10T21:13:21.908331+0000","last_unstale":"2024-09-10T21:13:22.916134+0000","last_undegraded":"2024-09-10T21:13:22.916134+0000","last_fullsized":"2024-09-10T21:13:22.916134+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:41.647046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.923539+0000","last_change":"2024-09-10T21:13:22.920840+0000","last_active":"2024-09-10T21:13:22.923539+0000","last_peered":"2024-09-10T21:13:22.923539+0000","last_clean":"2024-09-10T21:13:22.923539+0000","last_became_active":"2024-09-10T21:13:22.917562+0000","last_became_peered":"2024-09-10T21:13:22.917562+0000","last_unstale":"2024-09-10T21:13:22.923539+0000","last_undegraded":"2024-09-10T21:13:22.923539+0000","last_fullsized":"2024-09-10T21:13:22.923539+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:15.395667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.895394+0000","last_change":"2024-09-10T21:13:21.909382+0000","last_active":"2024-09-10T21:13:22.895394+0000","last_peered":"2024-09-10T21:13:22.895394+0000","last_clean":"2024-09-10T21:13:22.895394+0000","last_became_active":"2024-09-10T21:13:21.909059+0000","last_became_peered":"2024-09-10T21:13:21.909059+0000","last_unstale":"2024-09-10T21:13:22.895394+0000","last_undegraded":"2024-09-10T21:13:22.895394+0000","last_fullsized":"2024-09-10T21:13:22.895394+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:53.484906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902253+0000","last_change":"2024-09-10T21:13:22.908636+0000","last_active":"2024-09-10T21:13:23.902253+0000","last_peered":"2024-09-10T21:13:23.902253+0000","last_clean":"2024-09-10T21:13:23.902253+0000","last_became_active":"2024-09-10T21:13:22.908223+0000","last_became_peered":"2024-09-10T21:13:22.908223+0000","last_unstale":"2024-09-10T21:13:23.902253+0000","last_undegraded":"2024-09-10T21:13:23.902253+0000","last_fullsized":"2024-09-10T21:13:23.902253+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:08.036388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901033+0000","last_change":"2024-09-10T21:13:21.890375+0000","last_active":"2024-09-10T21:13:23.901033+0000","last_peered":"2024-09-10T21:13:23.901033+0000","last_clean":"2024-09-10T21:13:23.901033+0000","last_became_active":"2024-09-10T21:13:21.890028+0000","last_became_peered":"2024-09-10T21:13:21.890028+0000","last_unstale":"2024-09-10T21:13:23.901033+0000","last_undegraded":"2024-09-10T21:13:23.901033+0000","last_fullsized":"2024-09-10T21:13:23.901033+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:41.970949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918658+0000","last_change":"2024-09-10T21:13:22.914964+0000","last_active":"2024-09-10T21:13:22.918658+0000","last_peered":"2024-09-10T21:13:22.918658+0000","last_clean":"2024-09-10T21:13:22.918658+0000","last_became_active":"2024-09-10T21:13:22.914421+0000","last_became_peered":"2024-09-10T21:13:22.914421+0000","last_unstale":"2024-09-10T21:13:22.918658+0000","last_undegraded":"2024-09-10T21:13:22.918658+0000","last_fullsized":"2024-09-10T21:13:22.918658+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:56.608777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.888549+0000","last_change":"2024-09-10T21:13:21.900619+0000","last_active":"2024-09-10T21:13:22.888549+0000","last_peered":"2024-09-10T21:13:22.888549+0000","last_clean":"2024-09-10T21:13:22.888549+0000","last_became_active":"2024-09-10T21:13:21.900230+0000","last_became_peered":"2024-09-10T21:13:21.900230+0000","last_unstale":"2024-09-10T21:13:22.888549+0000","last_undegraded":"2024-09-10T21:13:22.888549+0000","last_fullsized":"2024-09-10T21:13:22.888549+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:59.280286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922816+0000","last_change":"2024-09-10T21:13:22.919434+0000","last_active":"2024-09-10T21:13:22.922816+0000","last_peered":"2024-09-10T21:13:22.922816+0000","last_clean":"2024-09-10T21:13:22.922816+0000","last_became_active":"2024-09-10T21:13:22.919119+0000","last_became_peered":"2024-09-10T21:13:22.919119+0000","last_unstale":"2024-09-10T21:13:22.922816+0000","last_undegraded":"2024-09-10T21:13:22.922816+0000","last_fullsized":"2024-09-10T21:13:22.922816+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:01.378449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.907283+0000","last_change":"2024-09-10T21:13:21.910037+0000","last_active":"2024-09-10T21:13:22.907283+0000","last_peered":"2024-09-10T21:13:22.907283+0000","last_clean":"2024-09-10T21:13:22.907283+0000","last_became_active":"2024-09-10T21:13:21.906750+0000","last_became_peered":"2024-09-10T21:13:21.906750+0000","last_unstale":"2024-09-10T21:13:22.907283+0000","last_undegraded":"2024-09-10T21:13:22.907283+0000","last_fullsized":"2024-09-10T21:13:22.907283+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:53.419904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.903943+0000","last_change":"2024-09-10T21:13:22.900722+0000","last_active":"2024-09-10T21:13:22.903943+0000","last_peered":"2024-09-10T21:13:22.903943+0000","last_clean":"2024-09-10T21:13:22.903943+0000","last_became_active":"2024-09-10T21:13:22.900463+0000","last_became_peered":"2024-09-10T21:13:22.900463+0000","last_unstale":"2024-09-10T21:13:22.903943+0000","last_undegraded":"2024-09-10T21:13:22.903943+0000","last_fullsized":"2024-09-10T21:13:22.903943+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:50.015582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902319+0000","last_change":"2024-09-10T21:13:21.893258+0000","last_active":"2024-09-10T21:13:23.902319+0000","last_peered":"2024-09-10T21:13:23.902319+0000","last_clean":"2024-09-10T21:13:23.902319+0000","last_became_active":"2024-09-10T21:13:21.892943+0000","last_became_peered":"2024-09-10T21:13:21.892943+0000","last_unstale":"2024-09-10T21:13:23.902319+0000","last_undegraded":"2024-09-10T21:13:23.902319+0000","last_fullsized":"2024-09-10T21:13:23.902319+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:46.844309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902312+0000","last_change":"2024-09-10T21:13:22.909927+0000","last_active":"2024-09-10T21:13:23.902312+0000","last_peered":"2024-09-10T21:13:23.902312+0000","last_clean":"2024-09-10T21:13:23.902312+0000","last_became_active":"2024-09-10T21:13:22.909504+0000","last_became_peered":"2024-09-10T21:13:22.909504+0000","last_unstale":"2024-09-10T21:13:23.902312+0000","last_undegraded":"2024-09-10T21:13:23.902312+0000","last_fullsized":"2024-09-10T21:13:23.902312+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:36:59.591146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"69'6","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.925300+0000","last_change":"2024-09-10T21:13:21.901366+0000","last_active":"2024-09-10T21:13:22.925300+0000","last_peered":"2024-09-10T21:13:22.925300+0000","last_clean":"2024-09-10T21:13:22.925300+0000","last_became_active":"2024-09-10T21:13:21.901017+0000","last_became_peered":"2024-09-10T21:13:21.901017+0000","last_unstale":"2024-09-10T21:13:22.925300+0000","last_undegraded":"2024-09-10T21:13:22.925300+0000","last_fullsized":"2024-09-10T21:13:22.925300+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:46.707163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901196+0000","last_change":"2024-09-10T21:13:22.904653+0000","last_active":"2024-09-10T21:13:23.901196+0000","last_peered":"2024-09-10T21:13:23.901196+0000","last_clean":"2024-09-10T21:13:23.901196+0000","last_became_active":"2024-09-10T21:13:22.904344+0000","last_became_peered":"2024-09-10T21:13:22.904344+0000","last_unstale":"2024-09-10T21:13:23.901196+0000","last_undegraded":"2024-09-10T21:13:23.901196+0000","last_fullsized":"2024-09-10T21:13:23.901196+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:57.783475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"69'2","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.912129+0000","last_change":"2024-09-10T21:13:21.901952+0000","last_active":"2024-09-10T21:13:22.912129+0000","last_peered":"2024-09-10T21:13:22.912129+0000","last_clean":"2024-09-10T21:13:22.912129+0000","last_became_active":"2024-09-10T21:13:21.901478+0000","last_became_peered":"2024-09-10T21:13:21.901478+0000","last_unstale":"2024-09-10T21:13:22.912129+0000","last_undegraded":"2024-09-10T21:13:22.912129+0000","last_fullsized":"2024-09-10T21:13:22.912129+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:30.021715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.908596+0000","last_change":"2024-09-10T21:13:22.898289+0000","last_active":"2024-09-10T21:13:22.908596+0000","last_peered":"2024-09-10T21:13:22.908596+0000","last_clean":"2024-09-10T21:13:22.908596+0000","last_became_active":"2024-09-10T21:13:22.897979+0000","last_became_peered":"2024-09-10T21:13:22.897979+0000","last_unstale":"2024-09-10T21:13:22.908596+0000","last_undegraded":"2024-09-10T21:13:22.908596+0000","last_fullsized":"2024-09-10T21:13:22.908596+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:10.730899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.892289+0000","last_change":"2024-09-10T21:13:21.904214+0000","last_active":"2024-09-10T21:13:22.892289+0000","last_peered":"2024-09-10T21:13:22.892289+0000","last_clean":"2024-09-10T21:13:22.892289+0000","last_became_active":"2024-09-10T21:13:21.903872+0000","last_became_peered":"2024-09-10T21:13:21.903872+0000","last_unstale":"2024-09-10T21:13:22.892289+0000","last_undegraded":"2024-09-10T21:13:22.892289+0000","last_fullsized":"2024-09-10T21:13:22.892289+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:03.795344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901444+0000","last_change":"2024-09-10T21:13:22.905275+0000","last_active":"2024-09-10T21:13:23.901444+0000","last_peered":"2024-09-10T21:13:23.901444+0000","last_clean":"2024-09-10T21:13:23.901444+0000","last_became_active":"2024-09-10T21:13:22.902236+0000","last_became_peered":"2024-09-10T21:13:22.902236+0000","last_unstale":"2024-09-10T21:13:23.901444+0000","last_undegraded":"2024-09-10T21:13:23.901444+0000","last_fullsized":"2024-09-10T21:13:23.901444+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:05.930480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.890219+0000","last_change":"2024-09-10T21:13:21.906146+0000","last_active":"2024-09-10T21:13:22.890219+0000","last_peered":"2024-09-10T21:13:22.890219+0000","last_clean":"2024-09-10T21:13:22.890219+0000","last_became_active":"2024-09-10T21:13:21.905879+0000","last_became_peered":"2024-09-10T21:13:21.905879+0000","last_unstale":"2024-09-10T21:13:22.890219+0000","last_undegraded":"2024-09-10T21:13:22.890219+0000","last_fullsized":"2024-09-10T21:13:22.890219+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:20.664766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.907318+0000","last_change":"2024-09-10T21:13:22.903404+0000","last_active":"2024-09-10T21:13:22.907318+0000","last_peered":"2024-09-10T21:13:22.907318+0000","last_clean":"2024-09-10T21:13:22.907318+0000","last_became_active":"2024-09-10T21:13:22.897729+0000","last_became_peered":"2024-09-10T21:13:22.897729+0000","last_unstale":"2024-09-10T21:13:22.907318+0000","last_undegraded":"2024-09-10T21:13:22.907318+0000","last_fullsized":"2024-09-10T21:13:22.907318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:18.592973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.916904+0000","last_change":"2024-09-10T21:13:21.900487+0000","last_active":"2024-09-10T21:13:22.916904+0000","last_peered":"2024-09-10T21:13:22.916904+0000","last_clean":"2024-09-10T21:13:22.916904+0000","last_became_active":"2024-09-10T21:13:21.900220+0000","last_became_peered":"2024-09-10T21:13:21.900220+0000","last_unstale":"2024-09-10T21:13:22.916904+0000","last_undegraded":"2024-09-10T21:13:22.916904+0000","last_fullsized":"2024-09-10T21:13:22.916904+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:06.348482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.920226+0000","last_change":"2024-09-10T21:13:22.910988+0000","last_active":"2024-09-10T21:13:22.920226+0000","last_peered":"2024-09-10T21:13:22.920226+0000","last_clean":"2024-09-10T21:13:22.920226+0000","last_became_active":"2024-09-10T21:13:22.910623+0000","last_became_peered":"2024-09-10T21:13:22.910623+0000","last_unstale":"2024-09-10T21:13:22.920226+0000","last_undegraded":"2024-09-10T21:13:22.920226+0000","last_fullsized":"2024-09-10T21:13:22.920226+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:18.535836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.921722+0000","last_change":"2024-09-10T21:13:21.899168+0000","last_active":"2024-09-10T21:13:22.921722+0000","last_peered":"2024-09-10T21:13:22.921722+0000","last_clean":"2024-09-10T21:13:22.921722+0000","last_became_active":"2024-09-10T21:13:21.898731+0000","last_became_peered":"2024-09-10T21:13:21.898731+0000","last_unstale":"2024-09-10T21:13:22.921722+0000","last_undegraded":"2024-09-10T21:13:22.921722+0000","last_fullsized":"2024-09-10T21:13:22.921722+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:21.312897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.923933+0000","last_change":"2024-09-10T21:13:22.922332+0000","last_active":"2024-09-10T21:13:22.923933+0000","last_peered":"2024-09-10T21:13:22.923933+0000","last_clean":"2024-09-10T21:13:22.923933+0000","last_became_active":"2024-09-10T21:13:22.922078+0000","last_became_peered":"2024-09-10T21:13:22.922078+0000","last_unstale":"2024-09-10T21:13:22.923933+0000","last_undegraded":"2024-09-10T21:13:22.923933+0000","last_fullsized":"2024-09-10T21:13:22.923933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:28.421242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.919719+0000","last_change":"2024-09-10T21:13:21.912484+0000","last_active":"2024-09-10T21:13:22.919719+0000","last_peered":"2024-09-10T21:13:22.919719+0000","last_clean":"2024-09-10T21:13:22.919719+0000","last_became_active":"2024-09-10T21:13:21.912214+0000","last_became_peered":"2024-09-10T21:13:21.912214+0000","last_unstale":"2024-09-10T21:13:22.919719+0000","last_undegraded":"2024-09-10T21:13:22.919719+0000","last_fullsized":"2024-09-10T21:13:22.919719+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:41.828958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922496+0000","last_change":"2024-09-10T21:13:22.919296+0000","last_active":"2024-09-10T21:13:22.922496+0000","last_peered":"2024-09-10T21:13:22.922496+0000","last_clean":"2024-09-10T21:13:22.922496+0000","last_became_active":"2024-09-10T21:13:22.919014+0000","last_became_peered":"2024-09-10T21:13:22.919014+0000","last_unstale":"2024-09-10T21:13:22.922496+0000","last_undegraded":"2024-09-10T21:13:22.922496+0000","last_fullsized":"2024-09-10T21:13:22.922496+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:10.730899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.915756+0000","last_change":"2024-09-10T21:13:21.906072+0000","last_active":"2024-09-10T21:13:22.915756+0000","last_peered":"2024-09-10T21:13:22.915756+0000","last_clean":"2024-09-10T21:13:22.915756+0000","last_became_active":"2024-09-10T21:13:21.905742+0000","last_became_peered":"2024-09-10T21:13:21.905742+0000","last_unstale":"2024-09-10T21:13:22.915756+0000","last_undegraded":"2024-09-10T21:13:22.915756+0000","last_fullsized":"2024-09-10T21:13:22.915756+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:42:26.027622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.910563+0000","last_change":"2024-09-10T21:13:22.903478+0000","last_active":"2024-09-10T21:13:22.910563+0000","last_peered":"2024-09-10T21:13:22.910563+0000","last_clean":"2024-09-10T21:13:22.910563+0000","last_became_active":"2024-09-10T21:13:22.903195+0000","last_became_peered":"2024-09-10T21:13:22.903195+0000","last_unstale":"2024-09-10T21:13:22.910563+0000","last_undegraded":"2024-09-10T21:13:22.910563+0000","last_fullsized":"2024-09-10T21:13:22.910563+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:08.970391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901193+0000","last_change":"2024-09-10T21:13:21.899616+0000","last_active":"2024-09-10T21:13:23.901193+0000","last_peered":"2024-09-10T21:13:23.901193+0000","last_clean":"2024-09-10T21:13:23.901193+0000","last_became_active":"2024-09-10T21:13:21.899337+0000","last_became_peered":"2024-09-10T21:13:21.899337+0000","last_unstale":"2024-09-10T21:13:23.901193+0000","last_undegraded":"2024-09-10T21:13:23.901193+0000","last_fullsized":"2024-09-10T21:13:23.901193+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:00.375700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.921826+0000","last_change":"2024-09-10T21:13:22.918519+0000","last_active":"2024-09-10T21:13:22.921826+0000","last_peered":"2024-09-10T21:13:22.921826+0000","last_clean":"2024-09-10T21:13:22.921826+0000","last_became_active":"2024-09-10T21:13:22.917932+0000","last_became_peered":"2024-09-10T21:13:22.917932+0000","last_unstale":"2024-09-10T21:13:22.921826+0000","last_undegraded":"2024-09-10T21:13:22.921826+0000","last_fullsized":"2024-09-10T21:13:22.921826+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:42.973697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901177+0000","last_change":"2024-09-10T21:13:21.888372+0000","last_active":"2024-09-10T21:13:23.901177+0000","last_peered":"2024-09-10T21:13:23.901177+0000","last_clean":"2024-09-10T21:13:23.901177+0000","last_became_active":"2024-09-10T21:13:21.888082+0000","last_became_peered":"2024-09-10T21:13:21.888082+0000","last_unstale":"2024-09-10T21:13:23.901177+0000","last_undegraded":"2024-09-10T21:13:23.901177+0000","last_fullsized":"2024-09-10T21:13:23.901177+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:10.740521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922244+0000","last_change":"2024-09-10T21:13:22.916504+0000","last_active":"2024-09-10T21:13:22.922244+0000","last_peered":"2024-09-10T21:13:22.922244+0000","last_clean":"2024-09-10T21:13:22.922244+0000","last_became_active":"2024-09-10T21:13:22.915699+0000","last_became_peered":"2024-09-10T21:13:22.915699+0000","last_unstale":"2024-09-10T21:13:22.922244+0000","last_undegraded":"2024-09-10T21:13:22.922244+0000","last_fullsized":"2024-09-10T21:13:22.922244+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:25.585671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.905804+0000","last_change":"2024-09-10T21:13:21.907164+0000","last_active":"2024-09-10T21:13:22.905804+0000","last_peered":"2024-09-10T21:13:22.905804+0000","last_clean":"2024-09-10T21:13:22.905804+0000","last_became_active":"2024-09-10T21:13:21.906610+0000","last_became_peered":"2024-09-10T21:13:21.906610+0000","last_unstale":"2024-09-10T21:13:22.905804+0000","last_undegraded":"2024-09-10T21:13:22.905804+0000","last_fullsized":"2024-09-10T21:13:22.905804+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:40.113319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902480+0000","last_change":"2024-09-10T21:13:22.898945+0000","last_active":"2024-09-10T21:13:23.902480+0000","last_peered":"2024-09-10T21:13:23.902480+0000","last_clean":"2024-09-10T21:13:23.902480+0000","last_became_active":"2024-09-10T21:13:22.898052+0000","last_became_peered":"2024-09-10T21:13:22.898052+0000","last_unstale":"2024-09-10T21:13:23.902480+0000","last_undegraded":"2024-09-10T21:13:23.902480+0000","last_fullsized":"2024-09-10T21:13:23.902480+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:55.405705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901250+0000","last_change":"2024-09-10T21:13:21.906141+0000","last_active":"2024-09-10T21:13:23.901250+0000","last_peered":"2024-09-10T21:13:23.901250+0000","last_clean":"2024-09-10T21:13:23.901250+0000","last_became_active":"2024-09-10T21:13:21.905746+0000","last_became_peered":"2024-09-10T21:13:21.905746+0000","last_unstale":"2024-09-10T21:13:23.901250+0000","last_undegraded":"2024-09-10T21:13:23.901250+0000","last_fullsized":"2024-09-10T21:13:23.901250+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:43.092284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902491+0000","last_change":"2024-09-10T21:13:22.916766+0000","last_active":"2024-09-10T21:13:23.902491+0000","last_peered":"2024-09-10T21:13:23.902491+0000","last_clean":"2024-09-10T21:13:23.902491+0000","last_became_active":"2024-09-10T21:13:22.916442+0000","last_became_peered":"2024-09-10T21:13:22.916442+0000","last_unstale":"2024-09-10T21:13:23.902491+0000","last_undegraded":"2024-09-10T21:13:23.902491+0000","last_fullsized":"2024-09-10T21:13:23.902491+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:36.488990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.921831+0000","last_change":"2024-09-10T21:13:21.903728+0000","last_active":"2024-09-10T21:13:22.921831+0000","last_peered":"2024-09-10T21:13:22.921831+0000","last_clean":"2024-09-10T21:13:22.921831+0000","last_became_active":"2024-09-10T21:13:21.902934+0000","last_became_peered":"2024-09-10T21:13:21.902934+0000","last_unstale":"2024-09-10T21:13:22.921831+0000","last_undegraded":"2024-09-10T21:13:22.921831+0000","last_fullsized":"2024-09-10T21:13:22.921831+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:47.969001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.923398+0000","last_change":"2024-09-10T21:13:22.919284+0000","last_active":"2024-09-10T21:13:22.923398+0000","last_peered":"2024-09-10T21:13:22.923398+0000","last_clean":"2024-09-10T21:13:22.923398+0000","last_became_active":"2024-09-10T21:13:22.918999+0000","last_became_peered":"2024-09-10T21:13:22.918999+0000","last_unstale":"2024-09-10T21:13:22.923398+0000","last_undegraded":"2024-09-10T21:13:22.923398+0000","last_fullsized":"2024-09-10T21:13:22.923398+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:12.826890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.889810+0000","last_change":"2024-09-10T21:13:21.908766+0000","last_active":"2024-09-10T21:13:23.889810+0000","last_peered":"2024-09-10T21:13:23.889810+0000","last_clean":"2024-09-10T21:13:23.889810+0000","last_became_active":"2024-09-10T21:13:21.908407+0000","last_became_peered":"2024-09-10T21:13:21.908407+0000","last_unstale":"2024-09-10T21:13:23.889810+0000","last_undegraded":"2024-09-10T21:13:23.889810+0000","last_fullsized":"2024-09-10T21:13:23.889810+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:50.408040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.915998+0000","last_change":"2024-09-10T21:13:22.912010+0000","last_active":"2024-09-10T21:13:22.915998+0000","last_peered":"2024-09-10T21:13:22.915998+0000","last_clean":"2024-09-10T21:13:22.915998+0000","last_became_active":"2024-09-10T21:13:22.911626+0000","last_became_peered":"2024-09-10T21:13:22.911626+0000","last_unstale":"2024-09-10T21:13:22.915998+0000","last_undegraded":"2024-09-10T21:13:22.915998+0000","last_fullsized":"2024-09-10T21:13:22.915998+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:57.866030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902425+0000","last_change":"2024-09-10T21:13:21.900741+0000","last_active":"2024-09-10T21:13:23.902425+0000","last_peered":"2024-09-10T21:13:23.902425+0000","last_clean":"2024-09-10T21:13:23.902425+0000","last_became_active":"2024-09-10T21:13:21.900316+0000","last_became_peered":"2024-09-10T21:13:21.900316+0000","last_unstale":"2024-09-10T21:13:23.902425+0000","last_undegraded":"2024-09-10T21:13:23.902425+0000","last_fullsized":"2024-09-10T21:13:23.902425+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:36.947920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.906343+0000","last_change":"2024-09-10T21:13:22.895387+0000","last_active":"2024-09-10T21:13:22.906343+0000","last_peered":"2024-09-10T21:13:22.906343+0000","last_clean":"2024-09-10T21:13:22.906343+0000","last_became_active":"2024-09-10T21:13:22.895123+0000","last_became_peered":"2024-09-10T21:13:22.895123+0000","last_unstale":"2024-09-10T21:13:22.906343+0000","last_undegraded":"2024-09-10T21:13:22.906343+0000","last_fullsized":"2024-09-10T21:13:22.906343+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:40.270334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902958+0000","last_change":"2024-09-10T21:13:21.893643+0000","last_active":"2024-09-10T21:13:23.902958+0000","last_peered":"2024-09-10T21:13:23.902958+0000","last_clean":"2024-09-10T21:13:23.902958+0000","last_became_active":"2024-09-10T21:13:21.893383+0000","last_became_peered":"2024-09-10T21:13:21.893383+0000","last_unstale":"2024-09-10T21:13:23.902958+0000","last_undegraded":"2024-09-10T21:13:23.902958+0000","last_fullsized":"2024-09-10T21:13:23.902958+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:10.261088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901948+0000","last_change":"2024-09-10T21:13:22.900413+0000","last_active":"2024-09-10T21:13:23.901948+0000","last_peered":"2024-09-10T21:13:23.901948+0000","last_clean":"2024-09-10T21:13:23.901948+0000","last_became_active":"2024-09-10T21:13:22.899234+0000","last_became_peered":"2024-09-10T21:13:22.899234+0000","last_unstale":"2024-09-10T21:13:23.901948+0000","last_undegraded":"2024-09-10T21:13:23.901948+0000","last_fullsized":"2024-09-10T21:13:23.901948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:27.125309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901976+0000","last_change":"2024-09-10T21:13:21.895050+0000","last_active":"2024-09-10T21:13:23.901976+0000","last_peered":"2024-09-10T21:13:23.901976+0000","last_clean":"2024-09-10T21:13:23.901976+0000","last_became_active":"2024-09-10T21:13:21.892497+0000","last_became_peered":"2024-09-10T21:13:21.892497+0000","last_unstale":"2024-09-10T21:13:23.901976+0000","last_undegraded":"2024-09-10T21:13:23.901976+0000","last_fullsized":"2024-09-10T21:13:23.901976+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:40.609267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922237+0000","last_change":"2024-09-10T21:13:22.921138+0000","last_active":"2024-09-10T21:13:22.922237+0000","last_peered":"2024-09-10T21:13:22.922237+0000","last_clean":"2024-09-10T21:13:22.922237+0000","last_became_active":"2024-09-10T21:13:22.920172+0000","last_became_peered":"2024-09-10T21:13:22.920172+0000","last_unstale":"2024-09-10T21:13:22.922237+0000","last_undegraded":"2024-09-10T21:13:22.922237+0000","last_fullsized":"2024-09-10T21:13:22.922237+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:21.882085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901976+0000","last_change":"2024-09-10T21:13:21.894991+0000","last_active":"2024-09-10T21:13:23.901976+0000","last_peered":"2024-09-10T21:13:23.901976+0000","last_clean":"2024-09-10T21:13:23.901976+0000","last_became_active":"2024-09-10T21:13:21.894201+0000","last_became_peered":"2024-09-10T21:13:21.894201+0000","last_unstale":"2024-09-10T21:13:23.901976+0000","last_undegraded":"2024-09-10T21:13:23.901976+0000","last_fullsized":"2024-09-10T21:13:23.901976+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:16.375733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922269+0000","last_change":"2024-09-10T21:13:22.917783+0000","last_active":"2024-09-10T21:13:22.922269+0000","last_peered":"2024-09-10T21:13:22.922269+0000","last_clean":"2024-09-10T21:13:22.922269+0000","last_became_active":"2024-09-10T21:13:22.917187+0000","last_became_peered":"2024-09-10T21:13:22.917187+0000","last_unstale":"2024-09-10T21:13:22.922269+0000","last_undegraded":"2024-09-10T21:13:22.922269+0000","last_fullsized":"2024-09-10T21:13:22.922269+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:35.277948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.905942+0000","last_change":"2024-09-10T21:13:21.906852+0000","last_active":"2024-09-10T21:13:22.905942+0000","last_peered":"2024-09-10T21:13:22.905942+0000","last_clean":"2024-09-10T21:13:22.905942+0000","last_became_active":"2024-09-10T21:13:21.906221+0000","last_became_peered":"2024-09-10T21:13:21.906221+0000","last_unstale":"2024-09-10T21:13:22.905942+0000","last_undegraded":"2024-09-10T21:13:22.905942+0000","last_fullsized":"2024-09-10T21:13:22.905942+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:03.864814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918213+0000","last_change":"2024-09-10T21:13:22.914627+0000","last_active":"2024-09-10T21:13:22.918213+0000","last_peered":"2024-09-10T21:13:22.918213+0000","last_clean":"2024-09-10T21:13:22.918213+0000","last_became_active":"2024-09-10T21:13:22.914389+0000","last_became_peered":"2024-09-10T21:13:22.914389+0000","last_unstale":"2024-09-10T21:13:22.918213+0000","last_undegraded":"2024-09-10T21:13:22.918213+0000","last_fullsized":"2024-09-10T21:13:22.918213+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:25.289658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.900655+0000","last_change":"2024-09-10T21:13:21.891567+0000","last_active":"2024-09-10T21:13:23.900655+0000","last_peered":"2024-09-10T21:13:23.900655+0000","last_clean":"2024-09-10T21:13:23.900655+0000","last_became_active":"2024-09-10T21:13:21.891342+0000","last_became_peered":"2024-09-10T21:13:21.891342+0000","last_unstale":"2024-09-10T21:13:23.900655+0000","last_undegraded":"2024-09-10T21:13:23.900655+0000","last_fullsized":"2024-09-10T21:13:23.900655+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:38:26.642457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901660+0000","last_change":"2024-09-10T21:13:22.907707+0000","last_active":"2024-09-10T21:13:23.901660+0000","last_peered":"2024-09-10T21:13:23.901660+0000","last_clean":"2024-09-10T21:13:23.901660+0000","last_became_active":"2024-09-10T21:13:22.907443+0000","last_became_peered":"2024-09-10T21:13:22.907443+0000","last_unstale":"2024-09-10T21:13:23.901660+0000","last_undegraded":"2024-09-10T21:13:23.901660+0000","last_fullsized":"2024-09-10T21:13:23.901660+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:43.420059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.920849+0000","last_change":"2024-09-10T21:13:21.905186+0000","last_active":"2024-09-10T21:13:22.920849+0000","last_peered":"2024-09-10T21:13:22.920849+0000","last_clean":"2024-09-10T21:13:22.920849+0000","last_became_active":"2024-09-10T21:13:21.904923+0000","last_became_peered":"2024-09-10T21:13:21.904923+0000","last_unstale":"2024-09-10T21:13:22.920849+0000","last_undegraded":"2024-09-10T21:13:22.920849+0000","last_fullsized":"2024-09-10T21:13:22.920849+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:09.229106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":193,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.890033+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:13:22.890033+0000","last_peered":"2024-09-10T21:13:22.890033+0000","last_clean":"2024-09-10T21:13:22.890033+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:13:22.890033+0000","last_undegraded":"2024-09-10T21:13:22.890033+0000","last_fullsized":"2024-09-10T21:13:22.890033+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":847872,"data_stored":350382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1429504,"data_stored":939194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":843776,"data_stored":349430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":843776,"data_stored":348862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":349318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":348806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36160,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335680,"statfs":{"total":96636764160,"available":96599736320,"internally_reserved":0,"allocated":819200,"data_stored":341961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":1425408,"data_stored":939104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:13:27.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:13:27.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.232+0000 7f5d5be00640 1 -- 172.21.15.16:0/235554250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c0d2820 msgr2=0x7f5d5c15fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:27.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.232+0000 7f5d5be00640 1 --2- 172.21.15.16:0/235554250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c0d2820 0x7f5d5c15fbe0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0730f0 tx=0x7f5d4401c830 comp rx=0 tx=0).stop 2024-09-10T21:13:27.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.232+0000 7f5d5be00640 1 -- 172.21.15.16:0/235554250 shutdown_connections 2024-09-10T21:13:27.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.232+0000 7f5d5be00640 1 --2- 172.21.15.16:0/235554250 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5d5c160120 0x7f5d5c160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.232+0000 7f5d5be00640 1 --2- 172.21.15.16:0/235554250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c0d2820 0x7f5d5c15fbe0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.232+0000 7f5d5be00640 1 --2- 172.21.15.16:0/235554250 >> v2:172.21.15.52:3300/0 conn(0x7f5d5c0cfd40 0x7f5d5c0d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.232+0000 7f5d5be00640 1 -- 172.21.15.16:0/235554250 >> 172.21.15.16:0/235554250 conn(0x7f5d5c0d1d00 msgr2=0x7f5d5c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.232+0000 7f5d5be00640 1 -- 172.21.15.16:0/235554250 shutdown_connections 2024-09-10T21:13:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.232+0000 7f5d5be00640 1 -- 172.21.15.16:0/235554250 wait complete. 2024-09-10T21:13:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.232+0000 7f5d5be00640 1 Processor -- start 2024-09-10T21:13:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.232+0000 7f5d5be00640 1 -- start start 2024-09-10T21:13:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d5c0cfd40 0x7f5d5c15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5be00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5d5c0d2820 0x7f5d5c1f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c160120 0x7f5d5c1f7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c0b2610 con 0x7f5d5c160120 2024-09-10T21:13:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5be00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c0b2490 con 0x7f5d5c0cfd40 2024-09-10T21:13:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5be00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5c0b2790 con 0x7f5d5c0d2820 2024-09-10T21:13:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c160120 0x7f5d5c1f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c160120 0x7f5d5c1f7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41884/0 (socket says 172.21.15.16:41884) 2024-09-10T21:13:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5b400640 1 -- 172.21.15.16:0/3123404937 learned_addr learned my addr 172.21.15.16:0/3123404937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5aa00640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d5c0cfd40 0x7f5d5c15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d52000640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5d5c0d2820 0x7f5d5c1f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5b400640 1 -- 172.21.15.16:0/3123404937 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5d5c0d2820 msgr2=0x7f5d5c1f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5b400640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5d5c0d2820 0x7f5d5c1f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5b400640 1 -- 172.21.15.16:0/3123404937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d5c0cfd40 msgr2=0x7f5d5c15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5b400640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d5c0cfd40 0x7f5d5c15a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5b400640 1 -- 172.21.15.16:0/3123404937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c0c75e0 con 0x7f5d5c160120 2024-09-10T21:13:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d52000640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5d5c0d2820 0x7f5d5c1f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5b400640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c160120 0x7f5d5c1f7630 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400d660 tx=0x7f5d5400db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:27.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d53e00640 1 -- 172.21.15.16:0/3123404937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54017070 con 0x7f5d5c160120 2024-09-10T21:13:27.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d53e00640 1 -- 172.21.15.16:0/3123404937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d54005310 con 0x7f5d5c160120 2024-09-10T21:13:27.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d53e00640 1 -- 172.21.15.16:0/3123404937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54009820 con 0x7f5d5c160120 2024-09-10T21:13:27.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5be00640 1 -- 172.21.15.16:0/3123404937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5c0c78c0 con 0x7f5d5c160120 2024-09-10T21:13:27.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5be00640 1 -- 172.21.15.16:0/3123404937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d5c0c7e80 con 0x7f5d5c160120 2024-09-10T21:13:27.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.236+0000 7f5d5be00640 1 -- 172.21.15.16:0/3123404937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d5c15fbe0 con 0x7f5d5c160120 2024-09-10T21:13:27.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.240+0000 7f5d53e00640 1 -- 172.21.15.16:0/3123404937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5d540099c0 con 0x7f5d5c160120 2024-09-10T21:13:27.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.240+0000 7f5d53e00640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5d380725e0 0x7f5d38074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:27.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.240+0000 7f5d53e00640 1 -- 172.21.15.16:0/3123404937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f5d540932b0 con 0x7f5d5c160120 2024-09-10T21:13:27.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.240+0000 7f5d5aa00640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5d380725e0 0x7f5d38074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:27.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.240+0000 7f5d5aa00640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5d380725e0 0x7f5d38074aa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c1fa640 tx=0x7f5d4c0095e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.244+0000 7f5d53e00640 1 -- 172.21.15.16:0/3123404937 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5d5405e2a0 con 0x7f5d5c160120 2024-09-10T21:13:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.412+0000 7f5d5be00640 1 -- 172.21.15.16:0/3123404937 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d5c0ce470 con 0x7f5d380725e0 2024-09-10T21:13:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.416+0000 7f5d53e00640 1 -- 172.21.15.16:0/3123404937 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252056 (secure 0 0 0) 0x7f5d5c0ce470 con 0x7f5d380725e0 2024-09-10T21:13:27.427 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:13:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.420+0000 7f5d5be00640 1 -- 172.21.15.16:0/3123404937 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5d380725e0 msgr2=0x7f5d38074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.420+0000 7f5d5be00640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5d380725e0 0x7f5d38074aa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c1fa640 tx=0x7f5d4c0095e0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.420+0000 7f5d5be00640 1 -- 172.21.15.16:0/3123404937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c160120 msgr2=0x7f5d5c1f7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.420+0000 7f5d5be00640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c160120 0x7f5d5c1f7630 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400d660 tx=0x7f5d5400db20 comp rx=0 tx=0).stop 2024-09-10T21:13:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.420+0000 7f5d5be00640 1 -- 172.21.15.16:0/3123404937 shutdown_connections 2024-09-10T21:13:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.420+0000 7f5d5be00640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5d380725e0 0x7f5d38074aa0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.420+0000 7f5d5be00640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c160120 0x7f5d5c1f7630 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.420+0000 7f5d5be00640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5d5c0d2820 0x7f5d5c1f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.420+0000 7f5d5be00640 1 --2- 172.21.15.16:0/3123404937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d5c0cfd40 0x7f5d5c15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.420+0000 7f5d5be00640 1 -- 172.21.15.16:0/3123404937 >> 172.21.15.16:0/3123404937 conn(0x7f5d5c0d1d00 msgr2=0x7f5d5c15e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.420+0000 7f5d5be00640 1 -- 172.21.15.16:0/3123404937 shutdown_connections 2024-09-10T21:13:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.420+0000 7f5d5be00640 1 -- 172.21.15.16:0/3123404937 wait complete. 2024-09-10T21:13:27.448 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-10T21:13:25.495345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265556,"kb_used_data":7696,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754709164,"statfs":{"total":773094113280,"available":772822183936,"internally_reserved":0,"allocated":7880704,"data_stored":3967057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.623087"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.912119+0000","last_change":"2024-09-10T21:13:22.903796+0000","last_active":"2024-09-10T21:13:22.912119+0000","last_peered":"2024-09-10T21:13:22.912119+0000","last_clean":"2024-09-10T21:13:22.912119+0000","last_became_active":"2024-09-10T21:13:22.903508+0000","last_became_peered":"2024-09-10T21:13:22.903508+0000","last_unstale":"2024-09-10T21:13:22.912119+0000","last_undegraded":"2024-09-10T21:13:22.912119+0000","last_fullsized":"2024-09-10T21:13:22.912119+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:35.266008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922798+0000","last_change":"2024-09-10T21:13:22.920023+0000","last_active":"2024-09-10T21:13:22.922798+0000","last_peered":"2024-09-10T21:13:22.922798+0000","last_clean":"2024-09-10T21:13:22.922798+0000","last_became_active":"2024-09-10T21:13:22.919291+0000","last_became_peered":"2024-09-10T21:13:22.919291+0000","last_unstale":"2024-09-10T21:13:22.922798+0000","last_undegraded":"2024-09-10T21:13:22.922798+0000","last_fullsized":"2024-09-10T21:13:22.922798+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:29.476609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922222+0000","last_change":"2024-09-10T21:13:22.911628+0000","last_active":"2024-09-10T21:13:22.922222+0000","last_peered":"2024-09-10T21:13:22.922222+0000","last_clean":"2024-09-10T21:13:22.922222+0000","last_became_active":"2024-09-10T21:13:22.911281+0000","last_became_peered":"2024-09-10T21:13:22.911281+0000","last_unstale":"2024-09-10T21:13:22.922222+0000","last_undegraded":"2024-09-10T21:13:22.922222+0000","last_fullsized":"2024-09-10T21:13:22.922222+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:07.351230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922257+0000","last_change":"2024-09-10T21:13:22.913645+0000","last_active":"2024-09-10T21:13:22.922257+0000","last_peered":"2024-09-10T21:13:22.922257+0000","last_clean":"2024-09-10T21:13:22.922257+0000","last_became_active":"2024-09-10T21:13:22.913364+0000","last_became_peered":"2024-09-10T21:13:22.913364+0000","last_unstale":"2024-09-10T21:13:22.922257+0000","last_undegraded":"2024-09-10T21:13:22.922257+0000","last_fullsized":"2024-09-10T21:13:22.922257+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:04.798092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901562+0000","last_change":"2024-09-10T21:13:22.913846+0000","last_active":"2024-09-10T21:13:23.901562+0000","last_peered":"2024-09-10T21:13:23.901562+0000","last_clean":"2024-09-10T21:13:23.901562+0000","last_became_active":"2024-09-10T21:13:22.913585+0000","last_became_peered":"2024-09-10T21:13:22.913585+0000","last_unstale":"2024-09-10T21:13:23.901562+0000","last_undegraded":"2024-09-10T21:13:23.901562+0000","last_fullsized":"2024-09-10T21:13:23.901562+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:42.541215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.900917+0000","last_change":"2024-09-10T21:13:22.894948+0000","last_active":"2024-09-10T21:13:23.900917+0000","last_peered":"2024-09-10T21:13:23.900917+0000","last_clean":"2024-09-10T21:13:23.900917+0000","last_became_active":"2024-09-10T21:13:22.894625+0000","last_became_peered":"2024-09-10T21:13:22.894625+0000","last_unstale":"2024-09-10T21:13:23.900917+0000","last_undegraded":"2024-09-10T21:13:23.900917+0000","last_fullsized":"2024-09-10T21:13:23.900917+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:12.944060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918505+0000","last_change":"2024-09-10T21:13:22.912267+0000","last_active":"2024-09-10T21:13:22.918505+0000","last_peered":"2024-09-10T21:13:22.918505+0000","last_clean":"2024-09-10T21:13:22.918505+0000","last_became_active":"2024-09-10T21:13:22.908721+0000","last_became_peered":"2024-09-10T21:13:22.908721+0000","last_unstale":"2024-09-10T21:13:22.918505+0000","last_undegraded":"2024-09-10T21:13:22.918505+0000","last_fullsized":"2024-09-10T21:13:22.918505+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:06.697602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.911388+0000","last_change":"2024-09-10T21:13:22.906168+0000","last_active":"2024-09-10T21:13:22.911388+0000","last_peered":"2024-09-10T21:13:22.911388+0000","last_clean":"2024-09-10T21:13:22.911388+0000","last_became_active":"2024-09-10T21:13:22.905916+0000","last_became_peered":"2024-09-10T21:13:22.905916+0000","last_unstale":"2024-09-10T21:13:22.911388+0000","last_undegraded":"2024-09-10T21:13:22.911388+0000","last_fullsized":"2024-09-10T21:13:22.911388+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:28.915207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922412+0000","last_change":"2024-09-10T21:13:22.917607+0000","last_active":"2024-09-10T21:13:22.922412+0000","last_peered":"2024-09-10T21:13:22.922412+0000","last_clean":"2024-09-10T21:13:22.922412+0000","last_became_active":"2024-09-10T21:13:22.917214+0000","last_became_peered":"2024-09-10T21:13:22.917214+0000","last_unstale":"2024-09-10T21:13:22.922412+0000","last_undegraded":"2024-09-10T21:13:22.922412+0000","last_fullsized":"2024-09-10T21:13:22.922412+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:40.270334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918561+0000","last_change":"2024-09-10T21:13:22.912745+0000","last_active":"2024-09-10T21:13:22.918561+0000","last_peered":"2024-09-10T21:13:22.918561+0000","last_clean":"2024-09-10T21:13:22.918561+0000","last_became_active":"2024-09-10T21:13:22.910798+0000","last_became_peered":"2024-09-10T21:13:22.910798+0000","last_unstale":"2024-09-10T21:13:22.918561+0000","last_undegraded":"2024-09-10T21:13:22.918561+0000","last_fullsized":"2024-09-10T21:13:22.918561+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:14.832743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.905362+0000","last_change":"2024-09-10T21:13:22.899726+0000","last_active":"2024-09-10T21:13:22.905362+0000","last_peered":"2024-09-10T21:13:22.905362+0000","last_clean":"2024-09-10T21:13:22.905362+0000","last_became_active":"2024-09-10T21:13:22.899476+0000","last_became_peered":"2024-09-10T21:13:22.899476+0000","last_unstale":"2024-09-10T21:13:22.905362+0000","last_undegraded":"2024-09-10T21:13:22.905362+0000","last_fullsized":"2024-09-10T21:13:22.905362+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:46.578364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902993+0000","last_change":"2024-09-10T21:13:22.907292+0000","last_active":"2024-09-10T21:13:23.902993+0000","last_peered":"2024-09-10T21:13:23.902993+0000","last_clean":"2024-09-10T21:13:23.902993+0000","last_became_active":"2024-09-10T21:13:22.898081+0000","last_became_peered":"2024-09-10T21:13:22.898081+0000","last_unstale":"2024-09-10T21:13:23.902993+0000","last_undegraded":"2024-09-10T21:13:23.902993+0000","last_fullsized":"2024-09-10T21:13:23.902993+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:16.374896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922347+0000","last_change":"2024-09-10T21:13:22.917376+0000","last_active":"2024-09-10T21:13:22.922347+0000","last_peered":"2024-09-10T21:13:22.922347+0000","last_clean":"2024-09-10T21:13:22.922347+0000","last_became_active":"2024-09-10T21:13:22.917053+0000","last_became_peered":"2024-09-10T21:13:22.917053+0000","last_unstale":"2024-09-10T21:13:22.922347+0000","last_undegraded":"2024-09-10T21:13:22.922347+0000","last_fullsized":"2024-09-10T21:13:22.922347+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:42.649794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.903127+0000","last_change":"2024-09-10T21:13:22.900730+0000","last_active":"2024-09-10T21:13:23.903127+0000","last_peered":"2024-09-10T21:13:23.903127+0000","last_clean":"2024-09-10T21:13:23.903127+0000","last_became_active":"2024-09-10T21:13:22.900385+0000","last_became_peered":"2024-09-10T21:13:22.900385+0000","last_unstale":"2024-09-10T21:13:23.903127+0000","last_undegraded":"2024-09-10T21:13:23.903127+0000","last_fullsized":"2024-09-10T21:13:23.903127+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:25.620749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901431+0000","last_change":"2024-09-10T21:13:22.908969+0000","last_active":"2024-09-10T21:13:23.901431+0000","last_peered":"2024-09-10T21:13:23.901431+0000","last_clean":"2024-09-10T21:13:23.901431+0000","last_became_active":"2024-09-10T21:13:22.908612+0000","last_became_peered":"2024-09-10T21:13:22.908612+0000","last_unstale":"2024-09-10T21:13:23.901431+0000","last_undegraded":"2024-09-10T21:13:23.901431+0000","last_fullsized":"2024-09-10T21:13:23.901431+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:58.136168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922158+0000","last_change":"2024-09-10T21:13:22.920560+0000","last_active":"2024-09-10T21:13:22.922158+0000","last_peered":"2024-09-10T21:13:22.922158+0000","last_clean":"2024-09-10T21:13:22.922158+0000","last_became_active":"2024-09-10T21:13:22.920155+0000","last_became_peered":"2024-09-10T21:13:22.920155+0000","last_unstale":"2024-09-10T21:13:22.922158+0000","last_undegraded":"2024-09-10T21:13:22.922158+0000","last_fullsized":"2024-09-10T21:13:22.922158+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:47.643798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902272+0000","last_change":"2024-09-10T21:13:22.902540+0000","last_active":"2024-09-10T21:13:23.902272+0000","last_peered":"2024-09-10T21:13:23.902272+0000","last_clean":"2024-09-10T21:13:23.902272+0000","last_became_active":"2024-09-10T21:13:22.902169+0000","last_became_peered":"2024-09-10T21:13:22.902169+0000","last_unstale":"2024-09-10T21:13:23.902272+0000","last_undegraded":"2024-09-10T21:13:23.902272+0000","last_fullsized":"2024-09-10T21:13:23.902272+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:36:09.142008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902256+0000","last_change":"2024-09-10T21:13:22.912041+0000","last_active":"2024-09-10T21:13:23.902256+0000","last_peered":"2024-09-10T21:13:23.902256+0000","last_clean":"2024-09-10T21:13:23.902256+0000","last_became_active":"2024-09-10T21:13:22.911678+0000","last_became_peered":"2024-09-10T21:13:22.911678+0000","last_unstale":"2024-09-10T21:13:23.902256+0000","last_undegraded":"2024-09-10T21:13:23.902256+0000","last_fullsized":"2024-09-10T21:13:23.902256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:58.781293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.911673+0000","last_change":"2024-09-10T21:13:22.903186+0000","last_active":"2024-09-10T21:13:22.911673+0000","last_peered":"2024-09-10T21:13:22.911673+0000","last_clean":"2024-09-10T21:13:22.911673+0000","last_became_active":"2024-09-10T21:13:22.902897+0000","last_became_peered":"2024-09-10T21:13:22.902897+0000","last_unstale":"2024-09-10T21:13:22.911673+0000","last_undegraded":"2024-09-10T21:13:22.911673+0000","last_fullsized":"2024-09-10T21:13:22.911673+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:47.545186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901857+0000","last_change":"2024-09-10T21:13:22.893957+0000","last_active":"2024-09-10T21:13:23.901857+0000","last_peered":"2024-09-10T21:13:23.901857+0000","last_clean":"2024-09-10T21:13:23.901857+0000","last_became_active":"2024-09-10T21:13:22.893678+0000","last_became_peered":"2024-09-10T21:13:22.893678+0000","last_unstale":"2024-09-10T21:13:23.901857+0000","last_undegraded":"2024-09-10T21:13:23.901857+0000","last_fullsized":"2024-09-10T21:13:23.901857+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:15.626682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.925059+0000","last_change":"2024-09-10T21:13:22.924024+0000","last_active":"2024-09-10T21:13:22.925059+0000","last_peered":"2024-09-10T21:13:22.925059+0000","last_clean":"2024-09-10T21:13:22.925059+0000","last_became_active":"2024-09-10T21:13:22.923663+0000","last_became_peered":"2024-09-10T21:13:22.923663+0000","last_unstale":"2024-09-10T21:13:22.925059+0000","last_undegraded":"2024-09-10T21:13:22.925059+0000","last_fullsized":"2024-09-10T21:13:22.925059+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:26.813935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.900996+0000","last_change":"2024-09-10T21:13:22.903097+0000","last_active":"2024-09-10T21:13:23.900996+0000","last_peered":"2024-09-10T21:13:23.900996+0000","last_clean":"2024-09-10T21:13:23.900996+0000","last_became_active":"2024-09-10T21:13:22.902716+0000","last_became_peered":"2024-09-10T21:13:22.902716+0000","last_unstale":"2024-09-10T21:13:23.900996+0000","last_undegraded":"2024-09-10T21:13:23.900996+0000","last_fullsized":"2024-09-10T21:13:23.900996+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:28.965753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918526+0000","last_change":"2024-09-10T21:13:22.915676+0000","last_active":"2024-09-10T21:13:22.918526+0000","last_peered":"2024-09-10T21:13:22.918526+0000","last_clean":"2024-09-10T21:13:22.918526+0000","last_became_active":"2024-09-10T21:13:22.915330+0000","last_became_peered":"2024-09-10T21:13:22.915330+0000","last_unstale":"2024-09-10T21:13:22.918526+0000","last_undegraded":"2024-09-10T21:13:22.918526+0000","last_fullsized":"2024-09-10T21:13:22.918526+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:52.872636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.897629+0000","last_change":"2024-09-10T21:13:22.893235+0000","last_active":"2024-09-10T21:13:22.897629+0000","last_peered":"2024-09-10T21:13:22.897629+0000","last_clean":"2024-09-10T21:13:22.897629+0000","last_became_active":"2024-09-10T21:13:22.892180+0000","last_became_peered":"2024-09-10T21:13:22.892180+0000","last_unstale":"2024-09-10T21:13:22.897629+0000","last_undegraded":"2024-09-10T21:13:22.897629+0000","last_fullsized":"2024-09-10T21:13:22.897629+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:20.849337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.923422+0000","last_change":"2024-09-10T21:13:22.921263+0000","last_active":"2024-09-10T21:13:22.923422+0000","last_peered":"2024-09-10T21:13:22.923422+0000","last_clean":"2024-09-10T21:13:22.923422+0000","last_became_active":"2024-09-10T21:13:22.920957+0000","last_became_peered":"2024-09-10T21:13:22.920957+0000","last_unstale":"2024-09-10T21:13:22.923422+0000","last_undegraded":"2024-09-10T21:13:22.923422+0000","last_fullsized":"2024-09-10T21:13:22.923422+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:35.263152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.902365+0000","last_change":"2024-09-10T21:13:22.897818+0000","last_active":"2024-09-10T21:13:22.902365+0000","last_peered":"2024-09-10T21:13:22.902365+0000","last_clean":"2024-09-10T21:13:22.902365+0000","last_became_active":"2024-09-10T21:13:22.897548+0000","last_became_peered":"2024-09-10T21:13:22.897548+0000","last_unstale":"2024-09-10T21:13:22.902365+0000","last_undegraded":"2024-09-10T21:13:22.902365+0000","last_fullsized":"2024-09-10T21:13:22.902365+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:29.463517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901931+0000","last_change":"2024-09-10T21:13:22.909907+0000","last_active":"2024-09-10T21:13:23.901931+0000","last_peered":"2024-09-10T21:13:23.901931+0000","last_clean":"2024-09-10T21:13:23.901931+0000","last_became_active":"2024-09-10T21:13:22.909634+0000","last_became_peered":"2024-09-10T21:13:22.909634+0000","last_unstale":"2024-09-10T21:13:23.901931+0000","last_undegraded":"2024-09-10T21:13:23.901931+0000","last_fullsized":"2024-09-10T21:13:23.901931+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:08.408152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922097+0000","last_change":"2024-09-10T21:13:22.918658+0000","last_active":"2024-09-10T21:13:22.922097+0000","last_peered":"2024-09-10T21:13:22.922097+0000","last_clean":"2024-09-10T21:13:22.922097+0000","last_became_active":"2024-09-10T21:13:22.918275+0000","last_became_peered":"2024-09-10T21:13:22.918275+0000","last_unstale":"2024-09-10T21:13:22.922097+0000","last_undegraded":"2024-09-10T21:13:22.922097+0000","last_fullsized":"2024-09-10T21:13:22.922097+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:26.044069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.921633+0000","last_change":"2024-09-10T21:13:22.908668+0000","last_active":"2024-09-10T21:13:22.921633+0000","last_peered":"2024-09-10T21:13:22.921633+0000","last_clean":"2024-09-10T21:13:22.921633+0000","last_became_active":"2024-09-10T21:13:22.907915+0000","last_became_peered":"2024-09-10T21:13:22.907915+0000","last_unstale":"2024-09-10T21:13:22.921633+0000","last_undegraded":"2024-09-10T21:13:22.921633+0000","last_fullsized":"2024-09-10T21:13:22.921633+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:21.667515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.911191+0000","last_change":"2024-09-10T21:13:22.903576+0000","last_active":"2024-09-10T21:13:22.911191+0000","last_peered":"2024-09-10T21:13:22.911191+0000","last_clean":"2024-09-10T21:13:22.911191+0000","last_became_active":"2024-09-10T21:13:22.903261+0000","last_became_peered":"2024-09-10T21:13:22.903261+0000","last_unstale":"2024-09-10T21:13:22.911191+0000","last_undegraded":"2024-09-10T21:13:22.911191+0000","last_fullsized":"2024-09-10T21:13:22.911191+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:39.122262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922023+0000","last_change":"2024-09-10T21:13:22.918311+0000","last_active":"2024-09-10T21:13:22.922023+0000","last_peered":"2024-09-10T21:13:22.922023+0000","last_clean":"2024-09-10T21:13:22.922023+0000","last_became_active":"2024-09-10T21:13:22.918038+0000","last_became_peered":"2024-09-10T21:13:22.918038+0000","last_unstale":"2024-09-10T21:13:22.922023+0000","last_undegraded":"2024-09-10T21:13:22.922023+0000","last_fullsized":"2024-09-10T21:13:22.922023+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:38:27.645205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922123+0000","last_change":"2024-09-10T21:13:22.918559+0000","last_active":"2024-09-10T21:13:22.922123+0000","last_peered":"2024-09-10T21:13:22.922123+0000","last_clean":"2024-09-10T21:13:22.922123+0000","last_became_active":"2024-09-10T21:13:22.918289+0000","last_became_peered":"2024-09-10T21:13:22.918289+0000","last_unstale":"2024-09-10T21:13:22.922123+0000","last_undegraded":"2024-09-10T21:13:22.922123+0000","last_fullsized":"2024-09-10T21:13:22.922123+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:11.743269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918068+0000","last_change":"2024-09-10T21:13:22.916148+0000","last_active":"2024-09-10T21:13:22.918068+0000","last_peered":"2024-09-10T21:13:22.918068+0000","last_clean":"2024-09-10T21:13:22.918068+0000","last_became_active":"2024-09-10T21:13:22.912870+0000","last_became_peered":"2024-09-10T21:13:22.912870+0000","last_unstale":"2024-09-10T21:13:22.918068+0000","last_undegraded":"2024-09-10T21:13:22.918068+0000","last_fullsized":"2024-09-10T21:13:22.918068+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:23:44.907538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901208+0000","last_change":"2024-09-10T21:13:21.891223+0000","last_active":"2024-09-10T21:13:23.901208+0000","last_peered":"2024-09-10T21:13:23.901208+0000","last_clean":"2024-09-10T21:13:23.901208+0000","last_became_active":"2024-09-10T21:13:21.890880+0000","last_became_peered":"2024-09-10T21:13:21.890880+0000","last_unstale":"2024-09-10T21:13:23.901208+0000","last_undegraded":"2024-09-10T21:13:23.901208+0000","last_fullsized":"2024-09-10T21:13:23.901208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:28.473861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.920982+0000","last_change":"2024-09-10T21:13:22.910860+0000","last_active":"2024-09-10T21:13:22.920982+0000","last_peered":"2024-09-10T21:13:22.920982+0000","last_clean":"2024-09-10T21:13:22.920982+0000","last_became_active":"2024-09-10T21:13:22.910569+0000","last_became_peered":"2024-09-10T21:13:22.910569+0000","last_unstale":"2024-09-10T21:13:22.920982+0000","last_undegraded":"2024-09-10T21:13:22.920982+0000","last_fullsized":"2024-09-10T21:13:22.920982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:10.231854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902611+0000","last_change":"2024-09-10T21:13:21.890910+0000","last_active":"2024-09-10T21:13:23.902611+0000","last_peered":"2024-09-10T21:13:23.902611+0000","last_clean":"2024-09-10T21:13:23.902611+0000","last_became_active":"2024-09-10T21:13:21.890621+0000","last_became_peered":"2024-09-10T21:13:21.890621+0000","last_unstale":"2024-09-10T21:13:23.902611+0000","last_undegraded":"2024-09-10T21:13:23.902611+0000","last_fullsized":"2024-09-10T21:13:23.902611+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:57.940612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901137+0000","last_change":"2024-09-10T21:13:22.894314+0000","last_active":"2024-09-10T21:13:23.901137+0000","last_peered":"2024-09-10T21:13:23.901137+0000","last_clean":"2024-09-10T21:13:23.901137+0000","last_became_active":"2024-09-10T21:13:22.894039+0000","last_became_peered":"2024-09-10T21:13:22.894039+0000","last_unstale":"2024-09-10T21:13:23.901137+0000","last_undegraded":"2024-09-10T21:13:23.901137+0000","last_fullsized":"2024-09-10T21:13:23.901137+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:33.037742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.912016+0000","last_change":"2024-09-10T21:13:21.911067+0000","last_active":"2024-09-10T21:13:22.912016+0000","last_peered":"2024-09-10T21:13:22.912016+0000","last_clean":"2024-09-10T21:13:22.912016+0000","last_became_active":"2024-09-10T21:13:21.910626+0000","last_became_peered":"2024-09-10T21:13:21.910626+0000","last_unstale":"2024-09-10T21:13:22.912016+0000","last_undegraded":"2024-09-10T21:13:22.912016+0000","last_fullsized":"2024-09-10T21:13:22.912016+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:37.367655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918469+0000","last_change":"2024-09-10T21:13:22.911184+0000","last_active":"2024-09-10T21:13:22.918469+0000","last_peered":"2024-09-10T21:13:22.918469+0000","last_clean":"2024-09-10T21:13:22.918469+0000","last_became_active":"2024-09-10T21:13:22.910849+0000","last_became_peered":"2024-09-10T21:13:22.910849+0000","last_unstale":"2024-09-10T21:13:22.918469+0000","last_undegraded":"2024-09-10T21:13:22.918469+0000","last_fullsized":"2024-09-10T21:13:22.918469+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:14.675460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901073+0000","last_change":"2024-09-10T21:13:21.903374+0000","last_active":"2024-09-10T21:13:23.901073+0000","last_peered":"2024-09-10T21:13:23.901073+0000","last_clean":"2024-09-10T21:13:23.901073+0000","last_became_active":"2024-09-10T21:13:21.903062+0000","last_became_peered":"2024-09-10T21:13:21.903062+0000","last_unstale":"2024-09-10T21:13:23.901073+0000","last_undegraded":"2024-09-10T21:13:23.901073+0000","last_fullsized":"2024-09-10T21:13:23.901073+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:25.041321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.911347+0000","last_change":"2024-09-10T21:13:22.906151+0000","last_active":"2024-09-10T21:13:22.911347+0000","last_peered":"2024-09-10T21:13:22.911347+0000","last_clean":"2024-09-10T21:13:22.911347+0000","last_became_active":"2024-09-10T21:13:22.905888+0000","last_became_peered":"2024-09-10T21:13:22.905888+0000","last_unstale":"2024-09-10T21:13:22.911347+0000","last_undegraded":"2024-09-10T21:13:22.911347+0000","last_fullsized":"2024-09-10T21:13:22.911347+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:16.225082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901882+0000","last_change":"2024-09-10T21:13:21.895415+0000","last_active":"2024-09-10T21:13:23.901882+0000","last_peered":"2024-09-10T21:13:23.901882+0000","last_clean":"2024-09-10T21:13:23.901882+0000","last_became_active":"2024-09-10T21:13:21.894307+0000","last_became_peered":"2024-09-10T21:13:21.894307+0000","last_unstale":"2024-09-10T21:13:23.901882+0000","last_undegraded":"2024-09-10T21:13:23.901882+0000","last_fullsized":"2024-09-10T21:13:23.901882+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:14.392919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901290+0000","last_change":"2024-09-10T21:13:22.894260+0000","last_active":"2024-09-10T21:13:23.901290+0000","last_peered":"2024-09-10T21:13:23.901290+0000","last_clean":"2024-09-10T21:13:23.901290+0000","last_became_active":"2024-09-10T21:13:22.893955+0000","last_became_peered":"2024-09-10T21:13:22.893955+0000","last_unstale":"2024-09-10T21:13:23.901290+0000","last_undegraded":"2024-09-10T21:13:23.901290+0000","last_fullsized":"2024-09-10T21:13:23.901290+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:49.018588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.916134+0000","last_change":"2024-09-10T21:13:21.908654+0000","last_active":"2024-09-10T21:13:22.916134+0000","last_peered":"2024-09-10T21:13:22.916134+0000","last_clean":"2024-09-10T21:13:22.916134+0000","last_became_active":"2024-09-10T21:13:21.908331+0000","last_became_peered":"2024-09-10T21:13:21.908331+0000","last_unstale":"2024-09-10T21:13:22.916134+0000","last_undegraded":"2024-09-10T21:13:22.916134+0000","last_fullsized":"2024-09-10T21:13:22.916134+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:41.647046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.923539+0000","last_change":"2024-09-10T21:13:22.920840+0000","last_active":"2024-09-10T21:13:22.923539+0000","last_peered":"2024-09-10T21:13:22.923539+0000","last_clean":"2024-09-10T21:13:22.923539+0000","last_became_active":"2024-09-10T21:13:22.917562+0000","last_became_peered":"2024-09-10T21:13:22.917562+0000","last_unstale":"2024-09-10T21:13:22.923539+0000","last_undegraded":"2024-09-10T21:13:22.923539+0000","last_fullsized":"2024-09-10T21:13:22.923539+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:15.395667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.895394+0000","last_change":"2024-09-10T21:13:21.909382+0000","last_active":"2024-09-10T21:13:22.895394+0000","last_peered":"2024-09-10T21:13:22.895394+0000","last_clean":"2024-09-10T21:13:22.895394+0000","last_became_active":"2024-09-10T21:13:21.909059+0000","last_became_peered":"2024-09-10T21:13:21.909059+0000","last_unstale":"2024-09-10T21:13:22.895394+0000","last_undegraded":"2024-09-10T21:13:22.895394+0000","last_fullsized":"2024-09-10T21:13:22.895394+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:53.484906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902253+0000","last_change":"2024-09-10T21:13:22.908636+0000","last_active":"2024-09-10T21:13:23.902253+0000","last_peered":"2024-09-10T21:13:23.902253+0000","last_clean":"2024-09-10T21:13:23.902253+0000","last_became_active":"2024-09-10T21:13:22.908223+0000","last_became_peered":"2024-09-10T21:13:22.908223+0000","last_unstale":"2024-09-10T21:13:23.902253+0000","last_undegraded":"2024-09-10T21:13:23.902253+0000","last_fullsized":"2024-09-10T21:13:23.902253+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:08.036388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901033+0000","last_change":"2024-09-10T21:13:21.890375+0000","last_active":"2024-09-10T21:13:23.901033+0000","last_peered":"2024-09-10T21:13:23.901033+0000","last_clean":"2024-09-10T21:13:23.901033+0000","last_became_active":"2024-09-10T21:13:21.890028+0000","last_became_peered":"2024-09-10T21:13:21.890028+0000","last_unstale":"2024-09-10T21:13:23.901033+0000","last_undegraded":"2024-09-10T21:13:23.901033+0000","last_fullsized":"2024-09-10T21:13:23.901033+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:41.970949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918658+0000","last_change":"2024-09-10T21:13:22.914964+0000","last_active":"2024-09-10T21:13:22.918658+0000","last_peered":"2024-09-10T21:13:22.918658+0000","last_clean":"2024-09-10T21:13:22.918658+0000","last_became_active":"2024-09-10T21:13:22.914421+0000","last_became_peered":"2024-09-10T21:13:22.914421+0000","last_unstale":"2024-09-10T21:13:22.918658+0000","last_undegraded":"2024-09-10T21:13:22.918658+0000","last_fullsized":"2024-09-10T21:13:22.918658+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:56.608777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.888549+0000","last_change":"2024-09-10T21:13:21.900619+0000","last_active":"2024-09-10T21:13:22.888549+0000","last_peered":"2024-09-10T21:13:22.888549+0000","last_clean":"2024-09-10T21:13:22.888549+0000","last_became_active":"2024-09-10T21:13:21.900230+0000","last_became_peered":"2024-09-10T21:13:21.900230+0000","last_unstale":"2024-09-10T21:13:22.888549+0000","last_undegraded":"2024-09-10T21:13:22.888549+0000","last_fullsized":"2024-09-10T21:13:22.888549+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:59.280286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922816+0000","last_change":"2024-09-10T21:13:22.919434+0000","last_active":"2024-09-10T21:13:22.922816+0000","last_peered":"2024-09-10T21:13:22.922816+0000","last_clean":"2024-09-10T21:13:22.922816+0000","last_became_active":"2024-09-10T21:13:22.919119+0000","last_became_peered":"2024-09-10T21:13:22.919119+0000","last_unstale":"2024-09-10T21:13:22.922816+0000","last_undegraded":"2024-09-10T21:13:22.922816+0000","last_fullsized":"2024-09-10T21:13:22.922816+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:01.378449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.907283+0000","last_change":"2024-09-10T21:13:21.910037+0000","last_active":"2024-09-10T21:13:22.907283+0000","last_peered":"2024-09-10T21:13:22.907283+0000","last_clean":"2024-09-10T21:13:22.907283+0000","last_became_active":"2024-09-10T21:13:21.906750+0000","last_became_peered":"2024-09-10T21:13:21.906750+0000","last_unstale":"2024-09-10T21:13:22.907283+0000","last_undegraded":"2024-09-10T21:13:22.907283+0000","last_fullsized":"2024-09-10T21:13:22.907283+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:53.419904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.903943+0000","last_change":"2024-09-10T21:13:22.900722+0000","last_active":"2024-09-10T21:13:22.903943+0000","last_peered":"2024-09-10T21:13:22.903943+0000","last_clean":"2024-09-10T21:13:22.903943+0000","last_became_active":"2024-09-10T21:13:22.900463+0000","last_became_peered":"2024-09-10T21:13:22.900463+0000","last_unstale":"2024-09-10T21:13:22.903943+0000","last_undegraded":"2024-09-10T21:13:22.903943+0000","last_fullsized":"2024-09-10T21:13:22.903943+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:50.015582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902319+0000","last_change":"2024-09-10T21:13:21.893258+0000","last_active":"2024-09-10T21:13:23.902319+0000","last_peered":"2024-09-10T21:13:23.902319+0000","last_clean":"2024-09-10T21:13:23.902319+0000","last_became_active":"2024-09-10T21:13:21.892943+0000","last_became_peered":"2024-09-10T21:13:21.892943+0000","last_unstale":"2024-09-10T21:13:23.902319+0000","last_undegraded":"2024-09-10T21:13:23.902319+0000","last_fullsized":"2024-09-10T21:13:23.902319+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:46.844309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902312+0000","last_change":"2024-09-10T21:13:22.909927+0000","last_active":"2024-09-10T21:13:23.902312+0000","last_peered":"2024-09-10T21:13:23.902312+0000","last_clean":"2024-09-10T21:13:23.902312+0000","last_became_active":"2024-09-10T21:13:22.909504+0000","last_became_peered":"2024-09-10T21:13:22.909504+0000","last_unstale":"2024-09-10T21:13:23.902312+0000","last_undegraded":"2024-09-10T21:13:23.902312+0000","last_fullsized":"2024-09-10T21:13:23.902312+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:36:59.591146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"69'6","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.925300+0000","last_change":"2024-09-10T21:13:21.901366+0000","last_active":"2024-09-10T21:13:22.925300+0000","last_peered":"2024-09-10T21:13:22.925300+0000","last_clean":"2024-09-10T21:13:22.925300+0000","last_became_active":"2024-09-10T21:13:21.901017+0000","last_became_peered":"2024-09-10T21:13:21.901017+0000","last_unstale":"2024-09-10T21:13:22.925300+0000","last_undegraded":"2024-09-10T21:13:22.925300+0000","last_fullsized":"2024-09-10T21:13:22.925300+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:46.707163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901196+0000","last_change":"2024-09-10T21:13:22.904653+0000","last_active":"2024-09-10T21:13:23.901196+0000","last_peered":"2024-09-10T21:13:23.901196+0000","last_clean":"2024-09-10T21:13:23.901196+0000","last_became_active":"2024-09-10T21:13:22.904344+0000","last_became_peered":"2024-09-10T21:13:22.904344+0000","last_unstale":"2024-09-10T21:13:23.901196+0000","last_undegraded":"2024-09-10T21:13:23.901196+0000","last_fullsized":"2024-09-10T21:13:23.901196+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:57.783475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"69'2","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.912129+0000","last_change":"2024-09-10T21:13:21.901952+0000","last_active":"2024-09-10T21:13:22.912129+0000","last_peered":"2024-09-10T21:13:22.912129+0000","last_clean":"2024-09-10T21:13:22.912129+0000","last_became_active":"2024-09-10T21:13:21.901478+0000","last_became_peered":"2024-09-10T21:13:21.901478+0000","last_unstale":"2024-09-10T21:13:22.912129+0000","last_undegraded":"2024-09-10T21:13:22.912129+0000","last_fullsized":"2024-09-10T21:13:22.912129+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:30.021715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.908596+0000","last_change":"2024-09-10T21:13:22.898289+0000","last_active":"2024-09-10T21:13:22.908596+0000","last_peered":"2024-09-10T21:13:22.908596+0000","last_clean":"2024-09-10T21:13:22.908596+0000","last_became_active":"2024-09-10T21:13:22.897979+0000","last_became_peered":"2024-09-10T21:13:22.897979+0000","last_unstale":"2024-09-10T21:13:22.908596+0000","last_undegraded":"2024-09-10T21:13:22.908596+0000","last_fullsized":"2024-09-10T21:13:22.908596+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:10.730899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.892289+0000","last_change":"2024-09-10T21:13:21.904214+0000","last_active":"2024-09-10T21:13:22.892289+0000","last_peered":"2024-09-10T21:13:22.892289+0000","last_clean":"2024-09-10T21:13:22.892289+0000","last_became_active":"2024-09-10T21:13:21.903872+0000","last_became_peered":"2024-09-10T21:13:21.903872+0000","last_unstale":"2024-09-10T21:13:22.892289+0000","last_undegraded":"2024-09-10T21:13:22.892289+0000","last_fullsized":"2024-09-10T21:13:22.892289+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:03.795344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901444+0000","last_change":"2024-09-10T21:13:22.905275+0000","last_active":"2024-09-10T21:13:23.901444+0000","last_peered":"2024-09-10T21:13:23.901444+0000","last_clean":"2024-09-10T21:13:23.901444+0000","last_became_active":"2024-09-10T21:13:22.902236+0000","last_became_peered":"2024-09-10T21:13:22.902236+0000","last_unstale":"2024-09-10T21:13:23.901444+0000","last_undegraded":"2024-09-10T21:13:23.901444+0000","last_fullsized":"2024-09-10T21:13:23.901444+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:05.930480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.890219+0000","last_change":"2024-09-10T21:13:21.906146+0000","last_active":"2024-09-10T21:13:22.890219+0000","last_peered":"2024-09-10T21:13:22.890219+0000","last_clean":"2024-09-10T21:13:22.890219+0000","last_became_active":"2024-09-10T21:13:21.905879+0000","last_became_peered":"2024-09-10T21:13:21.905879+0000","last_unstale":"2024-09-10T21:13:22.890219+0000","last_undegraded":"2024-09-10T21:13:22.890219+0000","last_fullsized":"2024-09-10T21:13:22.890219+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:20.664766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.907318+0000","last_change":"2024-09-10T21:13:22.903404+0000","last_active":"2024-09-10T21:13:22.907318+0000","last_peered":"2024-09-10T21:13:22.907318+0000","last_clean":"2024-09-10T21:13:22.907318+0000","last_became_active":"2024-09-10T21:13:22.897729+0000","last_became_peered":"2024-09-10T21:13:22.897729+0000","last_unstale":"2024-09-10T21:13:22.907318+0000","last_undegraded":"2024-09-10T21:13:22.907318+0000","last_fullsized":"2024-09-10T21:13:22.907318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:18.592973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.916904+0000","last_change":"2024-09-10T21:13:21.900487+0000","last_active":"2024-09-10T21:13:22.916904+0000","last_peered":"2024-09-10T21:13:22.916904+0000","last_clean":"2024-09-10T21:13:22.916904+0000","last_became_active":"2024-09-10T21:13:21.900220+0000","last_became_peered":"2024-09-10T21:13:21.900220+0000","last_unstale":"2024-09-10T21:13:22.916904+0000","last_undegraded":"2024-09-10T21:13:22.916904+0000","last_fullsized":"2024-09-10T21:13:22.916904+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:06.348482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.920226+0000","last_change":"2024-09-10T21:13:22.910988+0000","last_active":"2024-09-10T21:13:22.920226+0000","last_peered":"2024-09-10T21:13:22.920226+0000","last_clean":"2024-09-10T21:13:22.920226+0000","last_became_active":"2024-09-10T21:13:22.910623+0000","last_became_peered":"2024-09-10T21:13:22.910623+0000","last_unstale":"2024-09-10T21:13:22.920226+0000","last_undegraded":"2024-09-10T21:13:22.920226+0000","last_fullsized":"2024-09-10T21:13:22.920226+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:18.535836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.921722+0000","last_change":"2024-09-10T21:13:21.899168+0000","last_active":"2024-09-10T21:13:22.921722+0000","last_peered":"2024-09-10T21:13:22.921722+0000","last_clean":"2024-09-10T21:13:22.921722+0000","last_became_active":"2024-09-10T21:13:21.898731+0000","last_became_peered":"2024-09-10T21:13:21.898731+0000","last_unstale":"2024-09-10T21:13:22.921722+0000","last_undegraded":"2024-09-10T21:13:22.921722+0000","last_fullsized":"2024-09-10T21:13:22.921722+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:21.312897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.923933+0000","last_change":"2024-09-10T21:13:22.922332+0000","last_active":"2024-09-10T21:13:22.923933+0000","last_peered":"2024-09-10T21:13:22.923933+0000","last_clean":"2024-09-10T21:13:22.923933+0000","last_became_active":"2024-09-10T21:13:22.922078+0000","last_became_peered":"2024-09-10T21:13:22.922078+0000","last_unstale":"2024-09-10T21:13:22.923933+0000","last_undegraded":"2024-09-10T21:13:22.923933+0000","last_fullsized":"2024-09-10T21:13:22.923933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:28.421242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.919719+0000","last_change":"2024-09-10T21:13:21.912484+0000","last_active":"2024-09-10T21:13:22.919719+0000","last_peered":"2024-09-10T21:13:22.919719+0000","last_clean":"2024-09-10T21:13:22.919719+0000","last_became_active":"2024-09-10T21:13:21.912214+0000","last_became_peered":"2024-09-10T21:13:21.912214+0000","last_unstale":"2024-09-10T21:13:22.919719+0000","last_undegraded":"2024-09-10T21:13:22.919719+0000","last_fullsized":"2024-09-10T21:13:22.919719+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:41.828958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922496+0000","last_change":"2024-09-10T21:13:22.919296+0000","last_active":"2024-09-10T21:13:22.922496+0000","last_peered":"2024-09-10T21:13:22.922496+0000","last_clean":"2024-09-10T21:13:22.922496+0000","last_became_active":"2024-09-10T21:13:22.919014+0000","last_became_peered":"2024-09-10T21:13:22.919014+0000","last_unstale":"2024-09-10T21:13:22.922496+0000","last_undegraded":"2024-09-10T21:13:22.922496+0000","last_fullsized":"2024-09-10T21:13:22.922496+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:10.730899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.915756+0000","last_change":"2024-09-10T21:13:21.906072+0000","last_active":"2024-09-10T21:13:22.915756+0000","last_peered":"2024-09-10T21:13:22.915756+0000","last_clean":"2024-09-10T21:13:22.915756+0000","last_became_active":"2024-09-10T21:13:21.905742+0000","last_became_peered":"2024-09-10T21:13:21.905742+0000","last_unstale":"2024-09-10T21:13:22.915756+0000","last_undegraded":"2024-09-10T21:13:22.915756+0000","last_fullsized":"2024-09-10T21:13:22.915756+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:42:26.027622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.910563+0000","last_change":"2024-09-10T21:13:22.903478+0000","last_active":"2024-09-10T21:13:22.910563+0000","last_peered":"2024-09-10T21:13:22.910563+0000","last_clean":"2024-09-10T21:13:22.910563+0000","last_became_active":"2024-09-10T21:13:22.903195+0000","last_became_peered":"2024-09-10T21:13:22.903195+0000","last_unstale":"2024-09-10T21:13:22.910563+0000","last_undegraded":"2024-09-10T21:13:22.910563+0000","last_fullsized":"2024-09-10T21:13:22.910563+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:08.970391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901193+0000","last_change":"2024-09-10T21:13:21.899616+0000","last_active":"2024-09-10T21:13:23.901193+0000","last_peered":"2024-09-10T21:13:23.901193+0000","last_clean":"2024-09-10T21:13:23.901193+0000","last_became_active":"2024-09-10T21:13:21.899337+0000","last_became_peered":"2024-09-10T21:13:21.899337+0000","last_unstale":"2024-09-10T21:13:23.901193+0000","last_undegraded":"2024-09-10T21:13:23.901193+0000","last_fullsized":"2024-09-10T21:13:23.901193+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:00.375700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.921826+0000","last_change":"2024-09-10T21:13:22.918519+0000","last_active":"2024-09-10T21:13:22.921826+0000","last_peered":"2024-09-10T21:13:22.921826+0000","last_clean":"2024-09-10T21:13:22.921826+0000","last_became_active":"2024-09-10T21:13:22.917932+0000","last_became_peered":"2024-09-10T21:13:22.917932+0000","last_unstale":"2024-09-10T21:13:22.921826+0000","last_undegraded":"2024-09-10T21:13:22.921826+0000","last_fullsized":"2024-09-10T21:13:22.921826+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:42.973697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901177+0000","last_change":"2024-09-10T21:13:21.888372+0000","last_active":"2024-09-10T21:13:23.901177+0000","last_peered":"2024-09-10T21:13:23.901177+0000","last_clean":"2024-09-10T21:13:23.901177+0000","last_became_active":"2024-09-10T21:13:21.888082+0000","last_became_peered":"2024-09-10T21:13:21.888082+0000","last_unstale":"2024-09-10T21:13:23.901177+0000","last_undegraded":"2024-09-10T21:13:23.901177+0000","last_fullsized":"2024-09-10T21:13:23.901177+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:10.740521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922244+0000","last_change":"2024-09-10T21:13:22.916504+0000","last_active":"2024-09-10T21:13:22.922244+0000","last_peered":"2024-09-10T21:13:22.922244+0000","last_clean":"2024-09-10T21:13:22.922244+0000","last_became_active":"2024-09-10T21:13:22.915699+0000","last_became_peered":"2024-09-10T21:13:22.915699+0000","last_unstale":"2024-09-10T21:13:22.922244+0000","last_undegraded":"2024-09-10T21:13:22.922244+0000","last_fullsized":"2024-09-10T21:13:22.922244+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:25.585671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.905804+0000","last_change":"2024-09-10T21:13:21.907164+0000","last_active":"2024-09-10T21:13:22.905804+0000","last_peered":"2024-09-10T21:13:22.905804+0000","last_clean":"2024-09-10T21:13:22.905804+0000","last_became_active":"2024-09-10T21:13:21.906610+0000","last_became_peered":"2024-09-10T21:13:21.906610+0000","last_unstale":"2024-09-10T21:13:22.905804+0000","last_undegraded":"2024-09-10T21:13:22.905804+0000","last_fullsized":"2024-09-10T21:13:22.905804+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:40.113319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902480+0000","last_change":"2024-09-10T21:13:22.898945+0000","last_active":"2024-09-10T21:13:23.902480+0000","last_peered":"2024-09-10T21:13:23.902480+0000","last_clean":"2024-09-10T21:13:23.902480+0000","last_became_active":"2024-09-10T21:13:22.898052+0000","last_became_peered":"2024-09-10T21:13:22.898052+0000","last_unstale":"2024-09-10T21:13:23.902480+0000","last_undegraded":"2024-09-10T21:13:23.902480+0000","last_fullsized":"2024-09-10T21:13:23.902480+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:55.405705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901250+0000","last_change":"2024-09-10T21:13:21.906141+0000","last_active":"2024-09-10T21:13:23.901250+0000","last_peered":"2024-09-10T21:13:23.901250+0000","last_clean":"2024-09-10T21:13:23.901250+0000","last_became_active":"2024-09-10T21:13:21.905746+0000","last_became_peered":"2024-09-10T21:13:21.905746+0000","last_unstale":"2024-09-10T21:13:23.901250+0000","last_undegraded":"2024-09-10T21:13:23.901250+0000","last_fullsized":"2024-09-10T21:13:23.901250+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:43.092284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902491+0000","last_change":"2024-09-10T21:13:22.916766+0000","last_active":"2024-09-10T21:13:23.902491+0000","last_peered":"2024-09-10T21:13:23.902491+0000","last_clean":"2024-09-10T21:13:23.902491+0000","last_became_active":"2024-09-10T21:13:22.916442+0000","last_became_peered":"2024-09-10T21:13:22.916442+0000","last_unstale":"2024-09-10T21:13:23.902491+0000","last_undegraded":"2024-09-10T21:13:23.902491+0000","last_fullsized":"2024-09-10T21:13:23.902491+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:36.488990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.921831+0000","last_change":"2024-09-10T21:13:21.903728+0000","last_active":"2024-09-10T21:13:22.921831+0000","last_peered":"2024-09-10T21:13:22.921831+0000","last_clean":"2024-09-10T21:13:22.921831+0000","last_became_active":"2024-09-10T21:13:21.902934+0000","last_became_peered":"2024-09-10T21:13:21.902934+0000","last_unstale":"2024-09-10T21:13:22.921831+0000","last_undegraded":"2024-09-10T21:13:22.921831+0000","last_fullsized":"2024-09-10T21:13:22.921831+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:47.969001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.923398+0000","last_change":"2024-09-10T21:13:22.919284+0000","last_active":"2024-09-10T21:13:22.923398+0000","last_peered":"2024-09-10T21:13:22.923398+0000","last_clean":"2024-09-10T21:13:22.923398+0000","last_became_active":"2024-09-10T21:13:22.918999+0000","last_became_peered":"2024-09-10T21:13:22.918999+0000","last_unstale":"2024-09-10T21:13:22.923398+0000","last_undegraded":"2024-09-10T21:13:22.923398+0000","last_fullsized":"2024-09-10T21:13:22.923398+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:12.826890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.889810+0000","last_change":"2024-09-10T21:13:21.908766+0000","last_active":"2024-09-10T21:13:23.889810+0000","last_peered":"2024-09-10T21:13:23.889810+0000","last_clean":"2024-09-10T21:13:23.889810+0000","last_became_active":"2024-09-10T21:13:21.908407+0000","last_became_peered":"2024-09-10T21:13:21.908407+0000","last_unstale":"2024-09-10T21:13:23.889810+0000","last_undegraded":"2024-09-10T21:13:23.889810+0000","last_fullsized":"2024-09-10T21:13:23.889810+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:50.408040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.915998+0000","last_change":"2024-09-10T21:13:22.912010+0000","last_active":"2024-09-10T21:13:22.915998+0000","last_peered":"2024-09-10T21:13:22.915998+0000","last_clean":"2024-09-10T21:13:22.915998+0000","last_became_active":"2024-09-10T21:13:22.911626+0000","last_became_peered":"2024-09-10T21:13:22.911626+0000","last_unstale":"2024-09-10T21:13:22.915998+0000","last_undegraded":"2024-09-10T21:13:22.915998+0000","last_fullsized":"2024-09-10T21:13:22.915998+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:57.866030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902425+0000","last_change":"2024-09-10T21:13:21.900741+0000","last_active":"2024-09-10T21:13:23.902425+0000","last_peered":"2024-09-10T21:13:23.902425+0000","last_clean":"2024-09-10T21:13:23.902425+0000","last_became_active":"2024-09-10T21:13:21.900316+0000","last_became_peered":"2024-09-10T21:13:21.900316+0000","last_unstale":"2024-09-10T21:13:23.902425+0000","last_undegraded":"2024-09-10T21:13:23.902425+0000","last_fullsized":"2024-09-10T21:13:23.902425+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:36.947920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.906343+0000","last_change":"2024-09-10T21:13:22.895387+0000","last_active":"2024-09-10T21:13:22.906343+0000","last_peered":"2024-09-10T21:13:22.906343+0000","last_clean":"2024-09-10T21:13:22.906343+0000","last_became_active":"2024-09-10T21:13:22.895123+0000","last_became_peered":"2024-09-10T21:13:22.895123+0000","last_unstale":"2024-09-10T21:13:22.906343+0000","last_undegraded":"2024-09-10T21:13:22.906343+0000","last_fullsized":"2024-09-10T21:13:22.906343+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:40.270334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.902958+0000","last_change":"2024-09-10T21:13:21.893643+0000","last_active":"2024-09-10T21:13:23.902958+0000","last_peered":"2024-09-10T21:13:23.902958+0000","last_clean":"2024-09-10T21:13:23.902958+0000","last_became_active":"2024-09-10T21:13:21.893383+0000","last_became_peered":"2024-09-10T21:13:21.893383+0000","last_unstale":"2024-09-10T21:13:23.902958+0000","last_undegraded":"2024-09-10T21:13:23.902958+0000","last_fullsized":"2024-09-10T21:13:23.902958+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:10.261088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901948+0000","last_change":"2024-09-10T21:13:22.900413+0000","last_active":"2024-09-10T21:13:23.901948+0000","last_peered":"2024-09-10T21:13:23.901948+0000","last_clean":"2024-09-10T21:13:23.901948+0000","last_became_active":"2024-09-10T21:13:22.899234+0000","last_became_peered":"2024-09-10T21:13:22.899234+0000","last_unstale":"2024-09-10T21:13:23.901948+0000","last_undegraded":"2024-09-10T21:13:23.901948+0000","last_fullsized":"2024-09-10T21:13:23.901948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:27.125309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901976+0000","last_change":"2024-09-10T21:13:21.895050+0000","last_active":"2024-09-10T21:13:23.901976+0000","last_peered":"2024-09-10T21:13:23.901976+0000","last_clean":"2024-09-10T21:13:23.901976+0000","last_became_active":"2024-09-10T21:13:21.892497+0000","last_became_peered":"2024-09-10T21:13:21.892497+0000","last_unstale":"2024-09-10T21:13:23.901976+0000","last_undegraded":"2024-09-10T21:13:23.901976+0000","last_fullsized":"2024-09-10T21:13:23.901976+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:40.609267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922237+0000","last_change":"2024-09-10T21:13:22.921138+0000","last_active":"2024-09-10T21:13:22.922237+0000","last_peered":"2024-09-10T21:13:22.922237+0000","last_clean":"2024-09-10T21:13:22.922237+0000","last_became_active":"2024-09-10T21:13:22.920172+0000","last_became_peered":"2024-09-10T21:13:22.920172+0000","last_unstale":"2024-09-10T21:13:22.922237+0000","last_undegraded":"2024-09-10T21:13:22.922237+0000","last_fullsized":"2024-09-10T21:13:22.922237+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:21.882085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901976+0000","last_change":"2024-09-10T21:13:21.894991+0000","last_active":"2024-09-10T21:13:23.901976+0000","last_peered":"2024-09-10T21:13:23.901976+0000","last_clean":"2024-09-10T21:13:23.901976+0000","last_became_active":"2024-09-10T21:13:21.894201+0000","last_became_peered":"2024-09-10T21:13:21.894201+0000","last_unstale":"2024-09-10T21:13:23.901976+0000","last_undegraded":"2024-09-10T21:13:23.901976+0000","last_fullsized":"2024-09-10T21:13:23.901976+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:16.375733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.922269+0000","last_change":"2024-09-10T21:13:22.917783+0000","last_active":"2024-09-10T21:13:22.922269+0000","last_peered":"2024-09-10T21:13:22.922269+0000","last_clean":"2024-09-10T21:13:22.922269+0000","last_became_active":"2024-09-10T21:13:22.917187+0000","last_became_peered":"2024-09-10T21:13:22.917187+0000","last_unstale":"2024-09-10T21:13:22.922269+0000","last_undegraded":"2024-09-10T21:13:22.922269+0000","last_fullsized":"2024-09-10T21:13:22.922269+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:35.277948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.905942+0000","last_change":"2024-09-10T21:13:21.906852+0000","last_active":"2024-09-10T21:13:22.905942+0000","last_peered":"2024-09-10T21:13:22.905942+0000","last_clean":"2024-09-10T21:13:22.905942+0000","last_became_active":"2024-09-10T21:13:21.906221+0000","last_became_peered":"2024-09-10T21:13:21.906221+0000","last_unstale":"2024-09-10T21:13:22.905942+0000","last_undegraded":"2024-09-10T21:13:22.905942+0000","last_fullsized":"2024-09-10T21:13:22.905942+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:03.864814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.918213+0000","last_change":"2024-09-10T21:13:22.914627+0000","last_active":"2024-09-10T21:13:22.918213+0000","last_peered":"2024-09-10T21:13:22.918213+0000","last_clean":"2024-09-10T21:13:22.918213+0000","last_became_active":"2024-09-10T21:13:22.914389+0000","last_became_peered":"2024-09-10T21:13:22.914389+0000","last_unstale":"2024-09-10T21:13:22.918213+0000","last_undegraded":"2024-09-10T21:13:22.918213+0000","last_fullsized":"2024-09-10T21:13:22.918213+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:25.289658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.900655+0000","last_change":"2024-09-10T21:13:21.891567+0000","last_active":"2024-09-10T21:13:23.900655+0000","last_peered":"2024-09-10T21:13:23.900655+0000","last_clean":"2024-09-10T21:13:23.900655+0000","last_became_active":"2024-09-10T21:13:21.891342+0000","last_became_peered":"2024-09-10T21:13:21.891342+0000","last_unstale":"2024-09-10T21:13:23.900655+0000","last_undegraded":"2024-09-10T21:13:23.900655+0000","last_fullsized":"2024-09-10T21:13:23.900655+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:38:26.642457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:13:23.901660+0000","last_change":"2024-09-10T21:13:22.907707+0000","last_active":"2024-09-10T21:13:23.901660+0000","last_peered":"2024-09-10T21:13:23.901660+0000","last_clean":"2024-09-10T21:13:23.901660+0000","last_became_active":"2024-09-10T21:13:22.907443+0000","last_became_peered":"2024-09-10T21:13:22.907443+0000","last_unstale":"2024-09-10T21:13:23.901660+0000","last_undegraded":"2024-09-10T21:13:23.901660+0000","last_fullsized":"2024-09-10T21:13:23.901660+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:21.871774+0000","last_clean_scrub_stamp":"2024-09-10T21:13:21.871774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:43.420059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.920849+0000","last_change":"2024-09-10T21:13:21.905186+0000","last_active":"2024-09-10T21:13:22.920849+0000","last_peered":"2024-09-10T21:13:22.920849+0000","last_clean":"2024-09-10T21:13:22.920849+0000","last_became_active":"2024-09-10T21:13:21.904923+0000","last_became_peered":"2024-09-10T21:13:21.904923+0000","last_unstale":"2024-09-10T21:13:22.920849+0000","last_undegraded":"2024-09-10T21:13:22.920849+0000","last_fullsized":"2024-09-10T21:13:22.920849+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:20.869025+0000","last_clean_scrub_stamp":"2024-09-10T21:13:20.869025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:09.229106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":193,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:13:22.890033+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:13:22.890033+0000","last_peered":"2024-09-10T21:13:22.890033+0000","last_clean":"2024-09-10T21:13:22.890033+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:13:22.890033+0000","last_undegraded":"2024-09-10T21:13:22.890033+0000","last_fullsized":"2024-09-10T21:13:22.890033+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":847872,"data_stored":350382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1429504,"data_stored":939194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":843776,"data_stored":349430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":843776,"data_stored":348862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":349318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":348806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36160,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335680,"statfs":{"total":96636764160,"available":96599736320,"internally_reserved":0,"allocated":819200,"data_stored":341961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":1425408,"data_stored":939104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:13:27.452 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:13:27.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:13:27.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.635+0000 7f2973200640 1 -- 172.21.15.16:0/3322668811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c0cbee0 msgr2=0x7f296c0cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.635+0000 7f2973200640 1 --2- 172.21.15.16:0/3322668811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c0cbee0 0x7f296c0cc390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2964009c80 tx=0x7f296401cda0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.635+0000 7f2973200640 1 -- 172.21.15.16:0/3322668811 shutdown_connections 2024-09-10T21:13:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.635+0000 7f2973200640 1 --2- 172.21.15.16:0/3322668811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c0cbee0 0x7f296c0cc390 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.635+0000 7f2973200640 1 --2- 172.21.15.16:0/3322668811 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f296c06f0e0 0x7f296c06f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.635+0000 7f2973200640 1 -- 172.21.15.16:0/3322668811 >> 172.21.15.16:0/3322668811 conn(0x7f296c0b16a0 msgr2=0x7f296c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.635+0000 7f2973200640 1 -- 172.21.15.16:0/3322668811 shutdown_connections 2024-09-10T21:13:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.635+0000 7f2973200640 1 -- 172.21.15.16:0/3322668811 wait complete. 2024-09-10T21:13:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.635+0000 7f2973200640 1 Processor -- start 2024-09-10T21:13:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.635+0000 7f2973200640 1 -- start start 2024-09-10T21:13:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2973200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c06f0e0 0x7f296c15b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2973200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f296c0cbb00 0x7f296c1f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2973200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296c0cbee0 0x7f296c15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2973200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f296c168d00 con 0x7f296c0cbee0 2024-09-10T21:13:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2973200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f296c168b80 con 0x7f296c06f0e0 2024-09-10T21:13:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2973200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f296c168e80 con 0x7f296c0cbb00 2024-09-10T21:13:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2971200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296c0cbee0 0x7f296c15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2971200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296c0cbee0 0x7f296c15ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41888/0 (socket says 172.21.15.16:41888) 2024-09-10T21:13:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2971200640 1 -- 172.21.15.16:0/464343949 learned_addr learned my addr 172.21.15.16:0/464343949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f296be00640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c06f0e0 0x7f296c15b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2963e00640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f296c0cbb00 0x7f296c1f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2971200640 1 -- 172.21.15.16:0/464343949 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f296c0cbb00 msgr2=0x7f296c1f8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2971200640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f296c0cbb00 0x7f296c1f8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2971200640 1 -- 172.21.15.16:0/464343949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c06f0e0 msgr2=0x7f296c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2971200640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c06f0e0 0x7f296c15b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2971200640 1 -- 172.21.15.16:0/464343949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c1fac10 con 0x7f296c0cbee0 2024-09-10T21:13:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2963e00640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f296c0cbb00 0x7f296c1f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f296be00640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c06f0e0 0x7f296c15b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2971200640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296c0cbee0 0x7f296c15ebf0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2964009bd0 tx=0x7f2964005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2968c00640 1 -- 172.21.15.16:0/464343949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29640bb070 con 0x7f296c0cbee0 2024-09-10T21:13:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2968c00640 1 -- 172.21.15.16:0/464343949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2964004b40 con 0x7f296c0cbee0 2024-09-10T21:13:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2968c00640 1 -- 172.21.15.16:0/464343949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29640b2040 con 0x7f296c0cbee0 2024-09-10T21:13:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2973200640 1 -- 172.21.15.16:0/464343949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f296c1faef0 con 0x7f296c0cbee0 2024-09-10T21:13:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2973200640 1 -- 172.21.15.16:0/464343949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f296c1fb4b0 con 0x7f296c0cbee0 2024-09-10T21:13:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2973200640 1 -- 172.21.15.16:0/464343949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f296c06f520 con 0x7f296c0cbee0 2024-09-10T21:13:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.639+0000 7f2968c00640 1 -- 172.21.15.16:0/464343949 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f29640b5030 con 0x7f296c0cbee0 2024-09-10T21:13:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.643+0000 7f2968c00640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f29480725e0 0x7f2948074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.643+0000 7f2968c00640 1 -- 172.21.15.16:0/464343949 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f296412cbe0 con 0x7f296c0cbee0 2024-09-10T21:13:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.643+0000 7f296be00640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f29480725e0 0x7f2948074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.643+0000 7f296be00640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f29480725e0 0x7f2948074aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f295c001750 tx=0x7f295c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.647+0000 7f2968c00640 1 -- 172.21.15.16:0/464343949 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2964016990 con 0x7f296c0cbee0 2024-09-10T21:13:27.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.887+0000 7f2973200640 1 -- 172.21.15.16:0/464343949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f296c145390 con 0x7f296c0cbee0 2024-09-10T21:13:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.891+0000 7f2968c00640 1 -- 172.21.15.16:0/464343949 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) ==== 190+0+147 (secure 0 0 0) 0x7f29640f82b0 con 0x7f296c0cbee0 2024-09-10T21:13:27.898 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-10T21:13:27.898 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA== 2024-09-10T21:13:27.898 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-10T21:13:27.898 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-10T21:13:27.898 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:13:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-10T21:13:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.891+0000 7f2973200640 1 -- 172.21.15.16:0/464343949 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f29480725e0 msgr2=0x7f2948074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.891+0000 7f2973200640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f29480725e0 0x7f2948074aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f295c001750 tx=0x7f295c009040 comp rx=0 tx=0).stop 2024-09-10T21:13:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.891+0000 7f2973200640 1 -- 172.21.15.16:0/464343949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296c0cbee0 msgr2=0x7f296c15ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.891+0000 7f2973200640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296c0cbee0 0x7f296c15ebf0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2964009bd0 tx=0x7f2964005b00 comp rx=0 tx=0).stop 2024-09-10T21:13:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.891+0000 7f2973200640 1 -- 172.21.15.16:0/464343949 shutdown_connections 2024-09-10T21:13:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.891+0000 7f2973200640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f29480725e0 0x7f2948074aa0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.891+0000 7f2973200640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296c0cbee0 0x7f296c15ebf0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.891+0000 7f2973200640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f296c0cbb00 0x7f296c1f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.891+0000 7f2973200640 1 --2- 172.21.15.16:0/464343949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c06f0e0 0x7f296c15b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.891+0000 7f2973200640 1 -- 172.21.15.16:0/464343949 >> 172.21.15.16:0/464343949 conn(0x7f296c0b16a0 msgr2=0x7f296c13cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.891+0000 7f2973200640 1 -- 172.21.15.16:0/464343949 shutdown_connections 2024-09-10T21:13:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:27.891+0000 7f2973200640 1 -- 172.21.15.16:0/464343949 wait complete. 2024-09-10T21:13:27.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:28.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.059+0000 7fa461600640 1 -- 172.21.15.16:0/1570121601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c0716f0 msgr2=0x7fa45c0af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.059+0000 7fa461600640 1 --2- 172.21.15.16:0/1570121601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c0716f0 0x7fa45c0af580 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0cc360 tx=0x7fa44401c7a0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.059+0000 7fa461600640 1 -- 172.21.15.16:0/1570121601 shutdown_connections 2024-09-10T21:13:28.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.059+0000 7fa461600640 1 --2- 172.21.15.16:0/1570121601 >> v2:172.21.15.52:3300/0 conn(0x7fa45c0b13c0 0x7fa45c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.059+0000 7fa461600640 1 --2- 172.21.15.16:0/1570121601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c0716f0 0x7fa45c0af580 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.059+0000 7fa461600640 1 -- 172.21.15.16:0/1570121601 >> 172.21.15.16:0/1570121601 conn(0x7fa45c09b130 msgr2=0x7fa45c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.059+0000 7fa461600640 1 -- 172.21.15.16:0/1570121601 shutdown_connections 2024-09-10T21:13:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.059+0000 7fa461600640 1 -- 172.21.15.16:0/1570121601 wait complete. 2024-09-10T21:13:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa461600640 1 Processor -- start 2024-09-10T21:13:28.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa461600640 1 -- start start 2024-09-10T21:13:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa461600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa45c0716f0 0x7fa45c147140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa461600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa45c0b13c0 0x7fa45c0b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa461600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c15f400 0x7fa45c143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa461600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c168f60 con 0x7fa45c15f400 2024-09-10T21:13:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa461600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c168de0 con 0x7fa45c0b13c0 2024-09-10T21:13:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa461600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa45c1690e0 con 0x7fa45c0716f0 2024-09-10T21:13:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa45b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c15f400 0x7fa45c143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa45b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c15f400 0x7fa45c143ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41906/0 (socket says 172.21.15.16:41906) 2024-09-10T21:13:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa45b400640 1 -- 172.21.15.16:0/4074480570 learned_addr learned my addr 172.21.15.16:0/4074480570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa45aa00640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa45c0716f0 0x7fa45c147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa453e00640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa45c0b13c0 0x7fa45c0b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa45b400640 1 -- 172.21.15.16:0/4074480570 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa45c0716f0 msgr2=0x7fa45c147140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa45b400640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa45c0716f0 0x7fa45c147140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa45b400640 1 -- 172.21.15.16:0/4074480570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa45c0b13c0 msgr2=0x7fa45c0b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa45b400640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa45c0b13c0 0x7fa45c0b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa45b400640 1 -- 172.21.15.16:0/4074480570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45c0b5b00 con 0x7fa45c15f400 2024-09-10T21:13:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa453e00640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa45c0b13c0 0x7fa45c0b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa45b400640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c15f400 0x7fa45c143ba0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa45400ea20 tx=0x7fa45400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa452a00640 1 -- 172.21.15.16:0/4074480570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45400cde0 con 0x7fa45c15f400 2024-09-10T21:13:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa452a00640 1 -- 172.21.15.16:0/4074480570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa454004c40 con 0x7fa45c15f400 2024-09-10T21:13:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa452a00640 1 -- 172.21.15.16:0/4074480570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa454005280 con 0x7fa45c15f400 2024-09-10T21:13:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa461600640 1 -- 172.21.15.16:0/4074480570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45c0b5da0 con 0x7fa45c15f400 2024-09-10T21:13:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa461600640 1 -- 172.21.15.16:0/4074480570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45c0b6360 con 0x7fa45c15f400 2024-09-10T21:13:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.063+0000 7fa461600640 1 -- 172.21.15.16:0/4074480570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa45c070fe0 con 0x7fa45c15f400 2024-09-10T21:13:28.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.067+0000 7fa452a00640 1 -- 172.21.15.16:0/4074480570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa45400f040 con 0x7fa45c15f400 2024-09-10T21:13:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.067+0000 7fa452a00640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa438072610 0x7fa438074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.067+0000 7fa452a00640 1 -- 172.21.15.16:0/4074480570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fa454093950 con 0x7fa45c15f400 2024-09-10T21:13:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.067+0000 7fa45aa00640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa438072610 0x7fa438074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.067+0000 7fa45aa00640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa438072610 0x7fa438074ad0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa44c006fd0 tx=0x7fa44c008040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:28.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.071+0000 7fa452a00640 1 -- 172.21.15.16:0/4074480570 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa454016030 con 0x7fa45c15f400 2024-09-10T21:13:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.279+0000 7fa461600640 1 -- 172.21.15.16:0/4074480570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa45c070c00 con 0x7fa45c15f400 2024-09-10T21:13:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.279+0000 7fa452a00640 1 -- 172.21.15.16:0/4074480570 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5234 (secure 0 0 0) 0x7fa45405ef90 con 0x7fa45c15f400 2024-09-10T21:13:28.287 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:28.288 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-10T21:13:25:905095+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4029516332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4029516332},{"type":"v1","addr":"172.21.15.16:6836","nonce":4029516332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/806881470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":806881470},{"type":"v1","addr":"172.21.15.16:6837","nonce":806881470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":135167,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2949376643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2949376643},{"type":"v1","addr":"172.21.15.52:6834","nonce":2949376643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:22.294493+0000","modified":"2024-09-10T21:13:25.298310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135170},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135170":{"gid":135170,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/1574608409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1574608409},{"type":"v1","addr":"172.21.15.52:6835","nonce":1574608409}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135170,"qdb_cluster":[135170]},"id":5}]} 2024-09-10T21:13:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-10T21:13:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.283+0000 7fa461600640 1 -- 172.21.15.16:0/4074480570 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa438072610 msgr2=0x7fa438074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.283+0000 7fa461600640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa438072610 0x7fa438074ad0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa44c006fd0 tx=0x7fa44c008040 comp rx=0 tx=0).stop 2024-09-10T21:13:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.283+0000 7fa461600640 1 -- 172.21.15.16:0/4074480570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c15f400 msgr2=0x7fa45c143ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.283+0000 7fa461600640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c15f400 0x7fa45c143ba0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa45400ea20 tx=0x7fa45400eee0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.283+0000 7fa461600640 1 -- 172.21.15.16:0/4074480570 shutdown_connections 2024-09-10T21:13:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.283+0000 7fa461600640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa438072610 0x7fa438074ad0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.283+0000 7fa461600640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c15f400 0x7fa45c143ba0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.283+0000 7fa461600640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa45c0b13c0 0x7fa45c0b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.283+0000 7fa461600640 1 --2- 172.21.15.16:0/4074480570 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa45c0716f0 0x7fa45c147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.283+0000 7fa461600640 1 -- 172.21.15.16:0/4074480570 >> 172.21.15.16:0/4074480570 conn(0x7fa45c09b130 msgr2=0x7fa45c0cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:28.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.283+0000 7fa461600640 1 -- 172.21.15.16:0/4074480570 shutdown_connections 2024-09-10T21:13:28.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.283+0000 7fa461600640 1 -- 172.21.15.16:0/4074480570 wait complete. 2024-09-10T21:13:28.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:13:22.294493+0000', 'modified': '2024-09-10T21:13:25.298310+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 135170}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_135170': {'gid': 135170, 'name': 'b', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.52:6835/1574608409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6833', 'nonce': 1574608409}, {'type': 'v1', 'addr': '172.21.15.52:6835', 'nonce': 1574608409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 135170, 'qdb_cluster': [135170]} 2024-09-10T21:13:28.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:13:28.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T21:13:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.855+0000 7faf21600640 1 -- 172.21.15.16:0/3927470719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf1c15bcb0 msgr2=0x7faf1c0b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.855+0000 7faf21600640 1 --2- 172.21.15.16:0/3927470719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf1c15bcb0 0x7faf1c0b1780 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0b30c0 tx=0x7faf0c01c600 comp rx=0 tx=0).stop 2024-09-10T21:13:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.855+0000 7faf21600640 1 -- 172.21.15.16:0/3927470719 shutdown_connections 2024-09-10T21:13:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.855+0000 7faf21600640 1 --2- 172.21.15.16:0/3927470719 >> v2:172.21.15.52:3300/0 conn(0x7faf1c0d1d30 0x7faf1c0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.855+0000 7faf21600640 1 --2- 172.21.15.16:0/3927470719 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf1c0d0750 0x7faf1c0d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.855+0000 7faf21600640 1 --2- 172.21.15.16:0/3927470719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf1c15bcb0 0x7faf1c0b1780 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.855+0000 7faf21600640 1 -- 172.21.15.16:0/3927470719 >> 172.21.15.16:0/3927470719 conn(0x7faf1c09a9d0 msgr2=0x7faf1c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:28.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.855+0000 7faf21600640 1 -- 172.21.15.16:0/3927470719 shutdown_connections 2024-09-10T21:13:28.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.855+0000 7faf21600640 1 -- 172.21.15.16:0/3927470719 wait complete. 2024-09-10T21:13:28.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf21600640 1 Processor -- start 2024-09-10T21:13:28.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf21600640 1 -- start start 2024-09-10T21:13:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf21600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf1c0d0750 0x7faf1c0b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf21600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf1c0d1d30 0x7faf1c0b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf21600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf1c15bcb0 0x7faf1c0c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf21600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c0cd170 con 0x7faf1c0d1d30 2024-09-10T21:13:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf21600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c0ccff0 con 0x7faf1c15bcb0 2024-09-10T21:13:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf21600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c0cd2f0 con 0x7faf1c0d0750 2024-09-10T21:13:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf13400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf1c0d1d30 0x7faf1c0b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf13400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf1c0d1d30 0x7faf1c0b63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41924/0 (socket says 172.21.15.16:41924) 2024-09-10T21:13:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf13400640 1 -- 172.21.15.16:0/3666772453 learned_addr learned my addr 172.21.15.16:0/3666772453 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:28.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf20c00640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf1c15bcb0 0x7faf1c0c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf1be00640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf1c0d0750 0x7faf1c0b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf13400640 1 -- 172.21.15.16:0/3666772453 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf1c0d0750 msgr2=0x7faf1c0b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf13400640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf1c0d0750 0x7faf1c0b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf13400640 1 -- 172.21.15.16:0/3666772453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf1c15bcb0 msgr2=0x7faf1c0c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf13400640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf1c15bcb0 0x7faf1c0c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf13400640 1 -- 172.21.15.16:0/3666772453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c1f8030 con 0x7faf1c0d1d30 2024-09-10T21:13:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf20c00640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf1c15bcb0 0x7faf1c0c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf13400640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf1c0d1d30 0x7faf1c0b63e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7faf0400b550 tx=0x7faf0400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf18c00640 1 -- 172.21.15.16:0/3666772453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04014070 con 0x7faf1c0d1d30 2024-09-10T21:13:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf18c00640 1 -- 172.21.15.16:0/3666772453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf04010980 con 0x7faf1c0d1d30 2024-09-10T21:13:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf18c00640 1 -- 172.21.15.16:0/3666772453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04010dd0 con 0x7faf1c0d1d30 2024-09-10T21:13:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf21600640 1 -- 172.21.15.16:0/3666772453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1c0b6a10 con 0x7faf1c0d1d30 2024-09-10T21:13:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.859+0000 7faf21600640 1 -- 172.21.15.16:0/3666772453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1c0b6fd0 con 0x7faf1c0d1d30 2024-09-10T21:13:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.863+0000 7faf18c00640 1 -- 172.21.15.16:0/3666772453 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faf0400f3c0 con 0x7faf1c0d1d30 2024-09-10T21:13:28.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.863+0000 7faf18c00640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faef80725e0 0x7faef8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.863+0000 7faf18c00640 1 -- 172.21.15.16:0/3666772453 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7faf04093360 con 0x7faf1c0d1d30 2024-09-10T21:13:28.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.863+0000 7faf26020000 1 Processor -- start 2024-09-10T21:13:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.863+0000 7faf1be00640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faef80725e0 0x7faef8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.863+0000 7faf26020000 1 -- start start 2024-09-10T21:13:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.863+0000 7faf1be00640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faef80725e0 0x7faef8074aa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0af000 tx=0x7faf0c01ae40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.863+0000 7faf26020000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373c9d0 0x558c1373cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.863+0000 7faf26020000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c1373d2f0 0x558c136f08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.863+0000 7faf26020000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 0x558c136f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.863+0000 7faf26020000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558c136113f0 con 0x558c136f0e10 2024-09-10T21:13:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.863+0000 7faf26020000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558c1354eb20 con 0x558c1373d2f0 2024-09-10T21:13:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.863+0000 7faf26020000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x558c1353df50 con 0x558c1373c9d0 2024-09-10T21:13:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf20c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 0x558c136f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf13400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c1373d2f0 0x558c136f08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf1be00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373c9d0 0x558c1373cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf20c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 0x558c136f11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41936/0 (socket says 172.21.15.16:41936) 2024-09-10T21:13:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf20c00640 1 -- 172.21.15.16:0/2973129964 learned_addr learned my addr 172.21.15.16:0/2973129964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf20c00640 1 -- 172.21.15.16:0/2973129964 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373c9d0 msgr2=0x558c1373cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf20c00640 1 --2- 172.21.15.16:0/2973129964 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373c9d0 0x558c1373cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf20c00640 1 -- 172.21.15.16:0/2973129964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c1373d2f0 msgr2=0x558c136f08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf20c00640 1 --2- 172.21.15.16:0/2973129964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c1373d2f0 0x558c136f08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf20c00640 1 -- 172.21.15.16:0/2973129964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c13505590 con 0x558c136f0e10 2024-09-10T21:13:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf20c00640 1 --2- 172.21.15.16:0/2973129964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 0x558c136f11f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7faf1400aeb0 tx=0x7faf1400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c98529ed670c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf10c00640 1 -- 172.21.15.16:0/2973129964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14010840 con 0x558c136f0e10 2024-09-10T21:13:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf10c00640 1 -- 172.21.15.16:0/2973129964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf14010ad0 con 0x558c136f0e10 2024-09-10T21:13:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf10c00640 1 -- 172.21.15.16:0/2973129964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1401a410 con 0x558c136f0e10 2024-09-10T21:13:28.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 -- 172.21.15.16:0/2973129964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 msgr2=0x558c136f11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 --2- 172.21.15.16:0/2973129964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 0x558c136f11f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7faf1400aeb0 tx=0x7faf1400e720 comp rx=0 tx=0).stop 2024-09-10T21:13:28.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 -- 172.21.15.16:0/2973129964 shutdown_connections 2024-09-10T21:13:28.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 --2- 172.21.15.16:0/2973129964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 0x558c136f11f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 --2- 172.21.15.16:0/2973129964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c1373d2f0 0x558c136f08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 --2- 172.21.15.16:0/2973129964 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373c9d0 0x558c1373cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 -- 172.21.15.16:0/2973129964 >> 172.21.15.16:0/2973129964 conn(0x558c135da130 msgr2=0x558c1371fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:28.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 -- 172.21.15.16:0/2973129964 shutdown_connections 2024-09-10T21:13:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 -- 172.21.15.16:0/2973129964 wait complete. 2024-09-10T21:13:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 -- start start 2024-09-10T21:13:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 Processor -- start 2024-09-10T21:13:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 0x558c136f2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c1373c9d0 0x558c136f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 0x558c136f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558c134d85d0 con 0x558c136f0e10 2024-09-10T21:13:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558c13541d60 con 0x558c1373c9d0 2024-09-10T21:13:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x558c134e4500 con 0x558c1373d2f0 2024-09-10T21:13:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf26020000 10 client.0 _ll_register_callbacks cb 0x558c134e8810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.867+0000 7faf1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 0x558c136f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 0x558c136f2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41938/0 (socket says 172.21.15.16:41938) 2024-09-10T21:13:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf1be00640 1 -- 172.21.15.16:0/138016622 learned_addr learned my addr 172.21.15.16:0/138016622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf13400640 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c1373c9d0 0x558c136f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf20c00640 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 0x558c136f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf1be00640 1 -- 172.21.15.16:0/138016622 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 msgr2=0x558c136f2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf1be00640 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 0x558c136f2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf1be00640 1 -- 172.21.15.16:0/138016622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c1373c9d0 msgr2=0x558c136f2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf1be00640 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c1373c9d0 0x558c136f2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf1be00640 1 -- 172.21.15.16:0/138016622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c134dc170 con 0x558c136f0e10 2024-09-10T21:13:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf13400640 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c1373c9d0 0x558c136f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf1be00640 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 0x558c136f2240 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7faf0c01acb0 tx=0x7faf0c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c07da27df2386 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf09600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:13:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf09600640 1 -- 172.21.15.16:0/138016622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c005660 con 0x558c136f0e10 2024-09-10T21:13:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf09600640 1 -- 172.21.15.16:0/138016622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7faf0c0b8030 con 0x558c136f0e10 2024-09-10T21:13:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf26020000 1 -- 172.21.15.16:0/138016622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558c134d60f0 con 0x558c136f0e10 2024-09-10T21:13:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf09600640 1 -- 172.21.15.16:0/138016622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7faf0c003600 con 0x558c136f0e10 2024-09-10T21:13:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf26020000 10 client.6063 fetch_fsmap learned FSMap version 41 2024-09-10T21:13:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf26020000 1 -- 172.21.15.16:0/138016622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x558c135c2ef0 con 0x558c136f0e10 2024-09-10T21:13:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf09600640 1 -- 172.21.15.16:0/138016622 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x558c134d60f0 con 0x558c136f0e10 2024-09-10T21:13:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf09600640 1 -- 172.21.15.16:0/138016622 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 41) ==== 3502+0+0 (secure 0 0 0) 0x7faf0c0d2050 con 0x558c136f0e10 2024-09-10T21:13:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf26020000 10 client.6063 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T21:13:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf26020000 10 client.6063 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:13:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf26020000 20 client.6063 populate_metadata read hostname 'smithi016' 2024-09-10T21:13:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf26020000 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.52:6833/1574608409,v1:172.21.15.52:6835/1574608409] conn(0x558c1374bc70 0x558c1374e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf26020000 4 client.6063 mds_command: new command op to 135170 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:13:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf26020000 1 -- 172.21.15.16:0/138016622 --> [v2:172.21.15.52:6833/1574608409,v1:172.21.15.52:6835/1574608409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c13550a70 con 0x558c1374bc70 2024-09-10T21:13:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf13400640 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.52:6833/1574608409,v1:172.21.15.52:6835/1574608409] conn(0x558c1374bc70 0x558c1374e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf13400640 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.52:6833/1574608409,v1:172.21.15.52:6835/1574608409] conn(0x558c1374bc70 0x558c1374e060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8bb23092909ce85 server_cookie=b2696ed886359847 in_seq=0 out_seq=0 2024-09-10T21:13:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf09600640 10 client.6063 ms_handle_connect on v2:172.21.15.52:6833/1574608409 2024-09-10T21:13:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf09600640 1 -- 172.21.15.16:0/138016622 <== mds.0 v2:172.21.15.52:6833/1574608409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c13550a70 con 0x558c1374bc70 2024-09-10T21:13:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf09600640 10 client.6063 handle_command_reply: tid=0 2024-09-10T21:13:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.871+0000 7faf26020000 1 client.6063 shutdown 2024-09-10T21:13:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.875+0000 7faf26020000 1 -- 172.21.15.16:0/138016622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 msgr2=0x558c136f2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.875+0000 7faf26020000 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 0x558c136f2240 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7faf0c01acb0 tx=0x7faf0c009260 comp rx=0 tx=0).stop 2024-09-10T21:13:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.875+0000 7faf26020000 1 -- 172.21.15.16:0/138016622 shutdown_connections 2024-09-10T21:13:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.875+0000 7faf26020000 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.52:6833/1574608409,v1:172.21.15.52:6835/1574608409] conn(0x558c1374bc70 0x558c1374e060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.875+0000 7faf26020000 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 0x558c136f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.875+0000 7faf26020000 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c1373c9d0 0x558c136f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.875+0000 7faf26020000 1 --2- 172.21.15.16:0/138016622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c136f0e10 0x558c136f2240 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.875+0000 7faf09600640 0 client.6063 ms_handle_reset on v2:172.21.15.52:6833/1574608409 2024-09-10T21:13:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.875+0000 7faf26020000 1 -- 172.21.15.16:0/138016622 >> 172.21.15.16:0/138016622 conn(0x558c135da130 msgr2=0x558c136effd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.875+0000 7faf26020000 1 -- 172.21.15.16:0/138016622 shutdown_connections 2024-09-10T21:13:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.875+0000 7faf26020000 1 -- 172.21.15.16:0/138016622 wait complete. 2024-09-10T21:13:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.875+0000 7faf26020000 20 client.6063 trim_cache size 0 max 16384 2024-09-10T21:13:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.891+0000 7faf26020000 1 Processor -- start 2024-09-10T21:13:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.891+0000 7faf26020000 1 -- start start 2024-09-10T21:13:28.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.891+0000 7faf26020000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c136f0e10 0x558c137ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.891+0000 7faf26020000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 0x558c137ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.891+0000 7faf26020000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 0x558c137aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.891+0000 7faf26020000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c2030f0 con 0x558c1373c9d0 2024-09-10T21:13:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.891+0000 7faf26020000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c0b5430 con 0x558c136f0e10 2024-09-10T21:13:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.891+0000 7faf26020000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c202b50 con 0x558c1373d2f0 2024-09-10T21:13:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf13400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 0x558c137ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf13400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 0x558c137ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41944/0 (socket says 172.21.15.16:41944) 2024-09-10T21:13:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf13400640 1 -- 172.21.15.16:0/765692824 learned_addr learned my addr 172.21.15.16:0/765692824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf20c00640 1 --2- 172.21.15.16:0/765692824 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 0x558c137aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf1be00640 1 --2- 172.21.15.16:0/765692824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c136f0e10 0x558c137ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf13400640 1 -- 172.21.15.16:0/765692824 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 msgr2=0x558c137aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf13400640 1 --2- 172.21.15.16:0/765692824 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 0x558c137aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf13400640 1 -- 172.21.15.16:0/765692824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c136f0e10 msgr2=0x558c137ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf13400640 1 --2- 172.21.15.16:0/765692824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c136f0e10 0x558c137ae3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf13400640 1 -- 172.21.15.16:0/765692824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c13599d90 con 0x558c1373c9d0 2024-09-10T21:13:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf1be00640 1 --2- 172.21.15.16:0/765692824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c136f0e10 0x558c137ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf20c00640 1 --2- 172.21.15.16:0/765692824 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 0x558c137aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf13400640 1 --2- 172.21.15.16:0/765692824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 0x558c137ae900 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7faf0400b520 tx=0x7faf040047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ade6b4b9533452f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf0be00640 1 -- 172.21.15.16:0/765692824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0401e040 con 0x558c1373c9d0 2024-09-10T21:13:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf0be00640 1 -- 172.21.15.16:0/765692824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf04083030 con 0x558c1373c9d0 2024-09-10T21:13:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf0be00640 1 -- 172.21.15.16:0/765692824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0405a5b0 con 0x558c1373c9d0 2024-09-10T21:13:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 -- 172.21.15.16:0/765692824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 msgr2=0x558c137ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 --2- 172.21.15.16:0/765692824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 0x558c137ae900 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7faf0400b520 tx=0x7faf040047e0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 -- 172.21.15.16:0/765692824 shutdown_connections 2024-09-10T21:13:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 --2- 172.21.15.16:0/765692824 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 0x558c137aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 --2- 172.21.15.16:0/765692824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 0x558c137ae900 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 --2- 172.21.15.16:0/765692824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c136f0e10 0x558c137ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 -- 172.21.15.16:0/765692824 >> 172.21.15.16:0/765692824 conn(0x558c135da130 msgr2=0x558c13753e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 -- 172.21.15.16:0/765692824 shutdown_connections 2024-09-10T21:13:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 -- 172.21.15.16:0/765692824 wait complete. 2024-09-10T21:13:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 -- start start 2024-09-10T21:13:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 Processor -- start 2024-09-10T21:13:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c136f0e10 0x558c13752a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 0x558c1376e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 0x558c1376eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558c13541070 con 0x558c1373c9d0 2024-09-10T21:13:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558c13546390 con 0x558c136f0e10 2024-09-10T21:13:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x558c1354d8f0 con 0x558c1373d2f0 2024-09-10T21:13:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf26020000 10 client.0 _ll_register_callbacks cb 0x7faf0c004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf13400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 0x558c1376e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf13400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 0x558c1376e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41946/0 (socket says 172.21.15.16:41946) 2024-09-10T21:13:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf13400640 1 -- 172.21.15.16:0/339464955 learned_addr learned my addr 172.21.15.16:0/339464955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf1be00640 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c136f0e10 0x558c13752a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.895+0000 7faf20c00640 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 0x558c1376eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf13400640 1 -- 172.21.15.16:0/339464955 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 msgr2=0x558c1376eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf13400640 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 0x558c1376eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf13400640 1 -- 172.21.15.16:0/339464955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c136f0e10 msgr2=0x558c13752a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf13400640 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c136f0e10 0x558c13752a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf13400640 1 -- 172.21.15.16:0/339464955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c13504a90 con 0x558c1373c9d0 2024-09-10T21:13:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf1be00640 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c136f0e10 0x558c13752a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf13400640 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 0x558c1376e580 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7faf0400bf10 tx=0x7faf0405dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd010942ca1dc2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf09600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:13:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf09600640 1 -- 172.21.15.16:0/339464955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0401e040 con 0x558c1373c9d0 2024-09-10T21:13:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf09600640 1 -- 172.21.15.16:0/339464955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7faf040831e0 con 0x558c1373c9d0 2024-09-10T21:13:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 -- 172.21.15.16:0/339464955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf04083030 con 0x558c1373c9d0 2024-09-10T21:13:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf09600640 1 -- 172.21.15.16:0/339464955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7faf04093180 con 0x558c1373c9d0 2024-09-10T21:13:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 10 client.6069 fetch_fsmap learned FSMap version 41 2024-09-10T21:13:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 -- 172.21.15.16:0/339464955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x558c134e3050 con 0x558c1373c9d0 2024-09-10T21:13:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf09600640 1 -- 172.21.15.16:0/339464955 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf04083030 con 0x558c1373c9d0 2024-09-10T21:13:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf09600640 1 -- 172.21.15.16:0/339464955 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 41) ==== 3502+0+0 (secure 0 0 0) 0x7faf04091350 con 0x558c1373c9d0 2024-09-10T21:13:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 10 client.6069 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T21:13:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 10 client.6069 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:13:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 20 client.6069 populate_metadata read hostname 'smithi016' 2024-09-10T21:13:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.52:6833/1574608409,v1:172.21.15.52:6835/1574608409] conn(0x558c1376f000 0x558c137b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 4 client.6069 mds_command: new command op to 135170 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:13:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 -- 172.21.15.16:0/339464955 --> [v2:172.21.15.52:6833/1574608409,v1:172.21.15.52:6835/1574608409] -- command(tid 0: {"prefix": "status"}) -- 0x558c1354c520 con 0x558c1376f000 2024-09-10T21:13:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf1be00640 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.52:6833/1574608409,v1:172.21.15.52:6835/1574608409] conn(0x558c1376f000 0x558c137b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf1be00640 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.52:6833/1574608409,v1:172.21.15.52:6835/1574608409] conn(0x558c1376f000 0x558c137b2610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=172332df33356c7e server_cookie=8849b5b31d4b7615 in_seq=0 out_seq=0 2024-09-10T21:13:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf09600640 10 client.6069 ms_handle_connect on v2:172.21.15.52:6833/1574608409 2024-09-10T21:13:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf09600640 1 -- 172.21.15.16:0/339464955 <== mds.0 v2:172.21.15.52:6833/1574608409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x558c1354c520 con 0x558c1376f000 2024-09-10T21:13:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf09600640 10 client.6069 handle_command_reply: tid=0 2024-09-10T21:13:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 client.6069 shutdown 2024-09-10T21:13:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 -- 172.21.15.16:0/339464955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 msgr2=0x558c1376e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 0x558c1376e580 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7faf0400bf10 tx=0x7faf0405dab0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 -- 172.21.15.16:0/339464955 shutdown_connections 2024-09-10T21:13:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.52:6833/1574608409,v1:172.21.15.52:6835/1574608409] conn(0x558c1376f000 0x558c137b2610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x558c1373d2f0 0x558c1376eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1373c9d0 0x558c1376e580 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 --2- 172.21.15.16:0/339464955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c136f0e10 0x558c13752a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf09600640 0 client.6069 ms_handle_reset on v2:172.21.15.52:6833/1574608409 2024-09-10T21:13:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 -- 172.21.15.16:0/339464955 >> 172.21.15.16:0/339464955 conn(0x558c135da130 msgr2=0x558c136f0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 -- 172.21.15.16:0/339464955 shutdown_connections 2024-09-10T21:13:28.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 1 -- 172.21.15.16:0/339464955 wait complete. 2024-09-10T21:13:28.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf26020000 20 client.6069 trim_cache size 0 max 16384 2024-09-10T21:13:28.913 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:13:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "eb19be60-6d8f-4816-b778-3d7d6698304d", 2024-09-10T21:13:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-10T21:13:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 135170, 2024-09-10T21:13:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-10T21:13:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-10T21:13:28.913 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-10T21:13:28.914 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.0231798230000004, 2024-09-10T21:13:28.914 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 41, 2024-09-10T21:13:28.914 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 70, 2024-09-10T21:13:28.914 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 70, 2024-09-10T21:13:28.914 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 10.444992751999999 2024-09-10T21:13:28.914 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:13:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf21600640 1 -- 172.21.15.16:0/3666772453 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faef80725e0 msgr2=0x7faef8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf21600640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faef80725e0 0x7faef8074aa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0af000 tx=0x7faf0c01ae40 comp rx=0 tx=0).stop 2024-09-10T21:13:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf21600640 1 -- 172.21.15.16:0/3666772453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf1c0d1d30 msgr2=0x7faf1c0b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf21600640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf1c0d1d30 0x7faf1c0b63e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7faf0400b550 tx=0x7faf0400ba10 comp rx=0 tx=0).stop 2024-09-10T21:13:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf21600640 1 -- 172.21.15.16:0/3666772453 shutdown_connections 2024-09-10T21:13:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf21600640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faef80725e0 0x7faef8074aa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf21600640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf1c15bcb0 0x7faf1c0c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf21600640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf1c0d1d30 0x7faf1c0b63e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf21600640 1 --2- 172.21.15.16:0/3666772453 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf1c0d0750 0x7faf1c0b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf21600640 1 -- 172.21.15.16:0/3666772453 >> 172.21.15.16:0/3666772453 conn(0x7faf1c09a9d0 msgr2=0x7faf1c06ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf21600640 1 -- 172.21.15.16:0/3666772453 shutdown_connections 2024-09-10T21:13:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:28.899+0000 7faf21600640 1 -- 172.21.15.16:0/3666772453 wait complete. 2024-09-10T21:13:28.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:13:29.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.067+0000 7f0f87200640 1 -- 172.21.15.16:0/2464105009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f880cfd40 msgr2=0x7f0f880d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:29.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.067+0000 7f0f87200640 1 --2- 172.21.15.16:0/2464105009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f880cfd40 0x7f0f880d0120 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009840 tx=0x7f0f7801c6b0 comp rx=0 tx=0).stop 2024-09-10T21:13:29.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.067+0000 7f0f87200640 1 -- 172.21.15.16:0/2464105009 shutdown_connections 2024-09-10T21:13:29.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.067+0000 7f0f87200640 1 --2- 172.21.15.16:0/2464105009 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0f880d2820 0x7f0f8815fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:29.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.067+0000 7f0f87200640 1 --2- 172.21.15.16:0/2464105009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f880cfd40 0x7f0f880d0120 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:29.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.067+0000 7f0f87200640 1 -- 172.21.15.16:0/2464105009 >> 172.21.15.16:0/2464105009 conn(0x7f0f880d1d00 msgr2=0x7f0f880d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:29.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.067+0000 7f0f87200640 1 -- 172.21.15.16:0/2464105009 shutdown_connections 2024-09-10T21:13:29.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.067+0000 7f0f87200640 1 -- 172.21.15.16:0/2464105009 wait complete. 2024-09-10T21:13:29.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.067+0000 7f0f87200640 1 Processor -- start 2024-09-10T21:13:29.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.067+0000 7f0f87200640 1 -- start start 2024-09-10T21:13:29.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f87200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f880cfd40 0x7f0f881fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:29.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f87200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0f880d2820 0x7f0f880c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:29.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f87200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f88160120 0x7f0f880b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:29.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f87200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f88169360 con 0x7f0f88160120 2024-09-10T21:13:29.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f87200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881691e0 con 0x7f0f880cfd40 2024-09-10T21:13:29.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f87200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0f881694e0 con 0x7f0f880d2820 2024-09-10T21:13:29.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f86800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f88160120 0x7f0f880b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f86800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f88160120 0x7f0f880b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41958/0 (socket says 172.21.15.16:41958) 2024-09-10T21:13:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f86800640 1 -- 172.21.15.16:0/1581900035 learned_addr learned my addr 172.21.15.16:0/1581900035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f85e00640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f880cfd40 0x7f0f881fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f7fe00640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0f880d2820 0x7f0f880c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f86800640 1 -- 172.21.15.16:0/1581900035 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0f880d2820 msgr2=0x7f0f880c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f86800640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0f880d2820 0x7f0f880c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f86800640 1 -- 172.21.15.16:0/1581900035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f880cfd40 msgr2=0x7f0f881fbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f86800640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f880cfd40 0x7f0f881fbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f86800640 1 -- 172.21.15.16:0/1581900035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f881fb1f0 con 0x7f0f88160120 2024-09-10T21:13:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f86800640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f88160120 0x7f0f880b3780 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000cbc0 tx=0x7f0f8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f7e000640 1 -- 172.21.15.16:0/1581900035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80016020 con 0x7f0f88160120 2024-09-10T21:13:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f7e000640 1 -- 172.21.15.16:0/1581900035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f8000ab60 con 0x7f0f88160120 2024-09-10T21:13:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f87200640 1 -- 172.21.15.16:0/1581900035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f880ca530 con 0x7f0f88160120 2024-09-10T21:13:29.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f7e000640 1 -- 172.21.15.16:0/1581900035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80012420 con 0x7f0f88160120 2024-09-10T21:13:29.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f87200640 1 -- 172.21.15.16:0/1581900035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f881fb490 con 0x7f0f88160120 2024-09-10T21:13:29.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.071+0000 7f0f87200640 1 -- 172.21.15.16:0/1581900035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f8815fbe0 con 0x7f0f88160120 2024-09-10T21:13:29.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.075+0000 7f0f7e000640 1 -- 172.21.15.16:0/1581900035 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0f8001b400 con 0x7f0f88160120 2024-09-10T21:13:29.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.075+0000 7f0f7e000640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0f640725e0 0x7f0f64074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:29.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.075+0000 7f0f7e000640 1 -- 172.21.15.16:0/1581900035 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f0f80093830 con 0x7f0f88160120 2024-09-10T21:13:29.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.075+0000 7f0f85e00640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0f640725e0 0x7f0f64074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:29.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.075+0000 7f0f85e00640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0f640725e0 0x7f0f64074aa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0f780af000 tx=0x7f0f7801aef0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:29.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.079+0000 7f0f7e000640 1 -- 172.21.15.16:0/1581900035 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0f80020020 con 0x7f0f88160120 2024-09-10T21:13:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.291+0000 7f0f87200640 1 -- 172.21.15.16:0/1581900035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0f8815f770 con 0x7f0f88160120 2024-09-10T21:13:29.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.291+0000 7f0f7e000640 1 -- 172.21.15.16:0/1581900035 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) ==== 53+0+73 (secure 0 0 0) 0x7f0f8005e820 con 0x7f0f88160120 2024-09-10T21:13:29.299 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:13:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.291+0000 7f0f87200640 1 -- 172.21.15.16:0/1581900035 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0f640725e0 msgr2=0x7f0f64074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.291+0000 7f0f87200640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0f640725e0 0x7f0f64074aa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0f780af000 tx=0x7f0f7801aef0 comp rx=0 tx=0).stop 2024-09-10T21:13:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.291+0000 7f0f87200640 1 -- 172.21.15.16:0/1581900035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f88160120 msgr2=0x7f0f880b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.291+0000 7f0f87200640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f88160120 0x7f0f880b3780 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000cbc0 tx=0x7f0f8000ccb0 comp rx=0 tx=0).stop 2024-09-10T21:13:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.291+0000 7f0f87200640 1 -- 172.21.15.16:0/1581900035 shutdown_connections 2024-09-10T21:13:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.291+0000 7f0f87200640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0f640725e0 0x7f0f64074aa0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.291+0000 7f0f87200640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f88160120 0x7f0f880b3780 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.291+0000 7f0f87200640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0f880d2820 0x7f0f880c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.291+0000 7f0f87200640 1 --2- 172.21.15.16:0/1581900035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f880cfd40 0x7f0f881fbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.291+0000 7f0f87200640 1 -- 172.21.15.16:0/1581900035 >> 172.21.15.16:0/1581900035 conn(0x7f0f880d1d00 msgr2=0x7f0f88071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.295+0000 7f0f87200640 1 -- 172.21.15.16:0/1581900035 shutdown_connections 2024-09-10T21:13:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29.295+0000 7f0f87200640 1 -- 172.21.15.16:0/1581900035 wait complete. 2024-09-10T21:13:29.320 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:13:29.320 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:13:29.320 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-10T21:13:29.320 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:13:29.320 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:13:29.320 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:13:29.320 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:13:29.320 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:13:29.320 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:13:29.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:29.321 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-10T21:13:29.329 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:13:29.330 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:13:29.330 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:13:29.330 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:29.330 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:13:29.330 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:29.330 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:13:29.330 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:29.330 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-10T21:13:29.330 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:13:29.330 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:29.330 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-10T21:13:29.331 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:13:29.331 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:29.331 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-10T21:13:29.331 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:13:29.331 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42240sec preferred_lft 42240sec 2024-09-10T21:13:29.331 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-10T21:13:29.331 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:29.331 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:13:29.331 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:29.331 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-10T21:13:29.331 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:13:29.331 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:30:fb:74:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:29.331 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:13:29.332 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:29.332 INFO:teuthology.orchestra.run.smithi016.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:13:29.332 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:29.332 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:13:29.332 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:29.332 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:13:29.332 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:29.332 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:13:29.332 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:13:29.332 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:13:29.332 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:29.332 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:13:29.333 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:13:29.333 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:13:29.333 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:29.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:29.334 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:13:29.397 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-10T21:13:29.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:29.397 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-10T21:13:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-10T21:13:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:13:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:13:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:13:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:13:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:13:29.451 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:13:29.451 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:13:29.451 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:13:29.451 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:13:29.451 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:13:29.451 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:13:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:13:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:13:29.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:29.593 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:13:29.602 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:13:29.602 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:13:29.602 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:13:29.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:29.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-10T21:13:29.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:13:29.665 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:13:29.717 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:13:29.717 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:13:29.717 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:13:29.768 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:13:29.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:29.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:13:29.858 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:13:29.858 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:13:29.872 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:13:29.872 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:13:29.873 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:13:29.873 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:13:29.873 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:13:29.876 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:13:29.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:29.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:13:29.898 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:13:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:13:30.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:13:30.008 INFO:teuthology.orchestra.run.smithi016.stdout:client6078 2024-09-10T21:13:30.008 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:30.009 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:30.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:13:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.203+0000 7f76b1000640 1 -- 172.21.15.16:0/3346484158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac0cc140 msgr2=0x7f76ac0b02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.203+0000 7f76b1000640 1 --2- 172.21.15.16:0/3346484158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac0cc140 0x7f76ac0b02f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0b1c90 tx=0x7f769801f0b0 comp rx=0 tx=0).stop 2024-09-10T21:13:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.203+0000 7f76b1000640 1 -- 172.21.15.16:0/3346484158 shutdown_connections 2024-09-10T21:13:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.203+0000 7f76b1000640 1 --2- 172.21.15.16:0/3346484158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac0cc140 0x7f76ac0b02f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.203+0000 7f76b1000640 1 --2- 172.21.15.16:0/3346484158 >> v2:172.21.15.52:3300/0 conn(0x7f76ac15f0e0 0x7f76ac068990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.203+0000 7f76b1000640 1 -- 172.21.15.16:0/3346484158 >> 172.21.15.16:0/3346484158 conn(0x7f76ac0b1d40 msgr2=0x7f76ac0b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.203+0000 7f76b1000640 1 -- 172.21.15.16:0/3346484158 shutdown_connections 2024-09-10T21:13:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.203+0000 7f76b1000640 1 -- 172.21.15.16:0/3346484158 wait complete. 2024-09-10T21:13:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.203+0000 7f76b1000640 1 Processor -- start 2024-09-10T21:13:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.203+0000 7f76b1000640 1 -- start start 2024-09-10T21:13:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76b1000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76ac0b0830 0x7f76ac15db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76b1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac0cc140 0x7f76ac15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76b1000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76ac15f0e0 0x7f76ac1f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76b1000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac06da50 con 0x7f76ac0cc140 2024-09-10T21:13:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76b1000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac06d8d0 con 0x7f76ac0b0830 2024-09-10T21:13:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76b1000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac15f4c0 con 0x7f76ac15f0e0 2024-09-10T21:13:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76aaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac0cc140 0x7f76ac15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76aaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac0cc140 0x7f76ac15b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41976/0 (socket says 172.21.15.16:41976) 2024-09-10T21:13:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76aaa00640 1 -- 172.21.15.16:0/342198330 learned_addr learned my addr 172.21.15.16:0/342198330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76aaa00640 1 -- 172.21.15.16:0/342198330 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76ac15f0e0 msgr2=0x7f76ac1f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76aaa00640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76ac15f0e0 0x7f76ac1f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76aaa00640 1 -- 172.21.15.16:0/342198330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76ac0b0830 msgr2=0x7f76ac15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76ab400640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76ac0b0830 0x7f76ac15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76aaa00640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76ac0b0830 0x7f76ac15db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76aaa00640 1 -- 172.21.15.16:0/342198330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac15b6c0 con 0x7f76ac0cc140 2024-09-10T21:13:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76ab400640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76ac0b0830 0x7f76ac15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76aaa00640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac0cc140 0x7f76ac15b180 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f76980b3040 tx=0x7f76980a8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76a3400640 1 -- 172.21.15.16:0/342198330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76980b5040 con 0x7f76ac0cc140 2024-09-10T21:13:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76a3400640 1 -- 172.21.15.16:0/342198330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7698005340 con 0x7f76ac0cc140 2024-09-10T21:13:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76a3400640 1 -- 172.21.15.16:0/342198330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76980046d0 con 0x7f76ac0cc140 2024-09-10T21:13:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76b1000640 1 -- 172.21.15.16:0/342198330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ac1f5860 con 0x7f76ac0cc140 2024-09-10T21:13:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76b1000640 1 -- 172.21.15.16:0/342198330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ac1f5b50 con 0x7f76ac0cc140 2024-09-10T21:13:30.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76b1000640 1 -- 172.21.15.16:0/342198330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ac0b02f0 con 0x7f76ac0cc140 2024-09-10T21:13:30.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.207+0000 7f76a3400640 1 -- 172.21.15.16:0/342198330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7698004870 con 0x7f76ac0cc140 2024-09-10T21:13:30.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.211+0000 7f76a3400640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f768c0725d0 0x7f768c074a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:30.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.211+0000 7f76a3400640 1 -- 172.21.15.16:0/342198330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f7698130130 con 0x7f76ac0cc140 2024-09-10T21:13:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.211+0000 7f76ab400640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f768c0725d0 0x7f768c074a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.211+0000 7f76ab400640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f768c0725d0 0x7f768c074a90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f769c00a870 tx=0x7f769c008040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.215+0000 7f76a3400640 1 -- 172.21.15.16:0/342198330 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7698018c10 con 0x7f76ac0cc140 2024-09-10T21:13:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.423+0000 7f76b1000640 1 -- 172.21.15.16:0/342198330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f76ac06e8b0 con 0x7f76ac0cc140 2024-09-10T21:13:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.427+0000 7f76a3400640 1 -- 172.21.15.16:0/342198330 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) ==== 53+0+73 (secure 0 0 0) 0x7f76980fb780 con 0x7f76ac0cc140 2024-09-10T21:13:30.434 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:13:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.427+0000 7f76b1000640 1 -- 172.21.15.16:0/342198330 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f768c0725d0 msgr2=0x7f768c074a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.427+0000 7f76b1000640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f768c0725d0 0x7f768c074a90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f769c00a870 tx=0x7f769c008040 comp rx=0 tx=0).stop 2024-09-10T21:13:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.427+0000 7f76b1000640 1 -- 172.21.15.16:0/342198330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac0cc140 msgr2=0x7f76ac15b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.427+0000 7f76b1000640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac0cc140 0x7f76ac15b180 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f76980b3040 tx=0x7f76980a8de0 comp rx=0 tx=0).stop 2024-09-10T21:13:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.427+0000 7f76b1000640 1 -- 172.21.15.16:0/342198330 shutdown_connections 2024-09-10T21:13:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.427+0000 7f76b1000640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f768c0725d0 0x7f768c074a90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.427+0000 7f76b1000640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76ac15f0e0 0x7f76ac1f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.427+0000 7f76b1000640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac0cc140 0x7f76ac15b180 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.427+0000 7f76b1000640 1 --2- 172.21.15.16:0/342198330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76ac0b0830 0x7f76ac15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.427+0000 7f76b1000640 1 -- 172.21.15.16:0/342198330 >> 172.21.15.16:0/342198330 conn(0x7f76ac0b1d40 msgr2=0x7f76ac06fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.427+0000 7f76b1000640 1 -- 172.21.15.16:0/342198330 shutdown_connections 2024-09-10T21:13:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:30.427+0000 7f76b1000640 1 -- 172.21.15.16:0/342198330 wait complete. 2024-09-10T21:13:30.454 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:13:30.455 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:13:30.455 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-09-10T21:13:30.455 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T21:13:30.455 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:13:30.455 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:13:30.455 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:13:30.455 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:13:30.455 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:13:30.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:30.455 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-09-10T21:13:30.466 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:13:30.466 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:13:30.466 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:13:30.466 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:30.466 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:13:30.466 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:30.466 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:13:30.467 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:30.467 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-09-10T21:13:30.467 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:13:30.467 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:30.467 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-09-10T21:13:30.467 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:13:30.467 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:30.467 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-09-10T21:13:30.467 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:13:30.467 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42240sec preferred_lft 42240sec 2024-09-10T21:13:30.467 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-09-10T21:13:30.467 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:13:30.468 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:30.468 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-09-10T21:13:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:13:30.468 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:7e:b4:71:a2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:30.468 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:13:30.468 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:30.468 INFO:teuthology.orchestra.run.smithi052.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:13:30.468 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:13:30.468 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:13:30.469 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:30.469 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:13:30.469 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:30.469 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:13:30.469 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:13:30.469 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:13:30.469 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:30.469 INFO:teuthology.orchestra.run.smithi052.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:13:30.469 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:13:30.469 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:13:30.469 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:13:30.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:30.470 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:13:30.533 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-09-10T21:13:30.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:30.535 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-09-10T21:13:30.587 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-09-10T21:13:30.587 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:13:30.587 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:13:30.587 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:13:30.587 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:13:30.587 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:13:30.587 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:13:30.587 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:13:30.588 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:13:30.588 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:13:30.588 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:13:30.588 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:13:30.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:13:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:13:30.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:13:30.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:30.735 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:13:30.745 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:13:30.745 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:13:30.746 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:13:30.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:30.746 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-09-10T21:13:30.809 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:13:30.809 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T21:13:30.865 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:13:30.865 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:13:30.865 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:13:30.917 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T21:13:30.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:30.917 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:13:31.011 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:13:31.011 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:13:31.026 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:13:31.026 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:13:31.026 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:13:31.026 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 1@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:13:31.026 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:13:31.030 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:13:31.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:31.030 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:13:31.050 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:13:31.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:13:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:13:31.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:13:31.150 INFO:teuthology.orchestra.run.smithi052.stdout:client6087 2024-09-10T21:13:31.150 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:13:31.151 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:31.160 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-10T21:13:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 -- 172.21.15.16:0/3409211999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac4073750 msgr2=0x7f7ac40d1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 --2- 172.21.15.16:0/3409211999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac4073750 0x7f7ac40d1750 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40b0560 tx=0x7f7ab401c7a0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 -- 172.21.15.16:0/3409211999 shutdown_connections 2024-09-10T21:13:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 --2- 172.21.15.16:0/3409211999 >> v2:172.21.15.52:3300/0 conn(0x7f7ac4074240 0x7f7ac40cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 --2- 172.21.15.16:0/3409211999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac4073750 0x7f7ac40d1750 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 -- 172.21.15.16:0/3409211999 >> 172.21.15.16:0/3409211999 conn(0x7f7ac40d0c30 msgr2=0x7f7ac40d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 -- 172.21.15.16:0/3409211999 shutdown_connections 2024-09-10T21:13:31.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 -- 172.21.15.16:0/3409211999 wait complete. 2024-09-10T21:13:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 Processor -- start 2024-09-10T21:13:31.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 -- start start 2024-09-10T21:13:31.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ac406f750 0x7f7ac40b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ac4073750 0x7f7ac4155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac4074240 0x7f7ac4156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac416adb0 con 0x7f7ac4074240 2024-09-10T21:13:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac416ac30 con 0x7f7ac4073750 2024-09-10T21:13:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac416af30 con 0x7f7ac406f750 2024-09-10T21:13:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7ac3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac4074240 0x7f7ac4156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7ac3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac4074240 0x7f7ac4156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41994/0 (socket says 172.21.15.16:41994) 2024-09-10T21:13:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7ac3e00640 1 -- 172.21.15.16:0/3964003864 learned_addr learned my addr 172.21.15.16:0/3964003864 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7abbe00640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ac4073750 0x7f7ac4155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7ac3400640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ac406f750 0x7f7ac40b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7ac3e00640 1 -- 172.21.15.16:0/3964003864 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ac406f750 msgr2=0x7f7ac40b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7ac3e00640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ac406f750 0x7f7ac40b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7ac3e00640 1 -- 172.21.15.16:0/3964003864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ac4073750 msgr2=0x7f7ac4155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7ac3e00640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ac4073750 0x7f7ac4155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7ac3e00640 1 -- 172.21.15.16:0/3964003864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac40b6120 con 0x7f7ac4074240 2024-09-10T21:13:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7ac3400640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ac406f750 0x7f7ac40b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7ac3e00640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac4074240 0x7f7ac4156100 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00dac0 tx=0x7f7abc00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7abb400640 1 -- 172.21.15.16:0/3964003864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc017070 con 0x7f7ac4074240 2024-09-10T21:13:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7abb400640 1 -- 172.21.15.16:0/3964003864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7abc004b30 con 0x7f7ac4074240 2024-09-10T21:13:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7abb400640 1 -- 172.21.15.16:0/3964003864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc005020 con 0x7f7ac4074240 2024-09-10T21:13:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.311+0000 7f7aca800640 1 -- 172.21.15.16:0/3964003864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac40b5080 con 0x7f7ac4074240 2024-09-10T21:13:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.315+0000 7f7aca800640 1 -- 172.21.15.16:0/3964003864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac40b5640 con 0x7f7ac4074240 2024-09-10T21:13:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.315+0000 7f7aca800640 1 -- 172.21.15.16:0/3964003864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac40d1750 con 0x7f7ac4074240 2024-09-10T21:13:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.315+0000 7f7abb400640 1 -- 172.21.15.16:0/3964003864 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7abc0051c0 con 0x7f7ac4074240 2024-09-10T21:13:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.315+0000 7f7abb400640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7aa00725e0 0x7f7aa0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.315+0000 7f7abb400640 1 -- 172.21.15.16:0/3964003864 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f7abc093600 con 0x7f7ac4074240 2024-09-10T21:13:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.315+0000 7f7ac3400640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7aa00725e0 0x7f7aa0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.315+0000 7f7ac3400640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7aa00725e0 0x7f7aa0074aa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40af000 tx=0x7f7ab401afe0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:31.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.319+0000 7f7abb400640 1 -- 172.21.15.16:0/3964003864 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7abc020070 con 0x7f7ac4074240 2024-09-10T21:13:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.495+0000 7f7aca800640 1 -- 172.21.15.16:0/3964003864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7f7ac40ccf30 con 0x7f7ac4074240 2024-09-10T21:13:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.495+0000 7f7abb400640 1 -- 172.21.15.16:0/3964003864 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v36) ==== 106+0+4 (secure 0 0 0) 0x7f7abc05ecd0 con 0x7f7ac4074240 2024-09-10T21:13:31.503 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-10T21:13:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.495+0000 7f7aca800640 1 -- 172.21.15.16:0/3964003864 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7aa00725e0 msgr2=0x7f7aa0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.495+0000 7f7aca800640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7aa00725e0 0x7f7aa0074aa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40af000 tx=0x7f7ab401afe0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.495+0000 7f7aca800640 1 -- 172.21.15.16:0/3964003864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac4074240 msgr2=0x7f7ac4156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.495+0000 7f7aca800640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac4074240 0x7f7ac4156100 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00dac0 tx=0x7f7abc00df80 comp rx=0 tx=0).stop 2024-09-10T21:13:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.499+0000 7f7aca800640 1 -- 172.21.15.16:0/3964003864 shutdown_connections 2024-09-10T21:13:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.499+0000 7f7aca800640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7aa00725e0 0x7f7aa0074aa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.499+0000 7f7aca800640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ac4074240 0x7f7ac4156100 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.499+0000 7f7aca800640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ac4073750 0x7f7ac4155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.499+0000 7f7aca800640 1 --2- 172.21.15.16:0/3964003864 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ac406f750 0x7f7ac40b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.499+0000 7f7aca800640 1 -- 172.21.15.16:0/3964003864 >> 172.21.15.16:0/3964003864 conn(0x7f7ac40d0c30 msgr2=0x7f7ac406fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.499+0000 7f7aca800640 1 -- 172.21.15.16:0/3964003864 shutdown_connections 2024-09-10T21:13:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.499+0000 7f7aca800640 1 -- 172.21.15.16:0/3964003864 wait complete. 2024-09-10T21:13:31.524 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 200 2024-09-10T21:13:31.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.667+0000 7f9704c00640 1 -- 172.21.15.16:0/990247726 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97000ce8c0 msgr2=0x7f97000ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:31.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.667+0000 7f9704c00640 1 --2- 172.21.15.16:0/990247726 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97000ce8c0 0x7f97000ceca0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009900 tx=0x7f96f001c8b0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.667+0000 7f9704c00640 1 -- 172.21.15.16:0/990247726 shutdown_connections 2024-09-10T21:13:31.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.667+0000 7f9704c00640 1 --2- 172.21.15.16:0/990247726 >> v2:172.21.15.52:3300/0 conn(0x7f97000cf1e0 0x7f97000af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.667+0000 7f9704c00640 1 --2- 172.21.15.16:0/990247726 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97000ce8c0 0x7f97000ceca0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.667+0000 7f9704c00640 1 -- 172.21.15.16:0/990247726 >> 172.21.15.16:0/990247726 conn(0x7f97000d1c70 msgr2=0x7f97000d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:31.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.667+0000 7f9704c00640 1 -- 172.21.15.16:0/990247726 shutdown_connections 2024-09-10T21:13:31.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.667+0000 7f9704c00640 1 -- 172.21.15.16:0/990247726 wait complete. 2024-09-10T21:13:31.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.667+0000 7f9704c00640 1 Processor -- start 2024-09-10T21:13:31.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.667+0000 7f9704c00640 1 -- start start 2024-09-10T21:13:31.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.667+0000 7f9704c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97000b0980 0x7f970006f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.667+0000 7f9704c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97000ce8c0 0x7f97001f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f9704c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97000cf1e0 0x7f9700166400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f9704c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9700169270 con 0x7f97000cf1e0 2024-09-10T21:13:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f9704c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97001690f0 con 0x7f97000ce8c0 2024-09-10T21:13:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f9704c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f97001693f0 con 0x7f97000b0980 2024-09-10T21:13:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96ff400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97000b0980 0x7f970006f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96ffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97000cf1e0 0x7f9700166400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96ff400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97000b0980 0x7f970006f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:51212/0 (socket says 172.21.15.16:51212) 2024-09-10T21:13:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96ff400640 1 -- 172.21.15.16:0/551218436 learned_addr learned my addr 172.21.15.16:0/551218436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96f6a00640 1 --2- 172.21.15.16:0/551218436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97000ce8c0 0x7f97001f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96ff400640 1 -- 172.21.15.16:0/551218436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97000ce8c0 msgr2=0x7f97001f8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96ff400640 1 --2- 172.21.15.16:0/551218436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97000ce8c0 0x7f97001f8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96ff400640 1 -- 172.21.15.16:0/551218436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97000cf1e0 msgr2=0x7f9700166400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:31.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96ff400640 1 --2- 172.21.15.16:0/551218436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97000cf1e0 0x7f9700166400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96ff400640 1 -- 172.21.15.16:0/551218436 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97001f90a0 con 0x7f97000b0980 2024-09-10T21:13:31.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96ff400640 1 --2- 172.21.15.16:0/551218436 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97000b0980 0x7f970006f6b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009850 tx=0x7f96f0005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96f7e00640 1 -- 172.21.15.16:0/551218436 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f00b2040 con 0x7f97000b0980 2024-09-10T21:13:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96f7e00640 1 -- 172.21.15.16:0/551218436 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96f00053e0 con 0x7f97000b0980 2024-09-10T21:13:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f9704c00640 1 -- 172.21.15.16:0/551218436 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9700166940 con 0x7f97000b0980 2024-09-10T21:13:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96f7e00640 1 -- 172.21.15.16:0/551218436 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f00046d0 con 0x7f97000b0980 2024-09-10T21:13:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f9704c00640 1 -- 172.21.15.16:0/551218436 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f970006e8f0 con 0x7f97000b0980 2024-09-10T21:13:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f9704c00640 1 -- 172.21.15.16:0/551218436 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97000ceca0 con 0x7f97000b0980 2024-09-10T21:13:31.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.671+0000 7f96f7e00640 1 -- 172.21.15.16:0/551218436 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f96f0004890 con 0x7f97000b0980 2024-09-10T21:13:31.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.675+0000 7f96f7e00640 1 --2- 172.21.15.16:0/551218436 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f96dc0725e0 0x7f96dc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:31.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.675+0000 7f96f7e00640 1 -- 172.21.15.16:0/551218436 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f96f012d940 con 0x7f97000b0980 2024-09-10T21:13:31.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.675+0000 7f96f6a00640 1 --2- 172.21.15.16:0/551218436 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f96dc0725e0 0x7f96dc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:31.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.675+0000 7f96f6a00640 1 --2- 172.21.15.16:0/551218436 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f96dc0725e0 0x7f96dc074aa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f970015a4d0 tx=0x7f96e8009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:31.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.679+0000 7f96f7e00640 1 -- 172.21.15.16:0/551218436 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9700166940 con 0x7f97000b0980 2024-09-10T21:13:31.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.855+0000 7f9704c00640 1 -- 172.21.15.16:0/551218436 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) -- 0x7f97000afd90 con 0x7f97000b0980 2024-09-10T21:13:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.859+0000 7f96f7e00640 1 -- 172.21.15.16:0/551218436 <== mon.2 v2:172.21.15.52:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v37)=0 v37) ==== 123+0+0 (secure 0 0 0) 0x7f97001f90a0 con 0x7f97000b0980 2024-09-10T21:13:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.863+0000 7f9704c00640 1 -- 172.21.15.16:0/551218436 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f96dc0725e0 msgr2=0x7f96dc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.863+0000 7f9704c00640 1 --2- 172.21.15.16:0/551218436 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f96dc0725e0 0x7f96dc074aa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f970015a4d0 tx=0x7f96e8009040 comp rx=0 tx=0).stop 2024-09-10T21:13:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.863+0000 7f9704c00640 1 -- 172.21.15.16:0/551218436 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97000b0980 msgr2=0x7f970006f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:31.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.863+0000 7f9704c00640 1 --2- 172.21.15.16:0/551218436 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97000b0980 0x7f970006f6b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009850 tx=0x7f96f0005e70 comp rx=0 tx=0).stop 2024-09-10T21:13:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.863+0000 7f9704c00640 1 -- 172.21.15.16:0/551218436 shutdown_connections 2024-09-10T21:13:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.863+0000 7f9704c00640 1 --2- 172.21.15.16:0/551218436 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f96dc0725e0 0x7f96dc074aa0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.863+0000 7f9704c00640 1 --2- 172.21.15.16:0/551218436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97000cf1e0 0x7f9700166400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.863+0000 7f9704c00640 1 --2- 172.21.15.16:0/551218436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97000ce8c0 0x7f97001f8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.863+0000 7f9704c00640 1 --2- 172.21.15.16:0/551218436 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97000b0980 0x7f970006f6b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.863+0000 7f9704c00640 1 -- 172.21.15.16:0/551218436 >> 172.21.15.16:0/551218436 conn(0x7f97000d1c70 msgr2=0x7f97000b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.863+0000 7f9704c00640 1 -- 172.21.15.16:0/551218436 shutdown_connections 2024-09-10T21:13:31.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:31.863+0000 7f9704c00640 1 -- 172.21.15.16:0/551218436 wait complete. 2024-09-10T21:13:31.889 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:31.889 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:31.899 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-10T21:13:31.899 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-10T21:13:31.899 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-10T21:13:31.899 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:13:31.899 DEBUG:teuthology.orchestra.run.smithi016:> n = 600 2024-09-10T21:13:31.899 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-10T21:13:31.899 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-10T21:13:31.899 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-10T21:13:31.899 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/foo/" 2024-09-10T21:13:31.899 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:13:31.899 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-10T21:13:31.899 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-10T21:13:31.900 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:13:31.901 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-10T21:13:31.901 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-10T21:13:31.901 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-10T21:13:31.901 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-10T21:13:31.901 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-10T21:13:31.901 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-10T21:13:31.901 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-10T21:13:31.901 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-10T21:13:31.901 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-10T21:13:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:13:35.040 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:35.040 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:35.089 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:13:35.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:13:35.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:13:35.191 INFO:teuthology.orchestra.run.smithi016.stdout:client6078 2024-09-10T21:13:35.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:35.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.327+0000 7f8ae7a00640 1 -- 172.21.15.16:0/4010411664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ae00700a0 msgr2=0x7f8ae00ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:35.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.327+0000 7f8ae7a00640 1 --2- 172.21.15.16:0/4010411664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ae00700a0 0x7f8ae00ca170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009bc0 tx=0x7f8ad801c950 comp rx=0 tx=0).stop 2024-09-10T21:13:35.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae7a00640 1 -- 172.21.15.16:0/4010411664 shutdown_connections 2024-09-10T21:13:35.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae7a00640 1 --2- 172.21.15.16:0/4010411664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ae00700a0 0x7f8ae00ca170 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:35.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae7a00640 1 -- 172.21.15.16:0/4010411664 >> 172.21.15.16:0/4010411664 conn(0x7f8ae00cfb20 msgr2=0x7f8ae00cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:35.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae7a00640 1 -- 172.21.15.16:0/4010411664 shutdown_connections 2024-09-10T21:13:35.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae7a00640 1 -- 172.21.15.16:0/4010411664 wait complete. 2024-09-10T21:13:35.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae7a00640 1 Processor -- start 2024-09-10T21:13:35.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae7a00640 1 -- start start 2024-09-10T21:13:35.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae7a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ae00700a0 0x7f8ae01543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:35.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae7a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8ae00d06b0 0x7f8ae01639b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:35.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae7a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ae015a270 0x7f8ae00b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:35.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae7a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae0169800 con 0x7f8ae00700a0 2024-09-10T21:13:35.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae7a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae0169680 con 0x7f8ae015a270 2024-09-10T21:13:35.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae7a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae0169980 con 0x7f8ae00d06b0 2024-09-10T21:13:35.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ae00700a0 0x7f8ae01543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:35.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ae00700a0 0x7f8ae01543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42378/0 (socket says 172.21.15.16:42378) 2024-09-10T21:13:35.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae5000640 1 -- 172.21.15.16:0/905710219 learned_addr learned my addr 172.21.15.16:0/905710219 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:35.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae5a00640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ae015a270 0x7f8ae00b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:35.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8adfe00640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8ae00d06b0 0x7f8ae01639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:35.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae5000640 1 -- 172.21.15.16:0/905710219 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8ae00d06b0 msgr2=0x7f8ae01639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:35.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae5000640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8ae00d06b0 0x7f8ae01639b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:35.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae5000640 1 -- 172.21.15.16:0/905710219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ae015a270 msgr2=0x7f8ae00b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:35.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae5000640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ae015a270 0x7f8ae00b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:35.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae5000640 1 -- 172.21.15.16:0/905710219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae0155790 con 0x7f8ae00700a0 2024-09-10T21:13:35.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8ae5000640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ae00700a0 0x7f8ae01543f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c500 tx=0x7f8ad400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:35.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8adcc00640 1 -- 172.21.15.16:0/905710219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4018070 con 0x7f8ae00700a0 2024-09-10T21:13:35.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.331+0000 7f8adcc00640 1 -- 172.21.15.16:0/905710219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ad4003600 con 0x7f8ae00700a0 2024-09-10T21:13:35.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.335+0000 7f8adcc00640 1 -- 172.21.15.16:0/905710219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4003a90 con 0x7f8ae00700a0 2024-09-10T21:13:35.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.335+0000 7f8ae7a00640 1 -- 172.21.15.16:0/905710219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae015cb80 con 0x7f8ae00700a0 2024-09-10T21:13:35.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.335+0000 7f8ae7a00640 1 -- 172.21.15.16:0/905710219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae00c7a10 con 0x7f8ae00700a0 2024-09-10T21:13:35.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.335+0000 7f8ae7a00640 1 -- 172.21.15.16:0/905710219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae00ca170 con 0x7f8ae00700a0 2024-09-10T21:13:35.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.335+0000 7f8adcc00640 1 -- 172.21.15.16:0/905710219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8ad4003c30 con 0x7f8ae00700a0 2024-09-10T21:13:35.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.335+0000 7f8adcc00640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ac00725e0 0x7f8ac0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:35.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.335+0000 7f8adcc00640 1 -- 172.21.15.16:0/905710219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f8ad4092910 con 0x7f8ae00700a0 2024-09-10T21:13:35.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.335+0000 7f8adfe00640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ac00725e0 0x7f8ac0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:35.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.339+0000 7f8adfe00640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ac00725e0 0x7f8ac0074aa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8acc001ec0 tx=0x7f8acc009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:35.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.339+0000 7f8adcc00640 1 -- 172.21.15.16:0/905710219 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ad405e020 con 0x7f8ae00700a0 2024-09-10T21:13:35.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8ae7a00640 1 -- 172.21.15.16:0/905710219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ae00b5aa0 con 0x7f8ae00700a0 2024-09-10T21:13:35.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8adcc00640 1 -- 172.21.15.16:0/905710219 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5234 (secure 0 0 0) 0x7f8ad405d920 con 0x7f8ae00700a0 2024-09-10T21:13:35.558 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:35.558 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-10T21:13:25:905095+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4029516332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4029516332},{"type":"v1","addr":"172.21.15.16:6836","nonce":4029516332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/806881470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":806881470},{"type":"v1","addr":"172.21.15.16:6837","nonce":806881470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":135167,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2949376643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2949376643},{"type":"v1","addr":"172.21.15.52:6834","nonce":2949376643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:22.294493+0000","modified":"2024-09-10T21:13:25.298310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135170},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135170":{"gid":135170,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/1574608409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1574608409},{"type":"v1","addr":"172.21.15.52:6835","nonce":1574608409}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135170,"qdb_cluster":[135170]},"id":5}]} 2024-09-10T21:13:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-10T21:13:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8ae7a00640 1 -- 172.21.15.16:0/905710219 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ac00725e0 msgr2=0x7f8ac0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8ae7a00640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ac00725e0 0x7f8ac0074aa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8acc001ec0 tx=0x7f8acc009040 comp rx=0 tx=0).stop 2024-09-10T21:13:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8ae7a00640 1 -- 172.21.15.16:0/905710219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ae00700a0 msgr2=0x7f8ae01543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8ae7a00640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ae00700a0 0x7f8ae01543f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c500 tx=0x7f8ad400c9c0 comp rx=0 tx=0).stop 2024-09-10T21:13:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8ae7a00640 1 -- 172.21.15.16:0/905710219 shutdown_connections 2024-09-10T21:13:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8ae7a00640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ac00725e0 0x7f8ac0074aa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8ae7a00640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ae015a270 0x7f8ae00b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8ae7a00640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8ae00d06b0 0x7f8ae01639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8ae7a00640 1 --2- 172.21.15.16:0/905710219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ae00700a0 0x7f8ae01543f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8ae7a00640 1 -- 172.21.15.16:0/905710219 >> 172.21.15.16:0/905710219 conn(0x7f8ae00cfb20 msgr2=0x7f8ae0157870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8ae7a00640 1 -- 172.21.15.16:0/905710219 shutdown_connections 2024-09-10T21:13:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:35.551+0000 7f8ae7a00640 1 -- 172.21.15.16:0/905710219 wait complete. 2024-09-10T21:13:35.580 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-10T21:13:35.749 INFO:teuthology.orchestra.run.smithi052.stdout:[{"id":6087,"entity":{"name":{"type":"client","num":6087},"addr":{"type":"any","addr":"172.21.15.52:0","nonce":2937410031}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":4.7119621629999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9720558540647359,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6087 172.21.15.52:0/2937410031","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi052","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":6078,"entity":{"name":{"type":"client","num":6078},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1599252759}},"state":"open","num_leases":1,"num_caps":602,"request_load_avg":613,"uptime":5.8679528899999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":743.60207814328749,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":442.73262477916006,"halflife":1.5},"recall_caps_throttle2o":{"value":171.42491981387192,"halflife":0.5},"session_cache_liveness":{"value":1198.8565289824849,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":630,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6078 192.168.0.1:0/1599252759","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-10T21:13:35.750 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi052", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.52:0", "nonce": 2937410031, "type": "any" }, "name": { "num": 6087, "type": "client" } }, "id": 6087, "inst": "client.6087 172.21.15.52:0/2937410031", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.972055854064736 }, "state": "open", "uptime": 4.711962163 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1599252759, "type": "any" }, "name": { "num": 6078, "type": "client" } }, "id": 6078, "inst": "client.6078 192.168.0.1:0/1599252759", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 630, "num_caps": 602, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 743.6020781432875 }, "recall_caps_throttle": { "halflife": 1.5, "value": 442.73262477916006 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 171.42491981387192 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 613, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1198.8565289824849 }, "state": "open", "uptime": 5.86795289 } ] 2024-09-10T21:13:35.750 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:13:40.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.903+0000 7fd0af200640 1 -- 172.21.15.16:0/1118711232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b0160120 msgr2=0x7fd0b0160560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.903+0000 7fd0af200640 1 --2- 172.21.15.16:0/1118711232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b0160120 0x7fd0b0160560 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00730f0 tx=0x7fd0a801ccf0 comp rx=0 tx=0).stop 2024-09-10T21:13:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.903+0000 7fd0af200640 1 -- 172.21.15.16:0/1118711232 shutdown_connections 2024-09-10T21:13:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.903+0000 7fd0af200640 1 --2- 172.21.15.16:0/1118711232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b0160120 0x7fd0b0160560 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.903+0000 7fd0af200640 1 --2- 172.21.15.16:0/1118711232 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0b00d2820 0x7fd0b015fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.903+0000 7fd0af200640 1 --2- 172.21.15.16:0/1118711232 >> v2:172.21.15.52:3300/0 conn(0x7fd0b00cfd40 0x7fd0b00d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.903+0000 7fd0af200640 1 -- 172.21.15.16:0/1118711232 >> 172.21.15.16:0/1118711232 conn(0x7fd0b00d1d00 msgr2=0x7fd0b00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.903+0000 7fd0af200640 1 -- 172.21.15.16:0/1118711232 shutdown_connections 2024-09-10T21:13:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.903+0000 7fd0af200640 1 -- 172.21.15.16:0/1118711232 wait complete. 2024-09-10T21:13:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.903+0000 7fd0af200640 1 Processor -- start 2024-09-10T21:13:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.903+0000 7fd0af200640 1 -- start start 2024-09-10T21:13:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0af200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0b00cfd40 0x7fd0b0154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0af200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0b00d2820 0x7fd0b01fb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0af200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b0160120 0x7fd0b00b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0af200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b00b26a0 con 0x7fd0b0160120 2024-09-10T21:13:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0af200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b00b2520 con 0x7fd0b00d2820 2024-09-10T21:13:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0af200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b00b2820 con 0x7fd0b00cfd40 2024-09-10T21:13:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0ae800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b0160120 0x7fd0b00b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0ae800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b0160120 0x7fd0b00b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42400/0 (socket says 172.21.15.16:42400) 2024-09-10T21:13:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0ade00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0b00cfd40 0x7fd0b0154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0a7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0b00d2820 0x7fd0b01fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:40.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0ae800640 1 -- 172.21.15.16:0/162914555 learned_addr learned my addr 172.21.15.16:0/162914555 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0ae800640 1 -- 172.21.15.16:0/162914555 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0b00cfd40 msgr2=0x7fd0b0154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0ae800640 1 --2- 172.21.15.16:0/162914555 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0b00cfd40 0x7fd0b0154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0ae800640 1 -- 172.21.15.16:0/162914555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0b00d2820 msgr2=0x7fd0b01fb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0ae800640 1 --2- 172.21.15.16:0/162914555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0b00d2820 0x7fd0b01fb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0ae800640 1 -- 172.21.15.16:0/162914555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b015afd0 con 0x7fd0b0160120 2024-09-10T21:13:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0ae800640 1 --2- 172.21.15.16:0/162914555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b0160120 0x7fd0b00b6520 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd0a801b3a0 tx=0x7fd0a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0a6000640 1 -- 172.21.15.16:0/162914555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a80bb070 con 0x7fd0b0160120 2024-09-10T21:13:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0a6000640 1 -- 172.21.15.16:0/162914555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0a8004b40 con 0x7fd0b0160120 2024-09-10T21:13:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0a6000640 1 -- 172.21.15.16:0/162914555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a80b2040 con 0x7fd0b0160120 2024-09-10T21:13:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0af200640 1 -- 172.21.15.16:0/162914555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b00b6a60 con 0x7fd0b0160120 2024-09-10T21:13:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0af200640 1 -- 172.21.15.16:0/162914555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b00b4da0 con 0x7fd0b0160120 2024-09-10T21:13:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0af200640 1 -- 172.21.15.16:0/162914555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b00b4b30 con 0x7fd0b0160120 2024-09-10T21:13:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.907+0000 7fd0a6000640 1 -- 172.21.15.16:0/162914555 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd0a80b5030 con 0x7fd0b0160120 2024-09-10T21:13:40.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.911+0000 7fd0a6000640 1 --2- 172.21.15.16:0/162914555 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd08c0725e0 0x7fd08c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.911+0000 7fd0a6000640 1 -- 172.21.15.16:0/162914555 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fd0a812ce80 con 0x7fd0b0160120 2024-09-10T21:13:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.911+0000 7fd0ade00640 1 --2- 172.21.15.16:0/162914555 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd08c0725e0 0x7fd08c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:40.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.911+0000 7fd0ade00640 1 --2- 172.21.15.16:0/162914555 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd08c0725e0 0x7fd08c074aa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000bb70 tx=0x7fd0a000b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:40.915+0000 7fd0a6000640 1 -- 172.21.15.16:0/162914555 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd0a80f1de0 con 0x7fd0b0160120 2024-09-10T21:13:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.127+0000 7fd0af200640 1 -- 172.21.15.16:0/162914555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0b015fbe0 con 0x7fd0b0160120 2024-09-10T21:13:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.127+0000 7fd0a6000640 1 -- 172.21.15.16:0/162914555 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5234 (secure 0 0 0) 0x7fd0a8016970 con 0x7fd0b0160120 2024-09-10T21:13:41.137 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:41.137 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-10T21:13:25:905095+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4029516332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4029516332},{"type":"v1","addr":"172.21.15.16:6836","nonce":4029516332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/806881470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":806881470},{"type":"v1","addr":"172.21.15.16:6837","nonce":806881470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":135167,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2949376643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2949376643},{"type":"v1","addr":"172.21.15.52:6834","nonce":2949376643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:22.294493+0000","modified":"2024-09-10T21:13:25.298310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135170},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135170":{"gid":135170,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/1574608409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1574608409},{"type":"v1","addr":"172.21.15.52:6835","nonce":1574608409}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135170,"qdb_cluster":[135170]},"id":5}]} 2024-09-10T21:13:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-10T21:13:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.131+0000 7fd0af200640 1 -- 172.21.15.16:0/162914555 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd08c0725e0 msgr2=0x7fd08c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.131+0000 7fd0af200640 1 --2- 172.21.15.16:0/162914555 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd08c0725e0 0x7fd08c074aa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000bb70 tx=0x7fd0a000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.131+0000 7fd0af200640 1 -- 172.21.15.16:0/162914555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b0160120 msgr2=0x7fd0b00b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.131+0000 7fd0af200640 1 --2- 172.21.15.16:0/162914555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b0160120 0x7fd0b00b6520 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd0a801b3a0 tx=0x7fd0a8004270 comp rx=0 tx=0).stop 2024-09-10T21:13:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.131+0000 7fd0af200640 1 -- 172.21.15.16:0/162914555 shutdown_connections 2024-09-10T21:13:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.131+0000 7fd0af200640 1 --2- 172.21.15.16:0/162914555 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd08c0725e0 0x7fd08c074aa0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.131+0000 7fd0af200640 1 --2- 172.21.15.16:0/162914555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b0160120 0x7fd0b00b6520 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.131+0000 7fd0af200640 1 --2- 172.21.15.16:0/162914555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0b00d2820 0x7fd0b01fb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.131+0000 7fd0af200640 1 --2- 172.21.15.16:0/162914555 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0b00cfd40 0x7fd0b0154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.131+0000 7fd0af200640 1 -- 172.21.15.16:0/162914555 >> 172.21.15.16:0/162914555 conn(0x7fd0b00d1d00 msgr2=0x7fd0b015dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.131+0000 7fd0af200640 1 -- 172.21.15.16:0/162914555 shutdown_connections 2024-09-10T21:13:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.131+0000 7fd0af200640 1 -- 172.21.15.16:0/162914555 wait complete. 2024-09-10T21:13:41.160 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-10T21:13:41.338 INFO:teuthology.orchestra.run.smithi052.stdout:[{"id":6087,"entity":{"name":{"type":"client","num":6087},"addr":{"type":"any","addr":"172.21.15.52:0","nonce":2937410031}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":10.299917191,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9339305319445366,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6087 172.21.15.52:0/2937410031","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi052","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":6078,"entity":{"name":{"type":"client","num":6078},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1599252759}},"state":"open","num_leases":1,"num_caps":100,"request_load_avg":574,"uptime":11.455907917999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":2084.1102749424449,"halflife":60},"release_caps":{"value":487.94734598951919,"halflife":60},"recall_caps_throttle":{"value":415.60744760073084,"halflife":1.5},"recall_caps_throttle2o":{"value":34.804233856234795,"halflife":0.5},"session_cache_liveness":{"value":1183.4776822892604,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":630,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6078 192.168.0.1:0/1599252759","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-10T21:13:41.338 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi052", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.52:0", "nonce": 2937410031, "type": "any" }, "name": { "num": 6087, "type": "client" } }, "id": 6087, "inst": "client.6087 172.21.15.52:0/2937410031", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9339305319445366 }, "state": "open", "uptime": 10.299917191 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1599252759, "type": "any" }, "name": { "num": 6078, "type": "client" } }, "id": 6078, "inst": "client.6078 192.168.0.1:0/1599252759", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 630, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 2084.110274942445 }, "recall_caps_throttle": { "halflife": 1.5, "value": 415.60744760073084 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 34.804233856234795 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 487.9473459895192 }, "request_load_avg": 574, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1183.4776822892604 }, "state": "open", "uptime": 11.455907918 } ] 2024-09-10T21:13:41.339 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:13:41.339 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:13:41.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:13:41.431 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:13:41.524 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:13:41.524 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:13:41.524 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:41.524 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:41.533 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:41.533 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:41.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:13:41.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.771+0000 7f45ee800640 1 -- 172.21.15.16:0/686480828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e8074170 msgr2=0x7f45e8074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:41.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.771+0000 7f45ee800640 1 --2- 172.21.15.16:0/686480828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e8074170 0x7f45e8074620 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009bc0 tx=0x7f45e001cb10 comp rx=0 tx=0).stop 2024-09-10T21:13:41.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.771+0000 7f45ee800640 1 -- 172.21.15.16:0/686480828 shutdown_connections 2024-09-10T21:13:41.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.771+0000 7f45ee800640 1 --2- 172.21.15.16:0/686480828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e8074170 0x7f45e8074620 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.771+0000 7f45ee800640 1 --2- 172.21.15.16:0/686480828 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45e80cfc20 0x7f45e8068990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.771+0000 7f45ee800640 1 -- 172.21.15.16:0/686480828 >> 172.21.15.16:0/686480828 conn(0x7f45e80b1d60 msgr2=0x7f45e80b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:41.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.771+0000 7f45ee800640 1 -- 172.21.15.16:0/686480828 shutdown_connections 2024-09-10T21:13:41.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.771+0000 7f45ee800640 1 -- 172.21.15.16:0/686480828 wait complete. 2024-09-10T21:13:41.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ee800640 1 Processor -- start 2024-09-10T21:13:41.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ee800640 1 -- start start 2024-09-10T21:13:41.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ee800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e8074170 0x7f45e80c3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:41.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ee800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45e80b2880 0x7f45e8155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:41.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ee800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45e80cfc20 0x7f45e81556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:41.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ee800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45e8167c30 con 0x7f45e8074170 2024-09-10T21:13:41.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ee800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45e8167ab0 con 0x7f45e80cfc20 2024-09-10T21:13:41.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ee800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f45e8167db0 con 0x7f45e80b2880 2024-09-10T21:13:41.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ed400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e8074170 0x7f45e80c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:41.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ed400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e8074170 0x7f45e80c3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42410/0 (socket says 172.21.15.16:42410) 2024-09-10T21:13:41.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ed400640 1 -- 172.21.15.16:0/3528699349 learned_addr learned my addr 172.21.15.16:0/3528699349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:41.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ede00640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45e80cfc20 0x7f45e81556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:41.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45eca00640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45e80b2880 0x7f45e8155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:41.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ed400640 1 -- 172.21.15.16:0/3528699349 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45e80b2880 msgr2=0x7f45e8155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:41.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ed400640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45e80b2880 0x7f45e8155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ed400640 1 -- 172.21.15.16:0/3528699349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45e80cfc20 msgr2=0x7f45e81556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:41.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ed400640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45e80cfc20 0x7f45e81556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ed400640 1 -- 172.21.15.16:0/3528699349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e8099e10 con 0x7f45e8074170 2024-09-10T21:13:41.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45eca00640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45e80b2880 0x7f45e8155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:41.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ed400640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e8074170 0x7f45e80c3610 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00c500 tx=0x7f45dc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:41.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45e5600640 1 -- 172.21.15.16:0/3528699349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc017070 con 0x7f45e8074170 2024-09-10T21:13:41.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45e5600640 1 -- 172.21.15.16:0/3528699349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45dc003600 con 0x7f45e8074170 2024-09-10T21:13:41.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45e5600640 1 -- 172.21.15.16:0/3528699349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc003a50 con 0x7f45e8074170 2024-09-10T21:13:41.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ee800640 1 -- 172.21.15.16:0/3528699349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e8155f20 con 0x7f45e8074170 2024-09-10T21:13:41.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ee800640 1 -- 172.21.15.16:0/3528699349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e815a4f0 con 0x7f45e8074170 2024-09-10T21:13:41.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.775+0000 7f45ee800640 1 -- 172.21.15.16:0/3528699349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e8074620 con 0x7f45e8074170 2024-09-10T21:13:41.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.779+0000 7f45e5600640 1 -- 172.21.15.16:0/3528699349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f45dc003bf0 con 0x7f45e8074170 2024-09-10T21:13:41.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.779+0000 7f45e5600640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45c80725e0 0x7f45c8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:41.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.779+0000 7f45e5600640 1 -- 172.21.15.16:0/3528699349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f45dc092960 con 0x7f45e8074170 2024-09-10T21:13:41.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.779+0000 7f45eca00640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45c80725e0 0x7f45c8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:41.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.779+0000 7f45eca00640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45c80725e0 0x7f45c8074aa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f45e8165050 tx=0x7f45d400b5b0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:41.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.783+0000 7f45e5600640 1 -- 172.21.15.16:0/3528699349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f45dc05daa0 con 0x7f45e8074170 2024-09-10T21:13:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.971+0000 7f45ee800640 1 -- 172.21.15.16:0/3528699349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f45e80b5fb0 con 0x7f45e8074170 2024-09-10T21:13:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.971+0000 7f45e5600640 1 -- 172.21.15.16:0/3528699349 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7f45dc0624d0 con 0x7f45e8074170 2024-09-10T21:13:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:13:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.975+0000 7f45ee800640 1 -- 172.21.15.16:0/3528699349 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45c80725e0 msgr2=0x7f45c8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.975+0000 7f45ee800640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45c80725e0 0x7f45c8074aa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f45e8165050 tx=0x7f45d400b5b0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.975+0000 7f45ee800640 1 -- 172.21.15.16:0/3528699349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e8074170 msgr2=0x7f45e80c3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.975+0000 7f45ee800640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e8074170 0x7f45e80c3610 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00c500 tx=0x7f45dc00c9c0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.975+0000 7f45ee800640 1 -- 172.21.15.16:0/3528699349 shutdown_connections 2024-09-10T21:13:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.975+0000 7f45ee800640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45c80725e0 0x7f45c8074aa0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.975+0000 7f45ee800640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45e80cfc20 0x7f45e81556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.975+0000 7f45ee800640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45e80b2880 0x7f45e8155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.975+0000 7f45ee800640 1 --2- 172.21.15.16:0/3528699349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e8074170 0x7f45e80c3610 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:41.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.975+0000 7f45ee800640 1 -- 172.21.15.16:0/3528699349 >> 172.21.15.16:0/3528699349 conn(0x7f45e80b1d60 msgr2=0x7f45e80caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:41.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.975+0000 7f45ee800640 1 -- 172.21.15.16:0/3528699349 shutdown_connections 2024-09-10T21:13:41.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:41.975+0000 7f45ee800640 1 -- 172.21.15.16:0/3528699349 wait complete. 2024-09-10T21:13:42.002 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:13:42.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:42.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:13:42.144 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:13:42.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:42.145 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:13:42.155 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:13:42.155 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:13:42.155 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:42.165 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:13:42.166 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:42.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:13:42.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.363+0000 7f8376a00640 1 -- 172.21.15.16:0/2443472466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83700ce990 msgr2=0x7f83700ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:42.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.363+0000 7f8376a00640 1 --2- 172.21.15.16:0/2443472466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83700ce990 0x7f83700ca170 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8370076480 tx=0x7f836801ccf0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.363+0000 7f8376a00640 1 -- 172.21.15.16:0/2443472466 shutdown_connections 2024-09-10T21:13:42.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.363+0000 7f8376a00640 1 --2- 172.21.15.16:0/2443472466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83700ce990 0x7f83700ca170 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.363+0000 7f8376a00640 1 --2- 172.21.15.16:0/2443472466 >> v2:172.21.15.52:3300/0 conn(0x7f83700705e0 0x7f83700ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.363+0000 7f8376a00640 1 --2- 172.21.15.16:0/2443472466 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f837015a270 0x7f83700700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.363+0000 7f8376a00640 1 -- 172.21.15.16:0/2443472466 >> 172.21.15.16:0/2443472466 conn(0x7f83700cfb20 msgr2=0x7f83700cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:42.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.363+0000 7f8376a00640 1 -- 172.21.15.16:0/2443472466 shutdown_connections 2024-09-10T21:13:42.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.363+0000 7f8376a00640 1 -- 172.21.15.16:0/2443472466 wait complete. 2024-09-10T21:13:42.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.363+0000 7f8376a00640 1 Processor -- start 2024-09-10T21:13:42.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.363+0000 7f8376a00640 1 -- start start 2024-09-10T21:13:42.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.363+0000 7f8376a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f83700705e0 0x7f83701f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:42.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8376a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83700ce990 0x7f83701f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:42.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8376a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837015a270 0x7f83701561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:42.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8376a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8370072ef0 con 0x7f837015a270 2024-09-10T21:13:42.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8376a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8370072d70 con 0x7f83700ce990 2024-09-10T21:13:42.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8376a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8370073070 con 0x7f83700705e0 2024-09-10T21:13:42.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8374a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837015a270 0x7f83701561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:42.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8374a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837015a270 0x7f83701561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39824/0 (socket says 172.21.15.16:39824) 2024-09-10T21:13:42.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8374a00640 1 -- 172.21.15.16:0/1411723792 learned_addr learned my addr 172.21.15.16:0/1411723792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:42.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f836fe00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f83700705e0 0x7f83701f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:42.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8367e00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83700ce990 0x7f83701f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:42.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8374a00640 1 -- 172.21.15.16:0/1411723792 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f83700705e0 msgr2=0x7f83701f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8374a00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f83700705e0 0x7f83701f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8374a00640 1 -- 172.21.15.16:0/1411723792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83700ce990 msgr2=0x7f83701f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8374a00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83700ce990 0x7f83701f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8374a00640 1 -- 172.21.15.16:0/1411723792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83701f7ce0 con 0x7f837015a270 2024-09-10T21:13:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8374a00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837015a270 0x7f83701561b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f836801b3a0 tx=0x7f8368003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f836cc00640 1 -- 172.21.15.16:0/1411723792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83680bb070 con 0x7f837015a270 2024-09-10T21:13:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f836cc00640 1 -- 172.21.15.16:0/1411723792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8368004e00 con 0x7f837015a270 2024-09-10T21:13:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f836cc00640 1 -- 172.21.15.16:0/1411723792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83680b2040 con 0x7f837015a270 2024-09-10T21:13:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8376a00640 1 -- 172.21.15.16:0/1411723792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83701fc2a0 con 0x7f837015a270 2024-09-10T21:13:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8376a00640 1 -- 172.21.15.16:0/1411723792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83701f7f50 con 0x7f837015a270 2024-09-10T21:13:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f8376a00640 1 -- 172.21.15.16:0/1411723792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83700ca170 con 0x7f837015a270 2024-09-10T21:13:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.367+0000 7f836cc00640 1 -- 172.21.15.16:0/1411723792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f83680b5030 con 0x7f837015a270 2024-09-10T21:13:42.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.371+0000 7f836cc00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f834c072610 0x7f834c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:42.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.371+0000 7f836cc00640 1 -- 172.21.15.16:0/1411723792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f836812cd30 con 0x7f837015a270 2024-09-10T21:13:42.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.371+0000 7f836fe00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f834c072610 0x7f834c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:42.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.371+0000 7f836fe00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f834c072610 0x7f834c074ad0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f83700ce0d0 tx=0x7f836000b3e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:42.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.375+0000 7f836cc00640 1 -- 172.21.15.16:0/1411723792 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8368016970 con 0x7f837015a270 2024-09-10T21:13:42.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.559+0000 7f8376a00640 1 -- 172.21.15.16:0/1411723792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f83700b5aa0 con 0x7f837015a270 2024-09-10T21:13:42.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.559+0000 7f836cc00640 1 -- 172.21.15.16:0/1411723792 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7f83680f7da0 con 0x7f837015a270 2024-09-10T21:13:42.568 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:13:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.563+0000 7f8376a00640 1 -- 172.21.15.16:0/1411723792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f834c072610 msgr2=0x7f834c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.563+0000 7f8376a00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f834c072610 0x7f834c074ad0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f83700ce0d0 tx=0x7f836000b3e0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.563+0000 7f8376a00640 1 -- 172.21.15.16:0/1411723792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837015a270 msgr2=0x7f83701561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.563+0000 7f8376a00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837015a270 0x7f83701561b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f836801b3a0 tx=0x7f8368003f60 comp rx=0 tx=0).stop 2024-09-10T21:13:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.563+0000 7f8376a00640 1 -- 172.21.15.16:0/1411723792 shutdown_connections 2024-09-10T21:13:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.563+0000 7f8376a00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f834c072610 0x7f834c074ad0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.563+0000 7f8376a00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837015a270 0x7f83701561b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.563+0000 7f8376a00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83700ce990 0x7f83701f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.563+0000 7f8376a00640 1 --2- 172.21.15.16:0/1411723792 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f83700705e0 0x7f83701f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.563+0000 7f8376a00640 1 -- 172.21.15.16:0/1411723792 >> 172.21.15.16:0/1411723792 conn(0x7f83700cfb20 msgr2=0x7f8370162680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:42.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.563+0000 7f8376a00640 1 -- 172.21.15.16:0/1411723792 shutdown_connections 2024-09-10T21:13:42.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.563+0000 7f8376a00640 1 -- 172.21.15.16:0/1411723792 wait complete. 2024-09-10T21:13:42.590 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T21:13:42.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:42.590 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T21:13:42.618 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:13:42.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:13:42.618 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T21:13:42.673 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:13:42.674 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:42.674 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:42.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:13:42.683 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:13:42.683 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:13:42.683 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:42.729 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:13:42.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.867+0000 7f2b5e800640 1 -- 172.21.15.16:0/2850007834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b580b13c0 msgr2=0x7f2b58071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.867+0000 7f2b5e800640 1 --2- 172.21.15.16:0/2850007834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b580b13c0 0x7f2b58071520 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2b58073290 tx=0x7f2b4001c7a0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.867+0000 7f2b5e800640 1 -- 172.21.15.16:0/2850007834 shutdown_connections 2024-09-10T21:13:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.867+0000 7f2b5e800640 1 --2- 172.21.15.16:0/2850007834 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b580cfc80 0x7f2b580d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.867+0000 7f2b5e800640 1 --2- 172.21.15.16:0/2850007834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b580b13c0 0x7f2b58071520 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.867+0000 7f2b5e800640 1 --2- 172.21.15.16:0/2850007834 >> v2:172.21.15.52:3300/0 conn(0x7f2b5815f400 0x7f2b58070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.867+0000 7f2b5e800640 1 -- 172.21.15.16:0/2850007834 >> 172.21.15.16:0/2850007834 conn(0x7f2b5809b130 msgr2=0x7f2b580d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.867+0000 7f2b5e800640 1 -- 172.21.15.16:0/2850007834 shutdown_connections 2024-09-10T21:13:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.867+0000 7f2b5e800640 1 -- 172.21.15.16:0/2850007834 wait complete. 2024-09-10T21:13:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b5e800640 1 Processor -- start 2024-09-10T21:13:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b5e800640 1 -- start start 2024-09-10T21:13:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b5e800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b580b13c0 0x7f2b580b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b5e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b580cfc80 0x7f2b580c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b5e800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b5815f400 0x7f2b580b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b5e800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b580afd90 con 0x7f2b580cfc80 2024-09-10T21:13:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b5e800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b580d1260 con 0x7f2b5815f400 2024-09-10T21:13:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b5e800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2b580aff10 con 0x7f2b580b13c0 2024-09-10T21:13:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b4ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b580cfc80 0x7f2b580c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b4ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b580cfc80 0x7f2b580c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39838/0 (socket says 172.21.15.16:39838) 2024-09-10T21:13:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b4ea00640 1 -- 172.21.15.16:0/336571096 learned_addr learned my addr 172.21.15.16:0/336571096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b57400640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b580b13c0 0x7f2b580b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b57e00640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b5815f400 0x7f2b580b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b4ea00640 1 -- 172.21.15.16:0/336571096 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b580b13c0 msgr2=0x7f2b580b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b4ea00640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b580b13c0 0x7f2b580b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b4ea00640 1 -- 172.21.15.16:0/336571096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b5815f400 msgr2=0x7f2b580b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b4ea00640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b5815f400 0x7f2b580b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b4ea00640 1 -- 172.21.15.16:0/336571096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b580b6750 con 0x7f2b580cfc80 2024-09-10T21:13:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b57e00640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b5815f400 0x7f2b580b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b4ea00640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b580cfc80 0x7f2b580c9930 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2b400b0040 tx=0x7f2b400a5510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b4fe00640 1 -- 172.21.15.16:0/336571096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b400b2040 con 0x7f2b580cfc80 2024-09-10T21:13:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b4fe00640 1 -- 172.21.15.16:0/336571096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b40004380 con 0x7f2b580cfc80 2024-09-10T21:13:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b4fe00640 1 -- 172.21.15.16:0/336571096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40005290 con 0x7f2b580cfc80 2024-09-10T21:13:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b5e800640 1 -- 172.21.15.16:0/336571096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b580b6a30 con 0x7f2b580cfc80 2024-09-10T21:13:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b5e800640 1 -- 172.21.15.16:0/336571096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b580c9410 con 0x7f2b580cfc80 2024-09-10T21:13:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.871+0000 7f2b5e800640 1 -- 172.21.15.16:0/336571096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b580d0110 con 0x7f2b580cfc80 2024-09-10T21:13:42.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.875+0000 7f2b4fe00640 1 -- 172.21.15.16:0/336571096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b40005430 con 0x7f2b580cfc80 2024-09-10T21:13:42.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.875+0000 7f2b4fe00640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b340725e0 0x7f2b34074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:42.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.875+0000 7f2b4fe00640 1 -- 172.21.15.16:0/336571096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f2b4012cce0 con 0x7f2b580cfc80 2024-09-10T21:13:42.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.875+0000 7f2b57400640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b340725e0 0x7f2b34074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:42.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.875+0000 7f2b57400640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b340725e0 0x7f2b34074aa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2b48005fd0 tx=0x7f2b4800b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:42.879+0000 7f2b4fe00640 1 -- 172.21.15.16:0/336571096 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b400164b0 con 0x7f2b580cfc80 2024-09-10T21:13:43.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.087+0000 7f2b5e800640 1 -- 172.21.15.16:0/336571096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b580b2a60 con 0x7f2b580cfc80 2024-09-10T21:13:43.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.091+0000 7f2b4fe00640 1 -- 172.21.15.16:0/336571096 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5234 (secure 0 0 0) 0x7f2b400f83b0 con 0x7f2b580cfc80 2024-09-10T21:13:43.097 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:43.098 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-10T21:13:25:905095+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4029516332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4029516332},{"type":"v1","addr":"172.21.15.16:6836","nonce":4029516332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/806881470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":806881470},{"type":"v1","addr":"172.21.15.16:6837","nonce":806881470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":135167,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2949376643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2949376643},{"type":"v1","addr":"172.21.15.52:6834","nonce":2949376643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:22.294493+0000","modified":"2024-09-10T21:13:25.298310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135170},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135170":{"gid":135170,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/1574608409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1574608409},{"type":"v1","addr":"172.21.15.52:6835","nonce":1574608409}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135170,"qdb_cluster":[135170]},"id":5}]} 2024-09-10T21:13:43.098 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-10T21:13:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.091+0000 7f2b5e800640 1 -- 172.21.15.16:0/336571096 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b340725e0 msgr2=0x7f2b34074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.091+0000 7f2b5e800640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b340725e0 0x7f2b34074aa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2b48005fd0 tx=0x7f2b4800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.091+0000 7f2b5e800640 1 -- 172.21.15.16:0/336571096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b580cfc80 msgr2=0x7f2b580c9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:43.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.091+0000 7f2b5e800640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b580cfc80 0x7f2b580c9930 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2b400b0040 tx=0x7f2b400a5510 comp rx=0 tx=0).stop 2024-09-10T21:13:43.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.091+0000 7f2b5e800640 1 -- 172.21.15.16:0/336571096 shutdown_connections 2024-09-10T21:13:43.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.091+0000 7f2b5e800640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b340725e0 0x7f2b34074aa0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.091+0000 7f2b5e800640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b5815f400 0x7f2b580b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.091+0000 7f2b5e800640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b580cfc80 0x7f2b580c9930 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.091+0000 7f2b5e800640 1 --2- 172.21.15.16:0/336571096 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b580b13c0 0x7f2b580b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.091+0000 7f2b5e800640 1 -- 172.21.15.16:0/336571096 >> 172.21.15.16:0/336571096 conn(0x7f2b5809b130 msgr2=0x7f2b5815dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:43.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.091+0000 7f2b5e800640 1 -- 172.21.15.16:0/336571096 shutdown_connections 2024-09-10T21:13:43.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.091+0000 7f2b5e800640 1 -- 172.21.15.16:0/336571096 wait complete. 2024-09-10T21:13:43.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 -- 172.21.15.16:0/3286520428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6400d06d0 msgr2=0x7fb6400aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 --2- 172.21.15.16:0/3286520428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6400d06d0 0x7fb6400aed30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb638009bc0 tx=0x7fb63801cae0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 -- 172.21.15.16:0/3286520428 shutdown_connections 2024-09-10T21:13:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 --2- 172.21.15.16:0/3286520428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6400d06d0 0x7fb6400aed30 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 --2- 172.21.15.16:0/3286520428 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb64015f350 0x7fb6400cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 -- 172.21.15.16:0/3286520428 >> 172.21.15.16:0/3286520428 conn(0x7fb6400d1d80 msgr2=0x7fb6400d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 -- 172.21.15.16:0/3286520428 shutdown_connections 2024-09-10T21:13:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 -- 172.21.15.16:0/3286520428 wait complete. 2024-09-10T21:13:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 Processor -- start 2024-09-10T21:13:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 -- start start 2024-09-10T21:13:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6400cfb60 0x7fb64014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6400d06d0 0x7fb64014d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb64015f350 0x7fb6401f8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb640168c50 con 0x7fb6400cfb60 2024-09-10T21:13:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb640168ad0 con 0x7fb64015f350 2024-09-10T21:13:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb647a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb640168dd0 con 0x7fb6400d06d0 2024-09-10T21:13:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb645000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6400cfb60 0x7fb64014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb645000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6400cfb60 0x7fb64014eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39852/0 (socket says 172.21.15.16:39852) 2024-09-10T21:13:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.259+0000 7fb645000640 1 -- 172.21.15.16:0/3409860358 learned_addr learned my addr 172.21.15.16:0/3409860358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb645000640 1 -- 172.21.15.16:0/3409860358 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6400d06d0 msgr2=0x7fb64014d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb645000640 1 --2- 172.21.15.16:0/3409860358 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6400d06d0 0x7fb64014d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb645000640 1 -- 172.21.15.16:0/3409860358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb64015f350 msgr2=0x7fb6401f8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb645000640 1 --2- 172.21.15.16:0/3409860358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb64015f350 0x7fb6401f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb645000640 1 -- 172.21.15.16:0/3409860358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64015a1f0 con 0x7fb6400cfb60 2024-09-10T21:13:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb645000640 1 --2- 172.21.15.16:0/3409860358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6400cfb60 0x7fb64014eef0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb63400d950 tx=0x7fb63400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb63cc00640 1 -- 172.21.15.16:0/3409860358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634017070 con 0x7fb6400cfb60 2024-09-10T21:13:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb63cc00640 1 -- 172.21.15.16:0/3409860358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb634009d20 con 0x7fb6400cfb60 2024-09-10T21:13:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb63cc00640 1 -- 172.21.15.16:0/3409860358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6340106c0 con 0x7fb6400cfb60 2024-09-10T21:13:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb647a00640 1 -- 172.21.15.16:0/3409860358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64014e650 con 0x7fb6400cfb60 2024-09-10T21:13:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb647a00640 1 -- 172.21.15.16:0/3409860358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64014ec10 con 0x7fb6400cfb60 2024-09-10T21:13:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb647a00640 1 -- 172.21.15.16:0/3409860358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6400aed30 con 0x7fb6400cfb60 2024-09-10T21:13:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb63cc00640 1 -- 172.21.15.16:0/3409860358 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb634010860 con 0x7fb6400cfb60 2024-09-10T21:13:43.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb63cc00640 1 --2- 172.21.15.16:0/3409860358 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6200725e0 0x7fb620074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:43.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.263+0000 7fb63cc00640 1 -- 172.21.15.16:0/3409860358 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fb634093770 con 0x7fb6400cfb60 2024-09-10T21:13:43.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.267+0000 7fb63fe00640 1 --2- 172.21.15.16:0/3409860358 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6200725e0 0x7fb620074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:43.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.267+0000 7fb63fe00640 1 --2- 172.21.15.16:0/3409860358 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6200725e0 0x7fb620074aa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb62c007fd0 tx=0x7fb62c0078c0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:43.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.267+0000 7fb63cc00640 1 -- 172.21.15.16:0/3409860358 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb63405edb0 con 0x7fb6400cfb60 2024-09-10T21:13:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.475+0000 7fb647a00640 1 -- 172.21.15.16:0/3409860358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb64015bb10 con 0x7fb6400cfb60 2024-09-10T21:13:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.479+0000 7fb63cc00640 1 -- 172.21.15.16:0/3409860358 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5234 (secure 0 0 0) 0x7fb63405e6b0 con 0x7fb6400cfb60 2024-09-10T21:13:43.485 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:43.485 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-10T21:13:25:905095+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4029516332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4029516332},{"type":"v1","addr":"172.21.15.16:6836","nonce":4029516332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/806881470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":806881470},{"type":"v1","addr":"172.21.15.16:6837","nonce":806881470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":135167,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2949376643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2949376643},{"type":"v1","addr":"172.21.15.52:6834","nonce":2949376643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:22.294493+0000","modified":"2024-09-10T21:13:25.298310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135170},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135170":{"gid":135170,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/1574608409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1574608409},{"type":"v1","addr":"172.21.15.52:6835","nonce":1574608409}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135170,"qdb_cluster":[135170]},"id":5}]} 2024-09-10T21:13:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-10T21:13:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.479+0000 7fb647a00640 1 -- 172.21.15.16:0/3409860358 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6200725e0 msgr2=0x7fb620074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.479+0000 7fb647a00640 1 --2- 172.21.15.16:0/3409860358 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6200725e0 0x7fb620074aa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb62c007fd0 tx=0x7fb62c0078c0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.479+0000 7fb647a00640 1 -- 172.21.15.16:0/3409860358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6400cfb60 msgr2=0x7fb64014eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.479+0000 7fb647a00640 1 --2- 172.21.15.16:0/3409860358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6400cfb60 0x7fb64014eef0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb63400d950 tx=0x7fb63400de10 comp rx=0 tx=0).stop 2024-09-10T21:13:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.479+0000 7fb647a00640 1 -- 172.21.15.16:0/3409860358 shutdown_connections 2024-09-10T21:13:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.479+0000 7fb647a00640 1 --2- 172.21.15.16:0/3409860358 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb6200725e0 0x7fb620074aa0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.479+0000 7fb647a00640 1 --2- 172.21.15.16:0/3409860358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb64015f350 0x7fb6401f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.479+0000 7fb647a00640 1 --2- 172.21.15.16:0/3409860358 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6400d06d0 0x7fb64014d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.479+0000 7fb647a00640 1 --2- 172.21.15.16:0/3409860358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6400cfb60 0x7fb64014eef0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.479+0000 7fb647a00640 1 -- 172.21.15.16:0/3409860358 >> 172.21.15.16:0/3409860358 conn(0x7fb6400d1d80 msgr2=0x7fb6400b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.479+0000 7fb647a00640 1 -- 172.21.15.16:0/3409860358 shutdown_connections 2024-09-10T21:13:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.479+0000 7fb647a00640 1 -- 172.21.15.16:0/3409860358 wait complete. 2024-09-10T21:13:43.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:13:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 -- 172.21.15.16:0/508045185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f535406e640 msgr2=0x7f53540ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 --2- 172.21.15.16:0/508045185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f535406e640 0x7f53540ca170 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f535406daf0 tx=0x7f534001c830 comp rx=0 tx=0).stop 2024-09-10T21:13:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 -- 172.21.15.16:0/508045185 shutdown_connections 2024-09-10T21:13:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 --2- 172.21.15.16:0/508045185 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f53540ca6b0 0x7f53540726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 --2- 172.21.15.16:0/508045185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f535406e640 0x7f53540ca170 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 --2- 172.21.15.16:0/508045185 >> v2:172.21.15.52:3300/0 conn(0x7f53541599d0 0x7f535415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 -- 172.21.15.16:0/508045185 >> 172.21.15.16:0/508045185 conn(0x7f53540cfb20 msgr2=0x7f53540cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 -- 172.21.15.16:0/508045185 shutdown_connections 2024-09-10T21:13:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 -- 172.21.15.16:0/508045185 wait complete. 2024-09-10T21:13:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 Processor -- start 2024-09-10T21:13:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 -- start start 2024-09-10T21:13:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f535406e640 0x7f5354156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f53540ca6b0 0x7f53541f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53541599d0 0x7f53540b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5354074250 con 0x7f53541599d0 2024-09-10T21:13:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f53540740d0 con 0x7f535406e640 2024-09-10T21:13:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.647+0000 7f535bc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f53540743d0 con 0x7f53540ca6b0 2024-09-10T21:13:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5359c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53541599d0 0x7f53540b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5359c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53541599d0 0x7f53540b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39864/0 (socket says 172.21.15.16:39864) 2024-09-10T21:13:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5359c00640 1 -- 172.21.15.16:0/4034458373 learned_addr learned my addr 172.21.15.16:0/4034458373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5359200640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f535406e640 0x7f5354156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5353e00640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f53540ca6b0 0x7f53541f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5359c00640 1 -- 172.21.15.16:0/4034458373 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f53540ca6b0 msgr2=0x7f53541f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5359c00640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f53540ca6b0 0x7f53541f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5359c00640 1 -- 172.21.15.16:0/4034458373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f535406e640 msgr2=0x7f5354156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5359c00640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f535406e640 0x7f5354156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5359c00640 1 -- 172.21.15.16:0/4034458373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53541f9190 con 0x7f53541599d0 2024-09-10T21:13:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5359c00640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53541599d0 0x7f53540b2950 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f534c00e990 tx=0x7f534c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5350c00640 1 -- 172.21.15.16:0/4034458373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c004d30 con 0x7f53541599d0 2024-09-10T21:13:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5350c00640 1 -- 172.21.15.16:0/4034458373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f534c00cb00 con 0x7f53541599d0 2024-09-10T21:13:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5350c00640 1 -- 172.21.15.16:0/4034458373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c005020 con 0x7f53541599d0 2024-09-10T21:13:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f535bc00640 1 -- 172.21.15.16:0/4034458373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53540c8820 con 0x7f53541599d0 2024-09-10T21:13:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f535bc00640 1 -- 172.21.15.16:0/4034458373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53540c6d90 con 0x7f53541599d0 2024-09-10T21:13:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f535bc00640 1 -- 172.21.15.16:0/4034458373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53540c8d40 con 0x7f53541599d0 2024-09-10T21:13:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.651+0000 7f5350c00640 1 -- 172.21.15.16:0/4034458373 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f534c00f040 con 0x7f53541599d0 2024-09-10T21:13:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.655+0000 7f5350c00640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f53340725e0 0x7f5334074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.655+0000 7f5350c00640 1 -- 172.21.15.16:0/4034458373 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f534c093800 con 0x7f53541599d0 2024-09-10T21:13:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.655+0000 7f5359200640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f53340725e0 0x7f5334074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.655+0000 7f5359200640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f53340725e0 0x7f5334074aa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f53540b4460 tx=0x7f534800b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:43.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.659+0000 7f5350c00640 1 -- 172.21.15.16:0/4034458373 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f53540c8d40 con 0x7f53541599d0 2024-09-10T21:13:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.831+0000 7f535bc00640 1 -- 172.21.15.16:0/4034458373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53540748f0 con 0x7f53541599d0 2024-09-10T21:13:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.835+0000 7f5350c00640 1 -- 172.21.15.16:0/4034458373 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16620 (secure 0 0 0) 0x7f534c016030 con 0x7f53541599d0 2024-09-10T21:13:43.842 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:43.842 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:13:23.893029+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:13:20.010491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T21:13:21.255527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:13:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.835+0000 7f535bc00640 1 -- 172.21.15.16:0/4034458373 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f53340725e0 msgr2=0x7f5334074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.835+0000 7f535bc00640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f53340725e0 0x7f5334074aa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f53540b4460 tx=0x7f534800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.835+0000 7f535bc00640 1 -- 172.21.15.16:0/4034458373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53541599d0 msgr2=0x7f53540b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.835+0000 7f535bc00640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53541599d0 0x7f53540b2950 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f534c00e990 tx=0x7f534c00ee50 comp rx=0 tx=0).stop 2024-09-10T21:13:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.835+0000 7f535bc00640 1 -- 172.21.15.16:0/4034458373 shutdown_connections 2024-09-10T21:13:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.835+0000 7f535bc00640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f53340725e0 0x7f5334074aa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.835+0000 7f535bc00640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53541599d0 0x7f53540b2950 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.835+0000 7f535bc00640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f53540ca6b0 0x7f53541f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.835+0000 7f535bc00640 1 --2- 172.21.15.16:0/4034458373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f535406e640 0x7f5354156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:43.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.835+0000 7f535bc00640 1 -- 172.21.15.16:0/4034458373 >> 172.21.15.16:0/4034458373 conn(0x7f53540cfb20 msgr2=0x7f5354158ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:43.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.835+0000 7f535bc00640 1 -- 172.21.15.16:0/4034458373 shutdown_connections 2024-09-10T21:13:43.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:43.835+0000 7f535bc00640 1 -- 172.21.15.16:0/4034458373 wait complete. 2024-09-10T21:13:43.863 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:13:43.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:44.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 -- 172.21.15.16:0/1540399213 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0fc0b29a0 msgr2=0x7fa0fc0d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 --2- 172.21.15.16:0/1540399213 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0fc0b29a0 0x7fa0fc0d0ee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009bf0 tx=0x7fa0f401cbe0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 -- 172.21.15.16:0/1540399213 shutdown_connections 2024-09-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 --2- 172.21.15.16:0/1540399213 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0fc0b29a0 0x7fa0fc0d0ee0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 -- 172.21.15.16:0/1540399213 >> 172.21.15.16:0/1540399213 conn(0x7fa0fc0d1d80 msgr2=0x7fa0fc0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 -- 172.21.15.16:0/1540399213 shutdown_connections 2024-09-10T21:13:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 -- 172.21.15.16:0/1540399213 wait complete. 2024-09-10T21:13:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 Processor -- start 2024-09-10T21:13:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 -- start start 2024-09-10T21:13:44.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0fc0b29a0 0x7fa0fc1f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:44.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0fc161520 0x7fa0fc153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:44.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc161900 0x7fa0fc0b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:44.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc16c000 con 0x7fa0fc161900 2024-09-10T21:13:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc16be80 con 0x7fa0fc161520 2024-09-10T21:13:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa103600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa0fc16c180 con 0x7fa0fc0b29a0 2024-09-10T21:13:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa101600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc161900 0x7fa0fc0b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa101600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc161900 0x7fa0fc0b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39876/0 (socket says 172.21.15.16:39876) 2024-09-10T21:13:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa101600640 1 -- 172.21.15.16:0/673796164 learned_addr learned my addr 172.21.15.16:0/673796164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa100c00640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0fc0b29a0 0x7fa0fc1f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.003+0000 7fa0fbe00640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0fc161520 0x7fa0fc153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa101600640 1 -- 172.21.15.16:0/673796164 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0fc0b29a0 msgr2=0x7fa0fc1f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa101600640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0fc0b29a0 0x7fa0fc1f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa101600640 1 -- 172.21.15.16:0/673796164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0fc161520 msgr2=0x7fa0fc153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa101600640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0fc161520 0x7fa0fc153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa101600640 1 -- 172.21.15.16:0/673796164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc1540f0 con 0x7fa0fc161900 2024-09-10T21:13:44.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa0fbe00640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0fc161520 0x7fa0fc153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:44.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa101600640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc161900 0x7fa0fc0b6240 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40b0040 tx=0x7fa0f4005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:44.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa0f8c00640 1 -- 172.21.15.16:0/673796164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40bb070 con 0x7fa0fc161900 2024-09-10T21:13:44.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa0f8c00640 1 -- 172.21.15.16:0/673796164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0f4004d40 con 0x7fa0fc161900 2024-09-10T21:13:44.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa0f8c00640 1 -- 172.21.15.16:0/673796164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40b2040 con 0x7fa0fc161900 2024-09-10T21:13:44.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa103600640 1 -- 172.21.15.16:0/673796164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc1f77e0 con 0x7fa0fc161900 2024-09-10T21:13:44.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa103600640 1 -- 172.21.15.16:0/673796164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc1f7a80 con 0x7fa0fc161900 2024-09-10T21:13:44.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa103600640 1 -- 172.21.15.16:0/673796164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0fc0aea50 con 0x7fa0fc161900 2024-09-10T21:13:44.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa0f8c00640 1 -- 172.21.15.16:0/673796164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa0f40b5070 con 0x7fa0fc161900 2024-09-10T21:13:44.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa0f8c00640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0dc0725e0 0x7fa0dc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:44.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa0f8c00640 1 -- 172.21.15.16:0/673796164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fa0f412cd90 con 0x7fa0fc161900 2024-09-10T21:13:44.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.007+0000 7fa100c00640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0dc0725e0 0x7fa0dc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:44.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.011+0000 7fa100c00640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0dc0725e0 0x7fa0dc074aa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000f70 tx=0x7fa0f0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:44.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.011+0000 7fa0f8c00640 1 -- 172.21.15.16:0/673796164 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa0fc1540f0 con 0x7fa0fc161900 2024-09-10T21:13:44.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa103600640 1 -- 172.21.15.16:0/673796164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0fc0b0340 con 0x7fa0fc161900 2024-09-10T21:13:44.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa0f8c00640 1 -- 172.21.15.16:0/673796164 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5234 (secure 0 0 0) 0x7fa0f40167d0 con 0x7fa0fc161900 2024-09-10T21:13:44.234 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:44.234 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-10T21:13:25:905095+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4029516332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4029516332},{"type":"v1","addr":"172.21.15.16:6836","nonce":4029516332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/806881470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":806881470},{"type":"v1","addr":"172.21.15.16:6837","nonce":806881470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":135167,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2949376643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2949376643},{"type":"v1","addr":"172.21.15.52:6834","nonce":2949376643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:22.294493+0000","modified":"2024-09-10T21:13:25.298310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135170},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135170":{"gid":135170,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/1574608409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1574608409},{"type":"v1","addr":"172.21.15.52:6835","nonce":1574608409}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135170,"qdb_cluster":[135170]},"id":5}]} 2024-09-10T21:13:44.234 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-10T21:13:44.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa103600640 1 -- 172.21.15.16:0/673796164 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0dc0725e0 msgr2=0x7fa0dc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa103600640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0dc0725e0 0x7fa0dc074aa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000f70 tx=0x7fa0f0009040 comp rx=0 tx=0).stop 2024-09-10T21:13:44.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa103600640 1 -- 172.21.15.16:0/673796164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc161900 msgr2=0x7fa0fc0b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa103600640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc161900 0x7fa0fc0b6240 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40b0040 tx=0x7fa0f4005ac0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa103600640 1 -- 172.21.15.16:0/673796164 shutdown_connections 2024-09-10T21:13:44.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa103600640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0dc0725e0 0x7fa0dc074aa0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa103600640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc161900 0x7fa0fc0b6240 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa103600640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0fc161520 0x7fa0fc153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa103600640 1 --2- 172.21.15.16:0/673796164 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa0fc0b29a0 0x7fa0fc1f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa103600640 1 -- 172.21.15.16:0/673796164 >> 172.21.15.16:0/673796164 conn(0x7fa0fc0d1d80 msgr2=0x7fa0fc0d2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:44.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa103600640 1 -- 172.21.15.16:0/673796164 shutdown_connections 2024-09-10T21:13:44.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.227+0000 7fa103600640 1 -- 172.21.15.16:0/673796164 wait complete. 2024-09-10T21:13:44.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.395+0000 7f6275200640 1 -- 172.21.15.16:0/1058339496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62700b13c0 msgr2=0x7f6270071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.395+0000 7f6275200640 1 --2- 172.21.15.16:0/1058339496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62700b13c0 0x7f6270071520 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6270073290 tx=0x7f625801c7a0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.395+0000 7f6275200640 1 -- 172.21.15.16:0/1058339496 shutdown_connections 2024-09-10T21:13:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.395+0000 7f6275200640 1 --2- 172.21.15.16:0/1058339496 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62700cfc80 0x7f62700d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.395+0000 7f6275200640 1 --2- 172.21.15.16:0/1058339496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62700b13c0 0x7f6270071520 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.395+0000 7f6275200640 1 --2- 172.21.15.16:0/1058339496 >> v2:172.21.15.52:3300/0 conn(0x7f627015f400 0x7f6270070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.395+0000 7f6275200640 1 -- 172.21.15.16:0/1058339496 >> 172.21.15.16:0/1058339496 conn(0x7f627009b130 msgr2=0x7f62700d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.395+0000 7f6275200640 1 -- 172.21.15.16:0/1058339496 shutdown_connections 2024-09-10T21:13:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.395+0000 7f6275200640 1 -- 172.21.15.16:0/1058339496 wait complete. 2024-09-10T21:13:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.395+0000 7f6275200640 1 Processor -- start 2024-09-10T21:13:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.395+0000 7f6275200640 1 -- start start 2024-09-10T21:13:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6275200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62700b13c0 0x7f627006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6275200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62700cfc80 0x7f62700b7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6275200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f627015f400 0x7f62701f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6275200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62700afd90 con 0x7f627015f400 2024-09-10T21:13:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6275200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62700d1260 con 0x7f62700cfc80 2024-09-10T21:13:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6275200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f62700aff10 con 0x7f62700b13c0 2024-09-10T21:13:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f626e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f627015f400 0x7f62701f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f626e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f627015f400 0x7f62701f8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39894/0 (socket says 172.21.15.16:39894) 2024-09-10T21:13:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f626e800640 1 -- 172.21.15.16:0/763641833 learned_addr learned my addr 172.21.15.16:0/763641833 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f626de00640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62700b13c0 0x7f627006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6267e00640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62700cfc80 0x7f62700b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f626e800640 1 -- 172.21.15.16:0/763641833 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62700b13c0 msgr2=0x7f627006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f626e800640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62700b13c0 0x7f627006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f626e800640 1 -- 172.21.15.16:0/763641833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62700cfc80 msgr2=0x7f62700b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f626e800640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62700cfc80 0x7f62700b7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f626e800640 1 -- 172.21.15.16:0/763641833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62701ed8e0 con 0x7f627015f400 2024-09-10T21:13:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6267e00640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62700cfc80 0x7f62700b7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f626e800640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f627015f400 0x7f62701f8070 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f626800eae0 tx=0x7f626800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:44.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6266000640 1 -- 172.21.15.16:0/763641833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268004b30 con 0x7f627015f400 2024-09-10T21:13:44.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6266000640 1 -- 172.21.15.16:0/763641833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6268004dc0 con 0x7f627015f400 2024-09-10T21:13:44.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6266000640 1 -- 172.21.15.16:0/763641833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268005280 con 0x7f627015f400 2024-09-10T21:13:44.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6275200640 1 -- 172.21.15.16:0/763641833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62701edbc0 con 0x7f627015f400 2024-09-10T21:13:44.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6275200640 1 -- 172.21.15.16:0/763641833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62701ee180 con 0x7f627015f400 2024-09-10T21:13:44.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.399+0000 7f6275200640 1 -- 172.21.15.16:0/763641833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62700d0110 con 0x7f627015f400 2024-09-10T21:13:44.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.403+0000 7f6266000640 1 -- 172.21.15.16:0/763641833 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6268005420 con 0x7f627015f400 2024-09-10T21:13:44.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.403+0000 7f6266000640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f624c0725e0 0x7f624c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:44.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.403+0000 7f6266000640 1 -- 172.21.15.16:0/763641833 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f62680940a0 con 0x7f627015f400 2024-09-10T21:13:44.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.403+0000 7f626de00640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f624c0725e0 0x7f624c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:44.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.403+0000 7f626de00640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f624c0725e0 0x7f624c074aa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6270155b20 tx=0x7f626000b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:44.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.407+0000 7f6266000640 1 -- 172.21.15.16:0/763641833 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f626805f6e0 con 0x7f627015f400 2024-09-10T21:13:44.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6275200640 1 -- 172.21.15.16:0/763641833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62700cee30 con 0x7f627015f400 2024-09-10T21:13:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6266000640 1 -- 172.21.15.16:0/763641833 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5234 (secure 0 0 0) 0x7f626805efe0 con 0x7f627015f400 2024-09-10T21:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:44.622 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-10T21:13:25:905095+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4029516332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4029516332},{"type":"v1","addr":"172.21.15.16:6836","nonce":4029516332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/806881470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":806881470},{"type":"v1","addr":"172.21.15.16:6837","nonce":806881470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":135167,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2949376643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2949376643},{"type":"v1","addr":"172.21.15.52:6834","nonce":2949376643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:22.294493+0000","modified":"2024-09-10T21:13:25.298310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135170},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135170":{"gid":135170,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/1574608409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1574608409},{"type":"v1","addr":"172.21.15.52:6835","nonce":1574608409}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135170,"qdb_cluster":[135170]},"id":5}]} 2024-09-10T21:13:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-10T21:13:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6275200640 1 -- 172.21.15.16:0/763641833 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f624c0725e0 msgr2=0x7f624c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6275200640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f624c0725e0 0x7f624c074aa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6270155b20 tx=0x7f626000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6275200640 1 -- 172.21.15.16:0/763641833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f627015f400 msgr2=0x7f62701f8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6275200640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f627015f400 0x7f62701f8070 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f626800eae0 tx=0x7f626800c2d0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6275200640 1 -- 172.21.15.16:0/763641833 shutdown_connections 2024-09-10T21:13:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6275200640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f624c0725e0 0x7f624c074aa0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6275200640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f627015f400 0x7f62701f8070 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6275200640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62700cfc80 0x7f62700b7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6275200640 1 --2- 172.21.15.16:0/763641833 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62700b13c0 0x7f627006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6275200640 1 -- 172.21.15.16:0/763641833 >> 172.21.15.16:0/763641833 conn(0x7f627009b130 msgr2=0x7f62701590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6275200640 1 -- 172.21.15.16:0/763641833 shutdown_connections 2024-09-10T21:13:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.615+0000 7f6275200640 1 -- 172.21.15.16:0/763641833 wait complete. 2024-09-10T21:13:44.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:13:44.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.783+0000 7fa86cc00640 1 -- 172.21.15.16:0/159747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa86815f400 msgr2=0x7fa868070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.783+0000 7fa86cc00640 1 --2- 172.21.15.16:0/159747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa86815f400 0x7fa868070fe0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa858005fd0 tx=0x7fa85801c700 comp rx=0 tx=0).stop 2024-09-10T21:13:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.783+0000 7fa86cc00640 1 -- 172.21.15.16:0/159747 shutdown_connections 2024-09-10T21:13:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.783+0000 7fa86cc00640 1 --2- 172.21.15.16:0/159747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa86815f400 0x7fa868070fe0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.783+0000 7fa86cc00640 1 -- 172.21.15.16:0/159747 >> 172.21.15.16:0/159747 conn(0x7fa86809b130 msgr2=0x7fa8680d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.783+0000 7fa86cc00640 1 -- 172.21.15.16:0/159747 shutdown_connections 2024-09-10T21:13:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.783+0000 7fa86cc00640 1 -- 172.21.15.16:0/159747 wait complete. 2024-09-10T21:13:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.783+0000 7fa86cc00640 1 Processor -- start 2024-09-10T21:13:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa86cc00640 1 -- start start 2024-09-10T21:13:44.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa86cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680b0590 0x7fa868152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:44.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa86cc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8680b13c0 0x7fa8680cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:44.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa86cc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa86815f400 0x7fa8680b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:44.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa86cc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa86816a800 con 0x7fa8680b0590 2024-09-10T21:13:44.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa86cc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa8680d1260 con 0x7fa8680b13c0 2024-09-10T21:13:44.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa86cc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa86816a980 con 0x7fa86815f400 2024-09-10T21:13:44.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa865e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680b0590 0x7fa868152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:44.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa865e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680b0590 0x7fa868152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39912/0 (socket says 172.21.15.16:39912) 2024-09-10T21:13:44.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa865e00640 1 -- 172.21.15.16:0/273485197 learned_addr learned my addr 172.21.15.16:0/273485197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:44.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa865e00640 1 -- 172.21.15.16:0/273485197 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa86815f400 msgr2=0x7fa8680b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa865e00640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa86815f400 0x7fa8680b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa865e00640 1 -- 172.21.15.16:0/273485197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8680b13c0 msgr2=0x7fa8680cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa85d400640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8680b13c0 0x7fa8680cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa865e00640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8680b13c0 0x7fa8680cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa865e00640 1 -- 172.21.15.16:0/273485197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8680c8f80 con 0x7fa8680b0590 2024-09-10T21:13:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa85d400640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8680b13c0 0x7fa8680cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa865e00640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680b0590 0x7fa868152c40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa858019310 tx=0x7fa858003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:44.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa85ea00640 1 -- 172.21.15.16:0/273485197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8580be070 con 0x7fa8680b0590 2024-09-10T21:13:44.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa85ea00640 1 -- 172.21.15.16:0/273485197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa858004030 con 0x7fa8680b0590 2024-09-10T21:13:44.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa85ea00640 1 -- 172.21.15.16:0/273485197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858005c40 con 0x7fa8680b0590 2024-09-10T21:13:44.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa86cc00640 1 -- 172.21.15.16:0/273485197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86815d310 con 0x7fa8680b0590 2024-09-10T21:13:44.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa86cc00640 1 -- 172.21.15.16:0/273485197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8680c9260 con 0x7fa8680b0590 2024-09-10T21:13:44.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.787+0000 7fa86cc00640 1 -- 172.21.15.16:0/273485197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa868070fe0 con 0x7fa8680b0590 2024-09-10T21:13:44.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.791+0000 7fa85ea00640 1 -- 172.21.15.16:0/273485197 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa8580041e0 con 0x7fa8680b0590 2024-09-10T21:13:44.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.791+0000 7fa85ea00640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa844072610 0x7fa844074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:44.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.791+0000 7fa85ea00640 1 -- 172.21.15.16:0/273485197 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fa85812d4a0 con 0x7fa8680b0590 2024-09-10T21:13:44.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.791+0000 7fa85d400640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa844072610 0x7fa844074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.791+0000 7fa85d400640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa844072610 0x7fa844074ad0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa8680c5530 tx=0x7fa850009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.795+0000 7fa85ea00640 1 -- 172.21.15.16:0/273485197 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa8580ac070 con 0x7fa8680b0590 2024-09-10T21:13:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.971+0000 7fa86cc00640 1 -- 172.21.15.16:0/273485197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa86815e910 con 0x7fa8680b0590 2024-09-10T21:13:44.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.971+0000 7fa85ea00640 1 -- 172.21.15.16:0/273485197 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16620 (secure 0 0 0) 0x7fa858016410 con 0x7fa8680b0590 2024-09-10T21:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:44.979 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:13:23.893029+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:13:20.010491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T21:13:21.255527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:13:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.975+0000 7fa86cc00640 1 -- 172.21.15.16:0/273485197 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa844072610 msgr2=0x7fa844074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.975+0000 7fa86cc00640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa844072610 0x7fa844074ad0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa8680c5530 tx=0x7fa850009040 comp rx=0 tx=0).stop 2024-09-10T21:13:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.975+0000 7fa86cc00640 1 -- 172.21.15.16:0/273485197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680b0590 msgr2=0x7fa868152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.975+0000 7fa86cc00640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680b0590 0x7fa868152c40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa858019310 tx=0x7fa858003d40 comp rx=0 tx=0).stop 2024-09-10T21:13:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.975+0000 7fa86cc00640 1 -- 172.21.15.16:0/273485197 shutdown_connections 2024-09-10T21:13:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.975+0000 7fa86cc00640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa844072610 0x7fa844074ad0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.975+0000 7fa86cc00640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa86815f400 0x7fa8680b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.975+0000 7fa86cc00640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8680b13c0 0x7fa8680cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.975+0000 7fa86cc00640 1 --2- 172.21.15.16:0/273485197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680b0590 0x7fa868152c40 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.975+0000 7fa86cc00640 1 -- 172.21.15.16:0/273485197 >> 172.21.15.16:0/273485197 conn(0x7fa86809b130 msgr2=0x7fa8680ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.975+0000 7fa86cc00640 1 -- 172.21.15.16:0/273485197 shutdown_connections 2024-09-10T21:13:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:44.975+0000 7fa86cc00640 1 -- 172.21.15.16:0/273485197 wait complete. 2024-09-10T21:13:45.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:13:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 -- 172.21.15.16:0/1575116848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50ac0d4b00 msgr2=0x7f50ac0cf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 --2- 172.21.15.16:0/1575116848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50ac0d4b00 0x7f50ac0cf0c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5094009840 tx=0x7f509401c7f0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 -- 172.21.15.16:0/1575116848 shutdown_connections 2024-09-10T21:13:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 --2- 172.21.15.16:0/1575116848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50ac0d4b00 0x7f50ac0cf0c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 --2- 172.21.15.16:0/1575116848 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50ac15e200 0x7f50ac0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 -- 172.21.15.16:0/1575116848 >> 172.21.15.16:0/1575116848 conn(0x7f50ac0d3f70 msgr2=0x7f50ac0d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 -- 172.21.15.16:0/1575116848 shutdown_connections 2024-09-10T21:13:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 -- 172.21.15.16:0/1575116848 wait complete. 2024-09-10T21:13:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 Processor -- start 2024-09-10T21:13:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 -- start start 2024-09-10T21:13:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50ac0cf600 0x7f50ac14baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50ac0d4b00 0x7f50ac14f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50ac15e200 0x7f50ac0b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac16b4c0 con 0x7f50ac0cf600 2024-09-10T21:13:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac16b340 con 0x7f50ac15e200 2024-09-10T21:13:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50b2a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f50ac16b640 con 0x7f50ac0d4b00 2024-09-10T21:13:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.143+0000 7f50abe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50ac0cf600 0x7f50ac14baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50abe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50ac0cf600 0x7f50ac14baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39940/0 (socket says 172.21.15.16:39940) 2024-09-10T21:13:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50abe00640 1 -- 172.21.15.16:0/3591150701 learned_addr learned my addr 172.21.15.16:0/3591150701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50a3e00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50ac0d4b00 0x7f50ac14f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50b0a00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50ac15e200 0x7f50ac0b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50abe00640 1 -- 172.21.15.16:0/3591150701 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50ac0d4b00 msgr2=0x7f50ac14f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50abe00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50ac0d4b00 0x7f50ac14f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50abe00640 1 -- 172.21.15.16:0/3591150701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50ac15e200 msgr2=0x7f50ac0b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50abe00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50ac15e200 0x7f50ac0b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50abe00640 1 -- 172.21.15.16:0/3591150701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ac14c390 con 0x7f50ac0cf600 2024-09-10T21:13:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50b0a00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50ac15e200 0x7f50ac0b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50abe00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50ac0cf600 0x7f50ac14baa0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f509c0058d0 tx=0x7f509c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50a8c00640 1 -- 172.21.15.16:0/3591150701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c017070 con 0x7f50ac0cf600 2024-09-10T21:13:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50a8c00640 1 -- 172.21.15.16:0/3591150701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f509c009d20 con 0x7f50ac0cf600 2024-09-10T21:13:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50a8c00640 1 -- 172.21.15.16:0/3591150701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c0106c0 con 0x7f50ac0cf600 2024-09-10T21:13:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50b2a00640 1 -- 172.21.15.16:0/3591150701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ac1f7f50 con 0x7f50ac0cf600 2024-09-10T21:13:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50b2a00640 1 -- 172.21.15.16:0/3591150701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ac1f87b0 con 0x7f50ac0cf600 2024-09-10T21:13:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50b2a00640 1 -- 172.21.15.16:0/3591150701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50ac0d0960 con 0x7f50ac0cf600 2024-09-10T21:13:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.147+0000 7f50a8c00640 1 -- 172.21.15.16:0/3591150701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f509c010860 con 0x7f50ac0cf600 2024-09-10T21:13:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.151+0000 7f50a8c00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50880725e0 0x7f5088074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.151+0000 7f50a8c00640 1 -- 172.21.15.16:0/3591150701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f509c0936c0 con 0x7f50ac0cf600 2024-09-10T21:13:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.151+0000 7f50a3e00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50880725e0 0x7f5088074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.151+0000 7f50a3e00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50880725e0 0x7f5088074aa0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f50940af000 tx=0x7f509401b030 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:45.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.151+0000 7f50a8c00640 1 -- 172.21.15.16:0/3591150701 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f50ac14c390 con 0x7f50ac0cf600 2024-09-10T21:13:45.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.359+0000 7f50b2a00640 1 -- 172.21.15.16:0/3591150701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f50ac0cae00 con 0x7f50ac0cf600 2024-09-10T21:13:45.368 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-10T21:13:45.359+0000 7ff429400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:13:45.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.363+0000 7f50a8c00640 1 -- 172.21.15.16:0/3591150701 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) ==== 161+0+0 (secure 0 0 0) 0x7f509c05e6b0 con 0x7f50ac0cf600 2024-09-10T21:13:45.370 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:13:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.367+0000 7f50b2a00640 1 -- 172.21.15.16:0/3591150701 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50880725e0 msgr2=0x7f5088074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.367+0000 7f50b2a00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50880725e0 0x7f5088074aa0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f50940af000 tx=0x7f509401b030 comp rx=0 tx=0).stop 2024-09-10T21:13:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.367+0000 7f50b2a00640 1 -- 172.21.15.16:0/3591150701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50ac0cf600 msgr2=0x7f50ac14baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.367+0000 7f50b2a00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50ac0cf600 0x7f50ac14baa0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f509c0058d0 tx=0x7f509c00dde0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.367+0000 7f50b2a00640 1 -- 172.21.15.16:0/3591150701 shutdown_connections 2024-09-10T21:13:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.367+0000 7f50b2a00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50880725e0 0x7f5088074aa0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.367+0000 7f50b2a00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50ac15e200 0x7f50ac0b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.367+0000 7f50b2a00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50ac0d4b00 0x7f50ac14f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.367+0000 7f50b2a00640 1 --2- 172.21.15.16:0/3591150701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50ac0cf600 0x7f50ac14baa0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.367+0000 7f50b2a00640 1 -- 172.21.15.16:0/3591150701 >> 172.21.15.16:0/3591150701 conn(0x7f50ac0d3f70 msgr2=0x7f50ac154750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:45.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.367+0000 7f50b2a00640 1 -- 172.21.15.16:0/3591150701 shutdown_connections 2024-09-10T21:13:45.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.367+0000 7f50b2a00640 1 -- 172.21.15.16:0/3591150701 wait complete. 2024-09-10T21:13:45.394 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:13:45.443 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:13:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.535+0000 7ff976200640 1 -- 172.21.15.16:0/215885187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970161680 msgr2=0x7ff9700d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.535+0000 7ff976200640 1 --2- 172.21.15.16:0/215885187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970161680 0x7ff9700d0a30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff970162c10 tx=0x7ff96001c740 comp rx=0 tx=0).stop 2024-09-10T21:13:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.535+0000 7ff976200640 1 -- 172.21.15.16:0/215885187 shutdown_connections 2024-09-10T21:13:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.535+0000 7ff976200640 1 --2- 172.21.15.16:0/215885187 >> v2:172.21.15.52:3300/0 conn(0x7ff970162710 0x7ff9700b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.535+0000 7ff976200640 1 --2- 172.21.15.16:0/215885187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970161680 0x7ff9700d0a30 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.535+0000 7ff976200640 1 -- 172.21.15.16:0/215885187 >> 172.21.15.16:0/215885187 conn(0x7ff9700d1ce0 msgr2=0x7ff9700d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.535+0000 7ff976200640 1 -- 172.21.15.16:0/215885187 shutdown_connections 2024-09-10T21:13:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.535+0000 7ff976200640 1 -- 172.21.15.16:0/215885187 wait complete. 2024-09-10T21:13:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff976200640 1 Processor -- start 2024-09-10T21:13:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff976200640 1 -- start start 2024-09-10T21:13:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff976200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff970161680 0x7ff970157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff976200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff970161b70 0x7ff9700b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff976200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970162710 0x7ff9700c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff976200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff97016aeb0 con 0x7ff970162710 2024-09-10T21:13:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff976200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff9700b3570 con 0x7ff970161680 2024-09-10T21:13:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff976200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff97016b030 con 0x7ff970161b70 2024-09-10T21:13:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970162710 0x7ff9700c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970162710 0x7ff9700c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39950/0 (socket says 172.21.15.16:39950) 2024-09-10T21:13:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff96fe00640 1 -- 172.21.15.16:0/1126708826 learned_addr learned my addr 172.21.15.16:0/1126708826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff96fe00640 1 -- 172.21.15.16:0/1126708826 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff970161b70 msgr2=0x7ff9700b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff96fe00640 1 --2- 172.21.15.16:0/1126708826 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff970161b70 0x7ff9700b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff96fe00640 1 -- 172.21.15.16:0/1126708826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff970161680 msgr2=0x7ff970157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff96fe00640 1 --2- 172.21.15.16:0/1126708826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff970161680 0x7ff970157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff96fe00640 1 -- 172.21.15.16:0/1126708826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9700ce860 con 0x7ff970162710 2024-09-10T21:13:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff96fe00640 1 --2- 172.21.15.16:0/1126708826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970162710 0x7ff9700c8170 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff96800da10 tx=0x7ff96800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff967400640 1 -- 172.21.15.16:0/1126708826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968017070 con 0x7ff970162710 2024-09-10T21:13:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff967400640 1 -- 172.21.15.16:0/1126708826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff968004b30 con 0x7ff970162710 2024-09-10T21:13:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff967400640 1 -- 172.21.15.16:0/1126708826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96800be60 con 0x7ff970162710 2024-09-10T21:13:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff976200640 1 -- 172.21.15.16:0/1126708826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9700b63e0 con 0x7ff970162710 2024-09-10T21:13:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.539+0000 7ff976200640 1 -- 172.21.15.16:0/1126708826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9700cedc0 con 0x7ff970162710 2024-09-10T21:13:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.543+0000 7ff976200640 1 -- 172.21.15.16:0/1126708826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9700d0a30 con 0x7ff970162710 2024-09-10T21:13:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.543+0000 7ff967400640 1 -- 172.21.15.16:0/1126708826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff968010460 con 0x7ff970162710 2024-09-10T21:13:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.543+0000 7ff967400640 1 --2- 172.21.15.16:0/1126708826 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff94c0725e0 0x7ff94c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.543+0000 7ff967400640 1 -- 172.21.15.16:0/1126708826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7ff968093720 con 0x7ff970162710 2024-09-10T21:13:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.543+0000 7ff96f400640 1 --2- 172.21.15.16:0/1126708826 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff94c0725e0 0x7ff94c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.543+0000 7ff96f400640 1 --2- 172.21.15.16:0/1126708826 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff94c0725e0 0x7ff94c074aa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff9600af000 tx=0x7ff9600194e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.547+0000 7ff967400640 1 -- 172.21.15.16:0/1126708826 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff968020070 con 0x7ff970162710 2024-09-10T21:13:45.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:45.755+0000 7ff976200640 1 -- 172.21.15.16:0/1126708826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff9700704a0 con 0x7ff970162710 2024-09-10T21:13:46.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.371+0000 7ff967400640 1 -- 172.21.15.16:0/1126708826 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) ==== 104+0+0 (secure 0 0 0) 0x7ff9700ce860 con 0x7ff970162710 2024-09-10T21:13:46.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.371+0000 7ff976200640 1 -- 172.21.15.16:0/1126708826 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff94c0725e0 msgr2=0x7ff94c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:46.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.371+0000 7ff976200640 1 --2- 172.21.15.16:0/1126708826 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff94c0725e0 0x7ff94c074aa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff9600af000 tx=0x7ff9600194e0 comp rx=0 tx=0).stop 2024-09-10T21:13:46.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.371+0000 7ff976200640 1 -- 172.21.15.16:0/1126708826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970162710 msgr2=0x7ff9700c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:46.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.371+0000 7ff976200640 1 --2- 172.21.15.16:0/1126708826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970162710 0x7ff9700c8170 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff96800da10 tx=0x7ff96800ded0 comp rx=0 tx=0).stop 2024-09-10T21:13:46.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.371+0000 7ff976200640 1 -- 172.21.15.16:0/1126708826 shutdown_connections 2024-09-10T21:13:46.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.371+0000 7ff976200640 1 --2- 172.21.15.16:0/1126708826 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff94c0725e0 0x7ff94c074aa0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:46.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.371+0000 7ff976200640 1 --2- 172.21.15.16:0/1126708826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff970162710 0x7ff9700c8170 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:46.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.371+0000 7ff976200640 1 --2- 172.21.15.16:0/1126708826 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff970161b70 0x7ff9700b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:46.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.371+0000 7ff976200640 1 --2- 172.21.15.16:0/1126708826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff970161680 0x7ff970157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:46.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.371+0000 7ff976200640 1 -- 172.21.15.16:0/1126708826 >> 172.21.15.16:0/1126708826 conn(0x7ff9700d1ce0 msgr2=0x7ff9700af530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:46.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.371+0000 7ff976200640 1 -- 172.21.15.16:0/1126708826 shutdown_connections 2024-09-10T21:13:46.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.371+0000 7ff976200640 1 -- 172.21.15.16:0/1126708826 wait complete. 2024-09-10T21:13:46.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:13:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 -- 172.21.15.16:0/3488616303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c0cfc40 msgr2=0x7f9e7c15ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 --2- 172.21.15.16:0/3488616303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c0cfc40 0x7f9e7c15ff50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c073ae0 tx=0x7f9e6401c800 comp rx=0 tx=0).stop 2024-09-10T21:13:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 -- 172.21.15.16:0/3488616303 shutdown_connections 2024-09-10T21:13:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 --2- 172.21.15.16:0/3488616303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c0cfc40 0x7f9e7c15ff50 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 --2- 172.21.15.16:0/3488616303 >> v2:172.21.15.52:3300/0 conn(0x7f9e7c0ce7c0 0x7f9e7c0ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 -- 172.21.15.16:0/3488616303 >> 172.21.15.16:0/3488616303 conn(0x7f9e7c0d1e60 msgr2=0x7f9e7c0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 -- 172.21.15.16:0/3488616303 shutdown_connections 2024-09-10T21:13:46.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 -- 172.21.15.16:0/3488616303 wait complete. 2024-09-10T21:13:46.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 Processor -- start 2024-09-10T21:13:46.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 -- start start 2024-09-10T21:13:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e7c0ce7c0 0x7f9e7c0b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e7c0cfc40 0x7f9e7c15baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c160490 0x7f9e7c0b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c1693a0 con 0x7f9e7c160490 2024-09-10T21:13:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c169220 con 0x7f9e7c0cfc40 2024-09-10T21:13:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.547+0000 7f9e82200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7c169520 con 0x7f9e7c0ce7c0 2024-09-10T21:13:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c160490 0x7f9e7c0b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c160490 0x7f9e7c0b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39964/0 (socket says 172.21.15.16:39964) 2024-09-10T21:13:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e7be00640 1 -- 172.21.15.16:0/2859379031 learned_addr learned my addr 172.21.15.16:0/2859379031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e7b400640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e7c0ce7c0 0x7f9e7c0b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e73e00640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e7c0cfc40 0x7f9e7c15baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e7be00640 1 -- 172.21.15.16:0/2859379031 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e7c0ce7c0 msgr2=0x7f9e7c0b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e7be00640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e7c0ce7c0 0x7f9e7c0b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e7be00640 1 -- 172.21.15.16:0/2859379031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e7c0cfc40 msgr2=0x7f9e7c15baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e7be00640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e7c0cfc40 0x7f9e7c15baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e7be00640 1 -- 172.21.15.16:0/2859379031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c1536a0 con 0x7f9e7c160490 2024-09-10T21:13:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e7b400640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e7c0ce7c0 0x7f9e7c0b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e7be00640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c160490 0x7f9e7c0b7280 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400cb10 tx=0x7f9e7400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e73400640 1 -- 172.21.15.16:0/2859379031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74016020 con 0x7f9e7c160490 2024-09-10T21:13:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e73400640 1 -- 172.21.15.16:0/2859379031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e7400ce50 con 0x7f9e7c160490 2024-09-10T21:13:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e82200640 1 -- 172.21.15.16:0/2859379031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7c0b4f70 con 0x7f9e7c160490 2024-09-10T21:13:46.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e73400640 1 -- 172.21.15.16:0/2859379031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7400ae00 con 0x7f9e7c160490 2024-09-10T21:13:46.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e82200640 1 -- 172.21.15.16:0/2859379031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7c15ea80 con 0x7f9e7c160490 2024-09-10T21:13:46.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e82200640 1 -- 172.21.15.16:0/2859379031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e7c0ceba0 con 0x7f9e7c160490 2024-09-10T21:13:46.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.551+0000 7f9e73400640 1 -- 172.21.15.16:0/2859379031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9e74012420 con 0x7f9e7c160490 2024-09-10T21:13:46.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.555+0000 7f9e73400640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e580725e0 0x7f9e58074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:46.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.555+0000 7f9e73400640 1 -- 172.21.15.16:0/2859379031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f9e74093890 con 0x7f9e7c160490 2024-09-10T21:13:46.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.555+0000 7f9e7b400640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e580725e0 0x7f9e58074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:46.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.555+0000 7f9e7b400640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e580725e0 0x7f9e58074aa0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0058d0 tx=0x7f9e6c005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:46.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.559+0000 7f9e73400640 1 -- 172.21.15.16:0/2859379031 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9e74020020 con 0x7f9e7c160490 2024-09-10T21:13:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:46.735+0000 7f9e82200640 1 -- 172.21.15.16:0/2859379031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9e7c153910 con 0x7f9e7c160490 2024-09-10T21:13:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.579+0000 7f9e73400640 1 -- 172.21.15.16:0/2859379031 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) ==== 181+0+0 (secure 0 0 0) 0x7f9e7405ee70 con 0x7f9e7c160490 2024-09-10T21:13:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:13:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.587+0000 7f9e82200640 1 -- 172.21.15.16:0/2859379031 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e580725e0 msgr2=0x7f9e58074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.587+0000 7f9e82200640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e580725e0 0x7f9e58074aa0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0058d0 tx=0x7f9e6c005860 comp rx=0 tx=0).stop 2024-09-10T21:13:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.587+0000 7f9e82200640 1 -- 172.21.15.16:0/2859379031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c160490 msgr2=0x7f9e7c0b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.587+0000 7f9e82200640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c160490 0x7f9e7c0b7280 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400cb10 tx=0x7f9e7400cc00 comp rx=0 tx=0).stop 2024-09-10T21:13:47.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.587+0000 7f9e82200640 1 -- 172.21.15.16:0/2859379031 shutdown_connections 2024-09-10T21:13:47.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.587+0000 7f9e82200640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e580725e0 0x7f9e58074aa0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:47.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.587+0000 7f9e82200640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c160490 0x7f9e7c0b7280 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:47.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.587+0000 7f9e82200640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e7c0cfc40 0x7f9e7c15baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:47.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.587+0000 7f9e82200640 1 --2- 172.21.15.16:0/2859379031 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e7c0ce7c0 0x7f9e7c0b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:47.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.587+0000 7f9e82200640 1 -- 172.21.15.16:0/2859379031 >> 172.21.15.16:0/2859379031 conn(0x7f9e7c0d1e60 msgr2=0x7f9e7c0708b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:47.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.587+0000 7f9e82200640 1 -- 172.21.15.16:0/2859379031 shutdown_connections 2024-09-10T21:13:47.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.587+0000 7f9e82200640 1 -- 172.21.15.16:0/2859379031 wait complete. 2024-09-10T21:13:47.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:13:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.771+0000 7f85ef800640 1 -- 172.21.15.16:0/1102742766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e8160660 msgr2=0x7f85e80b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.771+0000 7f85ef800640 1 --2- 172.21.15.16:0/1102742766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e8160660 0x7f85e80b1860 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f85e8073ae0 tx=0x7f85e001cd80 comp rx=0 tx=0).stop 2024-09-10T21:13:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.771+0000 7f85ef800640 1 -- 172.21.15.16:0/1102742766 shutdown_connections 2024-09-10T21:13:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.771+0000 7f85ef800640 1 --2- 172.21.15.16:0/1102742766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e8160660 0x7f85e80b1860 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.771+0000 7f85ef800640 1 --2- 172.21.15.16:0/1102742766 >> v2:172.21.15.52:3300/0 conn(0x7f85e80ce7c0 0x7f85e80ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:47.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.771+0000 7f85ef800640 1 -- 172.21.15.16:0/1102742766 >> 172.21.15.16:0/1102742766 conn(0x7f85e80d1e60 msgr2=0x7f85e80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.771+0000 7f85ef800640 1 -- 172.21.15.16:0/1102742766 shutdown_connections 2024-09-10T21:13:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.771+0000 7f85ef800640 1 -- 172.21.15.16:0/1102742766 wait complete. 2024-09-10T21:13:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ef800640 1 Processor -- start 2024-09-10T21:13:47.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ef800640 1 -- start start 2024-09-10T21:13:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ef800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e80ce7c0 0x7f85e8154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ef800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f85e80cfc40 0x7f85e81f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ef800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e8160660 0x7f85e806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ef800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85e80b29c0 con 0x7f85e8160660 2024-09-10T21:13:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ef800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f85e80b2840 con 0x7f85e80ce7c0 2024-09-10T21:13:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ef800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f85e80b2b40 con 0x7f85e80cfc40 2024-09-10T21:13:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ed800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e8160660 0x7f85e806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ed800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e8160660 0x7f85e806ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39986/0 (socket says 172.21.15.16:39986) 2024-09-10T21:13:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ed800640 1 -- 172.21.15.16:0/3677425906 learned_addr learned my addr 172.21.15.16:0/3677425906 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85dfe00640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f85e80cfc40 0x7f85e81f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ece00640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e80ce7c0 0x7f85e8154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ed800640 1 -- 172.21.15.16:0/3677425906 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f85e80cfc40 msgr2=0x7f85e81f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ed800640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f85e80cfc40 0x7f85e81f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ed800640 1 -- 172.21.15.16:0/3677425906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e80ce7c0 msgr2=0x7f85e8154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ed800640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e80ce7c0 0x7f85e8154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ed800640 1 -- 172.21.15.16:0/3677425906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e815afd0 con 0x7f85e8160660 2024-09-10T21:13:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ece00640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e80ce7c0 0x7f85e8154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85dfe00640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f85e80cfc40 0x7f85e81f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ed800640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e8160660 0x7f85e806ed40 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009bd0 tx=0x7f85e0005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85e5600640 1 -- 172.21.15.16:0/3677425906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00bc070 con 0x7f85e8160660 2024-09-10T21:13:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85e5600640 1 -- 172.21.15.16:0/3677425906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85e00040f0 con 0x7f85e8160660 2024-09-10T21:13:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85e5600640 1 -- 172.21.15.16:0/3677425906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00b2040 con 0x7f85e8160660 2024-09-10T21:13:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ef800640 1 -- 172.21.15.16:0/3677425906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e81f7920 con 0x7f85e8160660 2024-09-10T21:13:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.775+0000 7f85ef800640 1 -- 172.21.15.16:0/3677425906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e81fbca0 con 0x7f85e8160660 2024-09-10T21:13:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.779+0000 7f85ef800640 1 -- 172.21.15.16:0/3677425906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e80ceba0 con 0x7f85e8160660 2024-09-10T21:13:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.779+0000 7f85e5600640 1 -- 172.21.15.16:0/3677425906 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f85e00b5070 con 0x7f85e8160660 2024-09-10T21:13:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.779+0000 7f85e5600640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f85c40725e0 0x7f85c4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.779+0000 7f85e5600640 1 -- 172.21.15.16:0/3677425906 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5357+0+0 (secure 0 0 0) 0x7f85e012cc10 con 0x7f85e8160660 2024-09-10T21:13:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.779+0000 7f85ece00640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f85c40725e0 0x7f85c4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.779+0000 7f85ece00640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f85c40725e0 0x7f85c4074aa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009b60 tx=0x7f85d80091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.783+0000 7f85e5600640 1 -- 172.21.15.16:0/3677425906 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f85e0016970 con 0x7f85e8160660 2024-09-10T21:13:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:47.959+0000 7f85ef800640 1 -- 172.21.15.16:0/3677425906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f85e815c240 con 0x7f85e8160660 2024-09-10T21:13:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.583+0000 7f85e5600640 1 -- 172.21.15.16:0/3677425906 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) ==== 169+0+0 (secure 0 0 0) 0x7f85e00f8440 con 0x7f85e8160660 2024-09-10T21:13:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-10T21:13:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.587+0000 7f85ef800640 1 -- 172.21.15.16:0/3677425906 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f85c40725e0 msgr2=0x7f85c4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.587+0000 7f85ef800640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f85c40725e0 0x7f85c4074aa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009b60 tx=0x7f85d80091e0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.587+0000 7f85ef800640 1 -- 172.21.15.16:0/3677425906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e8160660 msgr2=0x7f85e806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.587+0000 7f85ef800640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e8160660 0x7f85e806ed40 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009bd0 tx=0x7f85e0005ae0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.587+0000 7f85ef800640 1 -- 172.21.15.16:0/3677425906 shutdown_connections 2024-09-10T21:13:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.587+0000 7f85ef800640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f85c40725e0 0x7f85c4074aa0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.587+0000 7f85ef800640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e8160660 0x7f85e806ed40 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.587+0000 7f85ef800640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f85e80cfc40 0x7f85e81f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.587+0000 7f85ef800640 1 --2- 172.21.15.16:0/3677425906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85e80ce7c0 0x7f85e8154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.587+0000 7f85ef800640 1 -- 172.21.15.16:0/3677425906 >> 172.21.15.16:0/3677425906 conn(0x7f85e80d1e60 msgr2=0x7f85e80d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.587+0000 7f85ef800640 1 -- 172.21.15.16:0/3677425906 shutdown_connections 2024-09-10T21:13:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.587+0000 7f85ef800640 1 -- 172.21.15.16:0/3677425906 wait complete. 2024-09-10T21:13:48.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:13:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 -- 172.21.15.16:0/2458018926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a940b2860 msgr2=0x7f6a94160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 --2- 172.21.15.16:0/2458018926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a940b2860 0x7f6a94160510 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009bc0 tx=0x7f6a8c01cba0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 -- 172.21.15.16:0/2458018926 shutdown_connections 2024-09-10T21:13:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 --2- 172.21.15.16:0/2458018926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a940b2860 0x7f6a94160510 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 --2- 172.21.15.16:0/2458018926 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a940cffe0 0x7f6a940ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 -- 172.21.15.16:0/2458018926 >> 172.21.15.16:0/2458018926 conn(0x7f6a940d1d80 msgr2=0x7f6a940d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 -- 172.21.15.16:0/2458018926 shutdown_connections 2024-09-10T21:13:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 -- 172.21.15.16:0/2458018926 wait complete. 2024-09-10T21:13:48.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 Processor -- start 2024-09-10T21:13:48.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 -- start start 2024-09-10T21:13:48.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a940b2860 0x7f6a941471d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:48.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a940cffe0 0x7f6a94155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:48.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a940d03c0 0x7f6a940b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:48.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a940cd910 con 0x7f6a940b2860 2024-09-10T21:13:48.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a940cd790 con 0x7f6a940cffe0 2024-09-10T21:13:48.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a98e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6a940cda90 con 0x7f6a940d03c0 2024-09-10T21:13:48.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a91e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a940b2860 0x7f6a941471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:48.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a91e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a940b2860 0x7f6a941471d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39990/0 (socket says 172.21.15.16:39990) 2024-09-10T21:13:48.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a91e00640 1 -- 172.21.15.16:0/4264185953 learned_addr learned my addr 172.21.15.16:0/4264185953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:48.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a8be00640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a940cffe0 0x7f6a94155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:48.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a92800640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a940d03c0 0x7f6a940b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:48.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a91e00640 1 -- 172.21.15.16:0/4264185953 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a940d03c0 msgr2=0x7f6a940b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:48.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a91e00640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a940d03c0 0x7f6a940b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a91e00640 1 -- 172.21.15.16:0/4264185953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a940cffe0 msgr2=0x7f6a94155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:48.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a91e00640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a940cffe0 0x7f6a94155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a91e00640 1 -- 172.21.15.16:0/4264185953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a94158130 con 0x7f6a940b2860 2024-09-10T21:13:48.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a8be00640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a940cffe0 0x7f6a94155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:48.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.771+0000 7f6a92800640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a940d03c0 0x7f6a940b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:48.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.775+0000 7f6a91e00640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a940b2860 0x7f6a941471d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400d980 tx=0x7f6a8400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.775+0000 7f6a8a000640 1 -- 172.21.15.16:0/4264185953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84017070 con 0x7f6a940b2860 2024-09-10T21:13:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.775+0000 7f6a8a000640 1 -- 172.21.15.16:0/4264185953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a84009cc0 con 0x7f6a940b2860 2024-09-10T21:13:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.775+0000 7f6a8a000640 1 -- 172.21.15.16:0/4264185953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a840106c0 con 0x7f6a940b2860 2024-09-10T21:13:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.775+0000 7f6a98e00640 1 -- 172.21.15.16:0/4264185953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a94158410 con 0x7f6a940b2860 2024-09-10T21:13:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.775+0000 7f6a98e00640 1 -- 172.21.15.16:0/4264185953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a941589d0 con 0x7f6a940b2860 2024-09-10T21:13:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.775+0000 7f6a98e00640 1 -- 172.21.15.16:0/4264185953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a940ce2b0 con 0x7f6a940b2860 2024-09-10T21:13:48.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.775+0000 7f6a8a000640 1 -- 172.21.15.16:0/4264185953 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6a84010860 con 0x7f6a940b2860 2024-09-10T21:13:48.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.775+0000 7f6a8a000640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6a70072610 0x7f6a70074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.775+0000 7f6a8a000640 1 -- 172.21.15.16:0/4264185953 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7f6a840932e0 con 0x7f6a940b2860 2024-09-10T21:13:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.779+0000 7f6a8be00640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6a70072610 0x7f6a70074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.779+0000 7f6a8be00640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6a70072610 0x7f6a70074ad0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6a940ce700 tx=0x7f6a7c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.779+0000 7f6a8a000640 1 -- 172.21.15.16:0/4264185953 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6a8405ecb0 con 0x7f6a940b2860 2024-09-10T21:13:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.955+0000 7f6a98e00640 1 -- 172.21.15.16:0/4264185953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f6a940cf950 con 0x7f6a940b2860 2024-09-10T21:13:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.959+0000 7f6a8a000640 1 -- 172.21.15.16:0/4264185953 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v38) ==== 71+0+0 (secure 0 0 0) 0x7f6a840930c0 con 0x7f6a940b2860 2024-09-10T21:13:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.963+0000 7f6a98e00640 1 -- 172.21.15.16:0/4264185953 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6a70072610 msgr2=0x7f6a70074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.963+0000 7f6a98e00640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6a70072610 0x7f6a70074ad0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6a940ce700 tx=0x7f6a7c009040 comp rx=0 tx=0).stop 2024-09-10T21:13:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.963+0000 7f6a98e00640 1 -- 172.21.15.16:0/4264185953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a940b2860 msgr2=0x7f6a941471d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.963+0000 7f6a98e00640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a940b2860 0x7f6a941471d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400d980 tx=0x7f6a8400de40 comp rx=0 tx=0).stop 2024-09-10T21:13:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.963+0000 7f6a98e00640 1 -- 172.21.15.16:0/4264185953 shutdown_connections 2024-09-10T21:13:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.963+0000 7f6a98e00640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6a70072610 0x7f6a70074ad0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.963+0000 7f6a98e00640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a940d03c0 0x7f6a940b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.963+0000 7f6a98e00640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a940cffe0 0x7f6a94155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.963+0000 7f6a98e00640 1 --2- 172.21.15.16:0/4264185953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a940b2860 0x7f6a941471d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.963+0000 7f6a98e00640 1 -- 172.21.15.16:0/4264185953 >> 172.21.15.16:0/4264185953 conn(0x7f6a940d1d80 msgr2=0x7f6a940b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.963+0000 7f6a98e00640 1 -- 172.21.15.16:0/4264185953 shutdown_connections 2024-09-10T21:13:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:48.963+0000 7f6a98e00640 1 -- 172.21.15.16:0/4264185953 wait complete. 2024-09-10T21:13:48.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2024-09-10T21:13:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.127+0000 7f6db2200640 1 -- 172.21.15.16:0/440009525 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6dac0716f0 msgr2=0x7f6dac0af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.127+0000 7f6db2200640 1 --2- 172.21.15.16:0/440009525 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6dac0716f0 0x7f6dac0af580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009840 tx=0x7f6d9401c700 comp rx=0 tx=0).stop 2024-09-10T21:13:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.127+0000 7f6db2200640 1 -- 172.21.15.16:0/440009525 shutdown_connections 2024-09-10T21:13:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.127+0000 7f6db2200640 1 --2- 172.21.15.16:0/440009525 >> v2:172.21.15.52:3300/0 conn(0x7f6dac0b13c0 0x7f6dac0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.127+0000 7f6db2200640 1 --2- 172.21.15.16:0/440009525 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6dac0716f0 0x7f6dac0af580 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.127+0000 7f6db2200640 1 -- 172.21.15.16:0/440009525 >> 172.21.15.16:0/440009525 conn(0x7f6dac09b130 msgr2=0x7f6dac0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.127+0000 7f6db2200640 1 -- 172.21.15.16:0/440009525 shutdown_connections 2024-09-10T21:13:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.127+0000 7f6db2200640 1 -- 172.21.15.16:0/440009525 wait complete. 2024-09-10T21:13:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6db2200640 1 Processor -- start 2024-09-10T21:13:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6db2200640 1 -- start start 2024-09-10T21:13:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6db2200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dac0716f0 0x7f6dac0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6db2200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dac0b13c0 0x7f6dac165260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6db2200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6dac15f400 0x7f6dac0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6db2200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac168fb0 con 0x7f6dac0b13c0 2024-09-10T21:13:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6db2200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac168e30 con 0x7f6dac0716f0 2024-09-10T21:13:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6db2200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6dac169130 con 0x7f6dac15f400 2024-09-10T21:13:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6da3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dac0b13c0 0x7f6dac165260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6da3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dac0b13c0 0x7f6dac165260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40002/0 (socket says 172.21.15.16:40002) 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6da3e00640 1 -- 172.21.15.16:0/2851931341 learned_addr learned my addr 172.21.15.16:0/2851931341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6dab400640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dac0716f0 0x7f6dac0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6dabe00640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6dac15f400 0x7f6dac0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6da3e00640 1 -- 172.21.15.16:0/2851931341 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6dac15f400 msgr2=0x7f6dac0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6da3e00640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6dac15f400 0x7f6dac0b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6da3e00640 1 -- 172.21.15.16:0/2851931341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dac0716f0 msgr2=0x7f6dac0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6da3e00640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dac0716f0 0x7f6dac0cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6da3e00640 1 -- 172.21.15.16:0/2851931341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dac159e80 con 0x7f6dac0b13c0 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6dab400640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dac0716f0 0x7f6dac0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6da3e00640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dac0b13c0 0x7f6dac165260 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009790 tx=0x7f6d940a65f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6da3400640 1 -- 172.21.15.16:0/2851931341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d940bc030 con 0x7f6dac0b13c0 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.131+0000 7f6da3400640 1 -- 172.21.15.16:0/2851931341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d94004120 con 0x7f6dac0b13c0 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.135+0000 7f6da3400640 1 -- 172.21.15.16:0/2851931341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94005020 con 0x7f6dac0b13c0 2024-09-10T21:13:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.135+0000 7f6db2200640 1 -- 172.21.15.16:0/2851931341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dac159900 con 0x7f6dac0b13c0 2024-09-10T21:13:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.135+0000 7f6db2200640 1 -- 172.21.15.16:0/2851931341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dac164a60 con 0x7f6dac0b13c0 2024-09-10T21:13:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.135+0000 7f6db2200640 1 -- 172.21.15.16:0/2851931341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dac070fe0 con 0x7f6dac0b13c0 2024-09-10T21:13:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.135+0000 7f6da3400640 1 -- 172.21.15.16:0/2851931341 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6d940c9060 con 0x7f6dac0b13c0 2024-09-10T21:13:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.135+0000 7f6da3400640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d880725e0 0x7f6d88074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.135+0000 7f6da3400640 1 -- 172.21.15.16:0/2851931341 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7f6d9412dab0 con 0x7f6dac0b13c0 2024-09-10T21:13:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.135+0000 7f6dab400640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d880725e0 0x7f6d88074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.135+0000 7f6dab400640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d880725e0 0x7f6d88074aa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0016d0 tx=0x7f6d9c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.139+0000 7f6da3400640 1 -- 172.21.15.16:0/2851931341 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6d94016410 con 0x7f6dac0b13c0 2024-09-10T21:13:49.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.379+0000 7f6db2200640 1 -- 172.21.15.16:0/2851931341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) -- 0x7f6dac0cd7b0 con 0x7f6dac0b13c0 2024-09-10T21:13:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.591+0000 7f6da3400640 1 -- 172.21.15.16:0/2851931341 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v189) ==== 148+0+0 (secure 0 0 0) 0x7f6d940f95e0 con 0x7f6dac0b13c0 2024-09-10T21:13:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.595+0000 7f6db2200640 1 -- 172.21.15.16:0/2851931341 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d880725e0 msgr2=0x7f6d88074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.595+0000 7f6db2200640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d880725e0 0x7f6d88074aa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0016d0 tx=0x7f6d9c009040 comp rx=0 tx=0).stop 2024-09-10T21:13:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.595+0000 7f6db2200640 1 -- 172.21.15.16:0/2851931341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dac0b13c0 msgr2=0x7f6dac165260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.595+0000 7f6db2200640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dac0b13c0 0x7f6dac165260 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009790 tx=0x7f6d940a65f0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.595+0000 7f6db2200640 1 -- 172.21.15.16:0/2851931341 shutdown_connections 2024-09-10T21:13:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.595+0000 7f6db2200640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d880725e0 0x7f6d88074aa0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.595+0000 7f6db2200640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6dac15f400 0x7f6dac0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.595+0000 7f6db2200640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dac0b13c0 0x7f6dac165260 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.595+0000 7f6db2200640 1 --2- 172.21.15.16:0/2851931341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dac0716f0 0x7f6dac0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.595+0000 7f6db2200640 1 -- 172.21.15.16:0/2851931341 >> 172.21.15.16:0/2851931341 conn(0x7f6dac09b130 msgr2=0x7f6dac15a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:49.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.595+0000 7f6db2200640 1 -- 172.21.15.16:0/2851931341 shutdown_connections 2024-09-10T21:13:49.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.595+0000 7f6db2200640 1 -- 172.21.15.16:0/2851931341 wait complete. 2024-09-10T21:13:49.624 INFO:tasks.cephfs_test_runner:That the MDS will not let a client sit above mds_max_caps_per_client caps. ... ok 2024-09-10T21:13:49.624 INFO:tasks.cephfs_test_runner:Starting test: test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) When a client has inodes pinned in its cache (open files), that the MDS 2024-09-10T21:13:49.624 INFO:tasks.cephfs_test_runner:test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-10T21:13:49.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:13:49.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.763+0000 7fc6d0000640 1 -- 172.21.15.16:0/2795965533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c80b13c0 msgr2=0x7fc6c8071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.763+0000 7fc6d0000640 1 --2- 172.21.15.16:0/2795965533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c80b13c0 0x7fc6c8071520 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009900 tx=0x7fc6b401c7c0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.763+0000 7fc6d0000640 1 -- 172.21.15.16:0/2795965533 shutdown_connections 2024-09-10T21:13:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.763+0000 7fc6d0000640 1 --2- 172.21.15.16:0/2795965533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c80b13c0 0x7fc6c8071520 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.763+0000 7fc6d0000640 1 --2- 172.21.15.16:0/2795965533 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc6c815f400 0x7fc6c8070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.763+0000 7fc6d0000640 1 -- 172.21.15.16:0/2795965533 >> 172.21.15.16:0/2795965533 conn(0x7fc6c809b130 msgr2=0x7fc6c80d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.763+0000 7fc6d0000640 1 -- 172.21.15.16:0/2795965533 shutdown_connections 2024-09-10T21:13:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.763+0000 7fc6d0000640 1 -- 172.21.15.16:0/2795965533 wait complete. 2024-09-10T21:13:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6d0000640 1 Processor -- start 2024-09-10T21:13:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6d0000640 1 -- start start 2024-09-10T21:13:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6d0000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc6c80b13c0 0x7fc6c8163f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6d0000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6c80cfc80 0x7fc6c80cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6d0000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c815f400 0x7fc6c8165e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6d0000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c8169770 con 0x7fc6c815f400 2024-09-10T21:13:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6d0000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c81695f0 con 0x7fc6c80cfc80 2024-09-10T21:13:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6d0000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c81698f0 con 0x7fc6c80b13c0 2024-09-10T21:13:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6ce000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c815f400 0x7fc6c8165e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6cd600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc6c80b13c0 0x7fc6c8163f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6ce000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c815f400 0x7fc6c8165e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40030/0 (socket says 172.21.15.16:40030) 2024-09-10T21:13:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6ce000640 1 -- 172.21.15.16:0/119429405 learned_addr learned my addr 172.21.15.16:0/119429405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6ccc00640 1 --2- 172.21.15.16:0/119429405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6c80cfc80 0x7fc6c80cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6ce000640 1 -- 172.21.15.16:0/119429405 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc6c80b13c0 msgr2=0x7fc6c8163f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6ce000640 1 --2- 172.21.15.16:0/119429405 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc6c80b13c0 0x7fc6c8163f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6ce000640 1 -- 172.21.15.16:0/119429405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6c80cfc80 msgr2=0x7fc6c80cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6ce000640 1 --2- 172.21.15.16:0/119429405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6c80cfc80 0x7fc6c80cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6ce000640 1 -- 172.21.15.16:0/119429405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c8164460 con 0x7fc6c815f400 2024-09-10T21:13:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6ce000640 1 --2- 172.21.15.16:0/119429405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c815f400 0x7fc6c8165e00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000c7b0 tx=0x7fc6c000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6c5600640 1 -- 172.21.15.16:0/119429405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0004b70 con 0x7fc6c815f400 2024-09-10T21:13:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6c5600640 1 -- 172.21.15.16:0/119429405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6c0004e00 con 0x7fc6c815f400 2024-09-10T21:13:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6c5600640 1 -- 172.21.15.16:0/119429405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c000aab0 con 0x7fc6c815f400 2024-09-10T21:13:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6d0000640 1 -- 172.21.15.16:0/119429405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c80b59a0 con 0x7fc6c815f400 2024-09-10T21:13:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6d0000640 1 -- 172.21.15.16:0/119429405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c80ce0b0 con 0x7fc6c815f400 2024-09-10T21:13:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.767+0000 7fc6d0000640 1 -- 172.21.15.16:0/119429405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c80d00d0 con 0x7fc6c815f400 2024-09-10T21:13:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.771+0000 7fc6c5600640 1 -- 172.21.15.16:0/119429405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc6c000ac50 con 0x7fc6c815f400 2024-09-10T21:13:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.771+0000 7fc6c5600640 1 --2- 172.21.15.16:0/119429405 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc6a80725e0 0x7fc6a8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.771+0000 7fc6c5600640 1 -- 172.21.15.16:0/119429405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7fc6c0093200 con 0x7fc6c815f400 2024-09-10T21:13:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.771+0000 7fc6cd600640 1 --2- 172.21.15.16:0/119429405 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc6a80725e0 0x7fc6a8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.771+0000 7fc6cd600640 1 --2- 172.21.15.16:0/119429405 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc6a80725e0 0x7fc6a8074aa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc005fd0 tx=0x7fc6bc00b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:49.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.775+0000 7fc6c5600640 1 -- 172.21.15.16:0/119429405 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc6c0020070 con 0x7fc6c815f400 2024-09-10T21:13:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.951+0000 7fc6d0000640 1 -- 172.21.15.16:0/119429405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6c80ce390 con 0x7fc6c815f400 2024-09-10T21:13:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.951+0000 7fc6c5600640 1 -- 172.21.15.16:0/119429405 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+12985 (secure 0 0 0) 0x7fc6c005e4f0 con 0x7fc6c815f400 2024-09-10T21:13:49.959 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:49.959 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:13:48.586663+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6835/1574608409":"2024-09-11T21:13:45.366425+0000","172.21.15.52:6833/1574608409":"2024-09-11T21:13:45.366425+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:13:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.955+0000 7fc6d0000640 1 -- 172.21.15.16:0/119429405 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc6a80725e0 msgr2=0x7fc6a8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.955+0000 7fc6d0000640 1 --2- 172.21.15.16:0/119429405 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc6a80725e0 0x7fc6a8074aa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc005fd0 tx=0x7fc6bc00b2e0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.955+0000 7fc6d0000640 1 -- 172.21.15.16:0/119429405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c815f400 msgr2=0x7fc6c8165e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.955+0000 7fc6d0000640 1 --2- 172.21.15.16:0/119429405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c815f400 0x7fc6c8165e00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000c7b0 tx=0x7fc6c000cc70 comp rx=0 tx=0).stop 2024-09-10T21:13:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.955+0000 7fc6d0000640 1 -- 172.21.15.16:0/119429405 shutdown_connections 2024-09-10T21:13:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.955+0000 7fc6d0000640 1 --2- 172.21.15.16:0/119429405 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc6a80725e0 0x7fc6a8074aa0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.955+0000 7fc6d0000640 1 --2- 172.21.15.16:0/119429405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c815f400 0x7fc6c8165e00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.955+0000 7fc6d0000640 1 --2- 172.21.15.16:0/119429405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6c80cfc80 0x7fc6c80cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.955+0000 7fc6d0000640 1 --2- 172.21.15.16:0/119429405 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc6c80b13c0 0x7fc6c8163f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.955+0000 7fc6d0000640 1 -- 172.21.15.16:0/119429405 >> 172.21.15.16:0/119429405 conn(0x7fc6c809b130 msgr2=0x7fc6c815e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.955+0000 7fc6d0000640 1 -- 172.21.15.16:0/119429405 shutdown_connections 2024-09-10T21:13:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:49.955+0000 7fc6d0000640 1 -- 172.21.15.16:0/119429405 wait complete. 2024-09-10T21:13:49.981 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:10:17.493770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:13:49.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:13:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.127+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2775292816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe290003a60 msgr2=0x7fe290003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.127+0000 7fe2b1c00640 1 --2- 172.21.15.16:0/2775292816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe290003a60 0x7fe290003e80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00c310 tx=0x7fe29c01cf90 comp rx=0 tx=0).stop 2024-09-10T21:13:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.127+0000 7fe2ab400640 1 -- 172.21.15.16:0/2775292816 reap_dead start 2024-09-10T21:13:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.127+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2775292816 shutdown_connections 2024-09-10T21:13:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.127+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2775292816 >> 172.21.15.16:0/2775292816 conn(0x7fe2ac0d1ce0 msgr2=0x7fe2ac0d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.127+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2775292816 shutdown_connections 2024-09-10T21:13:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.127+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2775292816 wait complete. 2024-09-10T21:13:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.127+0000 7fe2b1c00640 1 Processor -- start 2024-09-10T21:13:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.127+0000 7fe2b1c00640 1 -- start start 2024-09-10T21:13:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2b1c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe290003a60 0x7fe2ac1500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2b1c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2ac0b05b0 0x7fe2ac157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2b1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2ac1563c0 0x7fe2ac156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2b1c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0b3600 con 0x7fe2ac1563c0 2024-09-10T21:13:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2b1c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0b3480 con 0x7fe2ac0b05b0 2024-09-10T21:13:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2b1c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac168ea0 con 0x7fe290003a60 2024-09-10T21:13:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2ab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2ac1563c0 0x7fe2ac156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2a3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2ac0b05b0 0x7fe2ac157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2ab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2ac1563c0 0x7fe2ac156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40048/0 (socket says 172.21.15.16:40048) 2024-09-10T21:13:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2ab400640 1 -- 172.21.15.16:0/2288968192 learned_addr learned my addr 172.21.15.16:0/2288968192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2aaa00640 1 --2- 172.21.15.16:0/2288968192 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe290003a60 0x7fe2ac1500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2ab400640 1 -- 172.21.15.16:0/2288968192 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe290003a60 msgr2=0x7fe2ac1500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2ab400640 1 --2- 172.21.15.16:0/2288968192 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe290003a60 0x7fe2ac1500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2ab400640 1 -- 172.21.15.16:0/2288968192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2ac0b05b0 msgr2=0x7fe2ac157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2ab400640 1 --2- 172.21.15.16:0/2288968192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2ac0b05b0 0x7fe2ac157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:50.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2ab400640 1 -- 172.21.15.16:0/2288968192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac1fb100 con 0x7fe2ac1563c0 2024-09-10T21:13:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2ab400640 1 --2- 172.21.15.16:0/2288968192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2ac1563c0 0x7fe2ac156850 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac06fa40 tx=0x7fe2a4010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2a2a00640 1 -- 172.21.15.16:0/2288968192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4003980 con 0x7fe2ac1563c0 2024-09-10T21:13:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2a2a00640 1 -- 172.21.15.16:0/2288968192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2a4015070 con 0x7fe2ac1563c0 2024-09-10T21:13:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2288968192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ac1fb370 con 0x7fe2ac1563c0 2024-09-10T21:13:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2a2a00640 1 -- 172.21.15.16:0/2288968192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4004030 con 0x7fe2ac1563c0 2024-09-10T21:13:50.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2288968192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ac1586f0 con 0x7fe2ac1563c0 2024-09-10T21:13:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.131+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2288968192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2ac0b24a0 con 0x7fe2ac1563c0 2024-09-10T21:13:50.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.135+0000 7fe2a2a00640 1 -- 172.21.15.16:0/2288968192 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe2a40041d0 con 0x7fe2ac1563c0 2024-09-10T21:13:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.135+0000 7fe2a2a00640 1 --2- 172.21.15.16:0/2288968192 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe288072610 0x7fe288074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.135+0000 7fe2a2a00640 1 -- 172.21.15.16:0/2288968192 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7fe2a4093880 con 0x7fe2ac1563c0 2024-09-10T21:13:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.139+0000 7fe2aaa00640 1 --2- 172.21.15.16:0/2288968192 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe288072610 0x7fe288074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.139+0000 7fe2aaa00640 1 --2- 172.21.15.16:0/2288968192 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe288072610 0x7fe288074ad0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0ac000 tx=0x7fe29c019dd0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.139+0000 7fe2a2a00640 1 -- 172.21.15.16:0/2288968192 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe2a405ec10 con 0x7fe2ac1563c0 2024-09-10T21:13:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.311+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2288968192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe2ac0704a0 con 0x7fe2ac1563c0 2024-09-10T21:13:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.315+0000 7fe2a2a00640 1 -- 172.21.15.16:0/2288968192 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7fe2a4063640 con 0x7fe2ac1563c0 2024-09-10T21:13:50.321 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-10T21:13:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.315+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2288968192 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe288072610 msgr2=0x7fe288074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.315+0000 7fe2b1c00640 1 --2- 172.21.15.16:0/2288968192 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe288072610 0x7fe288074ad0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0ac000 tx=0x7fe29c019dd0 comp rx=0 tx=0).stop 2024-09-10T21:13:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.315+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2288968192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2ac1563c0 msgr2=0x7fe2ac156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.315+0000 7fe2b1c00640 1 --2- 172.21.15.16:0/2288968192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2ac1563c0 0x7fe2ac156850 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac06fa40 tx=0x7fe2a4010f40 comp rx=0 tx=0).stop 2024-09-10T21:13:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.315+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2288968192 shutdown_connections 2024-09-10T21:13:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.315+0000 7fe2b1c00640 1 --2- 172.21.15.16:0/2288968192 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe288072610 0x7fe288074ad0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.315+0000 7fe2b1c00640 1 --2- 172.21.15.16:0/2288968192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2ac1563c0 0x7fe2ac156850 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.315+0000 7fe2b1c00640 1 --2- 172.21.15.16:0/2288968192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2ac0b05b0 0x7fe2ac157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.315+0000 7fe2b1c00640 1 --2- 172.21.15.16:0/2288968192 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe290003a60 0x7fe2ac1500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.315+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2288968192 >> 172.21.15.16:0/2288968192 conn(0x7fe2ac0d1ce0 msgr2=0x7fe2ac0b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.315+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2288968192 shutdown_connections 2024-09-10T21:13:50.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.315+0000 7fe2b1c00640 1 -- 172.21.15.16:0/2288968192 wait complete. 2024-09-10T21:13:50.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2024-09-10T21:13:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 -- 172.21.15.16:0/3582555433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8700b0590 msgr2=0x7fa870071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 --2- 172.21.15.16:0/3582555433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8700b0590 0x7fa870071520 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa868009c80 tx=0x7fa86801cca0 comp rx=0 tx=0).stop 2024-09-10T21:13:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 -- 172.21.15.16:0/3582555433 shutdown_connections 2024-09-10T21:13:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 --2- 172.21.15.16:0/3582555433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8700b0590 0x7fa870071520 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 --2- 172.21.15.16:0/3582555433 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa87015f400 0x7fa870070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 -- 172.21.15.16:0/3582555433 >> 172.21.15.16:0/3582555433 conn(0x7fa87009b130 msgr2=0x7fa8700d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 -- 172.21.15.16:0/3582555433 shutdown_connections 2024-09-10T21:13:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 -- 172.21.15.16:0/3582555433 wait complete. 2024-09-10T21:13:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 Processor -- start 2024-09-10T21:13:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 -- start start 2024-09-10T21:13:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8700b0590 0x7fa87015b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8700b13c0 0x7fa870156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa87015f400 0x7fa870158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa870168f60 con 0x7fa8700b13c0 2024-09-10T21:13:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa870168de0 con 0x7fa8700b0590 2024-09-10T21:13:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa875400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa8701690e0 con 0x7fa87015f400 2024-09-10T21:13:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa865e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8700b13c0 0x7fa870156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa865e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8700b13c0 0x7fa870156980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40058/0 (socket says 172.21.15.16:40058) 2024-09-10T21:13:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa865e00640 1 -- 172.21.15.16:0/4127191347 learned_addr learned my addr 172.21.15.16:0/4127191347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa86e800640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8700b0590 0x7fa87015b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.483+0000 7fa86f200640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa87015f400 0x7fa870158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.487+0000 7fa865e00640 1 -- 172.21.15.16:0/4127191347 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa87015f400 msgr2=0x7fa870158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.487+0000 7fa865e00640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa87015f400 0x7fa870158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.487+0000 7fa865e00640 1 -- 172.21.15.16:0/4127191347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8700b0590 msgr2=0x7fa87015b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.487+0000 7fa865e00640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8700b0590 0x7fa87015b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.487+0000 7fa865e00640 1 -- 172.21.15.16:0/4127191347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8701f8670 con 0x7fa8700b13c0 2024-09-10T21:13:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.487+0000 7fa865e00640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8700b13c0 0x7fa870156980 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa85800c3a0 tx=0x7fa85800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.487+0000 7fa867400640 1 -- 172.21.15.16:0/4127191347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858017070 con 0x7fa8700b13c0 2024-09-10T21:13:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.487+0000 7fa867400640 1 -- 172.21.15.16:0/4127191347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa858003600 con 0x7fa8700b13c0 2024-09-10T21:13:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.487+0000 7fa875400640 1 -- 172.21.15.16:0/4127191347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87015c020 con 0x7fa8700b13c0 2024-09-10T21:13:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.487+0000 7fa867400640 1 -- 172.21.15.16:0/4127191347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858003a50 con 0x7fa8700b13c0 2024-09-10T21:13:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.487+0000 7fa875400640 1 -- 172.21.15.16:0/4127191347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa870156ec0 con 0x7fa8700b13c0 2024-09-10T21:13:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.487+0000 7fa875400640 1 -- 172.21.15.16:0/4127191347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8700cfc80 con 0x7fa8700b13c0 2024-09-10T21:13:50.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.487+0000 7fa867400640 1 -- 172.21.15.16:0/4127191347 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa858003bf0 con 0x7fa8700b13c0 2024-09-10T21:13:50.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.491+0000 7fa867400640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa84c0725e0 0x7fa84c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:50.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.491+0000 7fa867400640 1 -- 172.21.15.16:0/4127191347 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7fa858092510 con 0x7fa8700b13c0 2024-09-10T21:13:50.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.491+0000 7fa86e800640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa84c0725e0 0x7fa84c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:50.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.491+0000 7fa86e800640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa84c0725e0 0x7fa84c074aa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa860005fd0 tx=0x7fa860005f40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.495+0000 7fa867400640 1 -- 172.21.15.16:0/4127191347 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa85805d800 con 0x7fa8700b13c0 2024-09-10T21:13:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:50.731+0000 7fa875400640 1 -- 172.21.15.16:0/4127191347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) -- 0x7fa8700b3060 con 0x7fa8700b13c0 2024-09-10T21:13:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.603+0000 7fa867400640 1 -- 172.21.15.16:0/4127191347 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v191) ==== 163+0+0 (secure 0 0 0) 0x7fa858062230 con 0x7fa8700b13c0 2024-09-10T21:13:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.607+0000 7fa875400640 1 -- 172.21.15.16:0/4127191347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa84c0725e0 msgr2=0x7fa84c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.607+0000 7fa875400640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa84c0725e0 0x7fa84c074aa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa860005fd0 tx=0x7fa860005f40 comp rx=0 tx=0).stop 2024-09-10T21:13:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.607+0000 7fa875400640 1 -- 172.21.15.16:0/4127191347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8700b13c0 msgr2=0x7fa870156980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.607+0000 7fa875400640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8700b13c0 0x7fa870156980 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa85800c3a0 tx=0x7fa85800c860 comp rx=0 tx=0).stop 2024-09-10T21:13:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.607+0000 7fa875400640 1 -- 172.21.15.16:0/4127191347 shutdown_connections 2024-09-10T21:13:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.607+0000 7fa875400640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa84c0725e0 0x7fa84c074aa0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.607+0000 7fa875400640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa87015f400 0x7fa870158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.607+0000 7fa875400640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8700b13c0 0x7fa870156980 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.607+0000 7fa875400640 1 --2- 172.21.15.16:0/4127191347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8700b0590 0x7fa87015b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.607+0000 7fa875400640 1 -- 172.21.15.16:0/4127191347 >> 172.21.15.16:0/4127191347 conn(0x7fa87009b130 msgr2=0x7fa87006e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.607+0000 7fa875400640 1 -- 172.21.15.16:0/4127191347 shutdown_connections 2024-09-10T21:13:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.607+0000 7fa875400640 1 -- 172.21.15.16:0/4127191347 wait complete. 2024-09-10T21:13:51.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:13:51.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.787+0000 7f815bc00640 1 -- 172.21.15.16:0/772140189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c004d50 msgr2=0x7f813c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:51.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.787+0000 7f815bc00640 1 --2- 172.21.15.16:0/772140189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c004d50 0x7f813c0051d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f814c009ee0 tx=0x7f814c01d5f0 comp rx=0 tx=0).stop 2024-09-10T21:13:51.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.787+0000 7f8159c00640 1 -- 172.21.15.16:0/772140189 reap_dead start 2024-09-10T21:13:51.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.787+0000 7f815bc00640 1 -- 172.21.15.16:0/772140189 shutdown_connections 2024-09-10T21:13:51.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.787+0000 7f815bc00640 1 -- 172.21.15.16:0/772140189 >> 172.21.15.16:0/772140189 conn(0x7f81540cfb20 msgr2=0x7f81540cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:51.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.787+0000 7f815bc00640 1 -- 172.21.15.16:0/772140189 shutdown_connections 2024-09-10T21:13:51.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.787+0000 7f815bc00640 1 -- 172.21.15.16:0/772140189 wait complete. 2024-09-10T21:13:51.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.787+0000 7f815bc00640 1 Processor -- start 2024-09-10T21:13:51.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.787+0000 7f815bc00640 1 -- start start 2024-09-10T21:13:51.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f815bc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f813c004d50 0x7f8154158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:51.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f815bc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8154158d70 0x7f81540c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:51.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f815bc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81540c8920 0x7f81540c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:51.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f815bc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81540ca730 con 0x7f8154158d70 2024-09-10T21:13:51.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f815bc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f81540ca5b0 con 0x7f81540c8920 2024-09-10T21:13:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f815bc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f81540ca8b0 con 0x7f813c004d50 2024-09-10T21:13:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8153e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8154158d70 0x7f81540c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8153e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8154158d70 0x7f81540c6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40082/0 (socket says 172.21.15.16:40082) 2024-09-10T21:13:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8153e00640 1 -- 172.21.15.16:0/1622309913 learned_addr learned my addr 172.21.15.16:0/1622309913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8159200640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f813c004d50 0x7f8154158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8159c00640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81540c8920 0x7f81540c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8153e00640 1 -- 172.21.15.16:0/1622309913 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f813c004d50 msgr2=0x7f8154158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8153e00640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f813c004d50 0x7f8154158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8153e00640 1 -- 172.21.15.16:0/1622309913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81540c8920 msgr2=0x7f81540c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8153e00640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81540c8920 0x7f81540c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8153e00640 1 -- 172.21.15.16:0/1622309913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81540c74f0 con 0x7f8154158d70 2024-09-10T21:13:51.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8153e00640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8154158d70 0x7f81540c6bf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f814000e440 tx=0x7f8140010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8150c00640 1 -- 172.21.15.16:0/1622309913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81400129d0 con 0x7f8154158d70 2024-09-10T21:13:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8150c00640 1 -- 172.21.15.16:0/1622309913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8140012c60 con 0x7f8154158d70 2024-09-10T21:13:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8150c00640 1 -- 172.21.15.16:0/1622309913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814001a660 con 0x7f8154158d70 2024-09-10T21:13:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f815bc00640 1 -- 172.21.15.16:0/1622309913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81540c77d0 con 0x7f8154158d70 2024-09-10T21:13:51.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f815bc00640 1 -- 172.21.15.16:0/1622309913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8154158a50 con 0x7f8154158d70 2024-09-10T21:13:51.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f815bc00640 1 -- 172.21.15.16:0/1622309913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f815406ef60 con 0x7f8154158d70 2024-09-10T21:13:51.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.791+0000 7f8150c00640 1 -- 172.21.15.16:0/1622309913 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f81400113d0 con 0x7f8154158d70 2024-09-10T21:13:51.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.795+0000 7f8150c00640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f81340725e0 0x7f8134074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:51.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.795+0000 7f8150c00640 1 -- 172.21.15.16:0/1622309913 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7f81400964d0 con 0x7f8154158d70 2024-09-10T21:13:51.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.795+0000 7f8159200640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f81340725e0 0x7f8134074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:51.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.795+0000 7f8159200640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f81340725e0 0x7f8134074aa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f814800ac50 tx=0x7f814800a2f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:51.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.799+0000 7f8150c00640 1 -- 172.21.15.16:0/1622309913 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8140016030 con 0x7f8154158d70 2024-09-10T21:13:51.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.971+0000 7f815bc00640 1 -- 172.21.15.16:0/1622309913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f81540b5aa0 con 0x7f8154158d70 2024-09-10T21:13:51.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.971+0000 7f8150c00640 1 -- 172.21.15.16:0/1622309913 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v38) ==== 59+0+1998 (secure 0 0 0) 0x7f8140061850 con 0x7f8154158d70 2024-09-10T21:13:51.980 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:13:51.980 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:13:51.980 INFO:teuthology.orchestra.run.smithi016.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:13:51.980 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:13:51.980 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4226.427518 2024-09-10T21:13:51.980 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5426.620519 2024-09-10T21:13:51.980 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4255.210203 2024-09-10T21:13:51.980 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4255.734871 2024-09-10T21:13:51.980 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4192.833008 2024-09-10T21:13:51.980 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4136.940759 2024-09-10T21:13:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4217.384701 2024-09-10T21:13:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5220.337972 2024-09-10T21:13:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-10T21:13:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:13:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-10T21:13:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-10T21:13:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:13:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:13:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-10T21:13:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:13:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:13:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:13:51.981 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-10T21:13:51.982 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-10T21:13:51.982 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-10T21:13:51.982 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:13:51.982 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:13:51.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.975+0000 7f815bc00640 1 -- 172.21.15.16:0/1622309913 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f81340725e0 msgr2=0x7f8134074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:51.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.975+0000 7f815bc00640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f81340725e0 0x7f8134074aa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f814800ac50 tx=0x7f814800a2f0 comp rx=0 tx=0).stop 2024-09-10T21:13:51.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.975+0000 7f815bc00640 1 -- 172.21.15.16:0/1622309913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8154158d70 msgr2=0x7f81540c6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:51.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.975+0000 7f815bc00640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8154158d70 0x7f81540c6bf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f814000e440 tx=0x7f8140010f90 comp rx=0 tx=0).stop 2024-09-10T21:13:51.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.975+0000 7f815bc00640 1 -- 172.21.15.16:0/1622309913 shutdown_connections 2024-09-10T21:13:51.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.975+0000 7f815bc00640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f81340725e0 0x7f8134074aa0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:51.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.975+0000 7f815bc00640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81540c8920 0x7f81540c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:51.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.975+0000 7f815bc00640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8154158d70 0x7f81540c6bf0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:51.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.975+0000 7f815bc00640 1 --2- 172.21.15.16:0/1622309913 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f813c004d50 0x7f8154158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:51.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.975+0000 7f815bc00640 1 -- 172.21.15.16:0/1622309913 >> 172.21.15.16:0/1622309913 conn(0x7f81540cfb20 msgr2=0x7f815415e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:51.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.975+0000 7f815bc00640 1 -- 172.21.15.16:0/1622309913 shutdown_connections 2024-09-10T21:13:51.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:51.975+0000 7f815bc00640 1 -- 172.21.15.16:0/1622309913 wait complete. 2024-09-10T21:13:52.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:13:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.143+0000 7fb2d4400640 1 -- 172.21.15.16:0/2561487712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc0cfe30 msgr2=0x7fb2cc0ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.143+0000 7fb2d4400640 1 --2- 172.21.15.16:0/2561487712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc0cfe30 0x7fb2cc0ce590 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009870 tx=0x7fb2c001f010 comp rx=0 tx=0).stop 2024-09-10T21:13:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.143+0000 7fb2d4400640 1 -- 172.21.15.16:0/2561487712 shutdown_connections 2024-09-10T21:13:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.143+0000 7fb2d4400640 1 --2- 172.21.15.16:0/2561487712 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb2cc0ce040 0x7fb2cc0b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.143+0000 7fb2d4400640 1 --2- 172.21.15.16:0/2561487712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc0cfe30 0x7fb2cc0ce590 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.143+0000 7fb2d4400640 1 -- 172.21.15.16:0/2561487712 >> 172.21.15.16:0/2561487712 conn(0x7fb2cc0d1df0 msgr2=0x7fb2cc0d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:52.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.143+0000 7fb2d4400640 1 -- 172.21.15.16:0/2561487712 shutdown_connections 2024-09-10T21:13:52.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.143+0000 7fb2d4400640 1 -- 172.21.15.16:0/2561487712 wait complete. 2024-09-10T21:13:52.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.143+0000 7fb2d4400640 1 Processor -- start 2024-09-10T21:13:52.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.143+0000 7fb2d4400640 1 -- start start 2024-09-10T21:13:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d4400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb2cc0ce040 0x7fb2cc1e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d4400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc0cfe30 0x7fb2cc156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d4400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2cc0d0210 0x7fb2cc0c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d4400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc0d2b00 con 0x7fb2cc0cfe30 2024-09-10T21:13:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d4400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc0b14f0 con 0x7fb2cc0d0210 2024-09-10T21:13:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d4400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb2cc0cf1e0 con 0x7fb2cc0ce040 2024-09-10T21:13:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc0cfe30 0x7fb2cc156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc0cfe30 0x7fb2cc156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56812/0 (socket says 172.21.15.16:56812) 2024-09-10T21:13:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d1000640 1 -- 172.21.15.16:0/2938456120 learned_addr learned my addr 172.21.15.16:0/2938456120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d2400640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2cc0d0210 0x7fb2cc0c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d1a00640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb2cc0ce040 0x7fb2cc1e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d1000640 1 -- 172.21.15.16:0/2938456120 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb2cc0ce040 msgr2=0x7fb2cc1e4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d1000640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb2cc0ce040 0x7fb2cc1e4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d1000640 1 -- 172.21.15.16:0/2938456120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2cc0d0210 msgr2=0x7fb2cc0c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d1000640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2cc0d0210 0x7fb2cc0c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d1000640 1 -- 172.21.15.16:0/2938456120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc156990 con 0x7fb2cc0cfe30 2024-09-10T21:13:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d1000640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc0cfe30 0x7fb2cc156450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800c400 tx=0x7fb2b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2c9600640 1 -- 172.21.15.16:0/2938456120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8011790 con 0x7fb2cc0cfe30 2024-09-10T21:13:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2c9600640 1 -- 172.21.15.16:0/2938456120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2b8017070 con 0x7fb2cc0cfe30 2024-09-10T21:13:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2c9600640 1 -- 172.21.15.16:0/2938456120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8011cc0 con 0x7fb2cc0cfe30 2024-09-10T21:13:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d4400640 1 -- 172.21.15.16:0/2938456120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2cc15b2b0 con 0x7fb2cc0cfe30 2024-09-10T21:13:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d4400640 1 -- 172.21.15.16:0/2938456120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2cc0c6a60 con 0x7fb2cc0cfe30 2024-09-10T21:13:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2d4400640 1 -- 172.21.15.16:0/2938456120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2cc0b1710 con 0x7fb2cc0cfe30 2024-09-10T21:13:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.147+0000 7fb2c9600640 1 -- 172.21.15.16:0/2938456120 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb2b801c460 con 0x7fb2cc0cfe30 2024-09-10T21:13:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.151+0000 7fb2c9600640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb2ac0725e0 0x7fb2ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.151+0000 7fb2c9600640 1 -- 172.21.15.16:0/2938456120 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7fb2b8092db0 con 0x7fb2cc0cfe30 2024-09-10T21:13:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.151+0000 7fb2d1a00640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb2ac0725e0 0x7fb2ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.151+0000 7fb2d1a00640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb2ac0725e0 0x7fb2ac074aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00b2000 tx=0x7fb2c001d850 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.155+0000 7fb2c9600640 1 -- 172.21.15.16:0/2938456120 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb2b8097070 con 0x7fb2cc0cfe30 2024-09-10T21:13:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.327+0000 7fb2d4400640 1 -- 172.21.15.16:0/2938456120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb2cc0b2cd0 con 0x7fb2cc0cfe30 2024-09-10T21:13:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.331+0000 7fb2c9600640 1 -- 172.21.15.16:0/2938456120 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v39)=0 v39) ==== 122+0+0 (secure 0 0 0) 0x7fb2b805e6f0 con 0x7fb2cc0cfe30 2024-09-10T21:13:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.335+0000 7fb2d4400640 1 -- 172.21.15.16:0/2938456120 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb2ac0725e0 msgr2=0x7fb2ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.335+0000 7fb2d4400640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb2ac0725e0 0x7fb2ac074aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00b2000 tx=0x7fb2c001d850 comp rx=0 tx=0).stop 2024-09-10T21:13:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.335+0000 7fb2d4400640 1 -- 172.21.15.16:0/2938456120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc0cfe30 msgr2=0x7fb2cc156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.335+0000 7fb2d4400640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc0cfe30 0x7fb2cc156450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800c400 tx=0x7fb2b800c8c0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.335+0000 7fb2d4400640 1 -- 172.21.15.16:0/2938456120 shutdown_connections 2024-09-10T21:13:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.335+0000 7fb2d4400640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb2ac0725e0 0x7fb2ac074aa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.335+0000 7fb2d4400640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2cc0d0210 0x7fb2cc0c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.335+0000 7fb2d4400640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc0cfe30 0x7fb2cc156450 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.335+0000 7fb2d4400640 1 --2- 172.21.15.16:0/2938456120 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb2cc0ce040 0x7fb2cc1e4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.335+0000 7fb2d4400640 1 -- 172.21.15.16:0/2938456120 >> 172.21.15.16:0/2938456120 conn(0x7fb2cc0d1df0 msgr2=0x7fb2cc0af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.335+0000 7fb2d4400640 1 -- 172.21.15.16:0/2938456120 shutdown_connections 2024-09-10T21:13:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.335+0000 7fb2d4400640 1 -- 172.21.15.16:0/2938456120 wait complete. 2024-09-10T21:13:52.360 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:13:52.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:13:52.435 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:13:52.515 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:13:52.516 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:52.516 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:52.525 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:13:52.528 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:13:52.528 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:52.538 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:13:52.538 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:13:52.538 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:52.576 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:13:52.576 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:13:52.577 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:13:52.589 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:13:52.589 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:13:52.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:13:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.759+0000 7ff66d200640 1 -- 172.21.15.16:0/569136602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815f450 msgr2=0x7ff6680cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.759+0000 7ff66d200640 1 --2- 172.21.15.16:0/569136602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815f450 0x7ff6680cfc80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff6680b3810 tx=0x7ff65801f070 comp rx=0 tx=0).stop 2024-09-10T21:13:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.759+0000 7ff66d200640 1 -- 172.21.15.16:0/569136602 shutdown_connections 2024-09-10T21:13:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.759+0000 7ff66d200640 1 --2- 172.21.15.16:0/569136602 >> v2:172.21.15.52:3300/0 conn(0x7ff6680b3200 0x7ff6680d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.759+0000 7ff66d200640 1 --2- 172.21.15.16:0/569136602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815f450 0x7ff6680cfc80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.759+0000 7ff66d200640 1 -- 172.21.15.16:0/569136602 >> 172.21.15.16:0/569136602 conn(0x7ff6680d1d80 msgr2=0x7ff6680d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.759+0000 7ff66d200640 1 -- 172.21.15.16:0/569136602 shutdown_connections 2024-09-10T21:13:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.759+0000 7ff66d200640 1 -- 172.21.15.16:0/569136602 wait complete. 2024-09-10T21:13:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.759+0000 7ff66d200640 1 Processor -- start 2024-09-10T21:13:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.759+0000 7ff66d200640 1 -- start start 2024-09-10T21:13:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.759+0000 7ff66d200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6680d0890 0x7ff6680b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff66d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815f450 0x7ff6680c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff66d200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6681f51d0 0x7ff6680ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff66d200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6680b2e30 con 0x7ff66815f450 2024-09-10T21:13:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff66d200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6680b2cb0 con 0x7ff6680d0890 2024-09-10T21:13:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff66d200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff6680b2fb0 con 0x7ff6681f51d0 2024-09-10T21:13:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815f450 0x7ff6680c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815f450 0x7ff6680c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56834/0 (socket says 172.21.15.16:56834) 2024-09-10T21:13:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff65fe00640 1 -- 172.21.15.16:0/2063916130 learned_addr learned my addr 172.21.15.16:0/2063916130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff666800640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6681f51d0 0x7ff6680ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff665e00640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6680d0890 0x7ff6680b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff65fe00640 1 -- 172.21.15.16:0/2063916130 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6681f51d0 msgr2=0x7ff6680ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff65fe00640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6681f51d0 0x7ff6680ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff65fe00640 1 -- 172.21.15.16:0/2063916130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6680d0890 msgr2=0x7ff6680b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff65fe00640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6680d0890 0x7ff6680b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff65fe00640 1 -- 172.21.15.16:0/2063916130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6681f59b0 con 0x7ff66815f450 2024-09-10T21:13:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff65fe00640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815f450 0x7ff6680c9c30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff65000d950 tx=0x7ff65000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff65e000640 1 -- 172.21.15.16:0/2063916130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650017070 con 0x7ff66815f450 2024-09-10T21:13:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff65e000640 1 -- 172.21.15.16:0/2063916130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff650009d20 con 0x7ff66815f450 2024-09-10T21:13:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff65e000640 1 -- 172.21.15.16:0/2063916130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6500106c0 con 0x7ff66815f450 2024-09-10T21:13:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff66d200640 1 -- 172.21.15.16:0/2063916130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6681f5c90 con 0x7ff66815f450 2024-09-10T21:13:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff66d200640 1 -- 172.21.15.16:0/2063916130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6681f6250 con 0x7ff66815f450 2024-09-10T21:13:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff66d200640 1 -- 172.21.15.16:0/2063916130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6680b1e30 con 0x7ff66815f450 2024-09-10T21:13:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.763+0000 7ff65e000640 1 -- 172.21.15.16:0/2063916130 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff650010860 con 0x7ff66815f450 2024-09-10T21:13:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.767+0000 7ff65e000640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6440725e0 0x7ff644074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.767+0000 7ff65e000640 1 -- 172.21.15.16:0/2063916130 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7ff650093290 con 0x7ff66815f450 2024-09-10T21:13:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.767+0000 7ff665e00640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6440725e0 0x7ff644074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.767+0000 7ff665e00640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6440725e0 0x7ff644074aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff6580b2000 tx=0x7ff65801d8b0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.771+0000 7ff65e000640 1 -- 172.21.15.16:0/2063916130 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff65005e610 con 0x7ff66815f450 2024-09-10T21:13:52.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.975+0000 7ff66d200640 1 -- 172.21.15.16:0/2063916130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff668074ba0 con 0x7ff66815f450 2024-09-10T21:13:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.979+0000 7ff65e000640 1 -- 172.21.15.16:0/2063916130 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+3912 (secure 0 0 0) 0x7ff650063040 con 0x7ff66815f450 2024-09-10T21:13:52.985 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:52.985 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":43,"btime":"2024-09-10T21:13:46:369177+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4029516332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4029516332},{"type":"v1","addr":"172.21.15.16:6836","nonce":4029516332}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/806881470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":806881470},{"type":"v1","addr":"172.21.15.16:6837","nonce":806881470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":135167,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2949376643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2949376643},{"type":"v1","addr":"172.21.15.52:6834","nonce":2949376643}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":135323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3549021127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3549021127},{"type":"v1","addr":"172.21.15.52:6835","nonce":3549021127}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[]} 2024-09-10T21:13:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 43 2024-09-10T21:13:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.979+0000 7ff66d200640 1 -- 172.21.15.16:0/2063916130 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6440725e0 msgr2=0x7ff644074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.979+0000 7ff66d200640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6440725e0 0x7ff644074aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff6580b2000 tx=0x7ff65801d8b0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.979+0000 7ff66d200640 1 -- 172.21.15.16:0/2063916130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815f450 msgr2=0x7ff6680c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.979+0000 7ff66d200640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815f450 0x7ff6680c9c30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff65000d950 tx=0x7ff65000de10 comp rx=0 tx=0).stop 2024-09-10T21:13:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.979+0000 7ff66d200640 1 -- 172.21.15.16:0/2063916130 shutdown_connections 2024-09-10T21:13:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.979+0000 7ff66d200640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6440725e0 0x7ff644074aa0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.979+0000 7ff66d200640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6681f51d0 0x7ff6680ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.979+0000 7ff66d200640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff66815f450 0x7ff6680c9c30 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.979+0000 7ff66d200640 1 --2- 172.21.15.16:0/2063916130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6680d0890 0x7ff6680b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.979+0000 7ff66d200640 1 -- 172.21.15.16:0/2063916130 >> 172.21.15.16:0/2063916130 conn(0x7ff6680d1d80 msgr2=0x7ff6680cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.979+0000 7ff66d200640 1 -- 172.21.15.16:0/2063916130 shutdown_connections 2024-09-10T21:13:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:52.979+0000 7ff66d200640 1 -- 172.21.15.16:0/2063916130 wait complete. 2024-09-10T21:13:53.008 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:13:53.008 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:13:53.009 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:13:53.009 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:13:53.009 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:13:53.009 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:13:53.009 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:13:53.010 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:13:53.010 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:13:53.010 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:13:53.010 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:13:53.010 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:13:53.010 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:13:53.011 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:13:53.011 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:13:53.011 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:13:53.021 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:13:53.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:13:53.023 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:13:53.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:13:53.028 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:13:53.028 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:13:53.029 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:13:53.029 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:13:53.070 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:13:53.071 DEBUG:teuthology.parallel:result is None 2024-09-10T21:13:53.073 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:13:53.073 DEBUG:teuthology.parallel:result is None 2024-09-10T21:13:53.076 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:13:53.076 DEBUG:teuthology.parallel:result is None 2024-09-10T21:13:53.078 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:13:53.078 DEBUG:teuthology.parallel:result is None 2024-09-10T21:13:53.078 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:13:53.158 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-10T21:13:53.159 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-10T21:13:53.171 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:13:53.172 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:13:53.174 INFO:tasks.ceph.mds.d.smithi052.stderr:2024-09-10T21:13:53.166+0000 7fa86cc00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-10T21:13:53.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.223+0000 7fb3e7200640 1 -- 172.21.15.16:0/2343399930 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3e8073c90 msgr2=0x7fb3e80cb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.223+0000 7fb3e7200640 1 --2- 172.21.15.16:0/2343399930 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3e8073c90 0x7fb3e80cb160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009870 tx=0x7fb3d001c860 comp rx=0 tx=0).stop 2024-09-10T21:13:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.223+0000 7fb3e7200640 1 -- 172.21.15.16:0/2343399930 shutdown_connections 2024-09-10T21:13:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.223+0000 7fb3e7200640 1 --2- 172.21.15.16:0/2343399930 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3e8073c90 0x7fb3e80cb160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.223+0000 7fb3e7200640 1 -- 172.21.15.16:0/2343399930 >> 172.21.15.16:0/2343399930 conn(0x7fb3e80cd590 msgr2=0x7fb3e80cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.223+0000 7fb3e7200640 1 -- 172.21.15.16:0/2343399930 shutdown_connections 2024-09-10T21:13:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.223+0000 7fb3e7200640 1 -- 172.21.15.16:0/2343399930 wait complete. 2024-09-10T21:13:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e7200640 1 Processor -- start 2024-09-10T21:13:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e7200640 1 -- start start 2024-09-10T21:13:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e8073c90 0x7fb3e80c3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e7200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3e80742e0 0x7fb3e8151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e7200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e80cc020 0x7fb3e8152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e81690f0 con 0x7fb3e8073c90 2024-09-10T21:13:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e7200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e8168f70 con 0x7fb3e80cc020 2024-09-10T21:13:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e7200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e8169270 con 0x7fb3e80742e0 2024-09-10T21:13:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e8073c90 0x7fb3e80c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e8073c90 0x7fb3e80c3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56882/0 (socket says 172.21.15.16:56882) 2024-09-10T21:13:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e5e00640 1 -- 172.21.15.16:0/31334671 learned_addr learned my addr 172.21.15.16:0/31334671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e6800640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e80cc020 0x7fb3e8152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3dfe00640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3e80742e0 0x7fb3e8151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e5e00640 1 -- 172.21.15.16:0/31334671 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3e80742e0 msgr2=0x7fb3e8151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e5e00640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3e80742e0 0x7fb3e8151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e5e00640 1 -- 172.21.15.16:0/31334671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e80cc020 msgr2=0x7fb3e8152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e5e00640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e80cc020 0x7fb3e8152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e5e00640 1 -- 172.21.15.16:0/31334671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e8152cd0 con 0x7fb3e8073c90 2024-09-10T21:13:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e5e00640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e8073c90 0x7fb3e80c3860 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800c400 tx=0x7fb3d800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3de000640 1 -- 172.21.15.16:0/31334671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8017070 con 0x7fb3e8073c90 2024-09-10T21:13:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3de000640 1 -- 172.21.15.16:0/31334671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3d8005400 con 0x7fb3e8073c90 2024-09-10T21:13:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3de000640 1 -- 172.21.15.16:0/31334671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8003860 con 0x7fb3e8073c90 2024-09-10T21:13:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e7200640 1 -- 172.21.15.16:0/31334671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e814feb0 con 0x7fb3e8073c90 2024-09-10T21:13:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.227+0000 7fb3e7200640 1 -- 172.21.15.16:0/31334671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e8150470 con 0x7fb3e8073c90 2024-09-10T21:13:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.231+0000 7fb3e7200640 1 -- 172.21.15.16:0/31334671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e80cb160 con 0x7fb3e8073c90 2024-09-10T21:13:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.231+0000 7fb3de000640 1 -- 172.21.15.16:0/31334671 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb3d8003a00 con 0x7fb3e8073c90 2024-09-10T21:13:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.231+0000 7fb3de000640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3c40725e0 0x7fb3c4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.231+0000 7fb3de000640 1 -- 172.21.15.16:0/31334671 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7fb3d8092530 con 0x7fb3e8073c90 2024-09-10T21:13:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.231+0000 7fb3dfe00640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3c40725e0 0x7fb3c4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.231+0000 7fb3dfe00640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3c40725e0 0x7fb3c4074aa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00af000 tx=0x7fb3d001b0a0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.235+0000 7fb3de000640 1 -- 172.21.15.16:0/31334671 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb3d805d8b0 con 0x7fb3e8073c90 2024-09-10T21:13:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.423+0000 7fb3e7200640 1 -- 172.21.15.16:0/31334671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb3e80b6a90 con 0x7fb3e8073c90 2024-09-10T21:13:53.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:13:53.573 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:13:53.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.615+0000 7fb3de000640 1 -- 172.21.15.16:0/31334671 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) ==== 97+0+0 (secure 0 0 0) 0x7fb3d80622e0 con 0x7fb3e8073c90 2024-09-10T21:13:53.621 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-10T21:13:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.619+0000 7fb3e7200640 1 -- 172.21.15.16:0/31334671 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3c40725e0 msgr2=0x7fb3c4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.619+0000 7fb3e7200640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3c40725e0 0x7fb3c4074aa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00af000 tx=0x7fb3d001b0a0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.619+0000 7fb3e7200640 1 -- 172.21.15.16:0/31334671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e8073c90 msgr2=0x7fb3e80c3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.619+0000 7fb3e7200640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e8073c90 0x7fb3e80c3860 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800c400 tx=0x7fb3d800c8c0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.619+0000 7fb3e7200640 1 -- 172.21.15.16:0/31334671 shutdown_connections 2024-09-10T21:13:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.619+0000 7fb3e7200640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3c40725e0 0x7fb3c4074aa0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.619+0000 7fb3e7200640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3e80cc020 0x7fb3e8152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.619+0000 7fb3e7200640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3e80742e0 0x7fb3e8151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.619+0000 7fb3e7200640 1 --2- 172.21.15.16:0/31334671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e8073c90 0x7fb3e80c3860 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.619+0000 7fb3e7200640 1 -- 172.21.15.16:0/31334671 >> 172.21.15.16:0/31334671 conn(0x7fb3e80cd590 msgr2=0x7fb3e80cf8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.619+0000 7fb3e7200640 1 -- 172.21.15.16:0/31334671 shutdown_connections 2024-09-10T21:13:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.619+0000 7fb3e7200640 1 -- 172.21.15.16:0/31334671 wait complete. 2024-09-10T21:13:53.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:13:53.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 -- 172.21.15.16:0/1312337211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c040706f0 msgr2=0x7f0c04165760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 --2- 172.21.15.16:0/1312337211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c040706f0 0x7f0c04165760 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009bc0 tx=0x7f0bfc01cba0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 -- 172.21.15.16:0/1312337211 shutdown_connections 2024-09-10T21:13:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 --2- 172.21.15.16:0/1312337211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c040706f0 0x7f0c04165760 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 -- 172.21.15.16:0/1312337211 >> 172.21.15.16:0/1312337211 conn(0x7f0c040cfb20 msgr2=0x7f0c040cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 -- 172.21.15.16:0/1312337211 shutdown_connections 2024-09-10T21:13:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 -- 172.21.15.16:0/1312337211 wait complete. 2024-09-10T21:13:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 Processor -- start 2024-09-10T21:13:53.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 -- start start 2024-09-10T21:13:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0c040706f0 0x7f0c040b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04154f90 0x7f0c041f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c0415bc70 0x7f0c041fafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04168c30 con 0x7f0c04154f90 2024-09-10T21:13:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04168ab0 con 0x7f0c0415bc70 2024-09-10T21:13:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c0c200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0c04168db0 con 0x7f0c040706f0 2024-09-10T21:13:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.827+0000 7f0c08e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04154f90 0x7f0c041f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c08e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04154f90 0x7f0c041f8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56902/0 (socket says 172.21.15.16:56902) 2024-09-10T21:13:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c08e00640 1 -- 172.21.15.16:0/2986249929 learned_addr learned my addr 172.21.15.16:0/2986249929 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c0a200640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c0415bc70 0x7f0c041fafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c09800640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0c040706f0 0x7f0c040b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c08e00640 1 -- 172.21.15.16:0/2986249929 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0c040706f0 msgr2=0x7f0c040b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c08e00640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0c040706f0 0x7f0c040b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c08e00640 1 -- 172.21.15.16:0/2986249929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c0415bc70 msgr2=0x7f0c041fafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c08e00640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c0415bc70 0x7f0c041fafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c08e00640 1 -- 172.21.15.16:0/2986249929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c041fb890 con 0x7f0c04154f90 2024-09-10T21:13:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c08e00640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04154f90 0x7f0c041f8360 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c400 tx=0x7f0bf000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c01600640 1 -- 172.21.15.16:0/2986249929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0011790 con 0x7f0c04154f90 2024-09-10T21:13:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c01600640 1 -- 172.21.15.16:0/2986249929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bf0017070 con 0x7f0c04154f90 2024-09-10T21:13:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c0c200640 1 -- 172.21.15.16:0/2986249929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c041fbb70 con 0x7f0c04154f90 2024-09-10T21:13:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c01600640 1 -- 172.21.15.16:0/2986249929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0011cc0 con 0x7f0c04154f90 2024-09-10T21:13:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c0c200640 1 -- 172.21.15.16:0/2986249929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c040b56f0 con 0x7f0c04154f90 2024-09-10T21:13:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c0c200640 1 -- 172.21.15.16:0/2986249929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c04072d40 con 0x7f0c04154f90 2024-09-10T21:13:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.831+0000 7f0c01600640 1 -- 172.21.15.16:0/2986249929 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0bf001c460 con 0x7f0c04154f90 2024-09-10T21:13:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.835+0000 7f0c01600640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0be4072610 0x7f0be4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.835+0000 7f0c01600640 1 -- 172.21.15.16:0/2986249929 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4857+0+0 (secure 0 0 0) 0x7f0bf0092d30 con 0x7f0c04154f90 2024-09-10T21:13:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.835+0000 7f0c09800640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0be4072610 0x7f0be4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.835+0000 7f0c09800640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0be4072610 0x7f0be4074ad0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0c040b1600 tx=0x7f0bf8009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:53.839+0000 7f0c01600640 1 -- 172.21.15.16:0/2986249929 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0bf005e6c0 con 0x7f0c04154f90 2024-09-10T21:13:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c0c200640 1 -- 172.21.15.16:0/2986249929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0c041510a0 con 0x7f0c04154f90 2024-09-10T21:13:54.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c01600640 1 -- 172.21.15.16:0/2986249929 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) ==== 80+0+5304 (secure 0 0 0) 0x7f0bf005dfc0 con 0x7f0c04154f90 2024-09-10T21:13:54.082 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:54.082 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:13:54.082 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-10T21:13:54.082 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.082 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-10T21:13:54.082 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmhg+lJBAAWjuYW75BGo2dmyAUi7s9+A==", 2024-09-10T21:13:54.082 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:54.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:13:54.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:13:54.083 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.083 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.083 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.083 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-10T21:13:54.083 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmQr7+MBAAOzXhWIirBOoi/7wKnrFgmg==", 2024-09-10T21:13:54.083 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:54.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:13:54.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:13:54.083 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.083 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.084 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.084 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-10T21:13:54.084 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmxxoALBAAgkmw91/VP1Uv2WMZsJKAyA==", 2024-09-10T21:13:54.084 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:54.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:13:54.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:13:54.084 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.084 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.085 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-10T21:13:54.085 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmA4tzOBAAiS+8n+Gokz6yXapVf2ecYg==", 2024-09-10T21:13:54.085 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.085 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:54.085 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:13:54.085 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:13:54.085 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.085 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-10T21:13:54.085 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHteBmRoahLRAA32PsUkqtGkeI7tMFRwas5Q==", 2024-09-10T21:13:54.086 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:54.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:54.086 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.086 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.086 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.086 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-10T21:13:54.086 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCLteBmGmEMBhAAr1yQEf/NkiN4+ezAQ9VR2A==", 2024-09-10T21:13:54.086 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:54.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:54.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.087 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.087 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.087 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-10T21:13:54.087 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCOteBmmMuAHBAAt+G/KApsZymjTLRSmyRbdw==", 2024-09-10T21:13:54.087 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:54.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:54.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.087 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.088 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.088 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-10T21:13:54.088 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCRteBmoQfNMBAA0vnIqlYoI13KNAAfNmJg3w==", 2024-09-10T21:13:54.088 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:54.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:54.088 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.088 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.088 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.088 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-10T21:13:54.089 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCWteBmc/7lDxAADlSi4UpcFJVNdp6YTkpdSw==", 2024-09-10T21:13:54.089 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:54.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:54.089 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.089 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.089 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-10T21:13:54.089 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZteBmBZ1LJhAAPcbZa1LQu4kNbSsJ/8Z81A==", 2024-09-10T21:13:54.089 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:54.090 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:54.090 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.090 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.090 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.090 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-10T21:13:54.090 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCcteBm7/czOxAAqREnY84+LF1N9dRXOhzz6g==", 2024-09-10T21:13:54.090 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.090 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:54.090 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:54.090 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.090 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-10T21:13:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgteBmG2J8FRAAB+cMFCnf9wJg1R3RD5SUJw==", 2024-09-10T21:13:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:13:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:13:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.091 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.092 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-10T21:13:54.092 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA==", 2024-09-10T21:13:54.092 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.092 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:54.092 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-10T21:13:54.092 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:13:54.092 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.092 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.092 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-10T21:13:54.092 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmuEm4ChAAe08QvhwIcN8ZHa3AoVcYww==", 2024-09-10T21:13:54.092 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:54.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:13:54.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:13:54.093 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:13:54.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.093 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.093 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-10T21:13:54.093 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBm50o1BxAAIFbVxVqU4rU8eKYpuZjTHA==", 2024-09-10T21:13:54.093 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:54.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:13:54.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:13:54.094 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:13:54.094 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.094 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.094 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-10T21:13:54.094 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmxtu0EBAAmN7+07lDiwjsIC/pNYUlSw==", 2024-09-10T21:13:54.094 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:13:54.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:13:54.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:13:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:13:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-10T21:13:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmfqgYCBAAm00tBe45phGieCcp1q46cw==", 2024-09-10T21:13:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:13:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:13:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:13:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.096 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-10T21:13:54.096 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmMVABIBAAoB72mPyXyUO6Y6k+fMqQQg==", 2024-09-10T21:13:54.096 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.096 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:13:54.096 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:13:54.096 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.096 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:13:54.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.096 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-10T21:13:54.096 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmeyYxHBAAvBkpTzjPQKAOlHzI1jQ46g==", 2024-09-10T21:13:54.096 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:13:54.097 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:13:54.097 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:13:54.097 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:13:54.097 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.097 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.097 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-10T21:13:54.097 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:13:54.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c0c200640 1 -- 172.21.15.16:0/2986249929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0be4072610 msgr2=0x7f0be4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c0c200640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0be4072610 0x7f0be4074ad0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0c040b1600 tx=0x7f0bf8009040 comp rx=0 tx=0).stop 2024-09-10T21:13:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c0c200640 1 -- 172.21.15.16:0/2986249929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04154f90 msgr2=0x7f0c041f8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c0c200640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04154f90 0x7f0c041f8360 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c400 tx=0x7f0bf000c8c0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c0c200640 1 -- 172.21.15.16:0/2986249929 shutdown_connections 2024-09-10T21:13:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c0c200640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0be4072610 0x7f0be4074ad0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c0c200640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c0415bc70 0x7f0c041fafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c0c200640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04154f90 0x7f0c041f8360 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c0c200640 1 --2- 172.21.15.16:0/2986249929 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0c040706f0 0x7f0c040b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c0c200640 1 -- 172.21.15.16:0/2986249929 >> 172.21.15.16:0/2986249929 conn(0x7f0c040cfb20 msgr2=0x7f0c040cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c0c200640 1 -- 172.21.15.16:0/2986249929 shutdown_connections 2024-09-10T21:13:54.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.075+0000 7f0c0c200640 1 -- 172.21.15.16:0/2986249929 wait complete. 2024-09-10T21:13:54.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:13:54.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.243+0000 7f7166200640 1 -- 172.21.15.16:0/3432323531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7144003a60 msgr2=0x7f7144003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:54.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.243+0000 7f7166200640 1 --2- 172.21.15.16:0/3432323531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7144003a60 0x7f7144003e80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7150009b30 tx=0x7f715001d210 comp rx=0 tx=0).stop 2024-09-10T21:13:54.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.243+0000 7f715fe00640 1 -- 172.21.15.16:0/3432323531 reap_dead start 2024-09-10T21:13:54.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.243+0000 7f7166200640 1 -- 172.21.15.16:0/3432323531 shutdown_connections 2024-09-10T21:13:54.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.243+0000 7f7166200640 1 -- 172.21.15.16:0/3432323531 >> 172.21.15.16:0/3432323531 conn(0x7f71600d0c30 msgr2=0x7f71600d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:54.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.243+0000 7f7166200640 1 -- 172.21.15.16:0/3432323531 shutdown_connections 2024-09-10T21:13:54.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.243+0000 7f7166200640 1 -- 172.21.15.16:0/3432323531 wait complete. 2024-09-10T21:13:54.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7166200640 1 Processor -- start 2024-09-10T21:13:54.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7166200640 1 -- start start 2024-09-10T21:13:54.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7166200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7144003a60 0x7f71600c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:54.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7166200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f716014fde0 0x7f716014f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:54.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7166200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600b4b90 0x7f71600b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:54.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7166200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f71600cf210 con 0x7f71600b4b90 2024-09-10T21:13:54.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7166200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71600cf090 con 0x7f716014fde0 2024-09-10T21:13:54.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7166200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7160169270 con 0x7f7144003a60 2024-09-10T21:13:54.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f715fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600b4b90 0x7f71600b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f715fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600b4b90 0x7f71600b5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56926/0 (socket says 172.21.15.16:56926) 2024-09-10T21:13:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f715fe00640 1 -- 172.21.15.16:0/966619949 learned_addr learned my addr 172.21.15.16:0/966619949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f715f400640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7144003a60 0x7f71600c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7157e00640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f716014fde0 0x7f716014f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f715fe00640 1 -- 172.21.15.16:0/966619949 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7144003a60 msgr2=0x7f71600c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f715fe00640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7144003a60 0x7f71600c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f715fe00640 1 -- 172.21.15.16:0/966619949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f716014fde0 msgr2=0x7f716014f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f715fe00640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f716014fde0 0x7f716014f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f715fe00640 1 -- 172.21.15.16:0/966619949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7160161470 con 0x7f71600b4b90 2024-09-10T21:13:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f715fe00640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600b4b90 0x7f71600b5010 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7160073750 tx=0x7f7158010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:54.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7157400640 1 -- 172.21.15.16:0/966619949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158003980 con 0x7f71600b4b90 2024-09-10T21:13:54.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7157400640 1 -- 172.21.15.16:0/966619949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7158015070 con 0x7f71600b4b90 2024-09-10T21:13:54.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7157400640 1 -- 172.21.15.16:0/966619949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158004030 con 0x7f71600b4b90 2024-09-10T21:13:54.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7166200640 1 -- 172.21.15.16:0/966619949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71600c7270 con 0x7f71600b4b90 2024-09-10T21:13:54.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.247+0000 7f7166200640 1 -- 172.21.15.16:0/966619949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7160161100 con 0x7f71600b4b90 2024-09-10T21:13:54.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.251+0000 7f7166200640 1 -- 172.21.15.16:0/966619949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71600ce800 con 0x7f71600b4b90 2024-09-10T21:13:54.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.251+0000 7f7157400640 1 -- 172.21.15.16:0/966619949 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f71580041d0 con 0x7f71600b4b90 2024-09-10T21:13:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.251+0000 7f7157400640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f713c072610 0x7f713c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.251+0000 7f7157400640 1 -- 172.21.15.16:0/966619949 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4857+0+0 (secure 0 0 0) 0x7f7158093930 con 0x7f71600b4b90 2024-09-10T21:13:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.251+0000 7f715f400640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f713c072610 0x7f713c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.251+0000 7f715f400640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f713c072610 0x7f713c074ad0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f71500ae000 tx=0x7f7150019fb0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:54.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.255+0000 7f7157400640 1 -- 172.21.15.16:0/966619949 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f715805f2c0 con 0x7f71600b4b90 2024-09-10T21:13:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.431+0000 7f7166200640 1 -- 172.21.15.16:0/966619949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f71600c4450 con 0x7f71600b4b90 2024-09-10T21:13:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.431+0000 7f7157400640 1 -- 172.21.15.16:0/966619949 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7f715805ebc0 con 0x7f71600b4b90 2024-09-10T21:13:54.440 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:54.440 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-10T21:13:54.440 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:13:54.440 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-10T21:13:54.440 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-10T21:13:54.441 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:13:54.441 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-10T21:13:54.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.435+0000 7f7166200640 1 -- 172.21.15.16:0/966619949 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f713c072610 msgr2=0x7f713c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:54.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.435+0000 7f7166200640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f713c072610 0x7f713c074ad0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f71500ae000 tx=0x7f7150019fb0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.435+0000 7f7166200640 1 -- 172.21.15.16:0/966619949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600b4b90 msgr2=0x7f71600b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.435+0000 7f7166200640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600b4b90 0x7f71600b5010 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7160073750 tx=0x7f7158010f40 comp rx=0 tx=0).stop 2024-09-10T21:13:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.435+0000 7f7166200640 1 -- 172.21.15.16:0/966619949 shutdown_connections 2024-09-10T21:13:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.435+0000 7f7166200640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f713c072610 0x7f713c074ad0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.435+0000 7f7166200640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600b4b90 0x7f71600b5010 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.435+0000 7f7166200640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f716014fde0 0x7f716014f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.435+0000 7f7166200640 1 --2- 172.21.15.16:0/966619949 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7144003a60 0x7f71600c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.435+0000 7f7166200640 1 -- 172.21.15.16:0/966619949 >> 172.21.15.16:0/966619949 conn(0x7f71600d0c30 msgr2=0x7f71600b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.435+0000 7f7166200640 1 -- 172.21.15.16:0/966619949 shutdown_connections 2024-09-10T21:13:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.435+0000 7f7166200640 1 -- 172.21.15.16:0/966619949 wait complete. 2024-09-10T21:13:54.462 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:13:54.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:13:54.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.603+0000 7fb344c00640 1 -- 172.21.15.16:0/3208551186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400ce790 msgr2=0x7fb3400d28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:54.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.603+0000 7fb344c00640 1 --2- 172.21.15.16:0/3208551186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400ce790 0x7fb3400d28a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb330009930 tx=0x7fb33001c850 comp rx=0 tx=0).stop 2024-09-10T21:13:54.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.603+0000 7fb344c00640 1 -- 172.21.15.16:0/3208551186 shutdown_connections 2024-09-10T21:13:54.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.603+0000 7fb344c00640 1 --2- 172.21.15.16:0/3208551186 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3400ceb70 0x7fb3400b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.603+0000 7fb344c00640 1 --2- 172.21.15.16:0/3208551186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400ce790 0x7fb3400d28a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.603+0000 7fb344c00640 1 -- 172.21.15.16:0/3208551186 >> 172.21.15.16:0/3208551186 conn(0x7fb3400d1d80 msgr2=0x7fb3400d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:54.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.603+0000 7fb344c00640 1 -- 172.21.15.16:0/3208551186 shutdown_connections 2024-09-10T21:13:54.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.603+0000 7fb344c00640 1 -- 172.21.15.16:0/3208551186 wait complete. 2024-09-10T21:13:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb344c00640 1 Processor -- start 2024-09-10T21:13:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb344c00640 1 -- start start 2024-09-10T21:13:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb344c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400ce790 0x7fb3400c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb344c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3400ceb70 0x7fb34014a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb344c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3400d0260 0x7fb3400b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb344c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb34015f3e0 con 0x7fb3400ce790 2024-09-10T21:13:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb344c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb3400b1930 con 0x7fb3400d0260 2024-09-10T21:13:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb344c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb34015f560 con 0x7fb3400ceb70 2024-09-10T21:13:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb33de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400ce790 0x7fb3400c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb33de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400ce790 0x7fb3400c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56944/0 (socket says 172.21.15.16:56944) 2024-09-10T21:13:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb33de00640 1 -- 172.21.15.16:0/3456422432 learned_addr learned my addr 172.21.15.16:0/3456422432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb33e800640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3400d0260 0x7fb3400b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb337e00640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3400ceb70 0x7fb34014a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb33de00640 1 -- 172.21.15.16:0/3456422432 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3400ceb70 msgr2=0x7fb34014a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb33de00640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3400ceb70 0x7fb34014a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb33de00640 1 -- 172.21.15.16:0/3456422432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3400d0260 msgr2=0x7fb3400b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb33de00640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3400d0260 0x7fb3400b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb33de00640 1 -- 172.21.15.16:0/3456422432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34014a6d0 con 0x7fb3400ce790 2024-09-10T21:13:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb337e00640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3400ceb70 0x7fb34014a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb33de00640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400ce790 0x7fb3400c81f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb330009850 tx=0x7fb3300a65b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb336000640 1 -- 172.21.15.16:0/3456422432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3300ac070 con 0x7fb3400ce790 2024-09-10T21:13:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb336000640 1 -- 172.21.15.16:0/3456422432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb330004310 con 0x7fb3400ce790 2024-09-10T21:13:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb336000640 1 -- 172.21.15.16:0/3456422432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3300053a0 con 0x7fb3400ce790 2024-09-10T21:13:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb344c00640 1 -- 172.21.15.16:0/3456422432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb340158220 con 0x7fb3400ce790 2024-09-10T21:13:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.607+0000 7fb344c00640 1 -- 172.21.15.16:0/3456422432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3400c68f0 con 0x7fb3400ce790 2024-09-10T21:13:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.611+0000 7fb344c00640 1 -- 172.21.15.16:0/3456422432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3400d28a0 con 0x7fb3400ce790 2024-09-10T21:13:54.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.611+0000 7fb336000640 1 -- 172.21.15.16:0/3456422432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb3300046d0 con 0x7fb3400ce790 2024-09-10T21:13:54.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.611+0000 7fb336000640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb31c0725e0 0x7fb31c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:54.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.611+0000 7fb336000640 1 -- 172.21.15.16:0/3456422432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4857+0+0 (secure 0 0 0) 0x7fb33012cf00 con 0x7fb3400ce790 2024-09-10T21:13:54.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.611+0000 7fb337e00640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb31c0725e0 0x7fb31c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:54.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.611+0000 7fb337e00640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb31c0725e0 0x7fb31c074aa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb3401581d0 tx=0x7fb328008040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:54.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.615+0000 7fb336000640 1 -- 172.21.15.16:0/3456422432 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb3300bf020 con 0x7fb3400ce790 2024-09-10T21:13:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:54.795+0000 7fb344c00640 1 -- 172.21.15.16:0/3456422432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb3400b7d00 con 0x7fb3400ce790 2024-09-10T21:13:55.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.627+0000 7fb336000640 1 -- 172.21.15.16:0/3456422432 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7fb3300164d0 con 0x7fb3400ce790 2024-09-10T21:13:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-10T21:13:55.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.631+0000 7fb344c00640 1 -- 172.21.15.16:0/3456422432 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb31c0725e0 msgr2=0x7fb31c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:55.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.631+0000 7fb344c00640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb31c0725e0 0x7fb31c074aa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb3401581d0 tx=0x7fb328008040 comp rx=0 tx=0).stop 2024-09-10T21:13:55.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.631+0000 7fb344c00640 1 -- 172.21.15.16:0/3456422432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400ce790 msgr2=0x7fb3400c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:55.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.631+0000 7fb344c00640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400ce790 0x7fb3400c81f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb330009850 tx=0x7fb3300a65b0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.631+0000 7fb344c00640 1 -- 172.21.15.16:0/3456422432 shutdown_connections 2024-09-10T21:13:55.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.631+0000 7fb344c00640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb31c0725e0 0x7fb31c074aa0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.631+0000 7fb344c00640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3400d0260 0x7fb3400b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.631+0000 7fb344c00640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb3400ceb70 0x7fb34014a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.631+0000 7fb344c00640 1 --2- 172.21.15.16:0/3456422432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3400ce790 0x7fb3400c81f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.631+0000 7fb344c00640 1 -- 172.21.15.16:0/3456422432 >> 172.21.15.16:0/3456422432 conn(0x7fb3400d1d80 msgr2=0x7fb3400ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.631+0000 7fb344c00640 1 -- 172.21.15.16:0/3456422432 shutdown_connections 2024-09-10T21:13:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.631+0000 7fb344c00640 1 -- 172.21.15.16:0/3456422432 wait complete. 2024-09-10T21:13:55.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:13:55.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 -- 172.21.15.16:0/694520224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c0d29b0 msgr2=0x7fb95c0b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 --2- 172.21.15.16:0/694520224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c0d29b0 0x7fb95c0b1880 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb95c06fa90 tx=0x7fb95401cae0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 -- 172.21.15.16:0/694520224 shutdown_connections 2024-09-10T21:13:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 --2- 172.21.15.16:0/694520224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c0d29b0 0x7fb95c0b1880 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 --2- 172.21.15.16:0/694520224 >> v2:172.21.15.52:3300/0 conn(0x7fb95c0d25d0 0x7fb95c0b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 -- 172.21.15.16:0/694520224 >> 172.21.15.16:0/694520224 conn(0x7fb95c0d3d90 msgr2=0x7fb95c0d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 -- 172.21.15.16:0/694520224 shutdown_connections 2024-09-10T21:13:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 -- 172.21.15.16:0/694520224 wait complete. 2024-09-10T21:13:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 Processor -- start 2024-09-10T21:13:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 -- start start 2024-09-10T21:13:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb95c0d1e20 0x7fb95c1530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c0d25d0 0x7fb95c0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb95c0d29b0 0x7fb95c1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c0b25d0 con 0x7fb95c0d25d0 2024-09-10T21:13:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c0b4550 con 0x7fb95c0d1e20 2024-09-10T21:13:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.823+0000 7fb961400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb95c0b2750 con 0x7fb95c0d29b0 2024-09-10T21:13:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb953e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c0d25d0 0x7fb95c0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb953e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c0d25d0 0x7fb95c0cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56968/0 (socket says 172.21.15.16:56968) 2024-09-10T21:13:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb953e00640 1 -- 172.21.15.16:0/2346887633 learned_addr learned my addr 172.21.15.16:0/2346887633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb95be00640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb95c0d1e20 0x7fb95c1530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb960a00640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb95c0d29b0 0x7fb95c1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb953e00640 1 -- 172.21.15.16:0/2346887633 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb95c0d29b0 msgr2=0x7fb95c1581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb953e00640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb95c0d29b0 0x7fb95c1581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb953e00640 1 -- 172.21.15.16:0/2346887633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb95c0d1e20 msgr2=0x7fb95c1530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb953e00640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb95c0d1e20 0x7fb95c1530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb953e00640 1 -- 172.21.15.16:0/2346887633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c0b8ef0 con 0x7fb95c0d25d0 2024-09-10T21:13:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb953e00640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c0d25d0 0x7fb95c0cd1b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb94400c480 tx=0x7fb94400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb958c00640 1 -- 172.21.15.16:0/2346887633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944012830 con 0x7fb95c0d25d0 2024-09-10T21:13:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb958c00640 1 -- 172.21.15.16:0/2346887633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb944017070 con 0x7fb95c0d25d0 2024-09-10T21:13:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb958c00640 1 -- 172.21.15.16:0/2346887633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944012d80 con 0x7fb95c0d25d0 2024-09-10T21:13:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb961400640 1 -- 172.21.15.16:0/2346887633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95c161200 con 0x7fb95c0d25d0 2024-09-10T21:13:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb961400640 1 -- 172.21.15.16:0/2346887633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95c152bd0 con 0x7fb95c0d25d0 2024-09-10T21:13:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb961400640 1 -- 172.21.15.16:0/2346887633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb95c0b1880 con 0x7fb95c0d25d0 2024-09-10T21:13:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.827+0000 7fb958c00640 1 -- 172.21.15.16:0/2346887633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb94401c460 con 0x7fb95c0d25d0 2024-09-10T21:13:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.831+0000 7fb958c00640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9380725e0 0x7fb938074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.831+0000 7fb958c00640 1 -- 172.21.15.16:0/2346887633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5227+0+0 (secure 0 0 0) 0x7fb944092d30 con 0x7fb95c0d25d0 2024-09-10T21:13:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.831+0000 7fb95be00640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9380725e0 0x7fb938074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.831+0000 7fb95be00640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9380725e0 0x7fb938074aa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00abe0 tx=0x7fb94c00a2d0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:55.835+0000 7fb958c00640 1 -- 172.21.15.16:0/2346887633 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb94405df90 con 0x7fb95c0d25d0 2024-09-10T21:13:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.007+0000 7fb961400640 1 -- 172.21.15.16:0/2346887633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb95c06def0 con 0x7fb95c0d25d0 2024-09-10T21:13:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.631+0000 7fb958c00640 1 -- 172.21.15.16:0/2346887633 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7fb9440629c0 con 0x7fb95c0d25d0 2024-09-10T21:13:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-10T21:13:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.635+0000 7fb961400640 1 -- 172.21.15.16:0/2346887633 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9380725e0 msgr2=0x7fb938074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.635+0000 7fb961400640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9380725e0 0x7fb938074aa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00abe0 tx=0x7fb94c00a2d0 comp rx=0 tx=0).stop 2024-09-10T21:13:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.635+0000 7fb961400640 1 -- 172.21.15.16:0/2346887633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c0d25d0 msgr2=0x7fb95c0cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.635+0000 7fb961400640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c0d25d0 0x7fb95c0cd1b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb94400c480 tx=0x7fb94400c940 comp rx=0 tx=0).stop 2024-09-10T21:13:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.635+0000 7fb961400640 1 -- 172.21.15.16:0/2346887633 shutdown_connections 2024-09-10T21:13:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.635+0000 7fb961400640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9380725e0 0x7fb938074aa0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.635+0000 7fb961400640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb95c0d29b0 0x7fb95c1581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.635+0000 7fb961400640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c0d25d0 0x7fb95c0cd1b0 secure :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb94400c480 tx=0x7fb94400c940 comp rx=0 tx=0).stop 2024-09-10T21:13:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.635+0000 7fb961400640 1 --2- 172.21.15.16:0/2346887633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb95c0d1e20 0x7fb95c1530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.635+0000 7fb961400640 1 -- 172.21.15.16:0/2346887633 >> 172.21.15.16:0/2346887633 conn(0x7fb95c0d3d90 msgr2=0x7fb95c06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.635+0000 7fb961400640 1 -- 172.21.15.16:0/2346887633 shutdown_connections 2024-09-10T21:13:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.635+0000 7fb961400640 1 -- 172.21.15.16:0/2346887633 wait complete. 2024-09-10T21:13:56.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:13:56.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.827+0000 7fd292800640 1 -- 172.21.15.16:0/749447684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd28c0cd300 msgr2=0x7fd28c0b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.827+0000 7fd292800640 1 --2- 172.21.15.16:0/749447684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd28c0cd300 0x7fd28c0b4b10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0733a0 tx=0x7fd27401c7a0 comp rx=0 tx=0).stop 2024-09-10T21:13:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.827+0000 7fd292800640 1 -- 172.21.15.16:0/749447684 shutdown_connections 2024-09-10T21:13:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.827+0000 7fd292800640 1 --2- 172.21.15.16:0/749447684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd28c0cd300 0x7fd28c0b4b10 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.827+0000 7fd292800640 1 --2- 172.21.15.16:0/749447684 >> v2:172.21.15.52:3300/0 conn(0x7fd28c15bdc0 0x7fd28c0d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.827+0000 7fd292800640 1 -- 172.21.15.16:0/749447684 >> 172.21.15.16:0/749447684 conn(0x7fd28c0d1ce0 msgr2=0x7fd28c0d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.827+0000 7fd292800640 1 -- 172.21.15.16:0/749447684 shutdown_connections 2024-09-10T21:13:56.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.827+0000 7fd292800640 1 -- 172.21.15.16:0/749447684 wait complete. 2024-09-10T21:13:56.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.827+0000 7fd292800640 1 Processor -- start 2024-09-10T21:13:56.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.827+0000 7fd292800640 1 -- start start 2024-09-10T21:13:56.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd292800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd28c0b5050 0x7fd28c0c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd292800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd28c0cd300 0x7fd28c150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd292800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd28c15bdc0 0x7fd28c1514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd292800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c168e90 con 0x7fd28c0b5050 2024-09-10T21:13:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd292800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c160290 con 0x7fd28c0cd300 2024-09-10T21:13:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd292800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd28c169010 con 0x7fd28c15bdc0 2024-09-10T21:13:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd28b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd28c0b5050 0x7fd28c0c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd28b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd28c0b5050 0x7fd28c0c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56984/0 (socket says 172.21.15.16:56984) 2024-09-10T21:13:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd28b400640 1 -- 172.21.15.16:0/2632666365 learned_addr learned my addr 172.21.15.16:0/2632666365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd282a00640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd28c0cd300 0x7fd28c150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd28be00640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd28c15bdc0 0x7fd28c1514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:56.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd28b400640 1 -- 172.21.15.16:0/2632666365 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd28c15bdc0 msgr2=0x7fd28c1514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:56.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd28b400640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd28c15bdc0 0x7fd28c1514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:56.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd28b400640 1 -- 172.21.15.16:0/2632666365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd28c0cd300 msgr2=0x7fd28c150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:56.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd28b400640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd28c0cd300 0x7fd28c150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:56.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd28b400640 1 -- 172.21.15.16:0/2632666365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c152ab0 con 0x7fd28c0b5050 2024-09-10T21:13:56.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd28b400640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd28c0b5050 0x7fd28c0c70c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00d950 tx=0x7fd27c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:56.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd283e00640 1 -- 172.21.15.16:0/2632666365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c017070 con 0x7fd28c0b5050 2024-09-10T21:13:56.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd283e00640 1 -- 172.21.15.16:0/2632666365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd27c009d20 con 0x7fd28c0b5050 2024-09-10T21:13:56.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd283e00640 1 -- 172.21.15.16:0/2632666365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0106c0 con 0x7fd28c0b5050 2024-09-10T21:13:56.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd292800640 1 -- 172.21.15.16:0/2632666365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c1f6470 con 0x7fd28c0b5050 2024-09-10T21:13:56.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd292800640 1 -- 172.21.15.16:0/2632666365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c0c59f0 con 0x7fd28c0b5050 2024-09-10T21:13:56.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.831+0000 7fd292800640 1 -- 172.21.15.16:0/2632666365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c0b4b10 con 0x7fd28c0b5050 2024-09-10T21:13:56.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.835+0000 7fd283e00640 1 -- 172.21.15.16:0/2632666365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd27c010860 con 0x7fd28c0b5050 2024-09-10T21:13:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.835+0000 7fd283e00640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd2680725e0 0x7fd268074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.835+0000 7fd283e00640 1 -- 172.21.15.16:0/2632666365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5609+0+0 (secure 0 0 0) 0x7fd27c093af0 con 0x7fd28c0b5050 2024-09-10T21:13:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.835+0000 7fd282a00640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd2680725e0 0x7fd268074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.835+0000 7fd282a00640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd2680725e0 0x7fd268074aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd2740af000 tx=0x7fd2740b2040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:56.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:56.839+0000 7fd283e00640 1 -- 172.21.15.16:0/2632666365 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd27c05ebc0 con 0x7fd28c0b5050 2024-09-10T21:13:57.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.047+0000 7fd292800640 1 -- 172.21.15.16:0/2632666365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd28c159210 con 0x7fd28c0b5050 2024-09-10T21:13:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.655+0000 7fd283e00640 1 -- 172.21.15.16:0/2632666365 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v46) ==== 174+0+0 (secure 0 0 0) 0x7fd27c0635f0 con 0x7fd28c0b5050 2024-09-10T21:13:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 12 and data pool 13 2024-09-10T21:13:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.663+0000 7fd292800640 1 -- 172.21.15.16:0/2632666365 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd2680725e0 msgr2=0x7fd268074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.663+0000 7fd292800640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd2680725e0 0x7fd268074aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd2740af000 tx=0x7fd2740b2040 comp rx=0 tx=0).stop 2024-09-10T21:13:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.663+0000 7fd292800640 1 -- 172.21.15.16:0/2632666365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd28c0b5050 msgr2=0x7fd28c0c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.663+0000 7fd292800640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd28c0b5050 0x7fd28c0c70c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00d950 tx=0x7fd27c00de10 comp rx=0 tx=0).stop 2024-09-10T21:13:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.663+0000 7fd292800640 1 -- 172.21.15.16:0/2632666365 shutdown_connections 2024-09-10T21:13:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.663+0000 7fd292800640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd2680725e0 0x7fd268074aa0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.663+0000 7fd292800640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd28c15bdc0 0x7fd28c1514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.663+0000 7fd292800640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd28c0cd300 0x7fd28c150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.663+0000 7fd292800640 1 --2- 172.21.15.16:0/2632666365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd28c0b5050 0x7fd28c0c70c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.663+0000 7fd292800640 1 -- 172.21.15.16:0/2632666365 >> 172.21.15.16:0/2632666365 conn(0x7fd28c0d1ce0 msgr2=0x7fd28c0b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.663+0000 7fd292800640 1 -- 172.21.15.16:0/2632666365 shutdown_connections 2024-09-10T21:13:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.663+0000 7fd292800640 1 -- 172.21.15.16:0/2632666365 wait complete. 2024-09-10T21:13:57.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:13:57.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.851+0000 7fdf08600640 1 -- 172.21.15.16:0/3345715991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf0015a270 msgr2=0x7fdf000ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.851+0000 7fdf08600640 1 --2- 172.21.15.16:0/3345715991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf0015a270 0x7fdf000ce610 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fdef400aa10 tx=0x7fdef401db70 comp rx=0 tx=0).stop 2024-09-10T21:13:57.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.851+0000 7fdf08600640 1 -- 172.21.15.16:0/3345715991 shutdown_connections 2024-09-10T21:13:57.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.851+0000 7fdf08600640 1 --2- 172.21.15.16:0/3345715991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf0015a270 0x7fdf000ce610 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.851+0000 7fdf08600640 1 -- 172.21.15.16:0/3345715991 >> 172.21.15.16:0/3345715991 conn(0x7fdf000cfb20 msgr2=0x7fdf000cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:57.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.851+0000 7fdf08600640 1 -- 172.21.15.16:0/3345715991 shutdown_connections 2024-09-10T21:13:57.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.851+0000 7fdf08600640 1 -- 172.21.15.16:0/3345715991 wait complete. 2024-09-10T21:13:57.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.851+0000 7fdf08600640 1 Processor -- start 2024-09-10T21:13:57.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.851+0000 7fdf08600640 1 -- start start 2024-09-10T21:13:57.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf08600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdf000ca170 0x7fdf000c2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf08600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf000d06b0 0x7fdf0014de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf08600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf0015a270 0x7fdf000b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf08600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf00168590 con 0x7fdf0015a270 2024-09-10T21:13:57.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf08600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf00168410 con 0x7fdf000d06b0 2024-09-10T21:13:57.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf08600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fdf00168710 con 0x7fdf000ca170 2024-09-10T21:13:57.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf06600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf0015a270 0x7fdf000b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf05c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdf000ca170 0x7fdf000c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf06600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf0015a270 0x7fdf000b4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57010/0 (socket says 172.21.15.16:57010) 2024-09-10T21:13:57.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf06600640 1 -- 172.21.15.16:0/709117582 learned_addr learned my addr 172.21.15.16:0/709117582 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:57.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf05200640 1 --2- 172.21.15.16:0/709117582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf000d06b0 0x7fdf0014de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf06600640 1 -- 172.21.15.16:0/709117582 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdf000ca170 msgr2=0x7fdf000c2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf06600640 1 --2- 172.21.15.16:0/709117582 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdf000ca170 0x7fdf000c2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf06600640 1 -- 172.21.15.16:0/709117582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf000d06b0 msgr2=0x7fdf0014de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf06600640 1 --2- 172.21.15.16:0/709117582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf000d06b0 0x7fdf0014de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf06600640 1 -- 172.21.15.16:0/709117582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf000c5d60 con 0x7fdf0015a270 2024-09-10T21:13:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf06600640 1 --2- 172.21.15.16:0/709117582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf0015a270 0x7fdf000b4ea0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fdef800c7c0 tx=0x7fdef800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdefd600640 1 -- 172.21.15.16:0/709117582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8003b80 con 0x7fdf0015a270 2024-09-10T21:13:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdefd600640 1 -- 172.21.15.16:0/709117582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdef8003d20 con 0x7fdf0015a270 2024-09-10T21:13:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf08600640 1 -- 172.21.15.16:0/709117582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0015ca30 con 0x7fdf0015a270 2024-09-10T21:13:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdefd600640 1 -- 172.21.15.16:0/709117582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef800aab0 con 0x7fdf0015a270 2024-09-10T21:13:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf08600640 1 -- 172.21.15.16:0/709117582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0015cff0 con 0x7fdf0015a270 2024-09-10T21:13:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdf08600640 1 -- 172.21.15.16:0/709117582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf000ce610 con 0x7fdf0015a270 2024-09-10T21:13:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.855+0000 7fdefd600640 1 -- 172.21.15.16:0/709117582 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdef800ac50 con 0x7fdf0015a270 2024-09-10T21:13:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.859+0000 7fdefd600640 1 --2- 172.21.15.16:0/709117582 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdee00725e0 0x7fdee0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.859+0000 7fdefd600640 1 -- 172.21.15.16:0/709117582 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5709+0+0 (secure 0 0 0) 0x7fdef8092ff0 con 0x7fdf0015a270 2024-09-10T21:13:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.859+0000 7fdf05c00640 1 --2- 172.21.15.16:0/709117582 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdee00725e0 0x7fdee0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.859+0000 7fdf05c00640 1 --2- 172.21.15.16:0/709117582 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdee00725e0 0x7fdee0074aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdef40b0000 tx=0x7fdef401c3b0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:57.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:57.863+0000 7fdefd600640 1 -- 172.21.15.16:0/709117582 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdef805e010 con 0x7fdf0015a270 2024-09-10T21:13:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.035+0000 7fdf08600640 1 -- 172.21.15.16:0/709117582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf0015dbc0 con 0x7fdf0015a270 2024-09-10T21:13:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.035+0000 7fdefd600640 1 -- 172.21.15.16:0/709117582 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16697 (secure 0 0 0) 0x7fdef8062a40 con 0x7fdf0015a270 2024-09-10T21:13:58.045 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:58.045 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:13:57.054876+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:13:54.810241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T21:13:56.024501+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:13:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.039+0000 7fdf08600640 1 -- 172.21.15.16:0/709117582 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdee00725e0 msgr2=0x7fdee0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.039+0000 7fdf08600640 1 --2- 172.21.15.16:0/709117582 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdee00725e0 0x7fdee0074aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdef40b0000 tx=0x7fdef401c3b0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.039+0000 7fdf08600640 1 -- 172.21.15.16:0/709117582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf0015a270 msgr2=0x7fdf000b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.039+0000 7fdf08600640 1 --2- 172.21.15.16:0/709117582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf0015a270 0x7fdf000b4ea0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fdef800c7c0 tx=0x7fdef800cc80 comp rx=0 tx=0).stop 2024-09-10T21:13:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.039+0000 7fdf08600640 1 -- 172.21.15.16:0/709117582 shutdown_connections 2024-09-10T21:13:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.039+0000 7fdf08600640 1 --2- 172.21.15.16:0/709117582 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdee00725e0 0x7fdee0074aa0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.039+0000 7fdf08600640 1 --2- 172.21.15.16:0/709117582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf0015a270 0x7fdf000b4ea0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.039+0000 7fdf08600640 1 --2- 172.21.15.16:0/709117582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf000d06b0 0x7fdf0014de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.039+0000 7fdf08600640 1 --2- 172.21.15.16:0/709117582 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdf000ca170 0x7fdf000c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.039+0000 7fdf08600640 1 -- 172.21.15.16:0/709117582 >> 172.21.15.16:0/709117582 conn(0x7fdf000cfb20 msgr2=0x7fdf000cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.039+0000 7fdf08600640 1 -- 172.21.15.16:0/709117582 shutdown_connections 2024-09-10T21:13:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.039+0000 7fdf08600640 1 -- 172.21.15.16:0/709117582 wait complete. 2024-09-10T21:13:58.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:13:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 -- 172.21.15.16:0/1127478856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75815f290 msgr2=0x7fc7580cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 --2- 172.21.15.16:0/1127478856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75815f290 0x7fc7580cd120 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009870 tx=0x7fc7580746e0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 -- 172.21.15.16:0/1127478856 shutdown_connections 2024-09-10T21:13:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 --2- 172.21.15.16:0/1127478856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75815f290 0x7fc7580cd120 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 -- 172.21.15.16:0/1127478856 >> 172.21.15.16:0/1127478856 conn(0x7fc7580d0c30 msgr2=0x7fc7580d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 -- 172.21.15.16:0/1127478856 shutdown_connections 2024-09-10T21:13:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 -- 172.21.15.16:0/1127478856 wait complete. 2024-09-10T21:13:58.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 Processor -- start 2024-09-10T21:13:58.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 -- start start 2024-09-10T21:13:58.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7580c41d0 0x7fc7580af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7580cb990 0x7fc7580ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc75815f290 0x7fc758070c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc75816a150 con 0x7fc7580cb990 2024-09-10T21:13:58.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc758169fd0 con 0x7fc75815f290 2024-09-10T21:13:58.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc75f400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc75816a2d0 con 0x7fc7580c41d0 2024-09-10T21:13:58.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc757e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7580cb990 0x7fc7580ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc757e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7580cb990 0x7fc7580ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57028/0 (socket says 172.21.15.16:57028) 2024-09-10T21:13:58.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.207+0000 7fc757e00640 1 -- 172.21.15.16:0/3746836759 learned_addr learned my addr 172.21.15.16:0/3746836759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:58.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc75ca00640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7580c41d0 0x7fc7580af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc757e00640 1 -- 172.21.15.16:0/3746836759 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7580c41d0 msgr2=0x7fc7580af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc75d400640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc75815f290 0x7fc758070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc757e00640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7580c41d0 0x7fc7580af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc757e00640 1 -- 172.21.15.16:0/3746836759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc75815f290 msgr2=0x7fc758070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc757e00640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc75815f290 0x7fc758070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc757e00640 1 -- 172.21.15.16:0/3746836759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7580afa70 con 0x7fc7580cb990 2024-09-10T21:13:58.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc75ca00640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7580c41d0 0x7fc7580af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:58.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc75d400640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc75815f290 0x7fc758070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:58.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc757e00640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7580cb990 0x7fc7580ad720 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc74400c500 tx=0x7fc74400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:58.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc754c00640 1 -- 172.21.15.16:0/3746836759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744017070 con 0x7fc7580cb990 2024-09-10T21:13:58.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc754c00640 1 -- 172.21.15.16:0/3746836759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7440053c0 con 0x7fc7580cb990 2024-09-10T21:13:58.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc75f400640 1 -- 172.21.15.16:0/3746836759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc758152800 con 0x7fc7580cb990 2024-09-10T21:13:58.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc754c00640 1 -- 172.21.15.16:0/3746836759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744012890 con 0x7fc7580cb990 2024-09-10T21:13:58.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc75f400640 1 -- 172.21.15.16:0/3746836759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc758152dc0 con 0x7fc7580cb990 2024-09-10T21:13:58.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc75f400640 1 -- 172.21.15.16:0/3746836759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7580b6bb0 con 0x7fc7580cb990 2024-09-10T21:13:58.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc754c00640 1 -- 172.21.15.16:0/3746836759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc744003600 con 0x7fc7580cb990 2024-09-10T21:13:58.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.211+0000 7fc754c00640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7380725e0 0x7fc738074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.215+0000 7fc754c00640 1 -- 172.21.15.16:0/3746836759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5709+0+0 (secure 0 0 0) 0x7fc744092810 con 0x7fc7580cb990 2024-09-10T21:13:58.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.215+0000 7fc75ca00640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7380725e0 0x7fc738074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.215+0000 7fc75ca00640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7380725e0 0x7fc738074aa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0af000 tx=0x7fc74c01af40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.219+0000 7fc754c00640 1 -- 172.21.15.16:0/3746836759 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc74405df00 con 0x7fc7580cb990 2024-09-10T21:13:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.391+0000 7fc75f400640 1 -- 172.21.15.16:0/3746836759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7580cd100 con 0x7fc7580cb990 2024-09-10T21:13:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.395+0000 7fc754c00640 1 -- 172.21.15.16:0/3746836759 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16697 (secure 0 0 0) 0x7fc74405d800 con 0x7fc7580cb990 2024-09-10T21:13:58.402 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:13:58.402 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:13:57.054876+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:13:54.810241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T21:13:56.024501+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:13:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.395+0000 7fc75f400640 1 -- 172.21.15.16:0/3746836759 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7380725e0 msgr2=0x7fc738074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.395+0000 7fc75f400640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7380725e0 0x7fc738074aa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0af000 tx=0x7fc74c01af40 comp rx=0 tx=0).stop 2024-09-10T21:13:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.395+0000 7fc75f400640 1 -- 172.21.15.16:0/3746836759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7580cb990 msgr2=0x7fc7580ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.395+0000 7fc75f400640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7580cb990 0x7fc7580ad720 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc74400c500 tx=0x7fc74400c9c0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.395+0000 7fc75f400640 1 -- 172.21.15.16:0/3746836759 shutdown_connections 2024-09-10T21:13:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.395+0000 7fc75f400640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7380725e0 0x7fc738074aa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.395+0000 7fc75f400640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc75815f290 0x7fc758070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.395+0000 7fc75f400640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7580cb990 0x7fc7580ad720 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.395+0000 7fc75f400640 1 --2- 172.21.15.16:0/3746836759 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7580c41d0 0x7fc7580af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.395+0000 7fc75f400640 1 -- 172.21.15.16:0/3746836759 >> 172.21.15.16:0/3746836759 conn(0x7fc7580d0c30 msgr2=0x7fc758157930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.395+0000 7fc75f400640 1 -- 172.21.15.16:0/3746836759 shutdown_connections 2024-09-10T21:13:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.395+0000 7fc75f400640 1 -- 172.21.15.16:0/3746836759 wait complete. 2024-09-10T21:13:58.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:13:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.567+0000 7f1bf4a00640 1 -- 172.21.15.16:0/412023667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd4003a40 msgr2=0x7f1bd4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.567+0000 7f1bf4a00640 1 --2- 172.21.15.16:0/412023667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd4003a40 0x7f1bd4003e60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009b60 tx=0x7f1be001d270 comp rx=0 tx=0).stop 2024-09-10T21:13:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.567+0000 7f1bef200640 1 -- 172.21.15.16:0/412023667 reap_dead start 2024-09-10T21:13:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.567+0000 7f1bf4a00640 1 -- 172.21.15.16:0/412023667 shutdown_connections 2024-09-10T21:13:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.567+0000 7f1bf4a00640 1 -- 172.21.15.16:0/412023667 >> 172.21.15.16:0/412023667 conn(0x7f1bf00d0b60 msgr2=0x7f1bf00d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.567+0000 7f1bf4a00640 1 -- 172.21.15.16:0/412023667 shutdown_connections 2024-09-10T21:13:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.567+0000 7f1bf4a00640 1 -- 172.21.15.16:0/412023667 wait complete. 2024-09-10T21:13:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.567+0000 7f1bf4a00640 1 Processor -- start 2024-09-10T21:13:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.567+0000 7f1bf4a00640 1 -- start start 2024-09-10T21:13:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bf4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd4003a40 0x7f1bf00b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bf4a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1bf00c9eb0 0x7f1bf01fba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bf4a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bf00ca330 0x7f1bf01fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bf4a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf0167180 con 0x7f1bd4003a40 2024-09-10T21:13:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bf4a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf0167000 con 0x7f1bf00ca330 2024-09-10T21:13:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bf4a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f1bf0167300 con 0x7f1bf00c9eb0 2024-09-10T21:13:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bee800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd4003a40 0x7f1bf00b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bee800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd4003a40 0x7f1bf00b64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57034/0 (socket says 172.21.15.16:57034) 2024-09-10T21:13:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bee800640 1 -- 172.21.15.16:0/4080022989 learned_addr learned my addr 172.21.15.16:0/4080022989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1be7e00640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1bf00c9eb0 0x7f1bf01fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bef200640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bf00ca330 0x7f1bf01fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bee800640 1 -- 172.21.15.16:0/4080022989 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1bf00c9eb0 msgr2=0x7f1bf01fba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bee800640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1bf00c9eb0 0x7f1bf01fba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bee800640 1 -- 172.21.15.16:0/4080022989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bf00ca330 msgr2=0x7f1bf01fbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bee800640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bf00ca330 0x7f1bf01fbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bee800640 1 -- 172.21.15.16:0/4080022989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf00c7e80 con 0x7f1bd4003a40 2024-09-10T21:13:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1be7e00640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1bf00c9eb0 0x7f1bf01fba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bef200640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bf00ca330 0x7f1bf01fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bee800640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd4003a40 0x7f1bf00b64a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1be00af040 tx=0x7f1be0005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1be6a00640 1 -- 172.21.15.16:0/4080022989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be00b1040 con 0x7f1bd4003a40 2024-09-10T21:13:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1be6a00640 1 -- 172.21.15.16:0/4080022989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1be00bb070 con 0x7f1bd4003a40 2024-09-10T21:13:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bf4a00640 1 -- 172.21.15.16:0/4080022989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf00c8160 con 0x7f1bd4003a40 2024-09-10T21:13:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1be6a00640 1 -- 172.21.15.16:0/4080022989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be00043a0 con 0x7f1bd4003a40 2024-09-10T21:13:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bf4a00640 1 -- 172.21.15.16:0/4080022989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf00c8720 con 0x7f1bd4003a40 2024-09-10T21:13:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1bf4a00640 1 -- 172.21.15.16:0/4080022989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf015e610 con 0x7f1bd4003a40 2024-09-10T21:13:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.571+0000 7f1be6a00640 1 -- 172.21.15.16:0/4080022989 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1bf00cc130 con 0x7f1bd4003a40 2024-09-10T21:13:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.575+0000 7f1be6a00640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1bcc0725e0 0x7f1bcc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.575+0000 7f1be6a00640 1 -- 172.21.15.16:0/4080022989 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5709+0+0 (secure 0 0 0) 0x7f1be012db50 con 0x7f1bd4003a40 2024-09-10T21:13:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.575+0000 7f1be7e00640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1bcc0725e0 0x7f1bcc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.575+0000 7f1be7e00640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1bcc0725e0 0x7f1bcc074aa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8000fc0 tx=0x7f1bd80067b0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.579+0000 7f1be6a00640 1 -- 172.21.15.16:0/4080022989 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1be0016f80 con 0x7f1bd4003a40 2024-09-10T21:13:58.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:58.791+0000 7f1bf4a00640 1 -- 172.21.15.16:0/4080022989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1bf00c7bd0 con 0x7f1bd4003a40 2024-09-10T21:13:59.637 INFO:tasks.ceph.mds.b.smithi052.stderr:2024-09-10T21:13:59.626+0000 7f31e4000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:13:59.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.659+0000 7f1be6a00640 1 -- 172.21.15.16:0/4080022989 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) ==== 118+0+0 (secure 0 0 0) 0x7f1be00f92b0 con 0x7f1bd4003a40 2024-09-10T21:13:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.663+0000 7f1bf4a00640 1 -- 172.21.15.16:0/4080022989 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1bcc0725e0 msgr2=0x7f1bcc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.663+0000 7f1bf4a00640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1bcc0725e0 0x7f1bcc074aa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8000fc0 tx=0x7f1bd80067b0 comp rx=0 tx=0).stop 2024-09-10T21:13:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.663+0000 7f1bf4a00640 1 -- 172.21.15.16:0/4080022989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd4003a40 msgr2=0x7f1bf00b64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.663+0000 7f1bf4a00640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd4003a40 0x7f1bf00b64a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1be00af040 tx=0x7f1be0005660 comp rx=0 tx=0).stop 2024-09-10T21:13:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.663+0000 7f1bf4a00640 1 -- 172.21.15.16:0/4080022989 shutdown_connections 2024-09-10T21:13:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.663+0000 7f1bf4a00640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1bcc0725e0 0x7f1bcc074aa0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.663+0000 7f1bf4a00640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bf00ca330 0x7f1bf01fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.663+0000 7f1bf4a00640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1bf00c9eb0 0x7f1bf01fba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.663+0000 7f1bf4a00640 1 --2- 172.21.15.16:0/4080022989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd4003a40 0x7f1bf00b64a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.663+0000 7f1bf4a00640 1 -- 172.21.15.16:0/4080022989 >> 172.21.15.16:0/4080022989 conn(0x7f1bf00d0b60 msgr2=0x7f1bf015e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.663+0000 7f1bf4a00640 1 -- 172.21.15.16:0/4080022989 shutdown_connections 2024-09-10T21:13:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.663+0000 7f1bf4a00640 1 -- 172.21.15.16:0/4080022989 wait complete. 2024-09-10T21:13:59.696 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:13:59.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:13:59.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.843+0000 7f6914800640 1 -- 172.21.15.16:0/1116905165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690c151210 msgr2=0x7f690c0726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:59.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.843+0000 7f6914800640 1 --2- 172.21.15.16:0/1116905165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690c151210 0x7f690c0726b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f690c1517f0 tx=0x7f690001c8a0 comp rx=0 tx=0).stop 2024-09-10T21:13:59.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.843+0000 7f6914800640 1 -- 172.21.15.16:0/1116905165 shutdown_connections 2024-09-10T21:13:59.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.843+0000 7f6914800640 1 --2- 172.21.15.16:0/1116905165 >> v2:172.21.15.52:3300/0 conn(0x7f690c15e2c0 0x7f690c15a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:59.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.843+0000 7f6914800640 1 --2- 172.21.15.16:0/1116905165 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f690c072bf0 0x7f690c15dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:59.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.843+0000 7f6914800640 1 --2- 172.21.15.16:0/1116905165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690c151210 0x7f690c0726b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:59.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.843+0000 7f6914800640 1 -- 172.21.15.16:0/1116905165 >> 172.21.15.16:0/1116905165 conn(0x7f690c0b16a0 msgr2=0x7f690c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:59.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.843+0000 7f6914800640 1 -- 172.21.15.16:0/1116905165 shutdown_connections 2024-09-10T21:13:59.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.843+0000 7f6914800640 1 -- 172.21.15.16:0/1116905165 wait complete. 2024-09-10T21:13:59.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.843+0000 7f6914800640 1 Processor -- start 2024-09-10T21:13:59.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.843+0000 7f6914800640 1 -- start start 2024-09-10T21:13:59.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6914800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690c072bf0 0x7f690c146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:59.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6914800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f690c151210 0x7f690c1fba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:59.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6914800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c15e2c0 0x7f690c1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:59.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6914800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f690c15b150 con 0x7f690c072bf0 2024-09-10T21:13:59.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6914800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f690c15afd0 con 0x7f690c15e2c0 2024-09-10T21:13:59.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6914800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f690c15b2d0 con 0x7f690c151210 2024-09-10T21:13:59.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6911e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690c072bf0 0x7f690c146840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:59.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6911e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690c072bf0 0x7f690c146840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57056/0 (socket says 172.21.15.16:57056) 2024-09-10T21:13:59.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6911e00640 1 -- 172.21.15.16:0/1705758683 learned_addr learned my addr 172.21.15.16:0/1705758683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:13:59.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6911400640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f690c151210 0x7f690c1fba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:59.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6912800640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c15e2c0 0x7f690c1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:59.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6911e00640 1 -- 172.21.15.16:0/1705758683 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f690c151210 msgr2=0x7f690c1fba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:59.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6911e00640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f690c151210 0x7f690c1fba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:59.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6911e00640 1 -- 172.21.15.16:0/1705758683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c15e2c0 msgr2=0x7f690c1591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:59.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6911e00640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c15e2c0 0x7f690c1591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:59.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6911e00640 1 -- 172.21.15.16:0/1705758683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690c145aa0 con 0x7f690c072bf0 2024-09-10T21:13:59.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6911e00640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690c072bf0 0x7f690c146840 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f690001af50 tx=0x7f69000a67d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:59.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6906000640 1 -- 172.21.15.16:0/1705758683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69000bd070 con 0x7f690c072bf0 2024-09-10T21:13:59.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6906000640 1 -- 172.21.15.16:0/1705758683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6900004400 con 0x7f690c072bf0 2024-09-10T21:13:59.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6906000640 1 -- 172.21.15.16:0/1705758683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69000052d0 con 0x7f690c072bf0 2024-09-10T21:13:59.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6914800640 1 -- 172.21.15.16:0/1705758683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690c145d60 con 0x7f690c072bf0 2024-09-10T21:13:59.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6914800640 1 -- 172.21.15.16:0/1705758683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f690c159710 con 0x7f690c072bf0 2024-09-10T21:13:59.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.847+0000 7f6914800640 1 -- 172.21.15.16:0/1705758683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f690c0726b0 con 0x7f690c072bf0 2024-09-10T21:13:59.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.851+0000 7f6906000640 1 -- 172.21.15.16:0/1705758683 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6900005470 con 0x7f690c072bf0 2024-09-10T21:13:59.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.851+0000 7f6906000640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f68f00725e0 0x7f68f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:59.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.851+0000 7f6906000640 1 -- 172.21.15.16:0/1705758683 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f690012d400 con 0x7f690c072bf0 2024-09-10T21:13:59.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.851+0000 7f6911400640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f68f00725e0 0x7f68f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:59.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.851+0000 7f6911400640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f68f00725e0 0x7f68f0074aa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f690c0ccc40 tx=0x7f68fc00b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:59.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:13:59.855+0000 7f6906000640 1 -- 172.21.15.16:0/1705758683 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6900016490 con 0x7f690c072bf0 2024-09-10T21:14:00.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.067+0000 7f6914800640 1 -- 172.21.15.16:0/1705758683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f690c141b70 con 0x7f690c072bf0 2024-09-10T21:14:00.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.663+0000 7f6906000640 1 -- 172.21.15.16:0/1705758683 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) ==== 122+0+0 (secure 0 0 0) 0x7f69000f8520 con 0x7f690c072bf0 2024-09-10T21:14:00.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.667+0000 7f6914800640 1 -- 172.21.15.16:0/1705758683 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f68f00725e0 msgr2=0x7f68f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.667+0000 7f6914800640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f68f00725e0 0x7f68f0074aa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f690c0ccc40 tx=0x7f68fc00b2e0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.667+0000 7f6914800640 1 -- 172.21.15.16:0/1705758683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690c072bf0 msgr2=0x7f690c146840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.667+0000 7f6914800640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690c072bf0 0x7f690c146840 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f690001af50 tx=0x7f69000a67d0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.667+0000 7f6914800640 1 -- 172.21.15.16:0/1705758683 shutdown_connections 2024-09-10T21:14:00.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.667+0000 7f6914800640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f68f00725e0 0x7f68f0074aa0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.667+0000 7f6914800640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c15e2c0 0x7f690c1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.667+0000 7f6914800640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f690c151210 0x7f690c1fba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.667+0000 7f6914800640 1 --2- 172.21.15.16:0/1705758683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690c072bf0 0x7f690c146840 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.667+0000 7f6914800640 1 -- 172.21.15.16:0/1705758683 >> 172.21.15.16:0/1705758683 conn(0x7f690c0b16a0 msgr2=0x7f690c0af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:00.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.667+0000 7f6914800640 1 -- 172.21.15.16:0/1705758683 shutdown_connections 2024-09-10T21:14:00.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.667+0000 7f6914800640 1 -- 172.21.15.16:0/1705758683 wait complete. 2024-09-10T21:14:00.697 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:14:00.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:14:00.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.839+0000 7f027f400640 1 -- 172.21.15.16:0/1086171971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02780b3200 msgr2=0x7f02780d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.839+0000 7f027f400640 1 --2- 172.21.15.16:0/1086171971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02780b3200 0x7f02780d01c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f02780b3810 tx=0x7f026401f090 comp rx=0 tx=0).stop 2024-09-10T21:14:00.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.839+0000 7f027f400640 1 -- 172.21.15.16:0/1086171971 shutdown_connections 2024-09-10T21:14:00.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.839+0000 7f027f400640 1 --2- 172.21.15.16:0/1086171971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02780b3200 0x7f02780d01c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.839+0000 7f027f400640 1 --2- 172.21.15.16:0/1086171971 >> v2:172.21.15.52:3300/0 conn(0x7f027815f450 0x7f02780cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.839+0000 7f027f400640 1 -- 172.21.15.16:0/1086171971 >> 172.21.15.16:0/1086171971 conn(0x7f02780d1d80 msgr2=0x7f02780d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:00.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.839+0000 7f027f400640 1 -- 172.21.15.16:0/1086171971 shutdown_connections 2024-09-10T21:14:00.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.839+0000 7f027f400640 1 -- 172.21.15.16:0/1086171971 wait complete. 2024-09-10T21:14:00.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027f400640 1 Processor -- start 2024-09-10T21:14:00.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027f400640 1 -- start start 2024-09-10T21:14:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02780d0890 0x7f02780b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027f400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f027815f450 0x7f02780b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02781f6f40 0x7f027815b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027f400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02780b2e30 con 0x7f02781f6f40 2024-09-10T21:14:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027f400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f02780b2cb0 con 0x7f02780d0890 2024-09-10T21:14:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027f400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f02780b2fb0 con 0x7f027815f450 2024-09-10T21:14:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02781f6f40 0x7f027815b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02781f6f40 0x7f027815b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57070/0 (socket says 172.21.15.16:57070) 2024-09-10T21:14:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027d400640 1 -- 172.21.15.16:0/3329922650 learned_addr learned my addr 172.21.15.16:0/3329922650 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f0277e00640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f027815f450 0x7f02780b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027ca00640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02780d0890 0x7f02780b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027d400640 1 -- 172.21.15.16:0/3329922650 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f027815f450 msgr2=0x7f02780b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027d400640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f027815f450 0x7f02780b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027d400640 1 -- 172.21.15.16:0/3329922650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02780d0890 msgr2=0x7f02780b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027d400640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02780d0890 0x7f02780b7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027d400640 1 -- 172.21.15.16:0/3329922650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02780b5c50 con 0x7f02781f6f40 2024-09-10T21:14:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027ca00640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02780d0890 0x7f02780b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027d400640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02781f6f40 0x7f027815b5f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f027000c7b0 tx=0x7f027000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f0274c00640 1 -- 172.21.15.16:0/3329922650 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270017070 con 0x7f02781f6f40 2024-09-10T21:14:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f0274c00640 1 -- 172.21.15.16:0/3329922650 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0270005070 con 0x7f02781f6f40 2024-09-10T21:14:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f0274c00640 1 -- 172.21.15.16:0/3329922650 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700054c0 con 0x7f02781f6f40 2024-09-10T21:14:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027f400640 1 -- 172.21.15.16:0/3329922650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02780b5970 con 0x7f02781f6f40 2024-09-10T21:14:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.843+0000 7f027f400640 1 -- 172.21.15.16:0/3329922650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02781f6740 con 0x7f02781f6f40 2024-09-10T21:14:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.847+0000 7f027f400640 1 -- 172.21.15.16:0/3329922650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02780b1e50 con 0x7f02781f6f40 2024-09-10T21:14:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.847+0000 7f0274c00640 1 -- 172.21.15.16:0/3329922650 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0270005660 con 0x7f02781f6f40 2024-09-10T21:14:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.847+0000 7f0274c00640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0258072610 0x7f0258074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.847+0000 7f0274c00640 1 -- 172.21.15.16:0/3329922650 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f0270092760 con 0x7f02781f6f40 2024-09-10T21:14:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.847+0000 7f027ca00640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0258072610 0x7f0258074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.847+0000 7f027ca00640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0258072610 0x7f0258074ad0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f026c006fd0 tx=0x7f026c008040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:00.851+0000 7f0274c00640 1 -- 172.21.15.16:0/3329922650 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f02780b5c50 con 0x7f02781f6f40 2024-09-10T21:14:01.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.063+0000 7f027f400640 1 -- 172.21.15.16:0/3329922650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0278074ba0 con 0x7f02781f6f40 2024-09-10T21:14:01.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.063+0000 7f0274c00640 1 -- 172.21.15.16:0/3329922650 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5222 (secure 0 0 0) 0x7f027005d750 con 0x7f02781f6f40 2024-09-10T21:14:01.070 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:01.071 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"btime":"2024-09-10T21:14:00:666666+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4199133238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4199133238},{"type":"v1","addr":"172.21.15.52:6835","nonce":4199133238}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2246088651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2246088651},{"type":"v1","addr":"172.21.15.16:6836","nonce":2246088651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1753951486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1753951486},{"type":"v1","addr":"172.21.15.16:6837","nonce":1753951486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:57.054843+0000","modified":"2024-09-10T21:14:00.072852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6276":{"gid":6276,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/943286634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":943286634},{"type":"v1","addr":"172.21.15.52:6834","nonce":943286634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6276,"qdb_cluster":[6276]},"id":6}]} 2024-09-10T21:14:01.071 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2024-09-10T21:14:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.063+0000 7f027f400640 1 -- 172.21.15.16:0/3329922650 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0258072610 msgr2=0x7f0258074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.063+0000 7f027f400640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0258072610 0x7f0258074ad0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f026c006fd0 tx=0x7f026c008040 comp rx=0 tx=0).stop 2024-09-10T21:14:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.063+0000 7f027f400640 1 -- 172.21.15.16:0/3329922650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02781f6f40 msgr2=0x7f027815b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.063+0000 7f027f400640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02781f6f40 0x7f027815b5f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f027000c7b0 tx=0x7f027000cc70 comp rx=0 tx=0).stop 2024-09-10T21:14:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.063+0000 7f027f400640 1 -- 172.21.15.16:0/3329922650 shutdown_connections 2024-09-10T21:14:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.063+0000 7f027f400640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0258072610 0x7f0258074ad0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.063+0000 7f027f400640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02781f6f40 0x7f027815b5f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.063+0000 7f027f400640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f027815f450 0x7f02780b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.063+0000 7f027f400640 1 --2- 172.21.15.16:0/3329922650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02780d0890 0x7f02780b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.063+0000 7f027f400640 1 -- 172.21.15.16:0/3329922650 >> 172.21.15.16:0/3329922650 conn(0x7f02780d1d80 msgr2=0x7f02780b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.067+0000 7f027f400640 1 -- 172.21.15.16:0/3329922650 shutdown_connections 2024-09-10T21:14:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.067+0000 7f027f400640 1 -- 172.21.15.16:0/3329922650 wait complete. 2024-09-10T21:14:01.093 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:14:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 -- 172.21.15.16:0/2445807793 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f281599d0 msgr2=0x7f5f0c003a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 --2- 172.21.15.16:0/2445807793 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f281599d0 0x7f5f0c003a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009b60 tx=0x7f5f1801d2b0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f27400640 1 -- 172.21.15.16:0/2445807793 reap_dead start 2024-09-10T21:14:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 -- 172.21.15.16:0/2445807793 shutdown_connections 2024-09-10T21:14:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 -- 172.21.15.16:0/2445807793 >> 172.21.15.16:0/2445807793 conn(0x7f5f280cfb20 msgr2=0x7f5f280cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 -- 172.21.15.16:0/2445807793 shutdown_connections 2024-09-10T21:14:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 -- 172.21.15.16:0/2445807793 wait complete. 2024-09-10T21:14:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 Processor -- start 2024-09-10T21:14:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 -- start start 2024-09-10T21:14:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f281599d0 0x7f5f28158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f28159390 0x7f5f280c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f281f7d40 0x7f5f280cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2815fbc0 con 0x7f5f281599d0 2024-09-10T21:14:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2815fa40 con 0x7f5f28159390 2024-09-10T21:14:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f2d600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5f28073030 con 0x7f5f281f7d40 2024-09-10T21:14:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f26a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f281599d0 0x7f5f28158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f26a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f281599d0 0x7f5f28158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57092/0 (socket says 172.21.15.16:57092) 2024-09-10T21:14:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f26a00640 1 -- 172.21.15.16:0/271132858 learned_addr learned my addr 172.21.15.16:0/271132858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f27400640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f281f7d40 0x7f5f280cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.235+0000 7f5f1e000640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f28159390 0x7f5f280c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f26a00640 1 -- 172.21.15.16:0/271132858 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f281f7d40 msgr2=0x7f5f280cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f26a00640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f281f7d40 0x7f5f280cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f26a00640 1 -- 172.21.15.16:0/271132858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f28159390 msgr2=0x7f5f280c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f26a00640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f28159390 0x7f5f280c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f26a00640 1 -- 172.21.15.16:0/271132858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f280ae180 con 0x7f5f281599d0 2024-09-10T21:14:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f27400640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f281f7d40 0x7f5f280cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f1e000640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f28159390 0x7f5f280c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f26a00640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f281599d0 0x7f5f28158d10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5f180af040 tx=0x7f5f1801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f1f400640 1 -- 172.21.15.16:0/271132858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f180b1040 con 0x7f5f281599d0 2024-09-10T21:14:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f1f400640 1 -- 172.21.15.16:0/271132858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f180bb070 con 0x7f5f281599d0 2024-09-10T21:14:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f2d600640 1 -- 172.21.15.16:0/271132858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f280ae460 con 0x7f5f281599d0 2024-09-10T21:14:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f1f400640 1 -- 172.21.15.16:0/271132858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18004290 con 0x7f5f281599d0 2024-09-10T21:14:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f2d600640 1 -- 172.21.15.16:0/271132858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f281f7820 con 0x7f5f281599d0 2024-09-10T21:14:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f2d600640 1 -- 172.21.15.16:0/271132858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f280ca170 con 0x7f5f281599d0 2024-09-10T21:14:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f1f400640 1 -- 172.21.15.16:0/271132858 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5f180046d0 con 0x7f5f281599d0 2024-09-10T21:14:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.239+0000 7f5f1f400640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f040725e0 0x7f5f04074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.243+0000 7f5f1f400640 1 -- 172.21.15.16:0/271132858 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f5f1812e1a0 con 0x7f5f281599d0 2024-09-10T21:14:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.243+0000 7f5f1e000640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f040725e0 0x7f5f04074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.243+0000 7f5f1e000640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f040725e0 0x7f5f04074aa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5f280ca450 tx=0x7f5f100091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:01.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.247+0000 7f5f1f400640 1 -- 172.21.15.16:0/271132858 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f18016fc0 con 0x7f5f281599d0 2024-09-10T21:14:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.419+0000 7f5f2d600640 1 -- 172.21.15.16:0/271132858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f281f75f0 con 0x7f5f281599d0 2024-09-10T21:14:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.419+0000 7f5f1f400640 1 -- 172.21.15.16:0/271132858 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16684 (secure 0 0 0) 0x7f5f180f9870 con 0x7f5f281599d0 2024-09-10T21:14:01.429 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:01.429 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:13:58.647363+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:13:54.810241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T21:13:56.024501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:14:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.423+0000 7f5f2d600640 1 -- 172.21.15.16:0/271132858 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f040725e0 msgr2=0x7f5f04074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.423+0000 7f5f2d600640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f040725e0 0x7f5f04074aa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5f280ca450 tx=0x7f5f100091e0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.423+0000 7f5f2d600640 1 -- 172.21.15.16:0/271132858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f281599d0 msgr2=0x7f5f28158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.423+0000 7f5f2d600640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f281599d0 0x7f5f28158d10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5f180af040 tx=0x7f5f1801baf0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.423+0000 7f5f2d600640 1 -- 172.21.15.16:0/271132858 shutdown_connections 2024-09-10T21:14:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.423+0000 7f5f2d600640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f040725e0 0x7f5f04074aa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.423+0000 7f5f2d600640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f281f7d40 0x7f5f280cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.423+0000 7f5f2d600640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f28159390 0x7f5f280c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.423+0000 7f5f2d600640 1 --2- 172.21.15.16:0/271132858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f281599d0 0x7f5f28158d10 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.423+0000 7f5f2d600640 1 -- 172.21.15.16:0/271132858 >> 172.21.15.16:0/271132858 conn(0x7f5f280cfb20 msgr2=0x7f5f2815ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:01.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.423+0000 7f5f2d600640 1 -- 172.21.15.16:0/271132858 shutdown_connections 2024-09-10T21:14:01.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.423+0000 7f5f2d600640 1 -- 172.21.15.16:0/271132858 wait complete. 2024-09-10T21:14:01.450 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:14:01.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:14:01.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 -- 172.21.15.16:0/1659048096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f314c15bb40 msgr2=0x7f314c06cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:01.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 --2- 172.21.15.16:0/1659048096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f314c15bb40 0x7f314c06cbe0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3140009840 tx=0x7f314001c760 comp rx=0 tx=0).stop 2024-09-10T21:14:01.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 -- 172.21.15.16:0/1659048096 shutdown_connections 2024-09-10T21:14:01.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 --2- 172.21.15.16:0/1659048096 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f314c0ccea0 0x7f314c0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 --2- 172.21.15.16:0/1659048096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f314c15bb40 0x7f314c06cbe0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 -- 172.21.15.16:0/1659048096 >> 172.21.15.16:0/1659048096 conn(0x7f314c0cd6b0 msgr2=0x7f314c0cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:01.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 -- 172.21.15.16:0/1659048096 shutdown_connections 2024-09-10T21:14:01.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 -- 172.21.15.16:0/1659048096 wait complete. 2024-09-10T21:14:01.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 Processor -- start 2024-09-10T21:14:01.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 -- start start 2024-09-10T21:14:01.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f314c06d120 0x7f314c0b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:01.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f314c0ccea0 0x7f314c0c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:01.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f314c15bb40 0x7f314c0c3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:01.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f314c168d70 con 0x7f314c15bb40 2024-09-10T21:14:01.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f314c0cc4e0 con 0x7f314c06d120 2024-09-10T21:14:01.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3153e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f314c168ef0 con 0x7f314c0ccea0 2024-09-10T21:14:01.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3151e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f314c15bb40 0x7f314c0c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:01.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3151e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f314c15bb40 0x7f314c0c3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57106/0 (socket says 172.21.15.16:57106) 2024-09-10T21:14:01.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3151400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f314c06d120 0x7f314c0b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:01.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3151e00640 1 -- 172.21.15.16:0/3299956316 learned_addr learned my addr 172.21.15.16:0/3299956316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:01.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.591+0000 7f3150a00640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f314c0ccea0 0x7f314c0c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:01.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3151e00640 1 -- 172.21.15.16:0/3299956316 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f314c0ccea0 msgr2=0x7f314c0c31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:01.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3151e00640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f314c0ccea0 0x7f314c0c31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3151e00640 1 -- 172.21.15.16:0/3299956316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f314c06d120 msgr2=0x7f314c0b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:01.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3151e00640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f314c06d120 0x7f314c0b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3151e00640 1 -- 172.21.15.16:0/3299956316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314c165890 con 0x7f314c15bb40 2024-09-10T21:14:01.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3151400640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f314c06d120 0x7f314c0b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:01.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3151e00640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f314c15bb40 0x7f314c0c3700 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f314400ef60 tx=0x7f314400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3149600640 1 -- 172.21.15.16:0/3299956316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144004b30 con 0x7f314c15bb40 2024-09-10T21:14:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3149600640 1 -- 172.21.15.16:0/3299956316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3144004dc0 con 0x7f314c15bb40 2024-09-10T21:14:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3153e00640 1 -- 172.21.15.16:0/3299956316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314c165b70 con 0x7f314c15bb40 2024-09-10T21:14:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3149600640 1 -- 172.21.15.16:0/3299956316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314400b7d0 con 0x7f314c15bb40 2024-09-10T21:14:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3153e00640 1 -- 172.21.15.16:0/3299956316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f314c166130 con 0x7f314c15bb40 2024-09-10T21:14:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3153e00640 1 -- 172.21.15.16:0/3299956316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314c166330 con 0x7f314c15bb40 2024-09-10T21:14:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.595+0000 7f3149600640 1 -- 172.21.15.16:0/3299956316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f314400b040 con 0x7f314c15bb40 2024-09-10T21:14:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.599+0000 7f3149600640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f312c0725e0 0x7f312c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.599+0000 7f3149600640 1 -- 172.21.15.16:0/3299956316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f3144093800 con 0x7f314c15bb40 2024-09-10T21:14:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.599+0000 7f3151400640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f312c0725e0 0x7f312c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.599+0000 7f3151400640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f312c0725e0 0x7f312c074aa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f31400af000 tx=0x7f314001afa0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:01.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.599+0000 7f3149600640 1 -- 172.21.15.16:0/3299956316 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f314405e7f0 con 0x7f314c15bb40 2024-09-10T21:14:01.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.771+0000 7f3153e00640 1 -- 172.21.15.16:0/3299956316 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f314c0d0850 con 0x7f312c0725e0 2024-09-10T21:14:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.779+0000 7f3149600640 1 -- 172.21.15.16:0/3299956316 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7f31400a5710 con 0x7f312c0725e0 2024-09-10T21:14:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:01.789 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:14:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.783+0000 7f3153e00640 1 -- 172.21.15.16:0/3299956316 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f312c0725e0 msgr2=0x7f312c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.783+0000 7f3153e00640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f312c0725e0 0x7f312c074aa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f31400af000 tx=0x7f314001afa0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.783+0000 7f3153e00640 1 -- 172.21.15.16:0/3299956316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f314c15bb40 msgr2=0x7f314c0c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.783+0000 7f3153e00640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f314c15bb40 0x7f314c0c3700 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f314400ef60 tx=0x7f314400ef90 comp rx=0 tx=0).stop 2024-09-10T21:14:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.783+0000 7f3153e00640 1 -- 172.21.15.16:0/3299956316 shutdown_connections 2024-09-10T21:14:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.783+0000 7f3153e00640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f312c0725e0 0x7f312c074aa0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.783+0000 7f3153e00640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f314c15bb40 0x7f314c0c3700 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.783+0000 7f3153e00640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f314c0ccea0 0x7f314c0c31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.783+0000 7f3153e00640 1 --2- 172.21.15.16:0/3299956316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f314c06d120 0x7f314c0b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.783+0000 7f3153e00640 1 -- 172.21.15.16:0/3299956316 >> 172.21.15.16:0/3299956316 conn(0x7f314c0cd6b0 msgr2=0x7f314c0b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.783+0000 7f3153e00640 1 -- 172.21.15.16:0/3299956316 shutdown_connections 2024-09-10T21:14:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.783+0000 7f3153e00640 1 -- 172.21.15.16:0/3299956316 wait complete. 2024-09-10T21:14:01.808 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-10T21:14:01.503825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270876,"kb_used_data":8860,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754703844,"statfs":{"total":773094113280,"available":772816736256,"internally_reserved":0,"allocated":9072640,"data_stored":4493099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.869327"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.679268+0000","last_change":"2024-09-10T21:13:57.675647+0000","last_active":"2024-09-10T21:13:57.679268+0000","last_peered":"2024-09-10T21:13:57.679268+0000","last_clean":"2024-09-10T21:13:57.679268+0000","last_became_active":"2024-09-10T21:13:57.675067+0000","last_became_peered":"2024-09-10T21:13:57.675067+0000","last_unstale":"2024-09-10T21:13:57.679268+0000","last_undegraded":"2024-09-10T21:13:57.679268+0000","last_fullsized":"2024-09-10T21:13:57.679268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:08.229499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652856+0000","last_change":"2024-09-10T21:13:57.666078+0000","last_active":"2024-09-10T21:13:58.652856+0000","last_peered":"2024-09-10T21:13:58.652856+0000","last_clean":"2024-09-10T21:13:58.652856+0000","last_became_active":"2024-09-10T21:13:57.665762+0000","last_became_peered":"2024-09-10T21:13:57.665762+0000","last_unstale":"2024-09-10T21:13:58.652856+0000","last_undegraded":"2024-09-10T21:13:58.652856+0000","last_fullsized":"2024-09-10T21:13:58.652856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:06.098117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652861+0000","last_change":"2024-09-10T21:13:57.665470+0000","last_active":"2024-09-10T21:13:58.652861+0000","last_peered":"2024-09-10T21:13:58.652861+0000","last_clean":"2024-09-10T21:13:58.652861+0000","last_became_active":"2024-09-10T21:13:57.665120+0000","last_became_peered":"2024-09-10T21:13:57.665120+0000","last_unstale":"2024-09-10T21:13:58.652861+0000","last_undegraded":"2024-09-10T21:13:58.652861+0000","last_fullsized":"2024-09-10T21:13:58.652861+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:56.313169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653406+0000","last_change":"2024-09-10T21:13:57.661056+0000","last_active":"2024-09-10T21:13:58.653406+0000","last_peered":"2024-09-10T21:13:58.653406+0000","last_clean":"2024-09-10T21:13:58.653406+0000","last_became_active":"2024-09-10T21:13:57.659861+0000","last_became_peered":"2024-09-10T21:13:57.659861+0000","last_unstale":"2024-09-10T21:13:58.653406+0000","last_undegraded":"2024-09-10T21:13:58.653406+0000","last_fullsized":"2024-09-10T21:13:58.653406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:31.700508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.690903+0000","last_change":"2024-09-10T21:13:57.681348+0000","last_active":"2024-09-10T21:13:58.690903+0000","last_peered":"2024-09-10T21:13:58.690903+0000","last_clean":"2024-09-10T21:13:58.690903+0000","last_became_active":"2024-09-10T21:13:57.680908+0000","last_became_peered":"2024-09-10T21:13:57.680908+0000","last_unstale":"2024-09-10T21:13:58.690903+0000","last_undegraded":"2024-09-10T21:13:58.690903+0000","last_fullsized":"2024-09-10T21:13:58.690903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:22.099540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651691+0000","last_change":"2024-09-10T21:13:57.659245+0000","last_active":"2024-09-10T21:13:58.651691+0000","last_peered":"2024-09-10T21:13:58.651691+0000","last_clean":"2024-09-10T21:13:58.651691+0000","last_became_active":"2024-09-10T21:13:57.658283+0000","last_became_peered":"2024-09-10T21:13:57.658283+0000","last_unstale":"2024-09-10T21:13:58.651691+0000","last_undegraded":"2024-09-10T21:13:58.651691+0000","last_fullsized":"2024-09-10T21:13:58.651691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:42.562833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652061+0000","last_change":"2024-09-10T21:13:57.665271+0000","last_active":"2024-09-10T21:13:58.652061+0000","last_peered":"2024-09-10T21:13:58.652061+0000","last_clean":"2024-09-10T21:13:58.652061+0000","last_became_active":"2024-09-10T21:13:57.664330+0000","last_became_peered":"2024-09-10T21:13:57.664330+0000","last_unstale":"2024-09-10T21:13:58.652061+0000","last_undegraded":"2024-09-10T21:13:58.652061+0000","last_fullsized":"2024-09-10T21:13:58.652061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:44.586271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654091+0000","last_change":"2024-09-10T21:13:57.677047+0000","last_active":"2024-09-10T21:13:58.654091+0000","last_peered":"2024-09-10T21:13:58.654091+0000","last_clean":"2024-09-10T21:13:58.654091+0000","last_became_active":"2024-09-10T21:13:57.676725+0000","last_became_peered":"2024-09-10T21:13:57.676725+0000","last_unstale":"2024-09-10T21:13:58.654091+0000","last_undegraded":"2024-09-10T21:13:58.654091+0000","last_fullsized":"2024-09-10T21:13:58.654091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:19.669090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.679329+0000","last_change":"2024-09-10T21:13:57.676001+0000","last_active":"2024-09-10T21:13:57.679329+0000","last_peered":"2024-09-10T21:13:57.679329+0000","last_clean":"2024-09-10T21:13:57.679329+0000","last_became_active":"2024-09-10T21:13:57.675597+0000","last_became_peered":"2024-09-10T21:13:57.675597+0000","last_unstale":"2024-09-10T21:13:57.679329+0000","last_undegraded":"2024-09-10T21:13:57.679329+0000","last_fullsized":"2024-09-10T21:13:57.679329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:01.532812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653273+0000","last_change":"2024-09-10T21:13:57.657599+0000","last_active":"2024-09-10T21:13:58.653273+0000","last_peered":"2024-09-10T21:13:58.653273+0000","last_clean":"2024-09-10T21:13:58.653273+0000","last_became_active":"2024-09-10T21:13:57.657327+0000","last_became_peered":"2024-09-10T21:13:57.657327+0000","last_unstale":"2024-09-10T21:13:58.653273+0000","last_undegraded":"2024-09-10T21:13:58.653273+0000","last_fullsized":"2024-09-10T21:13:58.653273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:25.230878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674453+0000","last_change":"2024-09-10T21:13:57.665075+0000","last_active":"2024-09-10T21:13:58.674453+0000","last_peered":"2024-09-10T21:13:58.674453+0000","last_clean":"2024-09-10T21:13:58.674453+0000","last_became_active":"2024-09-10T21:13:57.664732+0000","last_became_peered":"2024-09-10T21:13:57.664732+0000","last_unstale":"2024-09-10T21:13:58.674453+0000","last_undegraded":"2024-09-10T21:13:58.674453+0000","last_fullsized":"2024-09-10T21:13:58.674453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:51:18.181610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654237+0000","last_change":"2024-09-10T21:13:57.682795+0000","last_active":"2024-09-10T21:13:58.654237+0000","last_peered":"2024-09-10T21:13:58.654237+0000","last_clean":"2024-09-10T21:13:58.654237+0000","last_became_active":"2024-09-10T21:13:57.682524+0000","last_became_peered":"2024-09-10T21:13:57.682524+0000","last_unstale":"2024-09-10T21:13:58.654237+0000","last_undegraded":"2024-09-10T21:13:58.654237+0000","last_fullsized":"2024-09-10T21:13:58.654237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:56.643637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.690924+0000","last_change":"2024-09-10T21:13:57.665092+0000","last_active":"2024-09-10T21:13:58.690924+0000","last_peered":"2024-09-10T21:13:58.690924+0000","last_clean":"2024-09-10T21:13:58.690924+0000","last_became_active":"2024-09-10T21:13:57.664814+0000","last_became_peered":"2024-09-10T21:13:57.664814+0000","last_unstale":"2024-09-10T21:13:58.690924+0000","last_undegraded":"2024-09-10T21:13:58.690924+0000","last_fullsized":"2024-09-10T21:13:58.690924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:36.165681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.683678+0000","last_change":"2024-09-10T21:13:57.677622+0000","last_active":"2024-09-10T21:13:57.683678+0000","last_peered":"2024-09-10T21:13:57.683678+0000","last_clean":"2024-09-10T21:13:57.683678+0000","last_became_active":"2024-09-10T21:13:57.677036+0000","last_became_peered":"2024-09-10T21:13:57.677036+0000","last_unstale":"2024-09-10T21:13:57.683678+0000","last_undegraded":"2024-09-10T21:13:57.683678+0000","last_fullsized":"2024-09-10T21:13:57.683678+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:08.229499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652300+0000","last_change":"2024-09-10T21:13:57.655747+0000","last_active":"2024-09-10T21:13:58.652300+0000","last_peered":"2024-09-10T21:13:58.652300+0000","last_clean":"2024-09-10T21:13:58.652300+0000","last_became_active":"2024-09-10T21:13:57.655468+0000","last_became_peered":"2024-09-10T21:13:57.655468+0000","last_unstale":"2024-09-10T21:13:58.652300+0000","last_undegraded":"2024-09-10T21:13:58.652300+0000","last_fullsized":"2024-09-10T21:13:58.652300+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:58.927960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651474+0000","last_change":"2024-09-10T21:13:57.665659+0000","last_active":"2024-09-10T21:13:58.651474+0000","last_peered":"2024-09-10T21:13:58.651474+0000","last_clean":"2024-09-10T21:13:58.651474+0000","last_became_active":"2024-09-10T21:13:57.665302+0000","last_became_peered":"2024-09-10T21:13:57.665302+0000","last_unstale":"2024-09-10T21:13:58.651474+0000","last_undegraded":"2024-09-10T21:13:58.651474+0000","last_fullsized":"2024-09-10T21:13:58.651474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:22.405349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654568+0000","last_change":"2024-09-10T21:13:57.688758+0000","last_active":"2024-09-10T21:13:58.654568+0000","last_peered":"2024-09-10T21:13:58.654568+0000","last_clean":"2024-09-10T21:13:58.654568+0000","last_became_active":"2024-09-10T21:13:57.688387+0000","last_became_peered":"2024-09-10T21:13:57.688387+0000","last_unstale":"2024-09-10T21:13:58.654568+0000","last_undegraded":"2024-09-10T21:13:58.654568+0000","last_fullsized":"2024-09-10T21:13:58.654568+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:50.075360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674475+0000","last_change":"2024-09-10T21:13:57.686565+0000","last_active":"2024-09-10T21:13:58.674475+0000","last_peered":"2024-09-10T21:13:58.674475+0000","last_clean":"2024-09-10T21:13:58.674475+0000","last_became_active":"2024-09-10T21:13:57.681049+0000","last_became_peered":"2024-09-10T21:13:57.681049+0000","last_unstale":"2024-09-10T21:13:58.674475+0000","last_undegraded":"2024-09-10T21:13:58.674475+0000","last_fullsized":"2024-09-10T21:13:58.674475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:50.388234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.682383+0000","last_change":"2024-09-10T21:13:57.677237+0000","last_active":"2024-09-10T21:13:57.682383+0000","last_peered":"2024-09-10T21:13:57.682383+0000","last_clean":"2024-09-10T21:13:57.682383+0000","last_became_active":"2024-09-10T21:13:57.676855+0000","last_became_peered":"2024-09-10T21:13:57.676855+0000","last_unstale":"2024-09-10T21:13:57.682383+0000","last_undegraded":"2024-09-10T21:13:57.682383+0000","last_fullsized":"2024-09-10T21:13:57.682383+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:15.099468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.680569+0000","last_change":"2024-09-10T21:13:57.665394+0000","last_active":"2024-09-10T21:13:57.680569+0000","last_peered":"2024-09-10T21:13:57.680569+0000","last_clean":"2024-09-10T21:13:57.680569+0000","last_became_active":"2024-09-10T21:13:57.665126+0000","last_became_peered":"2024-09-10T21:13:57.665126+0000","last_unstale":"2024-09-10T21:13:57.680569+0000","last_undegraded":"2024-09-10T21:13:57.680569+0000","last_fullsized":"2024-09-10T21:13:57.680569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:31.157608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653101+0000","last_change":"2024-09-10T21:13:57.666294+0000","last_active":"2024-09-10T21:13:58.653101+0000","last_peered":"2024-09-10T21:13:58.653101+0000","last_clean":"2024-09-10T21:13:58.653101+0000","last_became_active":"2024-09-10T21:13:57.664767+0000","last_became_peered":"2024-09-10T21:13:57.664767+0000","last_unstale":"2024-09-10T21:13:58.653101+0000","last_undegraded":"2024-09-10T21:13:58.653101+0000","last_fullsized":"2024-09-10T21:13:58.653101+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:13.733070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.680505+0000","last_change":"2024-09-10T21:13:57.674589+0000","last_active":"2024-09-10T21:13:57.680505+0000","last_peered":"2024-09-10T21:13:57.680505+0000","last_clean":"2024-09-10T21:13:57.680505+0000","last_became_active":"2024-09-10T21:13:57.674324+0000","last_became_peered":"2024-09-10T21:13:57.674324+0000","last_unstale":"2024-09-10T21:13:57.680505+0000","last_undegraded":"2024-09-10T21:13:57.680505+0000","last_fullsized":"2024-09-10T21:13:57.680505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:41.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654071+0000","last_change":"2024-09-10T21:13:57.688753+0000","last_active":"2024-09-10T21:13:58.654071+0000","last_peered":"2024-09-10T21:13:58.654071+0000","last_clean":"2024-09-10T21:13:58.654071+0000","last_became_active":"2024-09-10T21:13:57.688416+0000","last_became_peered":"2024-09-10T21:13:57.688416+0000","last_unstale":"2024-09-10T21:13:58.654071+0000","last_undegraded":"2024-09-10T21:13:58.654071+0000","last_fullsized":"2024-09-10T21:13:58.654071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:03.676758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674071+0000","last_change":"2024-09-10T21:13:57.670146+0000","last_active":"2024-09-10T21:13:58.674071+0000","last_peered":"2024-09-10T21:13:58.674071+0000","last_clean":"2024-09-10T21:13:58.674071+0000","last_became_active":"2024-09-10T21:13:57.669808+0000","last_became_peered":"2024-09-10T21:13:57.669808+0000","last_unstale":"2024-09-10T21:13:58.674071+0000","last_undegraded":"2024-09-10T21:13:58.674071+0000","last_fullsized":"2024-09-10T21:13:58.674071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:45.864745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674573+0000","last_change":"2024-09-10T21:13:57.675388+0000","last_active":"2024-09-10T21:13:58.674573+0000","last_peered":"2024-09-10T21:13:58.674573+0000","last_clean":"2024-09-10T21:13:58.674573+0000","last_became_active":"2024-09-10T21:13:57.675081+0000","last_became_peered":"2024-09-10T21:13:57.675081+0000","last_unstale":"2024-09-10T21:13:58.674573+0000","last_undegraded":"2024-09-10T21:13:58.674573+0000","last_fullsized":"2024-09-10T21:13:58.674573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:43.169704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691792+0000","last_change":"2024-09-10T21:13:57.670121+0000","last_active":"2024-09-10T21:13:58.691792+0000","last_peered":"2024-09-10T21:13:58.691792+0000","last_clean":"2024-09-10T21:13:58.691792+0000","last_became_active":"2024-09-10T21:13:57.669843+0000","last_became_peered":"2024-09-10T21:13:57.669843+0000","last_unstale":"2024-09-10T21:13:58.691792+0000","last_undegraded":"2024-09-10T21:13:58.691792+0000","last_fullsized":"2024-09-10T21:13:58.691792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:35.478067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691543+0000","last_change":"2024-09-10T21:13:57.664843+0000","last_active":"2024-09-10T21:13:58.691543+0000","last_peered":"2024-09-10T21:13:58.691543+0000","last_clean":"2024-09-10T21:13:58.691543+0000","last_became_active":"2024-09-10T21:13:57.664513+0000","last_became_peered":"2024-09-10T21:13:57.664513+0000","last_unstale":"2024-09-10T21:13:58.691543+0000","last_undegraded":"2024-09-10T21:13:58.691543+0000","last_fullsized":"2024-09-10T21:13:58.691543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:10.771217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.677589+0000","last_change":"2024-09-10T21:13:57.673552+0000","last_active":"2024-09-10T21:13:57.677589+0000","last_peered":"2024-09-10T21:13:57.677589+0000","last_clean":"2024-09-10T21:13:57.677589+0000","last_became_active":"2024-09-10T21:13:57.673319+0000","last_became_peered":"2024-09-10T21:13:57.673319+0000","last_unstale":"2024-09-10T21:13:57.677589+0000","last_undegraded":"2024-09-10T21:13:57.677589+0000","last_fullsized":"2024-09-10T21:13:57.677589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:15.099468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653102+0000","last_change":"2024-09-10T21:13:57.669502+0000","last_active":"2024-09-10T21:13:58.653102+0000","last_peered":"2024-09-10T21:13:58.653102+0000","last_clean":"2024-09-10T21:13:58.653102+0000","last_became_active":"2024-09-10T21:13:57.662958+0000","last_became_peered":"2024-09-10T21:13:57.662958+0000","last_unstale":"2024-09-10T21:13:58.653102+0000","last_undegraded":"2024-09-10T21:13:58.653102+0000","last_fullsized":"2024-09-10T21:13:58.653102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:54.958432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.683653+0000","last_change":"2024-09-10T21:13:57.677331+0000","last_active":"2024-09-10T21:13:57.683653+0000","last_peered":"2024-09-10T21:13:57.683653+0000","last_clean":"2024-09-10T21:13:57.683653+0000","last_became_active":"2024-09-10T21:13:57.676705+0000","last_became_peered":"2024-09-10T21:13:57.676705+0000","last_unstale":"2024-09-10T21:13:57.683653+0000","last_undegraded":"2024-09-10T21:13:57.683653+0000","last_fullsized":"2024-09-10T21:13:57.683653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:47.869632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651612+0000","last_change":"2024-09-10T21:13:57.656607+0000","last_active":"2024-09-10T21:13:58.651612+0000","last_peered":"2024-09-10T21:13:58.651612+0000","last_clean":"2024-09-10T21:13:58.651612+0000","last_became_active":"2024-09-10T21:13:57.656304+0000","last_became_peered":"2024-09-10T21:13:57.656304+0000","last_unstale":"2024-09-10T21:13:58.651612+0000","last_undegraded":"2024-09-10T21:13:58.651612+0000","last_fullsized":"2024-09-10T21:13:58.651612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:22.505181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651475+0000","last_change":"2024-09-10T21:13:57.656632+0000","last_active":"2024-09-10T21:13:58.651475+0000","last_peered":"2024-09-10T21:13:58.651475+0000","last_clean":"2024-09-10T21:13:58.651475+0000","last_became_active":"2024-09-10T21:13:57.656351+0000","last_became_peered":"2024-09-10T21:13:57.656351+0000","last_unstale":"2024-09-10T21:13:58.651475+0000","last_undegraded":"2024-09-10T21:13:58.651475+0000","last_fullsized":"2024-09-10T21:13:58.651475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:49.208104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654355+0000","last_change":"2024-09-10T21:13:57.674210+0000","last_active":"2024-09-10T21:13:58.654355+0000","last_peered":"2024-09-10T21:13:58.654355+0000","last_clean":"2024-09-10T21:13:58.654355+0000","last_became_active":"2024-09-10T21:13:57.673931+0000","last_became_peered":"2024-09-10T21:13:57.673931+0000","last_unstale":"2024-09-10T21:13:58.654355+0000","last_undegraded":"2024-09-10T21:13:58.654355+0000","last_fullsized":"2024-09-10T21:13:58.654355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:42.562833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691757+0000","last_change":"2024-09-10T21:13:56.661701+0000","last_active":"2024-09-10T21:13:58.691757+0000","last_peered":"2024-09-10T21:13:58.691757+0000","last_clean":"2024-09-10T21:13:58.691757+0000","last_became_active":"2024-09-10T21:13:56.661429+0000","last_became_peered":"2024-09-10T21:13:56.661429+0000","last_unstale":"2024-09-10T21:13:58.691757+0000","last_undegraded":"2024-09-10T21:13:58.691757+0000","last_fullsized":"2024-09-10T21:13:58.691757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:19.079723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.679292+0000","last_change":"2024-09-10T21:13:57.675904+0000","last_active":"2024-09-10T21:13:57.679292+0000","last_peered":"2024-09-10T21:13:57.679292+0000","last_clean":"2024-09-10T21:13:57.679292+0000","last_became_active":"2024-09-10T21:13:57.675381+0000","last_became_peered":"2024-09-10T21:13:57.675381+0000","last_unstale":"2024-09-10T21:13:57.679292+0000","last_undegraded":"2024-09-10T21:13:57.679292+0000","last_fullsized":"2024-09-10T21:13:57.679292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:03.773571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652702+0000","last_change":"2024-09-10T21:13:56.649469+0000","last_active":"2024-09-10T21:13:58.652702+0000","last_peered":"2024-09-10T21:13:58.652702+0000","last_clean":"2024-09-10T21:13:58.652702+0000","last_became_active":"2024-09-10T21:13:56.649117+0000","last_became_peered":"2024-09-10T21:13:56.649117+0000","last_unstale":"2024-09-10T21:13:58.652702+0000","last_undegraded":"2024-09-10T21:13:58.652702+0000","last_fullsized":"2024-09-10T21:13:58.652702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:13.367739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654344+0000","last_change":"2024-09-10T21:13:57.672513+0000","last_active":"2024-09-10T21:13:58.654344+0000","last_peered":"2024-09-10T21:13:58.654344+0000","last_clean":"2024-09-10T21:13:58.654344+0000","last_became_active":"2024-09-10T21:13:57.672246+0000","last_became_peered":"2024-09-10T21:13:57.672246+0000","last_unstale":"2024-09-10T21:13:58.654344+0000","last_undegraded":"2024-09-10T21:13:58.654344+0000","last_fullsized":"2024-09-10T21:13:58.654344+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:22.505181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674367+0000","last_change":"2024-09-10T21:13:56.668321+0000","last_active":"2024-09-10T21:13:58.674367+0000","last_peered":"2024-09-10T21:13:58.674367+0000","last_clean":"2024-09-10T21:13:58.674367+0000","last_became_active":"2024-09-10T21:13:56.668033+0000","last_became_peered":"2024-09-10T21:13:56.668033+0000","last_unstale":"2024-09-10T21:13:58.674367+0000","last_undegraded":"2024-09-10T21:13:58.674367+0000","last_fullsized":"2024-09-10T21:13:58.674367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:52.899802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.677439+0000","last_change":"2024-09-10T21:13:57.673335+0000","last_active":"2024-09-10T21:13:57.677439+0000","last_peered":"2024-09-10T21:13:57.677439+0000","last_clean":"2024-09-10T21:13:57.677439+0000","last_became_active":"2024-09-10T21:13:57.673097+0000","last_became_peered":"2024-09-10T21:13:57.673097+0000","last_unstale":"2024-09-10T21:13:57.677439+0000","last_undegraded":"2024-09-10T21:13:57.677439+0000","last_fullsized":"2024-09-10T21:13:57.677439+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:55.041043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653394+0000","last_change":"2024-09-10T21:13:56.668298+0000","last_active":"2024-09-10T21:13:58.653394+0000","last_peered":"2024-09-10T21:13:58.653394+0000","last_clean":"2024-09-10T21:13:58.653394+0000","last_became_active":"2024-09-10T21:13:56.668017+0000","last_became_peered":"2024-09-10T21:13:56.668017+0000","last_unstale":"2024-09-10T21:13:58.653394+0000","last_undegraded":"2024-09-10T21:13:58.653394+0000","last_fullsized":"2024-09-10T21:13:58.653394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:36.389857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653070+0000","last_change":"2024-09-10T21:13:57.671156+0000","last_active":"2024-09-10T21:13:58.653070+0000","last_peered":"2024-09-10T21:13:58.653070+0000","last_clean":"2024-09-10T21:13:58.653070+0000","last_became_active":"2024-09-10T21:13:57.670810+0000","last_became_peered":"2024-09-10T21:13:57.670810+0000","last_unstale":"2024-09-10T21:13:58.653070+0000","last_undegraded":"2024-09-10T21:13:58.653070+0000","last_fullsized":"2024-09-10T21:13:58.653070+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:50.377859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654158+0000","last_change":"2024-09-10T21:13:56.669721+0000","last_active":"2024-09-10T21:13:58.654158+0000","last_peered":"2024-09-10T21:13:58.654158+0000","last_clean":"2024-09-10T21:13:58.654158+0000","last_became_active":"2024-09-10T21:13:56.669450+0000","last_became_peered":"2024-09-10T21:13:56.669450+0000","last_unstale":"2024-09-10T21:13:58.654158+0000","last_undegraded":"2024-09-10T21:13:58.654158+0000","last_fullsized":"2024-09-10T21:13:58.654158+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:34.038173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652674+0000","last_change":"2024-09-10T21:13:57.671315+0000","last_active":"2024-09-10T21:13:58.652674+0000","last_peered":"2024-09-10T21:13:58.652674+0000","last_clean":"2024-09-10T21:13:58.652674+0000","last_became_active":"2024-09-10T21:13:57.670995+0000","last_became_peered":"2024-09-10T21:13:57.670995+0000","last_unstale":"2024-09-10T21:13:58.652674+0000","last_undegraded":"2024-09-10T21:13:58.652674+0000","last_fullsized":"2024-09-10T21:13:58.652674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:52.795683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651803+0000","last_change":"2024-09-10T21:13:56.663217+0000","last_active":"2024-09-10T21:13:58.651803+0000","last_peered":"2024-09-10T21:13:58.651803+0000","last_clean":"2024-09-10T21:13:58.651803+0000","last_became_active":"2024-09-10T21:13:56.662670+0000","last_became_peered":"2024-09-10T21:13:56.662670+0000","last_unstale":"2024-09-10T21:13:58.651803+0000","last_undegraded":"2024-09-10T21:13:58.651803+0000","last_fullsized":"2024-09-10T21:13:58.651803+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:58.126472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.679899+0000","last_change":"2024-09-10T21:13:57.677205+0000","last_active":"2024-09-10T21:13:57.679899+0000","last_peered":"2024-09-10T21:13:57.679899+0000","last_clean":"2024-09-10T21:13:57.679899+0000","last_became_active":"2024-09-10T21:13:57.676882+0000","last_became_peered":"2024-09-10T21:13:57.676882+0000","last_unstale":"2024-09-10T21:13:57.679899+0000","last_undegraded":"2024-09-10T21:13:57.679899+0000","last_fullsized":"2024-09-10T21:13:57.679899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:01.033971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.656260+0000","last_change":"2024-09-10T21:13:56.656577+0000","last_active":"2024-09-10T21:13:57.656260+0000","last_peered":"2024-09-10T21:13:57.656260+0000","last_clean":"2024-09-10T21:13:57.656260+0000","last_became_active":"2024-09-10T21:13:56.656210+0000","last_became_peered":"2024-09-10T21:13:56.656210+0000","last_unstale":"2024-09-10T21:13:57.656260+0000","last_undegraded":"2024-09-10T21:13:57.656260+0000","last_fullsized":"2024-09-10T21:13:57.656260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:47.235368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.684843+0000","last_change":"2024-09-10T21:13:57.672623+0000","last_active":"2024-09-10T21:13:57.684843+0000","last_peered":"2024-09-10T21:13:57.684843+0000","last_clean":"2024-09-10T21:13:57.684843+0000","last_became_active":"2024-09-10T21:13:57.672317+0000","last_became_peered":"2024-09-10T21:13:57.672317+0000","last_unstale":"2024-09-10T21:13:57.684843+0000","last_undegraded":"2024-09-10T21:13:57.684843+0000","last_fullsized":"2024-09-10T21:13:57.684843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:55.041043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.655583+0000","last_change":"2024-09-10T21:13:56.659363+0000","last_active":"2024-09-10T21:13:57.655583+0000","last_peered":"2024-09-10T21:13:57.655583+0000","last_clean":"2024-09-10T21:13:57.655583+0000","last_became_active":"2024-09-10T21:13:56.658325+0000","last_became_peered":"2024-09-10T21:13:56.658325+0000","last_unstale":"2024-09-10T21:13:57.655583+0000","last_undegraded":"2024-09-10T21:13:57.655583+0000","last_fullsized":"2024-09-10T21:13:57.655583+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:33.843989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.687893+0000","last_change":"2024-09-10T21:13:57.683302+0000","last_active":"2024-09-10T21:13:57.687893+0000","last_peered":"2024-09-10T21:13:57.687893+0000","last_clean":"2024-09-10T21:13:57.687893+0000","last_became_active":"2024-09-10T21:13:57.681805+0000","last_became_peered":"2024-09-10T21:13:57.681805+0000","last_unstale":"2024-09-10T21:13:57.687893+0000","last_undegraded":"2024-09-10T21:13:57.687893+0000","last_fullsized":"2024-09-10T21:13:57.687893+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:01.532812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.660466+0000","last_change":"2024-09-10T21:13:56.667222+0000","last_active":"2024-09-10T21:13:57.660466+0000","last_peered":"2024-09-10T21:13:57.660466+0000","last_clean":"2024-09-10T21:13:57.660466+0000","last_became_active":"2024-09-10T21:13:56.666917+0000","last_became_peered":"2024-09-10T21:13:56.666917+0000","last_unstale":"2024-09-10T21:13:57.660466+0000","last_undegraded":"2024-09-10T21:13:57.660466+0000","last_fullsized":"2024-09-10T21:13:57.660466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:18.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.677548+0000","last_change":"2024-09-10T21:13:57.672392+0000","last_active":"2024-09-10T21:13:57.677548+0000","last_peered":"2024-09-10T21:13:57.677548+0000","last_clean":"2024-09-10T21:13:57.677548+0000","last_became_active":"2024-09-10T21:13:57.672119+0000","last_became_peered":"2024-09-10T21:13:57.672119+0000","last_unstale":"2024-09-10T21:13:57.677548+0000","last_undegraded":"2024-09-10T21:13:57.677548+0000","last_fullsized":"2024-09-10T21:13:57.677548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:31.157608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652794+0000","last_change":"2024-09-10T21:13:56.662063+0000","last_active":"2024-09-10T21:13:58.652794+0000","last_peered":"2024-09-10T21:13:58.652794+0000","last_clean":"2024-09-10T21:13:58.652794+0000","last_became_active":"2024-09-10T21:13:56.661748+0000","last_became_peered":"2024-09-10T21:13:56.661748+0000","last_unstale":"2024-09-10T21:13:58.652794+0000","last_undegraded":"2024-09-10T21:13:58.652794+0000","last_fullsized":"2024-09-10T21:13:58.652794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:28.991050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.673977+0000","last_change":"2024-09-10T21:13:57.677141+0000","last_active":"2024-09-10T21:13:58.673977+0000","last_peered":"2024-09-10T21:13:58.673977+0000","last_clean":"2024-09-10T21:13:58.673977+0000","last_became_active":"2024-09-10T21:13:57.672963+0000","last_became_peered":"2024-09-10T21:13:57.672963+0000","last_unstale":"2024-09-10T21:13:58.673977+0000","last_undegraded":"2024-09-10T21:13:58.673977+0000","last_fullsized":"2024-09-10T21:13:58.673977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:47.313402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"80'3","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.691715+0000","last_change":"2024-09-10T21:13:56.669641+0000","last_active":"2024-09-10T21:13:57.691715+0000","last_peered":"2024-09-10T21:13:57.691715+0000","last_clean":"2024-09-10T21:13:57.691715+0000","last_became_active":"2024-09-10T21:13:56.669354+0000","last_became_peered":"2024-09-10T21:13:56.669354+0000","last_unstale":"2024-09-10T21:13:57.691715+0000","last_undegraded":"2024-09-10T21:13:57.691715+0000","last_fullsized":"2024-09-10T21:13:57.691715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.444461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691501+0000","last_change":"2024-09-10T21:13:57.668851+0000","last_active":"2024-09-10T21:13:58.691501+0000","last_peered":"2024-09-10T21:13:58.691501+0000","last_clean":"2024-09-10T21:13:58.691501+0000","last_became_active":"2024-09-10T21:13:57.668223+0000","last_became_peered":"2024-09-10T21:13:57.668223+0000","last_unstale":"2024-09-10T21:13:58.691501+0000","last_undegraded":"2024-09-10T21:13:58.691501+0000","last_fullsized":"2024-09-10T21:13:58.691501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:09.188572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"80'6","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654342+0000","last_change":"2024-09-10T21:13:56.663493+0000","last_active":"2024-09-10T21:13:58.654342+0000","last_peered":"2024-09-10T21:13:58.654342+0000","last_clean":"2024-09-10T21:13:58.654342+0000","last_became_active":"2024-09-10T21:13:56.663206+0000","last_became_peered":"2024-09-10T21:13:56.663206+0000","last_unstale":"2024-09-10T21:13:58.654342+0000","last_undegraded":"2024-09-10T21:13:58.654342+0000","last_fullsized":"2024-09-10T21:13:58.654342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:28.923989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.673921+0000","last_change":"2024-09-10T21:13:57.676164+0000","last_active":"2024-09-10T21:13:58.673921+0000","last_peered":"2024-09-10T21:13:58.673921+0000","last_clean":"2024-09-10T21:13:58.673921+0000","last_became_active":"2024-09-10T21:13:57.675863+0000","last_became_peered":"2024-09-10T21:13:57.675863+0000","last_unstale":"2024-09-10T21:13:58.673921+0000","last_undegraded":"2024-09-10T21:13:58.673921+0000","last_fullsized":"2024-09-10T21:13:58.673921+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:56.243098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653255+0000","last_change":"2024-09-10T21:13:56.657951+0000","last_active":"2024-09-10T21:13:58.653255+0000","last_peered":"2024-09-10T21:13:58.653255+0000","last_clean":"2024-09-10T21:13:58.653255+0000","last_became_active":"2024-09-10T21:13:56.657546+0000","last_became_peered":"2024-09-10T21:13:56.657546+0000","last_unstale":"2024-09-10T21:13:58.653255+0000","last_undegraded":"2024-09-10T21:13:58.653255+0000","last_fullsized":"2024-09-10T21:13:58.653255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:09.190211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.676322+0000","last_change":"2024-09-10T21:13:57.671289+0000","last_active":"2024-09-10T21:13:57.676322+0000","last_peered":"2024-09-10T21:13:57.676322+0000","last_clean":"2024-09-10T21:13:57.676322+0000","last_became_active":"2024-09-10T21:13:57.670996+0000","last_became_peered":"2024-09-10T21:13:57.670996+0000","last_unstale":"2024-09-10T21:13:57.676322+0000","last_undegraded":"2024-09-10T21:13:57.676322+0000","last_fullsized":"2024-09-10T21:13:57.676322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:28:14.065978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652299+0000","last_change":"2024-09-10T21:13:56.656519+0000","last_active":"2024-09-10T21:13:58.652299+0000","last_peered":"2024-09-10T21:13:58.652299+0000","last_clean":"2024-09-10T21:13:58.652299+0000","last_became_active":"2024-09-10T21:13:56.656214+0000","last_became_peered":"2024-09-10T21:13:56.656214+0000","last_unstale":"2024-09-10T21:13:58.652299+0000","last_undegraded":"2024-09-10T21:13:58.652299+0000","last_fullsized":"2024-09-10T21:13:58.652299+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:06.514907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.683522+0000","last_change":"2024-09-10T21:13:57.680707+0000","last_active":"2024-09-10T21:13:57.683522+0000","last_peered":"2024-09-10T21:13:57.683522+0000","last_clean":"2024-09-10T21:13:57.683522+0000","last_became_active":"2024-09-10T21:13:57.676133+0000","last_became_peered":"2024-09-10T21:13:57.676133+0000","last_unstale":"2024-09-10T21:13:57.683522+0000","last_undegraded":"2024-09-10T21:13:57.683522+0000","last_fullsized":"2024-09-10T21:13:57.683522+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:03.773571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651428+0000","last_change":"2024-09-10T21:13:56.651265+0000","last_active":"2024-09-10T21:13:58.651428+0000","last_peered":"2024-09-10T21:13:58.651428+0000","last_clean":"2024-09-10T21:13:58.651428+0000","last_became_active":"2024-09-10T21:13:56.650676+0000","last_became_peered":"2024-09-10T21:13:56.650676+0000","last_unstale":"2024-09-10T21:13:58.651428+0000","last_undegraded":"2024-09-10T21:13:58.651428+0000","last_fullsized":"2024-09-10T21:13:58.651428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:59.285767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.673773+0000","last_change":"2024-09-10T21:13:57.688435+0000","last_active":"2024-09-10T21:13:58.673773+0000","last_peered":"2024-09-10T21:13:58.673773+0000","last_clean":"2024-09-10T21:13:58.673773+0000","last_became_active":"2024-09-10T21:13:57.685659+0000","last_became_peered":"2024-09-10T21:13:57.685659+0000","last_unstale":"2024-09-10T21:13:58.673773+0000","last_undegraded":"2024-09-10T21:13:58.673773+0000","last_fullsized":"2024-09-10T21:13:58.673773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:41.854723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"80'8","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674192+0000","last_change":"2024-09-10T21:13:56.648738+0000","last_active":"2024-09-10T21:13:58.674192+0000","last_peered":"2024-09-10T21:13:58.674192+0000","last_clean":"2024-09-10T21:13:58.674192+0000","last_became_active":"2024-09-10T21:13:56.648029+0000","last_became_peered":"2024-09-10T21:13:56.648029+0000","last_unstale":"2024-09-10T21:13:58.674192+0000","last_undegraded":"2024-09-10T21:13:58.674192+0000","last_fullsized":"2024-09-10T21:13:58.674192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:44.422283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654415+0000","last_change":"2024-09-10T21:13:57.678126+0000","last_active":"2024-09-10T21:13:58.654415+0000","last_peered":"2024-09-10T21:13:58.654415+0000","last_clean":"2024-09-10T21:13:58.654415+0000","last_became_active":"2024-09-10T21:13:57.677835+0000","last_became_peered":"2024-09-10T21:13:57.677835+0000","last_unstale":"2024-09-10T21:13:58.654415+0000","last_undegraded":"2024-09-10T21:13:58.654415+0000","last_fullsized":"2024-09-10T21:13:58.654415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:52.795683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691914+0000","last_change":"2024-09-10T21:13:56.664199+0000","last_active":"2024-09-10T21:13:58.691914+0000","last_peered":"2024-09-10T21:13:58.691914+0000","last_clean":"2024-09-10T21:13:58.691914+0000","last_became_active":"2024-09-10T21:13:56.664007+0000","last_became_peered":"2024-09-10T21:13:56.664007+0000","last_unstale":"2024-09-10T21:13:58.691914+0000","last_undegraded":"2024-09-10T21:13:58.691914+0000","last_fullsized":"2024-09-10T21:13:58.691914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:32.999753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653244+0000","last_change":"2024-09-10T21:13:57.669220+0000","last_active":"2024-09-10T21:13:58.653244+0000","last_peered":"2024-09-10T21:13:58.653244+0000","last_clean":"2024-09-10T21:13:58.653244+0000","last_became_active":"2024-09-10T21:13:57.668984+0000","last_became_peered":"2024-09-10T21:13:57.668984+0000","last_unstale":"2024-09-10T21:13:58.653244+0000","last_undegraded":"2024-09-10T21:13:58.653244+0000","last_fullsized":"2024-09-10T21:13:58.653244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:36.159803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.692188+0000","last_change":"2024-09-10T21:13:56.668547+0000","last_active":"2024-09-10T21:13:57.692188+0000","last_peered":"2024-09-10T21:13:57.692188+0000","last_clean":"2024-09-10T21:13:57.692188+0000","last_became_active":"2024-09-10T21:13:56.668237+0000","last_became_peered":"2024-09-10T21:13:56.668237+0000","last_unstale":"2024-09-10T21:13:57.692188+0000","last_undegraded":"2024-09-10T21:13:57.692188+0000","last_fullsized":"2024-09-10T21:13:57.692188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:18.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654507+0000","last_change":"2024-09-10T21:13:57.677601+0000","last_active":"2024-09-10T21:13:58.654507+0000","last_peered":"2024-09-10T21:13:58.654507+0000","last_clean":"2024-09-10T21:13:58.654507+0000","last_became_active":"2024-09-10T21:13:57.677383+0000","last_became_peered":"2024-09-10T21:13:57.677383+0000","last_unstale":"2024-09-10T21:13:58.654507+0000","last_undegraded":"2024-09-10T21:13:58.654507+0000","last_fullsized":"2024-09-10T21:13:58.654507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:56.313169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.656429+0000","last_change":"2024-09-10T21:13:56.660492+0000","last_active":"2024-09-10T21:13:57.656429+0000","last_peered":"2024-09-10T21:13:57.656429+0000","last_clean":"2024-09-10T21:13:57.656429+0000","last_became_active":"2024-09-10T21:13:56.660178+0000","last_became_peered":"2024-09-10T21:13:56.660178+0000","last_unstale":"2024-09-10T21:13:57.656429+0000","last_undegraded":"2024-09-10T21:13:57.656429+0000","last_fullsized":"2024-09-10T21:13:57.656429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:11.412884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652074+0000","last_change":"2024-09-10T21:13:57.669400+0000","last_active":"2024-09-10T21:13:58.652074+0000","last_peered":"2024-09-10T21:13:58.652074+0000","last_clean":"2024-09-10T21:13:58.652074+0000","last_became_active":"2024-09-10T21:13:57.669025+0000","last_became_peered":"2024-09-10T21:13:57.669025+0000","last_unstale":"2024-09-10T21:13:58.652074+0000","last_undegraded":"2024-09-10T21:13:58.652074+0000","last_fullsized":"2024-09-10T21:13:58.652074+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:22.674883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.692760+0000","last_change":"2024-09-10T21:13:56.660970+0000","last_active":"2024-09-10T21:13:57.692760+0000","last_peered":"2024-09-10T21:13:57.692760+0000","last_clean":"2024-09-10T21:13:57.692760+0000","last_became_active":"2024-09-10T21:13:56.660729+0000","last_became_peered":"2024-09-10T21:13:56.660729+0000","last_unstale":"2024-09-10T21:13:57.692760+0000","last_undegraded":"2024-09-10T21:13:57.692760+0000","last_fullsized":"2024-09-10T21:13:57.692760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.444461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691130+0000","last_change":"2024-09-10T21:13:57.668511+0000","last_active":"2024-09-10T21:13:58.691130+0000","last_peered":"2024-09-10T21:13:58.691130+0000","last_clean":"2024-09-10T21:13:58.691130+0000","last_became_active":"2024-09-10T21:13:57.668192+0000","last_became_peered":"2024-09-10T21:13:57.668192+0000","last_unstale":"2024-09-10T21:13:58.691130+0000","last_undegraded":"2024-09-10T21:13:58.691130+0000","last_fullsized":"2024-09-10T21:13:58.691130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:54.301500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652403+0000","last_change":"2024-09-10T21:13:56.660874+0000","last_active":"2024-09-10T21:13:58.652403+0000","last_peered":"2024-09-10T21:13:58.652403+0000","last_clean":"2024-09-10T21:13:58.652403+0000","last_became_active":"2024-09-10T21:13:56.660603+0000","last_became_peered":"2024-09-10T21:13:56.660603+0000","last_unstale":"2024-09-10T21:13:58.652403+0000","last_undegraded":"2024-09-10T21:13:58.652403+0000","last_fullsized":"2024-09-10T21:13:58.652403+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:07.314380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652050+0000","last_change":"2024-09-10T21:13:57.665726+0000","last_active":"2024-09-10T21:13:58.652050+0000","last_peered":"2024-09-10T21:13:58.652050+0000","last_clean":"2024-09-10T21:13:58.652050+0000","last_became_active":"2024-09-10T21:13:57.665385+0000","last_became_peered":"2024-09-10T21:13:57.665385+0000","last_unstale":"2024-09-10T21:13:58.652050+0000","last_undegraded":"2024-09-10T21:13:58.652050+0000","last_fullsized":"2024-09-10T21:13:58.652050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:17.657607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.673794+0000","last_change":"2024-09-10T21:13:56.646406+0000","last_active":"2024-09-10T21:13:58.673794+0000","last_peered":"2024-09-10T21:13:58.673794+0000","last_clean":"2024-09-10T21:13:58.673794+0000","last_became_active":"2024-09-10T21:13:56.646114+0000","last_became_peered":"2024-09-10T21:13:56.646114+0000","last_unstale":"2024-09-10T21:13:58.673794+0000","last_undegraded":"2024-09-10T21:13:58.673794+0000","last_fullsized":"2024-09-10T21:13:58.673794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:11.231979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654093+0000","last_change":"2024-09-10T21:13:57.674009+0000","last_active":"2024-09-10T21:13:58.654093+0000","last_peered":"2024-09-10T21:13:58.654093+0000","last_clean":"2024-09-10T21:13:58.654093+0000","last_became_active":"2024-09-10T21:13:57.672222+0000","last_became_peered":"2024-09-10T21:13:57.672222+0000","last_unstale":"2024-09-10T21:13:58.654093+0000","last_undegraded":"2024-09-10T21:13:58.654093+0000","last_fullsized":"2024-09-10T21:13:58.654093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:54.144078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.656649+0000","last_change":"2024-09-10T21:13:56.660430+0000","last_active":"2024-09-10T21:13:57.656649+0000","last_peered":"2024-09-10T21:13:57.656649+0000","last_clean":"2024-09-10T21:13:57.656649+0000","last_became_active":"2024-09-10T21:13:56.660136+0000","last_became_peered":"2024-09-10T21:13:56.660136+0000","last_unstale":"2024-09-10T21:13:57.656649+0000","last_undegraded":"2024-09-10T21:13:57.656649+0000","last_fullsized":"2024-09-10T21:13:57.656649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:52.177028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691822+0000","last_change":"2024-09-10T21:13:57.675247+0000","last_active":"2024-09-10T21:13:58.691822+0000","last_peered":"2024-09-10T21:13:58.691822+0000","last_clean":"2024-09-10T21:13:58.691822+0000","last_became_active":"2024-09-10T21:13:57.674950+0000","last_became_peered":"2024-09-10T21:13:57.674950+0000","last_unstale":"2024-09-10T21:13:58.691822+0000","last_undegraded":"2024-09-10T21:13:58.691822+0000","last_fullsized":"2024-09-10T21:13:58.691822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:24.803662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.692291+0000","last_change":"2024-09-10T21:13:56.660247+0000","last_active":"2024-09-10T21:13:57.692291+0000","last_peered":"2024-09-10T21:13:57.692291+0000","last_clean":"2024-09-10T21:13:57.692291+0000","last_became_active":"2024-09-10T21:13:56.659836+0000","last_became_peered":"2024-09-10T21:13:56.659836+0000","last_unstale":"2024-09-10T21:13:57.692291+0000","last_undegraded":"2024-09-10T21:13:57.692291+0000","last_fullsized":"2024-09-10T21:13:57.692291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:28:13.059565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691803+0000","last_change":"2024-09-10T21:13:57.666742+0000","last_active":"2024-09-10T21:13:58.691803+0000","last_peered":"2024-09-10T21:13:58.691803+0000","last_clean":"2024-09-10T21:13:58.691803+0000","last_became_active":"2024-09-10T21:13:57.666361+0000","last_became_peered":"2024-09-10T21:13:57.666361+0000","last_unstale":"2024-09-10T21:13:58.691803+0000","last_undegraded":"2024-09-10T21:13:58.691803+0000","last_fullsized":"2024-09-10T21:13:58.691803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:58.029945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"81'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.676033+0000","last_change":"2024-09-10T21:13:56.649564+0000","last_active":"2024-09-10T21:13:58.676033+0000","last_peered":"2024-09-10T21:13:58.676033+0000","last_clean":"2024-09-10T21:13:58.676033+0000","last_became_active":"2024-09-10T21:13:56.649247+0000","last_became_peered":"2024-09-10T21:13:56.649247+0000","last_unstale":"2024-09-10T21:13:58.676033+0000","last_undegraded":"2024-09-10T21:13:58.676033+0000","last_fullsized":"2024-09-10T21:13:58.676033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:32.437051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691976+0000","last_change":"2024-09-10T21:13:57.666799+0000","last_active":"2024-09-10T21:13:58.691976+0000","last_peered":"2024-09-10T21:13:58.691976+0000","last_clean":"2024-09-10T21:13:58.691976+0000","last_became_active":"2024-09-10T21:13:57.666531+0000","last_became_peered":"2024-09-10T21:13:57.666531+0000","last_unstale":"2024-09-10T21:13:58.691976+0000","last_undegraded":"2024-09-10T21:13:58.691976+0000","last_fullsized":"2024-09-10T21:13:58.691976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:32.980731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654562+0000","last_change":"2024-09-10T21:13:56.666102+0000","last_active":"2024-09-10T21:13:58.654562+0000","last_peered":"2024-09-10T21:13:58.654562+0000","last_clean":"2024-09-10T21:13:58.654562+0000","last_became_active":"2024-09-10T21:13:56.665831+0000","last_became_peered":"2024-09-10T21:13:56.665831+0000","last_unstale":"2024-09-10T21:13:58.654562+0000","last_undegraded":"2024-09-10T21:13:58.654562+0000","last_fullsized":"2024-09-10T21:13:58.654562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:58.126472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653052+0000","last_change":"2024-09-10T21:13:57.681000+0000","last_active":"2024-09-10T21:13:58.653052+0000","last_peered":"2024-09-10T21:13:58.653052+0000","last_clean":"2024-09-10T21:13:58.653052+0000","last_became_active":"2024-09-10T21:13:57.680591+0000","last_became_peered":"2024-09-10T21:13:57.680591+0000","last_unstale":"2024-09-10T21:13:58.653052+0000","last_undegraded":"2024-09-10T21:13:58.653052+0000","last_fullsized":"2024-09-10T21:13:58.653052+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:50.075360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652464+0000","last_change":"2024-09-10T21:13:56.649981+0000","last_active":"2024-09-10T21:13:58.652464+0000","last_peered":"2024-09-10T21:13:58.652464+0000","last_clean":"2024-09-10T21:13:58.652464+0000","last_became_active":"2024-09-10T21:13:56.649711+0000","last_became_peered":"2024-09-10T21:13:56.649711+0000","last_unstale":"2024-09-10T21:13:58.652464+0000","last_undegraded":"2024-09-10T21:13:58.652464+0000","last_fullsized":"2024-09-10T21:13:58.652464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:00.959573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.678403+0000","last_change":"2024-09-10T21:13:57.673209+0000","last_active":"2024-09-10T21:13:57.678403+0000","last_peered":"2024-09-10T21:13:57.678403+0000","last_clean":"2024-09-10T21:13:57.678403+0000","last_became_active":"2024-09-10T21:13:57.672936+0000","last_became_peered":"2024-09-10T21:13:57.672936+0000","last_unstale":"2024-09-10T21:13:57.678403+0000","last_undegraded":"2024-09-10T21:13:57.678403+0000","last_fullsized":"2024-09-10T21:13:57.678403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:41.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652590+0000","last_change":"2024-09-10T21:13:56.659154+0000","last_active":"2024-09-10T21:13:58.652590+0000","last_peered":"2024-09-10T21:13:58.652590+0000","last_clean":"2024-09-10T21:13:58.652590+0000","last_became_active":"2024-09-10T21:13:56.658628+0000","last_became_peered":"2024-09-10T21:13:56.658628+0000","last_unstale":"2024-09-10T21:13:58.652590+0000","last_undegraded":"2024-09-10T21:13:58.652590+0000","last_fullsized":"2024-09-10T21:13:58.652590+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:22.366656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674092+0000","last_change":"2024-09-10T21:13:57.667627+0000","last_active":"2024-09-10T21:13:58.674092+0000","last_peered":"2024-09-10T21:13:58.674092+0000","last_clean":"2024-09-10T21:13:58.674092+0000","last_became_active":"2024-09-10T21:13:57.667212+0000","last_became_peered":"2024-09-10T21:13:57.667212+0000","last_unstale":"2024-09-10T21:13:58.674092+0000","last_undegraded":"2024-09-10T21:13:58.674092+0000","last_fullsized":"2024-09-10T21:13:58.674092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:22.674883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651891+0000","last_change":"2024-09-10T21:13:56.662937+0000","last_active":"2024-09-10T21:13:58.651891+0000","last_peered":"2024-09-10T21:13:58.651891+0000","last_clean":"2024-09-10T21:13:58.651891+0000","last_became_active":"2024-09-10T21:13:56.662220+0000","last_became_peered":"2024-09-10T21:13:56.662220+0000","last_unstale":"2024-09-10T21:13:58.651891+0000","last_undegraded":"2024-09-10T21:13:58.651891+0000","last_fullsized":"2024-09-10T21:13:58.651891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:28.923989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.679875+0000","last_change":"2024-09-10T21:13:57.675310+0000","last_active":"2024-09-10T21:13:57.679875+0000","last_peered":"2024-09-10T21:13:57.679875+0000","last_clean":"2024-09-10T21:13:57.679875+0000","last_became_active":"2024-09-10T21:13:57.674398+0000","last_became_peered":"2024-09-10T21:13:57.674398+0000","last_unstale":"2024-09-10T21:13:57.679875+0000","last_undegraded":"2024-09-10T21:13:57.679875+0000","last_fullsized":"2024-09-10T21:13:57.679875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:47.869632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.658286+0000","last_change":"2024-09-10T21:13:56.665686+0000","last_active":"2024-09-10T21:13:57.658286+0000","last_peered":"2024-09-10T21:13:57.658286+0000","last_clean":"2024-09-10T21:13:57.658286+0000","last_became_active":"2024-09-10T21:13:56.665410+0000","last_became_peered":"2024-09-10T21:13:56.665410+0000","last_unstale":"2024-09-10T21:13:57.658286+0000","last_undegraded":"2024-09-10T21:13:57.658286+0000","last_fullsized":"2024-09-10T21:13:57.658286+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:47.725344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.679736+0000","last_change":"2024-09-10T21:13:57.678980+0000","last_active":"2024-09-10T21:13:57.679736+0000","last_peered":"2024-09-10T21:13:57.679736+0000","last_clean":"2024-09-10T21:13:57.679736+0000","last_became_active":"2024-09-10T21:13:57.678655+0000","last_became_peered":"2024-09-10T21:13:57.678655+0000","last_unstale":"2024-09-10T21:13:57.679736+0000","last_undegraded":"2024-09-10T21:13:57.679736+0000","last_fullsized":"2024-09-10T21:13:57.679736+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:42.885937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654048+0000","last_change":"2024-09-10T21:13:56.669362+0000","last_active":"2024-09-10T21:13:58.654048+0000","last_peered":"2024-09-10T21:13:58.654048+0000","last_clean":"2024-09-10T21:13:58.654048+0000","last_became_active":"2024-09-10T21:13:56.668961+0000","last_became_peered":"2024-09-10T21:13:56.668961+0000","last_unstale":"2024-09-10T21:13:58.654048+0000","last_undegraded":"2024-09-10T21:13:58.654048+0000","last_fullsized":"2024-09-10T21:13:58.654048+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:00.959573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674086+0000","last_change":"2024-09-10T21:13:57.673095+0000","last_active":"2024-09-10T21:13:58.674086+0000","last_peered":"2024-09-10T21:13:58.674086+0000","last_clean":"2024-09-10T21:13:58.674086+0000","last_became_active":"2024-09-10T21:13:57.671973+0000","last_became_peered":"2024-09-10T21:13:57.671973+0000","last_unstale":"2024-09-10T21:13:58.674086+0000","last_undegraded":"2024-09-10T21:13:58.674086+0000","last_fullsized":"2024-09-10T21:13:58.674086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:40.692031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691618+0000","last_change":"2024-09-10T21:13:56.658873+0000","last_active":"2024-09-10T21:13:58.691618+0000","last_peered":"2024-09-10T21:13:58.691618+0000","last_clean":"2024-09-10T21:13:58.691618+0000","last_became_active":"2024-09-10T21:13:56.658480+0000","last_became_peered":"2024-09-10T21:13:56.658480+0000","last_unstale":"2024-09-10T21:13:58.691618+0000","last_undegraded":"2024-09-10T21:13:58.691618+0000","last_fullsized":"2024-09-10T21:13:58.691618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:42.885620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":213,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.658725+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:13:57.658725+0000","last_peered":"2024-09-10T21:13:57.658725+0000","last_clean":"2024-09-10T21:13:57.658725+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:13:57.658725+0000","last_undegraded":"2024-09-10T21:13:57.658725+0000","last_fullsized":"2024-09-10T21:13:57.658725+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36308,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335532,"statfs":{"total":96636764160,"available":96599584768,"internally_reserved":0,"allocated":983040,"data_stored":412608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1597440,"data_stored":1010250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":991232,"data_stored":418270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509486,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":987136,"data_stored":412610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":974848,"data_stored":411936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1003520,"data_stored":419870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36288,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335552,"statfs":{"total":96636764160,"available":96599605248,"internally_reserved":0,"allocated":962560,"data_stored":405161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542190,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36888,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334952,"statfs":{"total":96636764160,"available":96598990848,"internally_reserved":0,"allocated":1572864,"data_stored":1002394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:14:01.813 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:14:02.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 -- 172.21.15.16:0/493761227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9108161560 msgr2=0x7f91080d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 --2- 172.21.15.16:0/493761227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9108161560 0x7f91080d4a20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f91080b6a70 tx=0x7f90f801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 -- 172.21.15.16:0/493761227 shutdown_connections 2024-09-10T21:14:02.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 --2- 172.21.15.16:0/493761227 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91080d0c80 0x7f91080cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 --2- 172.21.15.16:0/493761227 >> v2:172.21.15.52:3300/0 conn(0x7f91080b2c20 0x7f91080d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 --2- 172.21.15.16:0/493761227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9108161560 0x7f91080d4a20 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 -- 172.21.15.16:0/493761227 >> 172.21.15.16:0/493761227 conn(0x7f91080d3f00 msgr2=0x7f91080d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:02.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 -- 172.21.15.16:0/493761227 shutdown_connections 2024-09-10T21:14:02.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 -- 172.21.15.16:0/493761227 wait complete. 2024-09-10T21:14:02.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 Processor -- start 2024-09-10T21:14:02.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 -- start start 2024-09-10T21:14:02.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91080b2c20 0x7f910815a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91080d0c80 0x7f91080c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9108161560 0x7f910815e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f91080b18e0 con 0x7f91080d0c80 2024-09-10T21:14:02.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f91080b1760 con 0x7f9108161560 2024-09-10T21:14:02.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9107e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f91080b1a60 con 0x7f91080b2c20 2024-09-10T21:14:02.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f90ffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91080d0c80 0x7f91080c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f90ffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91080d0c80 0x7f91080c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57112/0 (socket says 172.21.15.16:57112) 2024-09-10T21:14:02.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f90ffe00640 1 -- 172.21.15.16:0/2987776084 learned_addr learned my addr 172.21.15.16:0/2987776084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:02.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.995+0000 7f9106a00640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91080b2c20 0x7f910815a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f9107400640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9108161560 0x7f910815e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f90ffe00640 1 -- 172.21.15.16:0/2987776084 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91080b2c20 msgr2=0x7f910815a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f90ffe00640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91080b2c20 0x7f910815a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f90ffe00640 1 -- 172.21.15.16:0/2987776084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9108161560 msgr2=0x7f910815e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f90ffe00640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9108161560 0x7f910815e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f90ffe00640 1 -- 172.21.15.16:0/2987776084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9108159c80 con 0x7f91080d0c80 2024-09-10T21:14:02.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f90ffe00640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91080d0c80 0x7f91080c8a00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f90f000d700 tx=0x7f90f000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:02.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f90fea00640 1 -- 172.21.15.16:0/2987776084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0017070 con 0x7f91080d0c80 2024-09-10T21:14:02.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f90fea00640 1 -- 172.21.15.16:0/2987776084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90f0009d20 con 0x7f91080d0c80 2024-09-10T21:14:02.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f90fea00640 1 -- 172.21.15.16:0/2987776084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0004280 con 0x7f91080d0c80 2024-09-10T21:14:02.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f9107e00640 1 -- 172.21.15.16:0/2987776084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91080c9400 con 0x7f91080d0c80 2024-09-10T21:14:02.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f9107e00640 1 -- 172.21.15.16:0/2987776084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91080c99c0 con 0x7f91080d0c80 2024-09-10T21:14:02.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f9107e00640 1 -- 172.21.15.16:0/2987776084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91080b1f80 con 0x7f91080d0c80 2024-09-10T21:14:02.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:01.999+0000 7f90fea00640 1 -- 172.21.15.16:0/2987776084 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f90f0003600 con 0x7f91080d0c80 2024-09-10T21:14:02.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.003+0000 7f90fea00640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90e40725e0 0x7f90e4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.003+0000 7f90fea00640 1 -- 172.21.15.16:0/2987776084 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f90f0093300 con 0x7f91080d0c80 2024-09-10T21:14:02.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.003+0000 7f9106a00640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90e40725e0 0x7f90e4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.003+0000 7f9106a00640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90e40725e0 0x7f90e4074aa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f90f80af000 tx=0x7f90f80a7040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:02.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.003+0000 7f90fea00640 1 -- 172.21.15.16:0/2987776084 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f90f005e2f0 con 0x7f91080d0c80 2024-09-10T21:14:02.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.175+0000 7f9107e00640 1 -- 172.21.15.16:0/2987776084 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f91080d0310 con 0x7f90e40725e0 2024-09-10T21:14:02.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.179+0000 7f90fea00640 1 -- 172.21.15.16:0/2987776084 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7f91080d0310 con 0x7f90e40725e0 2024-09-10T21:14:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:02.190 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:14:02.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.183+0000 7f9107e00640 1 -- 172.21.15.16:0/2987776084 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90e40725e0 msgr2=0x7f90e4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.183+0000 7f9107e00640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90e40725e0 0x7f90e4074aa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f90f80af000 tx=0x7f90f80a7040 comp rx=0 tx=0).stop 2024-09-10T21:14:02.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.183+0000 7f9107e00640 1 -- 172.21.15.16:0/2987776084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91080d0c80 msgr2=0x7f91080c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.183+0000 7f9107e00640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91080d0c80 0x7f91080c8a00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f90f000d700 tx=0x7f90f000dbc0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.183+0000 7f9107e00640 1 -- 172.21.15.16:0/2987776084 shutdown_connections 2024-09-10T21:14:02.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.183+0000 7f9107e00640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90e40725e0 0x7f90e4074aa0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.183+0000 7f9107e00640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9108161560 0x7f910815e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.183+0000 7f9107e00640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91080d0c80 0x7f91080c8a00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.183+0000 7f9107e00640 1 --2- 172.21.15.16:0/2987776084 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91080b2c20 0x7f910815a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.183+0000 7f9107e00640 1 -- 172.21.15.16:0/2987776084 >> 172.21.15.16:0/2987776084 conn(0x7f91080d3f00 msgr2=0x7f91080b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.183+0000 7f9107e00640 1 -- 172.21.15.16:0/2987776084 shutdown_connections 2024-09-10T21:14:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.183+0000 7f9107e00640 1 -- 172.21.15.16:0/2987776084 wait complete. 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-10T21:14:01.503825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270876,"kb_used_data":8860,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754703844,"statfs":{"total":773094113280,"available":772816736256,"internally_reserved":0,"allocated":9072640,"data_stored":4493099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.869327"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.679268+0000","last_change":"2024-09-10T21:13:57.675647+0000","last_active":"2024-09-10T21:13:57.679268+0000","last_peered":"2024-09-10T21:13:57.679268+0000","last_clean":"2024-09-10T21:13:57.679268+0000","last_became_active":"2024-09-10T21:13:57.675067+0000","last_became_peered":"2024-09-10T21:13:57.675067+0000","last_unstale":"2024-09-10T21:13:57.679268+0000","last_undegraded":"2024-09-10T21:13:57.679268+0000","last_fullsized":"2024-09-10T21:13:57.679268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:08.229499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652856+0000","last_change":"2024-09-10T21:13:57.666078+0000","last_active":"2024-09-10T21:13:58.652856+0000","last_peered":"2024-09-10T21:13:58.652856+0000","last_clean":"2024-09-10T21:13:58.652856+0000","last_became_active":"2024-09-10T21:13:57.665762+0000","last_became_peered":"2024-09-10T21:13:57.665762+0000","last_unstale":"2024-09-10T21:13:58.652856+0000","last_undegraded":"2024-09-10T21:13:58.652856+0000","last_fullsized":"2024-09-10T21:13:58.652856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:06.098117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652861+0000","last_change":"2024-09-10T21:13:57.665470+0000","last_active":"2024-09-10T21:13:58.652861+0000","last_peered":"2024-09-10T21:13:58.652861+0000","last_clean":"2024-09-10T21:13:58.652861+0000","last_became_active":"2024-09-10T21:13:57.665120+0000","last_became_peered":"2024-09-10T21:13:57.665120+0000","last_unstale":"2024-09-10T21:13:58.652861+0000","last_undegraded":"2024-09-10T21:13:58.652861+0000","last_fullsized":"2024-09-10T21:13:58.652861+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:56.313169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653406+0000","last_change":"2024-09-10T21:13:57.661056+0000","last_active":"2024-09-10T21:13:58.653406+0000","last_peered":"2024-09-10T21:13:58.653406+0000","last_clean":"2024-09-10T21:13:58.653406+0000","last_became_active":"2024-09-10T21:13:57.659861+0000","last_became_peered":"2024-09-10T21:13:57.659861+0000","last_unstale":"2024-09-10T21:13:58.653406+0000","last_undegraded":"2024-09-10T21:13:58.653406+0000","last_fullsized":"2024-09-10T21:13:58.653406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:31.700508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.690903+0000","last_change":"2024-09-10T21:13:57.681348+0000","last_active":"2024-09-10T21:13:58.690903+0000","last_peered":"2024-09-10T21:13:58.690903+0000","last_clean":"2024-09-10T21:13:58.690903+0000","last_became_active":"2024-09-10T21:13:57.680908+0000","last_became_peered":"2024-09-10T21:13:57.680908+0000","last_unstale":"2024-09-10T21:13:58.690903+0000","last_undegraded":"2024-09-10T21:13:58.690903+0000","last_fullsized":"2024-09-10T21:13:58.690903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:22.099540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651691+0000","last_change":"2024-09-10T21:13:57.659245+0000","last_active":"2024-09-10T21:13:58.651691+0000","last_peered":"2024-09-10T21:13:58.651691+0000","last_clean":"2024-09-10T21:13:58.651691+0000","last_became_active":"2024-09-10T21:13:57.658283+0000","last_became_peered":"2024-09-10T21:13:57.658283+0000","last_unstale":"2024-09-10T21:13:58.651691+0000","last_undegraded":"2024-09-10T21:13:58.651691+0000","last_fullsized":"2024-09-10T21:13:58.651691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:42.562833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652061+0000","last_change":"2024-09-10T21:13:57.665271+0000","last_active":"2024-09-10T21:13:58.652061+0000","last_peered":"2024-09-10T21:13:58.652061+0000","last_clean":"2024-09-10T21:13:58.652061+0000","last_became_active":"2024-09-10T21:13:57.664330+0000","last_became_peered":"2024-09-10T21:13:57.664330+0000","last_unstale":"2024-09-10T21:13:58.652061+0000","last_undegraded":"2024-09-10T21:13:58.652061+0000","last_fullsized":"2024-09-10T21:13:58.652061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:44.586271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654091+0000","last_change":"2024-09-10T21:13:57.677047+0000","last_active":"2024-09-10T21:13:58.654091+0000","last_peered":"2024-09-10T21:13:58.654091+0000","last_clean":"2024-09-10T21:13:58.654091+0000","last_became_active":"2024-09-10T21:13:57.676725+0000","last_became_peered":"2024-09-10T21:13:57.676725+0000","last_unstale":"2024-09-10T21:13:58.654091+0000","last_undegraded":"2024-09-10T21:13:58.654091+0000","last_fullsized":"2024-09-10T21:13:58.654091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:19.669090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.679329+0000","last_change":"2024-09-10T21:13:57.676001+0000","last_active":"2024-09-10T21:13:57.679329+0000","last_peered":"2024-09-10T21:13:57.679329+0000","last_clean":"2024-09-10T21:13:57.679329+0000","last_became_active":"2024-09-10T21:13:57.675597+0000","last_became_peered":"2024-09-10T21:13:57.675597+0000","last_unstale":"2024-09-10T21:13:57.679329+0000","last_undegraded":"2024-09-10T21:13:57.679329+0000","last_fullsized":"2024-09-10T21:13:57.679329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:01.532812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653273+0000","last_change":"2024-09-10T21:13:57.657599+0000","last_active":"2024-09-10T21:13:58.653273+0000","last_peered":"2024-09-10T21:13:58.653273+0000","last_clean":"2024-09-10T21:13:58.653273+0000","last_became_active":"2024-09-10T21:13:57.657327+0000","last_became_peered":"2024-09-10T21:13:57.657327+0000","last_unstale":"2024-09-10T21:13:58.653273+0000","last_undegraded":"2024-09-10T21:13:58.653273+0000","last_fullsized":"2024-09-10T21:13:58.653273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:25.230878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674453+0000","last_change":"2024-09-10T21:13:57.665075+0000","last_active":"2024-09-10T21:13:58.674453+0000","last_peered":"2024-09-10T21:13:58.674453+0000","last_clean":"2024-09-10T21:13:58.674453+0000","last_became_active":"2024-09-10T21:13:57.664732+0000","last_became_peered":"2024-09-10T21:13:57.664732+0000","last_unstale":"2024-09-10T21:13:58.674453+0000","last_undegraded":"2024-09-10T21:13:58.674453+0000","last_fullsized":"2024-09-10T21:13:58.674453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:51:18.181610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654237+0000","last_change":"2024-09-10T21:13:57.682795+0000","last_active":"2024-09-10T21:13:58.654237+0000","last_peered":"2024-09-10T21:13:58.654237+0000","last_clean":"2024-09-10T21:13:58.654237+0000","last_became_active":"2024-09-10T21:13:57.682524+0000","last_became_peered":"2024-09-10T21:13:57.682524+0000","last_unstale":"2024-09-10T21:13:58.654237+0000","last_undegraded":"2024-09-10T21:13:58.654237+0000","last_fullsized":"2024-09-10T21:13:58.654237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:56.643637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.690924+0000","last_change":"2024-09-10T21:13:57.665092+0000","last_active":"2024-09-10T21:13:58.690924+0000","last_peered":"2024-09-10T21:13:58.690924+0000","last_clean":"2024-09-10T21:13:58.690924+0000","last_became_active":"2024-09-10T21:13:57.664814+0000","last_became_peered":"2024-09-10T21:13:57.664814+0000","last_unstale":"2024-09-10T21:13:58.690924+0000","last_undegraded":"2024-09-10T21:13:58.690924+0000","last_fullsized":"2024-09-10T21:13:58.690924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:36.165681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.683678+0000","last_change":"2024-09-10T21:13:57.677622+0000","last_active":"2024-09-10T21:13:57.683678+0000","last_peered":"2024-09-10T21:13:57.683678+0000","last_clean":"2024-09-10T21:13:57.683678+0000","last_became_active":"2024-09-10T21:13:57.677036+0000","last_became_peered":"2024-09-10T21:13:57.677036+0000","last_unstale":"2024-09-10T21:13:57.683678+0000","last_undegraded":"2024-09-10T21:13:57.683678+0000","last_fullsized":"2024-09-10T21:13:57.683678+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:08.229499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652300+0000","last_change":"2024-09-10T21:13:57.655747+0000","last_active":"2024-09-10T21:13:58.652300+0000","last_peered":"2024-09-10T21:13:58.652300+0000","last_clean":"2024-09-10T21:13:58.652300+0000","last_became_active":"2024-09-10T21:13:57.655468+0000","last_became_peered":"2024-09-10T21:13:57.655468+0000","last_unstale":"2024-09-10T21:13:58.652300+0000","last_undegraded":"2024-09-10T21:13:58.652300+0000","last_fullsized":"2024-09-10T21:13:58.652300+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:58.927960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651474+0000","last_change":"2024-09-10T21:13:57.665659+0000","last_active":"2024-09-10T21:13:58.651474+0000","last_peered":"2024-09-10T21:13:58.651474+0000","last_clean":"2024-09-10T21:13:58.651474+0000","last_became_active":"2024-09-10T21:13:57.665302+0000","last_became_peered":"2024-09-10T21:13:57.665302+0000","last_unstale":"2024-09-10T21:13:58.651474+0000","last_undegraded":"2024-09-10T21:13:58.651474+0000","last_fullsized":"2024-09-10T21:13:58.651474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:22.405349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654568+0000","last_change":"2024-09-10T21:13:57.688758+0000","last_active":"2024-09-10T21:13:58.654568+0000","last_peered":"2024-09-10T21:13:58.654568+0000","last_clean":"2024-09-10T21:13:58.654568+0000","last_became_active":"2024-09-10T21:13:57.688387+0000","last_became_peered":"2024-09-10T21:13:57.688387+0000","last_unstale":"2024-09-10T21:13:58.654568+0000","last_undegraded":"2024-09-10T21:13:58.654568+0000","last_fullsized":"2024-09-10T21:13:58.654568+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:50.075360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674475+0000","last_change":"2024-09-10T21:13:57.686565+0000","last_active":"2024-09-10T21:13:58.674475+0000","last_peered":"2024-09-10T21:13:58.674475+0000","last_clean":"2024-09-10T21:13:58.674475+0000","last_became_active":"2024-09-10T21:13:57.681049+0000","last_became_peered":"2024-09-10T21:13:57.681049+0000","last_unstale":"2024-09-10T21:13:58.674475+0000","last_undegraded":"2024-09-10T21:13:58.674475+0000","last_fullsized":"2024-09-10T21:13:58.674475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:50.388234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.682383+0000","last_change":"2024-09-10T21:13:57.677237+0000","last_active":"2024-09-10T21:13:57.682383+0000","last_peered":"2024-09-10T21:13:57.682383+0000","last_clean":"2024-09-10T21:13:57.682383+0000","last_became_active":"2024-09-10T21:13:57.676855+0000","last_became_peered":"2024-09-10T21:13:57.676855+0000","last_unstale":"2024-09-10T21:13:57.682383+0000","last_undegraded":"2024-09-10T21:13:57.682383+0000","last_fullsized":"2024-09-10T21:13:57.682383+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:15.099468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.680569+0000","last_change":"2024-09-10T21:13:57.665394+0000","last_active":"2024-09-10T21:13:57.680569+0000","last_peered":"2024-09-10T21:13:57.680569+0000","last_clean":"2024-09-10T21:13:57.680569+0000","last_became_active":"2024-09-10T21:13:57.665126+0000","last_became_peered":"2024-09-10T21:13:57.665126+0000","last_unstale":"2024-09-10T21:13:57.680569+0000","last_undegraded":"2024-09-10T21:13:57.680569+0000","last_fullsized":"2024-09-10T21:13:57.680569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:31.157608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653101+0000","last_change":"2024-09-10T21:13:57.666294+0000","last_active":"2024-09-10T21:13:58.653101+0000","last_peered":"2024-09-10T21:13:58.653101+0000","last_clean":"2024-09-10T21:13:58.653101+0000","last_became_active":"2024-09-10T21:13:57.664767+0000","last_became_peered":"2024-09-10T21:13:57.664767+0000","last_unstale":"2024-09-10T21:13:58.653101+0000","last_undegraded":"2024-09-10T21:13:58.653101+0000","last_fullsized":"2024-09-10T21:13:58.653101+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:13.733070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.680505+0000","last_change":"2024-09-10T21:13:57.674589+0000","last_active":"2024-09-10T21:13:57.680505+0000","last_peered":"2024-09-10T21:13:57.680505+0000","last_clean":"2024-09-10T21:13:57.680505+0000","last_became_active":"2024-09-10T21:13:57.674324+0000","last_became_peered":"2024-09-10T21:13:57.674324+0000","last_unstale":"2024-09-10T21:13:57.680505+0000","last_undegraded":"2024-09-10T21:13:57.680505+0000","last_fullsized":"2024-09-10T21:13:57.680505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:41.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654071+0000","last_change":"2024-09-10T21:13:57.688753+0000","last_active":"2024-09-10T21:13:58.654071+0000","last_peered":"2024-09-10T21:13:58.654071+0000","last_clean":"2024-09-10T21:13:58.654071+0000","last_became_active":"2024-09-10T21:13:57.688416+0000","last_became_peered":"2024-09-10T21:13:57.688416+0000","last_unstale":"2024-09-10T21:13:58.654071+0000","last_undegraded":"2024-09-10T21:13:58.654071+0000","last_fullsized":"2024-09-10T21:13:58.654071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:03.676758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674071+0000","last_change":"2024-09-10T21:13:57.670146+0000","last_active":"2024-09-10T21:13:58.674071+0000","last_peered":"2024-09-10T21:13:58.674071+0000","last_clean":"2024-09-10T21:13:58.674071+0000","last_became_active":"2024-09-10T21:13:57.669808+0000","last_became_peered":"2024-09-10T21:13:57.669808+0000","last_unstale":"2024-09-10T21:13:58.674071+0000","last_undegraded":"2024-09-10T21:13:58.674071+0000","last_fullsized":"2024-09-10T21:13:58.674071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:45.864745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674573+0000","last_change":"2024-09-10T21:13:57.675388+0000","last_active":"2024-09-10T21:13:58.674573+0000","last_peered":"2024-09-10T21:13:58.674573+0000","last_clean":"2024-09-10T21:13:58.674573+0000","last_became_active":"2024-09-10T21:13:57.675081+0000","last_became_peered":"2024-09-10T21:13:57.675081+0000","last_unstale":"2024-09-10T21:13:58.674573+0000","last_undegraded":"2024-09-10T21:13:58.674573+0000","last_fullsized":"2024-09-10T21:13:58.674573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:43.169704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691792+0000","last_change":"2024-09-10T21:13:57.670121+0000","last_active":"2024-09-10T21:13:58.691792+0000","last_peered":"2024-09-10T21:13:58.691792+0000","last_clean":"2024-09-10T21:13:58.691792+0000","last_became_active":"2024-09-10T21:13:57.669843+0000","last_became_peered":"2024-09-10T21:13:57.669843+0000","last_unstale":"2024-09-10T21:13:58.691792+0000","last_undegraded":"2024-09-10T21:13:58.691792+0000","last_fullsized":"2024-09-10T21:13:58.691792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:35.478067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691543+0000","last_change":"2024-09-10T21:13:57.664843+0000","last_active":"2024-09-10T21:13:58.691543+0000","last_peered":"2024-09-10T21:13:58.691543+0000","last_clean":"2024-09-10T21:13:58.691543+0000","last_became_active":"2024-09-10T21:13:57.664513+0000","last_became_peered":"2024-09-10T21:13:57.664513+0000","last_unstale":"2024-09-10T21:13:58.691543+0000","last_undegraded":"2024-09-10T21:13:58.691543+0000","last_fullsized":"2024-09-10T21:13:58.691543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:10.771217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.677589+0000","last_change":"2024-09-10T21:13:57.673552+0000","last_active":"2024-09-10T21:13:57.677589+0000","last_peered":"2024-09-10T21:13:57.677589+0000","last_clean":"2024-09-10T21:13:57.677589+0000","last_became_active":"2024-09-10T21:13:57.673319+0000","last_became_peered":"2024-09-10T21:13:57.673319+0000","last_unstale":"2024-09-10T21:13:57.677589+0000","last_undegraded":"2024-09-10T21:13:57.677589+0000","last_fullsized":"2024-09-10T21:13:57.677589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:15.099468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653102+0000","last_change":"2024-09-10T21:13:57.669502+0000","last_active":"2024-09-10T21:13:58.653102+0000","last_peered":"2024-09-10T21:13:58.653102+0000","last_clean":"2024-09-10T21:13:58.653102+0000","last_became_active":"2024-09-10T21:13:57.662958+0000","last_became_peered":"2024-09-10T21:13:57.662958+0000","last_unstale":"2024-09-10T21:13:58.653102+0000","last_undegraded":"2024-09-10T21:13:58.653102+0000","last_fullsized":"2024-09-10T21:13:58.653102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:54.958432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.683653+0000","last_change":"2024-09-10T21:13:57.677331+0000","last_active":"2024-09-10T21:13:57.683653+0000","last_peered":"2024-09-10T21:13:57.683653+0000","last_clean":"2024-09-10T21:13:57.683653+0000","last_became_active":"2024-09-10T21:13:57.676705+0000","last_became_peered":"2024-09-10T21:13:57.676705+0000","last_unstale":"2024-09-10T21:13:57.683653+0000","last_undegraded":"2024-09-10T21:13:57.683653+0000","last_fullsized":"2024-09-10T21:13:57.683653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:47.869632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651612+0000","last_change":"2024-09-10T21:13:57.656607+0000","last_active":"2024-09-10T21:13:58.651612+0000","last_peered":"2024-09-10T21:13:58.651612+0000","last_clean":"2024-09-10T21:13:58.651612+0000","last_became_active":"2024-09-10T21:13:57.656304+0000","last_became_peered":"2024-09-10T21:13:57.656304+0000","last_unstale":"2024-09-10T21:13:58.651612+0000","last_undegraded":"2024-09-10T21:13:58.651612+0000","last_fullsized":"2024-09-10T21:13:58.651612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:22.505181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651475+0000","last_change":"2024-09-10T21:13:57.656632+0000","last_active":"2024-09-10T21:13:58.651475+0000","last_peered":"2024-09-10T21:13:58.651475+0000","last_clean":"2024-09-10T21:13:58.651475+0000","last_became_active":"2024-09-10T21:13:57.656351+0000","last_became_peered":"2024-09-10T21:13:57.656351+0000","last_unstale":"2024-09-10T21:13:58.651475+0000","last_undegraded":"2024-09-10T21:13:58.651475+0000","last_fullsized":"2024-09-10T21:13:58.651475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:49.208104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654355+0000","last_change":"2024-09-10T21:13:57.674210+0000","last_active":"2024-09-10T21:13:58.654355+0000","last_peered":"2024-09-10T21:13:58.654355+0000","last_clean":"2024-09-10T21:13:58.654355+0000","last_became_active":"2024-09-10T21:13:57.673931+0000","last_became_peered":"2024-09-10T21:13:57.673931+0000","last_unstale":"2024-09-10T21:13:58.654355+0000","last_undegraded":"2024-09-10T21:13:58.654355+0000","last_fullsized":"2024-09-10T21:13:58.654355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:42.562833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691757+0000","last_change":"2024-09-10T21:13:56.661701+0000","last_active":"2024-09-10T21:13:58.691757+0000","last_peered":"2024-09-10T21:13:58.691757+0000","last_clean":"2024-09-10T21:13:58.691757+0000","last_became_active":"2024-09-10T21:13:56.661429+0000","last_became_peered":"2024-09-10T21:13:56.661429+0000","last_unstale":"2024-09-10T21:13:58.691757+0000","last_undegraded":"2024-09-10T21:13:58.691757+0000","last_fullsized":"2024-09-10T21:13:58.691757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:19.079723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.679292+0000","last_change":"2024-09-10T21:13:57.675904+0000","last_active":"2024-09-10T21:13:57.679292+0000","last_peered":"2024-09-10T21:13:57.679292+0000","last_clean":"2024-09-10T21:13:57.679292+0000","last_became_active":"2024-09-10T21:13:57.675381+0000","last_became_peered":"2024-09-10T21:13:57.675381+0000","last_unstale":"2024-09-10T21:13:57.679292+0000","last_undegraded":"2024-09-10T21:13:57.679292+0000","last_fullsized":"2024-09-10T21:13:57.679292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:03.773571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652702+0000","last_change":"2024-09-10T21:13:56.649469+0000","last_active":"2024-09-10T21:13:58.652702+0000","last_peered":"2024-09-10T21:13:58.652702+0000","last_clean":"2024-09-10T21:13:58.652702+0000","last_became_active":"2024-09-10T21:13:56.649117+0000","last_became_peered":"2024-09-10T21:13:56.649117+0000","last_unstale":"2024-09-10T21:13:58.652702+0000","last_undegraded":"2024-09-10T21:13:58.652702+0000","last_fullsized":"2024-09-10T21:13:58.652702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:13.367739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654344+0000","last_change":"2024-09-10T21:13:57.672513+0000","last_active":"2024-09-10T21:13:58.654344+0000","last_peered":"2024-09-10T21:13:58.654344+0000","last_clean":"2024-09-10T21:13:58.654344+0000","last_became_active":"2024-09-10T21:13:57.672246+0000","last_became_peered":"2024-09-10T21:13:57.672246+0000","last_unstale":"2024-09-10T21:13:58.654344+0000","last_undegraded":"2024-09-10T21:13:58.654344+0000","last_fullsized":"2024-09-10T21:13:58.654344+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:22.505181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674367+0000","last_change":"2024-09-10T21:13:56.668321+0000","last_active":"2024-09-10T21:13:58.674367+0000","last_peered":"2024-09-10T21:13:58.674367+0000","last_clean":"2024-09-10T21:13:58.674367+0000","last_became_active":"2024-09-10T21:13:56.668033+0000","last_became_peered":"2024-09-10T21:13:56.668033+0000","last_unstale":"2024-09-10T21:13:58.674367+0000","last_undegraded":"2024-09-10T21:13:58.674367+0000","last_fullsized":"2024-09-10T21:13:58.674367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:52.899802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.677439+0000","last_change":"2024-09-10T21:13:57.673335+0000","last_active":"2024-09-10T21:13:57.677439+0000","last_peered":"2024-09-10T21:13:57.677439+0000","last_clean":"2024-09-10T21:13:57.677439+0000","last_became_active":"2024-09-10T21:13:57.673097+0000","last_became_peered":"2024-09-10T21:13:57.673097+0000","last_unstale":"2024-09-10T21:13:57.677439+0000","last_undegraded":"2024-09-10T21:13:57.677439+0000","last_fullsized":"2024-09-10T21:13:57.677439+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:55.041043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653394+0000","last_change":"2024-09-10T21:13:56.668298+0000","last_active":"2024-09-10T21:13:58.653394+0000","last_peered":"2024-09-10T21:13:58.653394+0000","last_clean":"2024-09-10T21:13:58.653394+0000","last_became_active":"2024-09-10T21:13:56.668017+0000","last_became_peered":"2024-09-10T21:13:56.668017+0000","last_unstale":"2024-09-10T21:13:58.653394+0000","last_undegraded":"2024-09-10T21:13:58.653394+0000","last_fullsized":"2024-09-10T21:13:58.653394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:36.389857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653070+0000","last_change":"2024-09-10T21:13:57.671156+0000","last_active":"2024-09-10T21:13:58.653070+0000","last_peered":"2024-09-10T21:13:58.653070+0000","last_clean":"2024-09-10T21:13:58.653070+0000","last_became_active":"2024-09-10T21:13:57.670810+0000","last_became_peered":"2024-09-10T21:13:57.670810+0000","last_unstale":"2024-09-10T21:13:58.653070+0000","last_undegraded":"2024-09-10T21:13:58.653070+0000","last_fullsized":"2024-09-10T21:13:58.653070+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:50.377859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654158+0000","last_change":"2024-09-10T21:13:56.669721+0000","last_active":"2024-09-10T21:13:58.654158+0000","last_peered":"2024-09-10T21:13:58.654158+0000","last_clean":"2024-09-10T21:13:58.654158+0000","last_became_active":"2024-09-10T21:13:56.669450+0000","last_became_peered":"2024-09-10T21:13:56.669450+0000","last_unstale":"2024-09-10T21:13:58.654158+0000","last_undegraded":"2024-09-10T21:13:58.654158+0000","last_fullsized":"2024-09-10T21:13:58.654158+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:34.038173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652674+0000","last_change":"2024-09-10T21:13:57.671315+0000","last_active":"2024-09-10T21:13:58.652674+0000","last_peered":"2024-09-10T21:13:58.652674+0000","last_clean":"2024-09-10T21:13:58.652674+0000","last_became_active":"2024-09-10T21:13:57.670995+0000","last_became_peered":"2024-09-10T21:13:57.670995+0000","last_unstale":"2024-09-10T21:13:58.652674+0000","last_undegraded":"2024-09-10T21:13:58.652674+0000","last_fullsized":"2024-09-10T21:13:58.652674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:52.795683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651803+0000","last_change":"2024-09-10T21:13:56.663217+0000","last_active":"2024-09-10T21:13:58.651803+0000","last_peered":"2024-09-10T21:13:58.651803+0000","last_clean":"2024-09-10T21:13:58.651803+0000","last_became_active":"2024-09-10T21:13:56.662670+0000","last_became_peered":"2024-09-10T21:13:56.662670+0000","last_unstale":"2024-09-10T21:13:58.651803+0000","last_undegraded":"2024-09-10T21:13:58.651803+0000","last_fullsized":"2024-09-10T21:13:58.651803+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:58.126472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.679899+0000","last_change":"2024-09-10T21:13:57.677205+0000","last_active":"2024-09-10T21:13:57.679899+0000","last_peered":"2024-09-10T21:13:57.679899+0000","last_clean":"2024-09-10T21:13:57.679899+0000","last_became_active":"2024-09-10T21:13:57.676882+0000","last_became_peered":"2024-09-10T21:13:57.676882+0000","last_unstale":"2024-09-10T21:13:57.679899+0000","last_undegraded":"2024-09-10T21:13:57.679899+0000","last_fullsized":"2024-09-10T21:13:57.679899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:01.033971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.656260+0000","last_change":"2024-09-10T21:13:56.656577+0000","last_active":"2024-09-10T21:13:57.656260+0000","last_peered":"2024-09-10T21:13:57.656260+0000","last_clean":"2024-09-10T21:13:57.656260+0000","last_became_active":"2024-09-10T21:13:56.656210+0000","last_became_peered":"2024-09-10T21:13:56.656210+0000","last_unstale":"2024-09-10T21:13:57.656260+0000","last_undegraded":"2024-09-10T21:13:57.656260+0000","last_fullsized":"2024-09-10T21:13:57.656260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:47.235368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.684843+0000","last_change":"2024-09-10T21:13:57.672623+0000","last_active":"2024-09-10T21:13:57.684843+0000","last_peered":"2024-09-10T21:13:57.684843+0000","last_clean":"2024-09-10T21:13:57.684843+0000","last_became_active":"2024-09-10T21:13:57.672317+0000","last_became_peered":"2024-09-10T21:13:57.672317+0000","last_unstale":"2024-09-10T21:13:57.684843+0000","last_undegraded":"2024-09-10T21:13:57.684843+0000","last_fullsized":"2024-09-10T21:13:57.684843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:55.041043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.655583+0000","last_change":"2024-09-10T21:13:56.659363+0000","last_active":"2024-09-10T21:13:57.655583+0000","last_peered":"2024-09-10T21:13:57.655583+0000","last_clean":"2024-09-10T21:13:57.655583+0000","last_became_active":"2024-09-10T21:13:56.658325+0000","last_became_peered":"2024-09-10T21:13:56.658325+0000","last_unstale":"2024-09-10T21:13:57.655583+0000","last_undegraded":"2024-09-10T21:13:57.655583+0000","last_fullsized":"2024-09-10T21:13:57.655583+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:33.843989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.687893+0000","last_change":"2024-09-10T21:13:57.683302+0000","last_active":"2024-09-10T21:13:57.687893+0000","last_peered":"2024-09-10T21:13:57.687893+0000","last_clean":"2024-09-10T21:13:57.687893+0000","last_became_active":"2024-09-10T21:13:57.681805+0000","last_became_peered":"2024-09-10T21:13:57.681805+0000","last_unstale":"2024-09-10T21:13:57.687893+0000","last_undegraded":"2024-09-10T21:13:57.687893+0000","last_fullsized":"2024-09-10T21:13:57.687893+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:01.532812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.660466+0000","last_change":"2024-09-10T21:13:56.667222+0000","last_active":"2024-09-10T21:13:57.660466+0000","last_peered":"2024-09-10T21:13:57.660466+0000","last_clean":"2024-09-10T21:13:57.660466+0000","last_became_active":"2024-09-10T21:13:56.666917+0000","last_became_peered":"2024-09-10T21:13:56.666917+0000","last_unstale":"2024-09-10T21:13:57.660466+0000","last_undegraded":"2024-09-10T21:13:57.660466+0000","last_fullsized":"2024-09-10T21:13:57.660466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:18.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.677548+0000","last_change":"2024-09-10T21:13:57.672392+0000","last_active":"2024-09-10T21:13:57.677548+0000","last_peered":"2024-09-10T21:13:57.677548+0000","last_clean":"2024-09-10T21:13:57.677548+0000","last_became_active":"2024-09-10T21:13:57.672119+0000","last_became_peered":"2024-09-10T21:13:57.672119+0000","last_unstale":"2024-09-10T21:13:57.677548+0000","last_undegraded":"2024-09-10T21:13:57.677548+0000","last_fullsized":"2024-09-10T21:13:57.677548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:31.157608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652794+0000","last_change":"2024-09-10T21:13:56.662063+0000","last_active":"2024-09-10T21:13:58.652794+0000","last_peered":"2024-09-10T21:13:58.652794+0000","last_clean":"2024-09-10T21:13:58.652794+0000","last_became_active":"2024-09-10T21:13:56.661748+0000","last_became_peered":"2024-09-10T21:13:56.661748+0000","last_unstale":"2024-09-10T21:13:58.652794+0000","last_undegraded":"2024-09-10T21:13:58.652794+0000","last_fullsized":"2024-09-10T21:13:58.652794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:28.991050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.673977+0000","last_change":"2024-09-10T21:13:57.677141+0000","last_active":"2024-09-10T21:13:58.673977+0000","last_peered":"2024-09-10T21:13:58.673977+0000","last_clean":"2024-09-10T21:13:58.673977+0000","last_became_active":"2024-09-10T21:13:57.672963+0000","last_became_peered":"2024-09-10T21:13:57.672963+0000","last_unstale":"2024-09-10T21:13:58.673977+0000","last_undegraded":"2024-09-10T21:13:58.673977+0000","last_fullsized":"2024-09-10T21:13:58.673977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:47.313402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"80'3","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.691715+0000","last_change":"2024-09-10T21:13:56.669641+0000","last_active":"2024-09-10T21:13:57.691715+0000","last_peered":"2024-09-10T21:13:57.691715+0000","last_clean":"2024-09-10T21:13:57.691715+0000","last_became_active":"2024-09-10T21:13:56.669354+0000","last_became_peered":"2024-09-10T21:13:56.669354+0000","last_unstale":"2024-09-10T21:13:57.691715+0000","last_undegraded":"2024-09-10T21:13:57.691715+0000","last_fullsized":"2024-09-10T21:13:57.691715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.444461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691501+0000","last_change":"2024-09-10T21:13:57.668851+0000","last_active":"2024-09-10T21:13:58.691501+0000","last_peered":"2024-09-10T21:13:58.691501+0000","last_clean":"2024-09-10T21:13:58.691501+0000","last_became_active":"2024-09-10T21:13:57.668223+0000","last_became_peered":"2024-09-10T21:13:57.668223+0000","last_unstale":"2024-09-10T21:13:58.691501+0000","last_undegraded":"2024-09-10T21:13:58.691501+0000","last_fullsized":"2024-09-10T21:13:58.691501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:09.188572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"80'6","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654342+0000","last_change":"2024-09-10T21:13:56.663493+0000","last_active":"2024-09-10T21:13:58.654342+0000","last_peered":"2024-09-10T21:13:58.654342+0000","last_clean":"2024-09-10T21:13:58.654342+0000","last_became_active":"2024-09-10T21:13:56.663206+0000","last_became_peered":"2024-09-10T21:13:56.663206+0000","last_unstale":"2024-09-10T21:13:58.654342+0000","last_undegraded":"2024-09-10T21:13:58.654342+0000","last_fullsized":"2024-09-10T21:13:58.654342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:28.923989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.673921+0000","last_change":"2024-09-10T21:13:57.676164+0000","last_active":"2024-09-10T21:13:58.673921+0000","last_peered":"2024-09-10T21:13:58.673921+0000","last_clean":"2024-09-10T21:13:58.673921+0000","last_became_active":"2024-09-10T21:13:57.675863+0000","last_became_peered":"2024-09-10T21:13:57.675863+0000","last_unstale":"2024-09-10T21:13:58.673921+0000","last_undegraded":"2024-09-10T21:13:58.673921+0000","last_fullsized":"2024-09-10T21:13:58.673921+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:56.243098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653255+0000","last_change":"2024-09-10T21:13:56.657951+0000","last_active":"2024-09-10T21:13:58.653255+0000","last_peered":"2024-09-10T21:13:58.653255+0000","last_clean":"2024-09-10T21:13:58.653255+0000","last_became_active":"2024-09-10T21:13:56.657546+0000","last_became_peered":"2024-09-10T21:13:56.657546+0000","last_unstale":"2024-09-10T21:13:58.653255+0000","last_undegraded":"2024-09-10T21:13:58.653255+0000","last_fullsized":"2024-09-10T21:13:58.653255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:09.190211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.676322+0000","last_change":"2024-09-10T21:13:57.671289+0000","last_active":"2024-09-10T21:13:57.676322+0000","last_peered":"2024-09-10T21:13:57.676322+0000","last_clean":"2024-09-10T21:13:57.676322+0000","last_became_active":"2024-09-10T21:13:57.670996+0000","last_became_peered":"2024-09-10T21:13:57.670996+0000","last_unstale":"2024-09-10T21:13:57.676322+0000","last_undegraded":"2024-09-10T21:13:57.676322+0000","last_fullsized":"2024-09-10T21:13:57.676322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:28:14.065978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652299+0000","last_change":"2024-09-10T21:13:56.656519+0000","last_active":"2024-09-10T21:13:58.652299+0000","last_peered":"2024-09-10T21:13:58.652299+0000","last_clean":"2024-09-10T21:13:58.652299+0000","last_became_active":"2024-09-10T21:13:56.656214+0000","last_became_peered":"2024-09-10T21:13:56.656214+0000","last_unstale":"2024-09-10T21:13:58.652299+0000","last_undegraded":"2024-09-10T21:13:58.652299+0000","last_fullsized":"2024-09-10T21:13:58.652299+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:06.514907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.683522+0000","last_change":"2024-09-10T21:13:57.680707+0000","last_active":"2024-09-10T21:13:57.683522+0000","last_peered":"2024-09-10T21:13:57.683522+0000","last_clean":"2024-09-10T21:13:57.683522+0000","last_became_active":"2024-09-10T21:13:57.676133+0000","last_became_peered":"2024-09-10T21:13:57.676133+0000","last_unstale":"2024-09-10T21:13:57.683522+0000","last_undegraded":"2024-09-10T21:13:57.683522+0000","last_fullsized":"2024-09-10T21:13:57.683522+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:03.773571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651428+0000","last_change":"2024-09-10T21:13:56.651265+0000","last_active":"2024-09-10T21:13:58.651428+0000","last_peered":"2024-09-10T21:13:58.651428+0000","last_clean":"2024-09-10T21:13:58.651428+0000","last_became_active":"2024-09-10T21:13:56.650676+0000","last_became_peered":"2024-09-10T21:13:56.650676+0000","last_unstale":"2024-09-10T21:13:58.651428+0000","last_undegraded":"2024-09-10T21:13:58.651428+0000","last_fullsized":"2024-09-10T21:13:58.651428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:59.285767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.673773+0000","last_change":"2024-09-10T21:13:57.688435+0000","last_active":"2024-09-10T21:13:58.673773+0000","last_peered":"2024-09-10T21:13:58.673773+0000","last_clean":"2024-09-10T21:13:58.673773+0000","last_became_active":"2024-09-10T21:13:57.685659+0000","last_became_peered":"2024-09-10T21:13:57.685659+0000","last_unstale":"2024-09-10T21:13:58.673773+0000","last_undegraded":"2024-09-10T21:13:58.673773+0000","last_fullsized":"2024-09-10T21:13:58.673773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:41.854723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"80'8","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674192+0000","last_change":"2024-09-10T21:13:56.648738+0000","last_active":"2024-09-10T21:13:58.674192+0000","last_peered":"2024-09-10T21:13:58.674192+0000","last_clean":"2024-09-10T21:13:58.674192+0000","last_became_active":"2024-09-10T21:13:56.648029+0000","last_became_peered":"2024-09-10T21:13:56.648029+0000","last_unstale":"2024-09-10T21:13:58.674192+0000","last_undegraded":"2024-09-10T21:13:58.674192+0000","last_fullsized":"2024-09-10T21:13:58.674192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:44.422283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654415+0000","last_change":"2024-09-10T21:13:57.678126+0000","last_active":"2024-09-10T21:13:58.654415+0000","last_peered":"2024-09-10T21:13:58.654415+0000","last_clean":"2024-09-10T21:13:58.654415+0000","last_became_active":"2024-09-10T21:13:57.677835+0000","last_became_peered":"2024-09-10T21:13:57.677835+0000","last_unstale":"2024-09-10T21:13:58.654415+0000","last_undegraded":"2024-09-10T21:13:58.654415+0000","last_fullsized":"2024-09-10T21:13:58.654415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:52.795683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691914+0000","last_change":"2024-09-10T21:13:56.664199+0000","last_active":"2024-09-10T21:13:58.691914+0000","last_peered":"2024-09-10T21:13:58.691914+0000","last_clean":"2024-09-10T21:13:58.691914+0000","last_became_active":"2024-09-10T21:13:56.664007+0000","last_became_peered":"2024-09-10T21:13:56.664007+0000","last_unstale":"2024-09-10T21:13:58.691914+0000","last_undegraded":"2024-09-10T21:13:58.691914+0000","last_fullsized":"2024-09-10T21:13:58.691914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:32.999753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653244+0000","last_change":"2024-09-10T21:13:57.669220+0000","last_active":"2024-09-10T21:13:58.653244+0000","last_peered":"2024-09-10T21:13:58.653244+0000","last_clean":"2024-09-10T21:13:58.653244+0000","last_became_active":"2024-09-10T21:13:57.668984+0000","last_became_peered":"2024-09-10T21:13:57.668984+0000","last_unstale":"2024-09-10T21:13:58.653244+0000","last_undegraded":"2024-09-10T21:13:58.653244+0000","last_fullsized":"2024-09-10T21:13:58.653244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:36.159803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.692188+0000","last_change":"2024-09-10T21:13:56.668547+0000","last_active":"2024-09-10T21:13:57.692188+0000","last_peered":"2024-09-10T21:13:57.692188+0000","last_clean":"2024-09-10T21:13:57.692188+0000","last_became_active":"2024-09-10T21:13:56.668237+0000","last_became_peered":"2024-09-10T21:13:56.668237+0000","last_unstale":"2024-09-10T21:13:57.692188+0000","last_undegraded":"2024-09-10T21:13:57.692188+0000","last_fullsized":"2024-09-10T21:13:57.692188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:18.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654507+0000","last_change":"2024-09-10T21:13:57.677601+0000","last_active":"2024-09-10T21:13:58.654507+0000","last_peered":"2024-09-10T21:13:58.654507+0000","last_clean":"2024-09-10T21:13:58.654507+0000","last_became_active":"2024-09-10T21:13:57.677383+0000","last_became_peered":"2024-09-10T21:13:57.677383+0000","last_unstale":"2024-09-10T21:13:58.654507+0000","last_undegraded":"2024-09-10T21:13:58.654507+0000","last_fullsized":"2024-09-10T21:13:58.654507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:56.313169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.656429+0000","last_change":"2024-09-10T21:13:56.660492+0000","last_active":"2024-09-10T21:13:57.656429+0000","last_peered":"2024-09-10T21:13:57.656429+0000","last_clean":"2024-09-10T21:13:57.656429+0000","last_became_active":"2024-09-10T21:13:56.660178+0000","last_became_peered":"2024-09-10T21:13:56.660178+0000","last_unstale":"2024-09-10T21:13:57.656429+0000","last_undegraded":"2024-09-10T21:13:57.656429+0000","last_fullsized":"2024-09-10T21:13:57.656429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:11.412884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652074+0000","last_change":"2024-09-10T21:13:57.669400+0000","last_active":"2024-09-10T21:13:58.652074+0000","last_peered":"2024-09-10T21:13:58.652074+0000","last_clean":"2024-09-10T21:13:58.652074+0000","last_became_active":"2024-09-10T21:13:57.669025+0000","last_became_peered":"2024-09-10T21:13:57.669025+0000","last_unstale":"2024-09-10T21:13:58.652074+0000","last_undegraded":"2024-09-10T21:13:58.652074+0000","last_fullsized":"2024-09-10T21:13:58.652074+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:22.674883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.692760+0000","last_change":"2024-09-10T21:13:56.660970+0000","last_active":"2024-09-10T21:13:57.692760+0000","last_peered":"2024-09-10T21:13:57.692760+0000","last_clean":"2024-09-10T21:13:57.692760+0000","last_became_active":"2024-09-10T21:13:56.660729+0000","last_became_peered":"2024-09-10T21:13:56.660729+0000","last_unstale":"2024-09-10T21:13:57.692760+0000","last_undegraded":"2024-09-10T21:13:57.692760+0000","last_fullsized":"2024-09-10T21:13:57.692760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:30.444461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691130+0000","last_change":"2024-09-10T21:13:57.668511+0000","last_active":"2024-09-10T21:13:58.691130+0000","last_peered":"2024-09-10T21:13:58.691130+0000","last_clean":"2024-09-10T21:13:58.691130+0000","last_became_active":"2024-09-10T21:13:57.668192+0000","last_became_peered":"2024-09-10T21:13:57.668192+0000","last_unstale":"2024-09-10T21:13:58.691130+0000","last_undegraded":"2024-09-10T21:13:58.691130+0000","last_fullsized":"2024-09-10T21:13:58.691130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:54.301500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652403+0000","last_change":"2024-09-10T21:13:56.660874+0000","last_active":"2024-09-10T21:13:58.652403+0000","last_peered":"2024-09-10T21:13:58.652403+0000","last_clean":"2024-09-10T21:13:58.652403+0000","last_became_active":"2024-09-10T21:13:56.660603+0000","last_became_peered":"2024-09-10T21:13:56.660603+0000","last_unstale":"2024-09-10T21:13:58.652403+0000","last_undegraded":"2024-09-10T21:13:58.652403+0000","last_fullsized":"2024-09-10T21:13:58.652403+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:07.314380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652050+0000","last_change":"2024-09-10T21:13:57.665726+0000","last_active":"2024-09-10T21:13:58.652050+0000","last_peered":"2024-09-10T21:13:58.652050+0000","last_clean":"2024-09-10T21:13:58.652050+0000","last_became_active":"2024-09-10T21:13:57.665385+0000","last_became_peered":"2024-09-10T21:13:57.665385+0000","last_unstale":"2024-09-10T21:13:58.652050+0000","last_undegraded":"2024-09-10T21:13:58.652050+0000","last_fullsized":"2024-09-10T21:13:58.652050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:17.657607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.673794+0000","last_change":"2024-09-10T21:13:56.646406+0000","last_active":"2024-09-10T21:13:58.673794+0000","last_peered":"2024-09-10T21:13:58.673794+0000","last_clean":"2024-09-10T21:13:58.673794+0000","last_became_active":"2024-09-10T21:13:56.646114+0000","last_became_peered":"2024-09-10T21:13:56.646114+0000","last_unstale":"2024-09-10T21:13:58.673794+0000","last_undegraded":"2024-09-10T21:13:58.673794+0000","last_fullsized":"2024-09-10T21:13:58.673794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:11.231979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654093+0000","last_change":"2024-09-10T21:13:57.674009+0000","last_active":"2024-09-10T21:13:58.654093+0000","last_peered":"2024-09-10T21:13:58.654093+0000","last_clean":"2024-09-10T21:13:58.654093+0000","last_became_active":"2024-09-10T21:13:57.672222+0000","last_became_peered":"2024-09-10T21:13:57.672222+0000","last_unstale":"2024-09-10T21:13:58.654093+0000","last_undegraded":"2024-09-10T21:13:58.654093+0000","last_fullsized":"2024-09-10T21:13:58.654093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:54.144078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.656649+0000","last_change":"2024-09-10T21:13:56.660430+0000","last_active":"2024-09-10T21:13:57.656649+0000","last_peered":"2024-09-10T21:13:57.656649+0000","last_clean":"2024-09-10T21:13:57.656649+0000","last_became_active":"2024-09-10T21:13:56.660136+0000","last_became_peered":"2024-09-10T21:13:56.660136+0000","last_unstale":"2024-09-10T21:13:57.656649+0000","last_undegraded":"2024-09-10T21:13:57.656649+0000","last_fullsized":"2024-09-10T21:13:57.656649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:52.177028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691822+0000","last_change":"2024-09-10T21:13:57.675247+0000","last_active":"2024-09-10T21:13:58.691822+0000","last_peered":"2024-09-10T21:13:58.691822+0000","last_clean":"2024-09-10T21:13:58.691822+0000","last_became_active":"2024-09-10T21:13:57.674950+0000","last_became_peered":"2024-09-10T21:13:57.674950+0000","last_unstale":"2024-09-10T21:13:58.691822+0000","last_undegraded":"2024-09-10T21:13:58.691822+0000","last_fullsized":"2024-09-10T21:13:58.691822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:24.803662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.692291+0000","last_change":"2024-09-10T21:13:56.660247+0000","last_active":"2024-09-10T21:13:57.692291+0000","last_peered":"2024-09-10T21:13:57.692291+0000","last_clean":"2024-09-10T21:13:57.692291+0000","last_became_active":"2024-09-10T21:13:56.659836+0000","last_became_peered":"2024-09-10T21:13:56.659836+0000","last_unstale":"2024-09-10T21:13:57.692291+0000","last_undegraded":"2024-09-10T21:13:57.692291+0000","last_fullsized":"2024-09-10T21:13:57.692291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:28:13.059565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691803+0000","last_change":"2024-09-10T21:13:57.666742+0000","last_active":"2024-09-10T21:13:58.691803+0000","last_peered":"2024-09-10T21:13:58.691803+0000","last_clean":"2024-09-10T21:13:58.691803+0000","last_became_active":"2024-09-10T21:13:57.666361+0000","last_became_peered":"2024-09-10T21:13:57.666361+0000","last_unstale":"2024-09-10T21:13:58.691803+0000","last_undegraded":"2024-09-10T21:13:58.691803+0000","last_fullsized":"2024-09-10T21:13:58.691803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:58.029945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"81'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.676033+0000","last_change":"2024-09-10T21:13:56.649564+0000","last_active":"2024-09-10T21:13:58.676033+0000","last_peered":"2024-09-10T21:13:58.676033+0000","last_clean":"2024-09-10T21:13:58.676033+0000","last_became_active":"2024-09-10T21:13:56.649247+0000","last_became_peered":"2024-09-10T21:13:56.649247+0000","last_unstale":"2024-09-10T21:13:58.676033+0000","last_undegraded":"2024-09-10T21:13:58.676033+0000","last_fullsized":"2024-09-10T21:13:58.676033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:32.437051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691976+0000","last_change":"2024-09-10T21:13:57.666799+0000","last_active":"2024-09-10T21:13:58.691976+0000","last_peered":"2024-09-10T21:13:58.691976+0000","last_clean":"2024-09-10T21:13:58.691976+0000","last_became_active":"2024-09-10T21:13:57.666531+0000","last_became_peered":"2024-09-10T21:13:57.666531+0000","last_unstale":"2024-09-10T21:13:58.691976+0000","last_undegraded":"2024-09-10T21:13:58.691976+0000","last_fullsized":"2024-09-10T21:13:58.691976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:32.980731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654562+0000","last_change":"2024-09-10T21:13:56.666102+0000","last_active":"2024-09-10T21:13:58.654562+0000","last_peered":"2024-09-10T21:13:58.654562+0000","last_clean":"2024-09-10T21:13:58.654562+0000","last_became_active":"2024-09-10T21:13:56.665831+0000","last_became_peered":"2024-09-10T21:13:56.665831+0000","last_unstale":"2024-09-10T21:13:58.654562+0000","last_undegraded":"2024-09-10T21:13:58.654562+0000","last_fullsized":"2024-09-10T21:13:58.654562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:58.126472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.653052+0000","last_change":"2024-09-10T21:13:57.681000+0000","last_active":"2024-09-10T21:13:58.653052+0000","last_peered":"2024-09-10T21:13:58.653052+0000","last_clean":"2024-09-10T21:13:58.653052+0000","last_became_active":"2024-09-10T21:13:57.680591+0000","last_became_peered":"2024-09-10T21:13:57.680591+0000","last_unstale":"2024-09-10T21:13:58.653052+0000","last_undegraded":"2024-09-10T21:13:58.653052+0000","last_fullsized":"2024-09-10T21:13:58.653052+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:50.075360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652464+0000","last_change":"2024-09-10T21:13:56.649981+0000","last_active":"2024-09-10T21:13:58.652464+0000","last_peered":"2024-09-10T21:13:58.652464+0000","last_clean":"2024-09-10T21:13:58.652464+0000","last_became_active":"2024-09-10T21:13:56.649711+0000","last_became_peered":"2024-09-10T21:13:56.649711+0000","last_unstale":"2024-09-10T21:13:58.652464+0000","last_undegraded":"2024-09-10T21:13:58.652464+0000","last_fullsized":"2024-09-10T21:13:58.652464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:00.959573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.678403+0000","last_change":"2024-09-10T21:13:57.673209+0000","last_active":"2024-09-10T21:13:57.678403+0000","last_peered":"2024-09-10T21:13:57.678403+0000","last_clean":"2024-09-10T21:13:57.678403+0000","last_became_active":"2024-09-10T21:13:57.672936+0000","last_became_peered":"2024-09-10T21:13:57.672936+0000","last_unstale":"2024-09-10T21:13:57.678403+0000","last_undegraded":"2024-09-10T21:13:57.678403+0000","last_fullsized":"2024-09-10T21:13:57.678403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:41.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.652590+0000","last_change":"2024-09-10T21:13:56.659154+0000","last_active":"2024-09-10T21:13:58.652590+0000","last_peered":"2024-09-10T21:13:58.652590+0000","last_clean":"2024-09-10T21:13:58.652590+0000","last_became_active":"2024-09-10T21:13:56.658628+0000","last_became_peered":"2024-09-10T21:13:56.658628+0000","last_unstale":"2024-09-10T21:13:58.652590+0000","last_undegraded":"2024-09-10T21:13:58.652590+0000","last_fullsized":"2024-09-10T21:13:58.652590+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:22.366656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674092+0000","last_change":"2024-09-10T21:13:57.667627+0000","last_active":"2024-09-10T21:13:58.674092+0000","last_peered":"2024-09-10T21:13:58.674092+0000","last_clean":"2024-09-10T21:13:58.674092+0000","last_became_active":"2024-09-10T21:13:57.667212+0000","last_became_peered":"2024-09-10T21:13:57.667212+0000","last_unstale":"2024-09-10T21:13:58.674092+0000","last_undegraded":"2024-09-10T21:13:58.674092+0000","last_fullsized":"2024-09-10T21:13:58.674092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:22.674883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.651891+0000","last_change":"2024-09-10T21:13:56.662937+0000","last_active":"2024-09-10T21:13:58.651891+0000","last_peered":"2024-09-10T21:13:58.651891+0000","last_clean":"2024-09-10T21:13:58.651891+0000","last_became_active":"2024-09-10T21:13:56.662220+0000","last_became_peered":"2024-09-10T21:13:56.662220+0000","last_unstale":"2024-09-10T21:13:58.651891+0000","last_undegraded":"2024-09-10T21:13:58.651891+0000","last_fullsized":"2024-09-10T21:13:58.651891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:28.923989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.679875+0000","last_change":"2024-09-10T21:13:57.675310+0000","last_active":"2024-09-10T21:13:57.679875+0000","last_peered":"2024-09-10T21:13:57.679875+0000","last_clean":"2024-09-10T21:13:57.679875+0000","last_became_active":"2024-09-10T21:13:57.674398+0000","last_became_peered":"2024-09-10T21:13:57.674398+0000","last_unstale":"2024-09-10T21:13:57.679875+0000","last_undegraded":"2024-09-10T21:13:57.679875+0000","last_fullsized":"2024-09-10T21:13:57.679875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:47.869632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.658286+0000","last_change":"2024-09-10T21:13:56.665686+0000","last_active":"2024-09-10T21:13:57.658286+0000","last_peered":"2024-09-10T21:13:57.658286+0000","last_clean":"2024-09-10T21:13:57.658286+0000","last_became_active":"2024-09-10T21:13:56.665410+0000","last_became_peered":"2024-09-10T21:13:56.665410+0000","last_unstale":"2024-09-10T21:13:57.658286+0000","last_undegraded":"2024-09-10T21:13:57.658286+0000","last_fullsized":"2024-09-10T21:13:57.658286+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:47.725344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.679736+0000","last_change":"2024-09-10T21:13:57.678980+0000","last_active":"2024-09-10T21:13:57.679736+0000","last_peered":"2024-09-10T21:13:57.679736+0000","last_clean":"2024-09-10T21:13:57.679736+0000","last_became_active":"2024-09-10T21:13:57.678655+0000","last_became_peered":"2024-09-10T21:13:57.678655+0000","last_unstale":"2024-09-10T21:13:57.679736+0000","last_undegraded":"2024-09-10T21:13:57.679736+0000","last_fullsized":"2024-09-10T21:13:57.679736+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:42.885937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.654048+0000","last_change":"2024-09-10T21:13:56.669362+0000","last_active":"2024-09-10T21:13:58.654048+0000","last_peered":"2024-09-10T21:13:58.654048+0000","last_clean":"2024-09-10T21:13:58.654048+0000","last_became_active":"2024-09-10T21:13:56.668961+0000","last_became_peered":"2024-09-10T21:13:56.668961+0000","last_unstale":"2024-09-10T21:13:58.654048+0000","last_undegraded":"2024-09-10T21:13:58.654048+0000","last_fullsized":"2024-09-10T21:13:58.654048+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:00.959573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.674086+0000","last_change":"2024-09-10T21:13:57.673095+0000","last_active":"2024-09-10T21:13:58.674086+0000","last_peered":"2024-09-10T21:13:58.674086+0000","last_clean":"2024-09-10T21:13:58.674086+0000","last_became_active":"2024-09-10T21:13:57.671973+0000","last_became_peered":"2024-09-10T21:13:57.671973+0000","last_unstale":"2024-09-10T21:13:58.674086+0000","last_undegraded":"2024-09-10T21:13:58.674086+0000","last_fullsized":"2024-09-10T21:13:58.674086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:56.633325+0000","last_clean_scrub_stamp":"2024-09-10T21:13:56.633325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:40.692031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T21:13:58.691618+0000","last_change":"2024-09-10T21:13:56.658873+0000","last_active":"2024-09-10T21:13:58.691618+0000","last_peered":"2024-09-10T21:13:58.691618+0000","last_clean":"2024-09-10T21:13:58.691618+0000","last_became_active":"2024-09-10T21:13:56.658480+0000","last_became_peered":"2024-09-10T21:13:56.658480+0000","last_unstale":"2024-09-10T21:13:58.691618+0000","last_undegraded":"2024-09-10T21:13:58.691618+0000","last_fullsized":"2024-09-10T21:13:58.691618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:13:55.626913+0000","last_clean_scrub_stamp":"2024-09-10T21:13:55.626913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:42.885620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":213,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T21:13:57.658725+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:13:57.658725+0000","last_peered":"2024-09-10T21:13:57.658725+0000","last_clean":"2024-09-10T21:13:57.658725+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:13:57.658725+0000","last_undegraded":"2024-09-10T21:13:57.658725+0000","last_fullsized":"2024-09-10T21:13:57.658725+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36308,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335532,"statfs":{"total":96636764160,"available":96599584768,"internally_reserved":0,"allocated":983040,"data_stored":412608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1597440,"data_stored":1010250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":991232,"data_stored":418270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509486,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":987136,"data_stored":412610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":974848,"data_stored":411936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1003520,"data_stored":419870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36288,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335552,"statfs":{"total":96636764160,"available":96599605248,"internally_reserved":0,"allocated":962560,"data_stored":405161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542190,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36888,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334952,"statfs":{"total":96636764160,"available":96598990848,"internally_reserved":0,"allocated":1572864,"data_stored":1002394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:14:02.213 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:14:02.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:14:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 -- 172.21.15.16:0/1952408775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f650415ad80 msgr2=0x7f65040b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 --2- 172.21.15.16:0/1952408775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f650415ad80 0x7f65040b3f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009840 tx=0x7f64ec01c6b0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 -- 172.21.15.16:0/1952408775 shutdown_connections 2024-09-10T21:14:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 --2- 172.21.15.16:0/1952408775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f650415ad80 0x7f65040b3f80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 -- 172.21.15.16:0/1952408775 >> 172.21.15.16:0/1952408775 conn(0x7f65040b16a0 msgr2=0x7f65040b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 -- 172.21.15.16:0/1952408775 shutdown_connections 2024-09-10T21:14:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 -- 172.21.15.16:0/1952408775 wait complete. 2024-09-10T21:14:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 Processor -- start 2024-09-10T21:14:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 -- start start 2024-09-10T21:14:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65040b33b0 0x7f650415df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f650415a9a0 0x7f6504150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650415ad80 0x7f6504151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6504168d20 con 0x7f650415ad80 2024-09-10T21:14:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6504168ba0 con 0x7f65040b33b0 2024-09-10T21:14:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f650b000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6504168ea0 con 0x7f650415a9a0 2024-09-10T21:14:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f6509000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650415ad80 0x7f6504151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f6509000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650415ad80 0x7f6504151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59038/0 (socket says 172.21.15.16:59038) 2024-09-10T21:14:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.399+0000 7f6509000640 1 -- 172.21.15.16:0/2579230686 learned_addr learned my addr 172.21.15.16:0/2579230686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f6503e00640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65040b33b0 0x7f650415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f64fbe00640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f650415a9a0 0x7f6504150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f6509000640 1 -- 172.21.15.16:0/2579230686 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f650415a9a0 msgr2=0x7f6504150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f6509000640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f650415a9a0 0x7f6504150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f6509000640 1 -- 172.21.15.16:0/2579230686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65040b33b0 msgr2=0x7f650415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f6509000640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65040b33b0 0x7f650415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f6509000640 1 -- 172.21.15.16:0/2579230686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65040aea70 con 0x7f650415ad80 2024-09-10T21:14:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f64fbe00640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f650415a9a0 0x7f6504150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:14:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f6509000640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650415ad80 0x7f6504151470 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00c7c0 tx=0x7f64fc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f6500c00640 1 -- 172.21.15.16:0/2579230686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc003b80 con 0x7f650415ad80 2024-09-10T21:14:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f6500c00640 1 -- 172.21.15.16:0/2579230686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64fc003d20 con 0x7f650415ad80 2024-09-10T21:14:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f6500c00640 1 -- 172.21.15.16:0/2579230686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00aab0 con 0x7f650415ad80 2024-09-10T21:14:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f650b000640 1 -- 172.21.15.16:0/2579230686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65040aed30 con 0x7f650415ad80 2024-09-10T21:14:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f650b000640 1 -- 172.21.15.16:0/2579230686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6504153ba0 con 0x7f650415ad80 2024-09-10T21:14:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f650b000640 1 -- 172.21.15.16:0/2579230686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f650415e560 con 0x7f650415ad80 2024-09-10T21:14:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.403+0000 7f6500c00640 1 -- 172.21.15.16:0/2579230686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f64fc00ac50 con 0x7f650415ad80 2024-09-10T21:14:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.407+0000 7f6500c00640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f64e00725e0 0x7f64e0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.407+0000 7f6500c00640 1 -- 172.21.15.16:0/2579230686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f64fc0931e0 con 0x7f650415ad80 2024-09-10T21:14:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.407+0000 7f6503e00640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f64e00725e0 0x7f64e0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.407+0000 7f6503e00640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f64e00725e0 0x7f64e0074aa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f64f4001230 tx=0x7f64f4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.411+0000 7f6500c00640 1 -- 172.21.15.16:0/2579230686 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f64fc05e7a0 con 0x7f650415ad80 2024-09-10T21:14:02.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.643+0000 7f650b000640 1 -- 172.21.15.16:0/2579230686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6504050f80 con 0x7f650415ad80 2024-09-10T21:14:02.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.647+0000 7f6500c00640 1 -- 172.21.15.16:0/2579230686 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) ==== 190+0+147 (secure 0 0 0) 0x7f64fc05e0a0 con 0x7f650415ad80 2024-09-10T21:14:02.653 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-10T21:14:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA== 2024-09-10T21:14:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-10T21:14:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-10T21:14:02.654 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:14:02.654 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-10T21:14:02.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.647+0000 7f650b000640 1 -- 172.21.15.16:0/2579230686 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f64e00725e0 msgr2=0x7f64e0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.647+0000 7f650b000640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f64e00725e0 0x7f64e0074aa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f64f4001230 tx=0x7f64f4009040 comp rx=0 tx=0).stop 2024-09-10T21:14:02.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.647+0000 7f650b000640 1 -- 172.21.15.16:0/2579230686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650415ad80 msgr2=0x7f6504151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.647+0000 7f650b000640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650415ad80 0x7f6504151470 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00c7c0 tx=0x7f64fc00cc80 comp rx=0 tx=0).stop 2024-09-10T21:14:02.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.647+0000 7f650b000640 1 -- 172.21.15.16:0/2579230686 shutdown_connections 2024-09-10T21:14:02.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.647+0000 7f650b000640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f64e00725e0 0x7f64e0074aa0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.647+0000 7f650b000640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650415ad80 0x7f6504151470 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.647+0000 7f650b000640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f650415a9a0 0x7f6504150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.647+0000 7f650b000640 1 --2- 172.21.15.16:0/2579230686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65040b33b0 0x7f650415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.647+0000 7f650b000640 1 -- 172.21.15.16:0/2579230686 >> 172.21.15.16:0/2579230686 conn(0x7f65040b16a0 msgr2=0x7f6504072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:02.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.647+0000 7f650b000640 1 -- 172.21.15.16:0/2579230686 shutdown_connections 2024-09-10T21:14:02.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.647+0000 7f650b000640 1 -- 172.21.15.16:0/2579230686 wait complete. 2024-09-10T21:14:02.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:14:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 -- 172.21.15.16:0/3814920869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5815f830 msgr2=0x7f9b580ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 --2- 172.21.15.16:0/3814920869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5815f830 0x7f9b580ca170 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9b5806e0e0 tx=0x7f9b4001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 -- 172.21.15.16:0/3814920869 shutdown_connections 2024-09-10T21:14:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 --2- 172.21.15.16:0/3814920869 >> v2:172.21.15.52:3300/0 conn(0x7f9b580ca6b0 0x7f9b580726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 --2- 172.21.15.16:0/3814920869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5815f830 0x7f9b580ca170 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 -- 172.21.15.16:0/3814920869 >> 172.21.15.16:0/3814920869 conn(0x7f9b580cfb20 msgr2=0x7f9b580cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 -- 172.21.15.16:0/3814920869 shutdown_connections 2024-09-10T21:14:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 -- 172.21.15.16:0/3814920869 wait complete. 2024-09-10T21:14:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 Processor -- start 2024-09-10T21:14:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 -- start start 2024-09-10T21:14:02.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b580ca6b0 0x7f9b581f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9b581599d0 0x7f9b58155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5815f830 0x7f9b580c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b58169360 con 0x7f9b5815f830 2024-09-10T21:14:02.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9b581691e0 con 0x7f9b580ca6b0 2024-09-10T21:14:02.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b5e800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9b581694e0 con 0x7f9b581599d0 2024-09-10T21:14:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b57e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5815f830 0x7f9b580c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b57e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5815f830 0x7f9b580c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59052/0 (socket says 172.21.15.16:59052) 2024-09-10T21:14:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b57e00640 1 -- 172.21.15.16:0/3863078530 learned_addr learned my addr 172.21.15.16:0/3863078530 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b57400640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b580ca6b0 0x7f9b581f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b4fe00640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9b581599d0 0x7f9b58155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b57e00640 1 -- 172.21.15.16:0/3863078530 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9b581599d0 msgr2=0x7f9b58155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b57e00640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9b581599d0 0x7f9b58155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b57e00640 1 -- 172.21.15.16:0/3863078530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b580ca6b0 msgr2=0x7f9b581f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b57e00640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b580ca6b0 0x7f9b581f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b57e00640 1 -- 172.21.15.16:0/3863078530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b58160c90 con 0x7f9b5815f830 2024-09-10T21:14:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.815+0000 7f9b4fe00640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9b581599d0 0x7f9b58155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.819+0000 7f9b57e00640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5815f830 0x7f9b580c82b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000e9f0 tx=0x7f9b5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.819+0000 7f9b4f400640 1 -- 172.21.15.16:0/3863078530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5000cd80 con 0x7f9b5815f830 2024-09-10T21:14:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.819+0000 7f9b4f400640 1 -- 172.21.15.16:0/3863078530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b50004c20 con 0x7f9b5815f830 2024-09-10T21:14:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.819+0000 7f9b5e800640 1 -- 172.21.15.16:0/3863078530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b581650e0 con 0x7f9b5815f830 2024-09-10T21:14:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.819+0000 7f9b4f400640 1 -- 172.21.15.16:0/3863078530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50005020 con 0x7f9b5815f830 2024-09-10T21:14:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.819+0000 7f9b5e800640 1 -- 172.21.15.16:0/3863078530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b581656a0 con 0x7f9b5815f830 2024-09-10T21:14:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.819+0000 7f9b5e800640 1 -- 172.21.15.16:0/3863078530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b580726b0 con 0x7f9b5815f830 2024-09-10T21:14:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.819+0000 7f9b4f400640 1 -- 172.21.15.16:0/3863078530 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9b5000f040 con 0x7f9b5815f830 2024-09-10T21:14:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.819+0000 7f9b4f400640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9b340725e0 0x7f9b34074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.819+0000 7f9b4f400640 1 -- 172.21.15.16:0/3863078530 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f9b50093900 con 0x7f9b5815f830 2024-09-10T21:14:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.819+0000 7f9b57400640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9b340725e0 0x7f9b34074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.823+0000 7f9b57400640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9b340725e0 0x7f9b34074aa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9b48007fd0 tx=0x7f9b48009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:02.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:02.823+0000 7f9b4f400640 1 -- 172.21.15.16:0/3863078530 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9b58160c90 con 0x7f9b5815f830 2024-09-10T21:14:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b5e800640 1 -- 172.21.15.16:0/3863078530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b580741b0 con 0x7f9b5815f830 2024-09-10T21:14:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b4f400640 1 -- 172.21.15.16:0/3863078530 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5222 (secure 0 0 0) 0x7f9b50093070 con 0x7f9b5815f830 2024-09-10T21:14:03.041 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:03.042 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"btime":"2024-09-10T21:14:00:666666+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4199133238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4199133238},{"type":"v1","addr":"172.21.15.52:6835","nonce":4199133238}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2246088651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2246088651},{"type":"v1","addr":"172.21.15.16:6836","nonce":2246088651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1753951486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1753951486},{"type":"v1","addr":"172.21.15.16:6837","nonce":1753951486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:57.054843+0000","modified":"2024-09-10T21:14:00.072852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6276":{"gid":6276,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/943286634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":943286634},{"type":"v1","addr":"172.21.15.52:6834","nonce":943286634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6276,"qdb_cluster":[6276]},"id":6}]} 2024-09-10T21:14:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2024-09-10T21:14:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b5e800640 1 -- 172.21.15.16:0/3863078530 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9b340725e0 msgr2=0x7f9b34074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b5e800640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9b340725e0 0x7f9b34074aa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9b48007fd0 tx=0x7f9b48009040 comp rx=0 tx=0).stop 2024-09-10T21:14:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b5e800640 1 -- 172.21.15.16:0/3863078530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5815f830 msgr2=0x7f9b580c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b5e800640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5815f830 0x7f9b580c82b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000e9f0 tx=0x7f9b5000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b5e800640 1 -- 172.21.15.16:0/3863078530 shutdown_connections 2024-09-10T21:14:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b5e800640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9b340725e0 0x7f9b34074aa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b5e800640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5815f830 0x7f9b580c82b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b5e800640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9b581599d0 0x7f9b58155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b5e800640 1 --2- 172.21.15.16:0/3863078530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b580ca6b0 0x7f9b581f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b5e800640 1 -- 172.21.15.16:0/3863078530 >> 172.21.15.16:0/3863078530 conn(0x7f9b580cfb20 msgr2=0x7f9b580b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b5e800640 1 -- 172.21.15.16:0/3863078530 shutdown_connections 2024-09-10T21:14:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.035+0000 7f9b5e800640 1 -- 172.21.15.16:0/3863078530 wait complete. 2024-09-10T21:14:03.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:13:57.054843+0000', 'modified': '2024-09-10T21:14:00.072852+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6276}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6276': {'gid': 6276, 'name': 'b', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.52:6834/943286634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6832', 'nonce': 943286634}, {'type': 'v1', 'addr': '172.21.15.52:6834', 'nonce': 943286634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6276, 'qdb_cluster': [6276]} 2024-09-10T21:14:03.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:14:03.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T21:14:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.611+0000 7fc28be00640 1 -- 172.21.15.16:0/3240653307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28c0b1c10 msgr2=0x7fc28c0cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.611+0000 7fc28be00640 1 --2- 172.21.15.16:0/3240653307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28c0b1c10 0x7fc28c0cd580 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0735e0 tx=0x7fc28401d750 comp rx=0 tx=0).stop 2024-09-10T21:14:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.611+0000 7fc28be00640 1 -- 172.21.15.16:0/3240653307 shutdown_connections 2024-09-10T21:14:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.611+0000 7fc28be00640 1 --2- 172.21.15.16:0/3240653307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28c0b1c10 0x7fc28c0cd580 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.611+0000 7fc28be00640 1 --2- 172.21.15.16:0/3240653307 >> v2:172.21.15.52:3300/0 conn(0x7fc28c0cfa40 0x7fc28c0cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.611+0000 7fc28be00640 1 -- 172.21.15.16:0/3240653307 >> 172.21.15.16:0/3240653307 conn(0x7fc28c0d1c10 msgr2=0x7fc28c0d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.611+0000 7fc28be00640 1 -- 172.21.15.16:0/3240653307 shutdown_connections 2024-09-10T21:14:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.611+0000 7fc28be00640 1 -- 172.21.15.16:0/3240653307 wait complete. 2024-09-10T21:14:03.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28be00640 1 Processor -- start 2024-09-10T21:14:03.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28be00640 1 -- start start 2024-09-10T21:14:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc28c0b1c10 0x7fc28c0b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28be00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc28c0cdca0 0x7fc28c0b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28c0cfa40 0x7fc28c15e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c0cee20 con 0x7fc28c0cfa40 2024-09-10T21:14:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28be00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c0ceca0 con 0x7fc28c0b1c10 2024-09-10T21:14:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28be00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc28c0cefa0 con 0x7fc28c0cdca0 2024-09-10T21:14:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28c0cfa40 0x7fc28c15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28c0cfa40 0x7fc28c15e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59074/0 (socket says 172.21.15.16:59074) 2024-09-10T21:14:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28b400640 1 -- 172.21.15.16:0/146994766 learned_addr learned my addr 172.21.15.16:0/146994766 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28aa00640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc28c0b1c10 0x7fc28c0b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc283e00640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc28c0cdca0 0x7fc28c0b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28b400640 1 -- 172.21.15.16:0/146994766 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc28c0cdca0 msgr2=0x7fc28c0b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28b400640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc28c0cdca0 0x7fc28c0b7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28b400640 1 -- 172.21.15.16:0/146994766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc28c0b1c10 msgr2=0x7fc28c0b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28b400640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc28c0b1c10 0x7fc28c0b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28b400640 1 -- 172.21.15.16:0/146994766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c0b57c0 con 0x7fc28c0cfa40 2024-09-10T21:14:03.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc28b400640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28c0cfa40 0x7fc28c15e680 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc284009bd0 tx=0x7fc2840a5990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.615+0000 7fc282a00640 1 -- 172.21.15.16:0/146994766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2840bc070 con 0x7fc28c0cfa40 2024-09-10T21:14:03.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.619+0000 7fc282a00640 1 -- 172.21.15.16:0/146994766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2840043b0 con 0x7fc28c0cfa40 2024-09-10T21:14:03.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.619+0000 7fc282a00640 1 -- 172.21.15.16:0/146994766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2840b2040 con 0x7fc28c0cfa40 2024-09-10T21:14:03.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.619+0000 7fc28be00640 1 -- 172.21.15.16:0/146994766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c0b5aa0 con 0x7fc28c0cfa40 2024-09-10T21:14:03.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.619+0000 7fc28be00640 1 -- 172.21.15.16:0/146994766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c0b6060 con 0x7fc28c0cfa40 2024-09-10T21:14:03.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.619+0000 7fc282a00640 1 -- 172.21.15.16:0/146994766 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc2840b5070 con 0x7fc28c0cfa40 2024-09-10T21:14:03.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.619+0000 7fc282a00640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc268072610 0x7fc268074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc282a00640 1 -- 172.21.15.16:0/146994766 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fc28412ce60 con 0x7fc28c0cfa40 2024-09-10T21:14:03.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 Processor -- start 2024-09-10T21:14:03.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc28aa00640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc268072610 0x7fc268074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 -- start start 2024-09-10T21:14:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc28aa00640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc268072610 0x7fc268074ad0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0098f0 tx=0x7fc27c006f90 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e869d0 0x55f632e86db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e872f0 0x55f632e3a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 0x55f632e3b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f632d5b3f0 con 0x55f632e3ae10 2024-09-10T21:14:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f632c98b20 con 0x55f632e869d0 2024-09-10T21:14:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55f632c87f50 con 0x55f632e872f0 2024-09-10T21:14:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc28b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 0x55f632e3b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc28b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 0x55f632e3b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59080/0 (socket says 172.21.15.16:59080) 2024-09-10T21:14:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc28b400640 1 -- 172.21.15.16:0/4031804526 learned_addr learned my addr 172.21.15.16:0/4031804526 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc283e00640 1 --2- 172.21.15.16:0/4031804526 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e872f0 0x55f632e3a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc28aa00640 1 --2- 172.21.15.16:0/4031804526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e869d0 0x55f632e86db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc28b400640 1 -- 172.21.15.16:0/4031804526 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e872f0 msgr2=0x55f632e3a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc28b400640 1 --2- 172.21.15.16:0/4031804526 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e872f0 0x55f632e3a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc28b400640 1 -- 172.21.15.16:0/4031804526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e869d0 msgr2=0x55f632e86db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc28b400640 1 --2- 172.21.15.16:0/4031804526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e869d0 0x55f632e86db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc28b400640 1 -- 172.21.15.16:0/4031804526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f632c4f590 con 0x55f632e3ae10 2024-09-10T21:14:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc28b400640 1 --2- 172.21.15.16:0/4031804526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 0x55f632e3b1f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc2840f7e70 tx=0x7fc2840f7cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6232512e224dca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc27b400640 1 -- 172.21.15.16:0/4031804526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2840f79f0 con 0x55f632e3ae10 2024-09-10T21:14:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc27b400640 1 -- 172.21.15.16:0/4031804526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2840fc910 con 0x55f632e3ae10 2024-09-10T21:14:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc27b400640 1 -- 172.21.15.16:0/4031804526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284101330 con 0x55f632e3ae10 2024-09-10T21:14:03.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 -- 172.21.15.16:0/4031804526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 msgr2=0x55f632e3b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 --2- 172.21.15.16:0/4031804526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 0x55f632e3b1f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc2840f7e70 tx=0x7fc2840f7cb0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 -- 172.21.15.16:0/4031804526 shutdown_connections 2024-09-10T21:14:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 --2- 172.21.15.16:0/4031804526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 0x55f632e3b1f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 --2- 172.21.15.16:0/4031804526 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e872f0 0x55f632e3a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 --2- 172.21.15.16:0/4031804526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e869d0 0x55f632e86db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 -- 172.21.15.16:0/4031804526 >> 172.21.15.16:0/4031804526 conn(0x55f632d24130 msgr2=0x55f632e69e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 -- 172.21.15.16:0/4031804526 shutdown_connections 2024-09-10T21:14:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 -- 172.21.15.16:0/4031804526 wait complete. 2024-09-10T21:14:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 -- start start 2024-09-10T21:14:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.623+0000 7fc294c73000 1 Processor -- start 2024-09-10T21:14:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e3ae10 0x55f632e676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 0x55f632e67e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 0x55f632e683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f632c225d0 con 0x55f632e872f0 2024-09-10T21:14:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f632c8bd60 con 0x55f632e3ae10 2024-09-10T21:14:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55f632c2e500 con 0x55f632e869d0 2024-09-10T21:14:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 10 client.0 _ll_register_callbacks cb 0x55f632c32810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc28b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 0x55f632e683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc28b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 0x55f632e683b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59094/0 (socket says 172.21.15.16:59094) 2024-09-10T21:14:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc28b400640 1 -- 172.21.15.16:0/1302203956 learned_addr learned my addr 172.21.15.16:0/1302203956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc283e00640 1 --2- 172.21.15.16:0/1302203956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 0x55f632e67e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc28aa00640 1 --2- 172.21.15.16:0/1302203956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e3ae10 0x55f632e676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc28b400640 1 -- 172.21.15.16:0/1302203956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 msgr2=0x55f632e67e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc28b400640 1 --2- 172.21.15.16:0/1302203956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 0x55f632e67e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc28b400640 1 -- 172.21.15.16:0/1302203956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e3ae10 msgr2=0x55f632e676a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc28b400640 1 --2- 172.21.15.16:0/1302203956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e3ae10 0x55f632e676a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc28b400640 1 -- 172.21.15.16:0/1302203956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f632c26170 con 0x55f632e872f0 2024-09-10T21:14:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc28b400640 1 --2- 172.21.15.16:0/1302203956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 0x55f632e683b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc2840987b0 tx=0x7fc2840f7ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267e9284e590d295 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc26fe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:14:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc26fe00640 1 -- 172.21.15.16:0/1302203956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284101330 con 0x55f632e872f0 2024-09-10T21:14:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc26fe00640 1 -- 172.21.15.16:0/1302203956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fc284125bd0 con 0x55f632e872f0 2024-09-10T21:14:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 1 -- 172.21.15.16:0/1302203956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f632c200f0 con 0x55f632e872f0 2024-09-10T21:14:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc26fe00640 1 -- 172.21.15.16:0/1302203956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc2840fc910 con 0x55f632e872f0 2024-09-10T21:14:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 10 client.6399 fetch_fsmap learned FSMap version 49 2024-09-10T21:14:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 1 -- 172.21.15.16:0/1302203956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55f632d0cef0 con 0x55f632e872f0 2024-09-10T21:14:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc26fe00640 1 -- 172.21.15.16:0/1302203956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f632c200f0 con 0x55f632e872f0 2024-09-10T21:14:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc26fe00640 1 -- 172.21.15.16:0/1302203956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) ==== 3502+0+0 (secure 0 0 0) 0x7fc28411a030 con 0x55f632e872f0 2024-09-10T21:14:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 10 client.6399 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:14:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 10 client.6399 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:14:03.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 20 client.6399 populate_metadata read hostname 'smithi016' 2024-09-10T21:14:03.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 1 --2- 172.21.15.16:0/1302203956 >> [v2:172.21.15.52:6832/943286634,v1:172.21.15.52:6834/943286634] conn(0x55f632e690f0 0x55f632e98c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 4 client.6399 mds_command: new command op to 6276 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:03.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc294c73000 1 -- 172.21.15.16:0/1302203956 --> [v2:172.21.15.52:6832/943286634,v1:172.21.15.52:6834/943286634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f632c9aa70 con 0x55f632e690f0 2024-09-10T21:14:03.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc28aa00640 1 --2- 172.21.15.16:0/1302203956 >> [v2:172.21.15.52:6832/943286634,v1:172.21.15.52:6834/943286634] conn(0x55f632e690f0 0x55f632e98c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc28aa00640 1 --2- 172.21.15.16:0/1302203956 >> [v2:172.21.15.52:6832/943286634,v1:172.21.15.52:6834/943286634] conn(0x55f632e690f0 0x55f632e98c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe943ab270e6265e server_cookie=4bd9f7cb76d017ba in_seq=0 out_seq=0 2024-09-10T21:14:03.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.627+0000 7fc26fe00640 10 client.6399 ms_handle_connect on v2:172.21.15.52:6832/943286634 2024-09-10T21:14:03.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc26fe00640 1 -- 172.21.15.16:0/1302203956 <== mds.0 v2:172.21.15.52:6832/943286634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f632c9aa70 con 0x55f632e690f0 2024-09-10T21:14:03.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc26fe00640 10 client.6399 handle_command_reply: tid=0 2024-09-10T21:14:03.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc294c73000 1 client.6399 shutdown 2024-09-10T21:14:03.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc294c73000 1 -- 172.21.15.16:0/1302203956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 msgr2=0x55f632e683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc294c73000 1 --2- 172.21.15.16:0/1302203956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 0x55f632e683b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc2840987b0 tx=0x7fc2840f7ee0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc294c73000 1 -- 172.21.15.16:0/1302203956 shutdown_connections 2024-09-10T21:14:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc294c73000 1 --2- 172.21.15.16:0/1302203956 >> [v2:172.21.15.52:6832/943286634,v1:172.21.15.52:6834/943286634] conn(0x55f632e690f0 0x55f632e98c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc294c73000 1 --2- 172.21.15.16:0/1302203956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 0x55f632e683b0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc294c73000 1 --2- 172.21.15.16:0/1302203956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 0x55f632e67e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc294c73000 1 --2- 172.21.15.16:0/1302203956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e3ae10 0x55f632e676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc26fe00640 0 client.6399 ms_handle_reset on v2:172.21.15.52:6832/943286634 2024-09-10T21:14:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc294c73000 1 -- 172.21.15.16:0/1302203956 >> 172.21.15.16:0/1302203956 conn(0x55f632d24130 msgr2=0x55f632e39fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc294c73000 1 -- 172.21.15.16:0/1302203956 shutdown_connections 2024-09-10T21:14:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc294c73000 1 -- 172.21.15.16:0/1302203956 wait complete. 2024-09-10T21:14:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.631+0000 7fc294c73000 20 client.6399 trim_cache size 0 max 16384 2024-09-10T21:14:03.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 Processor -- start 2024-09-10T21:14:03.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 -- start start 2024-09-10T21:14:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 0x55f632e85cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 0x55f632e86200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e872f0 0x55f632e98c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c202f30 con 0x55f632e3ae10 2024-09-10T21:14:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c15d460 con 0x55f632e872f0 2024-09-10T21:14:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc28c15dab0 con 0x55f632e869d0 2024-09-10T21:14:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc28aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 0x55f632e85cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc28aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 0x55f632e85cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59102/0 (socket says 172.21.15.16:59102) 2024-09-10T21:14:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc28aa00640 1 -- 172.21.15.16:0/1963138683 learned_addr learned my addr 172.21.15.16:0/1963138683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc28aa00640 1 -- 172.21.15.16:0/1963138683 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 msgr2=0x55f632e86200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc28aa00640 1 --2- 172.21.15.16:0/1963138683 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 0x55f632e86200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc28aa00640 1 -- 172.21.15.16:0/1963138683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e872f0 msgr2=0x55f632e98c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc28aa00640 1 --2- 172.21.15.16:0/1963138683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e872f0 0x55f632e98c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc28aa00640 1 -- 172.21.15.16:0/1963138683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f632e68720 con 0x55f632e3ae10 2024-09-10T21:14:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc28aa00640 1 --2- 172.21.15.16:0/1963138683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 0x55f632e85cc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55f632c99700 tx=0x7fc27c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108ddb14e01d65b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc27aa00640 1 -- 172.21.15.16:0/1963138683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c01b040 con 0x55f632e3ae10 2024-09-10T21:14:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc27aa00640 1 -- 172.21.15.16:0/1963138683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc27c02e070 con 0x55f632e3ae10 2024-09-10T21:14:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc27aa00640 1 -- 172.21.15.16:0/1963138683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c015990 con 0x55f632e3ae10 2024-09-10T21:14:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 -- 172.21.15.16:0/1963138683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 msgr2=0x55f632e85cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 --2- 172.21.15.16:0/1963138683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 0x55f632e85cc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55f632c99700 tx=0x7fc27c015000 comp rx=0 tx=0).stop 2024-09-10T21:14:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 -- 172.21.15.16:0/1963138683 shutdown_connections 2024-09-10T21:14:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 --2- 172.21.15.16:0/1963138683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e872f0 0x55f632e98c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 --2- 172.21.15.16:0/1963138683 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 0x55f632e86200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 --2- 172.21.15.16:0/1963138683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e3ae10 0x55f632e85cc0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 -- 172.21.15.16:0/1963138683 >> 172.21.15.16:0/1963138683 conn(0x55f632d24130 msgr2=0x55f632e855b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 -- 172.21.15.16:0/1963138683 shutdown_connections 2024-09-10T21:14:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 -- 172.21.15.16:0/1963138683 wait complete. 2024-09-10T21:14:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 -- start start 2024-09-10T21:14:03.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.651+0000 7fc294c73000 1 Processor -- start 2024-09-10T21:14:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e3ae10 0x55f632efd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 0x55f632efd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 0x55f632efdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f632c225d0 con 0x55f632e872f0 2024-09-10T21:14:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f632c53ec0 con 0x55f632e3ae10 2024-09-10T21:14:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55f632c8bd60 con 0x55f632e869d0 2024-09-10T21:14:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 10 client.0 _ll_register_callbacks cb 0x7fc2840f4620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc28b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 0x55f632efdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc28b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 0x55f632efdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59118/0 (socket says 172.21.15.16:59118) 2024-09-10T21:14:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc28b400640 1 -- 172.21.15.16:0/3849387954 learned_addr learned my addr 172.21.15.16:0/3849387954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc28aa00640 1 --2- 172.21.15.16:0/3849387954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e3ae10 0x55f632efd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc283e00640 1 --2- 172.21.15.16:0/3849387954 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 0x55f632efd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc28b400640 1 -- 172.21.15.16:0/3849387954 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 msgr2=0x55f632efd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc28b400640 1 --2- 172.21.15.16:0/3849387954 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 0x55f632efd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc28b400640 1 -- 172.21.15.16:0/3849387954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e3ae10 msgr2=0x55f632efd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc28b400640 1 --2- 172.21.15.16:0/3849387954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e3ae10 0x55f632efd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc28b400640 1 -- 172.21.15.16:0/3849387954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f632ce3d90 con 0x55f632e872f0 2024-09-10T21:14:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc28b400640 1 --2- 172.21.15.16:0/3849387954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 0x55f632efdaa0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc2840b0040 tx=0x7fc284101210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad36e70eaa1df48e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc26fe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:14:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc26fe00640 1 -- 172.21.15.16:0/3849387954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284101330 con 0x55f632e872f0 2024-09-10T21:14:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc26fe00640 1 -- 172.21.15.16:0/3849387954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fc2840f3a40 con 0x55f632e872f0 2024-09-10T21:14:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 1 -- 172.21.15.16:0/3849387954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc27c02e070 con 0x55f632e872f0 2024-09-10T21:14:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc26fe00640 1 -- 172.21.15.16:0/3849387954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc2840f7ee0 con 0x55f632e872f0 2024-09-10T21:14:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 10 client.6405 fetch_fsmap learned FSMap version 49 2024-09-10T21:14:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 1 -- 172.21.15.16:0/3849387954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55f632c4ea90 con 0x55f632e872f0 2024-09-10T21:14:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc26fe00640 1 -- 172.21.15.16:0/3849387954 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc27c02e070 con 0x55f632e872f0 2024-09-10T21:14:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc26fe00640 1 -- 172.21.15.16:0/3849387954 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 49) ==== 3502+0+0 (secure 0 0 0) 0x7fc28411a070 con 0x55f632e872f0 2024-09-10T21:14:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 10 client.6405 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:14:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 10 client.6405 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:14:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 20 client.6405 populate_metadata read hostname 'smithi016' 2024-09-10T21:14:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 1 --2- 172.21.15.16:0/3849387954 >> [v2:172.21.15.52:6832/943286634,v1:172.21.15.52:6834/943286634] conn(0x55f632e9b450 0x55f632e9b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 4 client.6405 mds_command: new command op to 6276 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:14:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 1 -- 172.21.15.16:0/3849387954 --> [v2:172.21.15.52:6832/943286634,v1:172.21.15.52:6834/943286634] -- command(tid 0: {"prefix": "status"}) -- 0x55f632c96520 con 0x55f632e9b450 2024-09-10T21:14:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc28aa00640 1 --2- 172.21.15.16:0/3849387954 >> [v2:172.21.15.52:6832/943286634,v1:172.21.15.52:6834/943286634] conn(0x55f632e9b450 0x55f632e9b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc28aa00640 1 --2- 172.21.15.16:0/3849387954 >> [v2:172.21.15.52:6832/943286634,v1:172.21.15.52:6834/943286634] conn(0x55f632e9b450 0x55f632e9b830 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7eb8f151452db19 server_cookie=6b77f116710bea7c in_seq=0 out_seq=0 2024-09-10T21:14:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc26fe00640 10 client.6405 ms_handle_connect on v2:172.21.15.52:6832/943286634 2024-09-10T21:14:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc26fe00640 1 -- 172.21.15.16:0/3849387954 <== mds.0 v2:172.21.15.52:6832/943286634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55f632c96520 con 0x55f632e9b450 2024-09-10T21:14:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc26fe00640 10 client.6405 handle_command_reply: tid=0 2024-09-10T21:14:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 1 client.6405 shutdown 2024-09-10T21:14:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 1 -- 172.21.15.16:0/3849387954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 msgr2=0x55f632efdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.655+0000 7fc294c73000 1 --2- 172.21.15.16:0/3849387954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 0x55f632efdaa0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc2840b0040 tx=0x7fc284101210 comp rx=0 tx=0).stop 2024-09-10T21:14:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc294c73000 1 -- 172.21.15.16:0/3849387954 shutdown_connections 2024-09-10T21:14:03.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc294c73000 1 --2- 172.21.15.16:0/3849387954 >> [v2:172.21.15.52:6832/943286634,v1:172.21.15.52:6834/943286634] conn(0x55f632e9b450 0x55f632e9b830 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc294c73000 1 --2- 172.21.15.16:0/3849387954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f632e872f0 0x55f632efdaa0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc294c73000 1 --2- 172.21.15.16:0/3849387954 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55f632e869d0 0x55f632efd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc26fe00640 0 client.6405 ms_handle_reset on v2:172.21.15.52:6832/943286634 2024-09-10T21:14:03.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc294c73000 1 --2- 172.21.15.16:0/3849387954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f632e3ae10 0x55f632efd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc294c73000 1 -- 172.21.15.16:0/3849387954 >> 172.21.15.16:0/3849387954 conn(0x55f632d24130 msgr2=0x55f632e85dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc294c73000 1 -- 172.21.15.16:0/3849387954 shutdown_connections 2024-09-10T21:14:03.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc294c73000 1 -- 172.21.15.16:0/3849387954 wait complete. 2024-09-10T21:14:03.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc294c73000 20 client.6405 trim_cache size 0 max 16384 2024-09-10T21:14:03.669 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:14:03.670 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "eb19be60-6d8f-4816-b778-3d7d6698304d", 2024-09-10T21:14:03.670 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-10T21:14:03.670 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6276, 2024-09-10T21:14:03.670 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-10T21:14:03.670 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-10T21:14:03.670 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-10T21:14:03.670 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.0008654269999999, 2024-09-10T21:14:03.670 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 49, 2024-09-10T21:14:03.670 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-10T21:14:03.670 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 81, 2024-09-10T21:14:03.670 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 10.487764898 2024-09-10T21:14:03.670 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:14:03.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc28be00640 1 -- 172.21.15.16:0/146994766 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc268072610 msgr2=0x7fc268074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc28be00640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc268072610 0x7fc268074ad0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0098f0 tx=0x7fc27c006f90 comp rx=0 tx=0).stop 2024-09-10T21:14:03.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc28be00640 1 -- 172.21.15.16:0/146994766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28c0cfa40 msgr2=0x7fc28c15e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc28be00640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28c0cfa40 0x7fc28c15e680 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc284009bd0 tx=0x7fc2840a5990 comp rx=0 tx=0).stop 2024-09-10T21:14:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc28be00640 1 -- 172.21.15.16:0/146994766 shutdown_connections 2024-09-10T21:14:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc28be00640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc268072610 0x7fc268074ad0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc28be00640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28c0cfa40 0x7fc28c15e680 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc28be00640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc28c0cdca0 0x7fc28c0b7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc28be00640 1 --2- 172.21.15.16:0/146994766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc28c0b1c10 0x7fc28c0b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc28be00640 1 -- 172.21.15.16:0/146994766 >> 172.21.15.16:0/146994766 conn(0x7fc28c0d1c10 msgr2=0x7fc28c0d22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc28be00640 1 -- 172.21.15.16:0/146994766 shutdown_connections 2024-09-10T21:14:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.659+0000 7fc28be00640 1 -- 172.21.15.16:0/146994766 wait complete. 2024-09-10T21:14:03.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:14:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.823+0000 7f1972200640 1 -- 172.21.15.16:0/3638277462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f196c0b3200 msgr2=0x7f196c0afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.823+0000 7f1972200640 1 --2- 172.21.15.16:0/3638277462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f196c0b3200 0x7f196c0afae0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f196c0b3810 tx=0x7f195401c880 comp rx=0 tx=0).stop 2024-09-10T21:14:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.823+0000 7f1972200640 1 -- 172.21.15.16:0/3638277462 shutdown_connections 2024-09-10T21:14:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.823+0000 7f1972200640 1 --2- 172.21.15.16:0/3638277462 >> v2:172.21.15.52:3300/0 conn(0x7f196c0d0890 0x7f196c0b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.823+0000 7f1972200640 1 --2- 172.21.15.16:0/3638277462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f196c0b3200 0x7f196c0afae0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.823+0000 7f1972200640 1 -- 172.21.15.16:0/3638277462 >> 172.21.15.16:0/3638277462 conn(0x7f196c0d1d80 msgr2=0x7f196c0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.823+0000 7f1972200640 1 -- 172.21.15.16:0/3638277462 shutdown_connections 2024-09-10T21:14:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.823+0000 7f1972200640 1 -- 172.21.15.16:0/3638277462 wait complete. 2024-09-10T21:14:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1972200640 1 Processor -- start 2024-09-10T21:14:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1972200640 1 -- start start 2024-09-10T21:14:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1972200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f196c0d0890 0x7f196c06f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1972200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f196c15f450 0x7f196c0c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1972200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f196c151270 0x7f196c15c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1972200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f196c168bd0 con 0x7f196c15f450 2024-09-10T21:14:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1972200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f196c168d50 con 0x7f196c151270 2024-09-10T21:14:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1972200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f196c168ed0 con 0x7f196c0d0890 2024-09-10T21:14:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1962a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f196c15f450 0x7f196c0c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1962a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f196c15f450 0x7f196c0c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59140/0 (socket says 172.21.15.16:59140) 2024-09-10T21:14:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1962a00640 1 -- 172.21.15.16:0/1235571751 learned_addr learned my addr 172.21.15.16:0/1235571751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f196b400640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f196c0d0890 0x7f196c06f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f196be00640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f196c151270 0x7f196c15c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1962a00640 1 -- 172.21.15.16:0/1235571751 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f196c0d0890 msgr2=0x7f196c06f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1962a00640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f196c0d0890 0x7f196c06f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1962a00640 1 -- 172.21.15.16:0/1235571751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f196c151270 msgr2=0x7f196c15c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1962a00640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f196c151270 0x7f196c15c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1962a00640 1 -- 172.21.15.16:0/1235571751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c06eae0 con 0x7f196c15f450 2024-09-10T21:14:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1962a00640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f196c15f450 0x7f196c0c90c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f19540b0040 tx=0x7f19540a5720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1963e00640 1 -- 172.21.15.16:0/1235571751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19540b2040 con 0x7f196c15f450 2024-09-10T21:14:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1963e00640 1 -- 172.21.15.16:0/1235571751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1954005070 con 0x7f196c15f450 2024-09-10T21:14:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1963e00640 1 -- 172.21.15.16:0/1235571751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19540044f0 con 0x7f196c15f450 2024-09-10T21:14:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1972200640 1 -- 172.21.15.16:0/1235571751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196c0b4f50 con 0x7f196c15f450 2024-09-10T21:14:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1972200640 1 -- 172.21.15.16:0/1235571751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196c0b5510 con 0x7f196c15f450 2024-09-10T21:14:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.827+0000 7f1972200640 1 -- 172.21.15.16:0/1235571751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196c0cfc80 con 0x7f196c15f450 2024-09-10T21:14:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.831+0000 7f1963e00640 1 -- 172.21.15.16:0/1235571751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f19540054c0 con 0x7f196c15f450 2024-09-10T21:14:03.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.831+0000 7f1963e00640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1948072610 0x7f1948074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.831+0000 7f1963e00640 1 -- 172.21.15.16:0/1235571751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f195412cd10 con 0x7f196c15f450 2024-09-10T21:14:03.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.831+0000 7f196b400640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1948072610 0x7f1948074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.831+0000 7f196b400640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1948072610 0x7f1948074ad0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f195c0098e0 tx=0x7f195c009060 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:03.835+0000 7f1963e00640 1 -- 172.21.15.16:0/1235571751 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1954132020 con 0x7f196c15f450 2024-09-10T21:14:04.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.047+0000 7f1972200640 1 -- 172.21.15.16:0/1235571751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f196c074ba0 con 0x7f196c15f450 2024-09-10T21:14:04.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.047+0000 7f1963e00640 1 -- 172.21.15.16:0/1235571751 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) ==== 53+0+73 (secure 0 0 0) 0x7f1954016470 con 0x7f196c15f450 2024-09-10T21:14:04.055 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:14:04.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.051+0000 7f1972200640 1 -- 172.21.15.16:0/1235571751 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1948072610 msgr2=0x7f1948074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:04.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.051+0000 7f1972200640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1948072610 0x7f1948074ad0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f195c0098e0 tx=0x7f195c009060 comp rx=0 tx=0).stop 2024-09-10T21:14:04.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.051+0000 7f1972200640 1 -- 172.21.15.16:0/1235571751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f196c15f450 msgr2=0x7f196c0c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:04.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.051+0000 7f1972200640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f196c15f450 0x7f196c0c90c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f19540b0040 tx=0x7f19540a5720 comp rx=0 tx=0).stop 2024-09-10T21:14:04.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.051+0000 7f1972200640 1 -- 172.21.15.16:0/1235571751 shutdown_connections 2024-09-10T21:14:04.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.051+0000 7f1972200640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1948072610 0x7f1948074ad0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:04.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.051+0000 7f1972200640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f196c151270 0x7f196c15c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:04.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.051+0000 7f1972200640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f196c15f450 0x7f196c0c90c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:04.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.051+0000 7f1972200640 1 --2- 172.21.15.16:0/1235571751 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f196c0d0890 0x7f196c06f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.051+0000 7f1972200640 1 -- 172.21.15.16:0/1235571751 >> 172.21.15.16:0/1235571751 conn(0x7f196c0d1d80 msgr2=0x7f196c0afa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.051+0000 7f1972200640 1 -- 172.21.15.16:0/1235571751 shutdown_connections 2024-09-10T21:14:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.051+0000 7f1972200640 1 -- 172.21.15.16:0/1235571751 wait complete. 2024-09-10T21:14:04.077 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:14:04.077 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:14:04.077 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-10T21:14:04.078 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:14:04.078 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:14:04.078 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:14:04.078 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:14:04.078 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:14:04.078 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:14:04.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:04.078 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-10T21:14:04.089 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:14:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:14:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:14:04.090 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:04.090 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:14:04.090 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:04.090 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:14:04.090 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:14:04.090 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-10T21:14:04.090 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:14:04.090 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:14:04.091 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-10T21:14:04.091 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:14:04.091 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:14:04.091 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-10T21:14:04.091 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:14:04.091 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42205sec preferred_lft 42205sec 2024-09-10T21:14:04.091 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-10T21:14:04.091 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:04.091 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:14:04.091 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:14:04.091 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-10T21:14:04.092 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:14:04.092 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:30:fb:74:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:14:04.092 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:14:04.092 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:04.092 INFO:teuthology.orchestra.run.smithi016.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:14:04.092 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:14:04.092 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:14:04.092 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:04.092 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:14:04.092 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:04.092 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:14:04.092 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:14:04.093 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:14:04.093 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:04.093 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:14:04.093 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:14:04.093 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:14:04.093 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:04.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:04.094 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:14:04.150 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-10T21:14:04.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:04.152 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-10T21:14:04.201 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-10T21:14:04.201 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:14:04.201 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:14:04.201 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:14:04.201 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:14:04.201 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:14:04.202 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:14:04.202 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:14:04.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:14:04.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:14:04.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:14:04.202 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:14:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:14:04.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:14:04.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:04.340 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:14:04.349 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:14:04.349 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:14:04.350 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:14:04.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:04.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-10T21:14:04.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:14:04.412 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:14:04.464 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:14:04.465 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:14:04.465 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:14:04.516 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:14:04.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:04.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:14:04.605 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:14:04.606 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:14:04.621 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:14:04.621 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:14:04.621 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:14:04.621 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:14:04.621 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:14:04.624 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:14:04.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:04.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:14:04.645 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:14:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:14:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:14:04.757 INFO:teuthology.orchestra.run.smithi016.stdout:client6007 2024-09-10T21:14:04.758 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:14:04.758 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:14:04.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:14:04.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.951+0000 7f8d09c00640 1 -- 172.21.15.16:0/1020229315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce8004810 msgr2=0x7f8ce8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:04.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.951+0000 7f8d09c00640 1 --2- 172.21.15.16:0/1020229315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce8004810 0x7f8ce8004c90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8d0408d620 tx=0x7f8cec01f680 comp rx=0 tx=0).stop 2024-09-10T21:14:04.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.951+0000 7f8d09c00640 1 -- 172.21.15.16:0/1020229315 shutdown_connections 2024-09-10T21:14:04.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.951+0000 7f8d09c00640 1 --2- 172.21.15.16:0/1020229315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce8004810 0x7f8ce8004c90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:04.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.951+0000 7f8d09c00640 1 -- 172.21.15.16:0/1020229315 >> 172.21.15.16:0/1020229315 conn(0x7f8d040d3f00 msgr2=0x7f8d040d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:04.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.951+0000 7f8d09c00640 1 -- 172.21.15.16:0/1020229315 shutdown_connections 2024-09-10T21:14:04.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.951+0000 7f8d09c00640 1 -- 172.21.15.16:0/1020229315 wait complete. 2024-09-10T21:14:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d09c00640 1 Processor -- start 2024-09-10T21:14:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d09c00640 1 -- start start 2024-09-10T21:14:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d09c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8004810 0x7f8d040ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d09c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d0415a780 0x7f8d041f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d09c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8d041f4d40 0x7f8d040cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d09c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d040d08c0 con 0x7f8d0415a780 2024-09-10T21:14:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d09c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8d040d0740 con 0x7f8ce8004810 2024-09-10T21:14:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d09c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8d040d0a40 con 0x7f8d041f4d40 2024-09-10T21:14:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8cfbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d0415a780 0x7f8d041f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8cfbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d0415a780 0x7f8d041f4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59162/0 (socket says 172.21.15.16:59162) 2024-09-10T21:14:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8cfbe00640 1 -- 172.21.15.16:0/1492326195 learned_addr learned my addr 172.21.15.16:0/1492326195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d03e00640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8004810 0x7f8d040ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d09200640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8d041f4d40 0x7f8d040cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8cfbe00640 1 -- 172.21.15.16:0/1492326195 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8d041f4d40 msgr2=0x7f8d040cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8cfbe00640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8d041f4d40 0x7f8d040cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8cfbe00640 1 -- 172.21.15.16:0/1492326195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8004810 msgr2=0x7f8d040ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8cfbe00640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8004810 0x7f8d040ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8cfbe00640 1 -- 172.21.15.16:0/1492326195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d040c95c0 con 0x7f8d0415a780 2024-09-10T21:14:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d09200640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8d041f4d40 0x7f8d040cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8cfbe00640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d0415a780 0x7f8d041f4800 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8cec026390 tx=0x7f8cec004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d00c00640 1 -- 172.21.15.16:0/1492326195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0b3030 con 0x7f8d0415a780 2024-09-10T21:14:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d00c00640 1 -- 172.21.15.16:0/1492326195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cec0b4040 con 0x7f8d0415a780 2024-09-10T21:14:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d09c00640 1 -- 172.21.15.16:0/1492326195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d041f5330 con 0x7f8d0415a780 2024-09-10T21:14:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d00c00640 1 -- 172.21.15.16:0/1492326195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec006ca0 con 0x7f8d0415a780 2024-09-10T21:14:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d09c00640 1 -- 172.21.15.16:0/1492326195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d040c9880 con 0x7f8d0415a780 2024-09-10T21:14:04.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.955+0000 7f8d09c00640 1 -- 172.21.15.16:0/1492326195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d040d0fa0 con 0x7f8d0415a780 2024-09-10T21:14:04.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.959+0000 7f8d00c00640 1 -- 172.21.15.16:0/1492326195 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8cec0ca060 con 0x7f8d0415a780 2024-09-10T21:14:04.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.959+0000 7f8d00c00640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ce0072610 0x7f8ce0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:04.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.959+0000 7f8d00c00640 1 -- 172.21.15.16:0/1492326195 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f8cec130cd0 con 0x7f8d0415a780 2024-09-10T21:14:04.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.959+0000 7f8d03e00640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ce0072610 0x7f8ce0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:04.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.959+0000 7f8d03e00640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ce0072610 0x7f8ce0074ad0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4000c10 tx=0x7f8cf4006d20 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:04.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:04.963+0000 7f8d00c00640 1 -- 172.21.15.16:0/1492326195 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8cec0ab070 con 0x7f8d0415a780 2024-09-10T21:14:05.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.175+0000 7f8d09c00640 1 -- 172.21.15.16:0/1492326195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8d040d0310 con 0x7f8d0415a780 2024-09-10T21:14:05.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.175+0000 7f8d00c00640 1 -- 172.21.15.16:0/1492326195 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) ==== 53+0+73 (secure 0 0 0) 0x7f8cec019430 con 0x7f8d0415a780 2024-09-10T21:14:05.184 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:14:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.179+0000 7f8d09c00640 1 -- 172.21.15.16:0/1492326195 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ce0072610 msgr2=0x7f8ce0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.179+0000 7f8d09c00640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ce0072610 0x7f8ce0074ad0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4000c10 tx=0x7f8cf4006d20 comp rx=0 tx=0).stop 2024-09-10T21:14:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.179+0000 7f8d09c00640 1 -- 172.21.15.16:0/1492326195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d0415a780 msgr2=0x7f8d041f4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.179+0000 7f8d09c00640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d0415a780 0x7f8d041f4800 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8cec026390 tx=0x7f8cec004c50 comp rx=0 tx=0).stop 2024-09-10T21:14:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.179+0000 7f8d09c00640 1 -- 172.21.15.16:0/1492326195 shutdown_connections 2024-09-10T21:14:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.179+0000 7f8d09c00640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8ce0072610 0x7f8ce0074ad0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.179+0000 7f8d09c00640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8d041f4d40 0x7f8d040cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.179+0000 7f8d09c00640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d0415a780 0x7f8d041f4800 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.179+0000 7f8d09c00640 1 --2- 172.21.15.16:0/1492326195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8004810 0x7f8d040ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.179+0000 7f8d09c00640 1 -- 172.21.15.16:0/1492326195 >> 172.21.15.16:0/1492326195 conn(0x7f8d040d3f00 msgr2=0x7f8d040cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:05.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.179+0000 7f8d09c00640 1 -- 172.21.15.16:0/1492326195 shutdown_connections 2024-09-10T21:14:05.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:05.179+0000 7f8d09c00640 1 -- 172.21.15.16:0/1492326195 wait complete. 2024-09-10T21:14:05.206 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:14:05.206 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:14:05.206 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-09-10T21:14:05.206 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T21:14:05.206 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:14:05.206 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:14:05.206 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:14:05.206 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:14:05.206 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:14:05.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:05.207 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-09-10T21:14:05.217 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:14:05.217 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:14:05.217 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:14:05.217 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:05.217 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:14:05.217 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:05.217 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:14:05.218 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:14:05.218 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-09-10T21:14:05.218 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:14:05.218 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:14:05.218 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-09-10T21:14:05.218 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:14:05.218 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-09-10T21:14:05.218 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-09-10T21:14:05.218 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:14:05.218 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42205sec preferred_lft 42205sec 2024-09-10T21:14:05.218 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-09-10T21:14:05.218 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:05.218 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:14:05.219 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-09-10T21:14:05.219 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-09-10T21:14:05.219 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:14:05.219 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:7e:b4:71:a2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:14:05.219 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:14:05.219 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:05.219 INFO:teuthology.orchestra.run.smithi052.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:14:05.219 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:14:05.219 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:14:05.219 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:05.219 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:14:05.219 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:05.219 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:14:05.220 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:14:05.220 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:14:05.220 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:05.220 INFO:teuthology.orchestra.run.smithi052.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:14:05.220 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:14:05.220 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:14:05.220 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:14:05.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:05.221 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:14:05.289 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-09-10T21:14:05.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:05.291 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-09-10T21:14:05.343 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-09-10T21:14:05.343 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:14:05.343 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:14:05.343 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:14:05.343 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:14:05.343 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:14:05.343 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:14:05.344 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:14:05.344 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:14:05.344 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:14:05.344 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:14:05.344 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:14:05.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:14:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:14:05.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:14:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:14:05.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:05.491 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:14:05.500 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:14:05.501 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:14:05.502 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:14:05.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:05.502 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-09-10T21:14:05.565 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:14:05.566 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T21:14:05.617 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:14:05.617 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:14:05.617 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:14:05.673 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T21:14:05.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:05.673 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:14:05.767 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:14:05.767 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:14:05.779 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:14:05.779 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:14:05.779 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:14:05.780 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 1@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:14:05.780 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:14:05.783 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:14:05.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:14:05.784 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:14:05.804 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:14:05.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:14:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:14:05.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:14:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:14:05.902 INFO:teuthology.orchestra.run.smithi052.stdout:client6019 2024-09-10T21:14:05.902 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:14:05.903 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:14:05.912 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-09-10T21:14:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.063+0000 7fe9f1c00640 1 -- 172.21.15.16:0/784930460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ec15dd70 msgr2=0x7fe9ec0cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.063+0000 7fe9f1c00640 1 --2- 172.21.15.16:0/784930460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ec15dd70 0x7fe9ec0cf840 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009900 tx=0x7fe9dc01c8b0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.063+0000 7fe9f1c00640 1 -- 172.21.15.16:0/784930460 shutdown_connections 2024-09-10T21:14:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.063+0000 7fe9f1c00640 1 --2- 172.21.15.16:0/784930460 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe9ec0d0680 0x7fe9ec162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.063+0000 7fe9f1c00640 1 --2- 172.21.15.16:0/784930460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ec15dd70 0x7fe9ec0cf840 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.063+0000 7fe9f1c00640 1 -- 172.21.15.16:0/784930460 >> 172.21.15.16:0/784930460 conn(0x7fe9ec0d1c70 msgr2=0x7fe9ec0d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.063+0000 7fe9f1c00640 1 -- 172.21.15.16:0/784930460 shutdown_connections 2024-09-10T21:14:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.063+0000 7fe9f1c00640 1 -- 172.21.15.16:0/784930460 wait complete. 2024-09-10T21:14:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1c00640 1 Processor -- start 2024-09-10T21:14:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1c00640 1 -- start start 2024-09-10T21:14:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ec0d0250 0x7fe9ec0ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe9ec0d0680 0x7fe9ec0b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ec15dd70 0x7fe9ec167c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec0cff00 con 0x7fe9ec15dd70 2024-09-10T21:14:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec0cfd80 con 0x7fe9ec0d0250 2024-09-10T21:14:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ec162540 con 0x7fe9ec0d0680 2024-09-10T21:14:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ec15dd70 0x7fe9ec167c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ec15dd70 0x7fe9ec167c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59174/0 (socket says 172.21.15.16:59174) 2024-09-10T21:14:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1200640 1 -- 172.21.15.16:0/2949333253 learned_addr learned my addr 172.21.15.16:0/2949333253 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9e3400640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe9ec0d0680 0x7fe9ec0b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9ebe00640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ec0d0250 0x7fe9ec0ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1200640 1 -- 172.21.15.16:0/2949333253 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe9ec0d0680 msgr2=0x7fe9ec0b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1200640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe9ec0d0680 0x7fe9ec0b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1200640 1 -- 172.21.15.16:0/2949333253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ec0d0250 msgr2=0x7fe9ec0ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1200640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ec0d0250 0x7fe9ec0ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1200640 1 -- 172.21.15.16:0/2949333253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec0c5870 con 0x7fe9ec15dd70 2024-09-10T21:14:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9ebe00640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ec0d0250 0x7fe9ec0ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1200640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ec15dd70 0x7fe9ec167c60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400ea50 tx=0x7fe9e400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9e8c00640 1 -- 172.21.15.16:0/2949333253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e400cdc0 con 0x7fe9ec15dd70 2024-09-10T21:14:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9e8c00640 1 -- 172.21.15.16:0/2949333253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9e4004c40 con 0x7fe9ec15dd70 2024-09-10T21:14:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1c00640 1 -- 172.21.15.16:0/2949333253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec0c5b50 con 0x7fe9ec15dd70 2024-09-10T21:14:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9e8c00640 1 -- 172.21.15.16:0/2949333253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4005280 con 0x7fe9ec15dd70 2024-09-10T21:14:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1c00640 1 -- 172.21.15.16:0/2949333253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ec0c6110 con 0x7fe9ec15dd70 2024-09-10T21:14:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.067+0000 7fe9f1c00640 1 -- 172.21.15.16:0/2949333253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ec0c6310 con 0x7fe9ec15dd70 2024-09-10T21:14:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.071+0000 7fe9e8c00640 1 -- 172.21.15.16:0/2949333253 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe9e400f040 con 0x7fe9ec15dd70 2024-09-10T21:14:06.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.071+0000 7fe9e8c00640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe9c80725e0 0x7fe9c8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.071+0000 7fe9e8c00640 1 -- 172.21.15.16:0/2949333253 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fe9e40939f0 con 0x7fe9ec15dd70 2024-09-10T21:14:06.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.071+0000 7fe9ebe00640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe9c80725e0 0x7fe9c8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.071+0000 7fe9ebe00640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe9c80725e0 0x7fe9c8074aa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0af000 tx=0x7fe9dc01b0f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:06.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.075+0000 7fe9e8c00640 1 -- 172.21.15.16:0/2949333253 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe9ec0c6310 con 0x7fe9ec15dd70 2024-09-10T21:14:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.247+0000 7fe9f1c00640 1 -- 172.21.15.16:0/2949333253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7fe9ec0cf840 con 0x7fe9ec15dd70 2024-09-10T21:14:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.251+0000 7fe9e8c00640 1 -- 172.21.15.16:0/2949333253 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v40)=0 v40) ==== 121+0+0 (secure 0 0 0) 0x7fe9e4016030 con 0x7fe9ec15dd70 2024-09-10T21:14:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.255+0000 7fe9f1c00640 1 -- 172.21.15.16:0/2949333253 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe9c80725e0 msgr2=0x7fe9c8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.255+0000 7fe9f1c00640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe9c80725e0 0x7fe9c8074aa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0af000 tx=0x7fe9dc01b0f0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.255+0000 7fe9f1c00640 1 -- 172.21.15.16:0/2949333253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ec15dd70 msgr2=0x7fe9ec167c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.255+0000 7fe9f1c00640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ec15dd70 0x7fe9ec167c60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400ea50 tx=0x7fe9e400ef10 comp rx=0 tx=0).stop 2024-09-10T21:14:06.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.255+0000 7fe9f1c00640 1 -- 172.21.15.16:0/2949333253 shutdown_connections 2024-09-10T21:14:06.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.255+0000 7fe9f1c00640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe9c80725e0 0x7fe9c8074aa0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.255+0000 7fe9f1c00640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ec15dd70 0x7fe9ec167c60 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.255+0000 7fe9f1c00640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe9ec0d0680 0x7fe9ec0b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.255+0000 7fe9f1c00640 1 --2- 172.21.15.16:0/2949333253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ec0d0250 0x7fe9ec0ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.255+0000 7fe9f1c00640 1 -- 172.21.15.16:0/2949333253 >> 172.21.15.16:0/2949333253 conn(0x7fe9ec0d1c70 msgr2=0x7fe9ec15ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:06.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.255+0000 7fe9f1c00640 1 -- 172.21.15.16:0/2949333253 shutdown_connections 2024-09-10T21:14:06.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.255+0000 7fe9f1c00640 1 -- 172.21.15.16:0/2949333253 wait complete. 2024-09-10T21:14:06.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2024-09-10T21:14:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.423+0000 7fab32400640 1 -- 172.21.15.16:0/3543781091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c15a9a0 msgr2=0x7fab2c0b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.423+0000 7fab32400640 1 --2- 172.21.15.16:0/3543781091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c15a9a0 0x7fab2c0b3f80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0cca90 tx=0x7fab1c01c830 comp rx=0 tx=0).stop 2024-09-10T21:14:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.423+0000 7fab32400640 1 -- 172.21.15.16:0/3543781091 shutdown_connections 2024-09-10T21:14:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.423+0000 7fab32400640 1 --2- 172.21.15.16:0/3543781091 >> v2:172.21.15.52:3300/0 conn(0x7fab2c0b33b0 0x7fab2c070070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.423+0000 7fab32400640 1 --2- 172.21.15.16:0/3543781091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c15a9a0 0x7fab2c0b3f80 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.423+0000 7fab32400640 1 -- 172.21.15.16:0/3543781091 >> 172.21.15.16:0/3543781091 conn(0x7fab2c0b16a0 msgr2=0x7fab2c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.423+0000 7fab32400640 1 -- 172.21.15.16:0/3543781091 shutdown_connections 2024-09-10T21:14:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.423+0000 7fab32400640 1 -- 172.21.15.16:0/3543781091 wait complete. 2024-09-10T21:14:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.423+0000 7fab32400640 1 Processor -- start 2024-09-10T21:14:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.423+0000 7fab32400640 1 -- start start 2024-09-10T21:14:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab32400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fab2c0b33b0 0x7fab2c06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab32400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab2c15a9a0 0x7fab2c0acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab32400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c15ad80 0x7fab2c0ad430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab32400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c167e70 con 0x7fab2c15ad80 2024-09-10T21:14:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab32400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c167cf0 con 0x7fab2c15a9a0 2024-09-10T21:14:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab32400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fab2c167ff0 con 0x7fab2c0b33b0 2024-09-10T21:14:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab2be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c15ad80 0x7fab2c0ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab2be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c15ad80 0x7fab2c0ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59192/0 (socket says 172.21.15.16:59192) 2024-09-10T21:14:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab2be00640 1 -- 172.21.15.16:0/2118401348 learned_addr learned my addr 172.21.15.16:0/2118401348 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab2b400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fab2c0b33b0 0x7fab2c06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab23e00640 1 --2- 172.21.15.16:0/2118401348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab2c15a9a0 0x7fab2c0acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab2be00640 1 -- 172.21.15.16:0/2118401348 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fab2c0b33b0 msgr2=0x7fab2c06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab2be00640 1 --2- 172.21.15.16:0/2118401348 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fab2c0b33b0 0x7fab2c06f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab2be00640 1 -- 172.21.15.16:0/2118401348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab2c15a9a0 msgr2=0x7fab2c0acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab2be00640 1 --2- 172.21.15.16:0/2118401348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab2c15a9a0 0x7fab2c0acef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab2be00640 1 -- 172.21.15.16:0/2118401348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c0ad970 con 0x7fab2c15ad80 2024-09-10T21:14:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab2be00640 1 --2- 172.21.15.16:0/2118401348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c15ad80 0x7fab2c0ad430 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fab2400ea20 tx=0x7fab2400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab23400640 1 -- 172.21.15.16:0/2118401348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2400cde0 con 0x7fab2c15ad80 2024-09-10T21:14:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab23400640 1 -- 172.21.15.16:0/2118401348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab24004c40 con 0x7fab2c15ad80 2024-09-10T21:14:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab23400640 1 -- 172.21.15.16:0/2118401348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24005280 con 0x7fab2c15ad80 2024-09-10T21:14:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab32400640 1 -- 172.21.15.16:0/2118401348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2c1f5e10 con 0x7fab2c15ad80 2024-09-10T21:14:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab32400640 1 -- 172.21.15.16:0/2118401348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2c1f60d0 con 0x7fab2c15ad80 2024-09-10T21:14:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab32400640 1 -- 172.21.15.16:0/2118401348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2c070070 con 0x7fab2c15ad80 2024-09-10T21:14:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.427+0000 7fab23400640 1 -- 172.21.15.16:0/2118401348 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fab2400f040 con 0x7fab2c15ad80 2024-09-10T21:14:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.431+0000 7fab23400640 1 --2- 172.21.15.16:0/2118401348 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fab080725e0 0x7fab08074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.431+0000 7fab23400640 1 -- 172.21.15.16:0/2118401348 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fab240938f0 con 0x7fab2c15ad80 2024-09-10T21:14:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.431+0000 7fab2b400640 1 --2- 172.21.15.16:0/2118401348 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fab080725e0 0x7fab08074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.431+0000 7fab2b400640 1 --2- 172.21.15.16:0/2118401348 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fab080725e0 0x7fab08074aa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0af000 tx=0x7fab1c01b070 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:06.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.435+0000 7fab23400640 1 -- 172.21.15.16:0/2118401348 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fab24016030 con 0x7fab2c15ad80 2024-09-10T21:14:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.611+0000 7fab32400640 1 -- 172.21.15.16:0/2118401348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7fab2c15e560 con 0x7fab2c15ad80 2024-09-10T21:14:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.615+0000 7fab23400640 1 -- 172.21.15.16:0/2118401348 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v41)=0 v41) ==== 119+0+0 (secure 0 0 0) 0x7fab2405e970 con 0x7fab2c15ad80 2024-09-10T21:14:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.619+0000 7fab32400640 1 -- 172.21.15.16:0/2118401348 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fab080725e0 msgr2=0x7fab08074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.619+0000 7fab32400640 1 --2- 172.21.15.16:0/2118401348 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fab080725e0 0x7fab08074aa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0af000 tx=0x7fab1c01b070 comp rx=0 tx=0).stop 2024-09-10T21:14:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.619+0000 7fab32400640 1 -- 172.21.15.16:0/2118401348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c15ad80 msgr2=0x7fab2c0ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.619+0000 7fab32400640 1 --2- 172.21.15.16:0/2118401348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c15ad80 0x7fab2c0ad430 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fab2400ea20 tx=0x7fab2400eee0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.619+0000 7fab32400640 1 -- 172.21.15.16:0/2118401348 shutdown_connections 2024-09-10T21:14:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.619+0000 7fab32400640 1 --2- 172.21.15.16:0/2118401348 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fab080725e0 0x7fab08074aa0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.619+0000 7fab32400640 1 --2- 172.21.15.16:0/2118401348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab2c15ad80 0x7fab2c0ad430 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.619+0000 7fab32400640 1 --2- 172.21.15.16:0/2118401348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab2c15a9a0 0x7fab2c0acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.619+0000 7fab32400640 1 --2- 172.21.15.16:0/2118401348 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fab2c0b33b0 0x7fab2c06f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.619+0000 7fab32400640 1 -- 172.21.15.16:0/2118401348 >> 172.21.15.16:0/2118401348 conn(0x7fab2c0b16a0 msgr2=0x7fab2c0ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.619+0000 7fab32400640 1 -- 172.21.15.16:0/2118401348 shutdown_connections 2024-09-10T21:14:06.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.619+0000 7fab32400640 1 -- 172.21.15.16:0/2118401348 wait complete. 2024-09-10T21:14:06.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2024-09-10T21:14:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 -- 172.21.15.16:0/1672831847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a680b33b0 msgr2=0x7f2a68070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 --2- 172.21.15.16:0/1672831847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a680b33b0 0x7f2a68070070 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009c80 tx=0x7f2a6001cd10 comp rx=0 tx=0).stop 2024-09-10T21:14:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 -- 172.21.15.16:0/1672831847 shutdown_connections 2024-09-10T21:14:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 --2- 172.21.15.16:0/1672831847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a680b33b0 0x7f2a68070070 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 --2- 172.21.15.16:0/1672831847 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a6815a9a0 0x7f2a680b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 -- 172.21.15.16:0/1672831847 >> 172.21.15.16:0/1672831847 conn(0x7f2a680b16a0 msgr2=0x7f2a680b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 -- 172.21.15.16:0/1672831847 shutdown_connections 2024-09-10T21:14:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 -- 172.21.15.16:0/1672831847 wait complete. 2024-09-10T21:14:06.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 Processor -- start 2024-09-10T21:14:06.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 -- start start 2024-09-10T21:14:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a680b33b0 0x7f2a68154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a6815a9a0 0x7f2a68164380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a6815ad80 0x7f2a681463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68167f80 con 0x7f2a680b33b0 2024-09-10T21:14:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68167e00 con 0x7f2a6815ad80 2024-09-10T21:14:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a6ce00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2a68168100 con 0x7f2a6815a9a0 2024-09-10T21:14:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a65e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a680b33b0 0x7f2a68154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a65e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a680b33b0 0x7f2a68154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59196/0 (socket says 172.21.15.16:59196) 2024-09-10T21:14:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a65e00640 1 -- 172.21.15.16:0/326574323 learned_addr learned my addr 172.21.15.16:0/326574323 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a66800640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a6815ad80 0x7f2a681463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.791+0000 7f2a5fe00640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a6815a9a0 0x7f2a68164380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a65e00640 1 -- 172.21.15.16:0/326574323 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a6815a9a0 msgr2=0x7f2a68164380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a65e00640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a6815a9a0 0x7f2a68164380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a65e00640 1 -- 172.21.15.16:0/326574323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a6815ad80 msgr2=0x7f2a681463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a65e00640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a6815ad80 0x7f2a681463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a65e00640 1 -- 172.21.15.16:0/326574323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a68162150 con 0x7f2a680b33b0 2024-09-10T21:14:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a65e00640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a680b33b0 0x7f2a68154c20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800e9f0 tx=0x7f2a5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a5e000640 1 -- 172.21.15.16:0/326574323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5800cd20 con 0x7f2a680b33b0 2024-09-10T21:14:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a5e000640 1 -- 172.21.15.16:0/326574323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a58010430 con 0x7f2a680b33b0 2024-09-10T21:14:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a5e000640 1 -- 172.21.15.16:0/326574323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58010860 con 0x7f2a680b33b0 2024-09-10T21:14:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a6ce00640 1 -- 172.21.15.16:0/326574323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a68162430 con 0x7f2a680b33b0 2024-09-10T21:14:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a6ce00640 1 -- 172.21.15.16:0/326574323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a681629f0 con 0x7f2a680b33b0 2024-09-10T21:14:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a6ce00640 1 -- 172.21.15.16:0/326574323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a68070070 con 0x7f2a680b33b0 2024-09-10T21:14:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a5e000640 1 -- 172.21.15.16:0/326574323 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2a58010a00 con 0x7f2a680b33b0 2024-09-10T21:14:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a5e000640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a440725e0 0x7f2a44074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.795+0000 7f2a5e000640 1 -- 172.21.15.16:0/326574323 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f2a58093dc0 con 0x7f2a680b33b0 2024-09-10T21:14:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.799+0000 7f2a5fe00640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a440725e0 0x7f2a44074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.799+0000 7f2a5fe00640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a440725e0 0x7f2a44074aa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2a68164a40 tx=0x7f2a50009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.803+0000 7f2a5e000640 1 -- 172.21.15.16:0/326574323 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2a5805ee70 con 0x7f2a680b33b0 2024-09-10T21:14:06.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.975+0000 7f2a6ce00640 1 -- 172.21.15.16:0/326574323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7f2a68072b30 con 0x7f2a680b33b0 2024-09-10T21:14:06.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.979+0000 7f2a5e000640 1 -- 172.21.15.16:0/326574323 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v42)=0 v42) ==== 128+0+0 (secure 0 0 0) 0x7f2a580638a0 con 0x7f2a680b33b0 2024-09-10T21:14:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.979+0000 7f2a6ce00640 1 -- 172.21.15.16:0/326574323 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a440725e0 msgr2=0x7f2a44074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.979+0000 7f2a6ce00640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a440725e0 0x7f2a44074aa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2a68164a40 tx=0x7f2a50009040 comp rx=0 tx=0).stop 2024-09-10T21:14:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.979+0000 7f2a6ce00640 1 -- 172.21.15.16:0/326574323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a680b33b0 msgr2=0x7f2a68154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.979+0000 7f2a6ce00640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a680b33b0 0x7f2a68154c20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800e9f0 tx=0x7f2a5800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.979+0000 7f2a6ce00640 1 -- 172.21.15.16:0/326574323 shutdown_connections 2024-09-10T21:14:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.979+0000 7f2a6ce00640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a440725e0 0x7f2a44074aa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.983+0000 7f2a6ce00640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a6815ad80 0x7f2a681463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.983+0000 7f2a6ce00640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a6815a9a0 0x7f2a68164380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.983+0000 7f2a6ce00640 1 --2- 172.21.15.16:0/326574323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a680b33b0 0x7f2a68154c20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.983+0000 7f2a6ce00640 1 -- 172.21.15.16:0/326574323 >> 172.21.15.16:0/326574323 conn(0x7f2a680b16a0 msgr2=0x7f2a68072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.983+0000 7f2a6ce00640 1 -- 172.21.15.16:0/326574323 shutdown_connections 2024-09-10T21:14:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:06.983+0000 7f2a6ce00640 1 -- 172.21.15.16:0/326574323 wait complete. 2024-09-10T21:14:07.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 800 2024-09-10T21:14:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.151+0000 7fa48e600640 1 -- 172.21.15.16:0/608771417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa48815dd80 msgr2=0x7fa48815e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.151+0000 7fa48e600640 1 --2- 172.21.15.16:0/608771417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa48815dd80 0x7fa48815e190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa47000a950 tx=0x7fa47001db70 comp rx=0 tx=0).stop 2024-09-10T21:14:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.151+0000 7fa48e600640 1 -- 172.21.15.16:0/608771417 shutdown_connections 2024-09-10T21:14:07.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.151+0000 7fa48e600640 1 --2- 172.21.15.16:0/608771417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa48815dd80 0x7fa48815e190 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.151+0000 7fa48e600640 1 -- 172.21.15.16:0/608771417 >> 172.21.15.16:0/608771417 conn(0x7fa4880b16a0 msgr2=0x7fa4880b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:07.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.151+0000 7fa48e600640 1 -- 172.21.15.16:0/608771417 shutdown_connections 2024-09-10T21:14:07.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.151+0000 7fa48e600640 1 -- 172.21.15.16:0/608771417 wait complete. 2024-09-10T21:14:07.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.151+0000 7fa48e600640 1 Processor -- start 2024-09-10T21:14:07.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa48e600640 1 -- start start 2024-09-10T21:14:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa48e600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa48806f800 0x7fa48815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa48e600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa488151210 0x7fa4880ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa48e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48815dd80 0x7fa4880ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa48e600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa488168340 con 0x7fa48815dd80 2024-09-10T21:14:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa48e600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa488072f50 con 0x7fa488151210 2024-09-10T21:14:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa48e600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa4881684c0 con 0x7fa48806f800 2024-09-10T21:14:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa487e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48815dd80 0x7fa4880ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa487e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48815dd80 0x7fa4880ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59212/0 (socket says 172.21.15.16:59212) 2024-09-10T21:14:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa487e00640 1 -- 172.21.15.16:0/2425279359 learned_addr learned my addr 172.21.15.16:0/2425279359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa47fe00640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa488151210 0x7fa4880ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa487400640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa48806f800 0x7fa48815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa487e00640 1 -- 172.21.15.16:0/2425279359 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa48806f800 msgr2=0x7fa48815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa487e00640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa48806f800 0x7fa48815b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa487e00640 1 -- 172.21.15.16:0/2425279359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa488151210 msgr2=0x7fa4880ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa487e00640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa488151210 0x7fa4880ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa487e00640 1 -- 172.21.15.16:0/2425279359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4881fb680 con 0x7fa48815dd80 2024-09-10T21:14:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa487e00640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48815dd80 0x7fa4880ce7f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa48000cbf0 tx=0x7fa48000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa47f400640 1 -- 172.21.15.16:0/2425279359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480016020 con 0x7fa48815dd80 2024-09-10T21:14:07.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa47f400640 1 -- 172.21.15.16:0/2425279359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa48000ab40 con 0x7fa48815dd80 2024-09-10T21:14:07.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa47f400640 1 -- 172.21.15.16:0/2425279359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480012420 con 0x7fa48815dd80 2024-09-10T21:14:07.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa48e600640 1 -- 172.21.15.16:0/2425279359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4881fb940 con 0x7fa48815dd80 2024-09-10T21:14:07.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa48e600640 1 -- 172.21.15.16:0/2425279359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4881fbf00 con 0x7fa48815dd80 2024-09-10T21:14:07.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.155+0000 7fa48e600640 1 -- 172.21.15.16:0/2425279359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48815e190 con 0x7fa48815dd80 2024-09-10T21:14:07.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.159+0000 7fa47f400640 1 -- 172.21.15.16:0/2425279359 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa48001b400 con 0x7fa48815dd80 2024-09-10T21:14:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.159+0000 7fa47f400640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa4640725e0 0x7fa464074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.159+0000 7fa47f400640 1 -- 172.21.15.16:0/2425279359 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fa4800937e0 con 0x7fa48815dd80 2024-09-10T21:14:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.159+0000 7fa487400640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa4640725e0 0x7fa464074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:07.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.159+0000 7fa487400640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa4640725e0 0x7fa464074aa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa478000c10 tx=0x7fa478009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:07.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.163+0000 7fa47f400640 1 -- 172.21.15.16:0/2425279359 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa480020020 con 0x7fa48815dd80 2024-09-10T21:14:07.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.335+0000 7fa48e600640 1 -- 172.21.15.16:0/2425279359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7fa4880b2c50 con 0x7fa48815dd80 2024-09-10T21:14:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.339+0000 7fa47f400640 1 -- 172.21.15.16:0/2425279359 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v43)=0 v43) ==== 124+0+0 (secure 0 0 0) 0x7fa48815e6d0 con 0x7fa48815dd80 2024-09-10T21:14:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.339+0000 7fa48e600640 1 -- 172.21.15.16:0/2425279359 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa4640725e0 msgr2=0x7fa464074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.339+0000 7fa48e600640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa4640725e0 0x7fa464074aa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa478000c10 tx=0x7fa478009040 comp rx=0 tx=0).stop 2024-09-10T21:14:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.339+0000 7fa48e600640 1 -- 172.21.15.16:0/2425279359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48815dd80 msgr2=0x7fa4880ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.339+0000 7fa48e600640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48815dd80 0x7fa4880ce7f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa48000cbf0 tx=0x7fa48000cce0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.343+0000 7fa48e600640 1 -- 172.21.15.16:0/2425279359 shutdown_connections 2024-09-10T21:14:07.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.343+0000 7fa48e600640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa4640725e0 0x7fa464074aa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.343+0000 7fa48e600640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48815dd80 0x7fa4880ce7f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.343+0000 7fa48e600640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa488151210 0x7fa4880ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.343+0000 7fa48e600640 1 --2- 172.21.15.16:0/2425279359 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa48806f800 0x7fa48815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.343+0000 7fa48e600640 1 -- 172.21.15.16:0/2425279359 >> 172.21.15.16:0/2425279359 conn(0x7fa4880b16a0 msgr2=0x7fa4880b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:07.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.343+0000 7fa48e600640 1 -- 172.21.15.16:0/2425279359 shutdown_connections 2024-09-10T21:14:07.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.343+0000 7fa48e600640 1 -- 172.21.15.16:0/2425279359 wait complete. 2024-09-10T21:14:07.369 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-10T21:14:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.511+0000 7fe6d9800640 1 -- 172.21.15.16:0/3323180589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b8007850 msgr2=0x7fe6b8009ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.511+0000 7fe6d9800640 1 --2- 172.21.15.16:0/3323180589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b8007850 0x7fe6b8009ce0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0016f0 tx=0x7fe6bc01c760 comp rx=0 tx=0).stop 2024-09-10T21:14:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.511+0000 7fe6d9800640 1 -- 172.21.15.16:0/3323180589 shutdown_connections 2024-09-10T21:14:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.511+0000 7fe6d9800640 1 --2- 172.21.15.16:0/3323180589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b8007850 0x7fe6b8009ce0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.511+0000 7fe6d9800640 1 -- 172.21.15.16:0/3323180589 >> 172.21.15.16:0/3323180589 conn(0x7fe6d40b16a0 msgr2=0x7fe6d40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.511+0000 7fe6d9800640 1 -- 172.21.15.16:0/3323180589 shutdown_connections 2024-09-10T21:14:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.511+0000 7fe6d9800640 1 -- 172.21.15.16:0/3323180589 wait complete. 2024-09-10T21:14:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.511+0000 7fe6d9800640 1 Processor -- start 2024-09-10T21:14:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.511+0000 7fe6d9800640 1 -- start start 2024-09-10T21:14:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d9800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6b8007850 0x7fe6d41f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d9800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe6d415b8c0 0x7fe6d415bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d9800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d414e280 0x7fe6d414e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d9800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d406f7f0 con 0x7fe6d414e280 2024-09-10T21:14:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d9800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d406f670 con 0x7fe6b8007850 2024-09-10T21:14:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d9800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d406f970 con 0x7fe6d415b8c0 2024-09-10T21:14:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d414e280 0x7fe6d414e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d2a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6b8007850 0x7fe6d41f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d414e280 0x7fe6d414e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59224/0 (socket says 172.21.15.16:59224) 2024-09-10T21:14:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d3400640 1 -- 172.21.15.16:0/707693501 learned_addr learned my addr 172.21.15.16:0/707693501 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6cbe00640 1 --2- 172.21.15.16:0/707693501 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe6d415b8c0 0x7fe6d415bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d3400640 1 -- 172.21.15.16:0/707693501 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe6d415b8c0 msgr2=0x7fe6d415bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d3400640 1 --2- 172.21.15.16:0/707693501 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe6d415b8c0 0x7fe6d415bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d3400640 1 -- 172.21.15.16:0/707693501 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6b8007850 msgr2=0x7fe6d41f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d3400640 1 --2- 172.21.15.16:0/707693501 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6b8007850 0x7fe6d41f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d3400640 1 -- 172.21.15.16:0/707693501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d414ec40 con 0x7fe6d414e280 2024-09-10T21:14:07.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d3400640 1 --2- 172.21.15.16:0/707693501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d414e280 0x7fe6d414e700 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe6d413d400 tx=0x7fe6cc006420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:07.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6caa00640 1 -- 172.21.15.16:0/707693501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc010690 con 0x7fe6d414e280 2024-09-10T21:14:07.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6caa00640 1 -- 172.21.15.16:0/707693501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6cc010940 con 0x7fe6d414e280 2024-09-10T21:14:07.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6caa00640 1 -- 172.21.15.16:0/707693501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc010d90 con 0x7fe6d414e280 2024-09-10T21:14:07.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d9800640 1 -- 172.21.15.16:0/707693501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d41fbbb0 con 0x7fe6d414e280 2024-09-10T21:14:07.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d9800640 1 -- 172.21.15.16:0/707693501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d41fc170 con 0x7fe6d414e280 2024-09-10T21:14:07.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.515+0000 7fe6d9800640 1 -- 172.21.15.16:0/707693501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d406ff00 con 0x7fe6d414e280 2024-09-10T21:14:07.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.519+0000 7fe6caa00640 1 -- 172.21.15.16:0/707693501 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe6cc01b410 con 0x7fe6d414e280 2024-09-10T21:14:07.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.519+0000 7fe6caa00640 1 --2- 172.21.15.16:0/707693501 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6b0072610 0x7fe6b0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:07.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.519+0000 7fe6caa00640 1 -- 172.21.15.16:0/707693501 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fe6cc093660 con 0x7fe6d414e280 2024-09-10T21:14:07.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.519+0000 7fe6d2a00640 1 --2- 172.21.15.16:0/707693501 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6b0072610 0x7fe6b0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:07.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.519+0000 7fe6d2a00640 1 --2- 172.21.15.16:0/707693501 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6b0072610 0x7fe6b0074ad0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe6d406c240 tx=0x7fe6c40091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.523+0000 7fe6caa00640 1 -- 172.21.15.16:0/707693501 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe6cc05e650 con 0x7fe6d414e280 2024-09-10T21:14:07.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.695+0000 7fe6d9800640 1 -- 172.21.15.16:0/707693501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7fe6d40b24d0 con 0x7fe6d414e280 2024-09-10T21:14:07.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.695+0000 7fe6caa00640 1 -- 172.21.15.16:0/707693501 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v43) ==== 106+0+4 (secure 0 0 0) 0x7fe6cc0673c0 con 0x7fe6d414e280 2024-09-10T21:14:07.702 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-10T21:14:07.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.695+0000 7fe6d9800640 1 -- 172.21.15.16:0/707693501 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6b0072610 msgr2=0x7fe6b0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.695+0000 7fe6d9800640 1 --2- 172.21.15.16:0/707693501 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6b0072610 0x7fe6b0074ad0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe6d406c240 tx=0x7fe6c40091e0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.695+0000 7fe6d9800640 1 -- 172.21.15.16:0/707693501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d414e280 msgr2=0x7fe6d414e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.695+0000 7fe6d9800640 1 --2- 172.21.15.16:0/707693501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d414e280 0x7fe6d414e700 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe6d413d400 tx=0x7fe6cc006420 comp rx=0 tx=0).stop 2024-09-10T21:14:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.695+0000 7fe6d9800640 1 -- 172.21.15.16:0/707693501 shutdown_connections 2024-09-10T21:14:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.695+0000 7fe6d9800640 1 --2- 172.21.15.16:0/707693501 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6b0072610 0x7fe6b0074ad0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.695+0000 7fe6d9800640 1 --2- 172.21.15.16:0/707693501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d414e280 0x7fe6d414e700 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.695+0000 7fe6d9800640 1 --2- 172.21.15.16:0/707693501 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe6d415b8c0 0x7fe6d415bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.695+0000 7fe6d9800640 1 --2- 172.21.15.16:0/707693501 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6b8007850 0x7fe6d41f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.695+0000 7fe6d9800640 1 -- 172.21.15.16:0/707693501 >> 172.21.15.16:0/707693501 conn(0x7fe6d40b16a0 msgr2=0x7fe6d40afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.695+0000 7fe6d9800640 1 -- 172.21.15.16:0/707693501 shutdown_connections 2024-09-10T21:14:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.699+0000 7fe6d9800640 1 -- 172.21.15.16:0/707693501 wait complete. 2024-09-10T21:14:07.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-09-10T21:14:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 -- 172.21.15.16:0/166469575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2010072bf0 msgr2=0x7f201015dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 --2- 172.21.15.16:0/166469575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2010072bf0 0x7f201015dd80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f20100b2470 tx=0x7f1ffc01dbf0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 -- 172.21.15.16:0/166469575 shutdown_connections 2024-09-10T21:14:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 --2- 172.21.15.16:0/166469575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2010072bf0 0x7f201015dd80 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 --2- 172.21.15.16:0/166469575 >> v2:172.21.15.52:3300/0 conn(0x7f2010151210 0x7f20100726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 -- 172.21.15.16:0/166469575 >> 172.21.15.16:0/166469575 conn(0x7f20100b16a0 msgr2=0x7f20100b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 -- 172.21.15.16:0/166469575 shutdown_connections 2024-09-10T21:14:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 -- 172.21.15.16:0/166469575 wait complete. 2024-09-10T21:14:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 Processor -- start 2024-09-10T21:14:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 -- start start 2024-09-10T21:14:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2010072bf0 0x7f20100aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2010151210 0x7f201015c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f201015e2c0 0x7f20100cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f201015b310 con 0x7f2010072bf0 2024-09-10T21:14:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f201015b190 con 0x7f2010151210 2024-09-10T21:14:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f2015a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f201015b490 con 0x7f201015e2c0 2024-09-10T21:14:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f200ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2010072bf0 0x7f20100aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f200ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2010072bf0 0x7f20100aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59250/0 (socket says 172.21.15.16:59250) 2024-09-10T21:14:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f200ea00640 1 -- 172.21.15.16:0/65876350 learned_addr learned my addr 172.21.15.16:0/65876350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f200e000640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2010151210 0x7f201015c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f200f400640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f201015e2c0 0x7f20100cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f200ea00640 1 -- 172.21.15.16:0/65876350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f201015e2c0 msgr2=0x7f20100cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f200ea00640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f201015e2c0 0x7f20100cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f200ea00640 1 -- 172.21.15.16:0/65876350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2010151210 msgr2=0x7f201015c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f200ea00640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2010151210 0x7f201015c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.867+0000 7f200ea00640 1 -- 172.21.15.16:0/65876350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20100add40 con 0x7f2010072bf0 2024-09-10T21:14:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.871+0000 7f200ea00640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2010072bf0 0x7f20100aede0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f200000e940 tx=0x7f200000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.871+0000 7f2006a00640 1 -- 172.21.15.16:0/65876350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200000cd80 con 0x7f2010072bf0 2024-09-10T21:14:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.871+0000 7f2006a00640 1 -- 172.21.15.16:0/65876350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2000005020 con 0x7f2010072bf0 2024-09-10T21:14:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.871+0000 7f2006a00640 1 -- 172.21.15.16:0/65876350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000005450 con 0x7f2010072bf0 2024-09-10T21:14:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.871+0000 7f2015a00640 1 -- 172.21.15.16:0/65876350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20100ae020 con 0x7f2010072bf0 2024-09-10T21:14:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.871+0000 7f2015a00640 1 -- 172.21.15.16:0/65876350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20100ae8c0 con 0x7f2010072bf0 2024-09-10T21:14:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.871+0000 7f2015a00640 1 -- 172.21.15.16:0/65876350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201015dd80 con 0x7f2010072bf0 2024-09-10T21:14:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.871+0000 7f2006a00640 1 -- 172.21.15.16:0/65876350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f20000047b0 con 0x7f2010072bf0 2024-09-10T21:14:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.875+0000 7f2006a00640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1ff00725e0 0x7f1ff0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.875+0000 7f2006a00640 1 -- 172.21.15.16:0/65876350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f2000093d00 con 0x7f2010072bf0 2024-09-10T21:14:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.875+0000 7f200e000640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1ff00725e0 0x7f1ff0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.875+0000 7f200e000640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1ff00725e0 0x7f1ff0074aa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0b0000 tx=0x7f1ffc01c430 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:07.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:07.879+0000 7f2006a00640 1 -- 172.21.15.16:0/65876350 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f200005ecf0 con 0x7f2010072bf0 2024-09-10T21:14:08.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2015a00640 1 -- 172.21.15.16:0/65876350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7f20100ae1f0 con 0x7f2010072bf0 2024-09-10T21:14:08.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2006a00640 1 -- 172.21.15.16:0/65876350 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v43) ==== 112+0+10 (secure 0 0 0) 0x7f2000063720 con 0x7f2010072bf0 2024-09-10T21:14:08.062 INFO:teuthology.orchestra.run.smithi016.stdout:60.000000 2024-09-10T21:14:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2015a00640 1 -- 172.21.15.16:0/65876350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1ff00725e0 msgr2=0x7f1ff0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2015a00640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1ff00725e0 0x7f1ff0074aa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0b0000 tx=0x7f1ffc01c430 comp rx=0 tx=0).stop 2024-09-10T21:14:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2015a00640 1 -- 172.21.15.16:0/65876350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2010072bf0 msgr2=0x7f20100aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2015a00640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2010072bf0 0x7f20100aede0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f200000e940 tx=0x7f200000ee00 comp rx=0 tx=0).stop 2024-09-10T21:14:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2015a00640 1 -- 172.21.15.16:0/65876350 shutdown_connections 2024-09-10T21:14:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2015a00640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1ff00725e0 0x7f1ff0074aa0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2015a00640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f201015e2c0 0x7f20100cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2015a00640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2010151210 0x7f201015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2015a00640 1 --2- 172.21.15.16:0/65876350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2010072bf0 0x7f20100aede0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2015a00640 1 -- 172.21.15.16:0/65876350 >> 172.21.15.16:0/65876350 conn(0x7f20100b16a0 msgr2=0x7f20100b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2015a00640 1 -- 172.21.15.16:0/65876350 shutdown_connections 2024-09-10T21:14:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.055+0000 7f2015a00640 1 -- 172.21.15.16:0/65876350 wait complete. 2024-09-10T21:14:08.084 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:14:08.084 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:14:08.093 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:14:08.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:14:08.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:14:08.198 INFO:teuthology.orchestra.run.smithi016.stdout:client6007 2024-09-10T21:14:08.198 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:14:08.198 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:14:08.207 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-10T21:14:08.208 DEBUG:teuthology.orchestra.run.smithi016:> import fcntl 2024-09-10T21:14:08.208 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2024-09-10T21:14:08.208 DEBUG:teuthology.orchestra.run.smithi016:> import time 2024-09-10T21:14:08.208 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-10T21:14:08.208 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:14:08.208 DEBUG:teuthology.orchestra.run.smithi016:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-10T21:14:08.208 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:14:08.208 DEBUG:teuthology.orchestra.run.smithi016:> n = 400 2024-09-10T21:14:08.208 DEBUG:teuthology.orchestra.run.smithi016:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-09-10T21:14:08.208 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:14:08.208 DEBUG:teuthology.orchestra.run.smithi016:> if not os.path.exists(abs_path): 2024-09-10T21:14:08.208 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(abs_path) 2024-09-10T21:14:08.208 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:14:08.209 DEBUG:teuthology.orchestra.run.smithi016:> handles = [] 2024-09-10T21:14:08.209 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, n): 2024-09-10T21:14:08.209 DEBUG:teuthology.orchestra.run.smithi016:> fname = "file_"+str(i) 2024-09-10T21:14:08.209 DEBUG:teuthology.orchestra.run.smithi016:> path = os.path.join(abs_path, fname) 2024-09-10T21:14:08.209 DEBUG:teuthology.orchestra.run.smithi016:> handles.append(open(path, '"'"'w'"'"')) 2024-09-10T21:14:08.209 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:14:08.209 DEBUG:teuthology.orchestra.run.smithi016:> print("waiting with handles open", file=sys.stderr) 2024-09-10T21:14:08.209 DEBUG:teuthology.orchestra.run.smithi016:> while True: 2024-09-10T21:14:08.210 DEBUG:teuthology.orchestra.run.smithi016:> print("open_n_background: keeping files open", file=sys.stderr) 2024-09-10T21:14:08.210 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-10T21:14:08.210 DEBUG:teuthology.orchestra.run.smithi016:> if os.read(0, 4096) == b"": 2024-09-10T21:14:08.210 DEBUG:teuthology.orchestra.run.smithi016:> break 2024-09-10T21:14:08.210 DEBUG:teuthology.orchestra.run.smithi016:> except BlockingIOError: 2024-09-10T21:14:08.210 DEBUG:teuthology.orchestra.run.smithi016:> pass 2024-09-10T21:14:08.210 DEBUG:teuthology.orchestra.run.smithi016:> time.sleep(2) 2024-09-10T21:14:08.210 DEBUG:teuthology.orchestra.run.smithi016:> print("stdin closed, goodbye!", file=sys.stderr) 2024-09-10T21:14:08.210 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-10T21:14:08.255 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:14:08.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 -- 172.21.15.16:0/1556277961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c900716f0 msgr2=0x7f7c900af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 --2- 172.21.15.16:0/1556277961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c900716f0 0x7f7c900af580 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009900 tx=0x7f7c7801c7c0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 -- 172.21.15.16:0/1556277961 shutdown_connections 2024-09-10T21:14:08.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 --2- 172.21.15.16:0/1556277961 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7c900b13c0 0x7f7c900afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 --2- 172.21.15.16:0/1556277961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c900716f0 0x7f7c900af580 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 -- 172.21.15.16:0/1556277961 >> 172.21.15.16:0/1556277961 conn(0x7f7c9009b130 msgr2=0x7f7c900d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:08.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 -- 172.21.15.16:0/1556277961 shutdown_connections 2024-09-10T21:14:08.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 -- 172.21.15.16:0/1556277961 wait complete. 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 Processor -- start 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 -- start start 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c900716f0 0x7f7c90155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c900b13c0 0x7f7c9006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7c9015f400 0x7f7c90156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90168fc0 con 0x7f7c900b13c0 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90168e40 con 0x7f7c900716f0 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c96000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7c90169140 con 0x7f7c9015f400 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c87e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c900b13c0 0x7f7c9006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c8f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c900716f0 0x7f7c90155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c8fe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7c9015f400 0x7f7c90156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c87e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c900b13c0 0x7f7c9006e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59262/0 (socket says 172.21.15.16:59262) 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c8f400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c900716f0 0x7f7c90155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:57910/0 (socket says 172.21.15.16:57910) 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.399+0000 7f7c87e00640 1 -- 172.21.15.16:0/3367168127 learned_addr learned my addr 172.21.15.16:0/3367168127 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.403+0000 7f7c87e00640 1 -- 172.21.15.16:0/3367168127 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7c9015f400 msgr2=0x7f7c90156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.403+0000 7f7c87e00640 1 --2- 172.21.15.16:0/3367168127 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7c9015f400 0x7f7c90156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.403+0000 7f7c87e00640 1 -- 172.21.15.16:0/3367168127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c900716f0 msgr2=0x7f7c90155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.403+0000 7f7c87e00640 1 --2- 172.21.15.16:0/3367168127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c900716f0 0x7f7c90155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.403+0000 7f7c87e00640 1 -- 172.21.15.16:0/3367168127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c90157c40 con 0x7f7c900b13c0 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.403+0000 7f7c87e00640 1 --2- 172.21.15.16:0/3367168127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c900b13c0 0x7f7c9006e7c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009850 tx=0x7f7c780a5590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.403+0000 7f7c87400640 1 -- 172.21.15.16:0/3367168127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c780b2040 con 0x7f7c900b13c0 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.403+0000 7f7c87400640 1 -- 172.21.15.16:0/3367168127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c78004380 con 0x7f7c900b13c0 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.403+0000 7f7c96000640 1 -- 172.21.15.16:0/3367168127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c901fb050 con 0x7f7c900b13c0 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.403+0000 7f7c87400640 1 -- 172.21.15.16:0/3367168127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78005290 con 0x7f7c900b13c0 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.403+0000 7f7c96000640 1 -- 172.21.15.16:0/3367168127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c901fb610 con 0x7f7c900b13c0 2024-09-10T21:14:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.403+0000 7f7c96000640 1 -- 172.21.15.16:0/3367168127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c900b7110 con 0x7f7c900b13c0 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.403+0000 7f7c87400640 1 -- 172.21.15.16:0/3367168127 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7c78005430 con 0x7f7c900b13c0 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.407+0000 7f7c87400640 1 --2- 172.21.15.16:0/3367168127 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7c6c0725e0 0x7f7c6c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.407+0000 7f7c87400640 1 -- 172.21.15.16:0/3367168127 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f7c7812ca90 con 0x7f7c900b13c0 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.407+0000 7f7c8f400640 1 --2- 172.21.15.16:0/3367168127 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7c6c0725e0 0x7f7c6c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.407+0000 7f7c8f400640 1 --2- 172.21.15.16:0/3367168127 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7c6c0725e0 0x7f7c6c074aa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c80001800 tx=0x7f7c80009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:08.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.411+0000 7f7c87400640 1 -- 172.21.15.16:0/3367168127 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7c780164d0 con 0x7f7c900b13c0 2024-09-10T21:14:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.635+0000 7f7c96000640 1 -- 172.21.15.16:0/3367168127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c900aef70 con 0x7f7c900b13c0 2024-09-10T21:14:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.635+0000 7f7c87400640 1 -- 172.21.15.16:0/3367168127 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5222 (secure 0 0 0) 0x7f7c780f7b10 con 0x7f7c900b13c0 2024-09-10T21:14:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:08.642 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"btime":"2024-09-10T21:14:00:666666+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4199133238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4199133238},{"type":"v1","addr":"172.21.15.52:6835","nonce":4199133238}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2246088651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2246088651},{"type":"v1","addr":"172.21.15.16:6836","nonce":2246088651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1753951486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1753951486},{"type":"v1","addr":"172.21.15.16:6837","nonce":1753951486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:57.054843+0000","modified":"2024-09-10T21:14:00.072852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6276":{"gid":6276,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/943286634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":943286634},{"type":"v1","addr":"172.21.15.52:6834","nonce":943286634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6276,"qdb_cluster":[6276]},"id":6}]} 2024-09-10T21:14:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2024-09-10T21:14:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.635+0000 7f7c96000640 1 -- 172.21.15.16:0/3367168127 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7c6c0725e0 msgr2=0x7f7c6c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.635+0000 7f7c96000640 1 --2- 172.21.15.16:0/3367168127 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7c6c0725e0 0x7f7c6c074aa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c80001800 tx=0x7f7c80009040 comp rx=0 tx=0).stop 2024-09-10T21:14:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.639+0000 7f7c96000640 1 -- 172.21.15.16:0/3367168127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c900b13c0 msgr2=0x7f7c9006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.639+0000 7f7c96000640 1 --2- 172.21.15.16:0/3367168127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c900b13c0 0x7f7c9006e7c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009850 tx=0x7f7c780a5590 comp rx=0 tx=0).stop 2024-09-10T21:14:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.639+0000 7f7c96000640 1 -- 172.21.15.16:0/3367168127 shutdown_connections 2024-09-10T21:14:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.639+0000 7f7c96000640 1 --2- 172.21.15.16:0/3367168127 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7c6c0725e0 0x7f7c6c074aa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.639+0000 7f7c96000640 1 --2- 172.21.15.16:0/3367168127 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7c9015f400 0x7f7c90156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.639+0000 7f7c96000640 1 --2- 172.21.15.16:0/3367168127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c900b13c0 0x7f7c9006e7c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.639+0000 7f7c96000640 1 --2- 172.21.15.16:0/3367168127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c900716f0 0x7f7c90155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.639+0000 7f7c96000640 1 -- 172.21.15.16:0/3367168127 >> 172.21.15.16:0/3367168127 conn(0x7f7c9009b130 msgr2=0x7f7c900af510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.639+0000 7f7c96000640 1 -- 172.21.15.16:0/3367168127 shutdown_connections 2024-09-10T21:14:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.639+0000 7f7c96000640 1 -- 172.21.15.16:0/3367168127 wait complete. 2024-09-10T21:14:08.666 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-10T21:14:08.827 INFO:teuthology.orchestra.run.smithi016.stderr:waiting with handles open 2024-09-10T21:14:08.827 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:08.842 INFO:teuthology.orchestra.run.smithi052.stdout:[{"id":6019,"entity":{"name":{"type":"client","num":6019},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2099841206}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":3.0519751930000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9906363586195552,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6019 192.168.0.1:0/2099841206","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi052","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":6007,"entity":{"name":{"type":"client","num":6007},"addr":{"type":"any","addr":"172.21.15.16:0","nonce":3935336498}},"state":"open","num_leases":1,"num_caps":402,"request_load_avg":425,"uptime":4.2119657689999999,"requests_in_flight":400,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":185.69941970324896,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":174.34802804412189,"halflife":1.5},"recall_caps_throttle2o":{"value":153.18817088169331,"halflife":0.5},"session_cache_liveness":{"value":804.53672846543918,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6007 172.21.15.16:0/3935336498","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-10T21:14:08.843 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi052", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2099841206, "type": "any" }, "name": { "num": 6019, "type": "client" } }, "id": 6019, "inst": "client.6019 192.168.0.1:0/2099841206", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9906363586195552 }, "state": "open", "uptime": 3.051975193 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 3935336498, "type": "any" }, "name": { "num": 6007, "type": "client" } }, "id": 6007, "inst": "client.6007 172.21.15.16:0/3935336498", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 402, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 185.69941970324896 }, "recall_caps_throttle": { "halflife": 1.5, "value": 174.3480280441219 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 153.1881708816933 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 425, "requests_in_flight": 400, "session_cache_liveness": { "halflife": 300, "value": 804.5367284654392 }, "state": "open", "uptime": 4.211965769 } ] 2024-09-10T21:14:08.843 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-09-10T21:14:08.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:14:08.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.983+0000 7f9ebda00640 1 -- 172.21.15.16:0/3431007579 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eb80b1b40 msgr2=0x7f9eb80b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.983+0000 7f9ebda00640 1 --2- 172.21.15.16:0/3431007579 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eb80b1b40 0x7f9eb80b1fb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009c80 tx=0x7f9eb001cd10 comp rx=0 tx=0).stop 2024-09-10T21:14:08.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.983+0000 7f9ebda00640 1 -- 172.21.15.16:0/3431007579 shutdown_connections 2024-09-10T21:14:08.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.983+0000 7f9ebda00640 1 --2- 172.21.15.16:0/3431007579 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eb80b1b40 0x7f9eb80b1fb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.983+0000 7f9ebda00640 1 --2- 172.21.15.16:0/3431007579 >> v2:172.21.15.52:3300/0 conn(0x7f9eb80d2820 0x7f9eb80b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.983+0000 7f9ebda00640 1 -- 172.21.15.16:0/3431007579 >> 172.21.15.16:0/3431007579 conn(0x7f9eb80d1d00 msgr2=0x7f9eb80d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:08.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.983+0000 7f9ebda00640 1 -- 172.21.15.16:0/3431007579 shutdown_connections 2024-09-10T21:14:08.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.983+0000 7f9ebda00640 1 -- 172.21.15.16:0/3431007579 wait complete. 2024-09-10T21:14:08.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.983+0000 7f9ebda00640 1 Processor -- start 2024-09-10T21:14:08.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.983+0000 7f9ebda00640 1 -- start start 2024-09-10T21:14:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebda00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9eb80b1b40 0x7f9eb80b6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebda00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eb80cfd40 0x7f9eb81f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eb80d2820 0x7f9eb8152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebda00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb80b0670 con 0x7f9eb80d2820 2024-09-10T21:14:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebda00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb80b04f0 con 0x7f9eb80b1b40 2024-09-10T21:14:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebda00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb8168eb0 con 0x7f9eb80cfd40 2024-09-10T21:14:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eb80d2820 0x7f9eb8152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eb80d2820 0x7f9eb8152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59274/0 (socket says 172.21.15.16:59274) 2024-09-10T21:14:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebd000640 1 -- 172.21.15.16:0/1873115513 learned_addr learned my addr 172.21.15.16:0/1873115513 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9eaf400640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eb80cfd40 0x7f9eb81f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9eb7e00640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9eb80b1b40 0x7f9eb80b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebd000640 1 -- 172.21.15.16:0/1873115513 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eb80cfd40 msgr2=0x7f9eb81f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebd000640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eb80cfd40 0x7f9eb81f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebd000640 1 -- 172.21.15.16:0/1873115513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9eb80b1b40 msgr2=0x7f9eb80b6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebd000640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9eb80b1b40 0x7f9eb80b6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebd000640 1 -- 172.21.15.16:0/1873115513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb80cdf90 con 0x7f9eb80d2820 2024-09-10T21:14:08.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebd000640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eb80d2820 0x7f9eb8152ca0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9eb001b3c0 tx=0x7f9eb00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:08.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9eb4c00640 1 -- 172.21.15.16:0/1873115513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00bb070 con 0x7f9eb80d2820 2024-09-10T21:14:08.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9eb4c00640 1 -- 172.21.15.16:0/1873115513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9eb0003e00 con 0x7f9eb80d2820 2024-09-10T21:14:08.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9eb4c00640 1 -- 172.21.15.16:0/1873115513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00b2040 con 0x7f9eb80d2820 2024-09-10T21:14:08.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebda00640 1 -- 172.21.15.16:0/1873115513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb8159af0 con 0x7f9eb80d2820 2024-09-10T21:14:08.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebda00640 1 -- 172.21.15.16:0/1873115513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb81f86c0 con 0x7f9eb80d2820 2024-09-10T21:14:08.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.987+0000 7f9ebda00640 1 -- 172.21.15.16:0/1873115513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb80b1fb0 con 0x7f9eb80d2820 2024-09-10T21:14:08.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.991+0000 7f9eb4c00640 1 -- 172.21.15.16:0/1873115513 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9eb00b5030 con 0x7f9eb80d2820 2024-09-10T21:14:08.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.991+0000 7f9eb4c00640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e940725e0 0x7f9e94074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.991+0000 7f9eb4c00640 1 -- 172.21.15.16:0/1873115513 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f9eb012cf00 con 0x7f9eb80d2820 2024-09-10T21:14:08.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.991+0000 7f9eb7e00640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e940725e0 0x7f9e94074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.991+0000 7f9eb7e00640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e940725e0 0x7f9e94074aa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8007fd0 tx=0x7f9ea8009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:09.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:08.995+0000 7f9eb4c00640 1 -- 172.21.15.16:0/1873115513 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9eb00f1de0 con 0x7f9eb80d2820 2024-09-10T21:14:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9ebda00640 1 -- 172.21.15.16:0/1873115513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9eb80cc8f0 con 0x7f9eb80d2820 2024-09-10T21:14:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9eb4c00640 1 -- 172.21.15.16:0/1873115513 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f9eb0016990 con 0x7f9eb80d2820 2024-09-10T21:14:09.218 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:09.218 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T21:14:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9ebda00640 1 -- 172.21.15.16:0/1873115513 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e940725e0 msgr2=0x7f9e94074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9ebda00640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e940725e0 0x7f9e94074aa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8007fd0 tx=0x7f9ea8009040 comp rx=0 tx=0).stop 2024-09-10T21:14:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9ebda00640 1 -- 172.21.15.16:0/1873115513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eb80d2820 msgr2=0x7f9eb8152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9ebda00640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eb80d2820 0x7f9eb8152ca0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9eb001b3c0 tx=0x7f9eb00045e0 comp rx=0 tx=0).stop 2024-09-10T21:14:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9ebda00640 1 -- 172.21.15.16:0/1873115513 shutdown_connections 2024-09-10T21:14:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9ebda00640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e940725e0 0x7f9e94074aa0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9ebda00640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eb80d2820 0x7f9eb8152ca0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9ebda00640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9eb80cfd40 0x7f9eb81f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9ebda00640 1 --2- 172.21.15.16:0/1873115513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9eb80b1b40 0x7f9eb80b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9ebda00640 1 -- 172.21.15.16:0/1873115513 >> 172.21.15.16:0/1873115513 conn(0x7f9eb80d1d00 msgr2=0x7f9eb80d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9ebda00640 1 -- 172.21.15.16:0/1873115513 shutdown_connections 2024-09-10T21:14:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:09.211+0000 7f9ebda00640 1 -- 172.21.15.16:0/1873115513 wait complete. 2024-09-10T21:14:09.240 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-10T21:14:09.240 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:14:10.829 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:14.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:14:14.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.391+0000 7fc714400640 1 -- 172.21.15.16:0/3049072840 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc70c0d06b0 msgr2=0x7fc70c0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.391+0000 7fc714400640 1 --2- 172.21.15.16:0/3049072840 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc70c0d06b0 0x7fc70c0700a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009900 tx=0x7fc6f801d280 comp rx=0 tx=0).stop 2024-09-10T21:14:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.391+0000 7fc714400640 1 -- 172.21.15.16:0/3049072840 shutdown_connections 2024-09-10T21:14:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.391+0000 7fc714400640 1 --2- 172.21.15.16:0/3049072840 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc70c0d06b0 0x7fc70c0700a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.391+0000 7fc714400640 1 --2- 172.21.15.16:0/3049072840 >> v2:172.21.15.52:3300/0 conn(0x7fc70c15a270 0x7fc70c0ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.391+0000 7fc714400640 1 -- 172.21.15.16:0/3049072840 >> 172.21.15.16:0/3049072840 conn(0x7fc70c0cfb20 msgr2=0x7fc70c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:14.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.391+0000 7fc714400640 1 -- 172.21.15.16:0/3049072840 shutdown_connections 2024-09-10T21:14:14.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.391+0000 7fc714400640 1 -- 172.21.15.16:0/3049072840 wait complete. 2024-09-10T21:14:14.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc714400640 1 Processor -- start 2024-09-10T21:14:14.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc714400640 1 -- start start 2024-09-10T21:14:14.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc714400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70c0705e0 0x7fc70c0c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:14.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc714400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc70c0d06b0 0x7fc70c1fb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:14.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc714400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc70c15a270 0x7fc70c158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:14.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc714400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c072be0 con 0x7fc70c0705e0 2024-09-10T21:14:14.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc714400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c072a60 con 0x7fc70c15a270 2024-09-10T21:14:14.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc714400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc70c072d60 con 0x7fc70c0d06b0 2024-09-10T21:14:14.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc711a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70c0705e0 0x7fc70c0c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:14.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc711a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70c0705e0 0x7fc70c0c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47182/0 (socket says 172.21.15.16:47182) 2024-09-10T21:14:14.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc711a00640 1 -- 172.21.15.16:0/853644146 learned_addr learned my addr 172.21.15.16:0/853644146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:14.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc712400640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc70c15a270 0x7fc70c158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:14.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc711000640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc70c0d06b0 0x7fc70c1fb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:14.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc711a00640 1 -- 172.21.15.16:0/853644146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc70c0d06b0 msgr2=0x7fc70c1fb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:14.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc711a00640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc70c0d06b0 0x7fc70c1fb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:14.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc711a00640 1 -- 172.21.15.16:0/853644146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc70c15a270 msgr2=0x7fc70c158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc711a00640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc70c15a270 0x7fc70c158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc711a00640 1 -- 172.21.15.16:0/853644146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c15ef80 con 0x7fc70c0705e0 2024-09-10T21:14:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc711a00640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70c0705e0 0x7fc70c0c6230 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc70000ea90 tx=0x7fc70000ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc709600640 1 -- 172.21.15.16:0/853644146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70000cda0 con 0x7fc70c0705e0 2024-09-10T21:14:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc709600640 1 -- 172.21.15.16:0/853644146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc700004110 con 0x7fc70c0705e0 2024-09-10T21:14:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc709600640 1 -- 172.21.15.16:0/853644146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700003600 con 0x7fc70c0705e0 2024-09-10T21:14:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc714400640 1 -- 172.21.15.16:0/853644146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c15f220 con 0x7fc70c0705e0 2024-09-10T21:14:14.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.395+0000 7fc714400640 1 -- 172.21.15.16:0/853644146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c15f7e0 con 0x7fc70c0705e0 2024-09-10T21:14:14.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.399+0000 7fc714400640 1 -- 172.21.15.16:0/853644146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c0700a0 con 0x7fc70c0705e0 2024-09-10T21:14:14.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.399+0000 7fc709600640 1 -- 172.21.15.16:0/853644146 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc70000b040 con 0x7fc70c0705e0 2024-09-10T21:14:14.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.399+0000 7fc709600640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc6ec0725e0 0x7fc6ec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:14.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.399+0000 7fc709600640 1 -- 172.21.15.16:0/853644146 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fc7000933c0 con 0x7fc70c0705e0 2024-09-10T21:14:14.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.399+0000 7fc711000640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc6ec0725e0 0x7fc6ec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:14.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.399+0000 7fc711000640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc6ec0725e0 0x7fc6ec074aa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc6f80af000 tx=0x7fc6f801b060 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:14.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.403+0000 7fc709600640 1 -- 172.21.15.16:0/853644146 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc70005eb30 con 0x7fc70c0705e0 2024-09-10T21:14:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.615+0000 7fc714400640 1 -- 172.21.15.16:0/853644146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc70c0b5aa0 con 0x7fc70c0705e0 2024-09-10T21:14:14.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.619+0000 7fc709600640 1 -- 172.21.15.16:0/853644146 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fc70005e430 con 0x7fc70c0705e0 2024-09-10T21:14:14.625 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:14.625 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:14:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.619+0000 7fc714400640 1 -- 172.21.15.16:0/853644146 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc6ec0725e0 msgr2=0x7fc6ec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.619+0000 7fc714400640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc6ec0725e0 0x7fc6ec074aa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc6f80af000 tx=0x7fc6f801b060 comp rx=0 tx=0).stop 2024-09-10T21:14:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.619+0000 7fc714400640 1 -- 172.21.15.16:0/853644146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70c0705e0 msgr2=0x7fc70c0c6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.619+0000 7fc714400640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70c0705e0 0x7fc70c0c6230 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc70000ea90 tx=0x7fc70000ef50 comp rx=0 tx=0).stop 2024-09-10T21:14:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.619+0000 7fc714400640 1 -- 172.21.15.16:0/853644146 shutdown_connections 2024-09-10T21:14:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.619+0000 7fc714400640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc6ec0725e0 0x7fc6ec074aa0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.619+0000 7fc714400640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc70c15a270 0x7fc70c158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.619+0000 7fc714400640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc70c0d06b0 0x7fc70c1fb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:14.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.619+0000 7fc714400640 1 --2- 172.21.15.16:0/853644146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70c0705e0 0x7fc70c0c6230 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:14.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.619+0000 7fc714400640 1 -- 172.21.15.16:0/853644146 >> 172.21.15.16:0/853644146 conn(0x7fc70c0cfb20 msgr2=0x7fc70c155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:14.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.619+0000 7fc714400640 1 -- 172.21.15.16:0/853644146 shutdown_connections 2024-09-10T21:14:14.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.619+0000 7fc714400640 1 -- 172.21.15.16:0/853644146 wait complete. 2024-09-10T21:14:14.646 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:14:14.647 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-09-10T21:14:14.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:14:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 -- 172.21.15.16:0/739052162 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a240705e0 msgr2=0x7f2a240ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 --2- 172.21.15.16:0/739052162 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a240705e0 0x7f2a240ca170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009bc0 tx=0x7f2a1c01cb10 comp rx=0 tx=0).stop 2024-09-10T21:14:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 -- 172.21.15.16:0/739052162 shutdown_connections 2024-09-10T21:14:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 --2- 172.21.15.16:0/739052162 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a240705e0 0x7f2a240ca170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:14.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 --2- 172.21.15.16:0/739052162 >> v2:172.21.15.52:3300/0 conn(0x7f2a240d06b0 0x7f2a240700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:14.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 -- 172.21.15.16:0/739052162 >> 172.21.15.16:0/739052162 conn(0x7f2a240cfb20 msgr2=0x7f2a240cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:14.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 -- 172.21.15.16:0/739052162 shutdown_connections 2024-09-10T21:14:14.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 -- 172.21.15.16:0/739052162 wait complete. 2024-09-10T21:14:14.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 Processor -- start 2024-09-10T21:14:14.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 -- start start 2024-09-10T21:14:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a240705e0 0x7f2a2414d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240d06b0 0x7f2a2415b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a2415a270 0x7f2a24156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241671f0 con 0x7f2a240d06b0 2024-09-10T21:14:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a24072fd0 con 0x7f2a240705e0 2024-09-10T21:14:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a2b000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2a24167370 con 0x7f2a2415a270 2024-09-10T21:14:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240d06b0 0x7f2a2415b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240d06b0 0x7f2a2415b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47190/0 (socket says 172.21.15.16:47190) 2024-09-10T21:14:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a1be00640 1 -- 172.21.15.16:0/2844467886 learned_addr learned my addr 172.21.15.16:0/2844467886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a23e00640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a240705e0 0x7f2a2414d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a29000640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a2415a270 0x7f2a24156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a1be00640 1 -- 172.21.15.16:0/2844467886 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a2415a270 msgr2=0x7f2a24156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a1be00640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a2415a270 0x7f2a24156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a1be00640 1 -- 172.21.15.16:0/2844467886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a240705e0 msgr2=0x7f2a2414d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a1be00640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a240705e0 0x7f2a2414d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a1be00640 1 -- 172.21.15.16:0/2844467886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2415c6b0 con 0x7f2a240d06b0 2024-09-10T21:14:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.787+0000 7f2a23e00640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a240705e0 0x7f2a2414d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.791+0000 7f2a1be00640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240d06b0 0x7f2a2415b290 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00eb20 tx=0x7f2a0c00c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.791+0000 7f2a20c00640 1 -- 172.21.15.16:0/2844467886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c00ce10 con 0x7f2a240d06b0 2024-09-10T21:14:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.791+0000 7f2a20c00640 1 -- 172.21.15.16:0/2844467886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a0c004110 con 0x7f2a240d06b0 2024-09-10T21:14:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.791+0000 7f2a2b000640 1 -- 172.21.15.16:0/2844467886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2415bb30 con 0x7f2a240d06b0 2024-09-10T21:14:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.791+0000 7f2a20c00640 1 -- 172.21.15.16:0/2844467886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c003600 con 0x7f2a240d06b0 2024-09-10T21:14:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.791+0000 7f2a2b000640 1 -- 172.21.15.16:0/2844467886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a2415c0f0 con 0x7f2a240d06b0 2024-09-10T21:14:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.791+0000 7f2a2b000640 1 -- 172.21.15.16:0/2844467886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a240ca170 con 0x7f2a240d06b0 2024-09-10T21:14:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.791+0000 7f2a20c00640 1 -- 172.21.15.16:0/2844467886 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2a0c00b040 con 0x7f2a240d06b0 2024-09-10T21:14:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.791+0000 7f2a20c00640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a000725e0 0x7f2a00074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.791+0000 7f2a20c00640 1 -- 172.21.15.16:0/2844467886 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f2a0c0935d0 con 0x7f2a240d06b0 2024-09-10T21:14:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.791+0000 7f2a23e00640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a000725e0 0x7f2a00074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.795+0000 7f2a23e00640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a000725e0 0x7f2a00074aa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2a14000f70 tx=0x7f2a14009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:14.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:14.795+0000 7f2a20c00640 1 -- 172.21.15.16:0/2844467886 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2a0c05ec10 con 0x7f2a240d06b0 2024-09-10T21:14:14.833 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:15.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.007+0000 7f2a2b000640 1 -- 172.21.15.16:0/2844467886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2a240b5aa0 con 0x7f2a240d06b0 2024-09-10T21:14:15.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.007+0000 7f2a20c00640 1 -- 172.21.15.16:0/2844467886 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f2a0c05e510 con 0x7f2a240d06b0 2024-09-10T21:14:15.015 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:15.015 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:14:15.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.011+0000 7f2a2b000640 1 -- 172.21.15.16:0/2844467886 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a000725e0 msgr2=0x7f2a00074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:15.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.011+0000 7f2a2b000640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a000725e0 0x7f2a00074aa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2a14000f70 tx=0x7f2a14009040 comp rx=0 tx=0).stop 2024-09-10T21:14:15.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.011+0000 7f2a2b000640 1 -- 172.21.15.16:0/2844467886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240d06b0 msgr2=0x7f2a2415b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:15.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.011+0000 7f2a2b000640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240d06b0 0x7f2a2415b290 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00eb20 tx=0x7f2a0c00c3d0 comp rx=0 tx=0).stop 2024-09-10T21:14:15.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.011+0000 7f2a2b000640 1 -- 172.21.15.16:0/2844467886 shutdown_connections 2024-09-10T21:14:15.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.011+0000 7f2a2b000640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a000725e0 0x7f2a00074aa0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:15.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.011+0000 7f2a2b000640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a2415a270 0x7f2a24156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:15.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.011+0000 7f2a2b000640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240d06b0 0x7f2a2415b290 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:15.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.011+0000 7f2a2b000640 1 --2- 172.21.15.16:0/2844467886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a240705e0 0x7f2a2414d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:15.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.011+0000 7f2a2b000640 1 -- 172.21.15.16:0/2844467886 >> 172.21.15.16:0/2844467886 conn(0x7f2a240cfb20 msgr2=0x7f2a240c5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:15.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.011+0000 7f2a2b000640 1 -- 172.21.15.16:0/2844467886 shutdown_connections 2024-09-10T21:14:15.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:15.011+0000 7f2a2b000640 1 -- 172.21.15.16:0/2844467886 wait complete. 2024-09-10T21:14:15.037 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:14:15.037 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:14:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:20.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:14:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.187+0000 7ff4c4e00640 1 -- 172.21.15.16:0/3525953619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c00cea70 msgr2=0x7ff4c00cef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.187+0000 7ff4c4e00640 1 --2- 172.21.15.16:0/3525953619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c00cea70 0x7ff4c00cef20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009c80 tx=0x7ff4b801cda0 comp rx=0 tx=0).stop 2024-09-10T21:14:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.187+0000 7ff4c4e00640 1 -- 172.21.15.16:0/3525953619 shutdown_connections 2024-09-10T21:14:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.187+0000 7ff4c4e00640 1 --2- 172.21.15.16:0/3525953619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c00cea70 0x7ff4c00cef20 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.187+0000 7ff4c4e00640 1 --2- 172.21.15.16:0/3525953619 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff4c006f370 0x7ff4c00ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.187+0000 7ff4c4e00640 1 -- 172.21.15.16:0/3525953619 >> 172.21.15.16:0/3525953619 conn(0x7ff4c00cd5f0 msgr2=0x7ff4c00cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.187+0000 7ff4c4e00640 1 -- 172.21.15.16:0/3525953619 shutdown_connections 2024-09-10T21:14:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.187+0000 7ff4c4e00640 1 -- 172.21.15.16:0/3525953619 wait complete. 2024-09-10T21:14:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4c4e00640 1 Processor -- start 2024-09-10T21:14:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4c4e00640 1 -- start start 2024-09-10T21:14:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4c4e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4c006f370 0x7ff4c0153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4c4e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff4c00cea70 0x7ff4c00b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4c4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c015dfd0 0x7ff4c00c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4c4e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c016af80 con 0x7ff4c015dfd0 2024-09-10T21:14:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4c4e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c016ae00 con 0x7ff4c006f370 2024-09-10T21:14:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4c4e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c016b100 con 0x7ff4c00cea70 2024-09-10T21:14:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c015dfd0 0x7ff4c00c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c015dfd0 0x7ff4c00c5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47202/0 (socket says 172.21.15.16:47202) 2024-09-10T21:14:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4bfe00640 1 -- 172.21.15.16:0/2099445829 learned_addr learned my addr 172.21.15.16:0/2099445829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4bf400640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4c006f370 0x7ff4c0153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4b7e00640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff4c00cea70 0x7ff4c00b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4bfe00640 1 -- 172.21.15.16:0/2099445829 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff4c00cea70 msgr2=0x7ff4c00b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4bfe00640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff4c00cea70 0x7ff4c00b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4bfe00640 1 -- 172.21.15.16:0/2099445829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4c006f370 msgr2=0x7ff4c0153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4bfe00640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4c006f370 0x7ff4c0153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4bfe00640 1 -- 172.21.15.16:0/2099445829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c00b3f60 con 0x7ff4c015dfd0 2024-09-10T21:14:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4bfe00640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c015dfd0 0x7ff4c00c5eb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009bd0 tx=0x7ff4b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4b7400640 1 -- 172.21.15.16:0/2099445829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80bb070 con 0x7ff4c015dfd0 2024-09-10T21:14:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4b7400640 1 -- 172.21.15.16:0/2099445829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4b8004a50 con 0x7ff4c015dfd0 2024-09-10T21:14:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4b7400640 1 -- 172.21.15.16:0/2099445829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80b2040 con 0x7ff4c015dfd0 2024-09-10T21:14:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4c4e00640 1 -- 172.21.15.16:0/2099445829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c0154cd0 con 0x7ff4c015dfd0 2024-09-10T21:14:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.191+0000 7ff4c4e00640 1 -- 172.21.15.16:0/2099445829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c00c56b0 con 0x7ff4c015dfd0 2024-09-10T21:14:20.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.195+0000 7ff4c4e00640 1 -- 172.21.15.16:0/2099445829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c00ce110 con 0x7ff4c015dfd0 2024-09-10T21:14:20.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.195+0000 7ff4b7400640 1 -- 172.21.15.16:0/2099445829 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff4b80b5030 con 0x7ff4c015dfd0 2024-09-10T21:14:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.195+0000 7ff4b7400640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff49c0725e0 0x7ff49c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.195+0000 7ff4b7400640 1 -- 172.21.15.16:0/2099445829 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7ff4b812cef0 con 0x7ff4c015dfd0 2024-09-10T21:14:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.195+0000 7ff4bf400640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff49c0725e0 0x7ff49c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.195+0000 7ff4bf400640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff49c0725e0 0x7ff49c074aa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0007fd0 tx=0x7ff4b0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.199+0000 7ff4b7400640 1 -- 172.21.15.16:0/2099445829 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff4b8016990 con 0x7ff4c015dfd0 2024-09-10T21:14:20.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.407+0000 7ff4c4e00640 1 -- 172.21.15.16:0/2099445829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff4c00cdc30 con 0x7ff4c015dfd0 2024-09-10T21:14:20.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.407+0000 7ff4b7400640 1 -- 172.21.15.16:0/2099445829 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7ff4b80f7f70 con 0x7ff4c015dfd0 2024-09-10T21:14:20.417 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:20.417 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:14:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.411+0000 7ff4c4e00640 1 -- 172.21.15.16:0/2099445829 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff49c0725e0 msgr2=0x7ff49c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.411+0000 7ff4c4e00640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff49c0725e0 0x7ff49c074aa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0007fd0 tx=0x7ff4b0009040 comp rx=0 tx=0).stop 2024-09-10T21:14:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.411+0000 7ff4c4e00640 1 -- 172.21.15.16:0/2099445829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c015dfd0 msgr2=0x7ff4c00c5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.411+0000 7ff4c4e00640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c015dfd0 0x7ff4c00c5eb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009bd0 tx=0x7ff4b8004270 comp rx=0 tx=0).stop 2024-09-10T21:14:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.411+0000 7ff4c4e00640 1 -- 172.21.15.16:0/2099445829 shutdown_connections 2024-09-10T21:14:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.411+0000 7ff4c4e00640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff49c0725e0 0x7ff49c074aa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.411+0000 7ff4c4e00640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c015dfd0 0x7ff4c00c5eb0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.411+0000 7ff4c4e00640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff4c00cea70 0x7ff4c00b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.411+0000 7ff4c4e00640 1 --2- 172.21.15.16:0/2099445829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4c006f370 0x7ff4c0153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.411+0000 7ff4c4e00640 1 -- 172.21.15.16:0/2099445829 >> 172.21.15.16:0/2099445829 conn(0x7ff4c00cd5f0 msgr2=0x7ff4c006d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.411+0000 7ff4c4e00640 1 -- 172.21.15.16:0/2099445829 shutdown_connections 2024-09-10T21:14:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:20.411+0000 7ff4c4e00640 1 -- 172.21.15.16:0/2099445829 wait complete. 2024-09-10T21:14:20.438 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:14:20.438 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:14:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:22.842 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:23.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:14:23.641 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:14:24.844 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:25.439 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:14:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.591+0000 7f2fda600640 1 -- 172.21.15.16:0/1870660717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd415f400 msgr2=0x7f2fd4070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.591+0000 7f2fda600640 1 --2- 172.21.15.16:0/1870660717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd415f400 0x7f2fd4070fe0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4074cc0 tx=0x7f2fc401c780 comp rx=0 tx=0).stop 2024-09-10T21:14:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.591+0000 7f2fda600640 1 -- 172.21.15.16:0/1870660717 shutdown_connections 2024-09-10T21:14:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.591+0000 7f2fda600640 1 --2- 172.21.15.16:0/1870660717 >> v2:172.21.15.52:3300/0 conn(0x7f2fd40cfc80 0x7f2fd40d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.591+0000 7f2fda600640 1 --2- 172.21.15.16:0/1870660717 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd40b13c0 0x7f2fd4071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.591+0000 7f2fda600640 1 --2- 172.21.15.16:0/1870660717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd415f400 0x7f2fd4070fe0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.591+0000 7f2fda600640 1 -- 172.21.15.16:0/1870660717 >> 172.21.15.16:0/1870660717 conn(0x7f2fd409b130 msgr2=0x7f2fd40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.591+0000 7f2fda600640 1 -- 172.21.15.16:0/1870660717 shutdown_connections 2024-09-10T21:14:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.591+0000 7f2fda600640 1 -- 172.21.15.16:0/1870660717 wait complete. 2024-09-10T21:14:25.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.591+0000 7f2fda600640 1 Processor -- start 2024-09-10T21:14:25.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.591+0000 7f2fda600640 1 -- start start 2024-09-10T21:14:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fda600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fd40b13c0 0x7f2fd4156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fda600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd40cfc80 0x7f2fd41555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fda600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd415f400 0x7f2fd40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fda600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd40afd90 con 0x7f2fd415f400 2024-09-10T21:14:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fda600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd40d1260 con 0x7f2fd40b13c0 2024-09-10T21:14:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fda600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd40aff10 con 0x7f2fd40cfc80 2024-09-10T21:14:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fd3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd415f400 0x7f2fd40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fd3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd415f400 0x7f2fd40b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55080/0 (socket says 172.21.15.16:55080) 2024-09-10T21:14:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fd3e00640 1 -- 172.21.15.16:0/2593950580 learned_addr learned my addr 172.21.15.16:0/2593950580 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fcbe00640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd40cfc80 0x7f2fd41555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fd3400640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fd40b13c0 0x7f2fd4156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fd3e00640 1 -- 172.21.15.16:0/2593950580 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd40cfc80 msgr2=0x7f2fd41555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fd3e00640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd40cfc80 0x7f2fd41555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fd3e00640 1 -- 172.21.15.16:0/2593950580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fd40b13c0 msgr2=0x7f2fd4156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fd3e00640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fd40b13c0 0x7f2fd4156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fd3e00640 1 -- 172.21.15.16:0/2593950580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd41651e0 con 0x7f2fd415f400 2024-09-10T21:14:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fd3400640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fd40b13c0 0x7f2fd4156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fd3e00640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd415f400 0x7f2fd40b4bb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00eb50 tx=0x7f2fcc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fcb400640 1 -- 172.21.15.16:0/2593950580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc004d30 con 0x7f2fd415f400 2024-09-10T21:14:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fcb400640 1 -- 172.21.15.16:0/2593950580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fcc00ee50 con 0x7f2fd415f400 2024-09-10T21:14:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fda600640 1 -- 172.21.15.16:0/2593950580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd41654c0 con 0x7f2fd415f400 2024-09-10T21:14:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fcb400640 1 -- 172.21.15.16:0/2593950580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc005280 con 0x7f2fd415f400 2024-09-10T21:14:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fda600640 1 -- 172.21.15.16:0/2593950580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd406ea80 con 0x7f2fd415f400 2024-09-10T21:14:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.595+0000 7f2fda600640 1 -- 172.21.15.16:0/2593950580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd40d00d0 con 0x7f2fd415f400 2024-09-10T21:14:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.599+0000 7f2fcb400640 1 -- 172.21.15.16:0/2593950580 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2fcc005420 con 0x7f2fd415f400 2024-09-10T21:14:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.599+0000 7f2fcb400640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2fb0072610 0x7f2fb0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.599+0000 7f2fcb400640 1 -- 172.21.15.16:0/2593950580 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f2fcc093ef0 con 0x7f2fd415f400 2024-09-10T21:14:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.599+0000 7f2fd3400640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2fb0072610 0x7f2fb0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.599+0000 7f2fd3400640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2fb0072610 0x7f2fb0074ad0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40af000 tx=0x7f2fc401afc0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:25.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.603+0000 7f2fcb400640 1 -- 172.21.15.16:0/2593950580 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2fcc05f5c0 con 0x7f2fd415f400 2024-09-10T21:14:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fda600640 1 -- 172.21.15.16:0/2593950580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2fd40b2630 con 0x7f2fd415f400 2024-09-10T21:14:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fcb400640 1 -- 172.21.15.16:0/2593950580 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f2fcc05eec0 con 0x7f2fd415f400 2024-09-10T21:14:25.822 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:25.822 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:14:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fda600640 1 -- 172.21.15.16:0/2593950580 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2fb0072610 msgr2=0x7f2fb0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fda600640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2fb0072610 0x7f2fb0074ad0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40af000 tx=0x7f2fc401afc0 comp rx=0 tx=0).stop 2024-09-10T21:14:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fda600640 1 -- 172.21.15.16:0/2593950580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd415f400 msgr2=0x7f2fd40b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fda600640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd415f400 0x7f2fd40b4bb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00eb50 tx=0x7f2fcc00c6a0 comp rx=0 tx=0).stop 2024-09-10T21:14:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fda600640 1 -- 172.21.15.16:0/2593950580 shutdown_connections 2024-09-10T21:14:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fda600640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2fb0072610 0x7f2fb0074ad0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fda600640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd415f400 0x7f2fd40b4bb0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fda600640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd40cfc80 0x7f2fd41555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fda600640 1 --2- 172.21.15.16:0/2593950580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fd40b13c0 0x7f2fd4156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fda600640 1 -- 172.21.15.16:0/2593950580 >> 172.21.15.16:0/2593950580 conn(0x7f2fd409b130 msgr2=0x7f2fd4158fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fda600640 1 -- 172.21.15.16:0/2593950580 shutdown_connections 2024-09-10T21:14:25.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:25.815+0000 7f2fda600640 1 -- 172.21.15.16:0/2593950580 wait complete. 2024-09-10T21:14:25.844 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:14:25.844 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:14:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:28.848 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:30.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:14:30.850 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:31.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.995+0000 7fb754e00640 1 -- 172.21.15.16:0/2484993470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7500b2290 msgr2=0x7fb7500b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:31.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.995+0000 7fb754e00640 1 --2- 172.21.15.16:0/2484993470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7500b2290 0x7fb7500b2740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb748009bc0 tx=0x7fb74801cb40 comp rx=0 tx=0).stop 2024-09-10T21:14:31.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.995+0000 7fb754e00640 1 -- 172.21.15.16:0/2484993470 shutdown_connections 2024-09-10T21:14:31.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.995+0000 7fb754e00640 1 --2- 172.21.15.16:0/2484993470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7500b2290 0x7fb7500b2740 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:31.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.995+0000 7fb754e00640 1 --2- 172.21.15.16:0/2484993470 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb750161900 0x7fb7500d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:31.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.995+0000 7fb754e00640 1 -- 172.21.15.16:0/2484993470 >> 172.21.15.16:0/2484993470 conn(0x7fb7500d1d80 msgr2=0x7fb7500d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:31.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.995+0000 7fb754e00640 1 -- 172.21.15.16:0/2484993470 shutdown_connections 2024-09-10T21:14:31.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.995+0000 7fb754e00640 1 -- 172.21.15.16:0/2484993470 wait complete. 2024-09-10T21:14:31.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb754e00640 1 Processor -- start 2024-09-10T21:14:31.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb754e00640 1 -- start start 2024-09-10T21:14:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb754e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7500b2290 0x7fb7500af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb754e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb750161520 0x7fb7500afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb754e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb750161900 0x7fb7501faed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb754e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb75016aff0 con 0x7fb7500b2290 2024-09-10T21:14:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb754e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb75016ae70 con 0x7fb750161520 2024-09-10T21:14:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb754e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb75016b170 con 0x7fb750161900 2024-09-10T21:14:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb74de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7500b2290 0x7fb7500af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb74de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7500b2290 0x7fb7500af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55086/0 (socket says 172.21.15.16:55086) 2024-09-10T21:14:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb74de00640 1 -- 172.21.15.16:0/992606644 learned_addr learned my addr 172.21.15.16:0/992606644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:31.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb74d400640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb750161520 0x7fb7500afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb74e800640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb750161900 0x7fb7501faed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb74de00640 1 -- 172.21.15.16:0/992606644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb750161900 msgr2=0x7fb7501faed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb74de00640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb750161900 0x7fb7501faed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb74de00640 1 -- 172.21.15.16:0/992606644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb750161520 msgr2=0x7fb7500afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb74de00640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb750161520 0x7fb7500afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb74de00640 1 -- 172.21.15.16:0/992606644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7500b7970 con 0x7fb7500b2290 2024-09-10T21:14:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb74d400640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb750161520 0x7fb7500afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb74de00640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7500b2290 0x7fb7500af8d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb74000c500 tx=0x7fb74000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb746000640 1 -- 172.21.15.16:0/992606644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740017070 con 0x7fb7500b2290 2024-09-10T21:14:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb746000640 1 -- 172.21.15.16:0/992606644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7400053c0 con 0x7fb7500b2290 2024-09-10T21:14:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb746000640 1 -- 172.21.15.16:0/992606644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740012890 con 0x7fb7500b2290 2024-09-10T21:14:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb754e00640 1 -- 172.21.15.16:0/992606644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7501f7820 con 0x7fb7500b2290 2024-09-10T21:14:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb754e00640 1 -- 172.21.15.16:0/992606644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7501f7de0 con 0x7fb7500b2290 2024-09-10T21:14:31.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:30.999+0000 7fb754e00640 1 -- 172.21.15.16:0/992606644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7500b2740 con 0x7fb7500b2290 2024-09-10T21:14:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.003+0000 7fb746000640 1 -- 172.21.15.16:0/992606644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb740003600 con 0x7fb7500b2290 2024-09-10T21:14:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.003+0000 7fb746000640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7300725e0 0x7fb730074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.003+0000 7fb746000640 1 -- 172.21.15.16:0/992606644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fb7400928f0 con 0x7fb7500b2290 2024-09-10T21:14:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.003+0000 7fb74d400640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7300725e0 0x7fb730074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.003+0000 7fb74d400640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7300725e0 0x7fb730074aa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb75006fe30 tx=0x7fb73c009260 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:31.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.007+0000 7fb746000640 1 -- 172.21.15.16:0/992606644 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb74005df40 con 0x7fb7500b2290 2024-09-10T21:14:31.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.215+0000 7fb754e00640 1 -- 172.21.15.16:0/992606644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb7500d0e20 con 0x7fb7500b2290 2024-09-10T21:14:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.215+0000 7fb746000640 1 -- 172.21.15.16:0/992606644 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fb74005d840 con 0x7fb7500b2290 2024-09-10T21:14:31.224 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:31.224 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:14:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.219+0000 7fb754e00640 1 -- 172.21.15.16:0/992606644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7300725e0 msgr2=0x7fb730074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.219+0000 7fb754e00640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7300725e0 0x7fb730074aa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb75006fe30 tx=0x7fb73c009260 comp rx=0 tx=0).stop 2024-09-10T21:14:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.219+0000 7fb754e00640 1 -- 172.21.15.16:0/992606644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7500b2290 msgr2=0x7fb7500af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.219+0000 7fb754e00640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7500b2290 0x7fb7500af8d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb74000c500 tx=0x7fb74000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:14:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.219+0000 7fb754e00640 1 -- 172.21.15.16:0/992606644 shutdown_connections 2024-09-10T21:14:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.219+0000 7fb754e00640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7300725e0 0x7fb730074aa0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.219+0000 7fb754e00640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb750161900 0x7fb7501faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.219+0000 7fb754e00640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb750161520 0x7fb7500afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.219+0000 7fb754e00640 1 --2- 172.21.15.16:0/992606644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7500b2290 0x7fb7500af8d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.219+0000 7fb754e00640 1 -- 172.21.15.16:0/992606644 >> 172.21.15.16:0/992606644 conn(0x7fb7500d1d80 msgr2=0x7fb750071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.219+0000 7fb754e00640 1 -- 172.21.15.16:0/992606644 shutdown_connections 2024-09-10T21:14:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:31.219+0000 7fb754e00640 1 -- 172.21.15.16:0/992606644 wait complete. 2024-09-10T21:14:31.246 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:14:31.246 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:14:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:36.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:14:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 -- 172.21.15.16:0/2311084730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d500716f0 msgr2=0x7f9d500af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 --2- 172.21.15.16:0/2311084730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d500716f0 0x7f9d500af580 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009840 tx=0x7f9d3801c700 comp rx=0 tx=0).stop 2024-09-10T21:14:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 -- 172.21.15.16:0/2311084730 shutdown_connections 2024-09-10T21:14:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 --2- 172.21.15.16:0/2311084730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d500716f0 0x7f9d500af580 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 --2- 172.21.15.16:0/2311084730 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d500b13c0 0x7f9d500afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 -- 172.21.15.16:0/2311084730 >> 172.21.15.16:0/2311084730 conn(0x7f9d5009b130 msgr2=0x7f9d500d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 -- 172.21.15.16:0/2311084730 shutdown_connections 2024-09-10T21:14:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 -- 172.21.15.16:0/2311084730 wait complete. 2024-09-10T21:14:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 Processor -- start 2024-09-10T21:14:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 -- start start 2024-09-10T21:14:36.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d500716f0 0x7f9d5015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:36.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d500b13c0 0x7f9d501f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:36.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d5015f400 0x7f9d501f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:36.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d50168fb0 con 0x7f9d500716f0 2024-09-10T21:14:36.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d50168e30 con 0x7f9d500b13c0 2024-09-10T21:14:36.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.399+0000 7f9d56000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9d50169130 con 0x7f9d5015f400 2024-09-10T21:14:36.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d4f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d500716f0 0x7f9d5015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:36.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d4f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d500716f0 0x7f9d5015a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37524/0 (socket says 172.21.15.16:37524) 2024-09-10T21:14:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d4f400640 1 -- 172.21.15.16:0/3217015204 learned_addr learned my addr 172.21.15.16:0/3217015204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d47e00640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d500b13c0 0x7f9d501f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d4f400640 1 -- 172.21.15.16:0/3217015204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d5015f400 msgr2=0x7f9d501f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d4fe00640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d5015f400 0x7f9d501f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d4f400640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d5015f400 0x7f9d501f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d4f400640 1 -- 172.21.15.16:0/3217015204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d500b13c0 msgr2=0x7f9d501f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d4f400640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d500b13c0 0x7f9d501f6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d4f400640 1 -- 172.21.15.16:0/3217015204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d50159840 con 0x7f9d500716f0 2024-09-10T21:14:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d47e00640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d500b13c0 0x7f9d501f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:14:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d4fe00640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d5015f400 0x7f9d501f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:14:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d4f400640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d500716f0 0x7f9d5015a8c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000c400 tx=0x7f9d4000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d47400640 1 -- 172.21.15.16:0/3217015204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40017070 con 0x7f9d500716f0 2024-09-10T21:14:36.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d47400640 1 -- 172.21.15.16:0/3217015204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d40003600 con 0x7f9d500716f0 2024-09-10T21:14:36.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d47400640 1 -- 172.21.15.16:0/3217015204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40003a30 con 0x7f9d500716f0 2024-09-10T21:14:36.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d56000640 1 -- 172.21.15.16:0/3217015204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d50159b00 con 0x7f9d500716f0 2024-09-10T21:14:36.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d56000640 1 -- 172.21.15.16:0/3217015204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d5015a0c0 con 0x7f9d500716f0 2024-09-10T21:14:36.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d56000640 1 -- 172.21.15.16:0/3217015204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d50071f20 con 0x7f9d500716f0 2024-09-10T21:14:36.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.403+0000 7f9d47400640 1 -- 172.21.15.16:0/3217015204 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9d40003bd0 con 0x7f9d500716f0 2024-09-10T21:14:36.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.407+0000 7f9d47400640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d2c072610 0x7f9d2c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:36.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.407+0000 7f9d47400640 1 -- 172.21.15.16:0/3217015204 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f9d40092830 con 0x7f9d500716f0 2024-09-10T21:14:36.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.407+0000 7f9d47e00640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d2c072610 0x7f9d2c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.407+0000 7f9d47e00640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d2c072610 0x7f9d2c074ad0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9d380af000 tx=0x7f9d3801af40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:36.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.411+0000 7f9d47400640 1 -- 172.21.15.16:0/3217015204 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9d4005df00 con 0x7f9d500716f0 2024-09-10T21:14:36.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d56000640 1 -- 172.21.15.16:0/3217015204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9d500714b0 con 0x7f9d500716f0 2024-09-10T21:14:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d47400640 1 -- 172.21.15.16:0/3217015204 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f9d4005d800 con 0x7f9d500716f0 2024-09-10T21:14:36.626 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:36.626 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:14:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d56000640 1 -- 172.21.15.16:0/3217015204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d2c072610 msgr2=0x7f9d2c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d56000640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d2c072610 0x7f9d2c074ad0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9d380af000 tx=0x7f9d3801af40 comp rx=0 tx=0).stop 2024-09-10T21:14:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d56000640 1 -- 172.21.15.16:0/3217015204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d500716f0 msgr2=0x7f9d5015a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d56000640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d500716f0 0x7f9d5015a8c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000c400 tx=0x7f9d4000c8c0 comp rx=0 tx=0).stop 2024-09-10T21:14:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d56000640 1 -- 172.21.15.16:0/3217015204 shutdown_connections 2024-09-10T21:14:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d56000640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9d2c072610 0x7f9d2c074ad0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d56000640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9d5015f400 0x7f9d501f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d56000640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d500b13c0 0x7f9d501f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d56000640 1 --2- 172.21.15.16:0/3217015204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d500716f0 0x7f9d5015a8c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d56000640 1 -- 172.21.15.16:0/3217015204 >> 172.21.15.16:0/3217015204 conn(0x7f9d5009b130 msgr2=0x7f9d500cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d56000640 1 -- 172.21.15.16:0/3217015204 shutdown_connections 2024-09-10T21:14:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:36.619+0000 7f9d56000640 1 -- 172.21.15.16:0/3217015204 wait complete. 2024-09-10T21:14:36.648 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:14:36.648 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:14:36.857 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:38.859 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:41.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:14:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.799+0000 7f59efe00640 1 -- 172.21.15.16:0/3250689430 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f59e80b13c0 msgr2=0x7f59e8071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.799+0000 7f59efe00640 1 --2- 172.21.15.16:0/3250689430 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f59e80b13c0 0x7f59e8071520 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009900 tx=0x7f59d401c7c0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.799+0000 7f59efe00640 1 -- 172.21.15.16:0/3250689430 shutdown_connections 2024-09-10T21:14:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.799+0000 7f59efe00640 1 --2- 172.21.15.16:0/3250689430 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f59e80b13c0 0x7f59e8071520 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.799+0000 7f59efe00640 1 --2- 172.21.15.16:0/3250689430 >> v2:172.21.15.52:3300/0 conn(0x7f59e815f400 0x7f59e8070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.799+0000 7f59efe00640 1 -- 172.21.15.16:0/3250689430 >> 172.21.15.16:0/3250689430 conn(0x7f59e809b130 msgr2=0x7f59e80d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.799+0000 7f59efe00640 1 -- 172.21.15.16:0/3250689430 shutdown_connections 2024-09-10T21:14:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.799+0000 7f59efe00640 1 -- 172.21.15.16:0/3250689430 wait complete. 2024-09-10T21:14:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59efe00640 1 Processor -- start 2024-09-10T21:14:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59efe00640 1 -- start start 2024-09-10T21:14:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59efe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59e80b13c0 0x7f59e81f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cfc80 0x7f59e8158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59efe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f59e815f400 0x7f59e80b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59efe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59e8169770 con 0x7f59e80cfc80 2024-09-10T21:14:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59efe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81695f0 con 0x7f59e80b13c0 2024-09-10T21:14:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59efe00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f59e81698f0 con 0x7f59e815f400 2024-09-10T21:14:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59eca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cfc80 0x7f59e8158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59eca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cfc80 0x7f59e8158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37540/0 (socket says 172.21.15.16:37540) 2024-09-10T21:14:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59eca00640 1 -- 172.21.15.16:0/1716871026 learned_addr learned my addr 172.21.15.16:0/1716871026 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59ed400640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59e80b13c0 0x7f59e81f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59ede00640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f59e815f400 0x7f59e80b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59eca00640 1 -- 172.21.15.16:0/1716871026 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f59e815f400 msgr2=0x7f59e80b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59eca00640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f59e815f400 0x7f59e80b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59eca00640 1 -- 172.21.15.16:0/1716871026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59e80b13c0 msgr2=0x7f59e81f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59eca00640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59e80b13c0 0x7f59e81f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59eca00640 1 -- 172.21.15.16:0/1716871026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e81fbc50 con 0x7f59e80cfc80 2024-09-10T21:14:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59eca00640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cfc80 0x7f59e8158c00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009850 tx=0x7f59d40a5590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59e5600640 1 -- 172.21.15.16:0/1716871026 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d40b2040 con 0x7f59e80cfc80 2024-09-10T21:14:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59e5600640 1 -- 172.21.15.16:0/1716871026 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59d4004380 con 0x7f59e80cfc80 2024-09-10T21:14:41.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59e5600640 1 -- 172.21.15.16:0/1716871026 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4005290 con 0x7f59e80cfc80 2024-09-10T21:14:41.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59efe00640 1 -- 172.21.15.16:0/1716871026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e81fbf30 con 0x7f59e80cfc80 2024-09-10T21:14:41.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.803+0000 7f59efe00640 1 -- 172.21.15.16:0/1716871026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e81fc4f0 con 0x7f59e80cfc80 2024-09-10T21:14:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.807+0000 7f59efe00640 1 -- 172.21.15.16:0/1716871026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e80d00d0 con 0x7f59e80cfc80 2024-09-10T21:14:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.807+0000 7f59e5600640 1 -- 172.21.15.16:0/1716871026 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f59d4005430 con 0x7f59e80cfc80 2024-09-10T21:14:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.807+0000 7f59e5600640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59c8072610 0x7f59c8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.807+0000 7f59e5600640 1 -- 172.21.15.16:0/1716871026 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f59d412cb60 con 0x7f59e80cfc80 2024-09-10T21:14:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.807+0000 7f59ed400640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59c8072610 0x7f59c8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.807+0000 7f59ed400640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59c8072610 0x7f59c8074ad0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f59dc005fd0 tx=0x7f59dc005f40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:41.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:41.811+0000 7f59e5600640 1 -- 172.21.15.16:0/1716871026 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f59d40164d0 con 0x7f59e80cfc80 2024-09-10T21:14:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59efe00640 1 -- 172.21.15.16:0/1716871026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f59e815a9e0 con 0x7f59e80cfc80 2024-09-10T21:14:42.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59e5600640 1 -- 172.21.15.16:0/1716871026 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f59d40f8230 con 0x7f59e80cfc80 2024-09-10T21:14:42.033 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:42.033 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:14:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59efe00640 1 -- 172.21.15.16:0/1716871026 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59c8072610 msgr2=0x7f59c8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59efe00640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59c8072610 0x7f59c8074ad0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f59dc005fd0 tx=0x7f59dc005f40 comp rx=0 tx=0).stop 2024-09-10T21:14:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59efe00640 1 -- 172.21.15.16:0/1716871026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cfc80 msgr2=0x7f59e8158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59efe00640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cfc80 0x7f59e8158c00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009850 tx=0x7f59d40a5590 comp rx=0 tx=0).stop 2024-09-10T21:14:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59efe00640 1 -- 172.21.15.16:0/1716871026 shutdown_connections 2024-09-10T21:14:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59efe00640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f59c8072610 0x7f59c8074ad0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59efe00640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f59e815f400 0x7f59e80b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59efe00640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cfc80 0x7f59e8158c00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59efe00640 1 --2- 172.21.15.16:0/1716871026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59e80b13c0 0x7f59e81f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:42.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59efe00640 1 -- 172.21.15.16:0/1716871026 >> 172.21.15.16:0/1716871026 conn(0x7f59e809b130 msgr2=0x7f59e806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:42.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59efe00640 1 -- 172.21.15.16:0/1716871026 shutdown_connections 2024-09-10T21:14:42.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:42.027+0000 7f59efe00640 1 -- 172.21.15.16:0/1716871026 wait complete. 2024-09-10T21:14:42.056 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:14:42.056 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:14:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:44.866 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:46.867 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:47.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:14:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.207+0000 7fbcdf800640 1 -- 172.21.15.16:0/263345787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd80716f0 msgr2=0x7fbcd80af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.207+0000 7fbcdf800640 1 --2- 172.21.15.16:0/263345787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd80716f0 0x7fbcd80af580 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009840 tx=0x7fbcc001c700 comp rx=0 tx=0).stop 2024-09-10T21:14:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.207+0000 7fbcdf800640 1 -- 172.21.15.16:0/263345787 shutdown_connections 2024-09-10T21:14:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.207+0000 7fbcdf800640 1 --2- 172.21.15.16:0/263345787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd80716f0 0x7fbcd80af580 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.207+0000 7fbcdf800640 1 -- 172.21.15.16:0/263345787 >> 172.21.15.16:0/263345787 conn(0x7fbcd809b130 msgr2=0x7fbcd80d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.207+0000 7fbcdf800640 1 -- 172.21.15.16:0/263345787 shutdown_connections 2024-09-10T21:14:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.207+0000 7fbcdf800640 1 -- 172.21.15.16:0/263345787 wait complete. 2024-09-10T21:14:47.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdf800640 1 Processor -- start 2024-09-10T21:14:47.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdf800640 1 -- start start 2024-09-10T21:14:47.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdf800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd80716f0 0x7fbcd815a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:47.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdf800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbcd80b13c0 0x7fbcd8153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:47.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdf800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcd815f400 0x7fbcd80cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdf800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd816a020 con 0x7fbcd80716f0 2024-09-10T21:14:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdf800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd8169ea0 con 0x7fbcd815f400 2024-09-10T21:14:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdf800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd816a1a0 con 0x7fbcd80b13c0 2024-09-10T21:14:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd80716f0 0x7fbcd815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd80716f0 0x7fbcd815a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35014/0 (socket says 172.21.15.16:35014) 2024-09-10T21:14:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdce00640 1 -- 172.21.15.16:0/1553981347 learned_addr learned my addr 172.21.15.16:0/1553981347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbccfe00640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbcd80b13c0 0x7fbcd8153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdd800640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcd815f400 0x7fbcd80cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdce00640 1 -- 172.21.15.16:0/1553981347 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbcd80b13c0 msgr2=0x7fbcd8153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdce00640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbcd80b13c0 0x7fbcd8153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdce00640 1 -- 172.21.15.16:0/1553981347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcd815f400 msgr2=0x7fbcd80cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdce00640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcd815f400 0x7fbcd80cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdce00640 1 -- 172.21.15.16:0/1553981347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd81f7f10 con 0x7fbcd80716f0 2024-09-10T21:14:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdce00640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd80716f0 0x7fbcd815a810 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800c400 tx=0x7fbcc800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcd5600640 1 -- 172.21.15.16:0/1553981347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8017070 con 0x7fbcd80716f0 2024-09-10T21:14:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcd5600640 1 -- 172.21.15.16:0/1553981347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcc8003600 con 0x7fbcd80716f0 2024-09-10T21:14:47.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcd5600640 1 -- 172.21.15.16:0/1553981347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8003a10 con 0x7fbcd80716f0 2024-09-10T21:14:47.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdf800640 1 -- 172.21.15.16:0/1553981347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd81f81f0 con 0x7fbcd80716f0 2024-09-10T21:14:47.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdf800640 1 -- 172.21.15.16:0/1553981347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd81f87b0 con 0x7fbcd80716f0 2024-09-10T21:14:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.211+0000 7fbcdf800640 1 -- 172.21.15.16:0/1553981347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd81f8af0 con 0x7fbcd80716f0 2024-09-10T21:14:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.215+0000 7fbcd5600640 1 -- 172.21.15.16:0/1553981347 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbcc8003bb0 con 0x7fbcd80716f0 2024-09-10T21:14:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.215+0000 7fbcd5600640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbcb40725e0 0x7fbcb4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.215+0000 7fbcd5600640 1 -- 172.21.15.16:0/1553981347 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fbcc8092830 con 0x7fbcd80716f0 2024-09-10T21:14:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.215+0000 7fbccfe00640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbcb40725e0 0x7fbcb4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.215+0000 7fbccfe00640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbcb40725e0 0x7fbcb4074aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00af000 tx=0x7fbcc001af40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:47.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.219+0000 7fbcd5600640 1 -- 172.21.15.16:0/1553981347 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbcd81f8af0 con 0x7fbcd80716f0 2024-09-10T21:14:47.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.427+0000 7fbcdf800640 1 -- 172.21.15.16:0/1553981347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbcd809b540 con 0x7fbcd80716f0 2024-09-10T21:14:47.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.427+0000 7fbcd5600640 1 -- 172.21.15.16:0/1553981347 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fbcc805d820 con 0x7fbcd80716f0 2024-09-10T21:14:47.435 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:47.435 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:14:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.431+0000 7fbcdf800640 1 -- 172.21.15.16:0/1553981347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbcb40725e0 msgr2=0x7fbcb4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.431+0000 7fbcdf800640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbcb40725e0 0x7fbcb4074aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00af000 tx=0x7fbcc001af40 comp rx=0 tx=0).stop 2024-09-10T21:14:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.431+0000 7fbcdf800640 1 -- 172.21.15.16:0/1553981347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd80716f0 msgr2=0x7fbcd815a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.431+0000 7fbcdf800640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd80716f0 0x7fbcd815a810 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800c400 tx=0x7fbcc800c8c0 comp rx=0 tx=0).stop 2024-09-10T21:14:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.431+0000 7fbcdf800640 1 -- 172.21.15.16:0/1553981347 shutdown_connections 2024-09-10T21:14:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.431+0000 7fbcdf800640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbcb40725e0 0x7fbcb4074aa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.431+0000 7fbcdf800640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcd815f400 0x7fbcd80cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.431+0000 7fbcdf800640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbcd80b13c0 0x7fbcd8153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.431+0000 7fbcdf800640 1 --2- 172.21.15.16:0/1553981347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd80716f0 0x7fbcd815a810 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.431+0000 7fbcdf800640 1 -- 172.21.15.16:0/1553981347 >> 172.21.15.16:0/1553981347 conn(0x7fbcd809b130 msgr2=0x7fbcd806e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.431+0000 7fbcdf800640 1 -- 172.21.15.16:0/1553981347 shutdown_connections 2024-09-10T21:14:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:47.431+0000 7fbcdf800640 1 -- 172.21.15.16:0/1553981347 wait complete. 2024-09-10T21:14:47.457 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:14:47.457 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:14:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:52.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:14:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.607+0000 7f561ce00640 1 -- 172.21.15.16:0/1308298650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56180b0590 msgr2=0x7f5618071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.607+0000 7f561ce00640 1 --2- 172.21.15.16:0/1308298650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56180b0590 0x7f5618071520 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5610009c80 tx=0x7f561001cca0 comp rx=0 tx=0).stop 2024-09-10T21:14:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.607+0000 7f561ce00640 1 -- 172.21.15.16:0/1308298650 shutdown_connections 2024-09-10T21:14:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.607+0000 7f561ce00640 1 --2- 172.21.15.16:0/1308298650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56180b0590 0x7f5618071520 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.607+0000 7f561ce00640 1 --2- 172.21.15.16:0/1308298650 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f561815f400 0x7f5618070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.607+0000 7f561ce00640 1 -- 172.21.15.16:0/1308298650 >> 172.21.15.16:0/1308298650 conn(0x7f561809b130 msgr2=0x7f56180d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.607+0000 7f561ce00640 1 -- 172.21.15.16:0/1308298650 shutdown_connections 2024-09-10T21:14:52.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.607+0000 7f561ce00640 1 -- 172.21.15.16:0/1308298650 wait complete. 2024-09-10T21:14:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f561ce00640 1 Processor -- start 2024-09-10T21:14:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f561ce00640 1 -- start start 2024-09-10T21:14:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f561ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56180b0590 0x7f56181fa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f561ce00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56180b13c0 0x7f56180b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f561ce00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f561815f400 0x7f5618165160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f561ce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5618168f60 con 0x7f56180b0590 2024-09-10T21:14:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f561ce00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5618168de0 con 0x7f56180b13c0 2024-09-10T21:14:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f561ce00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f56181690e0 con 0x7f561815f400 2024-09-10T21:14:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f5615e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56180b0590 0x7f56181fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f5615e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56180b0590 0x7f56181fa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44976/0 (socket says 172.21.15.16:44976) 2024-09-10T21:14:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f5615e00640 1 -- 172.21.15.16:0/1345392441 learned_addr learned my addr 172.21.15.16:0/1345392441 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f5616800640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f561815f400 0x7f5618165160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f560fe00640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56180b13c0 0x7f56180b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f5615e00640 1 -- 172.21.15.16:0/1345392441 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f561815f400 msgr2=0x7f5618165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f5615e00640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f561815f400 0x7f5618165160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f5615e00640 1 -- 172.21.15.16:0/1345392441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56180b13c0 msgr2=0x7f56180b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f5615e00640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56180b13c0 0x7f56180b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f5615e00640 1 -- 172.21.15.16:0/1345392441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56181640c0 con 0x7f56180b0590 2024-09-10T21:14:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f560fe00640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56180b13c0 0x7f56180b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f5615e00640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56180b0590 0x7f56181fa830 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f560800c970 tx=0x7f560800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f560e000640 1 -- 172.21.15.16:0/1345392441 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608007bb0 con 0x7f56180b0590 2024-09-10T21:14:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f560e000640 1 -- 172.21.15.16:0/1345392441 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5608007e40 con 0x7f56180b0590 2024-09-10T21:14:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f560e000640 1 -- 172.21.15.16:0/1345392441 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608005270 con 0x7f56180b0590 2024-09-10T21:14:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f561ce00640 1 -- 172.21.15.16:0/1345392441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56181643a0 con 0x7f56180b0590 2024-09-10T21:14:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.611+0000 7f561ce00640 1 -- 172.21.15.16:0/1345392441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5618164960 con 0x7f56180b0590 2024-09-10T21:14:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.615+0000 7f561ce00640 1 -- 172.21.15.16:0/1345392441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56180cfc80 con 0x7f56180b0590 2024-09-10T21:14:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.615+0000 7f560e000640 1 -- 172.21.15.16:0/1345392441 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5608005410 con 0x7f56180b0590 2024-09-10T21:14:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.615+0000 7f560e000640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55f40725e0 0x7f55f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.615+0000 7f560e000640 1 -- 172.21.15.16:0/1345392441 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f56080936e0 con 0x7f56180b0590 2024-09-10T21:14:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.615+0000 7f560fe00640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55f40725e0 0x7f55f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.615+0000 7f560fe00640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55f40725e0 0x7f55f4074aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f56180b7da0 tx=0x7f5600009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.619+0000 7f560e000640 1 -- 172.21.15.16:0/1345392441 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5608014070 con 0x7f56180b0590 2024-09-10T21:14:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.831+0000 7f561ce00640 1 -- 172.21.15.16:0/1345392441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f56180b2450 con 0x7f56180b0590 2024-09-10T21:14:52.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.831+0000 7f560e000640 1 -- 172.21.15.16:0/1345392441 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f560805e760 con 0x7f56180b0590 2024-09-10T21:14:52.840 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:52.840 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:14:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.835+0000 7f561ce00640 1 -- 172.21.15.16:0/1345392441 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55f40725e0 msgr2=0x7f55f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.835+0000 7f561ce00640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55f40725e0 0x7f55f4074aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f56180b7da0 tx=0x7f5600009040 comp rx=0 tx=0).stop 2024-09-10T21:14:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.835+0000 7f561ce00640 1 -- 172.21.15.16:0/1345392441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56180b0590 msgr2=0x7f56181fa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.835+0000 7f561ce00640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56180b0590 0x7f56181fa830 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f560800c970 tx=0x7f560800ce30 comp rx=0 tx=0).stop 2024-09-10T21:14:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.835+0000 7f561ce00640 1 -- 172.21.15.16:0/1345392441 shutdown_connections 2024-09-10T21:14:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.835+0000 7f561ce00640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55f40725e0 0x7f55f4074aa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.835+0000 7f561ce00640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f561815f400 0x7f5618165160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.835+0000 7f561ce00640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56180b13c0 0x7f56180b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.835+0000 7f561ce00640 1 --2- 172.21.15.16:0/1345392441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56180b0590 0x7f56181fa830 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.835+0000 7f561ce00640 1 -- 172.21.15.16:0/1345392441 >> 172.21.15.16:0/1345392441 conn(0x7f561809b130 msgr2=0x7f561815e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.835+0000 7f561ce00640 1 -- 172.21.15.16:0/1345392441 shutdown_connections 2024-09-10T21:14:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:52.835+0000 7f561ce00640 1 -- 172.21.15.16:0/1345392441 wait complete. 2024-09-10T21:14:52.861 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:14:52.861 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:14:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:53.734 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:14:53.738 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:14:54.875 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:14:57.862 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:14:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 -- 172.21.15.16:0/98258617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7de80b0590 msgr2=0x7f7de8071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 --2- 172.21.15.16:0/98258617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7de80b0590 0x7f7de8071520 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009bc0 tx=0x7f7de001c680 comp rx=0 tx=0).stop 2024-09-10T21:14:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 -- 172.21.15.16:0/98258617 shutdown_connections 2024-09-10T21:14:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 --2- 172.21.15.16:0/98258617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7de80b0590 0x7f7de8071520 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 --2- 172.21.15.16:0/98258617 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7de815f400 0x7f7de8070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 -- 172.21.15.16:0/98258617 >> 172.21.15.16:0/98258617 conn(0x7f7de809b130 msgr2=0x7f7de80d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 -- 172.21.15.16:0/98258617 shutdown_connections 2024-09-10T21:14:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 -- 172.21.15.16:0/98258617 wait complete. 2024-09-10T21:14:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 Processor -- start 2024-09-10T21:14:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 -- start start 2024-09-10T21:14:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de80b0590 0x7f7de80cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7de80b13c0 0x7f7de8151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7de815f400 0x7f7de8159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7de8169eb0 con 0x7f7de80b0590 2024-09-10T21:14:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7de8169d30 con 0x7f7de80b13c0 2024-09-10T21:14:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7def200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7de816a030 con 0x7f7de815f400 2024-09-10T21:14:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7de7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de80b0590 0x7f7de80cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7de7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de80b0590 0x7f7de80cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44992/0 (socket says 172.21.15.16:44992) 2024-09-10T21:14:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7de7e00640 1 -- 172.21.15.16:0/2959766896 learned_addr learned my addr 172.21.15.16:0/2959766896 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:14:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7ddfe00640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7de80b13c0 0x7f7de8151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7ded200640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7de815f400 0x7f7de8159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7de7e00640 1 -- 172.21.15.16:0/2959766896 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7de815f400 msgr2=0x7f7de8159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7de7e00640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7de815f400 0x7f7de8159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7de7e00640 1 -- 172.21.15.16:0/2959766896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7de80b13c0 msgr2=0x7f7de8151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7de7e00640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7de80b13c0 0x7f7de8151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.015+0000 7f7de7e00640 1 -- 172.21.15.16:0/2959766896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de80ce0f0 con 0x7f7de80b0590 2024-09-10T21:14:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.019+0000 7f7de7e00640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de80b0590 0x7f7de80cd780 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800ea10 tx=0x7f7dd800eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:58.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.019+0000 7f7de4c00640 1 -- 172.21.15.16:0/2959766896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800cde0 con 0x7f7de80b0590 2024-09-10T21:14:58.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.019+0000 7f7de4c00640 1 -- 172.21.15.16:0/2959766896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dd8004020 con 0x7f7de80b0590 2024-09-10T21:14:58.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.019+0000 7f7def200640 1 -- 172.21.15.16:0/2959766896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de81580a0 con 0x7f7de80b0590 2024-09-10T21:14:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.019+0000 7f7de4c00640 1 -- 172.21.15.16:0/2959766896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8004470 con 0x7f7de80b0590 2024-09-10T21:14:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.019+0000 7f7def200640 1 -- 172.21.15.16:0/2959766896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de815b930 con 0x7f7de80b0590 2024-09-10T21:14:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.019+0000 7f7def200640 1 -- 172.21.15.16:0/2959766896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de80cfc80 con 0x7f7de80b0590 2024-09-10T21:14:58.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.019+0000 7f7de4c00640 1 -- 172.21.15.16:0/2959766896 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7dd8020030 con 0x7f7de80b0590 2024-09-10T21:14:58.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.019+0000 7f7de4c00640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7dc4072610 0x7f7dc4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:58.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.019+0000 7f7de4c00640 1 -- 172.21.15.16:0/2959766896 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f7dd80939f0 con 0x7f7de80b0590 2024-09-10T21:14:58.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.019+0000 7f7ddfe00640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7dc4072610 0x7f7dc4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:58.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.023+0000 7f7ddfe00640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7dc4072610 0x7f7dc4074ad0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7de81665d0 tx=0x7f7dd0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:58.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.023+0000 7f7de4c00640 1 -- 172.21.15.16:0/2959766896 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7dd805f0c0 con 0x7f7de80b0590 2024-09-10T21:14:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7def200640 1 -- 172.21.15.16:0/2959766896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7de815c030 con 0x7f7de80b0590 2024-09-10T21:14:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7de4c00640 1 -- 172.21.15.16:0/2959766896 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f7dd805e9c0 con 0x7f7de80b0590 2024-09-10T21:14:58.242 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:14:58.242 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:14:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7def200640 1 -- 172.21.15.16:0/2959766896 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7dc4072610 msgr2=0x7f7dc4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7def200640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7dc4072610 0x7f7dc4074ad0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7de81665d0 tx=0x7f7dd0009040 comp rx=0 tx=0).stop 2024-09-10T21:14:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7def200640 1 -- 172.21.15.16:0/2959766896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de80b0590 msgr2=0x7f7de80cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7def200640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de80b0590 0x7f7de80cd780 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800ea10 tx=0x7f7dd800eed0 comp rx=0 tx=0).stop 2024-09-10T21:14:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7def200640 1 -- 172.21.15.16:0/2959766896 shutdown_connections 2024-09-10T21:14:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7def200640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7dc4072610 0x7f7dc4074ad0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7def200640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7de815f400 0x7f7de8159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7def200640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7de80b13c0 0x7f7de8151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7def200640 1 --2- 172.21.15.16:0/2959766896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de80b0590 0x7f7de80cd780 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7def200640 1 -- 172.21.15.16:0/2959766896 >> 172.21.15.16:0/2959766896 conn(0x7f7de809b130 msgr2=0x7f7de806e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7def200640 1 -- 172.21.15.16:0/2959766896 shutdown_connections 2024-09-10T21:14:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:14:58.235+0000 7f7def200640 1 -- 172.21.15.16:0/2959766896 wait complete. 2024-09-10T21:14:58.263 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:14:58.263 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:14:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:03.264 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.415+0000 7f0961c00640 1 -- 172.21.15.16:0/1820524458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c0b13c0 msgr2=0x7f095c071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.415+0000 7f0961c00640 1 --2- 172.21.15.16:0/1820524458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c0b13c0 0x7f095c071520 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f095c074cc0 tx=0x7f094401c7a0 comp rx=0 tx=0).stop 2024-09-10T21:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.415+0000 7f0961c00640 1 -- 172.21.15.16:0/1820524458 shutdown_connections 2024-09-10T21:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.415+0000 7f0961c00640 1 --2- 172.21.15.16:0/1820524458 >> v2:172.21.15.52:3300/0 conn(0x7f095c0cfc80 0x7f095c0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.415+0000 7f0961c00640 1 --2- 172.21.15.16:0/1820524458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c0b13c0 0x7f095c071520 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:03.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.415+0000 7f0961c00640 1 --2- 172.21.15.16:0/1820524458 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f095c15f400 0x7f095c070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.415+0000 7f0961c00640 1 -- 172.21.15.16:0/1820524458 >> 172.21.15.16:0/1820524458 conn(0x7f095c09b130 msgr2=0x7f095c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.415+0000 7f0961c00640 1 -- 172.21.15.16:0/1820524458 shutdown_connections 2024-09-10T21:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.415+0000 7f0961c00640 1 -- 172.21.15.16:0/1820524458 wait complete. 2024-09-10T21:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.415+0000 7f0961c00640 1 Processor -- start 2024-09-10T21:15:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.415+0000 7f0961c00640 1 -- start start 2024-09-10T21:15:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0961c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f095c0b13c0 0x7f095c159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0961c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f095c0cfc80 0x7f095c165130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0961c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c15f400 0x7f095c0b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0961c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f095c0afd90 con 0x7f095c15f400 2024-09-10T21:15:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0961c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f095c0d1260 con 0x7f095c0cfc80 2024-09-10T21:15:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0961c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f095c0aff10 con 0x7f095c0b13c0 2024-09-10T21:15:03.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f095b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c15f400 0x7f095c0b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f095aa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f095c0b13c0 0x7f095c159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0953e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f095c0cfc80 0x7f095c165130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f095b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c15f400 0x7f095c0b6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44644/0 (socket says 172.21.15.16:44644) 2024-09-10T21:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f095b400640 1 -- 172.21.15.16:0/2876274526 learned_addr learned my addr 172.21.15.16:0/2876274526 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f095b400640 1 -- 172.21.15.16:0/2876274526 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f095c0b13c0 msgr2=0x7f095c159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f095b400640 1 --2- 172.21.15.16:0/2876274526 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f095c0b13c0 0x7f095c159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f095b400640 1 -- 172.21.15.16:0/2876274526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f095c0cfc80 msgr2=0x7f095c165130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f095b400640 1 --2- 172.21.15.16:0/2876274526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f095c0cfc80 0x7f095c165130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f095b400640 1 -- 172.21.15.16:0/2876274526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c15aa10 con 0x7f095c15f400 2024-09-10T21:15:03.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f095b400640 1 --2- 172.21.15.16:0/2876274526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c15f400 0x7f095c0b6c80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f095400ccb0 tx=0x7f095400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:03.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0952a00640 1 -- 172.21.15.16:0/2876274526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095400f450 con 0x7f095c15f400 2024-09-10T21:15:03.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0952a00640 1 -- 172.21.15.16:0/2876274526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f095400f700 con 0x7f095c15f400 2024-09-10T21:15:03.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0952a00640 1 -- 172.21.15.16:0/2876274526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095400fb50 con 0x7f095c15f400 2024-09-10T21:15:03.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0961c00640 1 -- 172.21.15.16:0/2876274526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c1660f0 con 0x7f095c15f400 2024-09-10T21:15:03.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0961c00640 1 -- 172.21.15.16:0/2876274526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c164930 con 0x7f095c15f400 2024-09-10T21:15:03.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0961c00640 1 -- 172.21.15.16:0/2876274526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f095c165ee0 con 0x7f095c15f400 2024-09-10T21:15:03.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.419+0000 7f0952a00640 1 -- 172.21.15.16:0/2876274526 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f09540223f0 con 0x7f095c15f400 2024-09-10T21:15:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.423+0000 7f0952a00640 1 --2- 172.21.15.16:0/2876274526 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09380725e0 0x7f0938074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.423+0000 7f0952a00640 1 -- 172.21.15.16:0/2876274526 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f0954093ce0 con 0x7f095c15f400 2024-09-10T21:15:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.423+0000 7f095aa00640 1 --2- 172.21.15.16:0/2876274526 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09380725e0 0x7f0938074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.423+0000 7f095aa00640 1 --2- 172.21.15.16:0/2876274526 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09380725e0 0x7f0938074aa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f094c005fd0 tx=0x7f094c00a250 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.427+0000 7f0952a00640 1 -- 172.21.15.16:0/2876274526 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0954014030 con 0x7f095c15f400 2024-09-10T21:15:03.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.635+0000 7f0961c00640 1 -- 172.21.15.16:0/2876274526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f095c159fc0 con 0x7f095c15f400 2024-09-10T21:15:03.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.635+0000 7f0952a00640 1 -- 172.21.15.16:0/2876274526 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f095405ecd0 con 0x7f095c15f400 2024-09-10T21:15:03.643 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:15:03.643 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:15:03.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.639+0000 7f0961c00640 1 -- 172.21.15.16:0/2876274526 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09380725e0 msgr2=0x7f0938074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.639+0000 7f0961c00640 1 --2- 172.21.15.16:0/2876274526 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09380725e0 0x7f0938074aa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f094c005fd0 tx=0x7f094c00a250 comp rx=0 tx=0).stop 2024-09-10T21:15:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.639+0000 7f0961c00640 1 -- 172.21.15.16:0/2876274526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c15f400 msgr2=0x7f095c0b6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.639+0000 7f0961c00640 1 --2- 172.21.15.16:0/2876274526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c15f400 0x7f095c0b6c80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f095400ccb0 tx=0x7f095400a620 comp rx=0 tx=0).stop 2024-09-10T21:15:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.639+0000 7f0961c00640 1 -- 172.21.15.16:0/2876274526 shutdown_connections 2024-09-10T21:15:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.639+0000 7f0961c00640 1 --2- 172.21.15.16:0/2876274526 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09380725e0 0x7f0938074aa0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.639+0000 7f0961c00640 1 --2- 172.21.15.16:0/2876274526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c15f400 0x7f095c0b6c80 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.639+0000 7f0961c00640 1 --2- 172.21.15.16:0/2876274526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f095c0cfc80 0x7f095c165130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.639+0000 7f0961c00640 1 --2- 172.21.15.16:0/2876274526 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f095c0b13c0 0x7f095c159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.639+0000 7f0961c00640 1 -- 172.21.15.16:0/2876274526 >> 172.21.15.16:0/2876274526 conn(0x7f095c09b130 msgr2=0x7f095c0b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.639+0000 7f0961c00640 1 -- 172.21.15.16:0/2876274526 shutdown_connections 2024-09-10T21:15:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:03.639+0000 7f0961c00640 1 -- 172.21.15.16:0/2876274526 wait complete. 2024-09-10T21:15:03.665 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:15:03.665 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:15:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:06.886 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:08.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:15:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.818+0000 7fcf46400640 1 -- 172.21.15.16:0/3672696440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf400b13c0 msgr2=0x7fcf40071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.818+0000 7fcf46400640 1 --2- 172.21.15.16:0/3672696440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf400b13c0 0x7fcf40071520 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fcf40073290 tx=0x7fcf2801c7a0 comp rx=0 tx=0).stop 2024-09-10T21:15:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.818+0000 7fcf46400640 1 -- 172.21.15.16:0/3672696440 shutdown_connections 2024-09-10T21:15:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.818+0000 7fcf46400640 1 --2- 172.21.15.16:0/3672696440 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcf400cfc80 0x7fcf400d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.818+0000 7fcf46400640 1 --2- 172.21.15.16:0/3672696440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf400b13c0 0x7fcf40071520 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.818+0000 7fcf46400640 1 --2- 172.21.15.16:0/3672696440 >> v2:172.21.15.52:3300/0 conn(0x7fcf4015f400 0x7fcf40070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.818+0000 7fcf46400640 1 -- 172.21.15.16:0/3672696440 >> 172.21.15.16:0/3672696440 conn(0x7fcf4009b130 msgr2=0x7fcf400d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.818+0000 7fcf46400640 1 -- 172.21.15.16:0/3672696440 shutdown_connections 2024-09-10T21:15:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.818+0000 7fcf46400640 1 -- 172.21.15.16:0/3672696440 wait complete. 2024-09-10T21:15:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf46400640 1 Processor -- start 2024-09-10T21:15:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf46400640 1 -- start start 2024-09-10T21:15:08.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf46400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf400b13c0 0x7fcf40155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf46400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf400cfc80 0x7fcf400b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf46400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcf4015f400 0x7fcf400c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf46400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf400afd90 con 0x7fcf400b13c0 2024-09-10T21:15:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf46400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcf400d1260 con 0x7fcf400cfc80 2024-09-10T21:15:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf46400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fcf400aff10 con 0x7fcf4015f400 2024-09-10T21:15:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf400b13c0 0x7fcf40155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf400b13c0 0x7fcf40155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44664/0 (socket says 172.21.15.16:44664) 2024-09-10T21:15:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf3f400640 1 -- 172.21.15.16:0/3348126674 learned_addr learned my addr 172.21.15.16:0/3348126674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:15:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf3fe00640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcf4015f400 0x7fcf400c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf37e00640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf400cfc80 0x7fcf400b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf3f400640 1 -- 172.21.15.16:0/3348126674 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcf4015f400 msgr2=0x7fcf400c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf3f400640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcf4015f400 0x7fcf400c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf3f400640 1 -- 172.21.15.16:0/3348126674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf400cfc80 msgr2=0x7fcf400b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf3f400640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf400cfc80 0x7fcf400b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf3f400640 1 -- 172.21.15.16:0/3348126674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf400b5e10 con 0x7fcf400b13c0 2024-09-10T21:15:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf37e00640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf400cfc80 0x7fcf400b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:15:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf3f400640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf400b13c0 0x7fcf40155110 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000b440 tx=0x7fcf3000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf37400640 1 -- 172.21.15.16:0/3348126674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30015030 con 0x7fcf400b13c0 2024-09-10T21:15:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf37400640 1 -- 172.21.15.16:0/3348126674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf30005980 con 0x7fcf400b13c0 2024-09-10T21:15:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf37400640 1 -- 172.21.15.16:0/3348126674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30005dd0 con 0x7fcf400b13c0 2024-09-10T21:15:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf46400640 1 -- 172.21.15.16:0/3348126674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf400b60f0 con 0x7fcf400b13c0 2024-09-10T21:15:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.822+0000 7fcf46400640 1 -- 172.21.15.16:0/3348126674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf400b66b0 con 0x7fcf400b13c0 2024-09-10T21:15:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.826+0000 7fcf46400640 1 -- 172.21.15.16:0/3348126674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf400d0110 con 0x7fcf400b13c0 2024-09-10T21:15:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.826+0000 7fcf37400640 1 -- 172.21.15.16:0/3348126674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcf30022060 con 0x7fcf400b13c0 2024-09-10T21:15:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.826+0000 7fcf37400640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcf1c0725e0 0x7fcf1c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.826+0000 7fcf37400640 1 -- 172.21.15.16:0/3348126674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fcf30092fe0 con 0x7fcf400b13c0 2024-09-10T21:15:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.826+0000 7fcf37e00640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcf1c0725e0 0x7fcf1c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.826+0000 7fcf37e00640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcf1c0725e0 0x7fcf1c074aa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcf280af000 tx=0x7fcf2801afe0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:08.830+0000 7fcf37400640 1 -- 172.21.15.16:0/3348126674 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcf3000d040 con 0x7fcf400b13c0 2024-09-10T21:15:08.888 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:09.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.038+0000 7fcf46400640 1 -- 172.21.15.16:0/3348126674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcf400b33c0 con 0x7fcf400b13c0 2024-09-10T21:15:09.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.042+0000 7fcf37400640 1 -- 172.21.15.16:0/3348126674 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fcf3005e640 con 0x7fcf400b13c0 2024-09-10T21:15:09.045 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:15:09.045 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:15:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.042+0000 7fcf46400640 1 -- 172.21.15.16:0/3348126674 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcf1c0725e0 msgr2=0x7fcf1c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.042+0000 7fcf46400640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcf1c0725e0 0x7fcf1c074aa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcf280af000 tx=0x7fcf2801afe0 comp rx=0 tx=0).stop 2024-09-10T21:15:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.042+0000 7fcf46400640 1 -- 172.21.15.16:0/3348126674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf400b13c0 msgr2=0x7fcf40155110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.042+0000 7fcf46400640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf400b13c0 0x7fcf40155110 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000b440 tx=0x7fcf3000b900 comp rx=0 tx=0).stop 2024-09-10T21:15:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.042+0000 7fcf46400640 1 -- 172.21.15.16:0/3348126674 shutdown_connections 2024-09-10T21:15:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.042+0000 7fcf46400640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcf1c0725e0 0x7fcf1c074aa0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.042+0000 7fcf46400640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcf4015f400 0x7fcf400c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.042+0000 7fcf46400640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf400cfc80 0x7fcf400b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.042+0000 7fcf46400640 1 --2- 172.21.15.16:0/3348126674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf400b13c0 0x7fcf40155110 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.042+0000 7fcf46400640 1 -- 172.21.15.16:0/3348126674 >> 172.21.15.16:0/3348126674 conn(0x7fcf4009b130 msgr2=0x7fcf400cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.042+0000 7fcf46400640 1 -- 172.21.15.16:0/3348126674 shutdown_connections 2024-09-10T21:15:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:09.042+0000 7fcf46400640 1 -- 172.21.15.16:0/3348126674 wait complete. 2024-09-10T21:15:09.067 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:15:09.067 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:15:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:12.893 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:14.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:15:14.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.222+0000 7febf3e00640 1 -- 172.21.15.16:0/2003293737 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7febf4074860 msgr2=0x7febf40b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:14.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.222+0000 7febf3e00640 1 --2- 172.21.15.16:0/2003293737 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7febf4074860 0x7febf40b16b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7febec009bf0 tx=0x7febec01f2f0 comp rx=0 tx=0).stop 2024-09-10T21:15:14.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.222+0000 7febf3e00640 1 -- 172.21.15.16:0/2003293737 shutdown_connections 2024-09-10T21:15:14.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.222+0000 7febf3e00640 1 --2- 172.21.15.16:0/2003293737 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7febf4074860 0x7febf40b16b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:14.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.222+0000 7febf3e00640 1 -- 172.21.15.16:0/2003293737 >> 172.21.15.16:0/2003293737 conn(0x7febf40b1d40 msgr2=0x7febf40b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:14.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.222+0000 7febf3e00640 1 -- 172.21.15.16:0/2003293737 shutdown_connections 2024-09-10T21:15:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.222+0000 7febf3e00640 1 -- 172.21.15.16:0/2003293737 wait complete. 2024-09-10T21:15:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.222+0000 7febf3e00640 1 Processor -- start 2024-09-10T21:15:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.222+0000 7febf3e00640 1 -- start start 2024-09-10T21:15:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febf4074860 0x7febf40ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7febf40b2860 0x7febf4070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf40cc290 0x7febf4155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7febf4165ef0 con 0x7febf40cc290 2024-09-10T21:15:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7febf40b45c0 con 0x7febf4074860 2024-09-10T21:15:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7febf4166070 con 0x7febf40b2860 2024-09-10T21:15:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf40cc290 0x7febf4155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf40cc290 0x7febf4155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50300/0 (socket says 172.21.15.16:50300) 2024-09-10T21:15:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3400640 1 -- 172.21.15.16:0/3843968387 learned_addr learned my addr 172.21.15.16:0/3843968387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:15:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febebe00640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7febf40b2860 0x7febf4070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf2a00640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febf4074860 0x7febf40ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3400640 1 -- 172.21.15.16:0/3843968387 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7febf40b2860 msgr2=0x7febf4070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3400640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7febf40b2860 0x7febf4070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3400640 1 -- 172.21.15.16:0/3843968387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febf4074860 msgr2=0x7febf40ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3400640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febf4074860 0x7febf40ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3400640 1 -- 172.21.15.16:0/3843968387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf41567c0 con 0x7febf40cc290 2024-09-10T21:15:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf2a00640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febf4074860 0x7febf40ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:15:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3400640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf40cc290 0x7febf4155460 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7febec0b2040 tx=0x7febec003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febeaa00640 1 -- 172.21.15.16:0/3843968387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec0b4040 con 0x7febf40cc290 2024-09-10T21:15:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febeaa00640 1 -- 172.21.15.16:0/3843968387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febec0bf070 con 0x7febf40cc290 2024-09-10T21:15:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febeaa00640 1 -- 172.21.15.16:0/3843968387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec0afc50 con 0x7febf40cc290 2024-09-10T21:15:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3e00640 1 -- 172.21.15.16:0/3843968387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf4156aa0 con 0x7febf40cc290 2024-09-10T21:15:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3e00640 1 -- 172.21.15.16:0/3843968387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf4157580 con 0x7febf40cc290 2024-09-10T21:15:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febf3e00640 1 -- 172.21.15.16:0/3843968387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf40b16b0 con 0x7febf40cc290 2024-09-10T21:15:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.226+0000 7febeaa00640 1 -- 172.21.15.16:0/3843968387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7febec004040 con 0x7febf40cc290 2024-09-10T21:15:14.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.230+0000 7febeaa00640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7febd00725e0 0x7febd0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:14.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.230+0000 7febeaa00640 1 -- 172.21.15.16:0/3843968387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7febec1312a0 con 0x7febf40cc290 2024-09-10T21:15:14.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.230+0000 7febf2a00640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7febd00725e0 0x7febd0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:14.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.230+0000 7febf2a00640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7febd00725e0 0x7febd0074aa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7febe40010f0 tx=0x7febe4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:14.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.234+0000 7febeaa00640 1 -- 172.21.15.16:0/3843968387 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7febec018e50 con 0x7febf40cc290 2024-09-10T21:15:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.442+0000 7febf3e00640 1 -- 172.21.15.16:0/3843968387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7febf415d320 con 0x7febf40cc290 2024-09-10T21:15:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.442+0000 7febeaa00640 1 -- 172.21.15.16:0/3843968387 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7febec0fc8e0 con 0x7febf40cc290 2024-09-10T21:15:14.446 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:15:14.446 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:15:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.442+0000 7febf3e00640 1 -- 172.21.15.16:0/3843968387 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7febd00725e0 msgr2=0x7febd0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.442+0000 7febf3e00640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7febd00725e0 0x7febd0074aa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7febe40010f0 tx=0x7febe4009040 comp rx=0 tx=0).stop 2024-09-10T21:15:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.442+0000 7febf3e00640 1 -- 172.21.15.16:0/3843968387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf40cc290 msgr2=0x7febf4155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.442+0000 7febf3e00640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf40cc290 0x7febf4155460 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7febec0b2040 tx=0x7febec003f60 comp rx=0 tx=0).stop 2024-09-10T21:15:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.446+0000 7febf3e00640 1 -- 172.21.15.16:0/3843968387 shutdown_connections 2024-09-10T21:15:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.446+0000 7febf3e00640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7febd00725e0 0x7febd0074aa0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.446+0000 7febf3e00640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf40cc290 0x7febf4155460 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.446+0000 7febf3e00640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7febf40b2860 0x7febf4070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.446+0000 7febf3e00640 1 --2- 172.21.15.16:0/3843968387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febf4074860 0x7febf40ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.446+0000 7febf3e00640 1 -- 172.21.15.16:0/3843968387 >> 172.21.15.16:0/3843968387 conn(0x7febf40b1d40 msgr2=0x7febf4068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.446+0000 7febf3e00640 1 -- 172.21.15.16:0/3843968387 shutdown_connections 2024-09-10T21:15:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:14.446+0000 7febf3e00640 1 -- 172.21.15.16:0/3843968387 wait complete. 2024-09-10T21:15:14.467 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:15:14.467 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:15:14.895 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:19.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:15:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.622+0000 7f7595800640 1 -- 172.21.15.16:0/1169788466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759015f400 msgr2=0x7f7590070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.622+0000 7f7595800640 1 --2- 172.21.15.16:0/1169788466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759015f400 0x7f7590070fe0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f75800098f0 tx=0x7f758001c7c0 comp rx=0 tx=0).stop 2024-09-10T21:15:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.622+0000 7f7595800640 1 -- 172.21.15.16:0/1169788466 shutdown_connections 2024-09-10T21:15:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.622+0000 7f7595800640 1 --2- 172.21.15.16:0/1169788466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759015f400 0x7f7590070fe0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.622+0000 7f7595800640 1 -- 172.21.15.16:0/1169788466 >> 172.21.15.16:0/1169788466 conn(0x7f759009b130 msgr2=0x7f75900d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.622+0000 7f7595800640 1 -- 172.21.15.16:0/1169788466 shutdown_connections 2024-09-10T21:15:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.622+0000 7f7595800640 1 -- 172.21.15.16:0/1169788466 wait complete. 2024-09-10T21:15:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7595800640 1 Processor -- start 2024-09-10T21:15:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7595800640 1 -- start start 2024-09-10T21:15:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7595800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75900b0590 0x7f759014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7595800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75900b13c0 0x7f75901f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7595800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759015f400 0x7f7590150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7595800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f759016a800 con 0x7f759015f400 2024-09-10T21:15:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7595800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f75900d1260 con 0x7f75900b0590 2024-09-10T21:15:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7595800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f759016a980 con 0x7f75900b13c0 2024-09-10T21:15:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f758f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759015f400 0x7f7590150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f758f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759015f400 0x7f7590150af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50330/0 (socket says 172.21.15.16:50330) 2024-09-10T21:15:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f758f400640 1 -- 172.21.15.16:0/1041846765 learned_addr learned my addr 172.21.15.16:0/1041846765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:15:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f758ea00640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75900b0590 0x7f759014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7587e00640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75900b13c0 0x7f75901f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f758f400640 1 -- 172.21.15.16:0/1041846765 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75900b13c0 msgr2=0x7f75901f98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f758f400640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75900b13c0 0x7f75901f98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f758f400640 1 -- 172.21.15.16:0/1041846765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75900b0590 msgr2=0x7f759014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f758f400640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75900b0590 0x7f759014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f758f400640 1 -- 172.21.15.16:0/1041846765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75900b5320 con 0x7f759015f400 2024-09-10T21:15:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f758f400640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759015f400 0x7f7590150af0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f758800c7a0 tx=0x7f758800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7586a00640 1 -- 172.21.15.16:0/1041846765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588017070 con 0x7f759015f400 2024-09-10T21:15:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7586a00640 1 -- 172.21.15.16:0/1041846765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7588005070 con 0x7f759015f400 2024-09-10T21:15:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7586a00640 1 -- 172.21.15.16:0/1041846765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75880054c0 con 0x7f759015f400 2024-09-10T21:15:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7595800640 1 -- 172.21.15.16:0/1041846765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75900b5600 con 0x7f759015f400 2024-09-10T21:15:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7595800640 1 -- 172.21.15.16:0/1041846765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75900ceda0 con 0x7f759015f400 2024-09-10T21:15:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.626+0000 7f7595800640 1 -- 172.21.15.16:0/1041846765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7590070fe0 con 0x7f759015f400 2024-09-10T21:15:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.630+0000 7f7586a00640 1 -- 172.21.15.16:0/1041846765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7588005660 con 0x7f759015f400 2024-09-10T21:15:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.630+0000 7f7586a00640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f756c0725e0 0x7f756c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.630+0000 7f7586a00640 1 -- 172.21.15.16:0/1041846765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f7588092be0 con 0x7f759015f400 2024-09-10T21:15:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.630+0000 7f758ea00640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f756c0725e0 0x7f756c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.630+0000 7f758ea00640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f756c0725e0 0x7f756c074aa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f75800ae000 tx=0x7f75800b3040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:19.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.634+0000 7f7586a00640 1 -- 172.21.15.16:0/1041846765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f758805dbd0 con 0x7f759015f400 2024-09-10T21:15:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.842+0000 7f7595800640 1 -- 172.21.15.16:0/1041846765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f75900b5ab0 con 0x7f759015f400 2024-09-10T21:15:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.842+0000 7f7586a00640 1 -- 172.21.15.16:0/1041846765 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f7588062600 con 0x7f759015f400 2024-09-10T21:15:19.848 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:15:19.848 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:15:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.846+0000 7f7595800640 1 -- 172.21.15.16:0/1041846765 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f756c0725e0 msgr2=0x7f756c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.846+0000 7f7595800640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f756c0725e0 0x7f756c074aa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f75800ae000 tx=0x7f75800b3040 comp rx=0 tx=0).stop 2024-09-10T21:15:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.846+0000 7f7595800640 1 -- 172.21.15.16:0/1041846765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759015f400 msgr2=0x7f7590150af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.846+0000 7f7595800640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759015f400 0x7f7590150af0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f758800c7a0 tx=0x7f758800cc60 comp rx=0 tx=0).stop 2024-09-10T21:15:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.846+0000 7f7595800640 1 -- 172.21.15.16:0/1041846765 shutdown_connections 2024-09-10T21:15:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.846+0000 7f7595800640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f756c0725e0 0x7f756c074aa0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.846+0000 7f7595800640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759015f400 0x7f7590150af0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.846+0000 7f7595800640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75900b13c0 0x7f75901f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.846+0000 7f7595800640 1 --2- 172.21.15.16:0/1041846765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75900b0590 0x7f759014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.846+0000 7f7595800640 1 -- 172.21.15.16:0/1041846765 >> 172.21.15.16:0/1041846765 conn(0x7f759009b130 msgr2=0x7f75900af510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.846+0000 7f7595800640 1 -- 172.21.15.16:0/1041846765 shutdown_connections 2024-09-10T21:15:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:19.846+0000 7f7595800640 1 -- 172.21.15.16:0/1041846765 wait complete. 2024-09-10T21:15:19.870 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:15:19.871 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:15:20.900 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:23.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:15:23.813 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:15:24.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:15:24.904 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 -- 172.21.15.16:0/4095994039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc40af780 msgr2=0x7facc415f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 --2- 172.21.15.16:0/4095994039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc40af780 0x7facc415f1e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7facc40b0460 tx=0x7facb001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:15:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 -- 172.21.15.16:0/4095994039 shutdown_connections 2024-09-10T21:15:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 --2- 172.21.15.16:0/4095994039 >> v2:172.21.15.52:3300/0 conn(0x7facc4074280 0x7facc415f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 --2- 172.21.15.16:0/4095994039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc40af780 0x7facc415f1e0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 --2- 172.21.15.16:0/4095994039 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facc40737e0 0x7facc415eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 -- 172.21.15.16:0/4095994039 >> 172.21.15.16:0/4095994039 conn(0x7facc40d0a60 msgr2=0x7facc40d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 -- 172.21.15.16:0/4095994039 shutdown_connections 2024-09-10T21:15:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 -- 172.21.15.16:0/4095994039 wait complete. 2024-09-10T21:15:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 Processor -- start 2024-09-10T21:15:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 -- start start 2024-09-10T21:15:25.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facc40737e0 0x7facc40aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:25.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4074280 0x7facc415de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:25.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facc40af780 0x7facc40c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:25.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facc40cea10 con 0x7facc4074280 2024-09-10T21:15:25.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7facc4070fe0 con 0x7facc40af780 2024-09-10T21:15:25.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.026+0000 7faccac00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7facc40ceb90 con 0x7facc40737e0 2024-09-10T21:15:25.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4074280 0x7facc415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:25.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4074280 0x7facc415de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55462/0 (socket says 172.21.15.16:55462) 2024-09-10T21:15:25.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc8e00640 1 -- 172.21.15.16:0/3275147089 learned_addr learned my addr 172.21.15.16:0/3275147089 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:15:25.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facca200640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facc40af780 0x7facc40c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:25.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc9800640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facc40737e0 0x7facc40aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:25.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc8e00640 1 -- 172.21.15.16:0/3275147089 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facc40737e0 msgr2=0x7facc40aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:25.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc8e00640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facc40737e0 0x7facc40aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:25.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc8e00640 1 -- 172.21.15.16:0/3275147089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facc40af780 msgr2=0x7facc40c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:25.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc8e00640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facc40af780 0x7facc40c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:25.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc8e00640 1 -- 172.21.15.16:0/3275147089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc415d3c0 con 0x7facc4074280 2024-09-10T21:15:25.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc9800640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facc40737e0 0x7facc40aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:15:25.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc8e00640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4074280 0x7facc415de60 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7facb001af70 tx=0x7facb0005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:25.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc1600640 1 -- 172.21.15.16:0/3275147089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb00b2040 con 0x7facc4074280 2024-09-10T21:15:25.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc1600640 1 -- 172.21.15.16:0/3275147089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facb0004310 con 0x7facc4074280 2024-09-10T21:15:25.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7faccac00640 1 -- 172.21.15.16:0/3275147089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc40c51a0 con 0x7facc4074280 2024-09-10T21:15:25.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc1600640 1 -- 172.21.15.16:0/3275147089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb00052d0 con 0x7facc4074280 2024-09-10T21:15:25.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7faccac00640 1 -- 172.21.15.16:0/3275147089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc415d660 con 0x7facc4074280 2024-09-10T21:15:25.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7faccac00640 1 -- 172.21.15.16:0/3275147089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc40cf120 con 0x7facc4074280 2024-09-10T21:15:25.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.030+0000 7facc1600640 1 -- 172.21.15.16:0/3275147089 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7facb00046d0 con 0x7facc4074280 2024-09-10T21:15:25.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.034+0000 7facc1600640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faca4072610 0x7faca4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:25.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.034+0000 7facc1600640 1 -- 172.21.15.16:0/3275147089 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7facb012cae0 con 0x7facc4074280 2024-09-10T21:15:25.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.034+0000 7facc9800640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faca4072610 0x7faca4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:25.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.034+0000 7facc9800640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faca4072610 0x7faca4074ad0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7facb8009bb0 tx=0x7facb80091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.038+0000 7facc1600640 1 -- 172.21.15.16:0/3275147089 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7facb00bc060 con 0x7facc4074280 2024-09-10T21:15:25.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7faccac00640 1 -- 172.21.15.16:0/3275147089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7facc40cae70 con 0x7facc4074280 2024-09-10T21:15:25.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7facc1600640 1 -- 172.21.15.16:0/3275147089 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7facb00164b0 con 0x7facc4074280 2024-09-10T21:15:25.253 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:15:25.254 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:15:25.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7faccac00640 1 -- 172.21.15.16:0/3275147089 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faca4072610 msgr2=0x7faca4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:25.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7faccac00640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faca4072610 0x7faca4074ad0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7facb8009bb0 tx=0x7facb80091e0 comp rx=0 tx=0).stop 2024-09-10T21:15:25.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7faccac00640 1 -- 172.21.15.16:0/3275147089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4074280 msgr2=0x7facc415de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:25.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7faccac00640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4074280 0x7facc415de60 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7facb001af70 tx=0x7facb0005d60 comp rx=0 tx=0).stop 2024-09-10T21:15:25.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7faccac00640 1 -- 172.21.15.16:0/3275147089 shutdown_connections 2024-09-10T21:15:25.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7faccac00640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faca4072610 0x7faca4074ad0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:25.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7faccac00640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facc40af780 0x7facc40c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:25.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7faccac00640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4074280 0x7facc415de60 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:25.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7faccac00640 1 --2- 172.21.15.16:0/3275147089 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7facc40737e0 0x7facc40aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:25.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7faccac00640 1 -- 172.21.15.16:0/3275147089 >> 172.21.15.16:0/3275147089 conn(0x7facc40d0a60 msgr2=0x7facc415a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:25.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7faccac00640 1 -- 172.21.15.16:0/3275147089 shutdown_connections 2024-09-10T21:15:25.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:25.250+0000 7faccac00640 1 -- 172.21.15.16:0/3275147089 wait complete. 2024-09-10T21:15:25.275 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:15:25.275 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:15:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:30.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:15:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.430+0000 7fcf8be00640 1 -- 172.21.15.16:0/383447945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf841599d0 msgr2=0x7fcf8415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.430+0000 7fcf8be00640 1 --2- 172.21.15.16:0/383447945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf841599d0 0x7fcf8415f830 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fcf8406e0e0 tx=0x7fcf7801c770 comp rx=0 tx=0).stop 2024-09-10T21:15:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.430+0000 7fcf8be00640 1 -- 172.21.15.16:0/383447945 shutdown_connections 2024-09-10T21:15:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.430+0000 7fcf8be00640 1 --2- 172.21.15.16:0/383447945 >> v2:172.21.15.52:3300/0 conn(0x7fcf840726b0 0x7fcf84072b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.430+0000 7fcf8be00640 1 --2- 172.21.15.16:0/383447945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf841599d0 0x7fcf8415f830 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.430+0000 7fcf8be00640 1 -- 172.21.15.16:0/383447945 >> 172.21.15.16:0/383447945 conn(0x7fcf840cfb20 msgr2=0x7fcf840cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.430+0000 7fcf8be00640 1 -- 172.21.15.16:0/383447945 shutdown_connections 2024-09-10T21:15:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.430+0000 7fcf8be00640 1 -- 172.21.15.16:0/383447945 wait complete. 2024-09-10T21:15:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf8be00640 1 Processor -- start 2024-09-10T21:15:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf8be00640 1 -- start start 2024-09-10T21:15:30.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf8be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf8406e640 0x7fcf840c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:30.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf8be00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcf840726b0 0x7fcf840b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:30.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf8be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf841599d0 0x7fcf840ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:30.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf8be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf84168eb0 con 0x7fcf841599d0 2024-09-10T21:15:30.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf8be00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcf84168d30 con 0x7fcf8406e640 2024-09-10T21:15:30.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf8be00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fcf84169030 con 0x7fcf840726b0 2024-09-10T21:15:30.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf89e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf841599d0 0x7fcf840ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:30.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf89e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf841599d0 0x7fcf840ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55476/0 (socket says 172.21.15.16:55476) 2024-09-10T21:15:30.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf89e00640 1 -- 172.21.15.16:0/3110144453 learned_addr learned my addr 172.21.15.16:0/3110144453 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:15:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf88a00640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcf840726b0 0x7fcf840b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf89400640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf8406e640 0x7fcf840c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf89e00640 1 -- 172.21.15.16:0/3110144453 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcf840726b0 msgr2=0x7fcf840b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf89e00640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcf840726b0 0x7fcf840b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf89e00640 1 -- 172.21.15.16:0/3110144453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf8406e640 msgr2=0x7fcf840c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf89e00640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf8406e640 0x7fcf840c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf89e00640 1 -- 172.21.15.16:0/3110144453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf840b33f0 con 0x7fcf841599d0 2024-09-10T21:15:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf89e00640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf841599d0 0x7fcf840ce740 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00ea20 tx=0x7fcf7c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf81600640 1 -- 172.21.15.16:0/3110144453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c00cd80 con 0x7fcf841599d0 2024-09-10T21:15:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf81600640 1 -- 172.21.15.16:0/3110144453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf7c004c40 con 0x7fcf841599d0 2024-09-10T21:15:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf81600640 1 -- 172.21.15.16:0/3110144453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c005280 con 0x7fcf841599d0 2024-09-10T21:15:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf8be00640 1 -- 172.21.15.16:0/3110144453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf840b36d0 con 0x7fcf841599d0 2024-09-10T21:15:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf8be00640 1 -- 172.21.15.16:0/3110144453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf84157520 con 0x7fcf841599d0 2024-09-10T21:15:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.434+0000 7fcf8be00640 1 -- 172.21.15.16:0/3110144453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf840c3650 con 0x7fcf841599d0 2024-09-10T21:15:30.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.438+0000 7fcf81600640 1 -- 172.21.15.16:0/3110144453 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcf7c00f040 con 0x7fcf841599d0 2024-09-10T21:15:30.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.438+0000 7fcf81600640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcf640725e0 0x7fcf64074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:30.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.438+0000 7fcf81600640 1 -- 172.21.15.16:0/3110144453 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fcf7c093940 con 0x7fcf841599d0 2024-09-10T21:15:30.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.438+0000 7fcf89400640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcf640725e0 0x7fcf64074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:30.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.438+0000 7fcf89400640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcf640725e0 0x7fcf64074aa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcf780af000 tx=0x7fcf7801afb0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:30.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.442+0000 7fcf81600640 1 -- 172.21.15.16:0/3110144453 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcf840c3650 con 0x7fcf841599d0 2024-09-10T21:15:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.654+0000 7fcf8be00640 1 -- 172.21.15.16:0/3110144453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcf84072b40 con 0x7fcf841599d0 2024-09-10T21:15:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.658+0000 7fcf81600640 1 -- 172.21.15.16:0/3110144453 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fcf7c016030 con 0x7fcf841599d0 2024-09-10T21:15:30.660 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:15:30.660 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.658+0000 7fcf8be00640 1 -- 172.21.15.16:0/3110144453 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcf640725e0 msgr2=0x7fcf64074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.658+0000 7fcf8be00640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcf640725e0 0x7fcf64074aa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcf780af000 tx=0x7fcf7801afb0 comp rx=0 tx=0).stop 2024-09-10T21:15:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.658+0000 7fcf8be00640 1 -- 172.21.15.16:0/3110144453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf841599d0 msgr2=0x7fcf840ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.658+0000 7fcf8be00640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf841599d0 0x7fcf840ce740 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00ea20 tx=0x7fcf7c00eee0 comp rx=0 tx=0).stop 2024-09-10T21:15:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.658+0000 7fcf8be00640 1 -- 172.21.15.16:0/3110144453 shutdown_connections 2024-09-10T21:15:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.658+0000 7fcf8be00640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcf640725e0 0x7fcf64074aa0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.658+0000 7fcf8be00640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf841599d0 0x7fcf840ce740 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.658+0000 7fcf8be00640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcf840726b0 0x7fcf840b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.658+0000 7fcf8be00640 1 --2- 172.21.15.16:0/3110144453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf8406e640 0x7fcf840c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.658+0000 7fcf8be00640 1 -- 172.21.15.16:0/3110144453 >> 172.21.15.16:0/3110144453 conn(0x7fcf840cfb20 msgr2=0x7fcf840ca150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.658+0000 7fcf8be00640 1 -- 172.21.15.16:0/3110144453 shutdown_connections 2024-09-10T21:15:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:30.658+0000 7fcf8be00640 1 -- 172.21.15.16:0/3110144453 wait complete. 2024-09-10T21:15:30.682 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:15:30.682 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:15:30.910 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:35.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:15:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 -- 172.21.15.16:0/4239866887 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe340d0890 msgr2=0x7efe340b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 --2- 172.21.15.16:0/4239866887 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe340d0890 0x7efe340b0020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009bc0 tx=0x7efe2c01cba0 comp rx=0 tx=0).stop 2024-09-10T21:15:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 -- 172.21.15.16:0/4239866887 shutdown_connections 2024-09-10T21:15:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 --2- 172.21.15.16:0/4239866887 >> v2:172.21.15.52:3300/0 conn(0x7efe340b3200 0x7efe340afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 --2- 172.21.15.16:0/4239866887 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe340d0890 0x7efe340b0020 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 -- 172.21.15.16:0/4239866887 >> 172.21.15.16:0/4239866887 conn(0x7efe340d1d80 msgr2=0x7efe340d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 -- 172.21.15.16:0/4239866887 shutdown_connections 2024-09-10T21:15:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 -- 172.21.15.16:0/4239866887 wait complete. 2024-09-10T21:15:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 Processor -- start 2024-09-10T21:15:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 -- start start 2024-09-10T21:15:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe340d0890 0x7efe3414f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe3415f450 0x7efe3414eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe3406e700 0x7efe3414f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe34168d50 con 0x7efe340d0890 2024-09-10T21:15:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe34168bd0 con 0x7efe3415f450 2024-09-10T21:15:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3c200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7efe34168ed0 con 0x7efe3406e700 2024-09-10T21:15:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe39800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe340d0890 0x7efe3414f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe39800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe340d0890 0x7efe3414f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45782/0 (socket says 172.21.15.16:45782) 2024-09-10T21:15:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe39800640 1 -- 172.21.15.16:0/1989536837 learned_addr learned my addr 172.21.15.16:0/1989536837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:15:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe3a200640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe3406e700 0x7efe3414f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe38e00640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe3415f450 0x7efe3414eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe39800640 1 -- 172.21.15.16:0/1989536837 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe3406e700 msgr2=0x7efe3414f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe39800640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe3406e700 0x7efe3414f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe39800640 1 -- 172.21.15.16:0/1989536837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe3415f450 msgr2=0x7efe3414eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe39800640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe3415f450 0x7efe3414eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.838+0000 7efe39800640 1 -- 172.21.15.16:0/1989536837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3406ecc0 con 0x7efe340d0890 2024-09-10T21:15:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.842+0000 7efe39800640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe340d0890 0x7efe3414f230 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7efe2800c3e0 tx=0x7efe2800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.842+0000 7efe31600640 1 -- 172.21.15.16:0/1989536837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28011790 con 0x7efe340d0890 2024-09-10T21:15:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.842+0000 7efe31600640 1 -- 172.21.15.16:0/1989536837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe28017070 con 0x7efe340d0890 2024-09-10T21:15:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.842+0000 7efe3c200640 1 -- 172.21.15.16:0/1989536837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3406efa0 con 0x7efe340d0890 2024-09-10T21:15:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.842+0000 7efe31600640 1 -- 172.21.15.16:0/1989536837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28011ca0 con 0x7efe340d0890 2024-09-10T21:15:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.842+0000 7efe3c200640 1 -- 172.21.15.16:0/1989536837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe34153c30 con 0x7efe340d0890 2024-09-10T21:15:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.842+0000 7efe3c200640 1 -- 172.21.15.16:0/1989536837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe34153e30 con 0x7efe340d0890 2024-09-10T21:15:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.842+0000 7efe31600640 1 -- 172.21.15.16:0/1989536837 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efe2801c460 con 0x7efe340d0890 2024-09-10T21:15:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.842+0000 7efe31600640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efe14072610 0x7efe14074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.842+0000 7efe31600640 1 -- 172.21.15.16:0/1989536837 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7efe28092e60 con 0x7efe340d0890 2024-09-10T21:15:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.842+0000 7efe38e00640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efe14072610 0x7efe14074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.846+0000 7efe38e00640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efe14072610 0x7efe14074ad0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efe20009600 tx=0x7efe200091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:35.846+0000 7efe31600640 1 -- 172.21.15.16:0/1989536837 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efe34153e30 con 0x7efe340d0890 2024-09-10T21:15:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe3c200640 1 -- 172.21.15.16:0/1989536837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efe34074ba0 con 0x7efe340d0890 2024-09-10T21:15:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe31600640 1 -- 172.21.15.16:0/1989536837 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7efe2805dee0 con 0x7efe340d0890 2024-09-10T21:15:36.062 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:15:36.062 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:15:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe3c200640 1 -- 172.21.15.16:0/1989536837 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efe14072610 msgr2=0x7efe14074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe3c200640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efe14072610 0x7efe14074ad0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efe20009600 tx=0x7efe200091e0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe3c200640 1 -- 172.21.15.16:0/1989536837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe340d0890 msgr2=0x7efe3414f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe3c200640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe340d0890 0x7efe3414f230 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7efe2800c3e0 tx=0x7efe2800c8a0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe3c200640 1 -- 172.21.15.16:0/1989536837 shutdown_connections 2024-09-10T21:15:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe3c200640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efe14072610 0x7efe14074ad0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe3c200640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe3406e700 0x7efe3414f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe3c200640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe3415f450 0x7efe3414eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe3c200640 1 --2- 172.21.15.16:0/1989536837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe340d0890 0x7efe3414f230 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe3c200640 1 -- 172.21.15.16:0/1989536837 >> 172.21.15.16:0/1989536837 conn(0x7efe340d1d80 msgr2=0x7efe340711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe3c200640 1 -- 172.21.15.16:0/1989536837 shutdown_connections 2024-09-10T21:15:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:36.058+0000 7efe3c200640 1 -- 172.21.15.16:0/1989536837 wait complete. 2024-09-10T21:15:36.084 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:15:36.084 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:41.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.238+0000 7fed63800640 1 -- 172.21.15.16:0/3525948541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed44004da0 msgr2=0x7fed44005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.238+0000 7fed63800640 1 --2- 172.21.15.16:0/3525948541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed44004da0 0x7fed44005200 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fed5c074ce0 tx=0x7fed5401d590 comp rx=0 tx=0).stop 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.238+0000 7fed61800640 1 -- 172.21.15.16:0/3525948541 reap_dead start 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.238+0000 7fed63800640 1 -- 172.21.15.16:0/3525948541 shutdown_connections 2024-09-10T21:15:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.238+0000 7fed63800640 1 -- 172.21.15.16:0/3525948541 >> 172.21.15.16:0/3525948541 conn(0x7fed5c09b130 msgr2=0x7fed5c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.238+0000 7fed63800640 1 -- 172.21.15.16:0/3525948541 shutdown_connections 2024-09-10T21:15:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.238+0000 7fed63800640 1 -- 172.21.15.16:0/3525948541 wait complete. 2024-09-10T21:15:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed63800640 1 Processor -- start 2024-09-10T21:15:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed63800640 1 -- start start 2024-09-10T21:15:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed63800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed44004da0 0x7fed5c0b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed63800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5c0b4bb0 0x7fed5c1f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed63800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed5c0cb3e0 0x7fed5c158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed63800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c0b0830 con 0x7fed5c0b4bb0 2024-09-10T21:15:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed63800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c0b06b0 con 0x7fed5c0cb3e0 2024-09-10T21:15:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed63800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fed5c167bd0 con 0x7fed44004da0 2024-09-10T21:15:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed5be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5c0b4bb0 0x7fed5c1f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed5be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5c0b4bb0 0x7fed5c1f9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45806/0 (socket says 172.21.15.16:45806) 2024-09-10T21:15:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed5be00640 1 -- 172.21.15.16:0/211734077 learned_addr learned my addr 172.21.15.16:0/211734077 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:15:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed60e00640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed44004da0 0x7fed5c0b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed61800640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed5c0cb3e0 0x7fed5c158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed5be00640 1 -- 172.21.15.16:0/211734077 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed44004da0 msgr2=0x7fed5c0b7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed5be00640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed44004da0 0x7fed5c0b7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed5be00640 1 -- 172.21.15.16:0/211734077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed5c0cb3e0 msgr2=0x7fed5c158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed5be00640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed5c0cb3e0 0x7fed5c158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed5be00640 1 -- 172.21.15.16:0/211734077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c0b4ff0 con 0x7fed5c0b4bb0 2024-09-10T21:15:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed5be00640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5c0b4bb0 0x7fed5c1f9920 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fed48006630 tx=0x7fed480039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed58c00640 1 -- 172.21.15.16:0/211734077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48015070 con 0x7fed5c0b4bb0 2024-09-10T21:15:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed58c00640 1 -- 172.21.15.16:0/211734077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed48011900 con 0x7fed5c0b4bb0 2024-09-10T21:15:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed58c00640 1 -- 172.21.15.16:0/211734077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48011d50 con 0x7fed5c0b4bb0 2024-09-10T21:15:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed63800640 1 -- 172.21.15.16:0/211734077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5c0b66f0 con 0x7fed5c0b4bb0 2024-09-10T21:15:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.242+0000 7fed63800640 1 -- 172.21.15.16:0/211734077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5c0b6c70 con 0x7fed5c0b4bb0 2024-09-10T21:15:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.246+0000 7fed63800640 1 -- 172.21.15.16:0/211734077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed5c0af560 con 0x7fed5c0b4bb0 2024-09-10T21:15:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.246+0000 7fed58c00640 1 -- 172.21.15.16:0/211734077 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fed480103c0 con 0x7fed5c0b4bb0 2024-09-10T21:15:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.246+0000 7fed58c00640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed3c0725e0 0x7fed3c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.246+0000 7fed58c00640 1 -- 172.21.15.16:0/211734077 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fed48094340 con 0x7fed5c0b4bb0 2024-09-10T21:15:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.246+0000 7fed60e00640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed3c0725e0 0x7fed3c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.246+0000 7fed60e00640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed3c0725e0 0x7fed3c074aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fed50000c10 tx=0x7fed50006bd0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.250+0000 7fed58c00640 1 -- 172.21.15.16:0/211734077 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fed4805f980 con 0x7fed5c0b4bb0 2024-09-10T21:15:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.458+0000 7fed63800640 1 -- 172.21.15.16:0/211734077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fed5c0d06c0 con 0x7fed5c0b4bb0 2024-09-10T21:15:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.462+0000 7fed58c00640 1 -- 172.21.15.16:0/211734077 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fed4805f280 con 0x7fed5c0b4bb0 2024-09-10T21:15:41.464 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:15:41.465 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:15:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.462+0000 7fed63800640 1 -- 172.21.15.16:0/211734077 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed3c0725e0 msgr2=0x7fed3c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.462+0000 7fed63800640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed3c0725e0 0x7fed3c074aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fed50000c10 tx=0x7fed50006bd0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.462+0000 7fed63800640 1 -- 172.21.15.16:0/211734077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5c0b4bb0 msgr2=0x7fed5c1f9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.462+0000 7fed63800640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5c0b4bb0 0x7fed5c1f9920 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fed48006630 tx=0x7fed480039d0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.462+0000 7fed63800640 1 -- 172.21.15.16:0/211734077 shutdown_connections 2024-09-10T21:15:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.462+0000 7fed63800640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed3c0725e0 0x7fed3c074aa0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.462+0000 7fed63800640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed5c0cb3e0 0x7fed5c158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.462+0000 7fed63800640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed5c0b4bb0 0x7fed5c1f9920 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.462+0000 7fed63800640 1 --2- 172.21.15.16:0/211734077 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed44004da0 0x7fed5c0b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.462+0000 7fed63800640 1 -- 172.21.15.16:0/211734077 >> 172.21.15.16:0/211734077 conn(0x7fed5c09b130 msgr2=0x7fed5c0aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.462+0000 7fed63800640 1 -- 172.21.15.16:0/211734077 shutdown_connections 2024-09-10T21:15:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:41.462+0000 7fed63800640 1 -- 172.21.15.16:0/211734077 wait complete. 2024-09-10T21:15:41.486 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:15:41.486 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:15:42.923 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:46.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:15:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.634+0000 7fc2cb800640 1 -- 172.21.15.16:0/1649030947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8003ab0 msgr2=0x7fc2a8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.634+0000 7fc2cb800640 1 --2- 172.21.15.16:0/1649030947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8003ab0 0x7fc2a8003ed0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400c2e0 tx=0x7fc2b401cfa0 comp rx=0 tx=0).stop 2024-09-10T21:15:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.634+0000 7fc2c9800640 1 -- 172.21.15.16:0/1649030947 reap_dead start 2024-09-10T21:15:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.634+0000 7fc2cb800640 1 -- 172.21.15.16:0/1649030947 shutdown_connections 2024-09-10T21:15:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.634+0000 7fc2cb800640 1 -- 172.21.15.16:0/1649030947 >> 172.21.15.16:0/1649030947 conn(0x7fc2c40d1d80 msgr2=0x7fc2c40d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.634+0000 7fc2cb800640 1 -- 172.21.15.16:0/1649030947 shutdown_connections 2024-09-10T21:15:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.634+0000 7fc2cb800640 1 -- 172.21.15.16:0/1649030947 wait complete. 2024-09-10T21:15:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2cb800640 1 Processor -- start 2024-09-10T21:15:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2cb800640 1 -- start start 2024-09-10T21:15:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2cb800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8003ab0 0x7fc2c40ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2cb800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2c41f8620 0x7fc2c41f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2cb800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc2c41f7790 0x7fc2c40cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2cb800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c406e890 con 0x7fc2a8003ab0 2024-09-10T21:15:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2cb800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c406e710 con 0x7fc2c41f8620 2024-09-10T21:15:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2cb800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc2c4168e60 con 0x7fc2c41f7790 2024-09-10T21:15:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2c8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8003ab0 0x7fc2c40ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2c8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8003ab0 0x7fc2c40ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46742/0 (socket says 172.21.15.16:46742) 2024-09-10T21:15:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2c8e00640 1 -- 172.21.15.16:0/1467667490 learned_addr learned my addr 172.21.15.16:0/1467667490 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:15:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2c8e00640 1 -- 172.21.15.16:0/1467667490 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc2c41f7790 msgr2=0x7fc2c40cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2c8e00640 1 --2- 172.21.15.16:0/1467667490 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc2c41f7790 0x7fc2c40cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2c8e00640 1 -- 172.21.15.16:0/1467667490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2c41f8620 msgr2=0x7fc2c41f7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2c8e00640 1 --2- 172.21.15.16:0/1467667490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2c41f8620 0x7fc2c41f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2c8e00640 1 -- 172.21.15.16:0/1467667490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c40ce880 con 0x7fc2a8003ab0 2024-09-10T21:15:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2c8e00640 1 --2- 172.21.15.16:0/1467667490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8003ab0 0x7fc2c40ce280 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc2b401ca60 tx=0x7fc2b4006c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2c1600640 1 -- 172.21.15.16:0/1467667490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4001580 con 0x7fc2a8003ab0 2024-09-10T21:15:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2c1600640 1 -- 172.21.15.16:0/1467667490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2b4006280 con 0x7fc2a8003ab0 2024-09-10T21:15:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2c1600640 1 -- 172.21.15.16:0/1467667490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4003600 con 0x7fc2a8003ab0 2024-09-10T21:15:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2cb800640 1 -- 172.21.15.16:0/1467667490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c40ceb60 con 0x7fc2a8003ab0 2024-09-10T21:15:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.638+0000 7fc2cb800640 1 -- 172.21.15.16:0/1467667490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c40cf120 con 0x7fc2a8003ab0 2024-09-10T21:15:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.642+0000 7fc2cb800640 1 -- 172.21.15.16:0/1467667490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c40b2740 con 0x7fc2a8003ab0 2024-09-10T21:15:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.642+0000 7fc2c1600640 1 -- 172.21.15.16:0/1467667490 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc2b40a5ce0 con 0x7fc2a8003ab0 2024-09-10T21:15:46.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.642+0000 7fc2c1600640 1 --2- 172.21.15.16:0/1467667490 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc2a00725e0 0x7fc2a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:46.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.642+0000 7fc2c1600640 1 -- 172.21.15.16:0/1467667490 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fc2b412c6e0 con 0x7fc2a8003ab0 2024-09-10T21:15:46.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.642+0000 7fc2bbe00640 1 --2- 172.21.15.16:0/1467667490 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc2a00725e0 0x7fc2a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:46.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.642+0000 7fc2bbe00640 1 --2- 172.21.15.16:0/1467667490 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc2a00725e0 0x7fc2a0074aa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005d10 tx=0x7fc2ac005f50 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.646+0000 7fc2c1600640 1 -- 172.21.15.16:0/1467667490 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc2b4016d50 con 0x7fc2a8003ab0 2024-09-10T21:15:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.858+0000 7fc2cb800640 1 -- 172.21.15.16:0/1467667490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc2c40b0040 con 0x7fc2a8003ab0 2024-09-10T21:15:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.858+0000 7fc2c1600640 1 -- 172.21.15.16:0/1467667490 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fc2b40f7db0 con 0x7fc2a8003ab0 2024-09-10T21:15:46.863 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:15:46.863 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:15:46.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.862+0000 7fc2cb800640 1 -- 172.21.15.16:0/1467667490 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc2a00725e0 msgr2=0x7fc2a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:46.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.862+0000 7fc2cb800640 1 --2- 172.21.15.16:0/1467667490 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc2a00725e0 0x7fc2a0074aa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005d10 tx=0x7fc2ac005f50 comp rx=0 tx=0).stop 2024-09-10T21:15:46.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.862+0000 7fc2cb800640 1 -- 172.21.15.16:0/1467667490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8003ab0 msgr2=0x7fc2c40ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:46.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.862+0000 7fc2cb800640 1 --2- 172.21.15.16:0/1467667490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8003ab0 0x7fc2c40ce280 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc2b401ca60 tx=0x7fc2b4006c40 comp rx=0 tx=0).stop 2024-09-10T21:15:46.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.862+0000 7fc2cb800640 1 -- 172.21.15.16:0/1467667490 shutdown_connections 2024-09-10T21:15:46.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.862+0000 7fc2cb800640 1 --2- 172.21.15.16:0/1467667490 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc2a00725e0 0x7fc2a0074aa0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:46.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.862+0000 7fc2cb800640 1 --2- 172.21.15.16:0/1467667490 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc2c41f7790 0x7fc2c40cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:46.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.862+0000 7fc2cb800640 1 --2- 172.21.15.16:0/1467667490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2c41f8620 0x7fc2c41f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:46.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.862+0000 7fc2cb800640 1 --2- 172.21.15.16:0/1467667490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8003ab0 0x7fc2c40ce280 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:46.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.862+0000 7fc2cb800640 1 -- 172.21.15.16:0/1467667490 >> 172.21.15.16:0/1467667490 conn(0x7fc2c40d1d80 msgr2=0x7fc2c40b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:46.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.862+0000 7fc2cb800640 1 -- 172.21.15.16:0/1467667490 shutdown_connections 2024-09-10T21:15:46.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:46.862+0000 7fc2cb800640 1 -- 172.21.15.16:0/1467667490 wait complete. 2024-09-10T21:15:46.885 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:15:46.886 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:15:46.926 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:50.930 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:51.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:15:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.042+0000 7f0819000640 1 -- 172.21.15.16:0/1666419670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0814161590 msgr2=0x7f0814068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.042+0000 7f0819000640 1 --2- 172.21.15.16:0/1666419670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0814161590 0x7f0814068c70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f08140b62e0 tx=0x7f080401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:15:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.042+0000 7f0819000640 1 -- 172.21.15.16:0/1666419670 shutdown_connections 2024-09-10T21:15:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.042+0000 7f0819000640 1 --2- 172.21.15.16:0/1666419670 >> v2:172.21.15.52:3300/0 conn(0x7f0814074860 0x7f08140af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.042+0000 7f0819000640 1 --2- 172.21.15.16:0/1666419670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0814161590 0x7f0814068c70 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.042+0000 7f0819000640 1 -- 172.21.15.16:0/1666419670 >> 172.21.15.16:0/1666419670 conn(0x7f08140b1d40 msgr2=0x7f08140b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.042+0000 7f0819000640 1 -- 172.21.15.16:0/1666419670 shutdown_connections 2024-09-10T21:15:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.042+0000 7f0819000640 1 -- 172.21.15.16:0/1666419670 wait complete. 2024-09-10T21:15:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.042+0000 7f0819000640 1 Processor -- start 2024-09-10T21:15:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.042+0000 7f0819000640 1 -- start start 2024-09-10T21:15:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0819000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0814074860 0x7f08140c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0819000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08140cc4f0 0x7f0814157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0819000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0814161590 0x7f0814099050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0819000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f081416b1f0 con 0x7f0814074860 2024-09-10T21:15:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0819000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f081416b070 con 0x7f08140cc4f0 2024-09-10T21:15:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0819000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f081416b370 con 0x7f0814161590 2024-09-10T21:15:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0813400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0814074860 0x7f08140c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0813400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0814074860 0x7f08140c6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36990/0 (socket says 172.21.15.16:36990) 2024-09-10T21:15:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0813400640 1 -- 172.21.15.16:0/276722628 learned_addr learned my addr 172.21.15.16:0/276722628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:15:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f080be00640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08140cc4f0 0x7f0814157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0813e00640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0814161590 0x7f0814099050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0813400640 1 -- 172.21.15.16:0/276722628 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0814161590 msgr2=0x7f0814099050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0813400640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0814161590 0x7f0814099050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0813400640 1 -- 172.21.15.16:0/276722628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08140cc4f0 msgr2=0x7f0814157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0813400640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08140cc4f0 0x7f0814157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0813400640 1 -- 172.21.15.16:0/276722628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08140b1240 con 0x7f0814074860 2024-09-10T21:15:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0813400640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0814074860 0x7f08140c6d30 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f08040b0040 tx=0x7f08040a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f080b400640 1 -- 172.21.15.16:0/276722628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040bb070 con 0x7f0814074860 2024-09-10T21:15:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f080b400640 1 -- 172.21.15.16:0/276722628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08040044b0 con 0x7f0814074860 2024-09-10T21:15:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f080b400640 1 -- 172.21.15.16:0/276722628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040b2040 con 0x7f0814074860 2024-09-10T21:15:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0819000640 1 -- 172.21.15.16:0/276722628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0814098570 con 0x7f0814074860 2024-09-10T21:15:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0819000640 1 -- 172.21.15.16:0/276722628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0814158490 con 0x7f0814074860 2024-09-10T21:15:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f0819000640 1 -- 172.21.15.16:0/276722628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08140af7c0 con 0x7f0814074860 2024-09-10T21:15:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.046+0000 7f080b400640 1 -- 172.21.15.16:0/276722628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f08040052d0 con 0x7f0814074860 2024-09-10T21:15:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.050+0000 7f080b400640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07f00725e0 0x7f07f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.050+0000 7f080b400640 1 -- 172.21.15.16:0/276722628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f080412ca10 con 0x7f0814074860 2024-09-10T21:15:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.050+0000 7f080be00640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07f00725e0 0x7f07f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.050+0000 7f080be00640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07f00725e0 0x7f07f0074aa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f08140982d0 tx=0x7f07fc009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.054+0000 7f080b400640 1 -- 172.21.15.16:0/276722628 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f08040164b0 con 0x7f0814074860 2024-09-10T21:15:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.262+0000 7f0819000640 1 -- 172.21.15.16:0/276722628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f08140cca40 con 0x7f0814074860 2024-09-10T21:15:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.262+0000 7f080b400640 1 -- 172.21.15.16:0/276722628 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f08040f80e0 con 0x7f0814074860 2024-09-10T21:15:52.267 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:15:52.267 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:15:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.266+0000 7f0819000640 1 -- 172.21.15.16:0/276722628 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07f00725e0 msgr2=0x7f07f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.266+0000 7f0819000640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07f00725e0 0x7f07f0074aa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f08140982d0 tx=0x7f07fc009040 comp rx=0 tx=0).stop 2024-09-10T21:15:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.266+0000 7f0819000640 1 -- 172.21.15.16:0/276722628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0814074860 msgr2=0x7f08140c6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.266+0000 7f0819000640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0814074860 0x7f08140c6d30 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f08040b0040 tx=0x7f08040a57f0 comp rx=0 tx=0).stop 2024-09-10T21:15:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.266+0000 7f0819000640 1 -- 172.21.15.16:0/276722628 shutdown_connections 2024-09-10T21:15:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.266+0000 7f0819000640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07f00725e0 0x7f07f0074aa0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.266+0000 7f0819000640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0814161590 0x7f0814099050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.266+0000 7f0819000640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08140cc4f0 0x7f0814157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.266+0000 7f0819000640 1 --2- 172.21.15.16:0/276722628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0814074860 0x7f08140c6d30 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.266+0000 7f0819000640 1 -- 172.21.15.16:0/276722628 >> 172.21.15.16:0/276722628 conn(0x7f08140b1d40 msgr2=0x7f081415bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.266+0000 7f0819000640 1 -- 172.21.15.16:0/276722628 shutdown_connections 2024-09-10T21:15:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:52.266+0000 7f0819000640 1 -- 172.21.15.16:0/276722628 wait complete. 2024-09-10T21:15:52.289 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:15:52.289 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:15:52.932 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:53.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:15:53.869 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:15:54.935 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:15:57.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:15:57.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.442+0000 7ff10ea00640 1 -- 172.21.15.16:0/3034907209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1080ca170 msgr2=0x7ff1080ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:57.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.442+0000 7ff10ea00640 1 --2- 172.21.15.16:0/3034907209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1080ca170 0x7ff1080ca620 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff10000ad90 tx=0x7ff10001dc20 comp rx=0 tx=0).stop 2024-09-10T21:15:57.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.442+0000 7ff10ea00640 1 -- 172.21.15.16:0/3034907209 shutdown_connections 2024-09-10T21:15:57.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.442+0000 7ff10ea00640 1 --2- 172.21.15.16:0/3034907209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1080ca170 0x7ff1080ca620 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:57.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.442+0000 7ff10ea00640 1 --2- 172.21.15.16:0/3034907209 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff10815a270 0x7ff1080ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:57.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.442+0000 7ff10ea00640 1 -- 172.21.15.16:0/3034907209 >> 172.21.15.16:0/3034907209 conn(0x7ff1080cfb20 msgr2=0x7ff1080cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:57.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.442+0000 7ff10ea00640 1 -- 172.21.15.16:0/3034907209 shutdown_connections 2024-09-10T21:15:57.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.442+0000 7ff10ea00640 1 -- 172.21.15.16:0/3034907209 wait complete. 2024-09-10T21:15:57.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ea00640 1 Processor -- start 2024-09-10T21:15:57.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ea00640 1 -- start start 2024-09-10T21:15:57.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1080ca170 0x7ff1080c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ea00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff1080d06b0 0x7ff108155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10815a270 0x7ff108156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff108072c30 con 0x7ff10815a270 2024-09-10T21:15:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ea00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff108072ab0 con 0x7ff1080ca170 2024-09-10T21:15:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ea00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff108072db0 con 0x7ff1080d06b0 2024-09-10T21:15:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10815a270 0x7ff108156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10815a270 0x7ff108156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37010/0 (socket says 172.21.15.16:37010) 2024-09-10T21:15:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ca00640 1 -- 172.21.15.16:0/1954209134 learned_addr learned my addr 172.21.15.16:0/1954209134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:15:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff0ffe00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff1080d06b0 0x7ff108155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff107e00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1080ca170 0x7ff1080c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ca00640 1 -- 172.21.15.16:0/1954209134 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff1080d06b0 msgr2=0x7ff108155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ca00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff1080d06b0 0x7ff108155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ca00640 1 -- 172.21.15.16:0/1954209134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1080ca170 msgr2=0x7ff1080c3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ca00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1080ca170 0x7ff1080c3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ca00640 1 -- 172.21.15.16:0/1954209134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1080ce670 con 0x7ff10815a270 2024-09-10T21:15:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ca00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10815a270 0x7ff108156180 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff10000ace0 tx=0x7ff10000a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff104c00640 1 -- 172.21.15.16:0/1954209134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000bb020 con 0x7ff10815a270 2024-09-10T21:15:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff104c00640 1 -- 172.21.15.16:0/1954209134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff100004d40 con 0x7ff10815a270 2024-09-10T21:15:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff104c00640 1 -- 172.21.15.16:0/1954209134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000b3040 con 0x7ff10815a270 2024-09-10T21:15:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ea00640 1 -- 172.21.15.16:0/1954209134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1081f9470 con 0x7ff10815a270 2024-09-10T21:15:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.446+0000 7ff10ea00640 1 -- 172.21.15.16:0/1954209134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff10814f7f0 con 0x7ff10815a270 2024-09-10T21:15:57.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.450+0000 7ff10ea00640 1 -- 172.21.15.16:0/1954209134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1080b5aa0 con 0x7ff10815a270 2024-09-10T21:15:57.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.450+0000 7ff104c00640 1 -- 172.21.15.16:0/1954209134 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff100004260 con 0x7ff10815a270 2024-09-10T21:15:57.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.450+0000 7ff104c00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff0e4072610 0x7ff0e4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:57.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.450+0000 7ff104c00640 1 -- 172.21.15.16:0/1954209134 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7ff10012d860 con 0x7ff10815a270 2024-09-10T21:15:57.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.450+0000 7ff107e00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff0e4072610 0x7ff0e4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:57.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.454+0000 7ff107e00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff0e4072610 0x7ff0e4074ad0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009ad0 tx=0x7ff0f80091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:57.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.454+0000 7ff104c00640 1 -- 172.21.15.16:0/1954209134 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff1000c5060 con 0x7ff10815a270 2024-09-10T21:15:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.670+0000 7ff10ea00640 1 -- 172.21.15.16:0/1954209134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff1081f91c0 con 0x7ff10815a270 2024-09-10T21:15:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.670+0000 7ff104c00640 1 -- 172.21.15.16:0/1954209134 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7ff1000a7e20 con 0x7ff10815a270 2024-09-10T21:15:57.674 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:15:57.674 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:15:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.674+0000 7ff10ea00640 1 -- 172.21.15.16:0/1954209134 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff0e4072610 msgr2=0x7ff0e4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.674+0000 7ff10ea00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff0e4072610 0x7ff0e4074ad0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009ad0 tx=0x7ff0f80091e0 comp rx=0 tx=0).stop 2024-09-10T21:15:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.674+0000 7ff10ea00640 1 -- 172.21.15.16:0/1954209134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10815a270 msgr2=0x7ff108156180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.674+0000 7ff10ea00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10815a270 0x7ff108156180 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff10000ace0 tx=0x7ff10000a3f0 comp rx=0 tx=0).stop 2024-09-10T21:15:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.674+0000 7ff10ea00640 1 -- 172.21.15.16:0/1954209134 shutdown_connections 2024-09-10T21:15:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.674+0000 7ff10ea00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff0e4072610 0x7ff0e4074ad0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.674+0000 7ff10ea00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10815a270 0x7ff108156180 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.674+0000 7ff10ea00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff1080d06b0 0x7ff108155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.674+0000 7ff10ea00640 1 --2- 172.21.15.16:0/1954209134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1080ca170 0x7ff1080c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.674+0000 7ff10ea00640 1 -- 172.21.15.16:0/1954209134 >> 172.21.15.16:0/1954209134 conn(0x7ff1080cfb20 msgr2=0x7ff1080758a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.674+0000 7ff10ea00640 1 -- 172.21.15.16:0/1954209134 shutdown_connections 2024-09-10T21:15:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:15:57.674+0000 7ff10ea00640 1 -- 172.21.15.16:0/1954209134 wait complete. 2024-09-10T21:15:57.698 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:15:57.698 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:15:58.938 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:02.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:16:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.854+0000 7f4139c00640 1 -- 172.21.15.16:0/2336960461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4118004810 msgr2=0x7f4118004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.854+0000 7f4139c00640 1 --2- 172.21.15.16:0/2336960461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4118004810 0x7f4118004c90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f411c006f80 tx=0x7f411c01cee0 comp rx=0 tx=0).stop 2024-09-10T21:16:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.854+0000 7f4139c00640 1 -- 172.21.15.16:0/2336960461 shutdown_connections 2024-09-10T21:16:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.854+0000 7f4139c00640 1 --2- 172.21.15.16:0/2336960461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4118004810 0x7f4118004c90 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.854+0000 7f4139c00640 1 -- 172.21.15.16:0/2336960461 >> 172.21.15.16:0/2336960461 conn(0x7f413409b130 msgr2=0x7f41340d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.854+0000 7f4139c00640 1 -- 172.21.15.16:0/2336960461 shutdown_connections 2024-09-10T21:16:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.854+0000 7f4139c00640 1 -- 172.21.15.16:0/2336960461 wait complete. 2024-09-10T21:16:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4139c00640 1 Processor -- start 2024-09-10T21:16:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4139c00640 1 -- start start 2024-09-10T21:16:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4139c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4118004810 0x7f41341555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4139c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41341f88c0 0x7f413406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4139c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41340cdd50 0x7f41340ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4139c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41340d0490 con 0x7f41341f88c0 2024-09-10T21:16:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4139c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f41340d1260 con 0x7f41340cdd50 2024-09-10T21:16:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4139c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f41340d0610 con 0x7f4118004810 2024-09-10T21:16:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f412be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41341f88c0 0x7f413406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f412be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41341f88c0 0x7f413406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56526/0 (socket says 172.21.15.16:56526) 2024-09-10T21:16:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f412be00640 1 -- 172.21.15.16:0/3392482448 learned_addr learned my addr 172.21.15.16:0/3392482448 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4132a00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4118004810 0x7f41341555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4133400640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41340cdd50 0x7f41340ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f412be00640 1 -- 172.21.15.16:0/3392482448 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4118004810 msgr2=0x7f41341555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f412be00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4118004810 0x7f41341555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f412be00640 1 -- 172.21.15.16:0/3392482448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41340cdd50 msgr2=0x7f41340ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f412be00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41340cdd50 0x7f41340ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f412be00640 1 -- 172.21.15.16:0/3392482448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413415a7f0 con 0x7f41341f88c0 2024-09-10T21:16:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4132a00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4118004810 0x7f41341555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f412be00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41341f88c0 0x7f413406dcf0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f411c01c9a0 tx=0x7f411c0011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f412aa00640 1 -- 172.21.15.16:0/3392482448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c006090 con 0x7f41341f88c0 2024-09-10T21:16:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f412aa00640 1 -- 172.21.15.16:0/3392482448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f411c006340 con 0x7f41341f88c0 2024-09-10T21:16:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4139c00640 1 -- 172.21.15.16:0/3392482448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413406e520 con 0x7f41341f88c0 2024-09-10T21:16:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f412aa00640 1 -- 172.21.15.16:0/3392482448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c003600 con 0x7f41341f88c0 2024-09-10T21:16:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4139c00640 1 -- 172.21.15.16:0/3392482448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413406eae0 con 0x7f41341f88c0 2024-09-10T21:16:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.858+0000 7f4139c00640 1 -- 172.21.15.16:0/3392482448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41340af580 con 0x7f41341f88c0 2024-09-10T21:16:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.862+0000 7f412aa00640 1 -- 172.21.15.16:0/3392482448 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f411c0b6020 con 0x7f41341f88c0 2024-09-10T21:16:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.862+0000 7f412aa00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4110072610 0x7f4110074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.862+0000 7f412aa00640 1 -- 172.21.15.16:0/3392482448 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f411c12c610 con 0x7f41341f88c0 2024-09-10T21:16:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.862+0000 7f4132a00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4110072610 0x7f4110074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.862+0000 7f4132a00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4110072610 0x7f4110074ad0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4134052940 tx=0x7f4124006b70 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:02.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:02.866+0000 7f412aa00640 1 -- 172.21.15.16:0/3392482448 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f411c0bd020 con 0x7f41341f88c0 2024-09-10T21:16:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f4139c00640 1 -- 172.21.15.16:0/3392482448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f41340cc120 con 0x7f41341f88c0 2024-09-10T21:16:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f412aa00640 1 -- 172.21.15.16:0/3392482448 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f413415a7f0 con 0x7f41341f88c0 2024-09-10T21:16:03.086 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:03.086 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:16:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f4139c00640 1 -- 172.21.15.16:0/3392482448 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4110072610 msgr2=0x7f4110074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f4139c00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4110072610 0x7f4110074ad0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4134052940 tx=0x7f4124006b70 comp rx=0 tx=0).stop 2024-09-10T21:16:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f4139c00640 1 -- 172.21.15.16:0/3392482448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41341f88c0 msgr2=0x7f413406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f4139c00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41341f88c0 0x7f413406dcf0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f411c01c9a0 tx=0x7f411c0011a0 comp rx=0 tx=0).stop 2024-09-10T21:16:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f4139c00640 1 -- 172.21.15.16:0/3392482448 shutdown_connections 2024-09-10T21:16:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f4139c00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4110072610 0x7f4110074ad0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f4139c00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41340cdd50 0x7f41340ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f4139c00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41341f88c0 0x7f413406dcf0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f4139c00640 1 --2- 172.21.15.16:0/3392482448 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4118004810 0x7f41341555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f4139c00640 1 -- 172.21.15.16:0/3392482448 >> 172.21.15.16:0/3392482448 conn(0x7f413409b130 msgr2=0x7f41340b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f4139c00640 1 -- 172.21.15.16:0/3392482448 shutdown_connections 2024-09-10T21:16:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:03.082+0000 7f4139c00640 1 -- 172.21.15.16:0/3392482448 wait complete. 2024-09-10T21:16:03.108 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:16:03.108 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:16:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:06.944 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:08.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:16:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.262+0000 7f0e4f600640 1 -- 172.21.15.16:0/89409249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4815f400 msgr2=0x7f0e48070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.262+0000 7f0e4f600640 1 --2- 172.21.15.16:0/89409249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4815f400 0x7f0e48070fe0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0e480698f0 tx=0x7f0e3c01c780 comp rx=0 tx=0).stop 2024-09-10T21:16:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.262+0000 7f0e4f600640 1 -- 172.21.15.16:0/89409249 shutdown_connections 2024-09-10T21:16:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.262+0000 7f0e4f600640 1 --2- 172.21.15.16:0/89409249 >> v2:172.21.15.52:3300/0 conn(0x7f0e480b13c0 0x7f0e48071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.262+0000 7f0e4f600640 1 --2- 172.21.15.16:0/89409249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4815f400 0x7f0e48070fe0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.262+0000 7f0e4f600640 1 -- 172.21.15.16:0/89409249 >> 172.21.15.16:0/89409249 conn(0x7f0e4809b130 msgr2=0x7f0e480d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.262+0000 7f0e4f600640 1 -- 172.21.15.16:0/89409249 shutdown_connections 2024-09-10T21:16:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.262+0000 7f0e4f600640 1 -- 172.21.15.16:0/89409249 wait complete. 2024-09-10T21:16:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4f600640 1 Processor -- start 2024-09-10T21:16:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4f600640 1 -- start start 2024-09-10T21:16:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4f600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0e480b13c0 0x7f0e48153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4f600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e480cfc80 0x7f0e480cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4f600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4815f400 0x7f0e480cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4f600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e481695f0 con 0x7f0e4815f400 2024-09-10T21:16:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4f600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0e480d1270 con 0x7f0e480cfc80 2024-09-10T21:16:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4f600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0e48169770 con 0x7f0e480b13c0 2024-09-10T21:16:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4815f400 0x7f0e480cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4815f400 0x7f0e480cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56538/0 (socket says 172.21.15.16:56538) 2024-09-10T21:16:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4d600640 1 -- 172.21.15.16:0/1331084114 learned_addr learned my addr 172.21.15.16:0/1331084114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e47e00640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e480cfc80 0x7f0e480cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4cc00640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0e480b13c0 0x7f0e48153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4d600640 1 -- 172.21.15.16:0/1331084114 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0e480b13c0 msgr2=0x7f0e48153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4d600640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0e480b13c0 0x7f0e48153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4d600640 1 -- 172.21.15.16:0/1331084114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e480cfc80 msgr2=0x7f0e480cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4d600640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e480cfc80 0x7f0e480cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4d600640 1 -- 172.21.15.16:0/1331084114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e481585a0 con 0x7f0e4815f400 2024-09-10T21:16:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4d600640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4815f400 0x7f0e480cdf20 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000c760 tx=0x7f0e4000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e44c00640 1 -- 172.21.15.16:0/1331084114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40003db0 con 0x7f0e4815f400 2024-09-10T21:16:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e44c00640 1 -- 172.21.15.16:0/1331084114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e40005070 con 0x7f0e4815f400 2024-09-10T21:16:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e44c00640 1 -- 172.21.15.16:0/1331084114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e400054c0 con 0x7f0e4815f400 2024-09-10T21:16:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4f600640 1 -- 172.21.15.16:0/1331084114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e48158880 con 0x7f0e4815f400 2024-09-10T21:16:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4f600640 1 -- 172.21.15.16:0/1331084114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e48158e40 con 0x7f0e4815f400 2024-09-10T21:16:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.266+0000 7f0e4f600640 1 -- 172.21.15.16:0/1331084114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e480d00d0 con 0x7f0e4815f400 2024-09-10T21:16:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.270+0000 7f0e44c00640 1 -- 172.21.15.16:0/1331084114 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0e40005660 con 0x7f0e4815f400 2024-09-10T21:16:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.270+0000 7f0e44c00640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0e280725e0 0x7f0e28074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.270+0000 7f0e44c00640 1 -- 172.21.15.16:0/1331084114 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f0e40092f40 con 0x7f0e4815f400 2024-09-10T21:16:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.270+0000 7f0e4cc00640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0e280725e0 0x7f0e28074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.270+0000 7f0e4cc00640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0e280725e0 0x7f0e28074aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0af000 tx=0x7f0e3c01afc0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.274+0000 7f0e44c00640 1 -- 172.21.15.16:0/1331084114 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0e4005dfc0 con 0x7f0e4815f400 2024-09-10T21:16:08.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.482+0000 7f0e4f600640 1 -- 172.21.15.16:0/1331084114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0e4806fb50 con 0x7f0e4815f400 2024-09-10T21:16:08.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.486+0000 7f0e44c00640 1 -- 172.21.15.16:0/1331084114 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f0e400629f0 con 0x7f0e4815f400 2024-09-10T21:16:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:08.488 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:16:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.486+0000 7f0e4f600640 1 -- 172.21.15.16:0/1331084114 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0e280725e0 msgr2=0x7f0e28074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.486+0000 7f0e4f600640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0e280725e0 0x7f0e28074aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0af000 tx=0x7f0e3c01afc0 comp rx=0 tx=0).stop 2024-09-10T21:16:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.486+0000 7f0e4f600640 1 -- 172.21.15.16:0/1331084114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4815f400 msgr2=0x7f0e480cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.486+0000 7f0e4f600640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4815f400 0x7f0e480cdf20 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000c760 tx=0x7f0e4000cc20 comp rx=0 tx=0).stop 2024-09-10T21:16:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.486+0000 7f0e4f600640 1 -- 172.21.15.16:0/1331084114 shutdown_connections 2024-09-10T21:16:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.486+0000 7f0e4f600640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0e280725e0 0x7f0e28074aa0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.486+0000 7f0e4f600640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e4815f400 0x7f0e480cdf20 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.486+0000 7f0e4f600640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e480cfc80 0x7f0e480cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.486+0000 7f0e4f600640 1 --2- 172.21.15.16:0/1331084114 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0e480b13c0 0x7f0e48153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.486+0000 7f0e4f600640 1 -- 172.21.15.16:0/1331084114 >> 172.21.15.16:0/1331084114 conn(0x7f0e4809b130 msgr2=0x7f0e48071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.486+0000 7f0e4f600640 1 -- 172.21.15.16:0/1331084114 shutdown_connections 2024-09-10T21:16:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:08.486+0000 7f0e4f600640 1 -- 172.21.15.16:0/1331084114 wait complete. 2024-09-10T21:16:08.511 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:16:08.511 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:16:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:10.948 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:12.951 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:13.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:16:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.666+0000 7f8293e00640 1 -- 172.21.15.16:0/444431194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294161560 msgr2=0x7f82940d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.666+0000 7f8293e00640 1 --2- 172.21.15.16:0/444431194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294161560 0x7f82940d4a20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f82940b6a70 tx=0x7f828401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.666+0000 7f8293e00640 1 -- 172.21.15.16:0/444431194 shutdown_connections 2024-09-10T21:16:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.666+0000 7f8293e00640 1 --2- 172.21.15.16:0/444431194 >> v2:172.21.15.52:3300/0 conn(0x7f82940b2c20 0x7f82940d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.666+0000 7f8293e00640 1 --2- 172.21.15.16:0/444431194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294161560 0x7f82940d4a20 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.666+0000 7f8293e00640 1 -- 172.21.15.16:0/444431194 >> 172.21.15.16:0/444431194 conn(0x7f82940d3f00 msgr2=0x7f82940d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293e00640 1 -- 172.21.15.16:0/444431194 shutdown_connections 2024-09-10T21:16:13.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293e00640 1 -- 172.21.15.16:0/444431194 wait complete. 2024-09-10T21:16:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293e00640 1 Processor -- start 2024-09-10T21:16:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293e00640 1 -- start start 2024-09-10T21:16:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f82940b2c20 0x7f82941513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82940d0c80 0x7f829406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294161560 0x7f829406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f829416b460 con 0x7f8294161560 2024-09-10T21:16:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f829416b2e0 con 0x7f82940d0c80 2024-09-10T21:16:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f829416b5e0 con 0x7f82940b2c20 2024-09-10T21:16:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294161560 0x7f829406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294161560 0x7f829406d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50672/0 (socket says 172.21.15.16:50672) 2024-09-10T21:16:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293400640 1 -- 172.21.15.16:0/2152339940 learned_addr learned my addr 172.21.15.16:0/2152339940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f828be00640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82940d0c80 0x7f829406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8292a00640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f82940b2c20 0x7f82941513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293400640 1 -- 172.21.15.16:0/2152339940 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f82940b2c20 msgr2=0x7f82941513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293400640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f82940b2c20 0x7f82941513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293400640 1 -- 172.21.15.16:0/2152339940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82940d0c80 msgr2=0x7f829406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293400640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82940d0c80 0x7f829406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293400640 1 -- 172.21.15.16:0/2152339940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8294151910 con 0x7f8294161560 2024-09-10T21:16:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.670+0000 7f8293400640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294161560 0x7f829406d960 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f828c00c760 tx=0x7f828c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.674+0000 7f828b400640 1 -- 172.21.15.16:0/2152339940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c003db0 con 0x7f8294161560 2024-09-10T21:16:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.674+0000 7f828b400640 1 -- 172.21.15.16:0/2152339940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f828c005070 con 0x7f8294161560 2024-09-10T21:16:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.674+0000 7f828b400640 1 -- 172.21.15.16:0/2152339940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c0054c0 con 0x7f8294161560 2024-09-10T21:16:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.674+0000 7f8293e00640 1 -- 172.21.15.16:0/2152339940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82940b8760 con 0x7f8294161560 2024-09-10T21:16:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.674+0000 7f8293e00640 1 -- 172.21.15.16:0/2152339940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82940b8d20 con 0x7f8294161560 2024-09-10T21:16:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.674+0000 7f8293e00640 1 -- 172.21.15.16:0/2152339940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82940d0740 con 0x7f8294161560 2024-09-10T21:16:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.674+0000 7f828b400640 1 -- 172.21.15.16:0/2152339940 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f828c005660 con 0x7f8294161560 2024-09-10T21:16:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.678+0000 7f828b400640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f82700725e0 0x7f8270074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.678+0000 7f828b400640 1 -- 172.21.15.16:0/2152339940 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f828c092f40 con 0x7f8294161560 2024-09-10T21:16:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.678+0000 7f8292a00640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f82700725e0 0x7f8270074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.678+0000 7f8292a00640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f82700725e0 0x7f8270074aa0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f82840af000 tx=0x7f828401b100 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:13.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.682+0000 7f828b400640 1 -- 172.21.15.16:0/2152339940 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f828c05dfc0 con 0x7f8294161560 2024-09-10T21:16:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.890+0000 7f8293e00640 1 -- 172.21.15.16:0/2152339940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f82940d0310 con 0x7f8294161560 2024-09-10T21:16:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.894+0000 7f828b400640 1 -- 172.21.15.16:0/2152339940 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f828c0629f0 con 0x7f8294161560 2024-09-10T21:16:13.897 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:13.897 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:16:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.894+0000 7f8293e00640 1 -- 172.21.15.16:0/2152339940 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f82700725e0 msgr2=0x7f8270074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.894+0000 7f8293e00640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f82700725e0 0x7f8270074aa0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f82840af000 tx=0x7f828401b100 comp rx=0 tx=0).stop 2024-09-10T21:16:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.894+0000 7f8293e00640 1 -- 172.21.15.16:0/2152339940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294161560 msgr2=0x7f829406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.894+0000 7f8293e00640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294161560 0x7f829406d960 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f828c00c760 tx=0x7f828c00cc20 comp rx=0 tx=0).stop 2024-09-10T21:16:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.894+0000 7f8293e00640 1 -- 172.21.15.16:0/2152339940 shutdown_connections 2024-09-10T21:16:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.894+0000 7f8293e00640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f82700725e0 0x7f8270074aa0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.894+0000 7f8293e00640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294161560 0x7f829406d960 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.894+0000 7f8293e00640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82940d0c80 0x7f829406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.894+0000 7f8293e00640 1 --2- 172.21.15.16:0/2152339940 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f82940b2c20 0x7f82941513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.894+0000 7f8293e00640 1 -- 172.21.15.16:0/2152339940 >> 172.21.15.16:0/2152339940 conn(0x7f82940d3f00 msgr2=0x7f8294160680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.894+0000 7f8293e00640 1 -- 172.21.15.16:0/2152339940 shutdown_connections 2024-09-10T21:16:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:13.894+0000 7f8293e00640 1 -- 172.21.15.16:0/2152339940 wait complete. 2024-09-10T21:16:13.918 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:16:13.918 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:16:14.952 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:18.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:16:18.956 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 -- 172.21.15.16:0/2677133504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa815f450 msgr2=0x7f5fa80cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 --2- 172.21.15.16:0/2677133504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa815f450 0x7f5fa80cfc80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8073820 tx=0x7f5f9801f070 comp rx=0 tx=0).stop 2024-09-10T21:16:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 -- 172.21.15.16:0/2677133504 shutdown_connections 2024-09-10T21:16:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 --2- 172.21.15.16:0/2677133504 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5fa80b3200 0x7f5fa80d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 --2- 172.21.15.16:0/2677133504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa815f450 0x7f5fa80cfc80 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 -- 172.21.15.16:0/2677133504 >> 172.21.15.16:0/2677133504 conn(0x7f5fa80d1d80 msgr2=0x7f5fa80d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 -- 172.21.15.16:0/2677133504 shutdown_connections 2024-09-10T21:16:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 -- 172.21.15.16:0/2677133504 wait complete. 2024-09-10T21:16:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 Processor -- start 2024-09-10T21:16:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 -- start start 2024-09-10T21:16:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5fa80d0890 0x7f5fa80cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa815f450 0x7f5fa81f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fa80b7360 0x7f5fa806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa80b2e70 con 0x7f5fa815f450 2024-09-10T21:16:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa80b2cf0 con 0x7f5fa80b7360 2024-09-10T21:16:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5faee00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa80b2ff0 con 0x7f5fa80d0890 2024-09-10T21:16:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5f9fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa815f450 0x7f5fa81f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5f9fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa815f450 0x7f5fa81f5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50684/0 (socket says 172.21.15.16:50684) 2024-09-10T21:16:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5f9fe00640 1 -- 172.21.15.16:0/1202163538 learned_addr learned my addr 172.21.15.16:0/1202163538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5fa7e00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5fa80d0890 0x7f5fa80cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.074+0000 7f5face00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fa80b7360 0x7f5fa806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5f9fe00640 1 -- 172.21.15.16:0/1202163538 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5fa80d0890 msgr2=0x7f5fa80cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5f9fe00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5fa80d0890 0x7f5fa80cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5f9fe00640 1 -- 172.21.15.16:0/1202163538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fa80b7360 msgr2=0x7f5fa806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5f9fe00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fa80b7360 0x7f5fa806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5f9fe00640 1 -- 172.21.15.16:0/1202163538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa80ce7f0 con 0x7f5fa815f450 2024-09-10T21:16:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5face00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fa80b7360 0x7f5fa806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5f9fe00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa815f450 0x7f5fa81f5780 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000c970 tx=0x7f5f9000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5fa4c00640 1 -- 172.21.15.16:0/1202163538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90007c70 con 0x7f5fa815f450 2024-09-10T21:16:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5fa4c00640 1 -- 172.21.15.16:0/1202163538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f9000f450 con 0x7f5fa815f450 2024-09-10T21:16:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5fa4c00640 1 -- 172.21.15.16:0/1202163538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9000f8a0 con 0x7f5fa815f450 2024-09-10T21:16:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5faee00640 1 -- 172.21.15.16:0/1202163538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa80cdf50 con 0x7f5fa815f450 2024-09-10T21:16:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5faee00640 1 -- 172.21.15.16:0/1202163538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa80ce510 con 0x7f5fa815f450 2024-09-10T21:16:19.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5faee00640 1 -- 172.21.15.16:0/1202163538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa80b1e90 con 0x7f5fa815f450 2024-09-10T21:16:19.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5fa4c00640 1 -- 172.21.15.16:0/1202163538 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5f9000fa40 con 0x7f5fa815f450 2024-09-10T21:16:19.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5fa4c00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f840725e0 0x7f5f84074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.078+0000 7f5fa4c00640 1 -- 172.21.15.16:0/1202163538 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f5f90093920 con 0x7f5fa815f450 2024-09-10T21:16:19.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.082+0000 7f5fa7e00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f840725e0 0x7f5f84074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.082+0000 7f5fa7e00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f840725e0 0x7f5f84074aa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5f980b2000 tx=0x7f5f9801d910 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:19.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.082+0000 7f5fa4c00640 1 -- 172.21.15.16:0/1202163538 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f90014030 con 0x7f5fa815f450 2024-09-10T21:16:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5faee00640 1 -- 172.21.15.16:0/1202163538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5fa8074ba0 con 0x7f5fa815f450 2024-09-10T21:16:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5fa4c00640 1 -- 172.21.15.16:0/1202163538 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f5f9005f090 con 0x7f5fa815f450 2024-09-10T21:16:19.297 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:19.298 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:16:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5faee00640 1 -- 172.21.15.16:0/1202163538 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f840725e0 msgr2=0x7f5f84074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5faee00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f840725e0 0x7f5f84074aa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5f980b2000 tx=0x7f5f9801d910 comp rx=0 tx=0).stop 2024-09-10T21:16:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5faee00640 1 -- 172.21.15.16:0/1202163538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa815f450 msgr2=0x7f5fa81f5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5faee00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa815f450 0x7f5fa81f5780 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000c970 tx=0x7f5f9000ce30 comp rx=0 tx=0).stop 2024-09-10T21:16:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5faee00640 1 -- 172.21.15.16:0/1202163538 shutdown_connections 2024-09-10T21:16:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5faee00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5f840725e0 0x7f5f84074aa0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5faee00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fa80b7360 0x7f5fa806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5faee00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa815f450 0x7f5fa81f5780 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5faee00640 1 --2- 172.21.15.16:0/1202163538 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5fa80d0890 0x7f5fa80cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5faee00640 1 -- 172.21.15.16:0/1202163538 >> 172.21.15.16:0/1202163538 conn(0x7f5fa80d1d80 msgr2=0x7f5fa8070960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5faee00640 1 -- 172.21.15.16:0/1202163538 shutdown_connections 2024-09-10T21:16:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:19.294+0000 7f5faee00640 1 -- 172.21.15.16:0/1202163538 wait complete. 2024-09-10T21:16:19.319 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:16:19.320 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:16:20.958 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:23.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:16:23.942 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:16:24.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:16:24.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.474+0000 7f6882e00640 1 -- 172.21.15.16:0/4096132441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860004810 msgr2=0x7f6860004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:24.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.474+0000 7f6882e00640 1 --2- 172.21.15.16:0/4096132441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860004810 0x7f6860004c90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f687c0698f0 tx=0x7f686801c840 comp rx=0 tx=0).stop 2024-09-10T21:16:24.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.474+0000 7f6880e00640 1 -- 172.21.15.16:0/4096132441 reap_dead start 2024-09-10T21:16:24.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.474+0000 7f6882e00640 1 -- 172.21.15.16:0/4096132441 shutdown_connections 2024-09-10T21:16:24.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.474+0000 7f6882e00640 1 -- 172.21.15.16:0/4096132441 >> 172.21.15.16:0/4096132441 conn(0x7f687c09b130 msgr2=0x7f687c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:24.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.474+0000 7f6882e00640 1 -- 172.21.15.16:0/4096132441 shutdown_connections 2024-09-10T21:16:24.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.474+0000 7f6882e00640 1 -- 172.21.15.16:0/4096132441 wait complete. 2024-09-10T21:16:24.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6882e00640 1 Processor -- start 2024-09-10T21:16:24.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6882e00640 1 -- start start 2024-09-10T21:16:24.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6882e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860004810 0x7f687c159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:24.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6882e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c15a6c0 0x7f687c1f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:24.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6882e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f687c0cfe90 0x7f687c0d0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:24.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6882e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f687c167de0 con 0x7f6860004810 2024-09-10T21:16:24.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6882e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f687c0d1240 con 0x7f687c15a6c0 2024-09-10T21:16:24.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6882e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f687c167f60 con 0x7f687c0cfe90 2024-09-10T21:16:24.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f687be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860004810 0x7f687c159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:24.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f687be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860004810 0x7f687c159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56736/0 (socket says 172.21.15.16:56736) 2024-09-10T21:16:24.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f687be00640 1 -- 172.21.15.16:0/1601266420 learned_addr learned my addr 172.21.15.16:0/1601266420 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:24.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f687b400640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c15a6c0 0x7f687c1f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:24.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6880e00640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f687c0cfe90 0x7f687c0d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:24.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f687be00640 1 -- 172.21.15.16:0/1601266420 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f687c0cfe90 msgr2=0x7f687c0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:24.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f687be00640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f687c0cfe90 0x7f687c0d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:24.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f687be00640 1 -- 172.21.15.16:0/1601266420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c15a6c0 msgr2=0x7f687c1f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:24.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f687be00640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c15a6c0 0x7f687c1f8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:24.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f687be00640 1 -- 172.21.15.16:0/1601266420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c15ae40 con 0x7f6860004810 2024-09-10T21:16:24.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f687be00640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860004810 0x7f687c159f60 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f686c0118d0 tx=0x7f686c011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:24.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6873e00640 1 -- 172.21.15.16:0/1601266420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c01e070 con 0x7f6860004810 2024-09-10T21:16:24.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6873e00640 1 -- 172.21.15.16:0/1601266420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f686c00fbb0 con 0x7f6860004810 2024-09-10T21:16:24.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6873e00640 1 -- 172.21.15.16:0/1601266420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c019410 con 0x7f6860004810 2024-09-10T21:16:24.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6882e00640 1 -- 172.21.15.16:0/1601266420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687c15b120 con 0x7f6860004810 2024-09-10T21:16:24.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.478+0000 7f6882e00640 1 -- 172.21.15.16:0/1601266420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687c15b6e0 con 0x7f6860004810 2024-09-10T21:16:24.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.482+0000 7f6882e00640 1 -- 172.21.15.16:0/1601266420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f687c0cfc80 con 0x7f6860004810 2024-09-10T21:16:24.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.482+0000 7f6873e00640 1 -- 172.21.15.16:0/1601266420 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f686c0195b0 con 0x7f6860004810 2024-09-10T21:16:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.482+0000 7f6873e00640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f685c0725e0 0x7f685c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.482+0000 7f6873e00640 1 -- 172.21.15.16:0/1601266420 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f686c093c10 con 0x7f6860004810 2024-09-10T21:16:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.482+0000 7f687b400640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f685c0725e0 0x7f685c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:24.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.486+0000 7f687b400640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f685c0725e0 0x7f685c074aa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f687c09b860 tx=0x7f68680a8040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:24.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.486+0000 7f6873e00640 1 -- 172.21.15.16:0/1601266420 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f686c05ec90 con 0x7f6860004810 2024-09-10T21:16:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6882e00640 1 -- 172.21.15.16:0/1601266420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f687c15ba50 con 0x7f6860004810 2024-09-10T21:16:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6873e00640 1 -- 172.21.15.16:0/1601266420 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f687c15ba50 con 0x7f6860004810 2024-09-10T21:16:24.703 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:24.703 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:16:24.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6882e00640 1 -- 172.21.15.16:0/1601266420 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f685c0725e0 msgr2=0x7f685c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6882e00640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f685c0725e0 0x7f685c074aa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f687c09b860 tx=0x7f68680a8040 comp rx=0 tx=0).stop 2024-09-10T21:16:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6882e00640 1 -- 172.21.15.16:0/1601266420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860004810 msgr2=0x7f687c159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6882e00640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860004810 0x7f687c159f60 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f686c0118d0 tx=0x7f686c011d90 comp rx=0 tx=0).stop 2024-09-10T21:16:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6882e00640 1 -- 172.21.15.16:0/1601266420 shutdown_connections 2024-09-10T21:16:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6882e00640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f685c0725e0 0x7f685c074aa0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6882e00640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f687c0cfe90 0x7f687c0d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6882e00640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c15a6c0 0x7f687c1f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6882e00640 1 --2- 172.21.15.16:0/1601266420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6860004810 0x7f687c159f60 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6882e00640 1 -- 172.21.15.16:0/1601266420 >> 172.21.15.16:0/1601266420 conn(0x7f687c09b130 msgr2=0x7f687c0b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6882e00640 1 -- 172.21.15.16:0/1601266420 shutdown_connections 2024-09-10T21:16:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24.698+0000 7f6882e00640 1 -- 172.21.15.16:0/1601266420 wait complete. 2024-09-10T21:16:24.723 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-10T21:16:24.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-10T21:16:24.962 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:16:24.963 INFO:teuthology.orchestra.run.smithi016.stderr:stdin closed, goodbye! 2024-09-10T21:16:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:16:24.983 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:16:24.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:16:25.102 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:16:25.195 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:16:25.195 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:16:25.195 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 400\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi016') 2024-09-10T21:16:25.195 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:16:25.196 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:25.204 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:16:25.205 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:25.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:16:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.442+0000 7fd872800640 1 -- 172.21.15.16:0/84488616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd86c161620 msgr2=0x7fd86c070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.442+0000 7fd872800640 1 --2- 172.21.15.16:0/84488616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd86c161620 0x7fd86c070cf0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd85c005fd0 tx=0x7fd85c01c7f0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.442+0000 7fd872800640 1 -- 172.21.15.16:0/84488616 shutdown_connections 2024-09-10T21:16:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.442+0000 7fd872800640 1 --2- 172.21.15.16:0/84488616 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd86c071230 0x7fd86c0d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.442+0000 7fd872800640 1 --2- 172.21.15.16:0/84488616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd86c161620 0x7fd86c070cf0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.442+0000 7fd872800640 1 -- 172.21.15.16:0/84488616 >> 172.21.15.16:0/84488616 conn(0x7fd86c09b130 msgr2=0x7fd86c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.442+0000 7fd872800640 1 -- 172.21.15.16:0/84488616 shutdown_connections 2024-09-10T21:16:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.442+0000 7fd872800640 1 -- 172.21.15.16:0/84488616 wait complete. 2024-09-10T21:16:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd872800640 1 Processor -- start 2024-09-10T21:16:25.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd872800640 1 -- start start 2024-09-10T21:16:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd872800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd86c071230 0x7fd86c155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd872800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd86c0b0b40 0x7fd86c15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd872800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd86c161620 0x7fd86c150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd872800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c16b2c0 con 0x7fd86c0b0b40 2024-09-10T21:16:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd872800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c16b140 con 0x7fd86c071230 2024-09-10T21:16:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd872800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd86c16b440 con 0x7fd86c161620 2024-09-10T21:16:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd862a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd86c0b0b40 0x7fd86c15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd862a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd86c0b0b40 0x7fd86c15dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56756/0 (socket says 172.21.15.16:56756) 2024-09-10T21:16:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd862a00640 1 -- 172.21.15.16:0/293058148 learned_addr learned my addr 172.21.15.16:0/293058148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd862a00640 1 -- 172.21.15.16:0/293058148 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd86c161620 msgr2=0x7fd86c150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd862a00640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd86c161620 0x7fd86c150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd86b400640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd86c071230 0x7fd86c155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd862a00640 1 -- 172.21.15.16:0/293058148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd86c071230 msgr2=0x7fd86c155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd862a00640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd86c071230 0x7fd86c155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd862a00640 1 -- 172.21.15.16:0/293058148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86c0b55b0 con 0x7fd86c0b0b40 2024-09-10T21:16:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd86b400640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd86c071230 0x7fd86c155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd862a00640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd86c0b0b40 0x7fd86c15dbe0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd85400c900 tx=0x7fd85400cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd863e00640 1 -- 172.21.15.16:0/293058148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854007c70 con 0x7fd86c0b0b40 2024-09-10T21:16:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd863e00640 1 -- 172.21.15.16:0/293058148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd85400f450 con 0x7fd86c0b0b40 2024-09-10T21:16:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd863e00640 1 -- 172.21.15.16:0/293058148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85400f8a0 con 0x7fd86c0b0b40 2024-09-10T21:16:25.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd872800640 1 -- 172.21.15.16:0/293058148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86c15d3e0 con 0x7fd86c0b0b40 2024-09-10T21:16:25.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.446+0000 7fd872800640 1 -- 172.21.15.16:0/293058148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd86c1563e0 con 0x7fd86c0b0b40 2024-09-10T21:16:25.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.450+0000 7fd872800640 1 -- 172.21.15.16:0/293058148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd86c0d0590 con 0x7fd86c0b0b40 2024-09-10T21:16:25.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.450+0000 7fd863e00640 1 -- 172.21.15.16:0/293058148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd85400fa40 con 0x7fd86c0b0b40 2024-09-10T21:16:25.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.450+0000 7fd863e00640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd8480725e0 0x7fd848074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.450+0000 7fd863e00640 1 -- 172.21.15.16:0/293058148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fd854093af0 con 0x7fd86c0b0b40 2024-09-10T21:16:25.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.450+0000 7fd86b400640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd8480725e0 0x7fd848074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.450+0000 7fd86b400640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd8480725e0 0x7fd848074aa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0ae000 tx=0x7fd85c01b030 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.454+0000 7fd863e00640 1 -- 172.21.15.16:0/293058148 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd854014030 con 0x7fd86c0b0b40 2024-09-10T21:16:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.638+0000 7fd872800640 1 -- 172.21.15.16:0/293058148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd86c0b5840 con 0x7fd86c0b0b40 2024-09-10T21:16:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.638+0000 7fd863e00640 1 -- 172.21.15.16:0/293058148 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v81) ==== 80+0+0 (secure 0 0 0) 0x7fd85405eae0 con 0x7fd86c0b0b40 2024-09-10T21:16:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:16:25.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.642+0000 7fd872800640 1 -- 172.21.15.16:0/293058148 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd8480725e0 msgr2=0x7fd848074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.642+0000 7fd872800640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd8480725e0 0x7fd848074aa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0ae000 tx=0x7fd85c01b030 comp rx=0 tx=0).stop 2024-09-10T21:16:25.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.642+0000 7fd872800640 1 -- 172.21.15.16:0/293058148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd86c0b0b40 msgr2=0x7fd86c15dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.642+0000 7fd872800640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd86c0b0b40 0x7fd86c15dbe0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd85400c900 tx=0x7fd85400cdc0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.642+0000 7fd872800640 1 -- 172.21.15.16:0/293058148 shutdown_connections 2024-09-10T21:16:25.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.642+0000 7fd872800640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd8480725e0 0x7fd848074aa0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.642+0000 7fd872800640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd86c161620 0x7fd86c150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.642+0000 7fd872800640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd86c0b0b40 0x7fd86c15dbe0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.642+0000 7fd872800640 1 --2- 172.21.15.16:0/293058148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd86c071230 0x7fd86c155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.642+0000 7fd872800640 1 -- 172.21.15.16:0/293058148 >> 172.21.15.16:0/293058148 conn(0x7fd86c09b130 msgr2=0x7fd86c157800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.642+0000 7fd872800640 1 -- 172.21.15.16:0/293058148 shutdown_connections 2024-09-10T21:16:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.642+0000 7fd872800640 1 -- 172.21.15.16:0/293058148 wait complete. 2024-09-10T21:16:25.666 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:16:25.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:25.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:16:25.734 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:16:25.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:25.734 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:16:25.748 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:16:25.748 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:16:25.748 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:25.757 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:16:25.757 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:25.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:16:25.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 -- 172.21.15.16:0/3572414672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff9015ced0 msgr2=0x7eff900ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 --2- 172.21.15.16:0/3572414672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff9015ced0 0x7eff900ca170 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7eff900cf9a0 tx=0x7eff8801cd80 comp rx=0 tx=0).stop 2024-09-10T21:16:25.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 -- 172.21.15.16:0/3572414672 shutdown_connections 2024-09-10T21:16:25.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 --2- 172.21.15.16:0/3572414672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff9015ced0 0x7eff900ca170 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 --2- 172.21.15.16:0/3572414672 >> v2:172.21.15.52:3300/0 conn(0x7eff9006e640 0x7eff9015c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 --2- 172.21.15.16:0/3572414672 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7eff900b5050 0x7eff9015f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 -- 172.21.15.16:0/3572414672 >> 172.21.15.16:0/3572414672 conn(0x7eff900cfb20 msgr2=0x7eff900cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:25.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 -- 172.21.15.16:0/3572414672 shutdown_connections 2024-09-10T21:16:25.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 -- 172.21.15.16:0/3572414672 wait complete. 2024-09-10T21:16:25.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 Processor -- start 2024-09-10T21:16:25.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 -- start start 2024-09-10T21:16:25.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7eff9006e640 0x7eff900c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff900b5050 0x7eff900c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff9015ced0 0x7eff900b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff90074280 con 0x7eff900b5050 2024-09-10T21:16:25.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff90074100 con 0x7eff9015ced0 2024-09-10T21:16:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff96800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7eff90074400 con 0x7eff9006e640 2024-09-10T21:16:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff86a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff900b5050 0x7eff900c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff86a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff900b5050 0x7eff900c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56774/0 (socket says 172.21.15.16:56774) 2024-09-10T21:16:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.954+0000 7eff86a00640 1 -- 172.21.15.16:0/2493526581 learned_addr learned my addr 172.21.15.16:0/2493526581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff8f400640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7eff9006e640 0x7eff900c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff86a00640 1 -- 172.21.15.16:0/2493526581 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7eff9006e640 msgr2=0x7eff900c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff86a00640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7eff9006e640 0x7eff900c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff86a00640 1 -- 172.21.15.16:0/2493526581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff9015ced0 msgr2=0x7eff900b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff8fe00640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff9015ced0 0x7eff900b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff86a00640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff9015ced0 0x7eff900b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff86a00640 1 -- 172.21.15.16:0/2493526581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff900c8fa0 con 0x7eff900b5050 2024-09-10T21:16:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff8fe00640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff9015ced0 0x7eff900b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:25.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff86a00640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff900b5050 0x7eff900c8a60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7eff7800d9b0 tx=0x7eff7800de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:25.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff87e00640 1 -- 172.21.15.16:0/2493526581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78017070 con 0x7eff900b5050 2024-09-10T21:16:25.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff87e00640 1 -- 172.21.15.16:0/2493526581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff78004310 con 0x7eff900b5050 2024-09-10T21:16:25.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff87e00640 1 -- 172.21.15.16:0/2493526581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78009c30 con 0x7eff900b5050 2024-09-10T21:16:25.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff96800640 1 -- 172.21.15.16:0/2493526581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff901faf80 con 0x7eff900b5050 2024-09-10T21:16:25.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff96800640 1 -- 172.21.15.16:0/2493526581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff901fb540 con 0x7eff900b5050 2024-09-10T21:16:25.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff96800640 1 -- 172.21.15.16:0/2493526581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff900748b0 con 0x7eff900b5050 2024-09-10T21:16:25.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff87e00640 1 -- 172.21.15.16:0/2493526581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7eff78003600 con 0x7eff900b5050 2024-09-10T21:16:25.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.958+0000 7eff87e00640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7eff6c072610 0x7eff6c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.962+0000 7eff87e00640 1 -- 172.21.15.16:0/2493526581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7eff780933a0 con 0x7eff900b5050 2024-09-10T21:16:25.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.962+0000 7eff8f400640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7eff6c072610 0x7eff6c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.962+0000 7eff8f400640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7eff6c072610 0x7eff6c074ad0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7eff80005d90 tx=0x7eff80005ca0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:25.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:25.966+0000 7eff87e00640 1 -- 172.21.15.16:0/2493526581 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7eff900c8fa0 con 0x7eff900b5050 2024-09-10T21:16:26.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff96800640 1 -- 172.21.15.16:0/2493526581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7eff9015ca30 con 0x7eff900b5050 2024-09-10T21:16:26.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff87e00640 1 -- 172.21.15.16:0/2493526581 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v81) ==== 80+0+0 (secure 0 0 0) 0x7eff7805ea80 con 0x7eff900b5050 2024-09-10T21:16:26.153 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:16:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff96800640 1 -- 172.21.15.16:0/2493526581 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7eff6c072610 msgr2=0x7eff6c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff96800640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7eff6c072610 0x7eff6c074ad0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7eff80005d90 tx=0x7eff80005ca0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff96800640 1 -- 172.21.15.16:0/2493526581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff900b5050 msgr2=0x7eff900c8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff96800640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff900b5050 0x7eff900c8a60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7eff7800d9b0 tx=0x7eff7800de70 comp rx=0 tx=0).stop 2024-09-10T21:16:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff96800640 1 -- 172.21.15.16:0/2493526581 shutdown_connections 2024-09-10T21:16:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff96800640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7eff6c072610 0x7eff6c074ad0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff96800640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff9015ced0 0x7eff900b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff96800640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff900b5050 0x7eff900c8a60 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff96800640 1 --2- 172.21.15.16:0/2493526581 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7eff9006e640 0x7eff900c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff96800640 1 -- 172.21.15.16:0/2493526581 >> 172.21.15.16:0/2493526581 conn(0x7eff900cfb20 msgr2=0x7eff9015eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff96800640 1 -- 172.21.15.16:0/2493526581 shutdown_connections 2024-09-10T21:16:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.150+0000 7eff96800640 1 -- 172.21.15.16:0/2493526581 wait complete. 2024-09-10T21:16:26.174 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T21:16:26.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:26.175 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T21:16:26.205 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:16:26.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:26.205 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T21:16:26.261 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:16:26.262 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:16:26.262 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:26.271 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:16:26.271 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:16:26.271 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:16:26.271 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:26.316 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:16:26.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:16:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.458+0000 7f3ea4e00640 1 -- 172.21.15.16:0/1885176101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e840039f0 msgr2=0x7f3e84003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.458+0000 7f3ea4e00640 1 --2- 172.21.15.16:0/1885176101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e840039f0 0x7f3e84003dd0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009b90 tx=0x7f3e9001d270 comp rx=0 tx=0).stop 2024-09-10T21:16:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.458+0000 7f3e9e800640 1 -- 172.21.15.16:0/1885176101 reap_dead start 2024-09-10T21:16:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.458+0000 7f3ea4e00640 1 -- 172.21.15.16:0/1885176101 shutdown_connections 2024-09-10T21:16:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.458+0000 7f3ea4e00640 1 -- 172.21.15.16:0/1885176101 >> 172.21.15.16:0/1885176101 conn(0x7f3ea00cfb20 msgr2=0x7f3ea00cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.458+0000 7f3ea4e00640 1 -- 172.21.15.16:0/1885176101 shutdown_connections 2024-09-10T21:16:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.458+0000 7f3ea4e00640 1 -- 172.21.15.16:0/1885176101 wait complete. 2024-09-10T21:16:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3ea4e00640 1 Processor -- start 2024-09-10T21:16:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3ea4e00640 1 -- start start 2024-09-10T21:16:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3ea4e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e840039f0 0x7f3ea0157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3ea4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ea00b0c30 0x7f3ea015aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3ea4e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ea00b10b0 0x7f3ea015b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3ea4e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0072f50 con 0x7f3ea00b0c30 2024-09-10T21:16:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3ea4e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0072dd0 con 0x7f3e840039f0 2024-09-10T21:16:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3ea4e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea01681d0 con 0x7f3ea00b10b0 2024-09-10T21:16:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e97e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ea00b0c30 0x7f3ea015aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e97e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ea00b0c30 0x7f3ea015aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56790/0 (socket says 172.21.15.16:56790) 2024-09-10T21:16:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e97e00640 1 -- 172.21.15.16:0/3372213684 learned_addr learned my addr 172.21.15.16:0/3372213684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e9de00640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e840039f0 0x7f3ea0157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e9e800640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ea00b10b0 0x7f3ea015b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e97e00640 1 -- 172.21.15.16:0/3372213684 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ea00b10b0 msgr2=0x7f3ea015b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e97e00640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ea00b10b0 0x7f3ea015b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e97e00640 1 -- 172.21.15.16:0/3372213684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e840039f0 msgr2=0x7f3ea0157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e97e00640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e840039f0 0x7f3ea0157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e97e00640 1 -- 172.21.15.16:0/3372213684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea0162990 con 0x7f3ea00b0c30 2024-09-10T21:16:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e9e800640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ea00b10b0 0x7f3ea015b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e97e00640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ea00b0c30 0x7f3ea015aba0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3e88004e00 tx=0x7f3e88001740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e96000640 1 -- 172.21.15.16:0/3372213684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e88015020 con 0x7f3ea00b0c30 2024-09-10T21:16:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e96000640 1 -- 172.21.15.16:0/3372213684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e88011960 con 0x7f3ea00b0c30 2024-09-10T21:16:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3ea4e00640 1 -- 172.21.15.16:0/3372213684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea01648f0 con 0x7f3ea00b0c30 2024-09-10T21:16:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3e96000640 1 -- 172.21.15.16:0/3372213684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e88011d90 con 0x7f3ea00b0c30 2024-09-10T21:16:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.462+0000 7f3ea4e00640 1 -- 172.21.15.16:0/3372213684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea0164eb0 con 0x7f3ea00b0c30 2024-09-10T21:16:26.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.466+0000 7f3ea4e00640 1 -- 172.21.15.16:0/3372213684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea00ca170 con 0x7f3ea00b0c30 2024-09-10T21:16:26.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.466+0000 7f3e96000640 1 -- 172.21.15.16:0/3372213684 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3e8801a400 con 0x7f3ea00b0c30 2024-09-10T21:16:26.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.466+0000 7f3e96000640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3e7c072610 0x7f3e7c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.466+0000 7f3e96000640 1 -- 172.21.15.16:0/3372213684 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f3e88093fe0 con 0x7f3ea00b0c30 2024-09-10T21:16:26.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.466+0000 7f3e9de00640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3e7c072610 0x7f3e7c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.466+0000 7f3e9de00640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3e7c072610 0x7f3e7c074ad0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3e900ae000 tx=0x7f3e9001a010 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:26.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.470+0000 7f3e96000640 1 -- 172.21.15.16:0/3372213684 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3e8801f020 con 0x7f3ea00b0c30 2024-09-10T21:16:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.678+0000 7f3ea4e00640 1 -- 172.21.15.16:0/3372213684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ea0162b60 con 0x7f3ea00b0c30 2024-09-10T21:16:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.678+0000 7f3e96000640 1 -- 172.21.15.16:0/3372213684 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5223 (secure 0 0 0) 0x7f3e8805f6f0 con 0x7f3ea00b0c30 2024-09-10T21:16:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:26.682 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":53,"btime":"2024-09-10T21:15:21:728518+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4199133238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4199133238},{"type":"v1","addr":"172.21.15.52:6835","nonce":4199133238}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2246088651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2246088651},{"type":"v1","addr":"172.21.15.16:6836","nonce":2246088651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1753951486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1753951486},{"type":"v1","addr":"172.21.15.16:6837","nonce":1753951486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:57.054843+0000","modified":"2024-09-10T21:15:21.697154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6276":{"gid":6276,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":24,"addr":"172.21.15.52:6834/943286634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":943286634},{"type":"v1","addr":"172.21.15.52:6834","nonce":943286634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6276,"qdb_cluster":[6276]},"id":6}]} 2024-09-10T21:16:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 53 2024-09-10T21:16:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.682+0000 7f3ea4e00640 1 -- 172.21.15.16:0/3372213684 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3e7c072610 msgr2=0x7f3e7c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.682+0000 7f3ea4e00640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3e7c072610 0x7f3e7c074ad0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3e900ae000 tx=0x7f3e9001a010 comp rx=0 tx=0).stop 2024-09-10T21:16:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.682+0000 7f3ea4e00640 1 -- 172.21.15.16:0/3372213684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ea00b0c30 msgr2=0x7f3ea015aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.682+0000 7f3ea4e00640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ea00b0c30 0x7f3ea015aba0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3e88004e00 tx=0x7f3e88001740 comp rx=0 tx=0).stop 2024-09-10T21:16:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.682+0000 7f3ea4e00640 1 -- 172.21.15.16:0/3372213684 shutdown_connections 2024-09-10T21:16:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.682+0000 7f3ea4e00640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3e7c072610 0x7f3e7c074ad0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.682+0000 7f3ea4e00640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3ea00b10b0 0x7f3ea015b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.682+0000 7f3ea4e00640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ea00b0c30 0x7f3ea015aba0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.682+0000 7f3ea4e00640 1 --2- 172.21.15.16:0/3372213684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e840039f0 0x7f3ea0157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.682+0000 7f3ea4e00640 1 -- 172.21.15.16:0/3372213684 >> 172.21.15.16:0/3372213684 conn(0x7f3ea00cfb20 msgr2=0x7f3ea00b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.682+0000 7f3ea4e00640 1 -- 172.21.15.16:0/3372213684 shutdown_connections 2024-09-10T21:16:26.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.682+0000 7f3ea4e00640 1 -- 172.21.15.16:0/3372213684 wait complete. 2024-09-10T21:16:26.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:16:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.846+0000 7fe774e00640 1 -- 172.21.15.16:0/3802426035 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe7700d0ca0 msgr2=0x7fe770168d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.846+0000 7fe774e00640 1 --2- 172.21.15.16:0/3802426035 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe7700d0ca0 0x7fe770168d60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe768009bf0 tx=0x7fe76801cad0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.846+0000 7fe774e00640 1 -- 172.21.15.16:0/3802426035 shutdown_connections 2024-09-10T21:16:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.846+0000 7fe774e00640 1 --2- 172.21.15.16:0/3802426035 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe7700d0ca0 0x7fe770168d60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.846+0000 7fe774e00640 1 -- 172.21.15.16:0/3802426035 >> 172.21.15.16:0/3802426035 conn(0x7fe77009b130 msgr2=0x7fe7700d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.846+0000 7fe774e00640 1 -- 172.21.15.16:0/3802426035 shutdown_connections 2024-09-10T21:16:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.846+0000 7fe774e00640 1 -- 172.21.15.16:0/3802426035 wait complete. 2024-09-10T21:16:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe774e00640 1 Processor -- start 2024-09-10T21:16:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe774e00640 1 -- start start 2024-09-10T21:16:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe774e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe7700b0b40 0x7fe77015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe774e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7700d0ca0 0x7fe770151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe774e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe770161620 0x7fe7701563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe774e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe77016bf60 con 0x7fe7700d0ca0 2024-09-10T21:16:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe774e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe77016bde0 con 0x7fe770161620 2024-09-10T21:16:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe774e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe77016c0e0 con 0x7fe7700b0b40 2024-09-10T21:16:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe765400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7700d0ca0 0x7fe770151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe765400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7700d0ca0 0x7fe770151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56804/0 (socket says 172.21.15.16:56804) 2024-09-10T21:16:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe765400640 1 -- 172.21.15.16:0/4267692422 learned_addr learned my addr 172.21.15.16:0/4267692422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe76e800640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe770161620 0x7fe7701563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe76de00640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe7700b0b40 0x7fe77015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe765400640 1 -- 172.21.15.16:0/4267692422 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe7700b0b40 msgr2=0x7fe77015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe765400640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe7700b0b40 0x7fe77015c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe765400640 1 -- 172.21.15.16:0/4267692422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe770161620 msgr2=0x7fe7701563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe765400640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe770161620 0x7fe7701563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe765400640 1 -- 172.21.15.16:0/4267692422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7701519d0 con 0x7fe7700d0ca0 2024-09-10T21:16:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe76e800640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe770161620 0x7fe7701563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe765400640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7700d0ca0 0x7fe770151490 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe75800c490 tx=0x7fe75800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe766a00640 1 -- 172.21.15.16:0/4267692422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580052d0 con 0x7fe7700d0ca0 2024-09-10T21:16:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe766a00640 1 -- 172.21.15.16:0/4267692422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe758005490 con 0x7fe7700d0ca0 2024-09-10T21:16:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe766a00640 1 -- 172.21.15.16:0/4267692422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758012870 con 0x7fe7700d0ca0 2024-09-10T21:16:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe774e00640 1 -- 172.21.15.16:0/4267692422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7700b36f0 con 0x7fe7700d0ca0 2024-09-10T21:16:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.850+0000 7fe774e00640 1 -- 172.21.15.16:0/4267692422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7700b3cb0 con 0x7fe7700d0ca0 2024-09-10T21:16:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.854+0000 7fe774e00640 1 -- 172.21.15.16:0/4267692422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7700b40f0 con 0x7fe7700d0ca0 2024-09-10T21:16:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.854+0000 7fe766a00640 1 -- 172.21.15.16:0/4267692422 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe758012a10 con 0x7fe7700d0ca0 2024-09-10T21:16:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.854+0000 7fe766a00640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe74c0725e0 0x7fe74c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.854+0000 7fe766a00640 1 -- 172.21.15.16:0/4267692422 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fe758092df0 con 0x7fe7700d0ca0 2024-09-10T21:16:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.854+0000 7fe76de00640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe74c0725e0 0x7fe74c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.854+0000 7fe76de00640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe74c0725e0 0x7fe74c074aa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe760000fc0 tx=0x7fe760009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:26.858+0000 7fe766a00640 1 -- 172.21.15.16:0/4267692422 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe75805e4c0 con 0x7fe7700d0ca0 2024-09-10T21:16:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.066+0000 7fe774e00640 1 -- 172.21.15.16:0/4267692422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7700d0590 con 0x7fe7700d0ca0 2024-09-10T21:16:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.066+0000 7fe766a00640 1 -- 172.21.15.16:0/4267692422 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5223 (secure 0 0 0) 0x7fe75805ddc0 con 0x7fe7700d0ca0 2024-09-10T21:16:27.072 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:27.072 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":53,"btime":"2024-09-10T21:15:21:728518+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4199133238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4199133238},{"type":"v1","addr":"172.21.15.52:6835","nonce":4199133238}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2246088651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2246088651},{"type":"v1","addr":"172.21.15.16:6836","nonce":2246088651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1753951486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1753951486},{"type":"v1","addr":"172.21.15.16:6837","nonce":1753951486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:57.054843+0000","modified":"2024-09-10T21:15:21.697154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6276":{"gid":6276,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":24,"addr":"172.21.15.52:6834/943286634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":943286634},{"type":"v1","addr":"172.21.15.52:6834","nonce":943286634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6276,"qdb_cluster":[6276]},"id":6}]} 2024-09-10T21:16:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 53 2024-09-10T21:16:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.070+0000 7fe774e00640 1 -- 172.21.15.16:0/4267692422 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe74c0725e0 msgr2=0x7fe74c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.070+0000 7fe774e00640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe74c0725e0 0x7fe74c074aa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe760000fc0 tx=0x7fe760009040 comp rx=0 tx=0).stop 2024-09-10T21:16:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.070+0000 7fe774e00640 1 -- 172.21.15.16:0/4267692422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7700d0ca0 msgr2=0x7fe770151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.070+0000 7fe774e00640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7700d0ca0 0x7fe770151490 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe75800c490 tx=0x7fe75800c950 comp rx=0 tx=0).stop 2024-09-10T21:16:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.070+0000 7fe774e00640 1 -- 172.21.15.16:0/4267692422 shutdown_connections 2024-09-10T21:16:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.070+0000 7fe774e00640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe74c0725e0 0x7fe74c074aa0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.070+0000 7fe774e00640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe770161620 0x7fe7701563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.070+0000 7fe774e00640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7700d0ca0 0x7fe770151490 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.070+0000 7fe774e00640 1 --2- 172.21.15.16:0/4267692422 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe7700b0b40 0x7fe77015c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.070+0000 7fe774e00640 1 -- 172.21.15.16:0/4267692422 >> 172.21.15.16:0/4267692422 conn(0x7fe77009b130 msgr2=0x7fe77006de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.070+0000 7fe774e00640 1 -- 172.21.15.16:0/4267692422 shutdown_connections 2024-09-10T21:16:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.070+0000 7fe774e00640 1 -- 172.21.15.16:0/4267692422 wait complete. 2024-09-10T21:16:27.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:16:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 -- 172.21.15.16:0/2982015233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f00b3200 msgr2=0x7f30f00d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 --2- 172.21.15.16:0/2982015233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f00b3200 0x7f30f00d01c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009900 tx=0x7f30dc01f070 comp rx=0 tx=0).stop 2024-09-10T21:16:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 -- 172.21.15.16:0/2982015233 shutdown_connections 2024-09-10T21:16:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 --2- 172.21.15.16:0/2982015233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f00b3200 0x7f30f00d01c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 --2- 172.21.15.16:0/2982015233 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f30f015f450 0x7f30f00cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 -- 172.21.15.16:0/2982015233 >> 172.21.15.16:0/2982015233 conn(0x7f30f00d1d80 msgr2=0x7f30f00d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 -- 172.21.15.16:0/2982015233 shutdown_connections 2024-09-10T21:16:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 -- 172.21.15.16:0/2982015233 wait complete. 2024-09-10T21:16:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 Processor -- start 2024-09-10T21:16:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 -- start start 2024-09-10T21:16:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f30f00d0890 0x7f30f01566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f015f450 0x7f30f0155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f00ce2a0 0x7f30f00cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30f00b2e30 con 0x7f30f00ce2a0 2024-09-10T21:16:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f30f00b2cb0 con 0x7f30f015f450 2024-09-10T21:16:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30f6200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f30f00b2fb0 con 0x7f30f00d0890 2024-09-10T21:16:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f00ce2a0 0x7f30f00cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f00ce2a0 0x7f30f00cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56830/0 (socket says 172.21.15.16:56830) 2024-09-10T21:16:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30efe00640 1 -- 172.21.15.16:0/832858901 learned_addr learned my addr 172.21.15.16:0/832858901 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30ef400640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f30f00d0890 0x7f30f01566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30eea00640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f015f450 0x7f30f0155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30efe00640 1 -- 172.21.15.16:0/832858901 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f30f00d0890 msgr2=0x7f30f01566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30efe00640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f30f00d0890 0x7f30f01566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30efe00640 1 -- 172.21.15.16:0/832858901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f015f450 msgr2=0x7f30f0155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30efe00640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f015f450 0x7f30f0155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.238+0000 7f30efe00640 1 -- 172.21.15.16:0/832858901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f00ce7d0 con 0x7f30f00ce2a0 2024-09-10T21:16:27.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.242+0000 7f30efe00640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f00ce2a0 0x7f30f00cae60 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f30e800c7a0 tx=0x7f30e800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:27.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.242+0000 7f30e7400640 1 -- 172.21.15.16:0/832858901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8003ba0 con 0x7f30f00ce2a0 2024-09-10T21:16:27.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.242+0000 7f30e7400640 1 -- 172.21.15.16:0/832858901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30e8003d60 con 0x7f30f00ce2a0 2024-09-10T21:16:27.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.242+0000 7f30e7400640 1 -- 172.21.15.16:0/832858901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e800aa50 con 0x7f30f00ce2a0 2024-09-10T21:16:27.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.242+0000 7f30f6200640 1 -- 172.21.15.16:0/832858901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f00ceab0 con 0x7f30f00ce2a0 2024-09-10T21:16:27.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.242+0000 7f30f6200640 1 -- 172.21.15.16:0/832858901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f00c5950 con 0x7f30f00ce2a0 2024-09-10T21:16:27.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.242+0000 7f30f6200640 1 -- 172.21.15.16:0/832858901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f00b1e90 con 0x7f30f00ce2a0 2024-09-10T21:16:27.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.242+0000 7f30e7400640 1 -- 172.21.15.16:0/832858901 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f30e800abf0 con 0x7f30f00ce2a0 2024-09-10T21:16:27.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.242+0000 7f30e7400640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f30d00725e0 0x7f30d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:27.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.242+0000 7f30e7400640 1 -- 172.21.15.16:0/832858901 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f30e8092fc0 con 0x7f30f00ce2a0 2024-09-10T21:16:27.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.242+0000 7f30ef400640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f30d00725e0 0x7f30d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:27.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.246+0000 7f30ef400640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f30d00725e0 0x7f30d0074aa0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f30e000bb50 tx=0x7f30e000b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.246+0000 7f30e7400640 1 -- 172.21.15.16:0/832858901 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f30e805e040 con 0x7f30f00ce2a0 2024-09-10T21:16:27.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.422+0000 7f30f6200640 1 -- 172.21.15.16:0/832858901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30f006fce0 con 0x7f30f00ce2a0 2024-09-10T21:16:27.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.422+0000 7f30e7400640 1 -- 172.21.15.16:0/832858901 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16684 (secure 0 0 0) 0x7f30e8062a70 con 0x7f30f00ce2a0 2024-09-10T21:16:27.428 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:27.428 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:13:58.647363+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:13:54.810241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T21:13:56.024501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:16:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.426+0000 7f30f6200640 1 -- 172.21.15.16:0/832858901 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f30d00725e0 msgr2=0x7f30d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.426+0000 7f30f6200640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f30d00725e0 0x7f30d0074aa0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f30e000bb50 tx=0x7f30e000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.426+0000 7f30f6200640 1 -- 172.21.15.16:0/832858901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f00ce2a0 msgr2=0x7f30f00cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.426+0000 7f30f6200640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f00ce2a0 0x7f30f00cae60 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f30e800c7a0 tx=0x7f30e800cc60 comp rx=0 tx=0).stop 2024-09-10T21:16:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.426+0000 7f30f6200640 1 -- 172.21.15.16:0/832858901 shutdown_connections 2024-09-10T21:16:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.426+0000 7f30f6200640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f30d00725e0 0x7f30d0074aa0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.426+0000 7f30f6200640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30f00ce2a0 0x7f30f00cae60 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.426+0000 7f30f6200640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f015f450 0x7f30f0155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.426+0000 7f30f6200640 1 --2- 172.21.15.16:0/832858901 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f30f00d0890 0x7f30f01566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.426+0000 7f30f6200640 1 -- 172.21.15.16:0/832858901 >> 172.21.15.16:0/832858901 conn(0x7f30f00d1d80 msgr2=0x7f30f0071d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.426+0000 7f30f6200640 1 -- 172.21.15.16:0/832858901 shutdown_connections 2024-09-10T21:16:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.426+0000 7f30f6200640 1 -- 172.21.15.16:0/832858901 wait complete. 2024-09-10T21:16:27.450 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:16:27.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:16:27.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 -- 172.21.15.16:0/2078733469 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415a270 msgr2=0x7f1de406e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 --2- 172.21.15.16:0/2078733469 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415a270 0x7f1de406e640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009870 tx=0x7f1dcc01c7d0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 -- 172.21.15.16:0/2078733469 shutdown_connections 2024-09-10T21:16:27.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 --2- 172.21.15.16:0/2078733469 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415a270 0x7f1de406e640 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 -- 172.21.15.16:0/2078733469 >> 172.21.15.16:0/2078733469 conn(0x7f1de40cfb20 msgr2=0x7f1de40cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 -- 172.21.15.16:0/2078733469 shutdown_connections 2024-09-10T21:16:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 -- 172.21.15.16:0/2078733469 wait complete. 2024-09-10T21:16:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 Processor -- start 2024-09-10T21:16:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 -- start start 2024-09-10T21:16:27.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1de406eb80 0x7f1de40b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:27.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40d06b0 0x7f1de40c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:27.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415a270 0x7f1de40c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:27.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1de4169310 con 0x7f1de40d06b0 2024-09-10T21:16:27.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1de4072f50 con 0x7f1de406eb80 2024-09-10T21:16:27.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.594+0000 7f1de8e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f1de4169490 con 0x7f1de415a270 2024-09-10T21:16:27.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1ddbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40d06b0 0x7f1de40c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:27.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1ddbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40d06b0 0x7f1de40c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56832/0 (socket says 172.21.15.16:56832) 2024-09-10T21:16:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1ddbe00640 1 -- 172.21.15.16:0/718393730 learned_addr learned my addr 172.21.15.16:0/718393730 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1de1e00640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1de406eb80 0x7f1de40b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1ddbe00640 1 -- 172.21.15.16:0/718393730 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415a270 msgr2=0x7f1de40c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1de2800640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415a270 0x7f1de40c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1ddbe00640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415a270 0x7f1de40c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1ddbe00640 1 -- 172.21.15.16:0/718393730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1de406eb80 msgr2=0x7f1de40b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1ddbe00640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1de406eb80 0x7f1de40b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1ddbe00640 1 -- 172.21.15.16:0/718393730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de4074220 con 0x7f1de40d06b0 2024-09-10T21:16:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1de1e00640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1de406eb80 0x7f1de40b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1de2800640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415a270 0x7f1de40c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1ddbe00640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40d06b0 0x7f1de40c83c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc000c10 tx=0x7f1dcc0a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1dda000640 1 -- 172.21.15.16:0/718393730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc0b2040 con 0x7f1de40d06b0 2024-09-10T21:16:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1dda000640 1 -- 172.21.15.16:0/718393730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dcc004470 con 0x7f1de40d06b0 2024-09-10T21:16:27.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1de8e00640 1 -- 172.21.15.16:0/718393730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de41643e0 con 0x7f1de40d06b0 2024-09-10T21:16:27.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1dda000640 1 -- 172.21.15.16:0/718393730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc005280 con 0x7f1de40d06b0 2024-09-10T21:16:27.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1de8e00640 1 -- 172.21.15.16:0/718393730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de4164920 con 0x7f1de40d06b0 2024-09-10T21:16:27.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1de8e00640 1 -- 172.21.15.16:0/718393730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de406e640 con 0x7f1de40d06b0 2024-09-10T21:16:27.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.598+0000 7f1dda000640 1 -- 172.21.15.16:0/718393730 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1dcc0b5070 con 0x7f1de40d06b0 2024-09-10T21:16:27.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.602+0000 7f1dda000640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1dc0072610 0x7f1dc0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:27.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.602+0000 7f1dda000640 1 -- 172.21.15.16:0/718393730 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f1dcc12c9b0 con 0x7f1de40d06b0 2024-09-10T21:16:27.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.602+0000 7f1de1e00640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1dc0072610 0x7f1dc0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:27.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.602+0000 7f1de1e00640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1dc0072610 0x7f1dc0074ad0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4001230 tx=0x7f1dd4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.606+0000 7f1dda000640 1 -- 172.21.15.16:0/718393730 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1dcc0bc060 con 0x7f1de40d06b0 2024-09-10T21:16:27.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.814+0000 7f1de8e00640 1 -- 172.21.15.16:0/718393730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1de40b5aa0 con 0x7f1de40d06b0 2024-09-10T21:16:27.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.814+0000 7f1dda000640 1 -- 172.21.15.16:0/718393730 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5223 (secure 0 0 0) 0x7f1dcc016450 con 0x7f1de40d06b0 2024-09-10T21:16:27.818 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:27.818 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":53,"btime":"2024-09-10T21:15:21:728518+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4199133238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4199133238},{"type":"v1","addr":"172.21.15.52:6835","nonce":4199133238}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2246088651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2246088651},{"type":"v1","addr":"172.21.15.16:6836","nonce":2246088651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1753951486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1753951486},{"type":"v1","addr":"172.21.15.16:6837","nonce":1753951486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:57.054843+0000","modified":"2024-09-10T21:15:21.697154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6276":{"gid":6276,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":24,"addr":"172.21.15.52:6834/943286634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":943286634},{"type":"v1","addr":"172.21.15.52:6834","nonce":943286634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6276,"qdb_cluster":[6276]},"id":6}]} 2024-09-10T21:16:27.818 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 53 2024-09-10T21:16:27.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.818+0000 7f1de8e00640 1 -- 172.21.15.16:0/718393730 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1dc0072610 msgr2=0x7f1dc0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.818+0000 7f1de8e00640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1dc0072610 0x7f1dc0074ad0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4001230 tx=0x7f1dd4009040 comp rx=0 tx=0).stop 2024-09-10T21:16:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.818+0000 7f1de8e00640 1 -- 172.21.15.16:0/718393730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40d06b0 msgr2=0x7f1de40c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.818+0000 7f1de8e00640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40d06b0 0x7f1de40c83c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc000c10 tx=0x7f1dcc0a5710 comp rx=0 tx=0).stop 2024-09-10T21:16:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.818+0000 7f1de8e00640 1 -- 172.21.15.16:0/718393730 shutdown_connections 2024-09-10T21:16:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.818+0000 7f1de8e00640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1dc0072610 0x7f1dc0074ad0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.818+0000 7f1de8e00640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415a270 0x7f1de40c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.818+0000 7f1de8e00640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40d06b0 0x7f1de40c83c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.818+0000 7f1de8e00640 1 --2- 172.21.15.16:0/718393730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1de406eb80 0x7f1de40b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.818+0000 7f1de8e00640 1 -- 172.21.15.16:0/718393730 >> 172.21.15.16:0/718393730 conn(0x7f1de40cfb20 msgr2=0x7f1de4154d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.818+0000 7f1de8e00640 1 -- 172.21.15.16:0/718393730 shutdown_connections 2024-09-10T21:16:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.818+0000 7f1de8e00640 1 -- 172.21.15.16:0/718393730 wait complete. 2024-09-10T21:16:27.840 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:16:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.982+0000 7f3c1aa00640 1 -- 172.21.15.16:0/162486552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c140b13c0 msgr2=0x7f3c140afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.982+0000 7f3c1aa00640 1 --2- 172.21.15.16:0/162486552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c140b13c0 0x7f3c140afc90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009bc0 tx=0x7f3c0c01ca80 comp rx=0 tx=0).stop 2024-09-10T21:16:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.982+0000 7f3c1aa00640 1 -- 172.21.15.16:0/162486552 shutdown_connections 2024-09-10T21:16:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.982+0000 7f3c1aa00640 1 --2- 172.21.15.16:0/162486552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c140b13c0 0x7f3c140afc90 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.982+0000 7f3c1aa00640 1 -- 172.21.15.16:0/162486552 >> 172.21.15.16:0/162486552 conn(0x7f3c1409b130 msgr2=0x7f3c140d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.982+0000 7f3c1aa00640 1 -- 172.21.15.16:0/162486552 shutdown_connections 2024-09-10T21:16:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.982+0000 7f3c1aa00640 1 -- 172.21.15.16:0/162486552 wait complete. 2024-09-10T21:16:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c1aa00640 1 Processor -- start 2024-09-10T21:16:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c1aa00640 1 -- start start 2024-09-10T21:16:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c1aa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3c140716f0 0x7f3c14152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c1aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c140b13c0 0x7f3c14157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c1aa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c1415f400 0x7f3c1415c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c1aa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14169f70 con 0x7f3c140b13c0 2024-09-10T21:16:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c1aa00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14169df0 con 0x7f3c1415f400 2024-09-10T21:16:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c1aa00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1416a0f0 con 0x7f3c140716f0 2024-09-10T21:16:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c0b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c140b13c0 0x7f3c14157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c0b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c140b13c0 0x7f3c14157290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56836/0 (socket says 172.21.15.16:56836) 2024-09-10T21:16:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c0b400640 1 -- 172.21.15.16:0/3695206687 learned_addr learned my addr 172.21.15.16:0/3695206687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c18a00640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c1415f400 0x7f3c1415c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c13e00640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3c140716f0 0x7f3c14152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c0b400640 1 -- 172.21.15.16:0/3695206687 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3c140716f0 msgr2=0x7f3c14152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c0b400640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3c140716f0 0x7f3c14152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c0b400640 1 -- 172.21.15.16:0/3695206687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c1415f400 msgr2=0x7f3c1415c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c0b400640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c1415f400 0x7f3c1415c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c0b400640 1 -- 172.21.15.16:0/3695206687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c14158230 con 0x7f3c140b13c0 2024-09-10T21:16:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c0b400640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c140b13c0 0x7f3c14157290 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00c4e0 tx=0x7f3bfc00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c10c00640 1 -- 172.21.15.16:0/3695206687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc0052d0 con 0x7f3c140b13c0 2024-09-10T21:16:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c10c00640 1 -- 172.21.15.16:0/3695206687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bfc005470 con 0x7f3c140b13c0 2024-09-10T21:16:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c10c00640 1 -- 172.21.15.16:0/3695206687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc012890 con 0x7f3c140b13c0 2024-09-10T21:16:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c1aa00640 1 -- 172.21.15.16:0/3695206687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c14157c70 con 0x7f3c140b13c0 2024-09-10T21:16:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.986+0000 7f3c1aa00640 1 -- 172.21.15.16:0/3695206687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1415bad0 con 0x7f3c140b13c0 2024-09-10T21:16:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.990+0000 7f3c1aa00640 1 -- 172.21.15.16:0/3695206687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c14070fe0 con 0x7f3c140b13c0 2024-09-10T21:16:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.990+0000 7f3c10c00640 1 -- 172.21.15.16:0/3695206687 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3bfc003600 con 0x7f3c140b13c0 2024-09-10T21:16:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.990+0000 7f3c10c00640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3bf00725e0 0x7f3bf0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.990+0000 7f3c10c00640 1 -- 172.21.15.16:0/3695206687 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f3bfc092d10 con 0x7f3c140b13c0 2024-09-10T21:16:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.990+0000 7f3c13e00640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3bf00725e0 0x7f3bf0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.990+0000 7f3c13e00640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3bf00725e0 0x7f3bf0074aa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3c040010a0 tx=0x7f3c04009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:27.994+0000 7f3c10c00640 1 -- 172.21.15.16:0/3695206687 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3bfc098020 con 0x7f3c140b13c0 2024-09-10T21:16:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.202+0000 7f3c1aa00640 1 -- 172.21.15.16:0/3695206687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c14159b60 con 0x7f3c140b13c0 2024-09-10T21:16:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.202+0000 7f3c10c00640 1 -- 172.21.15.16:0/3695206687 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5223 (secure 0 0 0) 0x7f3bfc05dd00 con 0x7f3c140b13c0 2024-09-10T21:16:28.207 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:28.208 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":53,"btime":"2024-09-10T21:15:21:728518+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4199133238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4199133238},{"type":"v1","addr":"172.21.15.52:6835","nonce":4199133238}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2246088651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2246088651},{"type":"v1","addr":"172.21.15.16:6836","nonce":2246088651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1753951486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1753951486},{"type":"v1","addr":"172.21.15.16:6837","nonce":1753951486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:13:57.054843+0000","modified":"2024-09-10T21:15:21.697154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6276":{"gid":6276,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":24,"addr":"172.21.15.52:6834/943286634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":943286634},{"type":"v1","addr":"172.21.15.52:6834","nonce":943286634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6276,"qdb_cluster":[6276]},"id":6}]} 2024-09-10T21:16:28.208 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 53 2024-09-10T21:16:28.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.206+0000 7f3c1aa00640 1 -- 172.21.15.16:0/3695206687 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3bf00725e0 msgr2=0x7f3bf0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.206+0000 7f3c1aa00640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3bf00725e0 0x7f3bf0074aa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3c040010a0 tx=0x7f3c04009040 comp rx=0 tx=0).stop 2024-09-10T21:16:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.206+0000 7f3c1aa00640 1 -- 172.21.15.16:0/3695206687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c140b13c0 msgr2=0x7f3c14157290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.206+0000 7f3c1aa00640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c140b13c0 0x7f3c14157290 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00c4e0 tx=0x7f3bfc00c9a0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.206+0000 7f3c1aa00640 1 -- 172.21.15.16:0/3695206687 shutdown_connections 2024-09-10T21:16:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.206+0000 7f3c1aa00640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3bf00725e0 0x7f3bf0074aa0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.206+0000 7f3c1aa00640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c1415f400 0x7f3c1415c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.206+0000 7f3c1aa00640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c140b13c0 0x7f3c14157290 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.206+0000 7f3c1aa00640 1 --2- 172.21.15.16:0/3695206687 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3c140716f0 0x7f3c14152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.206+0000 7f3c1aa00640 1 -- 172.21.15.16:0/3695206687 >> 172.21.15.16:0/3695206687 conn(0x7f3c1409b130 msgr2=0x7f3c1415a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.206+0000 7f3c1aa00640 1 -- 172.21.15.16:0/3695206687 shutdown_connections 2024-09-10T21:16:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.206+0000 7f3c1aa00640 1 -- 172.21.15.16:0/3695206687 wait complete. 2024-09-10T21:16:28.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:16:28.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 -- 172.21.15.16:0/2887068894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66dc0aed30 msgr2=0x7f66dc0af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:28.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 --2- 172.21.15.16:0/2887068894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66dc0aed30 0x7f66dc0af1e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009c80 tx=0x7f66d401cce0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 -- 172.21.15.16:0/2887068894 shutdown_connections 2024-09-10T21:16:28.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 --2- 172.21.15.16:0/2887068894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66dc0aed30 0x7f66dc0af1e0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 --2- 172.21.15.16:0/2887068894 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66dc0cd130 0x7f66dc0d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 -- 172.21.15.16:0/2887068894 >> 172.21.15.16:0/2887068894 conn(0x7f66dc0d1d80 msgr2=0x7f66dc0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:28.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 -- 172.21.15.16:0/2887068894 shutdown_connections 2024-09-10T21:16:28.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 -- 172.21.15.16:0/2887068894 wait complete. 2024-09-10T21:16:28.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 Processor -- start 2024-09-10T21:16:28.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 -- start start 2024-09-10T21:16:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66dc0aed30 0x7f66dc151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66dc0cd130 0x7f66dc0719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66dc15f350 0x7f66dc071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc168eb0 con 0x7f66dc0aed30 2024-09-10T21:16:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc168d30 con 0x7f66dc15f350 2024-09-10T21:16:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.378+0000 7f66e2600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f66dc169030 con 0x7f66dc0cd130 2024-09-10T21:16:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66db400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66dc0aed30 0x7f66dc151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66db400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66dc0aed30 0x7f66dc151dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56846/0 (socket says 172.21.15.16:56846) 2024-09-10T21:16:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66db400640 1 -- 172.21.15.16:0/1456817250 learned_addr learned my addr 172.21.15.16:0/1456817250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66dbe00640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66dc15f350 0x7f66dc071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66d3e00640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66dc0cd130 0x7f66dc0719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66db400640 1 -- 172.21.15.16:0/1456817250 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66dc0cd130 msgr2=0x7f66dc0719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66db400640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66dc0cd130 0x7f66dc0719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66db400640 1 -- 172.21.15.16:0/1456817250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66dc15f350 msgr2=0x7f66dc071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66db400640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66dc15f350 0x7f66dc071ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66db400640 1 -- 172.21.15.16:0/1456817250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc152300 con 0x7f66dc0aed30 2024-09-10T21:16:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66dbe00640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66dc15f350 0x7f66dc071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66db400640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66dc0aed30 0x7f66dc151dc0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00b410 tx=0x7f66cc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66d3400640 1 -- 172.21.15.16:0/1456817250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc015030 con 0x7f66dc0aed30 2024-09-10T21:16:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66d3400640 1 -- 172.21.15.16:0/1456817250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66cc004020 con 0x7f66dc0aed30 2024-09-10T21:16:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66d3400640 1 -- 172.21.15.16:0/1456817250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc004470 con 0x7f66dc0aed30 2024-09-10T21:16:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66e2600640 1 -- 172.21.15.16:0/1456817250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66dc1527b0 con 0x7f66dc0aed30 2024-09-10T21:16:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66e2600640 1 -- 172.21.15.16:0/1456817250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66dc1568b0 con 0x7f66dc0aed30 2024-09-10T21:16:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66e2600640 1 -- 172.21.15.16:0/1456817250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66dc0af1e0 con 0x7f66dc0aed30 2024-09-10T21:16:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66d3400640 1 -- 172.21.15.16:0/1456817250 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f66cc022060 con 0x7f66dc0aed30 2024-09-10T21:16:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.382+0000 7f66d3400640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66b80725e0 0x7f66b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.386+0000 7f66d3400640 1 -- 172.21.15.16:0/1456817250 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f66cc092f10 con 0x7f66dc0aed30 2024-09-10T21:16:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.386+0000 7f66d3e00640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66b80725e0 0x7f66b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.386+0000 7f66d3e00640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66b80725e0 0x7f66b8074aa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f66c40058d0 tx=0x7f66c4005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.390+0000 7f66d3400640 1 -- 172.21.15.16:0/1456817250 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f66cc05e550 con 0x7f66dc0aed30 2024-09-10T21:16:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.562+0000 7f66e2600640 1 -- 172.21.15.16:0/1456817250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66dc0ccf10 con 0x7f66dc0aed30 2024-09-10T21:16:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.562+0000 7f66d3400640 1 -- 172.21.15.16:0/1456817250 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16684 (secure 0 0 0) 0x7f66cc05de50 con 0x7f66dc0aed30 2024-09-10T21:16:28.568 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:28.568 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:13:58.647363+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:13:54.810241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T21:13:56.024501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:16:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.566+0000 7f66e2600640 1 -- 172.21.15.16:0/1456817250 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66b80725e0 msgr2=0x7f66b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.566+0000 7f66e2600640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66b80725e0 0x7f66b8074aa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f66c40058d0 tx=0x7f66c4005860 comp rx=0 tx=0).stop 2024-09-10T21:16:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.566+0000 7f66e2600640 1 -- 172.21.15.16:0/1456817250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66dc0aed30 msgr2=0x7f66dc151dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.566+0000 7f66e2600640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66dc0aed30 0x7f66dc151dc0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00b410 tx=0x7f66cc00b8d0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.566+0000 7f66e2600640 1 -- 172.21.15.16:0/1456817250 shutdown_connections 2024-09-10T21:16:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.566+0000 7f66e2600640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66b80725e0 0x7f66b8074aa0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.566+0000 7f66e2600640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66dc15f350 0x7f66dc071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.566+0000 7f66e2600640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66dc0cd130 0x7f66dc0719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.566+0000 7f66e2600640 1 --2- 172.21.15.16:0/1456817250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66dc0aed30 0x7f66dc151dc0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.566+0000 7f66e2600640 1 -- 172.21.15.16:0/1456817250 >> 172.21.15.16:0/1456817250 conn(0x7f66dc0d1d80 msgr2=0x7f66dc0b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.566+0000 7f66e2600640 1 -- 172.21.15.16:0/1456817250 shutdown_connections 2024-09-10T21:16:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.566+0000 7f66e2600640 1 -- 172.21.15.16:0/1456817250 wait complete. 2024-09-10T21:16:28.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:16:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.734+0000 7f4a2fe00640 1 -- 172.21.15.16:0/1101744296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a14003ad0 msgr2=0x7f4a14003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.734+0000 7f4a2fe00640 1 --2- 172.21.15.16:0/1101744296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a14003ad0 0x7f4a14003ef0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009b60 tx=0x7f4a2001d270 comp rx=0 tx=0).stop 2024-09-10T21:16:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.734+0000 7f4a2f400640 1 -- 172.21.15.16:0/1101744296 reap_dead start 2024-09-10T21:16:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.734+0000 7f4a2fe00640 1 -- 172.21.15.16:0/1101744296 shutdown_connections 2024-09-10T21:16:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.734+0000 7f4a2fe00640 1 -- 172.21.15.16:0/1101744296 >> 172.21.15.16:0/1101744296 conn(0x7f4a300b1df0 msgr2=0x7f4a300b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.734+0000 7f4a2fe00640 1 -- 172.21.15.16:0/1101744296 shutdown_connections 2024-09-10T21:16:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.734+0000 7f4a2fe00640 1 -- 172.21.15.16:0/1101744296 wait complete. 2024-09-10T21:16:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2fe00640 1 Processor -- start 2024-09-10T21:16:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2fe00640 1 -- start start 2024-09-10T21:16:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a14003ad0 0x7f4a300c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a30159ef0 0x7f4a300ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2fe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4a3015a370 0x7f4a300aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a300cf060 con 0x7f4a14003ad0 2024-09-10T21:16:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2fe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4a300ceee0 con 0x7f4a30159ef0 2024-09-10T21:16:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2fe00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4a300cc400 con 0x7f4a3015a370 2024-09-10T21:16:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a14003ad0 0x7f4a300c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a14003ad0 0x7f4a300c6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56858/0 (socket says 172.21.15.16:56858) 2024-09-10T21:16:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2ea00640 1 -- 172.21.15.16:0/476715868 learned_addr learned my addr 172.21.15.16:0/476715868 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a27e00640 1 --2- 172.21.15.16:0/476715868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a30159ef0 0x7f4a300ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2f400640 1 --2- 172.21.15.16:0/476715868 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4a3015a370 0x7f4a300aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2ea00640 1 -- 172.21.15.16:0/476715868 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4a3015a370 msgr2=0x7f4a300aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2ea00640 1 --2- 172.21.15.16:0/476715868 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4a3015a370 0x7f4a300aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2ea00640 1 -- 172.21.15.16:0/476715868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a30159ef0 msgr2=0x7f4a300ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2ea00640 1 --2- 172.21.15.16:0/476715868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a30159ef0 0x7f4a300ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2ea00640 1 -- 172.21.15.16:0/476715868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a300ad170 con 0x7f4a14003ad0 2024-09-10T21:16:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2ea00640 1 --2- 172.21.15.16:0/476715868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a14003ad0 0x7f4a300c6310 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4a2001d0a0 tx=0x7f4a20005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a26a00640 1 -- 172.21.15.16:0/476715868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a200b1040 con 0x7f4a14003ad0 2024-09-10T21:16:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a26a00640 1 -- 172.21.15.16:0/476715868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a200bb070 con 0x7f4a14003ad0 2024-09-10T21:16:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2fe00640 1 -- 172.21.15.16:0/476715868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a300ad410 con 0x7f4a14003ad0 2024-09-10T21:16:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a26a00640 1 -- 172.21.15.16:0/476715868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20004290 con 0x7f4a14003ad0 2024-09-10T21:16:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.738+0000 7f4a2fe00640 1 -- 172.21.15.16:0/476715868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a300c6850 con 0x7f4a14003ad0 2024-09-10T21:16:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.742+0000 7f4a2fe00640 1 -- 172.21.15.16:0/476715868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a300b2c00 con 0x7f4a14003ad0 2024-09-10T21:16:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.742+0000 7f4a26a00640 1 -- 172.21.15.16:0/476715868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4a200046d0 con 0x7f4a14003ad0 2024-09-10T21:16:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.742+0000 7f4a26a00640 1 --2- 172.21.15.16:0/476715868 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a0c072640 0x7f4a0c074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.742+0000 7f4a26a00640 1 -- 172.21.15.16:0/476715868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f4a2012e0d0 con 0x7f4a14003ad0 2024-09-10T21:16:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.742+0000 7f4a27e00640 1 --2- 172.21.15.16:0/476715868 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a0c072640 0x7f4a0c074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.742+0000 7f4a27e00640 1 --2- 172.21.15.16:0/476715868 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a0c072640 0x7f4a0c074b00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4a30157e00 tx=0x7f4a180091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.746+0000 7f4a26a00640 1 -- 172.21.15.16:0/476715868 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4a20016f80 con 0x7f4a14003ad0 2024-09-10T21:16:28.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.958+0000 7f4a2fe00640 1 -- 172.21.15.16:0/476715868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4a300b6280 con 0x7f4a14003ad0 2024-09-10T21:16:28.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:28.958+0000 7f4a26a00640 1 -- 172.21.15.16:0/476715868 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v53) ==== 316+0+0 (secure 0 0 0) 0x7f4a200f97a0 con 0x7f4a14003ad0 2024-09-10T21:16:28.962 INFO:teuthology.orchestra.run.smithi016.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-09-10T21:16:28.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:16:28.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-09-10T21:16:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.118+0000 7f0287a00640 1 -- 172.21.15.16:0/1244735476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02800d0210 msgr2=0x7f02800ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.118+0000 7f0287a00640 1 --2- 172.21.15.16:0/1244735476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02800d0210 0x7f02800ceca0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f026c009870 tx=0x7f026c01f010 comp rx=0 tx=0).stop 2024-09-10T21:16:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.118+0000 7f0287a00640 1 -- 172.21.15.16:0/1244735476 shutdown_connections 2024-09-10T21:16:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.118+0000 7f0287a00640 1 --2- 172.21.15.16:0/1244735476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02800d0210 0x7f02800ceca0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.118+0000 7f0287a00640 1 --2- 172.21.15.16:0/1244735476 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02800ce040 0x7f02800b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.118+0000 7f0287a00640 1 -- 172.21.15.16:0/1244735476 >> 172.21.15.16:0/1244735476 conn(0x7f02800d1df0 msgr2=0x7f02800d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.118+0000 7f0287a00640 1 -- 172.21.15.16:0/1244735476 shutdown_connections 2024-09-10T21:16:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.118+0000 7f0287a00640 1 -- 172.21.15.16:0/1244735476 wait complete. 2024-09-10T21:16:29.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0287a00640 1 Processor -- start 2024-09-10T21:16:29.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0287a00640 1 -- start start 2024-09-10T21:16:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0287a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02800ce040 0x7f0280157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0287a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02800cfe30 0x7f028015d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0287a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02800d0210 0x7f02800b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0287a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02800b1460 con 0x7f02800d0210 2024-09-10T21:16:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0287a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f02800b12e0 con 0x7f02800ce040 2024-09-10T21:16:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0287a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f02800cf1e0 con 0x7f02800cfe30 2024-09-10T21:16:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0285a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02800d0210 0x7f02800b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0285a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02800d0210 0x7f02800b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56880/0 (socket says 172.21.15.16:56880) 2024-09-10T21:16:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0285a00640 1 -- 172.21.15.16:0/169456882 learned_addr learned my addr 172.21.15.16:0/169456882 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f027fe00640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02800cfe30 0x7f028015d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0285000640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02800ce040 0x7f0280157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0285a00640 1 -- 172.21.15.16:0/169456882 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02800cfe30 msgr2=0x7f028015d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0285a00640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02800cfe30 0x7f028015d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0285a00640 1 -- 172.21.15.16:0/169456882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02800ce040 msgr2=0x7f0280157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0285a00640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02800ce040 0x7f0280157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0285a00640 1 -- 172.21.15.16:0/169456882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02801588e0 con 0x7f02800d0210 2024-09-10T21:16:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0285a00640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02800d0210 0x7f02800b6a40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f027800ea20 tx=0x7f027800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f027cc00640 1 -- 172.21.15.16:0/169456882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027800cde0 con 0x7f02800d0210 2024-09-10T21:16:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f027cc00640 1 -- 172.21.15.16:0/169456882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0278004c40 con 0x7f02800d0210 2024-09-10T21:16:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f027cc00640 1 -- 172.21.15.16:0/169456882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278005280 con 0x7f02800d0210 2024-09-10T21:16:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0287a00640 1 -- 172.21.15.16:0/169456882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028015c9d0 con 0x7f02800d0210 2024-09-10T21:16:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0287a00640 1 -- 172.21.15.16:0/169456882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028015cf90 con 0x7f02800d0210 2024-09-10T21:16:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.122+0000 7f0287a00640 1 -- 172.21.15.16:0/169456882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02800b1710 con 0x7f02800d0210 2024-09-10T21:16:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.126+0000 7f027cc00640 1 -- 172.21.15.16:0/169456882 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f027800f040 con 0x7f02800d0210 2024-09-10T21:16:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.126+0000 7f027cc00640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0260072610 0x7f0260074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.126+0000 7f027cc00640 1 -- 172.21.15.16:0/169456882 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f0278093850 con 0x7f02800d0210 2024-09-10T21:16:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.126+0000 7f0285000640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0260072610 0x7f0260074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.126+0000 7f0285000640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0260072610 0x7f0260074ad0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0274001910 tx=0x7f0274001420 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.130+0000 7f027cc00640 1 -- 172.21.15.16:0/169456882 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0278016030 con 0x7f02800d0210 2024-09-10T21:16:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.338+0000 7f0287a00640 1 -- 172.21.15.16:0/169456882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0280071170 con 0x7f02800d0210 2024-09-10T21:16:29.341 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-10T21:16:29.338+0000 7ff429400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:16:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.338+0000 7f027cc00640 1 -- 172.21.15.16:0/169456882 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) ==== 191+0+0 (secure 0 0 0) 0x7f027805e8d0 con 0x7f02800d0210 2024-09-10T21:16:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:16:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.342+0000 7f0287a00640 1 -- 172.21.15.16:0/169456882 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0260072610 msgr2=0x7f0260074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.342+0000 7f0287a00640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0260072610 0x7f0260074ad0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0274001910 tx=0x7f0274001420 comp rx=0 tx=0).stop 2024-09-10T21:16:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.342+0000 7f0287a00640 1 -- 172.21.15.16:0/169456882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02800d0210 msgr2=0x7f02800b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.342+0000 7f0287a00640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02800d0210 0x7f02800b6a40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f027800ea20 tx=0x7f027800eee0 comp rx=0 tx=0).stop 2024-09-10T21:16:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.342+0000 7f0287a00640 1 -- 172.21.15.16:0/169456882 shutdown_connections 2024-09-10T21:16:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.342+0000 7f0287a00640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0260072610 0x7f0260074ad0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.342+0000 7f0287a00640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02800d0210 0x7f02800b6a40 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.342+0000 7f0287a00640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02800cfe30 0x7f028015d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.342+0000 7f0287a00640 1 --2- 172.21.15.16:0/169456882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02800ce040 0x7f0280157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.342+0000 7f0287a00640 1 -- 172.21.15.16:0/169456882 >> 172.21.15.16:0/169456882 conn(0x7f02800d1df0 msgr2=0x7f02800aec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.342+0000 7f0287a00640 1 -- 172.21.15.16:0/169456882 shutdown_connections 2024-09-10T21:16:29.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.342+0000 7f0287a00640 1 -- 172.21.15.16:0/169456882 wait complete. 2024-09-10T21:16:29.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:16:29.414 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:16:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.514+0000 7fd4c3e00640 1 -- 172.21.15.16:0/3936248732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a40051d0 msgr2=0x7fd4a400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.514+0000 7fd4c3e00640 1 --2- 172.21.15.16:0/3936248732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a40051d0 0x7fd4a400b9d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4001dc0 tx=0x7fd4b401cb30 comp rx=0 tx=0).stop 2024-09-10T21:16:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.514+0000 7fd4c3e00640 1 -- 172.21.15.16:0/3936248732 shutdown_connections 2024-09-10T21:16:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.514+0000 7fd4c3e00640 1 --2- 172.21.15.16:0/3936248732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a40051d0 0x7fd4a400b9d0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.514+0000 7fd4c3e00640 1 -- 172.21.15.16:0/3936248732 >> 172.21.15.16:0/3936248732 conn(0x7fd4bc0b16a0 msgr2=0x7fd4bc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.514+0000 7fd4c3e00640 1 -- 172.21.15.16:0/3936248732 shutdown_connections 2024-09-10T21:16:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.514+0000 7fd4c3e00640 1 -- 172.21.15.16:0/3936248732 wait complete. 2024-09-10T21:16:29.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c3e00640 1 Processor -- start 2024-09-10T21:16:29.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c3e00640 1 -- start start 2024-09-10T21:16:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a40051d0 0x7fd4bc1484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c3e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd4bc1e9860 0x7fd4bc0cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c3e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4bc0cce30 0x7fd4bc1e8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c3e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc072830 con 0x7fd4a40051d0 2024-09-10T21:16:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c3e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc0726b0 con 0x7fd4bc0cce30 2024-09-10T21:16:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c3e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd4bc0729b0 con 0x7fd4bc1e9860 2024-09-10T21:16:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a40051d0 0x7fd4bc1484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a40051d0 0x7fd4bc1484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56904/0 (socket says 172.21.15.16:56904) 2024-09-10T21:16:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c1400640 1 -- 172.21.15.16:0/2348088142 learned_addr learned my addr 172.21.15.16:0/2348088142 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:29.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c0a00640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd4bc1e9860 0x7fd4bc0cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:29.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c1e00640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4bc0cce30 0x7fd4bc1e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:29.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c1400640 1 -- 172.21.15.16:0/2348088142 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd4bc1e9860 msgr2=0x7fd4bc0cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:29.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c1400640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd4bc1e9860 0x7fd4bc0cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:29.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c1400640 1 -- 172.21.15.16:0/2348088142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4bc0cce30 msgr2=0x7fd4bc1e8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:29.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c1400640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4bc0cce30 0x7fd4bc1e8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:29.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c1400640 1 -- 172.21.15.16:0/2348088142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4bc150540 con 0x7fd4a40051d0 2024-09-10T21:16:29.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c0a00640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd4bc1e9860 0x7fd4bc0cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c1400640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a40051d0 0x7fd4bc1484a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000eaa0 tx=0x7fd4b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4b9600640 1 -- 172.21.15.16:0/2348088142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000dd00 con 0x7fd4a40051d0 2024-09-10T21:16:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4b9600640 1 -- 172.21.15.16:0/2348088142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4b0004050 con 0x7fd4a40051d0 2024-09-10T21:16:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c3e00640 1 -- 172.21.15.16:0/2348088142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4bc1e7a60 con 0x7fd4a40051d0 2024-09-10T21:16:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4b9600640 1 -- 172.21.15.16:0/2348088142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000dea0 con 0x7fd4a40051d0 2024-09-10T21:16:29.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c3e00640 1 -- 172.21.15.16:0/2348088142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4bc1489e0 con 0x7fd4a40051d0 2024-09-10T21:16:29.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.518+0000 7fd4c3e00640 1 -- 172.21.15.16:0/2348088142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4bc15df90 con 0x7fd4a40051d0 2024-09-10T21:16:29.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.522+0000 7fd4b9600640 1 -- 172.21.15.16:0/2348088142 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd4b000b040 con 0x7fd4a40051d0 2024-09-10T21:16:29.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.522+0000 7fd4b9600640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd49c0725e0 0x7fd49c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:29.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.522+0000 7fd4b9600640 1 -- 172.21.15.16:0/2348088142 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5795+0+0 (secure 0 0 0) 0x7fd4b0093a80 con 0x7fd4a40051d0 2024-09-10T21:16:29.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.522+0000 7fd4c0a00640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd49c0725e0 0x7fd49c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:29.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.522+0000 7fd4c0a00640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd49c0725e0 0x7fd49c074aa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc150da0 tx=0x7fd4a80067b0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:29.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.526+0000 7fd4b9600640 1 -- 172.21.15.16:0/2348088142 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd4b0095e20 con 0x7fd4a40051d0 2024-09-10T21:16:29.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:29.742+0000 7fd4c3e00640 1 -- 172.21.15.16:0/2348088142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd4bc0cd210 con 0x7fd4a40051d0 2024-09-10T21:16:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.346+0000 7fd4b9600640 1 -- 172.21.15.16:0/2348088142 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) ==== 104+0+0 (secure 0 0 0) 0x7fd4b005ea20 con 0x7fd4a40051d0 2024-09-10T21:16:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.346+0000 7fd4c3e00640 1 -- 172.21.15.16:0/2348088142 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd49c0725e0 msgr2=0x7fd49c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.346+0000 7fd4c3e00640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd49c0725e0 0x7fd49c074aa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc150da0 tx=0x7fd4a80067b0 comp rx=0 tx=0).stop 2024-09-10T21:16:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.346+0000 7fd4c3e00640 1 -- 172.21.15.16:0/2348088142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a40051d0 msgr2=0x7fd4bc1484a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.346+0000 7fd4c3e00640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a40051d0 0x7fd4bc1484a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000eaa0 tx=0x7fd4b000ef60 comp rx=0 tx=0).stop 2024-09-10T21:16:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.346+0000 7fd4c3e00640 1 -- 172.21.15.16:0/2348088142 shutdown_connections 2024-09-10T21:16:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.346+0000 7fd4c3e00640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd49c0725e0 0x7fd49c074aa0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.346+0000 7fd4c3e00640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd4bc0cce30 0x7fd4bc1e8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.346+0000 7fd4c3e00640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd4bc1e9860 0x7fd4bc0cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.346+0000 7fd4c3e00640 1 --2- 172.21.15.16:0/2348088142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a40051d0 0x7fd4bc1484a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.346+0000 7fd4c3e00640 1 -- 172.21.15.16:0/2348088142 >> 172.21.15.16:0/2348088142 conn(0x7fd4bc0b16a0 msgr2=0x7fd4bc0ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.346+0000 7fd4c3e00640 1 -- 172.21.15.16:0/2348088142 shutdown_connections 2024-09-10T21:16:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.346+0000 7fd4c3e00640 1 -- 172.21.15.16:0/2348088142 wait complete. 2024-09-10T21:16:30.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:16:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.522+0000 7f89c7600640 1 -- 172.21.15.16:0/2080186283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a8004b50 msgr2=0x7f89a8004f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.522+0000 7f89c7600640 1 --2- 172.21.15.16:0/2080186283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a8004b50 0x7f89a8004f70 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f89b400c2e0 tx=0x7f89b401cf30 comp rx=0 tx=0).stop 2024-09-10T21:16:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.522+0000 7f89c5600640 1 -- 172.21.15.16:0/2080186283 reap_dead start 2024-09-10T21:16:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.522+0000 7f89c7600640 1 -- 172.21.15.16:0/2080186283 shutdown_connections 2024-09-10T21:16:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.522+0000 7f89c7600640 1 -- 172.21.15.16:0/2080186283 >> 172.21.15.16:0/2080186283 conn(0x7f89c006d1e0 msgr2=0x7f89c00b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.522+0000 7f89c7600640 1 -- 172.21.15.16:0/2080186283 shutdown_connections 2024-09-10T21:16:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.522+0000 7f89c7600640 1 -- 172.21.15.16:0/2080186283 wait complete. 2024-09-10T21:16:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c7600640 1 Processor -- start 2024-09-10T21:16:30.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c7600640 1 -- start start 2024-09-10T21:16:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c7600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a8004b50 0x7f89c014f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c7600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f89c01f7e70 0x7f89c015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c7600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c015d150 0x7f89c01e9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c7600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89c015e680 con 0x7f89a8004b50 2024-09-10T21:16:30.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c7600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89c015e500 con 0x7f89c015d150 2024-09-10T21:16:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c7600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f89c015e800 con 0x7f89c01f7e70 2024-09-10T21:16:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a8004b50 0x7f89c014f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a8004b50 0x7f89c014f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56914/0 (socket says 172.21.15.16:56914) 2024-09-10T21:16:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c4c00640 1 -- 172.21.15.16:0/1499581644 learned_addr learned my addr 172.21.15.16:0/1499581644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89bfe00640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f89c01f7e70 0x7f89c015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c5600640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c015d150 0x7f89c01e9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c4c00640 1 -- 172.21.15.16:0/1499581644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f89c01f7e70 msgr2=0x7f89c015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c4c00640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f89c01f7e70 0x7f89c015cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c4c00640 1 -- 172.21.15.16:0/1499581644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c015d150 msgr2=0x7f89c01e9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c4c00640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c015d150 0x7f89c01e9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c4c00640 1 -- 172.21.15.16:0/1499581644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c00b10c0 con 0x7f89a8004b50 2024-09-10T21:16:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c5600640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c015d150 0x7f89c01e9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89bfe00640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f89c01f7e70 0x7f89c015cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c4c00640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a8004b50 0x7f89c014f010 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f89b401c9f0 tx=0x7f89b4001d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89bcc00640 1 -- 172.21.15.16:0/1499581644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b40014c0 con 0x7f89a8004b50 2024-09-10T21:16:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89bcc00640 1 -- 172.21.15.16:0/1499581644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89b4001660 con 0x7f89a8004b50 2024-09-10T21:16:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c7600640 1 -- 172.21.15.16:0/1499581644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c00b13a0 con 0x7f89a8004b50 2024-09-10T21:16:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89bcc00640 1 -- 172.21.15.16:0/1499581644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b40064e0 con 0x7f89a8004b50 2024-09-10T21:16:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c7600640 1 -- 172.21.15.16:0/1499581644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c01f7950 con 0x7f89a8004b50 2024-09-10T21:16:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.526+0000 7f89c7600640 1 -- 172.21.15.16:0/1499581644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c015ed40 con 0x7f89a8004b50 2024-09-10T21:16:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.530+0000 7f89bcc00640 1 -- 172.21.15.16:0/1499581644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f89b40c1030 con 0x7f89a8004b50 2024-09-10T21:16:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.530+0000 7f89bcc00640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f89a00725e0 0x7f89a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.530+0000 7f89bcc00640 1 -- 172.21.15.16:0/1499581644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5795+0+0 (secure 0 0 0) 0x7f89b412c950 con 0x7f89a8004b50 2024-09-10T21:16:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.530+0000 7f89bfe00640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f89a00725e0 0x7f89a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.530+0000 7f89bfe00640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f89a00725e0 0x7f89a0074aa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f89c01f50c0 tx=0x7f89ac0091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.534+0000 7f89bcc00640 1 -- 172.21.15.16:0/1499581644 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f89b4016d10 con 0x7f89a8004b50 2024-09-10T21:16:30.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:30.710+0000 7f89c7600640 1 -- 172.21.15.16:0/1499581644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f89c0160c30 con 0x7f89a8004b50 2024-09-10T21:16:31.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.346+0000 7f89bcc00640 1 -- 172.21.15.16:0/1499581644 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) ==== 181+0+0 (secure 0 0 0) 0x7f89b40f7fc0 con 0x7f89a8004b50 2024-09-10T21:16:31.351 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:16:31.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.350+0000 7f89c7600640 1 -- 172.21.15.16:0/1499581644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f89a00725e0 msgr2=0x7f89a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:31.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.350+0000 7f89c7600640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f89a00725e0 0x7f89a0074aa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f89c01f50c0 tx=0x7f89ac0091e0 comp rx=0 tx=0).stop 2024-09-10T21:16:31.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.350+0000 7f89c7600640 1 -- 172.21.15.16:0/1499581644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a8004b50 msgr2=0x7f89c014f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:31.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.350+0000 7f89c7600640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a8004b50 0x7f89c014f010 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f89b401c9f0 tx=0x7f89b4001d10 comp rx=0 tx=0).stop 2024-09-10T21:16:31.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.350+0000 7f89c7600640 1 -- 172.21.15.16:0/1499581644 shutdown_connections 2024-09-10T21:16:31.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.350+0000 7f89c7600640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f89a00725e0 0x7f89a0074aa0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:31.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.350+0000 7f89c7600640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c015d150 0x7f89c01e9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:31.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.350+0000 7f89c7600640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f89c01f7e70 0x7f89c015cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:31.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.350+0000 7f89c7600640 1 --2- 172.21.15.16:0/1499581644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a8004b50 0x7f89c014f010 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:31.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.350+0000 7f89c7600640 1 -- 172.21.15.16:0/1499581644 >> 172.21.15.16:0/1499581644 conn(0x7f89c006d1e0 msgr2=0x7f89c015efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:31.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.350+0000 7f89c7600640 1 -- 172.21.15.16:0/1499581644 shutdown_connections 2024-09-10T21:16:31.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.350+0000 7f89c7600640 1 -- 172.21.15.16:0/1499581644 wait complete. 2024-09-10T21:16:31.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:16:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.534+0000 7f0867200640 1 -- 172.21.15.16:0/3880267225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0844003ab0 msgr2=0x7f0844003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.534+0000 7f0867200640 1 --2- 172.21.15.16:0/3880267225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0844003ab0 0x7f0844003ed0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f08500016c0 tx=0x7f085001c730 comp rx=0 tx=0).stop 2024-09-10T21:16:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.534+0000 7f0867200640 1 -- 172.21.15.16:0/3880267225 shutdown_connections 2024-09-10T21:16:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.534+0000 7f0867200640 1 --2- 172.21.15.16:0/3880267225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0844003ab0 0x7f0844003ed0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.534+0000 7f0867200640 1 -- 172.21.15.16:0/3880267225 >> 172.21.15.16:0/3880267225 conn(0x7f0860073470 msgr2=0x7f0860074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.534+0000 7f0867200640 1 -- 172.21.15.16:0/3880267225 shutdown_connections 2024-09-10T21:16:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.534+0000 7f0867200640 1 -- 172.21.15.16:0/3880267225 wait complete. 2024-09-10T21:16:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.534+0000 7f0867200640 1 Processor -- start 2024-09-10T21:16:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.534+0000 7f0867200640 1 -- start start 2024-09-10T21:16:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0867200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0844003ab0 0x7f0860159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0867200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08601f5ad0 0x7f0860159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0867200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600c4eb0 0x7f086015b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0867200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08600cc4f0 con 0x7f08600c4eb0 2024-09-10T21:16:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0867200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f08600cc370 con 0x7f08601f5ad0 2024-09-10T21:16:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0867200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f08600ccda0 con 0x7f0844003ab0 2024-09-10T21:16:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0865200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600c4eb0 0x7f086015b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f085fe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0844003ab0 0x7f0860159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0865200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600c4eb0 0x7f086015b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56938/0 (socket says 172.21.15.16:56938) 2024-09-10T21:16:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0865200640 1 -- 172.21.15.16:0/3182863951 learned_addr learned my addr 172.21.15.16:0/3182863951 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0857e00640 1 --2- 172.21.15.16:0/3182863951 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08601f5ad0 0x7f0860159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0865200640 1 -- 172.21.15.16:0/3182863951 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0844003ab0 msgr2=0x7f0860159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0865200640 1 --2- 172.21.15.16:0/3182863951 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0844003ab0 0x7f0860159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0865200640 1 -- 172.21.15.16:0/3182863951 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08601f5ad0 msgr2=0x7f0860159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0865200640 1 --2- 172.21.15.16:0/3182863951 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08601f5ad0 0x7f0860159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0865200640 1 -- 172.21.15.16:0/3182863951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08600b2740 con 0x7f08600c4eb0 2024-09-10T21:16:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0865200640 1 --2- 172.21.15.16:0/3182863951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600c4eb0 0x7f086015b910 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f08600cf8b0 tx=0x7f0858010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f085cc00640 1 -- 172.21.15.16:0/3182863951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858003e50 con 0x7f08600c4eb0 2024-09-10T21:16:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f085cc00640 1 -- 172.21.15.16:0/3182863951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0858015070 con 0x7f08600c4eb0 2024-09-10T21:16:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f085cc00640 1 -- 172.21.15.16:0/3182863951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085800ed70 con 0x7f08600c4eb0 2024-09-10T21:16:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0867200640 1 -- 172.21.15.16:0/3182863951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086015da70 con 0x7f08600c4eb0 2024-09-10T21:16:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0867200640 1 -- 172.21.15.16:0/3182863951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08600c7f10 con 0x7f08600c4eb0 2024-09-10T21:16:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f0867200640 1 -- 172.21.15.16:0/3182863951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08600cd300 con 0x7f08600c4eb0 2024-09-10T21:16:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.538+0000 7f085cc00640 1 -- 172.21.15.16:0/3182863951 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0858004050 con 0x7f08600c4eb0 2024-09-10T21:16:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.542+0000 7f085cc00640 1 --2- 172.21.15.16:0/3182863951 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f083c072780 0x7f083c074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.542+0000 7f085cc00640 1 -- 172.21.15.16:0/3182863951 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5357+0+0 (secure 0 0 0) 0x7f0858093a80 con 0x7f08600c4eb0 2024-09-10T21:16:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.542+0000 7f085fe00640 1 --2- 172.21.15.16:0/3182863951 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f083c072780 0x7f083c074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.542+0000 7f085fe00640 1 --2- 172.21.15.16:0/3182863951 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f083c072780 0x7f083c074c40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0844004790 tx=0x7f0850019520 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.546+0000 7f085cc00640 1 -- 172.21.15.16:0/3182863951 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0858095e20 con 0x7f08600c4eb0 2024-09-10T21:16:31.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:31.718+0000 7f0867200640 1 -- 172.21.15.16:0/3182863951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f08600b6220 con 0x7f08600c4eb0 2024-09-10T21:16:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.354+0000 7f085cc00640 1 -- 172.21.15.16:0/3182863951 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) ==== 169+0+0 (secure 0 0 0) 0x7f085805ec60 con 0x7f08600c4eb0 2024-09-10T21:16:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-10T21:16:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.358+0000 7f0867200640 1 -- 172.21.15.16:0/3182863951 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f083c072780 msgr2=0x7f083c074c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.358+0000 7f0867200640 1 --2- 172.21.15.16:0/3182863951 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f083c072780 0x7f083c074c40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0844004790 tx=0x7f0850019520 comp rx=0 tx=0).stop 2024-09-10T21:16:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.358+0000 7f0867200640 1 -- 172.21.15.16:0/3182863951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600c4eb0 msgr2=0x7f086015b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.358+0000 7f0867200640 1 --2- 172.21.15.16:0/3182863951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600c4eb0 0x7f086015b910 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f08600cf8b0 tx=0x7f0858010f40 comp rx=0 tx=0).stop 2024-09-10T21:16:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.358+0000 7f0867200640 1 -- 172.21.15.16:0/3182863951 shutdown_connections 2024-09-10T21:16:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.358+0000 7f0867200640 1 --2- 172.21.15.16:0/3182863951 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f083c072780 0x7f083c074c40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.358+0000 7f0867200640 1 --2- 172.21.15.16:0/3182863951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08600c4eb0 0x7f086015b910 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.358+0000 7f0867200640 1 --2- 172.21.15.16:0/3182863951 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08601f5ad0 0x7f0860159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.358+0000 7f0867200640 1 --2- 172.21.15.16:0/3182863951 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0844003ab0 0x7f0860159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.358+0000 7f0867200640 1 -- 172.21.15.16:0/3182863951 >> 172.21.15.16:0/3182863951 conn(0x7f0860073470 msgr2=0x7f08600b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.358+0000 7f0867200640 1 -- 172.21.15.16:0/3182863951 shutdown_connections 2024-09-10T21:16:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.358+0000 7f0867200640 1 -- 172.21.15.16:0/3182863951 wait complete. 2024-09-10T21:16:32.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:16:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.538+0000 7f3af3000640 1 -- 172.21.15.16:0/226398059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0cfa40 msgr2=0x7f3aec0cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.538+0000 7f3af3000640 1 --2- 172.21.15.16:0/226398059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0cfa40 0x7f3aec0cfe20 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009870 tx=0x7f3ae001c790 comp rx=0 tx=0).stop 2024-09-10T21:16:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.538+0000 7f3af3000640 1 -- 172.21.15.16:0/226398059 shutdown_connections 2024-09-10T21:16:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.538+0000 7f3af3000640 1 --2- 172.21.15.16:0/226398059 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3aec0b1c10 0x7f3aec0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.538+0000 7f3af3000640 1 --2- 172.21.15.16:0/226398059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0cfa40 0x7f3aec0cfe20 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.538+0000 7f3af3000640 1 -- 172.21.15.16:0/226398059 >> 172.21.15.16:0/226398059 conn(0x7f3aec0d1c10 msgr2=0x7f3aec0d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.538+0000 7f3af3000640 1 -- 172.21.15.16:0/226398059 shutdown_connections 2024-09-10T21:16:32.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.538+0000 7f3af3000640 1 -- 172.21.15.16:0/226398059 wait complete. 2024-09-10T21:16:32.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af3000640 1 Processor -- start 2024-09-10T21:16:32.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af3000640 1 -- start start 2024-09-10T21:16:32.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af3000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3aec0b1c10 0x7f3aec159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:32.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af3000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0cdca0 0x7f3aec15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:32.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af3000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aec0cfa40 0x7f3aec0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:32.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af3000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec0cee20 con 0x7f3aec0cdca0 2024-09-10T21:16:32.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af3000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec0ceca0 con 0x7f3aec0cfa40 2024-09-10T21:16:32.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af3000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3aec0cefa0 con 0x7f3aec0b1c10 2024-09-10T21:16:32.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0cdca0 0x7f3aec15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:32.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0cdca0 0x7f3aec15aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45384/0 (socket says 172.21.15.16:45384) 2024-09-10T21:16:32.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af1200640 1 -- 172.21.15.16:0/1581730450 learned_addr learned my addr 172.21.15.16:0/1581730450 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:32.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af1c00640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3aec0b1c10 0x7f3aec159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:32.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af2600640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aec0cfa40 0x7f3aec0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:32.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af1200640 1 -- 172.21.15.16:0/1581730450 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3aec0b1c10 msgr2=0x7f3aec159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:32.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af1200640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3aec0b1c10 0x7f3aec159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af1200640 1 -- 172.21.15.16:0/1581730450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aec0cfa40 msgr2=0x7f3aec0b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:32.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af1200640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aec0cfa40 0x7f3aec0b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af1200640 1 -- 172.21.15.16:0/1581730450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aec0c8ce0 con 0x7f3aec0cdca0 2024-09-10T21:16:32.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af2600640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aec0cfa40 0x7f3aec0b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:32.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af1200640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0cdca0 0x7f3aec15aa80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800c400 tx=0x7f3ad800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:32.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3ae9600640 1 -- 172.21.15.16:0/1581730450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80068a0 con 0x7f3aec0cdca0 2024-09-10T21:16:32.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3ae9600640 1 -- 172.21.15.16:0/1581730450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ad8006b30 con 0x7f3aec0cdca0 2024-09-10T21:16:32.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3ae9600640 1 -- 172.21.15.16:0/1581730450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8007020 con 0x7f3aec0cdca0 2024-09-10T21:16:32.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af3000640 1 -- 172.21.15.16:0/1581730450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aec1f7fe0 con 0x7f3aec0cdca0 2024-09-10T21:16:32.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af3000640 1 -- 172.21.15.16:0/1581730450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aec1f7500 con 0x7f3aec0cdca0 2024-09-10T21:16:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.542+0000 7f3af3000640 1 -- 172.21.15.16:0/1581730450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aec0cd580 con 0x7f3aec0cdca0 2024-09-10T21:16:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.546+0000 7f3ae9600640 1 -- 172.21.15.16:0/1581730450 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3ad80071c0 con 0x7f3aec0cdca0 2024-09-10T21:16:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.546+0000 7f3ae9600640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3acc0725e0 0x7f3acc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.546+0000 7f3ae9600640 1 -- 172.21.15.16:0/1581730450 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7f3ad80929d0 con 0x7f3aec0cdca0 2024-09-10T21:16:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.546+0000 7f3af1c00640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3acc0725e0 0x7f3acc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.546+0000 7f3af1c00640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3acc0725e0 0x7f3acc074aa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00ae000 tx=0x7f3ae001afd0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.550+0000 7f3ae9600640 1 -- 172.21.15.16:0/1581730450 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3aec0c8ce0 con 0x7f3aec0cdca0 2024-09-10T21:16:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.726+0000 7f3af3000640 1 -- 172.21.15.16:0/1581730450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f3aec0760c0 con 0x7f3aec0cdca0 2024-09-10T21:16:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.730+0000 7f3ae9600640 1 -- 172.21.15.16:0/1581730450 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v44) ==== 71+0+0 (secure 0 0 0) 0x7f3ad805e440 con 0x7f3aec0cdca0 2024-09-10T21:16:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.730+0000 7f3af3000640 1 -- 172.21.15.16:0/1581730450 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3acc0725e0 msgr2=0x7f3acc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.730+0000 7f3af3000640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3acc0725e0 0x7f3acc074aa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00ae000 tx=0x7f3ae001afd0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.730+0000 7f3af3000640 1 -- 172.21.15.16:0/1581730450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0cdca0 msgr2=0x7f3aec15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.730+0000 7f3af3000640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0cdca0 0x7f3aec15aa80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800c400 tx=0x7f3ad800c8c0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.730+0000 7f3af3000640 1 -- 172.21.15.16:0/1581730450 shutdown_connections 2024-09-10T21:16:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.730+0000 7f3af3000640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3acc0725e0 0x7f3acc074aa0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.730+0000 7f3af3000640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aec0cfa40 0x7f3aec0b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.730+0000 7f3af3000640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0cdca0 0x7f3aec15aa80 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.730+0000 7f3af3000640 1 --2- 172.21.15.16:0/1581730450 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3aec0b1c10 0x7f3aec159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.730+0000 7f3af3000640 1 -- 172.21.15.16:0/1581730450 >> 172.21.15.16:0/1581730450 conn(0x7f3aec0d1c10 msgr2=0x7f3aec15be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.730+0000 7f3af3000640 1 -- 172.21.15.16:0/1581730450 shutdown_connections 2024-09-10T21:16:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.730+0000 7f3af3000640 1 -- 172.21.15.16:0/1581730450 wait complete. 2024-09-10T21:16:32.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2024-09-10T21:16:32.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.898+0000 7f8d02400640 1 -- 172.21.15.16:0/134818413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc15f450 msgr2=0x7f8cfc0cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.898+0000 7f8d02400640 1 --2- 172.21.15.16:0/134818413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc15f450 0x7f8cfc0cfc80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009840 tx=0x7f8cec01efb0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.898+0000 7f8d02400640 1 -- 172.21.15.16:0/134818413 shutdown_connections 2024-09-10T21:16:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.898+0000 7f8d02400640 1 --2- 172.21.15.16:0/134818413 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8cfc0d0890 0x7f8cfc0d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.898+0000 7f8d02400640 1 --2- 172.21.15.16:0/134818413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc15f450 0x7f8cfc0cfc80 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.898+0000 7f8d02400640 1 -- 172.21.15.16:0/134818413 >> 172.21.15.16:0/134818413 conn(0x7f8cfc0d1d80 msgr2=0x7f8cfc0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.898+0000 7f8d02400640 1 -- 172.21.15.16:0/134818413 shutdown_connections 2024-09-10T21:16:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.898+0000 7f8d02400640 1 -- 172.21.15.16:0/134818413 wait complete. 2024-09-10T21:16:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.898+0000 7f8d02400640 1 Processor -- start 2024-09-10T21:16:32.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.898+0000 7f8d02400640 1 -- start start 2024-09-10T21:16:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.898+0000 7f8d02400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8cfc0d0890 0x7f8cfc0ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.898+0000 7f8d02400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc15f450 0x7f8cfc0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8d02400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cfc1f6080 0x7f8cfc1f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8d02400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc0b2e30 con 0x7f8cfc15f450 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8d02400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc0b2cb0 con 0x7f8cfc1f6080 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8d02400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8cfc0b2fb0 con 0x7f8cfc0d0890 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cf3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc15f450 0x7f8cfc0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cf3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc15f450 0x7f8cfc0c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45398/0 (socket says 172.21.15.16:45398) 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cf3e00640 1 -- 172.21.15.16:0/1319975409 learned_addr learned my addr 172.21.15.16:0/1319975409 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cfb400640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8cfc0d0890 0x7f8cfc0ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cfbe00640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cfc1f6080 0x7f8cfc1f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cf3e00640 1 -- 172.21.15.16:0/1319975409 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8cfc0d0890 msgr2=0x7f8cfc0ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cf3e00640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8cfc0d0890 0x7f8cfc0ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cf3e00640 1 -- 172.21.15.16:0/1319975409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cfc1f6080 msgr2=0x7f8cfc1f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cf3e00640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cfc1f6080 0x7f8cfc1f6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cf3e00640 1 -- 172.21.15.16:0/1319975409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc0b5ca0 con 0x7f8cfc15f450 2024-09-10T21:16:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cf3e00640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc15f450 0x7f8cfc0c8050 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400c430 tx=0x7f8ce400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cf3400640 1 -- 172.21.15.16:0/1319975409 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4011790 con 0x7f8cfc15f450 2024-09-10T21:16:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cf3400640 1 -- 172.21.15.16:0/1319975409 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ce4017070 con 0x7f8cfc15f450 2024-09-10T21:16:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8d02400640 1 -- 172.21.15.16:0/1319975409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cfc0caf00 con 0x7f8cfc15f450 2024-09-10T21:16:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cf3400640 1 -- 172.21.15.16:0/1319975409 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4011ce0 con 0x7f8cfc15f450 2024-09-10T21:16:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8d02400640 1 -- 172.21.15.16:0/1319975409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cfc0cb470 con 0x7f8cfc15f450 2024-09-10T21:16:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8d02400640 1 -- 172.21.15.16:0/1319975409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cfc0b1e90 con 0x7f8cfc15f450 2024-09-10T21:16:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.902+0000 7f8cf3400640 1 -- 172.21.15.16:0/1319975409 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8ce401c460 con 0x7f8cfc15f450 2024-09-10T21:16:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.906+0000 7f8cf3400640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8cd80725e0 0x7f8cd8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.906+0000 7f8cf3400640 1 -- 172.21.15.16:0/1319975409 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7f8ce4092d00 con 0x7f8cfc15f450 2024-09-10T21:16:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.906+0000 7f8cfb400640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8cd80725e0 0x7f8cd8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.906+0000 7f8cfb400640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8cd80725e0 0x7f8cd8074aa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0b2000 tx=0x7f8cec01d850 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:32.910+0000 7f8cf3400640 1 -- 172.21.15.16:0/1319975409 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ce405e6d0 con 0x7f8cfc15f450 2024-09-10T21:16:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.146+0000 7f8d02400640 1 -- 172.21.15.16:0/1319975409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) -- 0x7f8cfc074ba0 con 0x7f8cfc15f450 2024-09-10T21:16:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.354+0000 7f8cf3400640 1 -- 172.21.15.16:0/1319975409 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v298) ==== 160+0+0 (secure 0 0 0) 0x7f8ce405dfd0 con 0x7f8cfc15f450 2024-09-10T21:16:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.358+0000 7f8d02400640 1 -- 172.21.15.16:0/1319975409 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8cd80725e0 msgr2=0x7f8cd8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.358+0000 7f8d02400640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8cd80725e0 0x7f8cd8074aa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0b2000 tx=0x7f8cec01d850 comp rx=0 tx=0).stop 2024-09-10T21:16:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.358+0000 7f8d02400640 1 -- 172.21.15.16:0/1319975409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc15f450 msgr2=0x7f8cfc0c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.358+0000 7f8d02400640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc15f450 0x7f8cfc0c8050 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400c430 tx=0x7f8ce400c8f0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.358+0000 7f8d02400640 1 -- 172.21.15.16:0/1319975409 shutdown_connections 2024-09-10T21:16:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.358+0000 7f8d02400640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8cd80725e0 0x7f8cd8074aa0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.358+0000 7f8d02400640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cfc1f6080 0x7f8cfc1f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.358+0000 7f8d02400640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc15f450 0x7f8cfc0c8050 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.358+0000 7f8d02400640 1 --2- 172.21.15.16:0/1319975409 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8cfc0d0890 0x7f8cfc0ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.358+0000 7f8d02400640 1 -- 172.21.15.16:0/1319975409 >> 172.21.15.16:0/1319975409 conn(0x7f8cfc0d1d80 msgr2=0x7f8cfc0cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.358+0000 7f8d02400640 1 -- 172.21.15.16:0/1319975409 shutdown_connections 2024-09-10T21:16:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.358+0000 7f8d02400640 1 -- 172.21.15.16:0/1319975409 wait complete. 2024-09-10T21:16:33.380 INFO:tasks.cephfs_test_runner:When a client has inodes pinned in its cache (open files), that the MDS ... ok 2024-09-10T21:16:33.380 INFO:tasks.cephfs_test_runner:Starting test: test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) When a client does not advance its oldest tid, the MDS should notice that 2024-09-10T21:16:33.381 INFO:tasks.cephfs_test_runner:test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-10T21:16:33.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:16:33.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 -- 172.21.15.16:0/2823555161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac0415f450 msgr2=0x7fac040cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:33.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 --2- 172.21.15.16:0/2823555161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac0415f450 0x7fac040cfc80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fac040b3810 tx=0x7fabf401efd0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 -- 172.21.15.16:0/2823555161 shutdown_connections 2024-09-10T21:16:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 --2- 172.21.15.16:0/2823555161 >> v2:172.21.15.52:3300/0 conn(0x7fac040d0890 0x7fac040d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 --2- 172.21.15.16:0/2823555161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac0415f450 0x7fac040cfc80 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 -- 172.21.15.16:0/2823555161 >> 172.21.15.16:0/2823555161 conn(0x7fac040d1d80 msgr2=0x7fac040d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 -- 172.21.15.16:0/2823555161 shutdown_connections 2024-09-10T21:16:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 -- 172.21.15.16:0/2823555161 wait complete. 2024-09-10T21:16:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 Processor -- start 2024-09-10T21:16:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 -- start start 2024-09-10T21:16:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac040d0890 0x7fac041553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac0415f450 0x7fac040c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fac041f5a20 0x7fac040ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac040b2e30 con 0x7fac040d0890 2024-09-10T21:16:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fac040b2cb0 con 0x7fac0415f450 2024-09-10T21:16:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.526+0000 7fac09e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fac040b2fb0 con 0x7fac041f5a20 2024-09-10T21:16:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac02a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac040d0890 0x7fac041553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac02a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac040d0890 0x7fac041553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45416/0 (socket says 172.21.15.16:45416) 2024-09-10T21:16:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac02a00640 1 -- 172.21.15.16:0/3927262444 learned_addr learned my addr 172.21.15.16:0/3927262444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fabfbe00640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac0415f450 0x7fac040c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac03400640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fac041f5a20 0x7fac040ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac02a00640 1 -- 172.21.15.16:0/3927262444 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fac041f5a20 msgr2=0x7fac040ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac02a00640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fac041f5a20 0x7fac040ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac02a00640 1 -- 172.21.15.16:0/3927262444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac0415f450 msgr2=0x7fac040c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac02a00640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac0415f450 0x7fac040c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac02a00640 1 -- 172.21.15.16:0/3927262444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac040b6d60 con 0x7fac040d0890 2024-09-10T21:16:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fabfbe00640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac0415f450 0x7fac040c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac03400640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fac041f5a20 0x7fac040ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac02a00640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac040d0890 0x7fac041553d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fabf4000c10 tx=0x7fabf40a8d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fabfb400640 1 -- 172.21.15.16:0/3927262444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf40bf070 con 0x7fac040d0890 2024-09-10T21:16:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fabfb400640 1 -- 172.21.15.16:0/3927262444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabf4005320 con 0x7fac040d0890 2024-09-10T21:16:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fabfb400640 1 -- 172.21.15.16:0/3927262444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf40b5040 con 0x7fac040d0890 2024-09-10T21:16:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac09e00640 1 -- 172.21.15.16:0/3927262444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac040b4e00 con 0x7fac040d0890 2024-09-10T21:16:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac09e00640 1 -- 172.21.15.16:0/3927262444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac040b53c0 con 0x7fac040d0890 2024-09-10T21:16:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fac09e00640 1 -- 172.21.15.16:0/3927262444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac040b1e30 con 0x7fac040d0890 2024-09-10T21:16:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.530+0000 7fabfb400640 1 -- 172.21.15.16:0/3927262444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fabf40b8070 con 0x7fac040d0890 2024-09-10T21:16:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.534+0000 7fabfb400640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fabe0072610 0x7fabe0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.534+0000 7fabfb400640 1 -- 172.21.15.16:0/3927262444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7fabf41301b0 con 0x7fac040d0890 2024-09-10T21:16:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.534+0000 7fabfbe00640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fabe0072610 0x7fabe0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.534+0000 7fabfbe00640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fabe0072610 0x7fabe0074ad0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fac040722a0 tx=0x7fabec009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:33.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.538+0000 7fabfb400640 1 -- 172.21.15.16:0/3927262444 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fabf4018b30 con 0x7fac040d0890 2024-09-10T21:16:33.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.714+0000 7fac09e00640 1 -- 172.21.15.16:0/3927262444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac04074ba0 con 0x7fac040d0890 2024-09-10T21:16:33.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.714+0000 7fabfb400640 1 -- 172.21.15.16:0/3927262444 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+12983 (secure 0 0 0) 0x7fabf40fbb80 con 0x7fac040d0890 2024-09-10T21:16:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:33.720 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:16:32.349985+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6834/943286634":"2024-09-11T21:16:29.339977+0000","172.21.15.52:6832/943286634":"2024-09-11T21:16:29.339977+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:16:33.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.718+0000 7fac09e00640 1 -- 172.21.15.16:0/3927262444 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fabe0072610 msgr2=0x7fabe0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:33.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.718+0000 7fac09e00640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fabe0072610 0x7fabe0074ad0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fac040722a0 tx=0x7fabec009040 comp rx=0 tx=0).stop 2024-09-10T21:16:33.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.718+0000 7fac09e00640 1 -- 172.21.15.16:0/3927262444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac040d0890 msgr2=0x7fac041553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:33.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.718+0000 7fac09e00640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac040d0890 0x7fac041553d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fabf4000c10 tx=0x7fabf40a8d50 comp rx=0 tx=0).stop 2024-09-10T21:16:33.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.718+0000 7fac09e00640 1 -- 172.21.15.16:0/3927262444 shutdown_connections 2024-09-10T21:16:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.718+0000 7fac09e00640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fabe0072610 0x7fabe0074ad0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.718+0000 7fac09e00640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fac041f5a20 0x7fac040ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.718+0000 7fac09e00640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac0415f450 0x7fac040c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.718+0000 7fac09e00640 1 --2- 172.21.15.16:0/3927262444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac040d0890 0x7fac041553d0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.718+0000 7fac09e00640 1 -- 172.21.15.16:0/3927262444 >> 172.21.15.16:0/3927262444 conn(0x7fac040d1d80 msgr2=0x7fac0406e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.718+0000 7fac09e00640 1 -- 172.21.15.16:0/3927262444 shutdown_connections 2024-09-10T21:16:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.718+0000 7fac09e00640 1 -- 172.21.15.16:0/3927262444 wait complete. 2024-09-10T21:16:33.742 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:10:17.493770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:16:33.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:16:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 -- 172.21.15.16:0/1095947225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f487c003c70 msgr2=0x7f487c004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 --2- 172.21.15.16:0/1095947225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f487c003c70 0x7f487c004070 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4888006f80 tx=0x7f488801cf00 comp rx=0 tx=0).stop 2024-09-10T21:16:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 -- 172.21.15.16:0/1095947225 shutdown_connections 2024-09-10T21:16:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 --2- 172.21.15.16:0/1095947225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f487c003c70 0x7f487c004070 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 -- 172.21.15.16:0/1095947225 >> 172.21.15.16:0/1095947225 conn(0x7f48980d1df0 msgr2=0x7f48980d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 -- 172.21.15.16:0/1095947225 shutdown_connections 2024-09-10T21:16:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 -- 172.21.15.16:0/1095947225 wait complete. 2024-09-10T21:16:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 Processor -- start 2024-09-10T21:16:33.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 -- start start 2024-09-10T21:16:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487c003c70 0x7f489806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:33.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f489814eec0 0x7f489814f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48980c7d10 0x7f48980c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48980b1460 con 0x7f487c003c70 2024-09-10T21:16:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f48980b12e0 con 0x7f489814eec0 2024-09-10T21:16:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.890+0000 7f489dc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f48980ce590 con 0x7f48980c7d10 2024-09-10T21:16:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f4896a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487c003c70 0x7f489806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f4896a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487c003c70 0x7f489806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45430/0 (socket says 172.21.15.16:45430) 2024-09-10T21:16:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f4896a00640 1 -- 172.21.15.16:0/1467616909 learned_addr learned my addr 172.21.15.16:0/1467616909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f4897400640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48980c7d10 0x7f48980c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f488fe00640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f489814eec0 0x7f489814f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f4896a00640 1 -- 172.21.15.16:0/1467616909 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48980c7d10 msgr2=0x7f48980c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f4896a00640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48980c7d10 0x7f48980c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f4896a00640 1 -- 172.21.15.16:0/1467616909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f489814eec0 msgr2=0x7f489814f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:33.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f4896a00640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f489814eec0 0x7f489814f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f4896a00640 1 -- 172.21.15.16:0/1467616909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48980cbb40 con 0x7f487c003c70 2024-09-10T21:16:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f4896a00640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487c003c70 0x7f489806dcf0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4888000f70 tx=0x7f4888001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f488ea00640 1 -- 172.21.15.16:0/1467616909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888001700 con 0x7f487c003c70 2024-09-10T21:16:33.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f488ea00640 1 -- 172.21.15.16:0/1467616909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4888006540 con 0x7f487c003c70 2024-09-10T21:16:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f488ea00640 1 -- 172.21.15.16:0/1467616909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888003860 con 0x7f487c003c70 2024-09-10T21:16:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f489dc00640 1 -- 172.21.15.16:0/1467616909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48980cbe00 con 0x7f487c003c70 2024-09-10T21:16:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f489dc00640 1 -- 172.21.15.16:0/1467616909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48980c7510 con 0x7f487c003c70 2024-09-10T21:16:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f489dc00640 1 -- 172.21.15.16:0/1467616909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48980ceaf0 con 0x7f487c003c70 2024-09-10T21:16:33.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.894+0000 7f488ea00640 1 -- 172.21.15.16:0/1467616909 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f48880b7030 con 0x7f487c003c70 2024-09-10T21:16:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.898+0000 7f488ea00640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4874072610 0x7f4874074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.898+0000 7f488ea00640 1 -- 172.21.15.16:0/1467616909 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7f488812cf40 con 0x7f487c003c70 2024-09-10T21:16:33.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.898+0000 7f488fe00640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4874072610 0x7f4874074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:33.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.898+0000 7f488fe00640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4874072610 0x7f4874074ad0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f489806f800 tx=0x7f48800091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:33.902+0000 7f488ea00640 1 -- 172.21.15.16:0/1467616909 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4888016cb0 con 0x7f487c003c70 2024-09-10T21:16:34.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.074+0000 7f489dc00640 1 -- 172.21.15.16:0/1467616909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f489806e230 con 0x7f487c003c70 2024-09-10T21:16:34.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.074+0000 7f488ea00640 1 -- 172.21.15.16:0/1467616909 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f48880f8880 con 0x7f487c003c70 2024-09-10T21:16:34.078 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-10T21:16:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.078+0000 7f489dc00640 1 -- 172.21.15.16:0/1467616909 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4874072610 msgr2=0x7f4874074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.078+0000 7f489dc00640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4874072610 0x7f4874074ad0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f489806f800 tx=0x7f48800091e0 comp rx=0 tx=0).stop 2024-09-10T21:16:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.078+0000 7f489dc00640 1 -- 172.21.15.16:0/1467616909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487c003c70 msgr2=0x7f489806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.078+0000 7f489dc00640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487c003c70 0x7f489806dcf0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4888000f70 tx=0x7f4888001cc0 comp rx=0 tx=0).stop 2024-09-10T21:16:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.078+0000 7f489dc00640 1 -- 172.21.15.16:0/1467616909 shutdown_connections 2024-09-10T21:16:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.078+0000 7f489dc00640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4874072610 0x7f4874074ad0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.078+0000 7f489dc00640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48980c7d10 0x7f48980c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.078+0000 7f489dc00640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f489814eec0 0x7f489814f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.078+0000 7f489dc00640 1 --2- 172.21.15.16:0/1467616909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487c003c70 0x7f489806dcf0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.078+0000 7f489dc00640 1 -- 172.21.15.16:0/1467616909 >> 172.21.15.16:0/1467616909 conn(0x7f48980d1df0 msgr2=0x7f48980b6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.078+0000 7f489dc00640 1 -- 172.21.15.16:0/1467616909 shutdown_connections 2024-09-10T21:16:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.078+0000 7f489dc00640 1 -- 172.21.15.16:0/1467616909 wait complete. 2024-09-10T21:16:34.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2024-09-10T21:16:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.242+0000 7f1890400640 1 -- 172.21.15.16:0/762351565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18880726b0 msgr2=0x7f1888072b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.242+0000 7f1890400640 1 --2- 172.21.15.16:0/762351565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18880726b0 0x7f1888072b60 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1880009c80 tx=0x7f188001cd10 comp rx=0 tx=0).stop 2024-09-10T21:16:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.242+0000 7f1890400640 1 -- 172.21.15.16:0/762351565 shutdown_connections 2024-09-10T21:16:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.242+0000 7f1890400640 1 --2- 172.21.15.16:0/762351565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18880726b0 0x7f1888072b60 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.242+0000 7f1890400640 1 --2- 172.21.15.16:0/762351565 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18881599d0 0x7f188815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.242+0000 7f1890400640 1 -- 172.21.15.16:0/762351565 >> 172.21.15.16:0/762351565 conn(0x7f18880cfb20 msgr2=0x7f18880cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.242+0000 7f1890400640 1 -- 172.21.15.16:0/762351565 shutdown_connections 2024-09-10T21:16:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.242+0000 7f1890400640 1 -- 172.21.15.16:0/762351565 wait complete. 2024-09-10T21:16:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.242+0000 7f1890400640 1 Processor -- start 2024-09-10T21:16:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.242+0000 7f1890400640 1 -- start start 2024-09-10T21:16:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f1890400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f188806e640 0x7f188814aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f1890400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18880726b0 0x7f18880caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f1890400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18881599d0 0x7f18880cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f1890400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1888168f10 con 0x7f188806e640 2024-09-10T21:16:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f1890400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1888168d90 con 0x7f18880726b0 2024-09-10T21:16:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f1890400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f1888169090 con 0x7f18881599d0 2024-09-10T21:16:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f188da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f188806e640 0x7f188814aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f188da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f188806e640 0x7f188814aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45450/0 (socket says 172.21.15.16:45450) 2024-09-10T21:16:34.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f188da00640 1 -- 172.21.15.16:0/685559046 learned_addr learned my addr 172.21.15.16:0/685559046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f188d000640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18880726b0 0x7f18880caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f188e400640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18881599d0 0x7f18880cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f188da00640 1 -- 172.21.15.16:0/685559046 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18881599d0 msgr2=0x7f18880cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f188da00640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18881599d0 0x7f18880cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f188da00640 1 -- 172.21.15.16:0/685559046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18880726b0 msgr2=0x7f18880caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f188da00640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18880726b0 0x7f18880caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f188da00640 1 -- 172.21.15.16:0/685559046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18880c7c00 con 0x7f188806e640 2024-09-10T21:16:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f188da00640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f188806e640 0x7f188814aef0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f187c00e7a0 tx=0x7f187c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f1885600640 1 -- 172.21.15.16:0/685559046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c00ccc0 con 0x7f188806e640 2024-09-10T21:16:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f1885600640 1 -- 172.21.15.16:0/685559046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f187c010430 con 0x7f188806e640 2024-09-10T21:16:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f1885600640 1 -- 172.21.15.16:0/685559046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c010860 con 0x7f188806e640 2024-09-10T21:16:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f1890400640 1 -- 172.21.15.16:0/685559046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18880c6e40 con 0x7f188806e640 2024-09-10T21:16:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f1890400640 1 -- 172.21.15.16:0/685559046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18880c7400 con 0x7f188806e640 2024-09-10T21:16:34.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.246+0000 7f1890400640 1 -- 172.21.15.16:0/685559046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18880c6bd0 con 0x7f188806e640 2024-09-10T21:16:34.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.250+0000 7f1885600640 1 -- 172.21.15.16:0/685559046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f187c010a00 con 0x7f188806e640 2024-09-10T21:16:34.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.250+0000 7f1885600640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18680725e0 0x7f1868074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:34.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.250+0000 7f1885600640 1 -- 172.21.15.16:0/685559046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7f187c0938d0 con 0x7f188806e640 2024-09-10T21:16:34.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.250+0000 7f188d000640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18680725e0 0x7f1868074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:34.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.250+0000 7f188d000640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18680725e0 0x7f1868074aa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f18880c6380 tx=0x7f1874009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:34.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.254+0000 7f1885600640 1 -- 172.21.15.16:0/685559046 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f187c05f340 con 0x7f188806e640 2024-09-10T21:16:34.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:34.490+0000 7f1890400640 1 -- 172.21.15.16:0/685559046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) -- 0x7f1888072b60 con 0x7f188806e640 2024-09-10T21:16:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.362+0000 7f1885600640 1 -- 172.21.15.16:0/685559046 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v300) ==== 153+0+0 (secure 0 0 0) 0x7f187c05ec40 con 0x7f188806e640 2024-09-10T21:16:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.366+0000 7f1890400640 1 -- 172.21.15.16:0/685559046 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18680725e0 msgr2=0x7f1868074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.366+0000 7f1890400640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18680725e0 0x7f1868074aa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f18880c6380 tx=0x7f1874009040 comp rx=0 tx=0).stop 2024-09-10T21:16:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.366+0000 7f1890400640 1 -- 172.21.15.16:0/685559046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f188806e640 msgr2=0x7f188814aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.366+0000 7f1890400640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f188806e640 0x7f188814aef0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f187c00e7a0 tx=0x7f187c00ec60 comp rx=0 tx=0).stop 2024-09-10T21:16:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.366+0000 7f1890400640 1 -- 172.21.15.16:0/685559046 shutdown_connections 2024-09-10T21:16:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.366+0000 7f1890400640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18680725e0 0x7f1868074aa0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.366+0000 7f1890400640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18881599d0 0x7f18880cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.366+0000 7f1890400640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18880726b0 0x7f18880caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.366+0000 7f1890400640 1 --2- 172.21.15.16:0/685559046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f188806e640 0x7f188814aef0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.366+0000 7f1890400640 1 -- 172.21.15.16:0/685559046 >> 172.21.15.16:0/685559046 conn(0x7f18880cfb20 msgr2=0x7f1888158a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.366+0000 7f1890400640 1 -- 172.21.15.16:0/685559046 shutdown_connections 2024-09-10T21:16:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.366+0000 7f1890400640 1 -- 172.21.15.16:0/685559046 wait complete. 2024-09-10T21:16:35.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:16:35.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.538+0000 7f0458000640 1 -- 172.21.15.16:0/3199977962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0450161520 msgr2=0x7f04500aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.538+0000 7f0458000640 1 --2- 172.21.15.16:0/3199977962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0450161520 0x7f04500aea50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f04500c8da0 tx=0x7f044401c890 comp rx=0 tx=0).stop 2024-09-10T21:16:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.538+0000 7f0458000640 1 -- 172.21.15.16:0/3199977962 shutdown_connections 2024-09-10T21:16:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.538+0000 7f0458000640 1 --2- 172.21.15.16:0/3199977962 >> v2:172.21.15.52:3300/0 conn(0x7f04500b27d0 0x7f045006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.538+0000 7f0458000640 1 --2- 172.21.15.16:0/3199977962 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0450161900 0x7f04500b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.538+0000 7f0458000640 1 --2- 172.21.15.16:0/3199977962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0450161520 0x7f04500aea50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.538+0000 7f0458000640 1 -- 172.21.15.16:0/3199977962 >> 172.21.15.16:0/3199977962 conn(0x7f04500d1d80 msgr2=0x7f04500d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.538+0000 7f0458000640 1 -- 172.21.15.16:0/3199977962 shutdown_connections 2024-09-10T21:16:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.538+0000 7f0458000640 1 -- 172.21.15.16:0/3199977962 wait complete. 2024-09-10T21:16:35.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0458000640 1 Processor -- start 2024-09-10T21:16:35.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0458000640 1 -- start start 2024-09-10T21:16:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0458000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04500b27d0 0x7f0450150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0458000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0450161520 0x7f04500b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0458000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0450161900 0x7f04500b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0458000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04500d0260 con 0x7f04500b27d0 2024-09-10T21:16:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0458000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f04500d00e0 con 0x7f0450161520 2024-09-10T21:16:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0458000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f04500d03e0 con 0x7f0450161900 2024-09-10T21:16:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0455600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04500b27d0 0x7f0450150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0455600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04500b27d0 0x7f0450150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45458/0 (socket says 172.21.15.16:45458) 2024-09-10T21:16:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0455600640 1 -- 172.21.15.16:0/940517469 learned_addr learned my addr 172.21.15.16:0/940517469 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0455600640 1 -- 172.21.15.16:0/940517469 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0450161900 msgr2=0x7f04500b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0455600640 1 --2- 172.21.15.16:0/940517469 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0450161900 0x7f04500b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0455600640 1 -- 172.21.15.16:0/940517469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0450161520 msgr2=0x7f04500b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0455600640 1 --2- 172.21.15.16:0/940517469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0450161520 0x7f04500b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0455600640 1 -- 172.21.15.16:0/940517469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04500c9620 con 0x7f04500b27d0 2024-09-10T21:16:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0455600640 1 --2- 172.21.15.16:0/940517469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04500b27d0 0x7f0450150210 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f04440b0040 tx=0x7f0444004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f044d600640 1 -- 172.21.15.16:0/940517469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04440b2040 con 0x7f04500b27d0 2024-09-10T21:16:35.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f044d600640 1 -- 172.21.15.16:0/940517469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0444005030 con 0x7f04500b27d0 2024-09-10T21:16:35.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f044d600640 1 -- 172.21.15.16:0/940517469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444005480 con 0x7f04500b27d0 2024-09-10T21:16:35.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0458000640 1 -- 172.21.15.16:0/940517469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04500cbbd0 con 0x7f04500b27d0 2024-09-10T21:16:35.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.542+0000 7f0458000640 1 -- 172.21.15.16:0/940517469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04500c9340 con 0x7f04500b27d0 2024-09-10T21:16:35.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.546+0000 7f0458000640 1 -- 172.21.15.16:0/940517469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04500aea50 con 0x7f04500b27d0 2024-09-10T21:16:35.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.546+0000 7f044d600640 1 -- 172.21.15.16:0/940517469 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f04440bf060 con 0x7f04500b27d0 2024-09-10T21:16:35.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.546+0000 7f044d600640 1 --2- 172.21.15.16:0/940517469 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f04300725e0 0x7f0430074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:35.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.546+0000 7f044d600640 1 -- 172.21.15.16:0/940517469 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7f044412c2b0 con 0x7f04500b27d0 2024-09-10T21:16:35.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.546+0000 7f0454c00640 1 --2- 172.21.15.16:0/940517469 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f04300725e0 0x7f0430074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:35.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.546+0000 7f0454c00640 1 --2- 172.21.15.16:0/940517469 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f04300725e0 0x7f0430074aa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f045013fa80 tx=0x7f043c005cc0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:35.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.550+0000 7f044d600640 1 -- 172.21.15.16:0/940517469 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f04440164b0 con 0x7f04500b27d0 2024-09-10T21:16:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.722+0000 7f0458000640 1 -- 172.21.15.16:0/940517469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f045015b2a0 con 0x7f04500b27d0 2024-09-10T21:16:35.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.726+0000 7f044d600640 1 -- 172.21.15.16:0/940517469 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v44) ==== 59+0+1998 (secure 0 0 0) 0x7f04440f7b70 con 0x7f04500b27d0 2024-09-10T21:16:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:16:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:16:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:16:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:16:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4226.427518 2024-09-10T21:16:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5426.620519 2024-09-10T21:16:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4255.210203 2024-09-10T21:16:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4255.734871 2024-09-10T21:16:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4192.833008 2024-09-10T21:16:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4136.940759 2024-09-10T21:16:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4217.384701 2024-09-10T21:16:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5220.337972 2024-09-10T21:16:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-10T21:16:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:16:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-10T21:16:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-10T21:16:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:16:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:16:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-10T21:16:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:16:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:16:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:16:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-10T21:16:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-10T21:16:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-10T21:16:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:16:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:16:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.726+0000 7f0458000640 1 -- 172.21.15.16:0/940517469 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f04300725e0 msgr2=0x7f0430074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.726+0000 7f0458000640 1 --2- 172.21.15.16:0/940517469 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f04300725e0 0x7f0430074aa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f045013fa80 tx=0x7f043c005cc0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.726+0000 7f0458000640 1 -- 172.21.15.16:0/940517469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04500b27d0 msgr2=0x7f0450150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.726+0000 7f0458000640 1 --2- 172.21.15.16:0/940517469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04500b27d0 0x7f0450150210 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f04440b0040 tx=0x7f0444004040 comp rx=0 tx=0).stop 2024-09-10T21:16:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.726+0000 7f0458000640 1 -- 172.21.15.16:0/940517469 shutdown_connections 2024-09-10T21:16:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.726+0000 7f0458000640 1 --2- 172.21.15.16:0/940517469 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f04300725e0 0x7f0430074aa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.726+0000 7f0458000640 1 --2- 172.21.15.16:0/940517469 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0450161900 0x7f04500b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.726+0000 7f0458000640 1 --2- 172.21.15.16:0/940517469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0450161520 0x7f04500b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.726+0000 7f0458000640 1 --2- 172.21.15.16:0/940517469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04500b27d0 0x7f0450150210 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.726+0000 7f0458000640 1 -- 172.21.15.16:0/940517469 >> 172.21.15.16:0/940517469 conn(0x7f04500d1d80 msgr2=0x7f0450070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.726+0000 7f0458000640 1 -- 172.21.15.16:0/940517469 shutdown_connections 2024-09-10T21:16:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.726+0000 7f0458000640 1 -- 172.21.15.16:0/940517469 wait complete. 2024-09-10T21:16:35.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:16:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.894+0000 7fb43b800640 1 -- 172.21.15.16:0/2360158735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4340cfe30 msgr2=0x7fb4340ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.894+0000 7fb43b800640 1 --2- 172.21.15.16:0/2360158735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4340cfe30 0x7fb4340ce590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb428009870 tx=0x7fb42801f010 comp rx=0 tx=0).stop 2024-09-10T21:16:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.894+0000 7fb43b800640 1 -- 172.21.15.16:0/2360158735 shutdown_connections 2024-09-10T21:16:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.894+0000 7fb43b800640 1 --2- 172.21.15.16:0/2360158735 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4340ce040 0x7fb4340b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.894+0000 7fb43b800640 1 --2- 172.21.15.16:0/2360158735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4340cfe30 0x7fb4340ce590 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.894+0000 7fb43b800640 1 -- 172.21.15.16:0/2360158735 >> 172.21.15.16:0/2360158735 conn(0x7fb4340d1df0 msgr2=0x7fb4340d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb43b800640 1 -- 172.21.15.16:0/2360158735 shutdown_connections 2024-09-10T21:16:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb43b800640 1 -- 172.21.15.16:0/2360158735 wait complete. 2024-09-10T21:16:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb43b800640 1 Processor -- start 2024-09-10T21:16:35.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb43b800640 1 -- start start 2024-09-10T21:16:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb43b800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4340ce040 0x7fb434158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb43b800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4340cfe30 0x7fb4340b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb43b800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4340d0210 0x7fb4341e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb43b800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340d2b00 con 0x7fb4340cfe30 2024-09-10T21:16:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb43b800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340b14f0 con 0x7fb4340d0210 2024-09-10T21:16:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb43b800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb4340cf1e0 con 0x7fb4340ce040 2024-09-10T21:16:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb438e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4340ce040 0x7fb434158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb433e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4340cfe30 0x7fb4340b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb438e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4340ce040 0x7fb434158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:41956/0 (socket says 172.21.15.16:41956) 2024-09-10T21:16:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb438e00640 1 -- 172.21.15.16:0/3988875964 learned_addr learned my addr 172.21.15.16:0/3988875964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb439800640 1 --2- 172.21.15.16:0/3988875964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4340d0210 0x7fb4341e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb438e00640 1 -- 172.21.15.16:0/3988875964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4340d0210 msgr2=0x7fb4341e5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb438e00640 1 --2- 172.21.15.16:0/3988875964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4340d0210 0x7fb4341e5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb438e00640 1 -- 172.21.15.16:0/3988875964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4340cfe30 msgr2=0x7fb4340b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb438e00640 1 --2- 172.21.15.16:0/3988875964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4340cfe30 0x7fb4340b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:35.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb438e00640 1 -- 172.21.15.16:0/3988875964 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4341e7240 con 0x7fb4340ce040 2024-09-10T21:16:35.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.898+0000 7fb438e00640 1 --2- 172.21.15.16:0/3988875964 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4340ce040 0x7fb434158120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb428009790 tx=0x7fb428005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:35.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.902+0000 7fb430c00640 1 -- 172.21.15.16:0/3988875964 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4280b5040 con 0x7fb4340ce040 2024-09-10T21:16:35.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.902+0000 7fb430c00640 1 -- 172.21.15.16:0/3988875964 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb428005220 con 0x7fb4340ce040 2024-09-10T21:16:35.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.902+0000 7fb430c00640 1 -- 172.21.15.16:0/3988875964 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4280046d0 con 0x7fb4340ce040 2024-09-10T21:16:35.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.902+0000 7fb43b800640 1 -- 172.21.15.16:0/3988875964 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4340c9430 con 0x7fb4340ce040 2024-09-10T21:16:35.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.902+0000 7fb43b800640 1 -- 172.21.15.16:0/3988875964 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4340c99f0 con 0x7fb4340ce040 2024-09-10T21:16:35.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.902+0000 7fb43b800640 1 -- 172.21.15.16:0/3988875964 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4341e7410 con 0x7fb4340ce040 2024-09-10T21:16:35.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.902+0000 7fb430c00640 1 -- 172.21.15.16:0/3988875964 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb4280b8070 con 0x7fb4340ce040 2024-09-10T21:16:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.902+0000 7fb430c00640 1 --2- 172.21.15.16:0/3988875964 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4140725e0 0x7fb414074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.902+0000 7fb430c00640 1 -- 172.21.15.16:0/3988875964 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7fb42812fea0 con 0x7fb4340ce040 2024-09-10T21:16:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.902+0000 7fb433e00640 1 --2- 172.21.15.16:0/3988875964 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4140725e0 0x7fb414074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.906+0000 7fb433e00640 1 --2- 172.21.15.16:0/3988875964 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4140725e0 0x7fb414074aa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb4340cb2e0 tx=0x7fb420009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:35.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:35.906+0000 7fb430c00640 1 -- 172.21.15.16:0/3988875964 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb428018b70 con 0x7fb4340ce040 2024-09-10T21:16:36.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.086+0000 7fb43b800640 1 -- 172.21.15.16:0/3988875964 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb4340b1710 con 0x7fb4340ce040 2024-09-10T21:16:36.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.094+0000 7fb430c00640 1 -- 172.21.15.16:0/3988875964 <== mon.2 v2:172.21.15.52:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7fb4280fb210 con 0x7fb4340ce040 2024-09-10T21:16:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.094+0000 7fb43b800640 1 -- 172.21.15.16:0/3988875964 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4140725e0 msgr2=0x7fb414074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.094+0000 7fb43b800640 1 --2- 172.21.15.16:0/3988875964 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4140725e0 0x7fb414074aa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb4340cb2e0 tx=0x7fb420009040 comp rx=0 tx=0).stop 2024-09-10T21:16:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.094+0000 7fb43b800640 1 -- 172.21.15.16:0/3988875964 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4340ce040 msgr2=0x7fb434158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.094+0000 7fb43b800640 1 --2- 172.21.15.16:0/3988875964 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4340ce040 0x7fb434158120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb428009790 tx=0x7fb428005dd0 comp rx=0 tx=0).stop 2024-09-10T21:16:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.094+0000 7fb43b800640 1 -- 172.21.15.16:0/3988875964 shutdown_connections 2024-09-10T21:16:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.094+0000 7fb43b800640 1 --2- 172.21.15.16:0/3988875964 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb4140725e0 0x7fb414074aa0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.094+0000 7fb43b800640 1 --2- 172.21.15.16:0/3988875964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4340d0210 0x7fb4341e5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.094+0000 7fb43b800640 1 --2- 172.21.15.16:0/3988875964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4340cfe30 0x7fb4340b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.094+0000 7fb43b800640 1 --2- 172.21.15.16:0/3988875964 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4340ce040 0x7fb434158120 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.094+0000 7fb43b800640 1 -- 172.21.15.16:0/3988875964 >> 172.21.15.16:0/3988875964 conn(0x7fb4340d1df0 msgr2=0x7fb4340ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.094+0000 7fb43b800640 1 -- 172.21.15.16:0/3988875964 shutdown_connections 2024-09-10T21:16:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.094+0000 7fb43b800640 1 -- 172.21.15.16:0/3988875964 wait complete. 2024-09-10T21:16:36.118 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:16:36.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:16:36.214 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:16:36.299 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:16:36.299 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:16:36.300 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:36.309 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:16:36.309 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:16:36.309 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:36.318 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:16:36.318 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:16:36.318 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:36.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:16:36.360 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:16:36.360 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:36.373 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:16:36.373 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:16:36.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:16:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.546+0000 7f914d400640 1 -- 172.21.15.16:0/1863438256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91480b13c0 msgr2=0x7f91480afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.546+0000 7f914d400640 1 --2- 172.21.15.16:0/1863438256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91480b13c0 0x7f91480afac0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9140009bc0 tx=0x7f914001ca80 comp rx=0 tx=0).stop 2024-09-10T21:16:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.546+0000 7f914d400640 1 -- 172.21.15.16:0/1863438256 shutdown_connections 2024-09-10T21:16:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.546+0000 7f914d400640 1 --2- 172.21.15.16:0/1863438256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91480b13c0 0x7f91480afac0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.546+0000 7f914d400640 1 --2- 172.21.15.16:0/1863438256 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91480716f0 0x7f91480af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.546+0000 7f914d400640 1 -- 172.21.15.16:0/1863438256 >> 172.21.15.16:0/1863438256 conn(0x7f914809b130 msgr2=0x7f91480d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.546+0000 7f914d400640 1 -- 172.21.15.16:0/1863438256 shutdown_connections 2024-09-10T21:16:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.546+0000 7f914d400640 1 -- 172.21.15.16:0/1863438256 wait complete. 2024-09-10T21:16:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.546+0000 7f914d400640 1 Processor -- start 2024-09-10T21:16:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f914d400640 1 -- start start 2024-09-10T21:16:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f914d400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91480716f0 0x7f914815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f914d400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91480b13c0 0x7f914815bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f914d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f914815f400 0x7f9148141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f914d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9148168f60 con 0x7f914815f400 2024-09-10T21:16:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f914d400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9148168de0 con 0x7f91480b13c0 2024-09-10T21:16:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f914d400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f91481690e0 con 0x7f91480716f0 2024-09-10T21:16:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f9147200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f914815f400 0x7f9148141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f9147200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f914815f400 0x7f9148141830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45502/0 (socket says 172.21.15.16:45502) 2024-09-10T21:16:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f9147200640 1 -- 172.21.15.16:0/3277475709 learned_addr learned my addr 172.21.15.16:0/3277475709 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f9146800640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91480716f0 0x7f914815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f913fe00640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91480b13c0 0x7f914815bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f9147200640 1 -- 172.21.15.16:0/3277475709 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91480716f0 msgr2=0x7f914815b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f9147200640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91480716f0 0x7f914815b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f9147200640 1 -- 172.21.15.16:0/3277475709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91480b13c0 msgr2=0x7f914815bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f9147200640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91480b13c0 0x7f914815bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f9147200640 1 -- 172.21.15.16:0/3277475709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9148163fc0 con 0x7f914815f400 2024-09-10T21:16:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f9147200640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f914815f400 0x7f9148141830 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f91400b0040 tx=0x7f9140005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f913ea00640 1 -- 172.21.15.16:0/3277475709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91400b2040 con 0x7f914815f400 2024-09-10T21:16:36.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f913ea00640 1 -- 172.21.15.16:0/3277475709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9140004040 con 0x7f914815f400 2024-09-10T21:16:36.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f913ea00640 1 -- 172.21.15.16:0/3277475709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140004430 con 0x7f914815f400 2024-09-10T21:16:36.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f914d400640 1 -- 172.21.15.16:0/3277475709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91481642a0 con 0x7f914815f400 2024-09-10T21:16:36.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f914d400640 1 -- 172.21.15.16:0/3277475709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914815c580 con 0x7f914815f400 2024-09-10T21:16:36.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.550+0000 7f914d400640 1 -- 172.21.15.16:0/3277475709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9148164690 con 0x7f914815f400 2024-09-10T21:16:36.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.554+0000 7f913ea00640 1 -- 172.21.15.16:0/3277475709 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f91400b5070 con 0x7f914815f400 2024-09-10T21:16:36.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.554+0000 7f913ea00640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91240725e0 0x7f9124074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:36.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.554+0000 7f913ea00640 1 -- 172.21.15.16:0/3277475709 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7f914012cce0 con 0x7f914815f400 2024-09-10T21:16:36.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.554+0000 7f9146800640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91240725e0 0x7f9124074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:36.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.554+0000 7f9146800640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91240725e0 0x7f9124074aa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9138001770 tx=0x7f9138009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:36.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.558+0000 7f913ea00640 1 -- 172.21.15.16:0/3277475709 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f91400c4060 con 0x7f914815f400 2024-09-10T21:16:36.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f914d400640 1 -- 172.21.15.16:0/3277475709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9148070fe0 con 0x7f914815f400 2024-09-10T21:16:36.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f913ea00640 1 -- 172.21.15.16:0/3277475709 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+3910 (secure 0 0 0) 0x7f9140016790 con 0x7f914815f400 2024-09-10T21:16:36.770 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:36.770 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":55,"btime":"2024-09-10T21:16:30:342614+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/4199133238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":4199133238},{"type":"v1","addr":"172.21.15.52:6835","nonce":4199133238}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/2246088651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2246088651},{"type":"v1","addr":"172.21.15.16:6836","nonce":2246088651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1753951486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1753951486},{"type":"v1","addr":"172.21.15.16:6837","nonce":1753951486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":135662,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/448352209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":448352209},{"type":"v1","addr":"172.21.15.52:6834","nonce":448352209}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[]} 2024-09-10T21:16:36.770 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 55 2024-09-10T21:16:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f914d400640 1 -- 172.21.15.16:0/3277475709 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91240725e0 msgr2=0x7f9124074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f914d400640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91240725e0 0x7f9124074aa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9138001770 tx=0x7f9138009040 comp rx=0 tx=0).stop 2024-09-10T21:16:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f914d400640 1 -- 172.21.15.16:0/3277475709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f914815f400 msgr2=0x7f9148141830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f914d400640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f914815f400 0x7f9148141830 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f91400b0040 tx=0x7f9140005b00 comp rx=0 tx=0).stop 2024-09-10T21:16:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f914d400640 1 -- 172.21.15.16:0/3277475709 shutdown_connections 2024-09-10T21:16:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f914d400640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91240725e0 0x7f9124074aa0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f914d400640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f914815f400 0x7f9148141830 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f914d400640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91480b13c0 0x7f914815bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f914d400640 1 --2- 172.21.15.16:0/3277475709 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91480716f0 0x7f914815b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f914d400640 1 -- 172.21.15.16:0/3277475709 >> 172.21.15.16:0/3277475709 conn(0x7f914809b130 msgr2=0x7f91480d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:36.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f914d400640 1 -- 172.21.15.16:0/3277475709 shutdown_connections 2024-09-10T21:16:36.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:36.766+0000 7f914d400640 1 -- 172.21.15.16:0/3277475709 wait complete. 2024-09-10T21:16:36.791 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:16:36.791 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:16:36.791 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:16:36.791 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:16:36.791 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:16:36.792 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:16:36.792 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:16:36.792 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:16:36.792 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:16:36.792 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:16:36.793 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:16:36.793 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:16:36.793 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:16:36.793 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:16:36.793 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:16:36.793 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:16:36.809 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:16:36.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:16:36.810 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:16:36.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:16:36.811 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:16:36.811 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:16:36.813 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:16:36.813 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:16:36.854 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:16:36.854 DEBUG:teuthology.parallel:result is None 2024-09-10T21:16:36.856 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:16:36.856 DEBUG:teuthology.parallel:result is None 2024-09-10T21:16:36.862 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:16:36.862 DEBUG:teuthology.parallel:result is None 2024-09-10T21:16:36.864 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:16:36.865 DEBUG:teuthology.parallel:result is None 2024-09-10T21:16:36.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:16:36.940 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-10T21:16:36.944 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-10T21:16:36.958 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:16:36.958 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:16:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 -- 172.21.15.16:0/1376216641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1520005220 msgr2=0x7f152000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 --2- 172.21.15.16:0/1376216641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1520005220 0x7f152000b9d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f153000c610 tx=0x7f153001f9a0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 -- 172.21.15.16:0/1376216641 shutdown_connections 2024-09-10T21:16:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 --2- 172.21.15.16:0/1376216641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1520005220 0x7f152000b9d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 -- 172.21.15.16:0/1376216641 >> 172.21.15.16:0/1376216641 conn(0x7f153809b130 msgr2=0x7f15380d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 -- 172.21.15.16:0/1376216641 shutdown_connections 2024-09-10T21:16:37.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 -- 172.21.15.16:0/1376216641 wait complete. 2024-09-10T21:16:37.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 Processor -- start 2024-09-10T21:16:37.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 -- start start 2024-09-10T21:16:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1520005220 0x7f15381f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15381f3ec0 0x7f15380c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15380b17e0 0x7f153815c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1538071080 con 0x7f15381f3ec0 2024-09-10T21:16:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1538070f00 con 0x7f1520005220 2024-09-10T21:16:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.018+0000 7f153fa00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f15380d0590 con 0x7f15380b17e0 2024-09-10T21:16:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f1537e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15381f3ec0 0x7f15380c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f1537e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15381f3ec0 0x7f15380c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45556/0 (socket says 172.21.15.16:45556) 2024-09-10T21:16:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f1537e00640 1 -- 172.21.15.16:0/3505817478 learned_addr learned my addr 172.21.15.16:0/3505817478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f153d000640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1520005220 0x7f15381f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f153da00640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15380b17e0 0x7f153815c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f1537e00640 1 -- 172.21.15.16:0/3505817478 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15380b17e0 msgr2=0x7f153815c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f1537e00640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15380b17e0 0x7f153815c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f1537e00640 1 -- 172.21.15.16:0/3505817478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1520005220 msgr2=0x7f15381f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f1537e00640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1520005220 0x7f15381f7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f1537e00640 1 -- 172.21.15.16:0/3505817478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15380c6200 con 0x7f15381f3ec0 2024-09-10T21:16:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f1537e00640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15381f3ec0 0x7f15380c5cc0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1524003e50 tx=0x7f1524001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f1534c00640 1 -- 172.21.15.16:0/3505817478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524011430 con 0x7f15381f3ec0 2024-09-10T21:16:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f1534c00640 1 -- 172.21.15.16:0/3505817478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1524016020 con 0x7f15381f3ec0 2024-09-10T21:16:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f1534c00640 1 -- 172.21.15.16:0/3505817478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524011890 con 0x7f15381f3ec0 2024-09-10T21:16:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f153fa00640 1 -- 172.21.15.16:0/3505817478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15381f8200 con 0x7f15381f3ec0 2024-09-10T21:16:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f153fa00640 1 -- 172.21.15.16:0/3505817478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15381f87c0 con 0x7f15381f3ec0 2024-09-10T21:16:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f153fa00640 1 -- 172.21.15.16:0/3505817478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15380d0af0 con 0x7f15381f3ec0 2024-09-10T21:16:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.022+0000 7f1534c00640 1 -- 172.21.15.16:0/3505817478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1524011a30 con 0x7f15381f3ec0 2024-09-10T21:16:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.026+0000 7f1534c00640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15180725e0 0x7f1518074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.026+0000 7f1534c00640 1 -- 172.21.15.16:0/3505817478 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7f15240937d0 con 0x7f15381f3ec0 2024-09-10T21:16:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.026+0000 7f153d000640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15180725e0 0x7f1518074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.026+0000 7f153d000640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15180725e0 0x7f1518074aa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f152c005fd0 tx=0x7f152c005f40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.030+0000 7f1534c00640 1 -- 172.21.15.16:0/3505817478 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1524020020 con 0x7f15381f3ec0 2024-09-10T21:16:37.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.214+0000 7f153fa00640 1 -- 172.21.15.16:0/3505817478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f15380008e0 con 0x7f15381f3ec0 2024-09-10T21:16:37.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.378+0000 7f1534c00640 1 -- 172.21.15.16:0/3505817478 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) ==== 97+0+0 (secure 0 0 0) 0x7f152405eac0 con 0x7f15381f3ec0 2024-09-10T21:16:37.383 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-10T21:16:37.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.382+0000 7f153fa00640 1 -- 172.21.15.16:0/3505817478 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15180725e0 msgr2=0x7f1518074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:37.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.382+0000 7f153fa00640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15180725e0 0x7f1518074aa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f152c005fd0 tx=0x7f152c005f40 comp rx=0 tx=0).stop 2024-09-10T21:16:37.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.382+0000 7f153fa00640 1 -- 172.21.15.16:0/3505817478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15381f3ec0 msgr2=0x7f15380c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:37.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.382+0000 7f153fa00640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15381f3ec0 0x7f15380c5cc0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1524003e50 tx=0x7f1524001810 comp rx=0 tx=0).stop 2024-09-10T21:16:37.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.382+0000 7f153fa00640 1 -- 172.21.15.16:0/3505817478 shutdown_connections 2024-09-10T21:16:37.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.382+0000 7f153fa00640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15180725e0 0x7f1518074aa0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.382+0000 7f153fa00640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15380b17e0 0x7f153815c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.382+0000 7f153fa00640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15381f3ec0 0x7f15380c5cc0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.382+0000 7f153fa00640 1 --2- 172.21.15.16:0/3505817478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1520005220 0x7f15381f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.382+0000 7f153fa00640 1 -- 172.21.15.16:0/3505817478 >> 172.21.15.16:0/3505817478 conn(0x7f153809b130 msgr2=0x7f153806e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:37.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.382+0000 7f153fa00640 1 -- 172.21.15.16:0/3505817478 shutdown_connections 2024-09-10T21:16:37.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.382+0000 7f153fa00640 1 -- 172.21.15.16:0/3505817478 wait complete. 2024-09-10T21:16:37.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:16:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.550+0000 7f74c2000640 1 -- 172.21.15.16:0/2465551092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc15f400 msgr2=0x7f74bc070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.550+0000 7f74c2000640 1 --2- 172.21.15.16:0/2465551092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc15f400 0x7f74bc070fe0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0cc360 tx=0x7f74ac01c780 comp rx=0 tx=0).stop 2024-09-10T21:16:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.550+0000 7f74c2000640 1 -- 172.21.15.16:0/2465551092 shutdown_connections 2024-09-10T21:16:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.550+0000 7f74c2000640 1 --2- 172.21.15.16:0/2465551092 >> v2:172.21.15.52:3300/0 conn(0x7f74bc0b0590 0x7f74bc071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.550+0000 7f74c2000640 1 --2- 172.21.15.16:0/2465551092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc15f400 0x7f74bc070fe0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.550+0000 7f74c2000640 1 -- 172.21.15.16:0/2465551092 >> 172.21.15.16:0/2465551092 conn(0x7f74bc09b130 msgr2=0x7f74bc0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.550+0000 7f74c2000640 1 -- 172.21.15.16:0/2465551092 shutdown_connections 2024-09-10T21:16:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74c2000640 1 -- 172.21.15.16:0/2465551092 wait complete. 2024-09-10T21:16:37.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74c2000640 1 Processor -- start 2024-09-10T21:16:37.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74c2000640 1 -- start start 2024-09-10T21:16:37.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74c2000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74bc0b0590 0x7f74bc15d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:37.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74c2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc0b13c0 0x7f74bc1fbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:37.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74c2000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f74bc15f400 0x7f74bc0cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:37.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74c2000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc168f60 con 0x7f74bc0b13c0 2024-09-10T21:16:37.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74c2000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc168de0 con 0x7f74bc0b0590 2024-09-10T21:16:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74c2000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc1690e0 con 0x7f74bc15f400 2024-09-10T21:16:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74b2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc0b13c0 0x7f74bc1fbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74b2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc0b13c0 0x7f74bc1fbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45572/0 (socket says 172.21.15.16:45572) 2024-09-10T21:16:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74b2a00640 1 -- 172.21.15.16:0/4181832421 learned_addr learned my addr 172.21.15.16:0/4181832421 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74bbe00640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f74bc15f400 0x7f74bc0cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74bb400640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74bc0b0590 0x7f74bc15d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74b2a00640 1 -- 172.21.15.16:0/4181832421 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f74bc15f400 msgr2=0x7f74bc0cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74b2a00640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f74bc15f400 0x7f74bc0cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74b2a00640 1 -- 172.21.15.16:0/4181832421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74bc0b0590 msgr2=0x7f74bc15d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74b2a00640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74bc0b0590 0x7f74bc15d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74b2a00640 1 -- 172.21.15.16:0/4181832421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc164130 con 0x7f74bc0b13c0 2024-09-10T21:16:37.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74b2a00640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc0b13c0 0x7f74bc1fbe30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f74a400c490 tx=0x7f74a4001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:37.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74b3e00640 1 -- 172.21.15.16:0/4181832421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4017070 con 0x7f74bc0b13c0 2024-09-10T21:16:37.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74b3e00640 1 -- 172.21.15.16:0/4181832421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74a4003600 con 0x7f74bc0b13c0 2024-09-10T21:16:37.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74b3e00640 1 -- 172.21.15.16:0/4181832421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4003a90 con 0x7f74bc0b13c0 2024-09-10T21:16:37.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.554+0000 7f74c2000640 1 -- 172.21.15.16:0/4181832421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc164410 con 0x7f74bc0b13c0 2024-09-10T21:16:37.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.558+0000 7f74c2000640 1 -- 172.21.15.16:0/4181832421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc0ce050 con 0x7f74bc0b13c0 2024-09-10T21:16:37.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.558+0000 7f74c2000640 1 -- 172.21.15.16:0/4181832421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74bc0cfc80 con 0x7f74bc0b13c0 2024-09-10T21:16:37.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.558+0000 7f74b3e00640 1 -- 172.21.15.16:0/4181832421 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f74a4003c30 con 0x7f74bc0b13c0 2024-09-10T21:16:37.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.558+0000 7f74b3e00640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f74980725e0 0x7f7498074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:37.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.558+0000 7f74b3e00640 1 -- 172.21.15.16:0/4181832421 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7f74a40924c0 con 0x7f74bc0b13c0 2024-09-10T21:16:37.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.558+0000 7f74bb400640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f74980725e0 0x7f7498074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.562+0000 7f74bb400640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f74980725e0 0x7f7498074aa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0ae000 tx=0x7f74ac0b3040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:37.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.562+0000 7f74b3e00640 1 -- 172.21.15.16:0/4181832421 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f74a405d890 con 0x7f74bc0b13c0 2024-09-10T21:16:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.798+0000 7f74c2000640 1 -- 172.21.15.16:0/4181832421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f74bc15adb0 con 0x7f74bc0b13c0 2024-09-10T21:16:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.798+0000 7f74b3e00640 1 -- 172.21.15.16:0/4181832421 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) ==== 80+0+5304 (secure 0 0 0) 0x7f74a40622c0 con 0x7f74bc0b13c0 2024-09-10T21:16:37.804 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:37.804 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:16:37.804 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-10T21:16:37.804 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.804 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-10T21:16:37.804 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmhg+lJBAAWjuYW75BGo2dmyAUi7s9+A==", 2024-09-10T21:16:37.804 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.804 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:16:37.804 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:16:37.804 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:16:37.804 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.805 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.805 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.805 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-10T21:16:37.805 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmQr7+MBAAOzXhWIirBOoi/7wKnrFgmg==", 2024-09-10T21:16:37.805 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.805 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:16:37.805 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:16:37.805 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:16:37.805 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.806 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.806 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.806 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-10T21:16:37.806 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmxxoALBAAgkmw91/VP1Uv2WMZsJKAyA==", 2024-09-10T21:16:37.806 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.806 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:16:37.806 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:16:37.806 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:16:37.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.806 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.806 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.807 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-10T21:16:37.807 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmA4tzOBAAiS+8n+Gokz6yXapVf2ecYg==", 2024-09-10T21:16:37.807 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.807 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:16:37.807 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:16:37.807 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:16:37.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.807 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.807 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.807 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-10T21:16:37.807 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHteBmRoahLRAA32PsUkqtGkeI7tMFRwas5Q==", 2024-09-10T21:16:37.808 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.808 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:16:37.808 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:16:37.808 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.808 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.808 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-10T21:16:37.808 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCLteBmGmEMBhAAr1yQEf/NkiN4+ezAQ9VR2A==", 2024-09-10T21:16:37.808 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.808 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:16:37.808 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:16:37.808 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.809 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.809 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.809 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-10T21:16:37.809 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCOteBmmMuAHBAAt+G/KApsZymjTLRSmyRbdw==", 2024-09-10T21:16:37.809 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.809 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:16:37.809 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:16:37.809 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.809 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.809 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.809 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-10T21:16:37.810 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCRteBmoQfNMBAA0vnIqlYoI13KNAAfNmJg3w==", 2024-09-10T21:16:37.810 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.810 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:16:37.810 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:16:37.810 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.810 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.810 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.810 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-10T21:16:37.810 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCWteBmc/7lDxAADlSi4UpcFJVNdp6YTkpdSw==", 2024-09-10T21:16:37.810 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.810 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:16:37.811 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:16:37.811 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.811 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.811 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.811 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-10T21:16:37.811 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZteBmBZ1LJhAAPcbZa1LQu4kNbSsJ/8Z81A==", 2024-09-10T21:16:37.811 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.811 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:16:37.811 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:16:37.812 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.812 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.812 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.812 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-10T21:16:37.812 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCcteBm7/czOxAAqREnY84+LF1N9dRXOhzz6g==", 2024-09-10T21:16:37.812 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.812 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:16:37.812 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:16:37.812 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.812 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.812 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.813 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-10T21:16:37.813 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgteBmG2J8FRAAB+cMFCnf9wJg1R3RD5SUJw==", 2024-09-10T21:16:37.813 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.813 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:16:37.813 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:16:37.813 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.813 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.813 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-10T21:16:37.813 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA==", 2024-09-10T21:16:37.813 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.814 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:16:37.814 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-10T21:16:37.814 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:16:37.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.814 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.814 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.814 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-10T21:16:37.814 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmuEm4ChAAe08QvhwIcN8ZHa3AoVcYww==", 2024-09-10T21:16:37.814 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.814 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:16:37.814 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:16:37.814 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:16:37.814 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:16:37.815 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.815 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.815 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-10T21:16:37.815 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBm50o1BxAAIFbVxVqU4rU8eKYpuZjTHA==", 2024-09-10T21:16:37.815 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.815 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:16:37.815 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:16:37.815 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:16:37.815 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:16:37.815 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.815 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.816 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-10T21:16:37.816 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmxtu0EBAAmN7+07lDiwjsIC/pNYUlSw==", 2024-09-10T21:16:37.816 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.816 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:16:37.816 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:16:37.816 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:16:37.816 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:16:37.816 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.817 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.817 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-10T21:16:37.817 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmfqgYCBAAm00tBe45phGieCcp1q46cw==", 2024-09-10T21:16:37.817 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.817 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:16:37.817 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:16:37.817 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:16:37.817 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.817 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.817 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.817 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmMVABIBAAoB72mPyXyUO6Y6k+fMqQQg==", 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmeyYxHBAAvBkpTzjPQKAOlHzI1jQ46g==", 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:16:37.818 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:16:37.819 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.819 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:37.819 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-10T21:16:37.819 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:16:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.802+0000 7f74c2000640 1 -- 172.21.15.16:0/4181832421 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f74980725e0 msgr2=0x7f7498074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.802+0000 7f74c2000640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f74980725e0 0x7f7498074aa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0ae000 tx=0x7f74ac0b3040 comp rx=0 tx=0).stop 2024-09-10T21:16:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.802+0000 7f74c2000640 1 -- 172.21.15.16:0/4181832421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc0b13c0 msgr2=0x7f74bc1fbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.802+0000 7f74c2000640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc0b13c0 0x7f74bc1fbe30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f74a400c490 tx=0x7f74a4001e80 comp rx=0 tx=0).stop 2024-09-10T21:16:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.802+0000 7f74c2000640 1 -- 172.21.15.16:0/4181832421 shutdown_connections 2024-09-10T21:16:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.802+0000 7f74c2000640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f74980725e0 0x7f7498074aa0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.802+0000 7f74c2000640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f74bc15f400 0x7f74bc0cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.802+0000 7f74c2000640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc0b13c0 0x7f74bc1fbe30 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.802+0000 7f74c2000640 1 --2- 172.21.15.16:0/4181832421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74bc0b0590 0x7f74bc15d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.802+0000 7f74c2000640 1 -- 172.21.15.16:0/4181832421 >> 172.21.15.16:0/4181832421 conn(0x7f74bc09b130 msgr2=0x7f74bc15b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.802+0000 7f74c2000640 1 -- 172.21.15.16:0/4181832421 shutdown_connections 2024-09-10T21:16:37.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.802+0000 7f74c2000640 1 -- 172.21.15.16:0/4181832421 wait complete. 2024-09-10T21:16:37.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:16:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.966+0000 7fb40cc00640 1 -- 172.21.15.16:0/2795357832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40815a270 msgr2=0x7fb4080ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.966+0000 7fb40cc00640 1 --2- 172.21.15.16:0/2795357832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40815a270 0x7fb4080ce610 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb408074fa0 tx=0x7fb3f801c800 comp rx=0 tx=0).stop 2024-09-10T21:16:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.966+0000 7fb40cc00640 1 -- 172.21.15.16:0/2795357832 shutdown_connections 2024-09-10T21:16:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.966+0000 7fb40cc00640 1 --2- 172.21.15.16:0/2795357832 >> v2:172.21.15.52:3300/0 conn(0x7fb4080ca170 0x7fb4080ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.966+0000 7fb40cc00640 1 --2- 172.21.15.16:0/2795357832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40815a270 0x7fb4080ce610 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.966+0000 7fb40cc00640 1 -- 172.21.15.16:0/2795357832 >> 172.21.15.16:0/2795357832 conn(0x7fb4080cfb20 msgr2=0x7fb4080cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:37.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.966+0000 7fb40cc00640 1 -- 172.21.15.16:0/2795357832 shutdown_connections 2024-09-10T21:16:37.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.966+0000 7fb40cc00640 1 -- 172.21.15.16:0/2795357832 wait complete. 2024-09-10T21:16:37.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb40cc00640 1 Processor -- start 2024-09-10T21:16:37.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb40cc00640 1 -- start start 2024-09-10T21:16:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb40cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4080ca170 0x7fb408150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb40cc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4080d06b0 0x7fb4080b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb40cc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb40815a270 0x7fb4080b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb40cc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb408072b50 con 0x7fb4080ca170 2024-09-10T21:16:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb40cc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb4080729d0 con 0x7fb40815a270 2024-09-10T21:16:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb40cc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb408072cd0 con 0x7fb4080d06b0 2024-09-10T21:16:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb405e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4080ca170 0x7fb408150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb405e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4080ca170 0x7fb408150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45600/0 (socket says 172.21.15.16:45600) 2024-09-10T21:16:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb405e00640 1 -- 172.21.15.16:0/2167954133 learned_addr learned my addr 172.21.15.16:0/2167954133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:37.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb406800640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb40815a270 0x7fb4080b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb3ffe00640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4080d06b0 0x7fb4080b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb405e00640 1 -- 172.21.15.16:0/2167954133 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4080d06b0 msgr2=0x7fb4080b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb405e00640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4080d06b0 0x7fb4080b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb405e00640 1 -- 172.21.15.16:0/2167954133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb40815a270 msgr2=0x7fb4080b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb405e00640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb40815a270 0x7fb4080b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb405e00640 1 -- 172.21.15.16:0/2167954133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb408150b60 con 0x7fb4080ca170 2024-09-10T21:16:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb3ffe00640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4080d06b0 0x7fb4080b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb405e00640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4080ca170 0x7fb408150620 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80b0040 tx=0x7fb3f80a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb3fe000640 1 -- 172.21.15.16:0/2167954133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80b2040 con 0x7fb4080ca170 2024-09-10T21:16:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb3fe000640 1 -- 172.21.15.16:0/2167954133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3f8004430 con 0x7fb4080ca170 2024-09-10T21:16:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb3fe000640 1 -- 172.21.15.16:0/2167954133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8005280 con 0x7fb4080ca170 2024-09-10T21:16:37.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb40cc00640 1 -- 172.21.15.16:0/2167954133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4080c6c90 con 0x7fb4080ca170 2024-09-10T21:16:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.970+0000 7fb40cc00640 1 -- 172.21.15.16:0/2167954133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4080c7250 con 0x7fb4080ca170 2024-09-10T21:16:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.974+0000 7fb40cc00640 1 -- 172.21.15.16:0/2167954133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4080ca5b0 con 0x7fb4080ca170 2024-09-10T21:16:37.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.974+0000 7fb3fe000640 1 -- 172.21.15.16:0/2167954133 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb3f8005420 con 0x7fb4080ca170 2024-09-10T21:16:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.974+0000 7fb3fe000640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3e40725e0 0x7fb3e4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.974+0000 7fb3fe000640 1 -- 172.21.15.16:0/2167954133 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7fb3f812ca60 con 0x7fb4080ca170 2024-09-10T21:16:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.974+0000 7fb3ffe00640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3e40725e0 0x7fb3e4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.974+0000 7fb3ffe00640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3e40725e0 0x7fb3e4074aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb4080c5960 tx=0x7fb3f0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:37.978+0000 7fb3fe000640 1 -- 172.21.15.16:0/2167954133 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb3f80164b0 con 0x7fb4080ca170 2024-09-10T21:16:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.154+0000 7fb40cc00640 1 -- 172.21.15.16:0/2167954133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb4080b5aa0 con 0x7fb4080ca170 2024-09-10T21:16:38.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.154+0000 7fb3fe000640 1 -- 172.21.15.16:0/2167954133 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) ==== 84+0+65 (secure 0 0 0) 0x7fb3f80f83f0 con 0x7fb4080ca170 2024-09-10T21:16:38.160 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:38.160 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-10T21:16:38.160 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:16:38.160 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-10T21:16:38.160 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-10T21:16:38.160 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:16:38.160 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-10T21:16:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.158+0000 7fb40cc00640 1 -- 172.21.15.16:0/2167954133 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3e40725e0 msgr2=0x7fb3e4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.158+0000 7fb40cc00640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3e40725e0 0x7fb3e4074aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb4080c5960 tx=0x7fb3f0009040 comp rx=0 tx=0).stop 2024-09-10T21:16:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.158+0000 7fb40cc00640 1 -- 172.21.15.16:0/2167954133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4080ca170 msgr2=0x7fb408150620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.158+0000 7fb40cc00640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4080ca170 0x7fb408150620 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80b0040 tx=0x7fb3f80a5750 comp rx=0 tx=0).stop 2024-09-10T21:16:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.158+0000 7fb40cc00640 1 -- 172.21.15.16:0/2167954133 shutdown_connections 2024-09-10T21:16:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.158+0000 7fb40cc00640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb3e40725e0 0x7fb3e4074aa0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.158+0000 7fb40cc00640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb40815a270 0x7fb4080b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.158+0000 7fb40cc00640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb4080d06b0 0x7fb4080b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.158+0000 7fb40cc00640 1 --2- 172.21.15.16:0/2167954133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4080ca170 0x7fb408150620 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.158+0000 7fb40cc00640 1 -- 172.21.15.16:0/2167954133 >> 172.21.15.16:0/2167954133 conn(0x7fb4080cfb20 msgr2=0x7fb4080b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.158+0000 7fb40cc00640 1 -- 172.21.15.16:0/2167954133 shutdown_connections 2024-09-10T21:16:38.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.158+0000 7fb40cc00640 1 -- 172.21.15.16:0/2167954133 wait complete. 2024-09-10T21:16:38.181 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:16:38.181 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:16:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.330+0000 7faf3ae00640 1 -- 172.21.15.16:0/954795989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf34072710 msgr2=0x7faf340cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.330+0000 7faf3ae00640 1 --2- 172.21.15.16:0/954795989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf34072710 0x7faf340cda80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009bc0 tx=0x7faf2c01cba0 comp rx=0 tx=0).stop 2024-09-10T21:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.330+0000 7faf3ae00640 1 -- 172.21.15.16:0/954795989 shutdown_connections 2024-09-10T21:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.330+0000 7faf3ae00640 1 --2- 172.21.15.16:0/954795989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf34072710 0x7faf340cda80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.330+0000 7faf3ae00640 1 --2- 172.21.15.16:0/954795989 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf340b3550 0x7faf3415b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.330+0000 7faf3ae00640 1 -- 172.21.15.16:0/954795989 >> 172.21.15.16:0/954795989 conn(0x7faf3406d1e0 msgr2=0x7faf340b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.330+0000 7faf3ae00640 1 -- 172.21.15.16:0/954795989 shutdown_connections 2024-09-10T21:16:38.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.330+0000 7faf3ae00640 1 -- 172.21.15.16:0/954795989 wait complete. 2024-09-10T21:16:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.330+0000 7faf3ae00640 1 Processor -- start 2024-09-10T21:16:38.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.330+0000 7faf3ae00640 1 -- start start 2024-09-10T21:16:38.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.330+0000 7faf3ae00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf34072710 0x7faf340aa790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.330+0000 7faf3ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf340b3550 0x7faf340ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf3ae00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf3415bcc0 0x7faf340aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf3ae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf34167c60 con 0x7faf340b3550 2024-09-10T21:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf3ae00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faf34167ae0 con 0x7faf34072710 2024-09-10T21:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf3ae00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7faf34167de0 con 0x7faf3415bcc0 2024-09-10T21:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf2be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf340b3550 0x7faf340ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf2be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf340b3550 0x7faf340ae7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45606/0 (socket says 172.21.15.16:45606) 2024-09-10T21:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf2be00640 1 -- 172.21.15.16:0/784343638 learned_addr learned my addr 172.21.15.16:0/784343638 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf38e00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf3415bcc0 0x7faf340aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:38.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf33e00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf34072710 0x7faf340aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf2be00640 1 -- 172.21.15.16:0/784343638 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf3415bcc0 msgr2=0x7faf340aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf2be00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf3415bcc0 0x7faf340aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf2be00640 1 -- 172.21.15.16:0/784343638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf34072710 msgr2=0x7faf340aa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf2be00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf34072710 0x7faf340aa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf2be00640 1 -- 172.21.15.16:0/784343638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf340a9430 con 0x7faf340b3550 2024-09-10T21:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf33e00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf34072710 0x7faf340aa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf2be00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf340b3550 0x7faf340ae7a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00ea10 tx=0x7faf1c00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:38.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf30c00640 1 -- 172.21.15.16:0/784343638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00cde0 con 0x7faf340b3550 2024-09-10T21:16:38.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf30c00640 1 -- 172.21.15.16:0/784343638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf1c004020 con 0x7faf340b3550 2024-09-10T21:16:38.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf3ae00640 1 -- 172.21.15.16:0/784343638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf340a96f0 con 0x7faf340b3550 2024-09-10T21:16:38.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf30c00640 1 -- 172.21.15.16:0/784343638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c004470 con 0x7faf340b3550 2024-09-10T21:16:38.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf3ae00640 1 -- 172.21.15.16:0/784343638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf340a9cb0 con 0x7faf340b3550 2024-09-10T21:16:38.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf3ae00640 1 -- 172.21.15.16:0/784343638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf340aa480 con 0x7faf340b3550 2024-09-10T21:16:38.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.334+0000 7faf30c00640 1 -- 172.21.15.16:0/784343638 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faf1c003600 con 0x7faf340b3550 2024-09-10T21:16:38.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.338+0000 7faf30c00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faf100725e0 0x7faf10074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:38.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.338+0000 7faf30c00640 1 -- 172.21.15.16:0/784343638 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7faf1c093910 con 0x7faf340b3550 2024-09-10T21:16:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.338+0000 7faf33e00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faf100725e0 0x7faf10074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.338+0000 7faf33e00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faf100725e0 0x7faf10074aa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faf3415c270 tx=0x7faf24009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:38.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.342+0000 7faf30c00640 1 -- 172.21.15.16:0/784343638 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faf1c05f2a0 con 0x7faf340b3550 2024-09-10T21:16:38.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:38.522+0000 7faf3ae00640 1 -- 172.21.15.16:0/784343638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7faf340cda80 con 0x7faf340b3550 2024-09-10T21:16:39.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.390+0000 7faf30c00640 1 -- 172.21.15.16:0/784343638 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v89) ==== 138+0+0 (secure 0 0 0) 0x7faf1c05eba0 con 0x7faf340b3550 2024-09-10T21:16:39.395 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-10T21:16:39.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.394+0000 7faf3ae00640 1 -- 172.21.15.16:0/784343638 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faf100725e0 msgr2=0x7faf10074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:39.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.394+0000 7faf3ae00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faf100725e0 0x7faf10074aa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faf3415c270 tx=0x7faf24009040 comp rx=0 tx=0).stop 2024-09-10T21:16:39.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.394+0000 7faf3ae00640 1 -- 172.21.15.16:0/784343638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf340b3550 msgr2=0x7faf340ae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.394+0000 7faf3ae00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf340b3550 0x7faf340ae7a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00ea10 tx=0x7faf1c00eed0 comp rx=0 tx=0).stop 2024-09-10T21:16:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.394+0000 7faf3ae00640 1 -- 172.21.15.16:0/784343638 shutdown_connections 2024-09-10T21:16:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.394+0000 7faf3ae00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faf100725e0 0x7faf10074aa0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.394+0000 7faf3ae00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf3415bcc0 0x7faf340aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.394+0000 7faf3ae00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf340b3550 0x7faf340ae7a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.394+0000 7faf3ae00640 1 --2- 172.21.15.16:0/784343638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf34072710 0x7faf340aa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.394+0000 7faf3ae00640 1 -- 172.21.15.16:0/784343638 >> 172.21.15.16:0/784343638 conn(0x7faf3406d1e0 msgr2=0x7faf3415c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.398+0000 7faf3ae00640 1 -- 172.21.15.16:0/784343638 shutdown_connections 2024-09-10T21:16:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.398+0000 7faf3ae00640 1 -- 172.21.15.16:0/784343638 wait complete. 2024-09-10T21:16:39.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:16:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 -- 172.21.15.16:0/1449777019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe43c164880 msgr2=0x7fe43c166c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 --2- 172.21.15.16:0/1449777019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe43c164880 0x7fe43c166c70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe434009bf0 tx=0x7fe43401cae0 comp rx=0 tx=0).stop 2024-09-10T21:16:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 -- 172.21.15.16:0/1449777019 shutdown_connections 2024-09-10T21:16:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 --2- 172.21.15.16:0/1449777019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe43c164880 0x7fe43c166c70 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 -- 172.21.15.16:0/1449777019 >> 172.21.15.16:0/1449777019 conn(0x7fe43c0d1b70 msgr2=0x7fe43c0d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 -- 172.21.15.16:0/1449777019 shutdown_connections 2024-09-10T21:16:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 -- 172.21.15.16:0/1449777019 wait complete. 2024-09-10T21:16:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 Processor -- start 2024-09-10T21:16:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 -- start start 2024-09-10T21:16:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe43c0cfc10 0x7fe43c151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe43c15be70 0x7fe43c1f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe43c164880 0x7fe43c06f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c169d60 con 0x7fe43c164880 2024-09-10T21:16:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c169be0 con 0x7fe43c15be70 2024-09-10T21:16:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe441200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe43c169ee0 con 0x7fe43c0cfc10 2024-09-10T21:16:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe43be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe43c164880 0x7fe43c06f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe43be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe43c164880 0x7fe43c06f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45628/0 (socket says 172.21.15.16:45628) 2024-09-10T21:16:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.582+0000 7fe43be00640 1 -- 172.21.15.16:0/2664578954 learned_addr learned my addr 172.21.15.16:0/2664578954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe43b400640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe43c0cfc10 0x7fe43c151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe433e00640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe43c15be70 0x7fe43c1f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe43be00640 1 -- 172.21.15.16:0/2664578954 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe43c0cfc10 msgr2=0x7fe43c151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe43be00640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe43c0cfc10 0x7fe43c151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe43be00640 1 -- 172.21.15.16:0/2664578954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe43c15be70 msgr2=0x7fe43c1f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe43be00640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe43c15be70 0x7fe43c1f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe43be00640 1 -- 172.21.15.16:0/2664578954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c1fb670 con 0x7fe43c164880 2024-09-10T21:16:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe433e00640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe43c15be70 0x7fe43c1f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe43b400640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe43c0cfc10 0x7fe43c151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe43be00640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe43c164880 0x7fe43c06f8c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe434009b10 tx=0x7fe4340a59b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe433400640 1 -- 172.21.15.16:0/2664578954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4340bb070 con 0x7fe43c164880 2024-09-10T21:16:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe433400640 1 -- 172.21.15.16:0/2664578954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4340a5e50 con 0x7fe43c164880 2024-09-10T21:16:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe433400640 1 -- 172.21.15.16:0/2664578954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4340b2040 con 0x7fe43c164880 2024-09-10T21:16:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe441200640 1 -- 172.21.15.16:0/2664578954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43c1fb950 con 0x7fe43c164880 2024-09-10T21:16:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe441200640 1 -- 172.21.15.16:0/2664578954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43c1f7c00 con 0x7fe43c164880 2024-09-10T21:16:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe441200640 1 -- 172.21.15.16:0/2664578954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe43c071b20 con 0x7fe43c164880 2024-09-10T21:16:39.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.586+0000 7fe433400640 1 -- 172.21.15.16:0/2664578954 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe4340b5030 con 0x7fe43c164880 2024-09-10T21:16:39.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.590+0000 7fe433400640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4180725e0 0x7fe418074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:39.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.590+0000 7fe433400640 1 -- 172.21.15.16:0/2664578954 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5227+0+0 (secure 0 0 0) 0x7fe43412cfe0 con 0x7fe43c164880 2024-09-10T21:16:39.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.590+0000 7fe43b400640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4180725e0 0x7fe418074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:39.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.590+0000 7fe43b400640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4180725e0 0x7fe418074aa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe42c000f70 tx=0x7fe42c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:39.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.594+0000 7fe433400640 1 -- 172.21.15.16:0/2664578954 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe434016790 con 0x7fe43c164880 2024-09-10T21:16:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:39.770+0000 7fe441200640 1 -- 172.21.15.16:0/2664578954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe43c0b2b70 con 0x7fe43c164880 2024-09-10T21:16:40.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.398+0000 7fe433400640 1 -- 172.21.15.16:0/2664578954 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v90) ==== 170+0+0 (secure 0 0 0) 0x7fe4340f8950 con 0x7fe43c164880 2024-09-10T21:16:40.403 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-10T21:16:40.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.402+0000 7fe441200640 1 -- 172.21.15.16:0/2664578954 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4180725e0 msgr2=0x7fe418074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:40.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.402+0000 7fe441200640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4180725e0 0x7fe418074aa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe42c000f70 tx=0x7fe42c009040 comp rx=0 tx=0).stop 2024-09-10T21:16:40.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.402+0000 7fe441200640 1 -- 172.21.15.16:0/2664578954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe43c164880 msgr2=0x7fe43c06f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:40.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.402+0000 7fe441200640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe43c164880 0x7fe43c06f8c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe434009b10 tx=0x7fe4340a59b0 comp rx=0 tx=0).stop 2024-09-10T21:16:40.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.402+0000 7fe441200640 1 -- 172.21.15.16:0/2664578954 shutdown_connections 2024-09-10T21:16:40.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.406+0000 7fe441200640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4180725e0 0x7fe418074aa0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:40.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.406+0000 7fe441200640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe43c164880 0x7fe43c06f8c0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:40.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.406+0000 7fe441200640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe43c15be70 0x7fe43c1f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:40.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.406+0000 7fe441200640 1 --2- 172.21.15.16:0/2664578954 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe43c0cfc10 0x7fe43c151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:40.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.406+0000 7fe441200640 1 -- 172.21.15.16:0/2664578954 >> 172.21.15.16:0/2664578954 conn(0x7fe43c0d1b70 msgr2=0x7fe43c070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.406+0000 7fe441200640 1 -- 172.21.15.16:0/2664578954 shutdown_connections 2024-09-10T21:16:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.406+0000 7fe441200640 1 -- 172.21.15.16:0/2664578954 wait complete. 2024-09-10T21:16:40.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:16:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.594+0000 7fe532400640 1 -- 172.21.15.16:0/1315119875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c151210 msgr2=0x7fe52c0726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.594+0000 7fe532400640 1 --2- 172.21.15.16:0/1315119875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c151210 0x7fe52c0726b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00a950 tx=0x7fe51c01db70 comp rx=0 tx=0).stop 2024-09-10T21:16:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.594+0000 7fe532400640 1 -- 172.21.15.16:0/1315119875 shutdown_connections 2024-09-10T21:16:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.594+0000 7fe532400640 1 --2- 172.21.15.16:0/1315119875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c151210 0x7fe52c0726b0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.594+0000 7fe532400640 1 -- 172.21.15.16:0/1315119875 >> 172.21.15.16:0/1315119875 conn(0x7fe52c0b16a0 msgr2=0x7fe52c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe532400640 1 -- 172.21.15.16:0/1315119875 shutdown_connections 2024-09-10T21:16:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe532400640 1 -- 172.21.15.16:0/1315119875 wait complete. 2024-09-10T21:16:40.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe532400640 1 Processor -- start 2024-09-10T21:16:40.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe532400640 1 -- start start 2024-09-10T21:16:40.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe532400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe52c072bf0 0x7fe52c141110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:40.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe532400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c151210 0x7fe52c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:40.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe532400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe52c15e490 0x7fe52c15d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:40.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe532400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c0706b0 con 0x7fe52c151210 2024-09-10T21:16:40.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe532400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c070530 con 0x7fe52c15e490 2024-09-10T21:16:40.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe532400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe52c070830 con 0x7fe52c072bf0 2024-09-10T21:16:40.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe522a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c151210 0x7fe52c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:40.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe522a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c151210 0x7fe52c1537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45642/0 (socket says 172.21.15.16:45642) 2024-09-10T21:16:40.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe522a00640 1 -- 172.21.15.16:0/731937359 learned_addr learned my addr 172.21.15.16:0/731937359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe52b400640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe52c072bf0 0x7fe52c141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe52be00640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe52c15e490 0x7fe52c15d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe522a00640 1 -- 172.21.15.16:0/731937359 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe52c072bf0 msgr2=0x7fe52c141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe522a00640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe52c072bf0 0x7fe52c141110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe522a00640 1 -- 172.21.15.16:0/731937359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe52c15e490 msgr2=0x7fe52c15d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe522a00640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe52c15e490 0x7fe52c15d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe522a00640 1 -- 172.21.15.16:0/731937359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c1f9040 con 0x7fe52c151210 2024-09-10T21:16:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe522a00640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c151210 0x7fe52c1537c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c450 tx=0x7fe51400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe523e00640 1 -- 172.21.15.16:0/731937359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514005480 con 0x7fe52c151210 2024-09-10T21:16:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe523e00640 1 -- 172.21.15.16:0/731937359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe514003600 con 0x7fe52c151210 2024-09-10T21:16:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe523e00640 1 -- 172.21.15.16:0/731937359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514003a30 con 0x7fe52c151210 2024-09-10T21:16:40.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.598+0000 7fe532400640 1 -- 172.21.15.16:0/731937359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52c1f9320 con 0x7fe52c151210 2024-09-10T21:16:40.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.602+0000 7fe532400640 1 -- 172.21.15.16:0/731937359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52c1f98e0 con 0x7fe52c151210 2024-09-10T21:16:40.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.602+0000 7fe532400640 1 -- 172.21.15.16:0/731937359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe52c0726b0 con 0x7fe52c151210 2024-09-10T21:16:40.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.602+0000 7fe523e00640 1 -- 172.21.15.16:0/731937359 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe514003bd0 con 0x7fe52c151210 2024-09-10T21:16:40.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.606+0000 7fe523e00640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe5080725e0 0x7fe508074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:40.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.606+0000 7fe523e00640 1 -- 172.21.15.16:0/731937359 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5609+0+0 (secure 0 0 0) 0x7fe5140937c0 con 0x7fe52c151210 2024-09-10T21:16:40.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.606+0000 7fe52b400640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe5080725e0 0x7fe508074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:40.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.606+0000 7fe52b400640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe5080725e0 0x7fe508074aa0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0b0000 tx=0x7fe51c01c3b0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.606+0000 7fe523e00640 1 -- 172.21.15.16:0/731937359 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe51405efe0 con 0x7fe52c151210 2024-09-10T21:16:40.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:40.818+0000 7fe532400640 1 -- 172.21.15.16:0/731937359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe52c15b580 con 0x7fe52c151210 2024-09-10T21:16:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.402+0000 7fe523e00640 1 -- 172.21.15.16:0/731937359 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v58) ==== 174+0+0 (secure 0 0 0) 0x7fe51405e8e0 con 0x7fe52c151210 2024-09-10T21:16:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 14 and data pool 15 2024-09-10T21:16:41.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.406+0000 7fe532400640 1 -- 172.21.15.16:0/731937359 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe5080725e0 msgr2=0x7fe508074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:41.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.406+0000 7fe532400640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe5080725e0 0x7fe508074aa0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0b0000 tx=0x7fe51c01c3b0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.406+0000 7fe532400640 1 -- 172.21.15.16:0/731937359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c151210 msgr2=0x7fe52c1537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:41.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.406+0000 7fe532400640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c151210 0x7fe52c1537c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c450 tx=0x7fe51400c910 comp rx=0 tx=0).stop 2024-09-10T21:16:41.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.406+0000 7fe532400640 1 -- 172.21.15.16:0/731937359 shutdown_connections 2024-09-10T21:16:41.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.406+0000 7fe532400640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe5080725e0 0x7fe508074aa0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.406+0000 7fe532400640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe52c15e490 0x7fe52c15d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.406+0000 7fe532400640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c151210 0x7fe52c1537c0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.406+0000 7fe532400640 1 --2- 172.21.15.16:0/731937359 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe52c072bf0 0x7fe52c141110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.406+0000 7fe532400640 1 -- 172.21.15.16:0/731937359 >> 172.21.15.16:0/731937359 conn(0x7fe52c0b16a0 msgr2=0x7fe52c06f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:41.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.406+0000 7fe532400640 1 -- 172.21.15.16:0/731937359 shutdown_connections 2024-09-10T21:16:41.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.406+0000 7fe532400640 1 -- 172.21.15.16:0/731937359 wait complete. 2024-09-10T21:16:41.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:16:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.598+0000 7fa05a400640 1 -- 172.21.15.16:0/2500831469 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa054154f90 msgr2=0x7fa054072d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.598+0000 7fa05a400640 1 --2- 172.21.15.16:0/2500831469 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa054154f90 0x7fa054072d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa044009840 tx=0x7fa04401c710 comp rx=0 tx=0).stop 2024-09-10T21:16:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.598+0000 7fa05a400640 1 -- 172.21.15.16:0/2500831469 shutdown_connections 2024-09-10T21:16:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.598+0000 7fa05a400640 1 --2- 172.21.15.16:0/2500831469 >> v2:172.21.15.52:3300/0 conn(0x7fa0540706f0 0x7fa0540ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.598+0000 7fa05a400640 1 --2- 172.21.15.16:0/2500831469 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa054154f90 0x7fa054072d40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.598+0000 7fa05a400640 1 -- 172.21.15.16:0/2500831469 >> 172.21.15.16:0/2500831469 conn(0x7fa0540cfb20 msgr2=0x7fa0540cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.598+0000 7fa05a400640 1 -- 172.21.15.16:0/2500831469 shutdown_connections 2024-09-10T21:16:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.598+0000 7fa05a400640 1 -- 172.21.15.16:0/2500831469 wait complete. 2024-09-10T21:16:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa05a400640 1 Processor -- start 2024-09-10T21:16:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa05a400640 1 -- start start 2024-09-10T21:16:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa05a400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540706f0 0x7fa054141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa05a400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa054154f90 0x7fa0541422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa05a400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa05415bc70 0x7fa0540c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa05a400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa054167b00 con 0x7fa0540706f0 2024-09-10T21:16:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa05a400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa054167980 con 0x7fa054154f90 2024-09-10T21:16:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa05a400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa054167c80 con 0x7fa05415bc70 2024-09-10T21:16:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa053400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540706f0 0x7fa054141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa053400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540706f0 0x7fa054141da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45658/0 (socket says 172.21.15.16:45658) 2024-09-10T21:16:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa053400640 1 -- 172.21.15.16:0/803685446 learned_addr learned my addr 172.21.15.16:0/803685446 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa053e00640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa05415bc70 0x7fa0540c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa04be00640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa054154f90 0x7fa0541422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa053400640 1 -- 172.21.15.16:0/803685446 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa05415bc70 msgr2=0x7fa0540c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa053400640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa05415bc70 0x7fa0540c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa053400640 1 -- 172.21.15.16:0/803685446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa054154f90 msgr2=0x7fa0541422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa053400640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa054154f90 0x7fa0541422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa053400640 1 -- 172.21.15.16:0/803685446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0540c4e50 con 0x7fa0540706f0 2024-09-10T21:16:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa04be00640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa054154f90 0x7fa0541422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa053400640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540706f0 0x7fa054141da0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa04401adc0 tx=0x7fa044004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa04b400640 1 -- 172.21.15.16:0/803685446 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0440b2040 con 0x7fa0540706f0 2024-09-10T21:16:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa04b400640 1 -- 172.21.15.16:0/803685446 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa044005310 con 0x7fa0540706f0 2024-09-10T21:16:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa05a400640 1 -- 172.21.15.16:0/803685446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0540c5190 con 0x7fa0540706f0 2024-09-10T21:16:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.602+0000 7fa04b400640 1 -- 172.21.15.16:0/803685446 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0440a5890 con 0x7fa0540706f0 2024-09-10T21:16:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.606+0000 7fa05a400640 1 -- 172.21.15.16:0/803685446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0540c4b50 con 0x7fa0540706f0 2024-09-10T21:16:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.606+0000 7fa05a400640 1 -- 172.21.15.16:0/803685446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0540c5370 con 0x7fa0540706f0 2024-09-10T21:16:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.606+0000 7fa04b400640 1 -- 172.21.15.16:0/803685446 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa0440b5070 con 0x7fa0540706f0 2024-09-10T21:16:41.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.606+0000 7fa04b400640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0300725e0 0x7fa030074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:41.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.606+0000 7fa04b400640 1 -- 172.21.15.16:0/803685446 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5709+0+0 (secure 0 0 0) 0x7fa04412bfa0 con 0x7fa0540706f0 2024-09-10T21:16:41.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.606+0000 7fa04be00640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0300725e0 0x7fa030074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.610+0000 7fa04be00640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0300725e0 0x7fa030074aa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa0540cef40 tx=0x7fa03c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.610+0000 7fa04b400640 1 -- 172.21.15.16:0/803685446 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa0540c5190 con 0x7fa0540706f0 2024-09-10T21:16:41.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.786+0000 7fa05a400640 1 -- 172.21.15.16:0/803685446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0540ca530 con 0x7fa0540706f0 2024-09-10T21:16:41.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.786+0000 7fa04b400640 1 -- 172.21.15.16:0/803685446 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16633 (secure 0 0 0) 0x7fa044016330 con 0x7fa0540706f0 2024-09-10T21:16:41.790 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:41.791 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:16:40.823290+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:16:38.533164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T21:16:39.781552+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:16:41.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.790+0000 7fa05a400640 1 -- 172.21.15.16:0/803685446 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0300725e0 msgr2=0x7fa030074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:41.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.790+0000 7fa05a400640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0300725e0 0x7fa030074aa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa0540cef40 tx=0x7fa03c009040 comp rx=0 tx=0).stop 2024-09-10T21:16:41.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.790+0000 7fa05a400640 1 -- 172.21.15.16:0/803685446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540706f0 msgr2=0x7fa054141da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:41.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.790+0000 7fa05a400640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540706f0 0x7fa054141da0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa04401adc0 tx=0x7fa044004620 comp rx=0 tx=0).stop 2024-09-10T21:16:41.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.790+0000 7fa05a400640 1 -- 172.21.15.16:0/803685446 shutdown_connections 2024-09-10T21:16:41.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.790+0000 7fa05a400640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa0300725e0 0x7fa030074aa0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.790+0000 7fa05a400640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa05415bc70 0x7fa0540c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.790+0000 7fa05a400640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa054154f90 0x7fa0541422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.790+0000 7fa05a400640 1 --2- 172.21.15.16:0/803685446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540706f0 0x7fa054141da0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.790+0000 7fa05a400640 1 -- 172.21.15.16:0/803685446 >> 172.21.15.16:0/803685446 conn(0x7fa0540cfb20 msgr2=0x7fa05414f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:41.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.790+0000 7fa05a400640 1 -- 172.21.15.16:0/803685446 shutdown_connections 2024-09-10T21:16:41.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.790+0000 7fa05a400640 1 -- 172.21.15.16:0/803685446 wait complete. 2024-09-10T21:16:41.812 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:16:41.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.954+0000 7f54c0e00640 1 -- 172.21.15.16:0/2105133167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54bc0b0590 msgr2=0x7f54bc071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:41.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.954+0000 7f54c0e00640 1 --2- 172.21.15.16:0/2105133167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54bc0b0590 0x7f54bc071520 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009c80 tx=0x7f54b401cc80 comp rx=0 tx=0).stop 2024-09-10T21:16:41.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.954+0000 7f54c0e00640 1 -- 172.21.15.16:0/2105133167 shutdown_connections 2024-09-10T21:16:41.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.954+0000 7f54c0e00640 1 --2- 172.21.15.16:0/2105133167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54bc0b0590 0x7f54bc071520 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.954+0000 7f54c0e00640 1 --2- 172.21.15.16:0/2105133167 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f54bc15f400 0x7f54bc070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.954+0000 7f54c0e00640 1 -- 172.21.15.16:0/2105133167 >> 172.21.15.16:0/2105133167 conn(0x7f54bc09b130 msgr2=0x7f54bc0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:41.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.954+0000 7f54c0e00640 1 -- 172.21.15.16:0/2105133167 shutdown_connections 2024-09-10T21:16:41.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.954+0000 7f54c0e00640 1 -- 172.21.15.16:0/2105133167 wait complete. 2024-09-10T21:16:41.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54c0e00640 1 Processor -- start 2024-09-10T21:16:41.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54c0e00640 1 -- start start 2024-09-10T21:16:41.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54c0e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54bc0b0590 0x7f54bc0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:41.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54c0e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f54bc0b13c0 0x7f54bc14f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:41.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54c0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54bc15f400 0x7f54bc14f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:41.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54c0e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc168f60 con 0x7f54bc15f400 2024-09-10T21:16:41.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54c0e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc168de0 con 0x7f54bc0b0590 2024-09-10T21:16:41.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54c0e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f54bc1690e0 con 0x7f54bc0b13c0 2024-09-10T21:16:41.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54ba800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54bc15f400 0x7f54bc14f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:41.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54ba800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54bc15f400 0x7f54bc14f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45676/0 (socket says 172.21.15.16:45676) 2024-09-10T21:16:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54ba800640 1 -- 172.21.15.16:0/3538343793 learned_addr learned my addr 172.21.15.16:0/3538343793 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54b3e00640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f54bc0b13c0 0x7f54bc14f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54b9e00640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54bc0b0590 0x7f54bc0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54ba800640 1 -- 172.21.15.16:0/3538343793 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f54bc0b13c0 msgr2=0x7f54bc14f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54ba800640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f54bc0b13c0 0x7f54bc14f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54ba800640 1 -- 172.21.15.16:0/3538343793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54bc0b0590 msgr2=0x7f54bc0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54ba800640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54bc0b0590 0x7f54bc0c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54ba800640 1 -- 172.21.15.16:0/3538343793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54bc14f030 con 0x7f54bc15f400 2024-09-10T21:16:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54b3e00640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f54bc0b13c0 0x7f54bc14f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54b9e00640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54bc0b0590 0x7f54bc0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54ba800640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54bc15f400 0x7f54bc14f760 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009bd0 tx=0x7f54b4009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54b2000640 1 -- 172.21.15.16:0/3538343793 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b40bb070 con 0x7f54bc15f400 2024-09-10T21:16:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54b2000640 1 -- 172.21.15.16:0/3538343793 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54b4004e00 con 0x7f54bc15f400 2024-09-10T21:16:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54b2000640 1 -- 172.21.15.16:0/3538343793 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b40b2040 con 0x7f54bc15f400 2024-09-10T21:16:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54c0e00640 1 -- 172.21.15.16:0/3538343793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54bc150a50 con 0x7f54bc15f400 2024-09-10T21:16:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.958+0000 7f54c0e00640 1 -- 172.21.15.16:0/3538343793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54bc0b6010 con 0x7f54bc15f400 2024-09-10T21:16:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.962+0000 7f54c0e00640 1 -- 172.21.15.16:0/3538343793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54bc0cfc80 con 0x7f54bc15f400 2024-09-10T21:16:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.962+0000 7f54b2000640 1 -- 172.21.15.16:0/3538343793 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f54b40b5030 con 0x7f54bc15f400 2024-09-10T21:16:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.962+0000 7f54b2000640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54980725e0 0x7f5498074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.962+0000 7f54b2000640 1 -- 172.21.15.16:0/3538343793 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5709+0+0 (secure 0 0 0) 0x7f54b412cd20 con 0x7f54bc15f400 2024-09-10T21:16:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.962+0000 7f54b9e00640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54980725e0 0x7f5498074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.962+0000 7f54b9e00640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54980725e0 0x7f5498074aa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f54ac005fd0 tx=0x7f54ac005cd0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:41.966+0000 7f54b2000640 1 -- 172.21.15.16:0/3538343793 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f54b40f1de0 con 0x7f54bc15f400 2024-09-10T21:16:42.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.138+0000 7f54c0e00640 1 -- 172.21.15.16:0/3538343793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54bc0b18d0 con 0x7f54bc15f400 2024-09-10T21:16:42.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.142+0000 7f54b2000640 1 -- 172.21.15.16:0/3538343793 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16633 (secure 0 0 0) 0x7f54bc150a50 con 0x7f54bc15f400 2024-09-10T21:16:42.145 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:42.145 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:16:40.823290+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:16:38.533164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T21:16:39.781552+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:16:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.142+0000 7f54c0e00640 1 -- 172.21.15.16:0/3538343793 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54980725e0 msgr2=0x7f5498074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.142+0000 7f54c0e00640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54980725e0 0x7f5498074aa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f54ac005fd0 tx=0x7f54ac005cd0 comp rx=0 tx=0).stop 2024-09-10T21:16:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.142+0000 7f54c0e00640 1 -- 172.21.15.16:0/3538343793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54bc15f400 msgr2=0x7f54bc14f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.142+0000 7f54c0e00640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54bc15f400 0x7f54bc14f760 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009bd0 tx=0x7f54b4009390 comp rx=0 tx=0).stop 2024-09-10T21:16:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.142+0000 7f54c0e00640 1 -- 172.21.15.16:0/3538343793 shutdown_connections 2024-09-10T21:16:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.142+0000 7f54c0e00640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54980725e0 0x7f5498074aa0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.142+0000 7f54c0e00640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54bc15f400 0x7f54bc14f760 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.142+0000 7f54c0e00640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f54bc0b13c0 0x7f54bc14f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.142+0000 7f54c0e00640 1 --2- 172.21.15.16:0/3538343793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54bc0b0590 0x7f54bc0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.142+0000 7f54c0e00640 1 -- 172.21.15.16:0/3538343793 >> 172.21.15.16:0/3538343793 conn(0x7f54bc09b130 msgr2=0x7f54bc0ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.142+0000 7f54c0e00640 1 -- 172.21.15.16:0/3538343793 shutdown_connections 2024-09-10T21:16:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.142+0000 7f54c0e00640 1 -- 172.21.15.16:0/3538343793 wait complete. 2024-09-10T21:16:42.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:16:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.310+0000 7f8909e00640 1 -- 172.21.15.16:0/192731192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f890415bdc0 msgr2=0x7f89040d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.310+0000 7f8909e00640 1 --2- 172.21.15.16:0/192731192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f890415bdc0 0x7f89040d2800 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009840 tx=0x7f88f401c700 comp rx=0 tx=0).stop 2024-09-10T21:16:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.310+0000 7f8909e00640 1 -- 172.21.15.16:0/192731192 shutdown_connections 2024-09-10T21:16:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.310+0000 7f8909e00640 1 --2- 172.21.15.16:0/192731192 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f89040cd300 0x7f89040b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.310+0000 7f8909e00640 1 --2- 172.21.15.16:0/192731192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f890415bdc0 0x7f89040d2800 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.310+0000 7f8909e00640 1 -- 172.21.15.16:0/192731192 >> 172.21.15.16:0/192731192 conn(0x7f89040d1ce0 msgr2=0x7f89040d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.310+0000 7f8909e00640 1 -- 172.21.15.16:0/192731192 shutdown_connections 2024-09-10T21:16:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.310+0000 7f8909e00640 1 -- 172.21.15.16:0/192731192 wait complete. 2024-09-10T21:16:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f8909e00640 1 Processor -- start 2024-09-10T21:16:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f8909e00640 1 -- start start 2024-09-10T21:16:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f8909e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f89040b5050 0x7f890406f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f8909e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89040cd300 0x7f8904153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f8909e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f890415bdc0 0x7f8904154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f8909e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8904168e90 con 0x7f89040cd300 2024-09-10T21:16:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f8909e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89041602c0 con 0x7f890415bdc0 2024-09-10T21:16:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f8909e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8904169010 con 0x7f89040b5050 2024-09-10T21:16:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f88fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89040cd300 0x7f8904153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f88fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89040cd300 0x7f8904153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47656/0 (socket says 172.21.15.16:47656) 2024-09-10T21:16:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f88fbe00640 1 -- 172.21.15.16:0/4020389167 learned_addr learned my addr 172.21.15.16:0/4020389167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f8902a00640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f89040b5050 0x7f890406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f8903400640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f890415bdc0 0x7f8904154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f88fbe00640 1 -- 172.21.15.16:0/4020389167 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f89040b5050 msgr2=0x7f890406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f88fbe00640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f89040b5050 0x7f890406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f88fbe00640 1 -- 172.21.15.16:0/4020389167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f890415bdc0 msgr2=0x7f8904154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f88fbe00640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f890415bdc0 0x7f8904154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f88fbe00640 1 -- 172.21.15.16:0/4020389167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8904164e40 con 0x7f89040cd300 2024-09-10T21:16:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f88fbe00640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89040cd300 0x7f8904153df0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00ea10 tx=0x7f88ec00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f88fb400640 1 -- 172.21.15.16:0/4020389167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00cde0 con 0x7f89040cd300 2024-09-10T21:16:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f88fb400640 1 -- 172.21.15.16:0/4020389167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88ec004020 con 0x7f89040cd300 2024-09-10T21:16:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f88fb400640 1 -- 172.21.15.16:0/4020389167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec004470 con 0x7f89040cd300 2024-09-10T21:16:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f8909e00640 1 -- 172.21.15.16:0/4020389167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8904165100 con 0x7f89040cd300 2024-09-10T21:16:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.314+0000 7f8909e00640 1 -- 172.21.15.16:0/4020389167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89041656c0 con 0x7f89040cd300 2024-09-10T21:16:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.318+0000 7f8909e00640 1 -- 172.21.15.16:0/4020389167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89040b4b10 con 0x7f89040cd300 2024-09-10T21:16:42.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.318+0000 7f88fb400640 1 -- 172.21.15.16:0/4020389167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f88ec003600 con 0x7f89040cd300 2024-09-10T21:16:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.318+0000 7f88fb400640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88e00725e0 0x7f88e0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.318+0000 7f88fb400640 1 -- 172.21.15.16:0/4020389167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5709+0+0 (secure 0 0 0) 0x7f88ec093cc0 con 0x7f89040cd300 2024-09-10T21:16:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.318+0000 7f8902a00640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88e00725e0 0x7f88e0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.318+0000 7f8902a00640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88e00725e0 0x7f88e0074aa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f88f40af000 tx=0x7f88f401af40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:42.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.322+0000 7f88fb400640 1 -- 172.21.15.16:0/4020389167 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f88ec09a080 con 0x7f89040cd300 2024-09-10T21:16:42.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:42.534+0000 7f8909e00640 1 -- 172.21.15.16:0/4020389167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f89040704a0 con 0x7f89040cd300 2024-09-10T21:16:43.394 INFO:tasks.ceph.mds.d.smithi052.stderr:2024-09-10T21:16:43.389+0000 7f1ef1600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:16:43.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.426+0000 7f88fb400640 1 -- 172.21.15.16:0/4020389167 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) ==== 118+0+0 (secure 0 0 0) 0x7f88ec05f300 con 0x7f89040cd300 2024-09-10T21:16:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.430+0000 7f8909e00640 1 -- 172.21.15.16:0/4020389167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88e00725e0 msgr2=0x7f88e0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.430+0000 7f8909e00640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88e00725e0 0x7f88e0074aa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f88f40af000 tx=0x7f88f401af40 comp rx=0 tx=0).stop 2024-09-10T21:16:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.430+0000 7f8909e00640 1 -- 172.21.15.16:0/4020389167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89040cd300 msgr2=0x7f8904153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.430+0000 7f8909e00640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89040cd300 0x7f8904153df0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00ea10 tx=0x7f88ec00eed0 comp rx=0 tx=0).stop 2024-09-10T21:16:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.430+0000 7f8909e00640 1 -- 172.21.15.16:0/4020389167 shutdown_connections 2024-09-10T21:16:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.430+0000 7f8909e00640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f88e00725e0 0x7f88e0074aa0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.430+0000 7f8909e00640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f890415bdc0 0x7f8904154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:43.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.430+0000 7f8909e00640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89040cd300 0x7f8904153df0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.430+0000 7f8909e00640 1 --2- 172.21.15.16:0/4020389167 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f89040b5050 0x7f890406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.430+0000 7f8909e00640 1 -- 172.21.15.16:0/4020389167 >> 172.21.15.16:0/4020389167 conn(0x7f89040d1ce0 msgr2=0x7f8904155ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.430+0000 7f8909e00640 1 -- 172.21.15.16:0/4020389167 shutdown_connections 2024-09-10T21:16:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.430+0000 7f8909e00640 1 -- 172.21.15.16:0/4020389167 wait complete. 2024-09-10T21:16:43.458 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:16:43.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:16:43.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.610+0000 7f4d96200640 1 -- 172.21.15.16:0/665154186 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d900b3200 msgr2=0x7f4d9006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:43.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.610+0000 7f4d96200640 1 --2- 172.21.15.16:0/665154186 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d900b3200 0x7f4d9006dcf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4d900b1220 tx=0x7f4d7801c820 comp rx=0 tx=0).stop 2024-09-10T21:16:43.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.610+0000 7f4d96200640 1 -- 172.21.15.16:0/665154186 shutdown_connections 2024-09-10T21:16:43.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.610+0000 7f4d96200640 1 --2- 172.21.15.16:0/665154186 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d900b3200 0x7f4d9006dcf0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:43.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.610+0000 7f4d96200640 1 --2- 172.21.15.16:0/665154186 >> v2:172.21.15.52:3300/0 conn(0x7f4d90161690 0x7f4d900aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:43.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.610+0000 7f4d96200640 1 -- 172.21.15.16:0/665154186 >> 172.21.15.16:0/665154186 conn(0x7f4d900d1df0 msgr2=0x7f4d900d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:43.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.610+0000 7f4d96200640 1 -- 172.21.15.16:0/665154186 shutdown_connections 2024-09-10T21:16:43.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.610+0000 7f4d96200640 1 -- 172.21.15.16:0/665154186 wait complete. 2024-09-10T21:16:43.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d96200640 1 Processor -- start 2024-09-10T21:16:43.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d96200640 1 -- start start 2024-09-10T21:16:43.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d96200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d9006e230 0x7f4d90151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:43.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d96200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d900b3200 0x7f4d90152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:43.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d96200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90161690 0x7f4d9015dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:43.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d96200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9016b770 con 0x7f4d90161690 2024-09-10T21:16:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d96200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9016b5f0 con 0x7f4d9006e230 2024-09-10T21:16:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d96200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4d9016b8f0 con 0x7f4d900b3200 2024-09-10T21:16:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d8fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90161690 0x7f4d9015dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d8fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90161690 0x7f4d9015dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47660/0 (socket says 172.21.15.16:47660) 2024-09-10T21:16:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d8fe00640 1 -- 172.21.15.16:0/4148416016 learned_addr learned my addr 172.21.15.16:0/4148416016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d86a00640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d900b3200 0x7f4d90152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d8f400640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d9006e230 0x7f4d90151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d8fe00640 1 -- 172.21.15.16:0/4148416016 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d900b3200 msgr2=0x7f4d90152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d8fe00640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d900b3200 0x7f4d90152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d8fe00640 1 -- 172.21.15.16:0/4148416016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d9006e230 msgr2=0x7f4d90151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d8fe00640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d9006e230 0x7f4d90151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d8fe00640 1 -- 172.21.15.16:0/4148416016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d901f8140 con 0x7f4d90161690 2024-09-10T21:16:43.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d86a00640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d900b3200 0x7f4d90152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:43.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d8f400640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d9006e230 0x7f4d90151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:43.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d8fe00640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90161690 0x7f4d9015dc80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800cb10 tx=0x7f4d8800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:43.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d87e00640 1 -- 172.21.15.16:0/4148416016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88016070 con 0x7f4d90161690 2024-09-10T21:16:43.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d87e00640 1 -- 172.21.15.16:0/4148416016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d8800ce50 con 0x7f4d90161690 2024-09-10T21:16:43.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d87e00640 1 -- 172.21.15.16:0/4148416016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88003980 con 0x7f4d90161690 2024-09-10T21:16:43.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d96200640 1 -- 172.21.15.16:0/4148416016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d901f8400 con 0x7f4d90161690 2024-09-10T21:16:43.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.614+0000 7f4d96200640 1 -- 172.21.15.16:0/4148416016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d901f89c0 con 0x7f4d90161690 2024-09-10T21:16:43.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.618+0000 7f4d96200640 1 -- 172.21.15.16:0/4148416016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d9006dcf0 con 0x7f4d90161690 2024-09-10T21:16:43.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.618+0000 7f4d87e00640 1 -- 172.21.15.16:0/4148416016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4d8800a850 con 0x7f4d90161690 2024-09-10T21:16:43.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.618+0000 7f4d87e00640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4d6c0725e0 0x7f4d6c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:43.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.618+0000 7f4d87e00640 1 -- 172.21.15.16:0/4148416016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f4d88093160 con 0x7f4d90161690 2024-09-10T21:16:43.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.618+0000 7f4d8f400640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4d6c0725e0 0x7f4d6c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:43.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.618+0000 7f4d8f400640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4d6c0725e0 0x7f4d6c074aa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4d9009ffb0 tx=0x7f4d800091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:43.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.622+0000 7f4d87e00640 1 -- 172.21.15.16:0/4148416016 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4d8805e830 con 0x7f4d90161690 2024-09-10T21:16:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:43.830+0000 7f4d96200640 1 -- 172.21.15.16:0/4148416016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4d900cf9c0 con 0x7f4d90161690 2024-09-10T21:16:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.434+0000 7f4d87e00640 1 -- 172.21.15.16:0/4148416016 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) ==== 122+0+0 (secure 0 0 0) 0x7f4d8805e130 con 0x7f4d90161690 2024-09-10T21:16:44.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.438+0000 7f4d96200640 1 -- 172.21.15.16:0/4148416016 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4d6c0725e0 msgr2=0x7f4d6c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:44.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.438+0000 7f4d96200640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4d6c0725e0 0x7f4d6c074aa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4d9009ffb0 tx=0x7f4d800091e0 comp rx=0 tx=0).stop 2024-09-10T21:16:44.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.438+0000 7f4d96200640 1 -- 172.21.15.16:0/4148416016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90161690 msgr2=0x7f4d9015dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.438+0000 7f4d96200640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90161690 0x7f4d9015dc80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800cb10 tx=0x7f4d8800cc00 comp rx=0 tx=0).stop 2024-09-10T21:16:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.438+0000 7f4d96200640 1 -- 172.21.15.16:0/4148416016 shutdown_connections 2024-09-10T21:16:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.438+0000 7f4d96200640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4d6c0725e0 0x7f4d6c074aa0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.438+0000 7f4d96200640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d90161690 0x7f4d9015dc80 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.438+0000 7f4d96200640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d900b3200 0x7f4d90152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.438+0000 7f4d96200640 1 --2- 172.21.15.16:0/4148416016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d9006e230 0x7f4d90151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.438+0000 7f4d96200640 1 -- 172.21.15.16:0/4148416016 >> 172.21.15.16:0/4148416016 conn(0x7f4d900d1df0 msgr2=0x7f4d900b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.438+0000 7f4d96200640 1 -- 172.21.15.16:0/4148416016 shutdown_connections 2024-09-10T21:16:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.438+0000 7f4d96200640 1 -- 172.21.15.16:0/4148416016 wait complete. 2024-09-10T21:16:44.466 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:16:44.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:16:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 -- 172.21.15.16:0/2568220273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55840cea00 msgr2=0x7f558406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 --2- 172.21.15.16:0/2568220273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55840cea00 0x7f558406dcf0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f557c009c80 tx=0x7f557c01f560 comp rx=0 tx=0).stop 2024-09-10T21:16:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 -- 172.21.15.16:0/2568220273 shutdown_connections 2024-09-10T21:16:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 --2- 172.21.15.16:0/2568220273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55840cea00 0x7f558406dcf0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 --2- 172.21.15.16:0/2568220273 >> v2:172.21.15.52:3300/0 conn(0x7f5584161410 0x7f55840b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 -- 172.21.15.16:0/2568220273 >> 172.21.15.16:0/2568220273 conn(0x7f55840d1d00 msgr2=0x7f55840d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 -- 172.21.15.16:0/2568220273 shutdown_connections 2024-09-10T21:16:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 -- 172.21.15.16:0/2568220273 wait complete. 2024-09-10T21:16:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 Processor -- start 2024-09-10T21:16:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 -- start start 2024-09-10T21:16:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55840b2ed0 0x7f55840cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55840cea00 0x7f55841f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5584161410 0x7f55840c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f558406e3b0 con 0x7f55840b2ed0 2024-09-10T21:16:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f558406e230 con 0x7f5584161410 2024-09-10T21:16:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.614+0000 7f558a000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f558406e530 con 0x7f55840cea00 2024-09-10T21:16:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5588c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55840b2ed0 0x7f55840cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5588c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55840b2ed0 0x7f55840cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47676/0 (socket says 172.21.15.16:47676) 2024-09-10T21:16:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5588c00640 1 -- 172.21.15.16:0/2726244852 learned_addr learned my addr 172.21.15.16:0/2726244852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5589600640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5584161410 0x7f55840c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5583e00640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55840cea00 0x7f55841f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5588c00640 1 -- 172.21.15.16:0/2726244852 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55840cea00 msgr2=0x7f55841f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5588c00640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55840cea00 0x7f55841f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5588c00640 1 -- 172.21.15.16:0/2726244852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5584161410 msgr2=0x7f55840c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5588c00640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5584161410 0x7f55840c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5588c00640 1 -- 172.21.15.16:0/2726244852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55841f4920 con 0x7f55840b2ed0 2024-09-10T21:16:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5588c00640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55840b2ed0 0x7f55840cb590 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f557800d750 tx=0x7f557800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5580c00640 1 -- 172.21.15.16:0/2726244852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578017070 con 0x7f55840b2ed0 2024-09-10T21:16:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5580c00640 1 -- 172.21.15.16:0/2726244852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5578009d20 con 0x7f55840b2ed0 2024-09-10T21:16:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5580c00640 1 -- 172.21.15.16:0/2726244852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578005280 con 0x7f55840b2ed0 2024-09-10T21:16:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f558a000640 1 -- 172.21.15.16:0/2726244852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55841f4bb0 con 0x7f55840b2ed0 2024-09-10T21:16:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f558a000640 1 -- 172.21.15.16:0/2726244852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55841f5170 con 0x7f55840b2ed0 2024-09-10T21:16:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f558a000640 1 -- 172.21.15.16:0/2726244852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55841f53b0 con 0x7f55840b2ed0 2024-09-10T21:16:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.618+0000 7f5580c00640 1 -- 172.21.15.16:0/2726244852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5578005420 con 0x7f55840b2ed0 2024-09-10T21:16:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.622+0000 7f5580c00640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55640725e0 0x7f5564074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.622+0000 7f5580c00640 1 -- 172.21.15.16:0/2726244852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f5578093370 con 0x7f55840b2ed0 2024-09-10T21:16:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.622+0000 7f5583e00640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55640725e0 0x7f5564074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.622+0000 7f5583e00640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55640725e0 0x7f5564074aa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5584160770 tx=0x7f5570009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.626+0000 7f5580c00640 1 -- 172.21.15.16:0/2726244852 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f55841f53b0 con 0x7f55840b2ed0 2024-09-10T21:16:44.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.834+0000 7f558a000640 1 -- 172.21.15.16:0/2726244852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f558406dcf0 con 0x7f55840b2ed0 2024-09-10T21:16:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.834+0000 7f5580c00640 1 -- 172.21.15.16:0/2726244852 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5229 (secure 0 0 0) 0x7f557800de60 con 0x7f55840b2ed0 2024-09-10T21:16:44.838 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:44.838 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":62,"btime":"2024-09-10T21:16:44:430553+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/895621033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":895621033},{"type":"v1","addr":"172.21.15.52:6835","nonce":895621033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6789,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2281471863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2281471863},{"type":"v1","addr":"172.21.15.16:6835","nonce":2281471863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1879915011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1879915011},{"type":"v1","addr":"172.21.15.16:6837","nonce":1879915011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:16:40.823255+0000","modified":"2024-09-10T21:16:43.834820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135725":{"gid":135725,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/585458541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":585458541},{"type":"v1","addr":"172.21.15.52:6834","nonce":585458541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135725,"qdb_cluster":[135725]},"id":7}]} 2024-09-10T21:16:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 62 2024-09-10T21:16:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.838+0000 7f558a000640 1 -- 172.21.15.16:0/2726244852 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55640725e0 msgr2=0x7f5564074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.838+0000 7f558a000640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55640725e0 0x7f5564074aa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5584160770 tx=0x7f5570009040 comp rx=0 tx=0).stop 2024-09-10T21:16:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.838+0000 7f558a000640 1 -- 172.21.15.16:0/2726244852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55840b2ed0 msgr2=0x7f55840cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.838+0000 7f558a000640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55840b2ed0 0x7f55840cb590 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f557800d750 tx=0x7f557800dc10 comp rx=0 tx=0).stop 2024-09-10T21:16:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.838+0000 7f558a000640 1 -- 172.21.15.16:0/2726244852 shutdown_connections 2024-09-10T21:16:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.838+0000 7f558a000640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55640725e0 0x7f5564074aa0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.838+0000 7f558a000640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5584161410 0x7f55840c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.838+0000 7f558a000640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55840cea00 0x7f55841f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.838+0000 7f558a000640 1 --2- 172.21.15.16:0/2726244852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55840b2ed0 0x7f55840cb590 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.838+0000 7f558a000640 1 -- 172.21.15.16:0/2726244852 >> 172.21.15.16:0/2726244852 conn(0x7f55840d1d00 msgr2=0x7f55840cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.838+0000 7f558a000640 1 -- 172.21.15.16:0/2726244852 shutdown_connections 2024-09-10T21:16:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:44.838+0000 7f558a000640 1 -- 172.21.15.16:0/2726244852 wait complete. 2024-09-10T21:16:44.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:16:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.002+0000 7fe25ae00640 1 -- 172.21.15.16:0/1412398342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2540716f0 msgr2=0x7fe2540af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.002+0000 7fe25ae00640 1 --2- 172.21.15.16:0/1412398342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2540716f0 0x7fe2540af580 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009840 tx=0x7fe23c01c700 comp rx=0 tx=0).stop 2024-09-10T21:16:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.002+0000 7fe25ae00640 1 -- 172.21.15.16:0/1412398342 shutdown_connections 2024-09-10T21:16:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.002+0000 7fe25ae00640 1 --2- 172.21.15.16:0/1412398342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2540716f0 0x7fe2540af580 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.002+0000 7fe25ae00640 1 --2- 172.21.15.16:0/1412398342 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2540b13c0 0x7fe2540afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.002+0000 7fe25ae00640 1 -- 172.21.15.16:0/1412398342 >> 172.21.15.16:0/1412398342 conn(0x7fe25409b130 msgr2=0x7fe2540d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.002+0000 7fe25ae00640 1 -- 172.21.15.16:0/1412398342 shutdown_connections 2024-09-10T21:16:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.002+0000 7fe25ae00640 1 -- 172.21.15.16:0/1412398342 wait complete. 2024-09-10T21:16:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe25ae00640 1 Processor -- start 2024-09-10T21:16:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe25ae00640 1 -- start start 2024-09-10T21:16:45.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe25ae00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2540716f0 0x7fe25414d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:45.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe25ae00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2540b13c0 0x7fe2540b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:45.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe25ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25415f400 0x7fe2540cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:45.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe25ae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe254168fb0 con 0x7fe25415f400 2024-09-10T21:16:45.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe25ae00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe254168e30 con 0x7fe2540b13c0 2024-09-10T21:16:45.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe25ae00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe254169130 con 0x7fe2540716f0 2024-09-10T21:16:45.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe258e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25415f400 0x7fe2540cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:45.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe258e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25415f400 0x7fe2540cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47698/0 (socket says 172.21.15.16:47698) 2024-09-10T21:16:45.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe258e00640 1 -- 172.21.15.16:0/3690638575 learned_addr learned my addr 172.21.15.16:0/3690638575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:45.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe253e00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2540716f0 0x7fe25414d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:45.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe24be00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2540b13c0 0x7fe2540b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:45.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe258e00640 1 -- 172.21.15.16:0/3690638575 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2540716f0 msgr2=0x7fe25414d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe258e00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2540716f0 0x7fe25414d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe258e00640 1 -- 172.21.15.16:0/3690638575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2540b13c0 msgr2=0x7fe2540b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe258e00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2540b13c0 0x7fe2540b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe258e00640 1 -- 172.21.15.16:0/3690638575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2540cdf80 con 0x7fe25415f400 2024-09-10T21:16:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe24be00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2540b13c0 0x7fe2540b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe258e00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25415f400 0x7fe2540cd3b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00e9b0 tx=0x7fe24c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe250c00640 1 -- 172.21.15.16:0/3690638575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c004d30 con 0x7fe25415f400 2024-09-10T21:16:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe250c00640 1 -- 172.21.15.16:0/3690638575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe24c00cac0 con 0x7fe25415f400 2024-09-10T21:16:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe250c00640 1 -- 172.21.15.16:0/3690638575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c005020 con 0x7fe25415f400 2024-09-10T21:16:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe25ae00640 1 -- 172.21.15.16:0/3690638575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2540ce220 con 0x7fe25415f400 2024-09-10T21:16:45.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe25ae00640 1 -- 172.21.15.16:0/3690638575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2540ca2d0 con 0x7fe25415f400 2024-09-10T21:16:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.006+0000 7fe25ae00640 1 -- 172.21.15.16:0/3690638575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2540c89a0 con 0x7fe25415f400 2024-09-10T21:16:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.010+0000 7fe250c00640 1 -- 172.21.15.16:0/3690638575 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe24c022060 con 0x7fe25415f400 2024-09-10T21:16:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.010+0000 7fe250c00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe2300725e0 0x7fe230074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.010+0000 7fe250c00640 1 -- 172.21.15.16:0/3690638575 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fe24c093790 con 0x7fe25415f400 2024-09-10T21:16:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.010+0000 7fe253e00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe2300725e0 0x7fe230074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.010+0000 7fe253e00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe2300725e0 0x7fe230074aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe2440015c0 tx=0x7fe244009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:45.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.014+0000 7fe250c00640 1 -- 172.21.15.16:0/3690638575 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe24c016030 con 0x7fe25415f400 2024-09-10T21:16:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.186+0000 7fe25ae00640 1 -- 172.21.15.16:0/3690638575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe254070f90 con 0x7fe25415f400 2024-09-10T21:16:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.186+0000 7fe250c00640 1 -- 172.21.15.16:0/3690638575 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16620 (secure 0 0 0) 0x7fe24c05ee60 con 0x7fe25415f400 2024-09-10T21:16:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:45.191 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":92,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:16:42.404534+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:16:38.533164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T21:16:39.781552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:16:45.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.190+0000 7fe25ae00640 1 -- 172.21.15.16:0/3690638575 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe2300725e0 msgr2=0x7fe230074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.190+0000 7fe25ae00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe2300725e0 0x7fe230074aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe2440015c0 tx=0x7fe244009040 comp rx=0 tx=0).stop 2024-09-10T21:16:45.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.190+0000 7fe25ae00640 1 -- 172.21.15.16:0/3690638575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25415f400 msgr2=0x7fe2540cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.190+0000 7fe25ae00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25415f400 0x7fe2540cd3b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00e9b0 tx=0x7fe24c00ee70 comp rx=0 tx=0).stop 2024-09-10T21:16:45.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.190+0000 7fe25ae00640 1 -- 172.21.15.16:0/3690638575 shutdown_connections 2024-09-10T21:16:45.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.190+0000 7fe25ae00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe2300725e0 0x7fe230074aa0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.190+0000 7fe25ae00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25415f400 0x7fe2540cd3b0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.190+0000 7fe25ae00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2540b13c0 0x7fe2540b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.190+0000 7fe25ae00640 1 --2- 172.21.15.16:0/3690638575 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe2540716f0 0x7fe25414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.190+0000 7fe25ae00640 1 -- 172.21.15.16:0/3690638575 >> 172.21.15.16:0/3690638575 conn(0x7fe25409b130 msgr2=0x7fe25415e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:45.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.190+0000 7fe25ae00640 1 -- 172.21.15.16:0/3690638575 shutdown_connections 2024-09-10T21:16:45.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.190+0000 7fe25ae00640 1 -- 172.21.15.16:0/3690638575 wait complete. 2024-09-10T21:16:45.213 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:16:45.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:16:45.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 -- 172.21.15.16:0/937372212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90881599d0 msgr2=0x7f908815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 --2- 172.21.15.16:0/937372212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90881599d0 0x7f908815f830 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f90880caa90 tx=0x7f907801c830 comp rx=0 tx=0).stop 2024-09-10T21:16:45.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 -- 172.21.15.16:0/937372212 shutdown_connections 2024-09-10T21:16:45.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 --2- 172.21.15.16:0/937372212 >> v2:172.21.15.52:3300/0 conn(0x7f90880ca6b0 0x7f90880726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 --2- 172.21.15.16:0/937372212 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f908806e640 0x7f90880ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 --2- 172.21.15.16:0/937372212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90881599d0 0x7f908815f830 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 -- 172.21.15.16:0/937372212 >> 172.21.15.16:0/937372212 conn(0x7f90880cfb20 msgr2=0x7f90880cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:45.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 -- 172.21.15.16:0/937372212 shutdown_connections 2024-09-10T21:16:45.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 -- 172.21.15.16:0/937372212 wait complete. 2024-09-10T21:16:45.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 Processor -- start 2024-09-10T21:16:45.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 -- start start 2024-09-10T21:16:45.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908806e640 0x7f90880b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:45.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90880ca6b0 0x7f90881510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:45.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f90881599d0 0x7f90881f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:45.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90880742a0 con 0x7f90880ca6b0 2024-09-10T21:16:45.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9088074120 con 0x7f908806e640 2024-09-10T21:16:45.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f908d000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9088074420 con 0x7f90881599d0 2024-09-10T21:16:45.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f907d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90880ca6b0 0x7f90881510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:45.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f9085e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908806e640 0x7f90880b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:45.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f9086800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f90881599d0 0x7f90881f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:45.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f907d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90880ca6b0 0x7f90881510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47714/0 (socket says 172.21.15.16:47714) 2024-09-10T21:16:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f9085e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908806e640 0x7f90880b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:55246/0 (socket says 172.21.15.16:55246) 2024-09-10T21:16:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.358+0000 7f907d400640 1 -- 172.21.15.16:0/1474154073 learned_addr learned my addr 172.21.15.16:0/1474154073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.362+0000 7f907d400640 1 -- 172.21.15.16:0/1474154073 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f90881599d0 msgr2=0x7f90881f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.362+0000 7f907d400640 1 --2- 172.21.15.16:0/1474154073 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f90881599d0 0x7f90881f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.362+0000 7f907d400640 1 -- 172.21.15.16:0/1474154073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908806e640 msgr2=0x7f90880b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.362+0000 7f907d400640 1 --2- 172.21.15.16:0/1474154073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908806e640 0x7f90880b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.362+0000 7f907d400640 1 -- 172.21.15.16:0/1474154073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90880c6f20 con 0x7f90880ca6b0 2024-09-10T21:16:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.362+0000 7f907d400640 1 --2- 172.21.15.16:0/1474154073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90880ca6b0 0x7f90881510d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f907000c970 tx=0x7f907000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.362+0000 7f907ea00640 1 -- 172.21.15.16:0/1474154073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070007bb0 con 0x7f90880ca6b0 2024-09-10T21:16:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.362+0000 7f907ea00640 1 -- 172.21.15.16:0/1474154073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9070007e40 con 0x7f90880ca6b0 2024-09-10T21:16:45.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.362+0000 7f907ea00640 1 -- 172.21.15.16:0/1474154073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070004290 con 0x7f90880ca6b0 2024-09-10T21:16:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.362+0000 7f908d000640 1 -- 172.21.15.16:0/1474154073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90880c9920 con 0x7f90880ca6b0 2024-09-10T21:16:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.362+0000 7f908d000640 1 -- 172.21.15.16:0/1474154073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90880c7200 con 0x7f90880ca6b0 2024-09-10T21:16:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.362+0000 7f908d000640 1 -- 172.21.15.16:0/1474154073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9088074940 con 0x7f90880ca6b0 2024-09-10T21:16:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.362+0000 7f907ea00640 1 -- 172.21.15.16:0/1474154073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9070003600 con 0x7f90880ca6b0 2024-09-10T21:16:45.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.366+0000 7f907ea00640 1 --2- 172.21.15.16:0/1474154073 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90640725e0 0x7f9064074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:45.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.366+0000 7f907ea00640 1 -- 172.21.15.16:0/1474154073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f9070093800 con 0x7f90880ca6b0 2024-09-10T21:16:45.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.366+0000 7f9085e00640 1 --2- 172.21.15.16:0/1474154073 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90640725e0 0x7f9064074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:45.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.366+0000 7f9085e00640 1 --2- 172.21.15.16:0/1474154073 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90640725e0 0x7f9064074aa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f90780af000 tx=0x7f907801b070 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:45.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.370+0000 7f907ea00640 1 -- 172.21.15.16:0/1474154073 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9070014070 con 0x7f90880ca6b0 2024-09-10T21:16:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.542+0000 7f908d000640 1 -- 172.21.15.16:0/1474154073 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9088070ff0 con 0x7f90640725e0 2024-09-10T21:16:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.546+0000 7f907ea00640 1 -- 172.21.15.16:0/1474154073 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7f9088070ff0 con 0x7f90640725e0 2024-09-10T21:16:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:16:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.550+0000 7f908d000640 1 -- 172.21.15.16:0/1474154073 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90640725e0 msgr2=0x7f9064074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.550+0000 7f908d000640 1 --2- 172.21.15.16:0/1474154073 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90640725e0 0x7f9064074aa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f90780af000 tx=0x7f907801b070 comp rx=0 tx=0).stop 2024-09-10T21:16:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.550+0000 7f908d000640 1 -- 172.21.15.16:0/1474154073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90880ca6b0 msgr2=0x7f90881510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.550+0000 7f908d000640 1 --2- 172.21.15.16:0/1474154073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90880ca6b0 0x7f90881510d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f907000c970 tx=0x7f907000ce30 comp rx=0 tx=0).stop 2024-09-10T21:16:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.550+0000 7f908d000640 1 -- 172.21.15.16:0/1474154073 shutdown_connections 2024-09-10T21:16:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.550+0000 7f908d000640 1 --2- 172.21.15.16:0/1474154073 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90640725e0 0x7f9064074aa0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.550+0000 7f908d000640 1 --2- 172.21.15.16:0/1474154073 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f90881599d0 0x7f90881f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.550+0000 7f908d000640 1 --2- 172.21.15.16:0/1474154073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90880ca6b0 0x7f90881510d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.550+0000 7f908d000640 1 --2- 172.21.15.16:0/1474154073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908806e640 0x7f90880b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.550+0000 7f908d000640 1 -- 172.21.15.16:0/1474154073 >> 172.21.15.16:0/1474154073 conn(0x7f90880cfb20 msgr2=0x7f908815b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.550+0000 7f908d000640 1 -- 172.21.15.16:0/1474154073 shutdown_connections 2024-09-10T21:16:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.550+0000 7f908d000640 1 -- 172.21.15.16:0/1474154073 wait complete. 2024-09-10T21:16:45.574 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-10T21:16:45.544402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":272024,"kb_used_data":10020,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754702696,"statfs":{"total":773094113280,"available":772815560704,"internally_reserved":0,"allocated":10260480,"data_stored":5018144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001041"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994174+0000","last_change":"2024-09-10T21:16:41.423752+0000","last_active":"2024-09-10T21:16:42.994174+0000","last_peered":"2024-09-10T21:16:42.994174+0000","last_clean":"2024-09-10T21:16:42.994174+0000","last_became_active":"2024-09-10T21:16:41.423494+0000","last_became_peered":"2024-09-10T21:16:41.423494+0000","last_unstale":"2024-09-10T21:16:42.994174+0000","last_undegraded":"2024-09-10T21:16:42.994174+0000","last_fullsized":"2024-09-10T21:16:42.994174+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:37.210871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993972+0000","last_change":"2024-09-10T21:16:41.428205+0000","last_active":"2024-09-10T21:16:42.993972+0000","last_peered":"2024-09-10T21:16:42.993972+0000","last_clean":"2024-09-10T21:16:42.993972+0000","last_became_active":"2024-09-10T21:16:41.427942+0000","last_became_peered":"2024-09-10T21:16:41.427942+0000","last_unstale":"2024-09-10T21:16:42.993972+0000","last_undegraded":"2024-09-10T21:16:42.993972+0000","last_fullsized":"2024-09-10T21:16:42.993972+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:28.078428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716031+0000","last_change":"2024-09-10T21:16:41.436587+0000","last_active":"2024-09-10T21:16:42.716031+0000","last_peered":"2024-09-10T21:16:42.716031+0000","last_clean":"2024-09-10T21:16:42.716031+0000","last_became_active":"2024-09-10T21:16:41.436318+0000","last_became_peered":"2024-09-10T21:16:41.436318+0000","last_unstale":"2024-09-10T21:16:42.716031+0000","last_undegraded":"2024-09-10T21:16:42.716031+0000","last_fullsized":"2024-09-10T21:16:42.716031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:46.536158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994126+0000","last_change":"2024-09-10T21:16:41.438751+0000","last_active":"2024-09-10T21:16:42.994126+0000","last_peered":"2024-09-10T21:16:42.994126+0000","last_clean":"2024-09-10T21:16:42.994126+0000","last_became_active":"2024-09-10T21:16:41.438379+0000","last_became_peered":"2024-09-10T21:16:41.438379+0000","last_unstale":"2024-09-10T21:16:42.994126+0000","last_undegraded":"2024-09-10T21:16:42.994126+0000","last_fullsized":"2024-09-10T21:16:42.994126+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:58.747445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577120+0000","last_change":"2024-09-10T21:16:41.434896+0000","last_active":"2024-09-10T21:16:43.577120+0000","last_peered":"2024-09-10T21:16:43.577120+0000","last_clean":"2024-09-10T21:16:43.577120+0000","last_became_active":"2024-09-10T21:16:41.434616+0000","last_became_peered":"2024-09-10T21:16:41.434616+0000","last_unstale":"2024-09-10T21:16:43.577120+0000","last_undegraded":"2024-09-10T21:16:43.577120+0000","last_fullsized":"2024-09-10T21:16:43.577120+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:51.759196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577325+0000","last_change":"2024-09-10T21:16:41.434347+0000","last_active":"2024-09-10T21:16:43.577325+0000","last_peered":"2024-09-10T21:16:43.577325+0000","last_clean":"2024-09-10T21:16:43.577325+0000","last_became_active":"2024-09-10T21:16:41.434066+0000","last_became_peered":"2024-09-10T21:16:41.434066+0000","last_unstale":"2024-09-10T21:16:43.577325+0000","last_undegraded":"2024-09-10T21:16:43.577325+0000","last_fullsized":"2024-09-10T21:16:43.577325+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:13.060545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577472+0000","last_change":"2024-09-10T21:16:41.426916+0000","last_active":"2024-09-10T21:16:43.577472+0000","last_peered":"2024-09-10T21:16:43.577472+0000","last_clean":"2024-09-10T21:16:43.577472+0000","last_became_active":"2024-09-10T21:16:41.426635+0000","last_became_peered":"2024-09-10T21:16:41.426635+0000","last_unstale":"2024-09-10T21:16:43.577472+0000","last_undegraded":"2024-09-10T21:16:43.577472+0000","last_fullsized":"2024-09-10T21:16:43.577472+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:16.164214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716088+0000","last_change":"2024-09-10T21:16:41.440606+0000","last_active":"2024-09-10T21:16:42.716088+0000","last_peered":"2024-09-10T21:16:42.716088+0000","last_clean":"2024-09-10T21:16:42.716088+0000","last_became_active":"2024-09-10T21:16:41.440341+0000","last_became_peered":"2024-09-10T21:16:41.440341+0000","last_unstale":"2024-09-10T21:16:42.716088+0000","last_undegraded":"2024-09-10T21:16:42.716088+0000","last_fullsized":"2024-09-10T21:16:42.716088+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:37.536912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577540+0000","last_change":"2024-09-10T21:16:41.417164+0000","last_active":"2024-09-10T21:16:43.577540+0000","last_peered":"2024-09-10T21:16:43.577540+0000","last_clean":"2024-09-10T21:16:43.577540+0000","last_became_active":"2024-09-10T21:16:41.416874+0000","last_became_peered":"2024-09-10T21:16:41.416874+0000","last_unstale":"2024-09-10T21:16:43.577540+0000","last_undegraded":"2024-09-10T21:16:43.577540+0000","last_fullsized":"2024-09-10T21:16:43.577540+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:23.816050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409983+0000","last_change":"2024-09-10T21:16:41.426064+0000","last_active":"2024-09-10T21:16:42.409983+0000","last_peered":"2024-09-10T21:16:42.409983+0000","last_clean":"2024-09-10T21:16:42.409983+0000","last_became_active":"2024-09-10T21:16:41.425728+0000","last_became_peered":"2024-09-10T21:16:41.425728+0000","last_unstale":"2024-09-10T21:16:42.409983+0000","last_undegraded":"2024-09-10T21:16:42.409983+0000","last_fullsized":"2024-09-10T21:16:42.409983+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:30.322922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577290+0000","last_change":"2024-09-10T21:16:41.433075+0000","last_active":"2024-09-10T21:16:43.577290+0000","last_peered":"2024-09-10T21:16:43.577290+0000","last_clean":"2024-09-10T21:16:43.577290+0000","last_became_active":"2024-09-10T21:16:41.432842+0000","last_became_peered":"2024-09-10T21:16:41.432842+0000","last_unstale":"2024-09-10T21:16:43.577290+0000","last_undegraded":"2024-09-10T21:16:43.577290+0000","last_fullsized":"2024-09-10T21:16:43.577290+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:28.896020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.429743+0000","last_change":"2024-09-10T21:16:41.426102+0000","last_active":"2024-09-10T21:16:41.429743+0000","last_peered":"2024-09-10T21:16:41.429743+0000","last_clean":"2024-09-10T21:16:41.429743+0000","last_became_active":"2024-09-10T21:16:41.425670+0000","last_became_peered":"2024-09-10T21:16:41.425670+0000","last_unstale":"2024-09-10T21:16:41.429743+0000","last_undegraded":"2024-09-10T21:16:41.429743+0000","last_fullsized":"2024-09-10T21:16:41.429743+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:30.975065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.408951+0000","last_change":"2024-09-10T21:16:41.440725+0000","last_active":"2024-09-10T21:16:42.408951+0000","last_peered":"2024-09-10T21:16:42.408951+0000","last_clean":"2024-09-10T21:16:42.408951+0000","last_became_active":"2024-09-10T21:16:41.440424+0000","last_became_peered":"2024-09-10T21:16:41.440424+0000","last_unstale":"2024-09-10T21:16:42.408951+0000","last_undegraded":"2024-09-10T21:16:42.408951+0000","last_fullsized":"2024-09-10T21:16:42.408951+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:09.755499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993707+0000","last_change":"2024-09-10T21:16:41.437646+0000","last_active":"2024-09-10T21:16:42.993707+0000","last_peered":"2024-09-10T21:16:42.993707+0000","last_clean":"2024-09-10T21:16:42.993707+0000","last_became_active":"2024-09-10T21:16:41.436118+0000","last_became_peered":"2024-09-10T21:16:41.436118+0000","last_unstale":"2024-09-10T21:16:42.993707+0000","last_undegraded":"2024-09-10T21:16:42.993707+0000","last_fullsized":"2024-09-10T21:16:42.993707+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:20.366275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993923+0000","last_change":"2024-09-10T21:16:41.440098+0000","last_active":"2024-09-10T21:16:42.993923+0000","last_peered":"2024-09-10T21:16:42.993923+0000","last_clean":"2024-09-10T21:16:42.993923+0000","last_became_active":"2024-09-10T21:16:41.438021+0000","last_became_peered":"2024-09-10T21:16:41.438021+0000","last_unstale":"2024-09-10T21:16:42.993923+0000","last_undegraded":"2024-09-10T21:16:42.993923+0000","last_fullsized":"2024-09-10T21:16:42.993923+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:58.480418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577314+0000","last_change":"2024-09-10T21:16:41.414370+0000","last_active":"2024-09-10T21:16:43.577314+0000","last_peered":"2024-09-10T21:16:43.577314+0000","last_clean":"2024-09-10T21:16:43.577314+0000","last_became_active":"2024-09-10T21:16:41.414084+0000","last_became_peered":"2024-09-10T21:16:41.414084+0000","last_unstale":"2024-09-10T21:16:43.577314+0000","last_undegraded":"2024-09-10T21:16:43.577314+0000","last_fullsized":"2024-09-10T21:16:43.577314+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:27.159253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.419068+0000","last_change":"2024-09-10T21:16:41.440034+0000","last_active":"2024-09-10T21:16:43.419068+0000","last_peered":"2024-09-10T21:16:43.419068+0000","last_clean":"2024-09-10T21:16:43.419068+0000","last_became_active":"2024-09-10T21:16:41.439734+0000","last_became_peered":"2024-09-10T21:16:41.439734+0000","last_unstale":"2024-09-10T21:16:43.419068+0000","last_undegraded":"2024-09-10T21:16:43.419068+0000","last_fullsized":"2024-09-10T21:16:43.419068+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:37.688627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994011+0000","last_change":"2024-09-10T21:16:41.442571+0000","last_active":"2024-09-10T21:16:42.994011+0000","last_peered":"2024-09-10T21:16:42.994011+0000","last_clean":"2024-09-10T21:16:42.994011+0000","last_became_active":"2024-09-10T21:16:41.442254+0000","last_became_peered":"2024-09-10T21:16:41.442254+0000","last_unstale":"2024-09-10T21:16:42.994011+0000","last_undegraded":"2024-09-10T21:16:42.994011+0000","last_fullsized":"2024-09-10T21:16:42.994011+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:56.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418589+0000","last_change":"2024-09-10T21:16:41.423464+0000","last_active":"2024-09-10T21:16:43.418589+0000","last_peered":"2024-09-10T21:16:43.418589+0000","last_clean":"2024-09-10T21:16:43.418589+0000","last_became_active":"2024-09-10T21:16:41.423196+0000","last_became_peered":"2024-09-10T21:16:41.423196+0000","last_unstale":"2024-09-10T21:16:43.418589+0000","last_undegraded":"2024-09-10T21:16:43.418589+0000","last_fullsized":"2024-09-10T21:16:43.418589+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:39.236278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.445252+0000","last_change":"2024-09-10T21:16:41.444047+0000","last_active":"2024-09-10T21:16:41.445252+0000","last_peered":"2024-09-10T21:16:41.445252+0000","last_clean":"2024-09-10T21:16:41.445252+0000","last_became_active":"2024-09-10T21:16:41.442968+0000","last_became_peered":"2024-09-10T21:16:41.442968+0000","last_unstale":"2024-09-10T21:16:41.445252+0000","last_undegraded":"2024-09-10T21:16:41.445252+0000","last_fullsized":"2024-09-10T21:16:41.445252+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:02.662401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716434+0000","last_change":"2024-09-10T21:16:41.426567+0000","last_active":"2024-09-10T21:16:42.716434+0000","last_peered":"2024-09-10T21:16:42.716434+0000","last_clean":"2024-09-10T21:16:42.716434+0000","last_became_active":"2024-09-10T21:16:41.425971+0000","last_became_peered":"2024-09-10T21:16:41.425971+0000","last_unstale":"2024-09-10T21:16:42.716434+0000","last_undegraded":"2024-09-10T21:16:42.716434+0000","last_fullsized":"2024-09-10T21:16:42.716434+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:28.078428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.715986+0000","last_change":"2024-09-10T21:16:41.428575+0000","last_active":"2024-09-10T21:16:42.715986+0000","last_peered":"2024-09-10T21:16:42.715986+0000","last_clean":"2024-09-10T21:16:42.715986+0000","last_became_active":"2024-09-10T21:16:41.427872+0000","last_became_peered":"2024-09-10T21:16:41.427872+0000","last_unstale":"2024-09-10T21:16:42.715986+0000","last_undegraded":"2024-09-10T21:16:42.715986+0000","last_fullsized":"2024-09-10T21:16:42.715986+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:02.880139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994142+0000","last_change":"2024-09-10T21:16:41.434011+0000","last_active":"2024-09-10T21:16:42.994142+0000","last_peered":"2024-09-10T21:16:42.994142+0000","last_clean":"2024-09-10T21:16:42.994142+0000","last_became_active":"2024-09-10T21:16:41.433634+0000","last_became_peered":"2024-09-10T21:16:41.433634+0000","last_unstale":"2024-09-10T21:16:42.994142+0000","last_undegraded":"2024-09-10T21:16:42.994142+0000","last_fullsized":"2024-09-10T21:16:42.994142+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:38.449390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409568+0000","last_change":"2024-09-10T21:16:41.425079+0000","last_active":"2024-09-10T21:16:42.409568+0000","last_peered":"2024-09-10T21:16:42.409568+0000","last_clean":"2024-09-10T21:16:42.409568+0000","last_became_active":"2024-09-10T21:16:41.424723+0000","last_became_peered":"2024-09-10T21:16:41.424723+0000","last_unstale":"2024-09-10T21:16:42.409568+0000","last_undegraded":"2024-09-10T21:16:42.409568+0000","last_fullsized":"2024-09-10T21:16:42.409568+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:43.752527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.410117+0000","last_change":"2024-09-10T21:16:41.431761+0000","last_active":"2024-09-10T21:16:42.410117+0000","last_peered":"2024-09-10T21:16:42.410117+0000","last_clean":"2024-09-10T21:16:42.410117+0000","last_became_active":"2024-09-10T21:16:41.430910+0000","last_became_peered":"2024-09-10T21:16:41.430910+0000","last_unstale":"2024-09-10T21:16:42.410117+0000","last_undegraded":"2024-09-10T21:16:42.410117+0000","last_fullsized":"2024-09-10T21:16:42.410117+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:41.791454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409093+0000","last_change":"2024-09-10T21:16:41.447054+0000","last_active":"2024-09-10T21:16:42.409093+0000","last_peered":"2024-09-10T21:16:42.409093+0000","last_clean":"2024-09-10T21:16:42.409093+0000","last_became_active":"2024-09-10T21:16:41.445151+0000","last_became_peered":"2024-09-10T21:16:41.445151+0000","last_unstale":"2024-09-10T21:16:42.409093+0000","last_undegraded":"2024-09-10T21:16:42.409093+0000","last_fullsized":"2024-09-10T21:16:42.409093+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:11.443350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409143+0000","last_change":"2024-09-10T21:16:41.431004+0000","last_active":"2024-09-10T21:16:42.409143+0000","last_peered":"2024-09-10T21:16:42.409143+0000","last_clean":"2024-09-10T21:16:42.409143+0000","last_became_active":"2024-09-10T21:16:41.430720+0000","last_became_peered":"2024-09-10T21:16:41.430720+0000","last_unstale":"2024-09-10T21:16:42.409143+0000","last_undegraded":"2024-09-10T21:16:42.409143+0000","last_fullsized":"2024-09-10T21:16:42.409143+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:14.238476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409104+0000","last_change":"2024-09-10T21:16:41.429634+0000","last_active":"2024-09-10T21:16:42.409104+0000","last_peered":"2024-09-10T21:16:42.409104+0000","last_clean":"2024-09-10T21:16:42.409104+0000","last_became_active":"2024-09-10T21:16:41.429375+0000","last_became_peered":"2024-09-10T21:16:41.429375+0000","last_unstale":"2024-09-10T21:16:42.409104+0000","last_undegraded":"2024-09-10T21:16:42.409104+0000","last_fullsized":"2024-09-10T21:16:42.409104+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:40.455286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409160+0000","last_change":"2024-09-10T21:16:41.440627+0000","last_active":"2024-09-10T21:16:42.409160+0000","last_peered":"2024-09-10T21:16:42.409160+0000","last_clean":"2024-09-10T21:16:42.409160+0000","last_became_active":"2024-09-10T21:16:41.439989+0000","last_became_peered":"2024-09-10T21:16:41.439989+0000","last_unstale":"2024-09-10T21:16:42.409160+0000","last_undegraded":"2024-09-10T21:16:42.409160+0000","last_fullsized":"2024-09-10T21:16:42.409160+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:38.967436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.408861+0000","last_change":"2024-09-10T21:16:41.444696+0000","last_active":"2024-09-10T21:16:42.408861+0000","last_peered":"2024-09-10T21:16:42.408861+0000","last_clean":"2024-09-10T21:16:42.408861+0000","last_became_active":"2024-09-10T21:16:41.444358+0000","last_became_peered":"2024-09-10T21:16:41.444358+0000","last_unstale":"2024-09-10T21:16:42.408861+0000","last_undegraded":"2024-09-10T21:16:42.408861+0000","last_fullsized":"2024-09-10T21:16:42.408861+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:58.598089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716262+0000","last_change":"2024-09-10T21:16:41.429087+0000","last_active":"2024-09-10T21:16:42.716262+0000","last_peered":"2024-09-10T21:16:42.716262+0000","last_clean":"2024-09-10T21:16:42.716262+0000","last_became_active":"2024-09-10T21:16:41.426303+0000","last_became_peered":"2024-09-10T21:16:41.426303+0000","last_unstale":"2024-09-10T21:16:42.716262+0000","last_undegraded":"2024-09-10T21:16:42.716262+0000","last_fullsized":"2024-09-10T21:16:42.716262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:00.068361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993967+0000","last_change":"2024-09-10T21:16:41.433802+0000","last_active":"2024-09-10T21:16:42.993967+0000","last_peered":"2024-09-10T21:16:42.993967+0000","last_clean":"2024-09-10T21:16:42.993967+0000","last_became_active":"2024-09-10T21:16:41.433094+0000","last_became_peered":"2024-09-10T21:16:41.433094+0000","last_unstale":"2024-09-10T21:16:42.993967+0000","last_undegraded":"2024-09-10T21:16:42.993967+0000","last_fullsized":"2024-09-10T21:16:42.993967+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:33.343383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418915+0000","last_change":"2024-09-10T21:16:41.425350+0000","last_active":"2024-09-10T21:16:43.418915+0000","last_peered":"2024-09-10T21:16:43.418915+0000","last_clean":"2024-09-10T21:16:43.418915+0000","last_became_active":"2024-09-10T21:16:41.425074+0000","last_became_peered":"2024-09-10T21:16:41.425074+0000","last_unstale":"2024-09-10T21:16:43.418915+0000","last_undegraded":"2024-09-10T21:16:43.418915+0000","last_fullsized":"2024-09-10T21:16:43.418915+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:02.490697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"91'4","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409230+0000","last_change":"2024-09-10T21:16:40.428950+0000","last_active":"2024-09-10T21:16:42.409230+0000","last_peered":"2024-09-10T21:16:42.409230+0000","last_clean":"2024-09-10T21:16:42.409230+0000","last_became_active":"2024-09-10T21:16:40.428382+0000","last_became_peered":"2024-09-10T21:16:40.428382+0000","last_unstale":"2024-09-10T21:16:42.409230+0000","last_undegraded":"2024-09-10T21:16:42.409230+0000","last_fullsized":"2024-09-10T21:16:42.409230+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:56.746204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409288+0000","last_change":"2024-09-10T21:16:41.438522+0000","last_active":"2024-09-10T21:16:42.409288+0000","last_peered":"2024-09-10T21:16:42.409288+0000","last_clean":"2024-09-10T21:16:42.409288+0000","last_became_active":"2024-09-10T21:16:41.438268+0000","last_became_peered":"2024-09-10T21:16:41.438268+0000","last_unstale":"2024-09-10T21:16:42.409288+0000","last_undegraded":"2024-09-10T21:16:42.409288+0000","last_fullsized":"2024-09-10T21:16:42.409288+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:53.733853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577055+0000","last_change":"2024-09-10T21:16:40.428909+0000","last_active":"2024-09-10T21:16:43.577055+0000","last_peered":"2024-09-10T21:16:43.577055+0000","last_clean":"2024-09-10T21:16:43.577055+0000","last_became_active":"2024-09-10T21:16:40.428630+0000","last_became_peered":"2024-09-10T21:16:40.428630+0000","last_unstale":"2024-09-10T21:16:43.577055+0000","last_undegraded":"2024-09-10T21:16:43.577055+0000","last_fullsized":"2024-09-10T21:16:43.577055+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:35.886637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994373+0000","last_change":"2024-09-10T21:16:41.425798+0000","last_active":"2024-09-10T21:16:42.994373+0000","last_peered":"2024-09-10T21:16:42.994373+0000","last_clean":"2024-09-10T21:16:42.994373+0000","last_became_active":"2024-09-10T21:16:41.425522+0000","last_became_peered":"2024-09-10T21:16:41.425522+0000","last_unstale":"2024-09-10T21:16:42.994373+0000","last_undegraded":"2024-09-10T21:16:42.994373+0000","last_fullsized":"2024-09-10T21:16:42.994373+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:51.184355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.715874+0000","last_change":"2024-09-10T21:16:40.425190+0000","last_active":"2024-09-10T21:16:42.715874+0000","last_peered":"2024-09-10T21:16:42.715874+0000","last_clean":"2024-09-10T21:16:42.715874+0000","last_became_active":"2024-09-10T21:16:40.424907+0000","last_became_peered":"2024-09-10T21:16:40.424907+0000","last_unstale":"2024-09-10T21:16:42.715874+0000","last_undegraded":"2024-09-10T21:16:42.715874+0000","last_fullsized":"2024-09-10T21:16:42.715874+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:20.839168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409967+0000","last_change":"2024-09-10T21:16:41.423976+0000","last_active":"2024-09-10T21:16:42.409967+0000","last_peered":"2024-09-10T21:16:42.409967+0000","last_clean":"2024-09-10T21:16:42.409967+0000","last_became_active":"2024-09-10T21:16:41.423532+0000","last_became_peered":"2024-09-10T21:16:41.423532+0000","last_unstale":"2024-09-10T21:16:42.409967+0000","last_undegraded":"2024-09-10T21:16:42.409967+0000","last_fullsized":"2024-09-10T21:16:42.409967+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:01.515800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994302+0000","last_change":"2024-09-10T21:16:40.427336+0000","last_active":"2024-09-10T21:16:42.994302+0000","last_peered":"2024-09-10T21:16:42.994302+0000","last_clean":"2024-09-10T21:16:42.994302+0000","last_became_active":"2024-09-10T21:16:40.427035+0000","last_became_peered":"2024-09-10T21:16:40.427035+0000","last_unstale":"2024-09-10T21:16:42.994302+0000","last_undegraded":"2024-09-10T21:16:42.994302+0000","last_fullsized":"2024-09-10T21:16:42.994302+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:20.839168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.439242+0000","last_change":"2024-09-10T21:16:41.437269+0000","last_active":"2024-09-10T21:16:41.439242+0000","last_peered":"2024-09-10T21:16:41.439242+0000","last_clean":"2024-09-10T21:16:41.439242+0000","last_became_active":"2024-09-10T21:16:41.436719+0000","last_became_peered":"2024-09-10T21:16:41.436719+0000","last_unstale":"2024-09-10T21:16:41.439242+0000","last_undegraded":"2024-09-10T21:16:41.439242+0000","last_fullsized":"2024-09-10T21:16:41.439242+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:28:42.112331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993559+0000","last_change":"2024-09-10T21:16:40.420080+0000","last_active":"2024-09-10T21:16:42.993559+0000","last_peered":"2024-09-10T21:16:42.993559+0000","last_clean":"2024-09-10T21:16:42.993559+0000","last_became_active":"2024-09-10T21:16:40.419674+0000","last_became_peered":"2024-09-10T21:16:40.419674+0000","last_unstale":"2024-09-10T21:16:42.993559+0000","last_undegraded":"2024-09-10T21:16:42.993559+0000","last_fullsized":"2024-09-10T21:16:42.993559+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:57.278459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577213+0000","last_change":"2024-09-10T21:16:41.431653+0000","last_active":"2024-09-10T21:16:43.577213+0000","last_peered":"2024-09-10T21:16:43.577213+0000","last_clean":"2024-09-10T21:16:43.577213+0000","last_became_active":"2024-09-10T21:16:41.431357+0000","last_became_peered":"2024-09-10T21:16:41.431357+0000","last_unstale":"2024-09-10T21:16:43.577213+0000","last_undegraded":"2024-09-10T21:16:43.577213+0000","last_fullsized":"2024-09-10T21:16:43.577213+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:48.253724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.715735+0000","last_change":"2024-09-10T21:16:40.429453+0000","last_active":"2024-09-10T21:16:42.715735+0000","last_peered":"2024-09-10T21:16:42.715735+0000","last_clean":"2024-09-10T21:16:42.715735+0000","last_became_active":"2024-09-10T21:16:40.429183+0000","last_became_peered":"2024-09-10T21:16:40.429183+0000","last_unstale":"2024-09-10T21:16:42.715735+0000","last_undegraded":"2024-09-10T21:16:42.715735+0000","last_fullsized":"2024-09-10T21:16:42.715735+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:19.094153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.408567+0000","last_change":"2024-09-10T21:16:41.443869+0000","last_active":"2024-09-10T21:16:42.408567+0000","last_peered":"2024-09-10T21:16:42.408567+0000","last_clean":"2024-09-10T21:16:42.408567+0000","last_became_active":"2024-09-10T21:16:41.443593+0000","last_became_peered":"2024-09-10T21:16:41.443593+0000","last_unstale":"2024-09-10T21:16:42.408567+0000","last_undegraded":"2024-09-10T21:16:42.408567+0000","last_fullsized":"2024-09-10T21:16:42.408567+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:11.319468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.576989+0000","last_change":"2024-09-10T21:16:40.428282+0000","last_active":"2024-09-10T21:16:43.576989+0000","last_peered":"2024-09-10T21:16:43.576989+0000","last_clean":"2024-09-10T21:16:43.576989+0000","last_became_active":"2024-09-10T21:16:40.427970+0000","last_became_peered":"2024-09-10T21:16:40.427970+0000","last_unstale":"2024-09-10T21:16:43.576989+0000","last_undegraded":"2024-09-10T21:16:43.576989+0000","last_fullsized":"2024-09-10T21:16:43.576989+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:04.829527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.441077+0000","last_change":"2024-09-10T21:16:41.438427+0000","last_active":"2024-09-10T21:16:41.441077+0000","last_peered":"2024-09-10T21:16:41.441077+0000","last_clean":"2024-09-10T21:16:41.441077+0000","last_became_active":"2024-09-10T21:16:41.438089+0000","last_became_peered":"2024-09-10T21:16:41.438089+0000","last_unstale":"2024-09-10T21:16:41.441077+0000","last_undegraded":"2024-09-10T21:16:41.441077+0000","last_fullsized":"2024-09-10T21:16:41.441077+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:08.631169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418068+0000","last_change":"2024-09-10T21:16:40.418368+0000","last_active":"2024-09-10T21:16:43.418068+0000","last_peered":"2024-09-10T21:16:43.418068+0000","last_clean":"2024-09-10T21:16:43.418068+0000","last_became_active":"2024-09-10T21:16:40.417936+0000","last_became_peered":"2024-09-10T21:16:40.417936+0000","last_unstale":"2024-09-10T21:16:43.418068+0000","last_undegraded":"2024-09-10T21:16:43.418068+0000","last_fullsized":"2024-09-10T21:16:43.418068+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:36.208124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994265+0000","last_change":"2024-09-10T21:16:41.433447+0000","last_active":"2024-09-10T21:16:42.994265+0000","last_peered":"2024-09-10T21:16:42.994265+0000","last_clean":"2024-09-10T21:16:42.994265+0000","last_became_active":"2024-09-10T21:16:41.433157+0000","last_became_peered":"2024-09-10T21:16:41.433157+0000","last_unstale":"2024-09-10T21:16:42.994265+0000","last_undegraded":"2024-09-10T21:16:42.994265+0000","last_fullsized":"2024-09-10T21:16:42.994265+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:42.591342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716068+0000","last_change":"2024-09-10T21:16:40.423681+0000","last_active":"2024-09-10T21:16:42.716068+0000","last_peered":"2024-09-10T21:16:42.716068+0000","last_clean":"2024-09-10T21:16:42.716068+0000","last_became_active":"2024-09-10T21:16:40.423401+0000","last_became_peered":"2024-09-10T21:16:40.423401+0000","last_unstale":"2024-09-10T21:16:42.716068+0000","last_undegraded":"2024-09-10T21:16:42.716068+0000","last_fullsized":"2024-09-10T21:16:42.716068+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:15.821464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409469+0000","last_change":"2024-09-10T21:16:41.427532+0000","last_active":"2024-09-10T21:16:42.409469+0000","last_peered":"2024-09-10T21:16:42.409469+0000","last_clean":"2024-09-10T21:16:42.409469+0000","last_became_active":"2024-09-10T21:16:41.426963+0000","last_became_peered":"2024-09-10T21:16:41.426963+0000","last_unstale":"2024-09-10T21:16:42.409469+0000","last_undegraded":"2024-09-10T21:16:42.409469+0000","last_fullsized":"2024-09-10T21:16:42.409469+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:14.204203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994158+0000","last_change":"2024-09-10T21:16:40.427823+0000","last_active":"2024-09-10T21:16:42.994158+0000","last_peered":"2024-09-10T21:16:42.994158+0000","last_clean":"2024-09-10T21:16:42.994158+0000","last_became_active":"2024-09-10T21:16:40.427245+0000","last_became_peered":"2024-09-10T21:16:40.427245+0000","last_unstale":"2024-09-10T21:16:42.994158+0000","last_undegraded":"2024-09-10T21:16:42.994158+0000","last_fullsized":"2024-09-10T21:16:42.994158+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:19.094153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577466+0000","last_change":"2024-09-10T21:16:41.425413+0000","last_active":"2024-09-10T21:16:43.577466+0000","last_peered":"2024-09-10T21:16:43.577466+0000","last_clean":"2024-09-10T21:16:43.577466+0000","last_became_active":"2024-09-10T21:16:41.423100+0000","last_became_peered":"2024-09-10T21:16:41.423100+0000","last_unstale":"2024-09-10T21:16:43.577466+0000","last_undegraded":"2024-09-10T21:16:43.577466+0000","last_fullsized":"2024-09-10T21:16:43.577466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:08.007186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"91'3","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577588+0000","last_change":"2024-09-10T21:16:40.424166+0000","last_active":"2024-09-10T21:16:43.577588+0000","last_peered":"2024-09-10T21:16:43.577588+0000","last_clean":"2024-09-10T21:16:43.577588+0000","last_became_active":"2024-09-10T21:16:40.423841+0000","last_became_peered":"2024-09-10T21:16:40.423841+0000","last_unstale":"2024-09-10T21:16:43.577588+0000","last_undegraded":"2024-09-10T21:16:43.577588+0000","last_fullsized":"2024-09-10T21:16:43.577588+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:49.846416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409358+0000","last_change":"2024-09-10T21:16:41.422291+0000","last_active":"2024-09-10T21:16:42.409358+0000","last_peered":"2024-09-10T21:16:42.409358+0000","last_clean":"2024-09-10T21:16:42.409358+0000","last_became_active":"2024-09-10T21:16:41.421664+0000","last_became_peered":"2024-09-10T21:16:41.421664+0000","last_unstale":"2024-09-10T21:16:42.409358+0000","last_undegraded":"2024-09-10T21:16:42.409358+0000","last_fullsized":"2024-09-10T21:16:42.409358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:34.519014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"91'6","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.450639+0000","last_change":"2024-09-10T21:16:40.428188+0000","last_active":"2024-09-10T21:16:41.450639+0000","last_peered":"2024-09-10T21:16:41.450639+0000","last_clean":"2024-09-10T21:16:41.450639+0000","last_became_active":"2024-09-10T21:16:40.427871+0000","last_became_peered":"2024-09-10T21:16:40.427871+0000","last_unstale":"2024-09-10T21:16:41.450639+0000","last_undegraded":"2024-09-10T21:16:41.450639+0000","last_fullsized":"2024-09-10T21:16:41.450639+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:16.689109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993884+0000","last_change":"2024-09-10T21:16:41.422770+0000","last_active":"2024-09-10T21:16:42.993884+0000","last_peered":"2024-09-10T21:16:42.993884+0000","last_clean":"2024-09-10T21:16:42.993884+0000","last_became_active":"2024-09-10T21:16:41.422454+0000","last_became_peered":"2024-09-10T21:16:41.422454+0000","last_unstale":"2024-09-10T21:16:42.993884+0000","last_undegraded":"2024-09-10T21:16:42.993884+0000","last_fullsized":"2024-09-10T21:16:42.993884+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:49.093104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418280+0000","last_change":"2024-09-10T21:16:40.424578+0000","last_active":"2024-09-10T21:16:43.418280+0000","last_peered":"2024-09-10T21:16:43.418280+0000","last_clean":"2024-09-10T21:16:43.418280+0000","last_became_active":"2024-09-10T21:16:40.424242+0000","last_became_peered":"2024-09-10T21:16:40.424242+0000","last_unstale":"2024-09-10T21:16:43.418280+0000","last_undegraded":"2024-09-10T21:16:43.418280+0000","last_fullsized":"2024-09-10T21:16:43.418280+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:17.071296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716475+0000","last_change":"2024-09-10T21:16:41.420701+0000","last_active":"2024-09-10T21:16:42.716475+0000","last_peered":"2024-09-10T21:16:42.716475+0000","last_clean":"2024-09-10T21:16:42.716475+0000","last_became_active":"2024-09-10T21:16:41.420312+0000","last_became_peered":"2024-09-10T21:16:41.420312+0000","last_unstale":"2024-09-10T21:16:42.716475+0000","last_undegraded":"2024-09-10T21:16:42.716475+0000","last_fullsized":"2024-09-10T21:16:42.716475+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:56.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993859+0000","last_change":"2024-09-10T21:16:40.431389+0000","last_active":"2024-09-10T21:16:42.993859+0000","last_peered":"2024-09-10T21:16:42.993859+0000","last_clean":"2024-09-10T21:16:42.993859+0000","last_became_active":"2024-09-10T21:16:40.430988+0000","last_became_peered":"2024-09-10T21:16:40.430988+0000","last_unstale":"2024-09-10T21:16:42.993859+0000","last_undegraded":"2024-09-10T21:16:42.993859+0000","last_fullsized":"2024-09-10T21:16:42.993859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:15.821464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577597+0000","last_change":"2024-09-10T21:16:41.434437+0000","last_active":"2024-09-10T21:16:43.577597+0000","last_peered":"2024-09-10T21:16:43.577597+0000","last_clean":"2024-09-10T21:16:43.577597+0000","last_became_active":"2024-09-10T21:16:41.434128+0000","last_became_peered":"2024-09-10T21:16:41.434128+0000","last_unstale":"2024-09-10T21:16:43.577597+0000","last_undegraded":"2024-09-10T21:16:43.577597+0000","last_fullsized":"2024-09-10T21:16:43.577597+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:12.163978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.418653+0000","last_change":"2024-09-10T21:16:40.423517+0000","last_active":"2024-09-10T21:16:41.418653+0000","last_peered":"2024-09-10T21:16:41.418653+0000","last_clean":"2024-09-10T21:16:41.418653+0000","last_became_active":"2024-09-10T21:16:40.423179+0000","last_became_peered":"2024-09-10T21:16:40.423179+0000","last_unstale":"2024-09-10T21:16:41.418653+0000","last_undegraded":"2024-09-10T21:16:41.418653+0000","last_fullsized":"2024-09-10T21:16:41.418653+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:12.096394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716141+0000","last_change":"2024-09-10T21:16:41.434641+0000","last_active":"2024-09-10T21:16:42.716141+0000","last_peered":"2024-09-10T21:16:42.716141+0000","last_clean":"2024-09-10T21:16:42.716141+0000","last_became_active":"2024-09-10T21:16:41.429531+0000","last_became_peered":"2024-09-10T21:16:41.429531+0000","last_unstale":"2024-09-10T21:16:42.716141+0000","last_undegraded":"2024-09-10T21:16:42.716141+0000","last_fullsized":"2024-09-10T21:16:42.716141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:27.207535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"91'8","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994136+0000","last_change":"2024-09-10T21:16:40.422493+0000","last_active":"2024-09-10T21:16:42.994136+0000","last_peered":"2024-09-10T21:16:42.994136+0000","last_clean":"2024-09-10T21:16:42.994136+0000","last_became_active":"2024-09-10T21:16:40.422131+0000","last_became_peered":"2024-09-10T21:16:40.422131+0000","last_unstale":"2024-09-10T21:16:42.994136+0000","last_undegraded":"2024-09-10T21:16:42.994136+0000","last_fullsized":"2024-09-10T21:16:42.994136+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:29.923997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409780+0000","last_change":"2024-09-10T21:16:41.427245+0000","last_active":"2024-09-10T21:16:42.409780+0000","last_peered":"2024-09-10T21:16:42.409780+0000","last_clean":"2024-09-10T21:16:42.409780+0000","last_became_active":"2024-09-10T21:16:41.426624+0000","last_became_peered":"2024-09-10T21:16:41.426624+0000","last_unstale":"2024-09-10T21:16:42.409780+0000","last_undegraded":"2024-09-10T21:16:42.409780+0000","last_fullsized":"2024-09-10T21:16:42.409780+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:17.160075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"91'2","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.442183+0000","last_change":"2024-09-10T21:16:40.428161+0000","last_active":"2024-09-10T21:16:41.442183+0000","last_peered":"2024-09-10T21:16:41.442183+0000","last_clean":"2024-09-10T21:16:41.442183+0000","last_became_active":"2024-09-10T21:16:40.427892+0000","last_became_peered":"2024-09-10T21:16:40.427892+0000","last_unstale":"2024-09-10T21:16:41.442183+0000","last_undegraded":"2024-09-10T21:16:41.442183+0000","last_fullsized":"2024-09-10T21:16:41.442183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:12.829874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409367+0000","last_change":"2024-09-10T21:16:41.427978+0000","last_active":"2024-09-10T21:16:42.409367+0000","last_peered":"2024-09-10T21:16:42.409367+0000","last_clean":"2024-09-10T21:16:42.409367+0000","last_became_active":"2024-09-10T21:16:41.427247+0000","last_became_peered":"2024-09-10T21:16:41.427247+0000","last_unstale":"2024-09-10T21:16:42.409367+0000","last_undegraded":"2024-09-10T21:16:42.409367+0000","last_fullsized":"2024-09-10T21:16:42.409367+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:35.359461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577495+0000","last_change":"2024-09-10T21:16:40.423515+0000","last_active":"2024-09-10T21:16:43.577495+0000","last_peered":"2024-09-10T21:16:43.577495+0000","last_clean":"2024-09-10T21:16:43.577495+0000","last_became_active":"2024-09-10T21:16:40.423214+0000","last_became_peered":"2024-09-10T21:16:40.423214+0000","last_unstale":"2024-09-10T21:16:43.577495+0000","last_undegraded":"2024-09-10T21:16:43.577495+0000","last_fullsized":"2024-09-10T21:16:43.577495+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:52.123893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.441923+0000","last_change":"2024-09-10T21:16:41.438731+0000","last_active":"2024-09-10T21:16:41.441923+0000","last_peered":"2024-09-10T21:16:41.441923+0000","last_clean":"2024-09-10T21:16:41.441923+0000","last_became_active":"2024-09-10T21:16:41.437383+0000","last_became_peered":"2024-09-10T21:16:41.437383+0000","last_unstale":"2024-09-10T21:16:41.441923+0000","last_undegraded":"2024-09-10T21:16:41.441923+0000","last_fullsized":"2024-09-10T21:16:41.441923+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:58.782973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418481+0000","last_change":"2024-09-10T21:16:40.422872+0000","last_active":"2024-09-10T21:16:43.418481+0000","last_peered":"2024-09-10T21:16:43.418481+0000","last_clean":"2024-09-10T21:16:43.418481+0000","last_became_active":"2024-09-10T21:16:40.422557+0000","last_became_peered":"2024-09-10T21:16:40.422557+0000","last_unstale":"2024-09-10T21:16:43.418481+0000","last_undegraded":"2024-09-10T21:16:43.418481+0000","last_fullsized":"2024-09-10T21:16:43.418481+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:00.308807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418131+0000","last_change":"2024-09-10T21:16:41.428480+0000","last_active":"2024-09-10T21:16:43.418131+0000","last_peered":"2024-09-10T21:16:43.418131+0000","last_clean":"2024-09-10T21:16:43.418131+0000","last_became_active":"2024-09-10T21:16:41.428211+0000","last_became_peered":"2024-09-10T21:16:41.428211+0000","last_unstale":"2024-09-10T21:16:43.418131+0000","last_undegraded":"2024-09-10T21:16:43.418131+0000","last_fullsized":"2024-09-10T21:16:43.418131+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:35.128280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993604+0000","last_change":"2024-09-10T21:16:40.427377+0000","last_active":"2024-09-10T21:16:42.993604+0000","last_peered":"2024-09-10T21:16:42.993604+0000","last_clean":"2024-09-10T21:16:42.993604+0000","last_became_active":"2024-09-10T21:16:40.427084+0000","last_became_peered":"2024-09-10T21:16:40.427084+0000","last_unstale":"2024-09-10T21:16:42.993604+0000","last_undegraded":"2024-09-10T21:16:42.993604+0000","last_fullsized":"2024-09-10T21:16:42.993604+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:47.824551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418289+0000","last_change":"2024-09-10T21:16:41.440826+0000","last_active":"2024-09-10T21:16:43.418289+0000","last_peered":"2024-09-10T21:16:43.418289+0000","last_clean":"2024-09-10T21:16:43.418289+0000","last_became_active":"2024-09-10T21:16:41.440512+0000","last_became_peered":"2024-09-10T21:16:41.440512+0000","last_unstale":"2024-09-10T21:16:43.418289+0000","last_undegraded":"2024-09-10T21:16:43.418289+0000","last_fullsized":"2024-09-10T21:16:43.418289+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:09.617565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.408881+0000","last_change":"2024-09-10T21:16:40.432497+0000","last_active":"2024-09-10T21:16:42.408881+0000","last_peered":"2024-09-10T21:16:42.408881+0000","last_clean":"2024-09-10T21:16:42.408881+0000","last_became_active":"2024-09-10T21:16:40.432182+0000","last_became_peered":"2024-09-10T21:16:40.432182+0000","last_unstale":"2024-09-10T21:16:42.408881+0000","last_undegraded":"2024-09-10T21:16:42.408881+0000","last_fullsized":"2024-09-10T21:16:42.408881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:29.320175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994269+0000","last_change":"2024-09-10T21:16:41.431642+0000","last_active":"2024-09-10T21:16:42.994269+0000","last_peered":"2024-09-10T21:16:42.994269+0000","last_clean":"2024-09-10T21:16:42.994269+0000","last_became_active":"2024-09-10T21:16:41.431353+0000","last_became_peered":"2024-09-10T21:16:41.431353+0000","last_unstale":"2024-09-10T21:16:42.994269+0000","last_undegraded":"2024-09-10T21:16:42.994269+0000","last_fullsized":"2024-09-10T21:16:42.994269+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:14.550533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409747+0000","last_change":"2024-09-10T21:16:40.428078+0000","last_active":"2024-09-10T21:16:42.409747+0000","last_peered":"2024-09-10T21:16:42.409747+0000","last_clean":"2024-09-10T21:16:42.409747+0000","last_became_active":"2024-09-10T21:16:40.427704+0000","last_became_peered":"2024-09-10T21:16:40.427704+0000","last_unstale":"2024-09-10T21:16:42.409747+0000","last_undegraded":"2024-09-10T21:16:42.409747+0000","last_fullsized":"2024-09-10T21:16:42.409747+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:44.983295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.408958+0000","last_change":"2024-09-10T21:16:41.442032+0000","last_active":"2024-09-10T21:16:42.408958+0000","last_peered":"2024-09-10T21:16:42.408958+0000","last_clean":"2024-09-10T21:16:42.408958+0000","last_became_active":"2024-09-10T21:16:41.441718+0000","last_became_peered":"2024-09-10T21:16:41.441718+0000","last_unstale":"2024-09-10T21:16:42.408958+0000","last_undegraded":"2024-09-10T21:16:42.408958+0000","last_fullsized":"2024-09-10T21:16:42.408958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:12.027369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.715609+0000","last_change":"2024-09-10T21:16:40.420060+0000","last_active":"2024-09-10T21:16:42.715609+0000","last_peered":"2024-09-10T21:16:42.715609+0000","last_clean":"2024-09-10T21:16:42.715609+0000","last_became_active":"2024-09-10T21:16:40.419491+0000","last_became_peered":"2024-09-10T21:16:40.419491+0000","last_unstale":"2024-09-10T21:16:42.715609+0000","last_undegraded":"2024-09-10T21:16:42.715609+0000","last_fullsized":"2024-09-10T21:16:42.715609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:57.278459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418951+0000","last_change":"2024-09-10T21:16:41.430611+0000","last_active":"2024-09-10T21:16:43.418951+0000","last_peered":"2024-09-10T21:16:43.418951+0000","last_clean":"2024-09-10T21:16:43.418951+0000","last_became_active":"2024-09-10T21:16:41.430001+0000","last_became_peered":"2024-09-10T21:16:41.430001+0000","last_unstale":"2024-09-10T21:16:43.418951+0000","last_undegraded":"2024-09-10T21:16:43.418951+0000","last_fullsized":"2024-09-10T21:16:43.418951+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:31.634366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"91'2","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.446581+0000","last_change":"2024-09-10T21:16:40.423506+0000","last_active":"2024-09-10T21:16:41.446581+0000","last_peered":"2024-09-10T21:16:41.446581+0000","last_clean":"2024-09-10T21:16:41.446581+0000","last_became_active":"2024-09-10T21:16:40.422925+0000","last_became_peered":"2024-09-10T21:16:40.422925+0000","last_unstale":"2024-09-10T21:16:41.446581+0000","last_undegraded":"2024-09-10T21:16:41.446581+0000","last_fullsized":"2024-09-10T21:16:41.446581+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:35.124240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.419066+0000","last_change":"2024-09-10T21:16:41.427878+0000","last_active":"2024-09-10T21:16:43.419066+0000","last_peered":"2024-09-10T21:16:43.419066+0000","last_clean":"2024-09-10T21:16:43.419066+0000","last_became_active":"2024-09-10T21:16:41.427616+0000","last_became_peered":"2024-09-10T21:16:41.427616+0000","last_unstale":"2024-09-10T21:16:43.419066+0000","last_undegraded":"2024-09-10T21:16:43.419066+0000","last_fullsized":"2024-09-10T21:16:43.419066+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:59.659550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"92'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.420523+0000","last_change":"2024-09-10T21:16:40.421767+0000","last_active":"2024-09-10T21:16:43.420523+0000","last_peered":"2024-09-10T21:16:43.420523+0000","last_clean":"2024-09-10T21:16:43.420523+0000","last_became_active":"2024-09-10T21:16:40.421481+0000","last_became_peered":"2024-09-10T21:16:40.421481+0000","last_unstale":"2024-09-10T21:16:43.420523+0000","last_undegraded":"2024-09-10T21:16:43.420523+0000","last_fullsized":"2024-09-10T21:16:43.420523+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:13.547786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716038+0000","last_change":"2024-09-10T21:16:41.439815+0000","last_active":"2024-09-10T21:16:42.716038+0000","last_peered":"2024-09-10T21:16:42.716038+0000","last_clean":"2024-09-10T21:16:42.716038+0000","last_became_active":"2024-09-10T21:16:41.439134+0000","last_became_peered":"2024-09-10T21:16:41.439134+0000","last_unstale":"2024-09-10T21:16:42.716038+0000","last_undegraded":"2024-09-10T21:16:42.716038+0000","last_fullsized":"2024-09-10T21:16:42.716038+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:47.005003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418678+0000","last_change":"2024-09-10T21:16:40.409420+0000","last_active":"2024-09-10T21:16:43.418678+0000","last_peered":"2024-09-10T21:16:43.418678+0000","last_clean":"2024-09-10T21:16:43.418678+0000","last_became_active":"2024-09-10T21:16:40.409105+0000","last_became_peered":"2024-09-10T21:16:40.409105+0000","last_unstale":"2024-09-10T21:16:43.418678+0000","last_undegraded":"2024-09-10T21:16:43.418678+0000","last_fullsized":"2024-09-10T21:16:43.418678+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:50.181608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577527+0000","last_change":"2024-09-10T21:16:41.427746+0000","last_active":"2024-09-10T21:16:43.577527+0000","last_peered":"2024-09-10T21:16:43.577527+0000","last_clean":"2024-09-10T21:16:43.577527+0000","last_became_active":"2024-09-10T21:16:41.427457+0000","last_became_peered":"2024-09-10T21:16:41.427457+0000","last_unstale":"2024-09-10T21:16:43.577527+0000","last_undegraded":"2024-09-10T21:16:43.577527+0000","last_fullsized":"2024-09-10T21:16:43.577527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:07.209017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409245+0000","last_change":"2024-09-10T21:16:40.421114+0000","last_active":"2024-09-10T21:16:42.409245+0000","last_peered":"2024-09-10T21:16:42.409245+0000","last_clean":"2024-09-10T21:16:42.409245+0000","last_became_active":"2024-09-10T21:16:40.420628+0000","last_became_peered":"2024-09-10T21:16:40.420628+0000","last_unstale":"2024-09-10T21:16:42.409245+0000","last_undegraded":"2024-09-10T21:16:42.409245+0000","last_fullsized":"2024-09-10T21:16:42.409245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:05.194554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418341+0000","last_change":"2024-09-10T21:16:41.429688+0000","last_active":"2024-09-10T21:16:43.418341+0000","last_peered":"2024-09-10T21:16:43.418341+0000","last_clean":"2024-09-10T21:16:43.418341+0000","last_became_active":"2024-09-10T21:16:41.429399+0000","last_became_peered":"2024-09-10T21:16:41.429399+0000","last_unstale":"2024-09-10T21:16:43.418341+0000","last_undegraded":"2024-09-10T21:16:43.418341+0000","last_fullsized":"2024-09-10T21:16:43.418341+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:44.235771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716548+0000","last_change":"2024-09-10T21:16:40.425176+0000","last_active":"2024-09-10T21:16:42.716548+0000","last_peered":"2024-09-10T21:16:42.716548+0000","last_clean":"2024-09-10T21:16:42.716548+0000","last_became_active":"2024-09-10T21:16:40.424894+0000","last_became_peered":"2024-09-10T21:16:40.424894+0000","last_unstale":"2024-09-10T21:16:42.716548+0000","last_undegraded":"2024-09-10T21:16:42.716548+0000","last_fullsized":"2024-09-10T21:16:42.716548+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:16.920456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577362+0000","last_change":"2024-09-10T21:16:41.435563+0000","last_active":"2024-09-10T21:16:43.577362+0000","last_peered":"2024-09-10T21:16:43.577362+0000","last_clean":"2024-09-10T21:16:43.577362+0000","last_became_active":"2024-09-10T21:16:41.435299+0000","last_became_peered":"2024-09-10T21:16:41.435299+0000","last_unstale":"2024-09-10T21:16:43.577362+0000","last_undegraded":"2024-09-10T21:16:43.577362+0000","last_fullsized":"2024-09-10T21:16:43.577362+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:04.510513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.408918+0000","last_change":"2024-09-10T21:16:40.416609+0000","last_active":"2024-09-10T21:16:42.408918+0000","last_peered":"2024-09-10T21:16:42.408918+0000","last_clean":"2024-09-10T21:16:42.408918+0000","last_became_active":"2024-09-10T21:16:40.416278+0000","last_became_peered":"2024-09-10T21:16:40.416278+0000","last_unstale":"2024-09-10T21:16:42.408918+0000","last_undegraded":"2024-09-10T21:16:42.408918+0000","last_fullsized":"2024-09-10T21:16:42.408918+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:40.788707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418206+0000","last_change":"2024-09-10T21:16:41.418720+0000","last_active":"2024-09-10T21:16:43.418206+0000","last_peered":"2024-09-10T21:16:43.418206+0000","last_clean":"2024-09-10T21:16:43.418206+0000","last_became_active":"2024-09-10T21:16:41.418184+0000","last_became_peered":"2024-09-10T21:16:41.418184+0000","last_unstale":"2024-09-10T21:16:43.418206+0000","last_undegraded":"2024-09-10T21:16:43.418206+0000","last_fullsized":"2024-09-10T21:16:43.418206+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:26.250271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409007+0000","last_change":"2024-09-10T21:16:40.429961+0000","last_active":"2024-09-10T21:16:42.409007+0000","last_peered":"2024-09-10T21:16:42.409007+0000","last_clean":"2024-09-10T21:16:42.409007+0000","last_became_active":"2024-09-10T21:16:40.429695+0000","last_became_peered":"2024-09-10T21:16:40.429695+0000","last_unstale":"2024-09-10T21:16:42.409007+0000","last_undegraded":"2024-09-10T21:16:42.409007+0000","last_fullsized":"2024-09-10T21:16:42.409007+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:00.686235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.428138+0000","last_change":"2024-09-10T21:16:41.425291+0000","last_active":"2024-09-10T21:16:41.428138+0000","last_peered":"2024-09-10T21:16:41.428138+0000","last_clean":"2024-09-10T21:16:41.428138+0000","last_became_active":"2024-09-10T21:16:41.424986+0000","last_became_peered":"2024-09-10T21:16:41.424986+0000","last_unstale":"2024-09-10T21:16:41.428138+0000","last_undegraded":"2024-09-10T21:16:41.428138+0000","last_fullsized":"2024-09-10T21:16:41.428138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:53.979110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.715868+0000","last_change":"2024-09-10T21:16:40.419789+0000","last_active":"2024-09-10T21:16:42.715868+0000","last_peered":"2024-09-10T21:16:42.715868+0000","last_clean":"2024-09-10T21:16:42.715868+0000","last_became_active":"2024-09-10T21:16:40.419132+0000","last_became_peered":"2024-09-10T21:16:40.419132+0000","last_unstale":"2024-09-10T21:16:42.715868+0000","last_undegraded":"2024-09-10T21:16:42.715868+0000","last_fullsized":"2024-09-10T21:16:42.715868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:29.923997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716296+0000","last_change":"2024-09-10T21:16:41.428232+0000","last_active":"2024-09-10T21:16:42.716296+0000","last_peered":"2024-09-10T21:16:42.716296+0000","last_clean":"2024-09-10T21:16:42.716296+0000","last_became_active":"2024-09-10T21:16:41.427936+0000","last_became_peered":"2024-09-10T21:16:41.427936+0000","last_unstale":"2024-09-10T21:16:42.716296+0000","last_undegraded":"2024-09-10T21:16:42.716296+0000","last_fullsized":"2024-09-10T21:16:42.716296+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:33.343383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"91'1","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.444514+0000","last_change":"2024-09-10T21:16:40.430194+0000","last_active":"2024-09-10T21:16:41.444514+0000","last_peered":"2024-09-10T21:16:41.444514+0000","last_clean":"2024-09-10T21:16:41.444514+0000","last_became_active":"2024-09-10T21:16:40.429293+0000","last_became_peered":"2024-09-10T21:16:40.429293+0000","last_unstale":"2024-09-10T21:16:41.444514+0000","last_undegraded":"2024-09-10T21:16:41.444514+0000","last_fullsized":"2024-09-10T21:16:41.444514+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:23.946617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":234,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.419435+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:16:41.419435+0000","last_peered":"2024-09-10T21:16:41.419435+0000","last_clean":"2024-09-10T21:16:41.419435+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:16:41.419435+0000","last_undegraded":"2024-09-10T21:16:41.419435+0000","last_fullsized":"2024-09-10T21:16:41.419435+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509520,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36452,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335388,"statfs":{"total":96636764160,"available":96599437312,"internally_reserved":0,"allocated":1130496,"data_stored":481436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509522,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1732608,"data_stored":1067172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509520,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1134592,"data_stored":482018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509519,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":475330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1134592,"data_stored":481506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509520,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":481436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542224,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36456,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335384,"statfs":{"total":96636764160,"available":96599433216,"internally_reserved":0,"allocated":1134592,"data_stored":482018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37048,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334792,"statfs":{"total":96636764160,"available":96598827008,"internally_reserved":0,"allocated":1732608,"data_stored":1067228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:16:45.579 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:16:45.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.762+0000 7f029ac00640 1 -- 172.21.15.16:0/4072837908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f027c004d50 msgr2=0x7f027c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.762+0000 7f029ac00640 1 --2- 172.21.15.16:0/4072837908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f027c004d50 0x7f027c0051d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f028c00c610 tx=0x7f028c01d2a0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.762+0000 7f029a200640 1 -- 172.21.15.16:0/4072837908 reap_dead start 2024-09-10T21:16:45.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.762+0000 7f029ac00640 1 -- 172.21.15.16:0/4072837908 shutdown_connections 2024-09-10T21:16:45.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.762+0000 7f029ac00640 1 -- 172.21.15.16:0/4072837908 >> 172.21.15.16:0/4072837908 conn(0x7f02940d3f00 msgr2=0x7f02940d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:45.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.762+0000 7f029ac00640 1 -- 172.21.15.16:0/4072837908 shutdown_connections 2024-09-10T21:16:45.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.762+0000 7f029ac00640 1 -- 172.21.15.16:0/4072837908 wait complete. 2024-09-10T21:16:45.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029ac00640 1 Processor -- start 2024-09-10T21:16:45.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029ac00640 1 -- start start 2024-09-10T21:16:45.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029ac00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c004d50 0x7f02940c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029ac00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02941f5320 0x7f02940c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029ac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02941f7bb0 0x7f02941f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029ac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02940d4a20 con 0x7f02941f7bb0 2024-09-10T21:16:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029ac00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f02940d12a0 con 0x7f027c004d50 2024-09-10T21:16:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029ac00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f02940d4ba0 con 0x7f02941f5320 2024-09-10T21:16:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02941f7bb0 0x7f02941f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02941f7bb0 0x7f02941f8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47720/0 (socket says 172.21.15.16:47720) 2024-09-10T21:16:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029a200640 1 -- 172.21.15.16:0/4100219786 learned_addr learned my addr 172.21.15.16:0/4100219786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f0299800640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c004d50 0x7f02940c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f0298e00640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02941f5320 0x7f02940c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:45.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029a200640 1 -- 172.21.15.16:0/4100219786 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02941f5320 msgr2=0x7f02940c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029a200640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02941f5320 0x7f02940c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029a200640 1 -- 172.21.15.16:0/4100219786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c004d50 msgr2=0x7f02940c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029a200640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c004d50 0x7f02940c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029a200640 1 -- 172.21.15.16:0/4100219786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02941f4590 con 0x7f02941f7bb0 2024-09-10T21:16:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029a200640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02941f7bb0 0x7f02941f8040 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f028c098e70 tx=0x7f028c006580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f0291600640 1 -- 172.21.15.16:0/4100219786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c004e60 con 0x7f02941f7bb0 2024-09-10T21:16:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f0291600640 1 -- 172.21.15.16:0/4100219786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f028c0065f0 con 0x7f02941f7bb0 2024-09-10T21:16:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f0291600640 1 -- 172.21.15.16:0/4100219786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c001fd0 con 0x7f02941f7bb0 2024-09-10T21:16:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.766+0000 7f029ac00640 1 -- 172.21.15.16:0/4100219786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02941f88c0 con 0x7f02941f7bb0 2024-09-10T21:16:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.770+0000 7f029ac00640 1 -- 172.21.15.16:0/4100219786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02941f4b20 con 0x7f02941f7bb0 2024-09-10T21:16:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.770+0000 7f029ac00640 1 -- 172.21.15.16:0/4100219786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02940d0740 con 0x7f02941f7bb0 2024-09-10T21:16:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.770+0000 7f0291600640 1 -- 172.21.15.16:0/4100219786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f028c0c2070 con 0x7f02941f7bb0 2024-09-10T21:16:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.770+0000 7f0291600640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0274072610 0x7f0274074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.770+0000 7f0291600640 1 -- 172.21.15.16:0/4100219786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f028c12d8b0 con 0x7f02941f7bb0 2024-09-10T21:16:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.770+0000 7f0299800640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0274072610 0x7f0274074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.770+0000 7f0299800640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0274072610 0x7f0274074ad0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f029415df80 tx=0x7f02880091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.774+0000 7f0291600640 1 -- 172.21.15.16:0/4100219786 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f028c0bf060 con 0x7f02941f7bb0 2024-09-10T21:16:45.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.946+0000 7f029ac00640 1 -- 172.21.15.16:0/4100219786 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02940d45f0 con 0x7f0274072610 2024-09-10T21:16:45.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.950+0000 7f0291600640 1 -- 172.21.15.16:0/4100219786 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7f028805e8c0 con 0x7f0274072610 2024-09-10T21:16:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:16:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.954+0000 7f029ac00640 1 -- 172.21.15.16:0/4100219786 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0274072610 msgr2=0x7f0274074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.954+0000 7f029ac00640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0274072610 0x7f0274074ad0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f029415df80 tx=0x7f02880091e0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.954+0000 7f029ac00640 1 -- 172.21.15.16:0/4100219786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02941f7bb0 msgr2=0x7f02941f8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.954+0000 7f029ac00640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02941f7bb0 0x7f02941f8040 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f028c098e70 tx=0x7f028c006580 comp rx=0 tx=0).stop 2024-09-10T21:16:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.954+0000 7f029ac00640 1 -- 172.21.15.16:0/4100219786 shutdown_connections 2024-09-10T21:16:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.954+0000 7f029ac00640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0274072610 0x7f0274074ad0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.954+0000 7f029ac00640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02941f7bb0 0x7f02941f8040 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.954+0000 7f029ac00640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f02941f5320 0x7f02940c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.954+0000 7f029ac00640 1 --2- 172.21.15.16:0/4100219786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c004d50 0x7f02940c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.954+0000 7f029ac00640 1 -- 172.21.15.16:0/4100219786 >> 172.21.15.16:0/4100219786 conn(0x7f02940d3f00 msgr2=0x7f029415f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.954+0000 7f029ac00640 1 -- 172.21.15.16:0/4100219786 shutdown_connections 2024-09-10T21:16:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:45.954+0000 7f029ac00640 1 -- 172.21.15.16:0/4100219786 wait complete. 2024-09-10T21:16:45.976 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-10T21:16:45.544402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":272024,"kb_used_data":10020,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754702696,"statfs":{"total":773094113280,"available":772815560704,"internally_reserved":0,"allocated":10260480,"data_stored":5018144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001041"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994174+0000","last_change":"2024-09-10T21:16:41.423752+0000","last_active":"2024-09-10T21:16:42.994174+0000","last_peered":"2024-09-10T21:16:42.994174+0000","last_clean":"2024-09-10T21:16:42.994174+0000","last_became_active":"2024-09-10T21:16:41.423494+0000","last_became_peered":"2024-09-10T21:16:41.423494+0000","last_unstale":"2024-09-10T21:16:42.994174+0000","last_undegraded":"2024-09-10T21:16:42.994174+0000","last_fullsized":"2024-09-10T21:16:42.994174+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:37.210871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993972+0000","last_change":"2024-09-10T21:16:41.428205+0000","last_active":"2024-09-10T21:16:42.993972+0000","last_peered":"2024-09-10T21:16:42.993972+0000","last_clean":"2024-09-10T21:16:42.993972+0000","last_became_active":"2024-09-10T21:16:41.427942+0000","last_became_peered":"2024-09-10T21:16:41.427942+0000","last_unstale":"2024-09-10T21:16:42.993972+0000","last_undegraded":"2024-09-10T21:16:42.993972+0000","last_fullsized":"2024-09-10T21:16:42.993972+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:28.078428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716031+0000","last_change":"2024-09-10T21:16:41.436587+0000","last_active":"2024-09-10T21:16:42.716031+0000","last_peered":"2024-09-10T21:16:42.716031+0000","last_clean":"2024-09-10T21:16:42.716031+0000","last_became_active":"2024-09-10T21:16:41.436318+0000","last_became_peered":"2024-09-10T21:16:41.436318+0000","last_unstale":"2024-09-10T21:16:42.716031+0000","last_undegraded":"2024-09-10T21:16:42.716031+0000","last_fullsized":"2024-09-10T21:16:42.716031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:46.536158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994126+0000","last_change":"2024-09-10T21:16:41.438751+0000","last_active":"2024-09-10T21:16:42.994126+0000","last_peered":"2024-09-10T21:16:42.994126+0000","last_clean":"2024-09-10T21:16:42.994126+0000","last_became_active":"2024-09-10T21:16:41.438379+0000","last_became_peered":"2024-09-10T21:16:41.438379+0000","last_unstale":"2024-09-10T21:16:42.994126+0000","last_undegraded":"2024-09-10T21:16:42.994126+0000","last_fullsized":"2024-09-10T21:16:42.994126+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:58.747445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577120+0000","last_change":"2024-09-10T21:16:41.434896+0000","last_active":"2024-09-10T21:16:43.577120+0000","last_peered":"2024-09-10T21:16:43.577120+0000","last_clean":"2024-09-10T21:16:43.577120+0000","last_became_active":"2024-09-10T21:16:41.434616+0000","last_became_peered":"2024-09-10T21:16:41.434616+0000","last_unstale":"2024-09-10T21:16:43.577120+0000","last_undegraded":"2024-09-10T21:16:43.577120+0000","last_fullsized":"2024-09-10T21:16:43.577120+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:51.759196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577325+0000","last_change":"2024-09-10T21:16:41.434347+0000","last_active":"2024-09-10T21:16:43.577325+0000","last_peered":"2024-09-10T21:16:43.577325+0000","last_clean":"2024-09-10T21:16:43.577325+0000","last_became_active":"2024-09-10T21:16:41.434066+0000","last_became_peered":"2024-09-10T21:16:41.434066+0000","last_unstale":"2024-09-10T21:16:43.577325+0000","last_undegraded":"2024-09-10T21:16:43.577325+0000","last_fullsized":"2024-09-10T21:16:43.577325+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:13.060545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577472+0000","last_change":"2024-09-10T21:16:41.426916+0000","last_active":"2024-09-10T21:16:43.577472+0000","last_peered":"2024-09-10T21:16:43.577472+0000","last_clean":"2024-09-10T21:16:43.577472+0000","last_became_active":"2024-09-10T21:16:41.426635+0000","last_became_peered":"2024-09-10T21:16:41.426635+0000","last_unstale":"2024-09-10T21:16:43.577472+0000","last_undegraded":"2024-09-10T21:16:43.577472+0000","last_fullsized":"2024-09-10T21:16:43.577472+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:16.164214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716088+0000","last_change":"2024-09-10T21:16:41.440606+0000","last_active":"2024-09-10T21:16:42.716088+0000","last_peered":"2024-09-10T21:16:42.716088+0000","last_clean":"2024-09-10T21:16:42.716088+0000","last_became_active":"2024-09-10T21:16:41.440341+0000","last_became_peered":"2024-09-10T21:16:41.440341+0000","last_unstale":"2024-09-10T21:16:42.716088+0000","last_undegraded":"2024-09-10T21:16:42.716088+0000","last_fullsized":"2024-09-10T21:16:42.716088+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:37.536912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577540+0000","last_change":"2024-09-10T21:16:41.417164+0000","last_active":"2024-09-10T21:16:43.577540+0000","last_peered":"2024-09-10T21:16:43.577540+0000","last_clean":"2024-09-10T21:16:43.577540+0000","last_became_active":"2024-09-10T21:16:41.416874+0000","last_became_peered":"2024-09-10T21:16:41.416874+0000","last_unstale":"2024-09-10T21:16:43.577540+0000","last_undegraded":"2024-09-10T21:16:43.577540+0000","last_fullsized":"2024-09-10T21:16:43.577540+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:23.816050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409983+0000","last_change":"2024-09-10T21:16:41.426064+0000","last_active":"2024-09-10T21:16:42.409983+0000","last_peered":"2024-09-10T21:16:42.409983+0000","last_clean":"2024-09-10T21:16:42.409983+0000","last_became_active":"2024-09-10T21:16:41.425728+0000","last_became_peered":"2024-09-10T21:16:41.425728+0000","last_unstale":"2024-09-10T21:16:42.409983+0000","last_undegraded":"2024-09-10T21:16:42.409983+0000","last_fullsized":"2024-09-10T21:16:42.409983+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:30.322922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577290+0000","last_change":"2024-09-10T21:16:41.433075+0000","last_active":"2024-09-10T21:16:43.577290+0000","last_peered":"2024-09-10T21:16:43.577290+0000","last_clean":"2024-09-10T21:16:43.577290+0000","last_became_active":"2024-09-10T21:16:41.432842+0000","last_became_peered":"2024-09-10T21:16:41.432842+0000","last_unstale":"2024-09-10T21:16:43.577290+0000","last_undegraded":"2024-09-10T21:16:43.577290+0000","last_fullsized":"2024-09-10T21:16:43.577290+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:28.896020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.429743+0000","last_change":"2024-09-10T21:16:41.426102+0000","last_active":"2024-09-10T21:16:41.429743+0000","last_peered":"2024-09-10T21:16:41.429743+0000","last_clean":"2024-09-10T21:16:41.429743+0000","last_became_active":"2024-09-10T21:16:41.425670+0000","last_became_peered":"2024-09-10T21:16:41.425670+0000","last_unstale":"2024-09-10T21:16:41.429743+0000","last_undegraded":"2024-09-10T21:16:41.429743+0000","last_fullsized":"2024-09-10T21:16:41.429743+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:30.975065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.408951+0000","last_change":"2024-09-10T21:16:41.440725+0000","last_active":"2024-09-10T21:16:42.408951+0000","last_peered":"2024-09-10T21:16:42.408951+0000","last_clean":"2024-09-10T21:16:42.408951+0000","last_became_active":"2024-09-10T21:16:41.440424+0000","last_became_peered":"2024-09-10T21:16:41.440424+0000","last_unstale":"2024-09-10T21:16:42.408951+0000","last_undegraded":"2024-09-10T21:16:42.408951+0000","last_fullsized":"2024-09-10T21:16:42.408951+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:09.755499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993707+0000","last_change":"2024-09-10T21:16:41.437646+0000","last_active":"2024-09-10T21:16:42.993707+0000","last_peered":"2024-09-10T21:16:42.993707+0000","last_clean":"2024-09-10T21:16:42.993707+0000","last_became_active":"2024-09-10T21:16:41.436118+0000","last_became_peered":"2024-09-10T21:16:41.436118+0000","last_unstale":"2024-09-10T21:16:42.993707+0000","last_undegraded":"2024-09-10T21:16:42.993707+0000","last_fullsized":"2024-09-10T21:16:42.993707+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:20.366275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993923+0000","last_change":"2024-09-10T21:16:41.440098+0000","last_active":"2024-09-10T21:16:42.993923+0000","last_peered":"2024-09-10T21:16:42.993923+0000","last_clean":"2024-09-10T21:16:42.993923+0000","last_became_active":"2024-09-10T21:16:41.438021+0000","last_became_peered":"2024-09-10T21:16:41.438021+0000","last_unstale":"2024-09-10T21:16:42.993923+0000","last_undegraded":"2024-09-10T21:16:42.993923+0000","last_fullsized":"2024-09-10T21:16:42.993923+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:58.480418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577314+0000","last_change":"2024-09-10T21:16:41.414370+0000","last_active":"2024-09-10T21:16:43.577314+0000","last_peered":"2024-09-10T21:16:43.577314+0000","last_clean":"2024-09-10T21:16:43.577314+0000","last_became_active":"2024-09-10T21:16:41.414084+0000","last_became_peered":"2024-09-10T21:16:41.414084+0000","last_unstale":"2024-09-10T21:16:43.577314+0000","last_undegraded":"2024-09-10T21:16:43.577314+0000","last_fullsized":"2024-09-10T21:16:43.577314+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:27.159253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.419068+0000","last_change":"2024-09-10T21:16:41.440034+0000","last_active":"2024-09-10T21:16:43.419068+0000","last_peered":"2024-09-10T21:16:43.419068+0000","last_clean":"2024-09-10T21:16:43.419068+0000","last_became_active":"2024-09-10T21:16:41.439734+0000","last_became_peered":"2024-09-10T21:16:41.439734+0000","last_unstale":"2024-09-10T21:16:43.419068+0000","last_undegraded":"2024-09-10T21:16:43.419068+0000","last_fullsized":"2024-09-10T21:16:43.419068+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:37.688627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994011+0000","last_change":"2024-09-10T21:16:41.442571+0000","last_active":"2024-09-10T21:16:42.994011+0000","last_peered":"2024-09-10T21:16:42.994011+0000","last_clean":"2024-09-10T21:16:42.994011+0000","last_became_active":"2024-09-10T21:16:41.442254+0000","last_became_peered":"2024-09-10T21:16:41.442254+0000","last_unstale":"2024-09-10T21:16:42.994011+0000","last_undegraded":"2024-09-10T21:16:42.994011+0000","last_fullsized":"2024-09-10T21:16:42.994011+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:56.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418589+0000","last_change":"2024-09-10T21:16:41.423464+0000","last_active":"2024-09-10T21:16:43.418589+0000","last_peered":"2024-09-10T21:16:43.418589+0000","last_clean":"2024-09-10T21:16:43.418589+0000","last_became_active":"2024-09-10T21:16:41.423196+0000","last_became_peered":"2024-09-10T21:16:41.423196+0000","last_unstale":"2024-09-10T21:16:43.418589+0000","last_undegraded":"2024-09-10T21:16:43.418589+0000","last_fullsized":"2024-09-10T21:16:43.418589+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:39.236278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.445252+0000","last_change":"2024-09-10T21:16:41.444047+0000","last_active":"2024-09-10T21:16:41.445252+0000","last_peered":"2024-09-10T21:16:41.445252+0000","last_clean":"2024-09-10T21:16:41.445252+0000","last_became_active":"2024-09-10T21:16:41.442968+0000","last_became_peered":"2024-09-10T21:16:41.442968+0000","last_unstale":"2024-09-10T21:16:41.445252+0000","last_undegraded":"2024-09-10T21:16:41.445252+0000","last_fullsized":"2024-09-10T21:16:41.445252+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:02.662401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716434+0000","last_change":"2024-09-10T21:16:41.426567+0000","last_active":"2024-09-10T21:16:42.716434+0000","last_peered":"2024-09-10T21:16:42.716434+0000","last_clean":"2024-09-10T21:16:42.716434+0000","last_became_active":"2024-09-10T21:16:41.425971+0000","last_became_peered":"2024-09-10T21:16:41.425971+0000","last_unstale":"2024-09-10T21:16:42.716434+0000","last_undegraded":"2024-09-10T21:16:42.716434+0000","last_fullsized":"2024-09-10T21:16:42.716434+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:28.078428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.715986+0000","last_change":"2024-09-10T21:16:41.428575+0000","last_active":"2024-09-10T21:16:42.715986+0000","last_peered":"2024-09-10T21:16:42.715986+0000","last_clean":"2024-09-10T21:16:42.715986+0000","last_became_active":"2024-09-10T21:16:41.427872+0000","last_became_peered":"2024-09-10T21:16:41.427872+0000","last_unstale":"2024-09-10T21:16:42.715986+0000","last_undegraded":"2024-09-10T21:16:42.715986+0000","last_fullsized":"2024-09-10T21:16:42.715986+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:02.880139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994142+0000","last_change":"2024-09-10T21:16:41.434011+0000","last_active":"2024-09-10T21:16:42.994142+0000","last_peered":"2024-09-10T21:16:42.994142+0000","last_clean":"2024-09-10T21:16:42.994142+0000","last_became_active":"2024-09-10T21:16:41.433634+0000","last_became_peered":"2024-09-10T21:16:41.433634+0000","last_unstale":"2024-09-10T21:16:42.994142+0000","last_undegraded":"2024-09-10T21:16:42.994142+0000","last_fullsized":"2024-09-10T21:16:42.994142+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:38.449390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409568+0000","last_change":"2024-09-10T21:16:41.425079+0000","last_active":"2024-09-10T21:16:42.409568+0000","last_peered":"2024-09-10T21:16:42.409568+0000","last_clean":"2024-09-10T21:16:42.409568+0000","last_became_active":"2024-09-10T21:16:41.424723+0000","last_became_peered":"2024-09-10T21:16:41.424723+0000","last_unstale":"2024-09-10T21:16:42.409568+0000","last_undegraded":"2024-09-10T21:16:42.409568+0000","last_fullsized":"2024-09-10T21:16:42.409568+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:43.752527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.410117+0000","last_change":"2024-09-10T21:16:41.431761+0000","last_active":"2024-09-10T21:16:42.410117+0000","last_peered":"2024-09-10T21:16:42.410117+0000","last_clean":"2024-09-10T21:16:42.410117+0000","last_became_active":"2024-09-10T21:16:41.430910+0000","last_became_peered":"2024-09-10T21:16:41.430910+0000","last_unstale":"2024-09-10T21:16:42.410117+0000","last_undegraded":"2024-09-10T21:16:42.410117+0000","last_fullsized":"2024-09-10T21:16:42.410117+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:41.791454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409093+0000","last_change":"2024-09-10T21:16:41.447054+0000","last_active":"2024-09-10T21:16:42.409093+0000","last_peered":"2024-09-10T21:16:42.409093+0000","last_clean":"2024-09-10T21:16:42.409093+0000","last_became_active":"2024-09-10T21:16:41.445151+0000","last_became_peered":"2024-09-10T21:16:41.445151+0000","last_unstale":"2024-09-10T21:16:42.409093+0000","last_undegraded":"2024-09-10T21:16:42.409093+0000","last_fullsized":"2024-09-10T21:16:42.409093+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:11.443350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409143+0000","last_change":"2024-09-10T21:16:41.431004+0000","last_active":"2024-09-10T21:16:42.409143+0000","last_peered":"2024-09-10T21:16:42.409143+0000","last_clean":"2024-09-10T21:16:42.409143+0000","last_became_active":"2024-09-10T21:16:41.430720+0000","last_became_peered":"2024-09-10T21:16:41.430720+0000","last_unstale":"2024-09-10T21:16:42.409143+0000","last_undegraded":"2024-09-10T21:16:42.409143+0000","last_fullsized":"2024-09-10T21:16:42.409143+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:14.238476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409104+0000","last_change":"2024-09-10T21:16:41.429634+0000","last_active":"2024-09-10T21:16:42.409104+0000","last_peered":"2024-09-10T21:16:42.409104+0000","last_clean":"2024-09-10T21:16:42.409104+0000","last_became_active":"2024-09-10T21:16:41.429375+0000","last_became_peered":"2024-09-10T21:16:41.429375+0000","last_unstale":"2024-09-10T21:16:42.409104+0000","last_undegraded":"2024-09-10T21:16:42.409104+0000","last_fullsized":"2024-09-10T21:16:42.409104+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:40.455286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409160+0000","last_change":"2024-09-10T21:16:41.440627+0000","last_active":"2024-09-10T21:16:42.409160+0000","last_peered":"2024-09-10T21:16:42.409160+0000","last_clean":"2024-09-10T21:16:42.409160+0000","last_became_active":"2024-09-10T21:16:41.439989+0000","last_became_peered":"2024-09-10T21:16:41.439989+0000","last_unstale":"2024-09-10T21:16:42.409160+0000","last_undegraded":"2024-09-10T21:16:42.409160+0000","last_fullsized":"2024-09-10T21:16:42.409160+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:38.967436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.408861+0000","last_change":"2024-09-10T21:16:41.444696+0000","last_active":"2024-09-10T21:16:42.408861+0000","last_peered":"2024-09-10T21:16:42.408861+0000","last_clean":"2024-09-10T21:16:42.408861+0000","last_became_active":"2024-09-10T21:16:41.444358+0000","last_became_peered":"2024-09-10T21:16:41.444358+0000","last_unstale":"2024-09-10T21:16:42.408861+0000","last_undegraded":"2024-09-10T21:16:42.408861+0000","last_fullsized":"2024-09-10T21:16:42.408861+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:58.598089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716262+0000","last_change":"2024-09-10T21:16:41.429087+0000","last_active":"2024-09-10T21:16:42.716262+0000","last_peered":"2024-09-10T21:16:42.716262+0000","last_clean":"2024-09-10T21:16:42.716262+0000","last_became_active":"2024-09-10T21:16:41.426303+0000","last_became_peered":"2024-09-10T21:16:41.426303+0000","last_unstale":"2024-09-10T21:16:42.716262+0000","last_undegraded":"2024-09-10T21:16:42.716262+0000","last_fullsized":"2024-09-10T21:16:42.716262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:00.068361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993967+0000","last_change":"2024-09-10T21:16:41.433802+0000","last_active":"2024-09-10T21:16:42.993967+0000","last_peered":"2024-09-10T21:16:42.993967+0000","last_clean":"2024-09-10T21:16:42.993967+0000","last_became_active":"2024-09-10T21:16:41.433094+0000","last_became_peered":"2024-09-10T21:16:41.433094+0000","last_unstale":"2024-09-10T21:16:42.993967+0000","last_undegraded":"2024-09-10T21:16:42.993967+0000","last_fullsized":"2024-09-10T21:16:42.993967+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:33.343383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418915+0000","last_change":"2024-09-10T21:16:41.425350+0000","last_active":"2024-09-10T21:16:43.418915+0000","last_peered":"2024-09-10T21:16:43.418915+0000","last_clean":"2024-09-10T21:16:43.418915+0000","last_became_active":"2024-09-10T21:16:41.425074+0000","last_became_peered":"2024-09-10T21:16:41.425074+0000","last_unstale":"2024-09-10T21:16:43.418915+0000","last_undegraded":"2024-09-10T21:16:43.418915+0000","last_fullsized":"2024-09-10T21:16:43.418915+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:02.490697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"91'4","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409230+0000","last_change":"2024-09-10T21:16:40.428950+0000","last_active":"2024-09-10T21:16:42.409230+0000","last_peered":"2024-09-10T21:16:42.409230+0000","last_clean":"2024-09-10T21:16:42.409230+0000","last_became_active":"2024-09-10T21:16:40.428382+0000","last_became_peered":"2024-09-10T21:16:40.428382+0000","last_unstale":"2024-09-10T21:16:42.409230+0000","last_undegraded":"2024-09-10T21:16:42.409230+0000","last_fullsized":"2024-09-10T21:16:42.409230+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:56.746204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409288+0000","last_change":"2024-09-10T21:16:41.438522+0000","last_active":"2024-09-10T21:16:42.409288+0000","last_peered":"2024-09-10T21:16:42.409288+0000","last_clean":"2024-09-10T21:16:42.409288+0000","last_became_active":"2024-09-10T21:16:41.438268+0000","last_became_peered":"2024-09-10T21:16:41.438268+0000","last_unstale":"2024-09-10T21:16:42.409288+0000","last_undegraded":"2024-09-10T21:16:42.409288+0000","last_fullsized":"2024-09-10T21:16:42.409288+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:53.733853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577055+0000","last_change":"2024-09-10T21:16:40.428909+0000","last_active":"2024-09-10T21:16:43.577055+0000","last_peered":"2024-09-10T21:16:43.577055+0000","last_clean":"2024-09-10T21:16:43.577055+0000","last_became_active":"2024-09-10T21:16:40.428630+0000","last_became_peered":"2024-09-10T21:16:40.428630+0000","last_unstale":"2024-09-10T21:16:43.577055+0000","last_undegraded":"2024-09-10T21:16:43.577055+0000","last_fullsized":"2024-09-10T21:16:43.577055+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:35.886637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994373+0000","last_change":"2024-09-10T21:16:41.425798+0000","last_active":"2024-09-10T21:16:42.994373+0000","last_peered":"2024-09-10T21:16:42.994373+0000","last_clean":"2024-09-10T21:16:42.994373+0000","last_became_active":"2024-09-10T21:16:41.425522+0000","last_became_peered":"2024-09-10T21:16:41.425522+0000","last_unstale":"2024-09-10T21:16:42.994373+0000","last_undegraded":"2024-09-10T21:16:42.994373+0000","last_fullsized":"2024-09-10T21:16:42.994373+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:51.184355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.715874+0000","last_change":"2024-09-10T21:16:40.425190+0000","last_active":"2024-09-10T21:16:42.715874+0000","last_peered":"2024-09-10T21:16:42.715874+0000","last_clean":"2024-09-10T21:16:42.715874+0000","last_became_active":"2024-09-10T21:16:40.424907+0000","last_became_peered":"2024-09-10T21:16:40.424907+0000","last_unstale":"2024-09-10T21:16:42.715874+0000","last_undegraded":"2024-09-10T21:16:42.715874+0000","last_fullsized":"2024-09-10T21:16:42.715874+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:20.839168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409967+0000","last_change":"2024-09-10T21:16:41.423976+0000","last_active":"2024-09-10T21:16:42.409967+0000","last_peered":"2024-09-10T21:16:42.409967+0000","last_clean":"2024-09-10T21:16:42.409967+0000","last_became_active":"2024-09-10T21:16:41.423532+0000","last_became_peered":"2024-09-10T21:16:41.423532+0000","last_unstale":"2024-09-10T21:16:42.409967+0000","last_undegraded":"2024-09-10T21:16:42.409967+0000","last_fullsized":"2024-09-10T21:16:42.409967+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:01.515800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994302+0000","last_change":"2024-09-10T21:16:40.427336+0000","last_active":"2024-09-10T21:16:42.994302+0000","last_peered":"2024-09-10T21:16:42.994302+0000","last_clean":"2024-09-10T21:16:42.994302+0000","last_became_active":"2024-09-10T21:16:40.427035+0000","last_became_peered":"2024-09-10T21:16:40.427035+0000","last_unstale":"2024-09-10T21:16:42.994302+0000","last_undegraded":"2024-09-10T21:16:42.994302+0000","last_fullsized":"2024-09-10T21:16:42.994302+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:20.839168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.439242+0000","last_change":"2024-09-10T21:16:41.437269+0000","last_active":"2024-09-10T21:16:41.439242+0000","last_peered":"2024-09-10T21:16:41.439242+0000","last_clean":"2024-09-10T21:16:41.439242+0000","last_became_active":"2024-09-10T21:16:41.436719+0000","last_became_peered":"2024-09-10T21:16:41.436719+0000","last_unstale":"2024-09-10T21:16:41.439242+0000","last_undegraded":"2024-09-10T21:16:41.439242+0000","last_fullsized":"2024-09-10T21:16:41.439242+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:28:42.112331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993559+0000","last_change":"2024-09-10T21:16:40.420080+0000","last_active":"2024-09-10T21:16:42.993559+0000","last_peered":"2024-09-10T21:16:42.993559+0000","last_clean":"2024-09-10T21:16:42.993559+0000","last_became_active":"2024-09-10T21:16:40.419674+0000","last_became_peered":"2024-09-10T21:16:40.419674+0000","last_unstale":"2024-09-10T21:16:42.993559+0000","last_undegraded":"2024-09-10T21:16:42.993559+0000","last_fullsized":"2024-09-10T21:16:42.993559+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:57.278459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577213+0000","last_change":"2024-09-10T21:16:41.431653+0000","last_active":"2024-09-10T21:16:43.577213+0000","last_peered":"2024-09-10T21:16:43.577213+0000","last_clean":"2024-09-10T21:16:43.577213+0000","last_became_active":"2024-09-10T21:16:41.431357+0000","last_became_peered":"2024-09-10T21:16:41.431357+0000","last_unstale":"2024-09-10T21:16:43.577213+0000","last_undegraded":"2024-09-10T21:16:43.577213+0000","last_fullsized":"2024-09-10T21:16:43.577213+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:48.253724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.715735+0000","last_change":"2024-09-10T21:16:40.429453+0000","last_active":"2024-09-10T21:16:42.715735+0000","last_peered":"2024-09-10T21:16:42.715735+0000","last_clean":"2024-09-10T21:16:42.715735+0000","last_became_active":"2024-09-10T21:16:40.429183+0000","last_became_peered":"2024-09-10T21:16:40.429183+0000","last_unstale":"2024-09-10T21:16:42.715735+0000","last_undegraded":"2024-09-10T21:16:42.715735+0000","last_fullsized":"2024-09-10T21:16:42.715735+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:19.094153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.408567+0000","last_change":"2024-09-10T21:16:41.443869+0000","last_active":"2024-09-10T21:16:42.408567+0000","last_peered":"2024-09-10T21:16:42.408567+0000","last_clean":"2024-09-10T21:16:42.408567+0000","last_became_active":"2024-09-10T21:16:41.443593+0000","last_became_peered":"2024-09-10T21:16:41.443593+0000","last_unstale":"2024-09-10T21:16:42.408567+0000","last_undegraded":"2024-09-10T21:16:42.408567+0000","last_fullsized":"2024-09-10T21:16:42.408567+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:11.319468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.576989+0000","last_change":"2024-09-10T21:16:40.428282+0000","last_active":"2024-09-10T21:16:43.576989+0000","last_peered":"2024-09-10T21:16:43.576989+0000","last_clean":"2024-09-10T21:16:43.576989+0000","last_became_active":"2024-09-10T21:16:40.427970+0000","last_became_peered":"2024-09-10T21:16:40.427970+0000","last_unstale":"2024-09-10T21:16:43.576989+0000","last_undegraded":"2024-09-10T21:16:43.576989+0000","last_fullsized":"2024-09-10T21:16:43.576989+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:04.829527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.441077+0000","last_change":"2024-09-10T21:16:41.438427+0000","last_active":"2024-09-10T21:16:41.441077+0000","last_peered":"2024-09-10T21:16:41.441077+0000","last_clean":"2024-09-10T21:16:41.441077+0000","last_became_active":"2024-09-10T21:16:41.438089+0000","last_became_peered":"2024-09-10T21:16:41.438089+0000","last_unstale":"2024-09-10T21:16:41.441077+0000","last_undegraded":"2024-09-10T21:16:41.441077+0000","last_fullsized":"2024-09-10T21:16:41.441077+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:08.631169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418068+0000","last_change":"2024-09-10T21:16:40.418368+0000","last_active":"2024-09-10T21:16:43.418068+0000","last_peered":"2024-09-10T21:16:43.418068+0000","last_clean":"2024-09-10T21:16:43.418068+0000","last_became_active":"2024-09-10T21:16:40.417936+0000","last_became_peered":"2024-09-10T21:16:40.417936+0000","last_unstale":"2024-09-10T21:16:43.418068+0000","last_undegraded":"2024-09-10T21:16:43.418068+0000","last_fullsized":"2024-09-10T21:16:43.418068+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:36.208124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994265+0000","last_change":"2024-09-10T21:16:41.433447+0000","last_active":"2024-09-10T21:16:42.994265+0000","last_peered":"2024-09-10T21:16:42.994265+0000","last_clean":"2024-09-10T21:16:42.994265+0000","last_became_active":"2024-09-10T21:16:41.433157+0000","last_became_peered":"2024-09-10T21:16:41.433157+0000","last_unstale":"2024-09-10T21:16:42.994265+0000","last_undegraded":"2024-09-10T21:16:42.994265+0000","last_fullsized":"2024-09-10T21:16:42.994265+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:42.591342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716068+0000","last_change":"2024-09-10T21:16:40.423681+0000","last_active":"2024-09-10T21:16:42.716068+0000","last_peered":"2024-09-10T21:16:42.716068+0000","last_clean":"2024-09-10T21:16:42.716068+0000","last_became_active":"2024-09-10T21:16:40.423401+0000","last_became_peered":"2024-09-10T21:16:40.423401+0000","last_unstale":"2024-09-10T21:16:42.716068+0000","last_undegraded":"2024-09-10T21:16:42.716068+0000","last_fullsized":"2024-09-10T21:16:42.716068+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:15.821464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409469+0000","last_change":"2024-09-10T21:16:41.427532+0000","last_active":"2024-09-10T21:16:42.409469+0000","last_peered":"2024-09-10T21:16:42.409469+0000","last_clean":"2024-09-10T21:16:42.409469+0000","last_became_active":"2024-09-10T21:16:41.426963+0000","last_became_peered":"2024-09-10T21:16:41.426963+0000","last_unstale":"2024-09-10T21:16:42.409469+0000","last_undegraded":"2024-09-10T21:16:42.409469+0000","last_fullsized":"2024-09-10T21:16:42.409469+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:14.204203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994158+0000","last_change":"2024-09-10T21:16:40.427823+0000","last_active":"2024-09-10T21:16:42.994158+0000","last_peered":"2024-09-10T21:16:42.994158+0000","last_clean":"2024-09-10T21:16:42.994158+0000","last_became_active":"2024-09-10T21:16:40.427245+0000","last_became_peered":"2024-09-10T21:16:40.427245+0000","last_unstale":"2024-09-10T21:16:42.994158+0000","last_undegraded":"2024-09-10T21:16:42.994158+0000","last_fullsized":"2024-09-10T21:16:42.994158+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:19.094153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577466+0000","last_change":"2024-09-10T21:16:41.425413+0000","last_active":"2024-09-10T21:16:43.577466+0000","last_peered":"2024-09-10T21:16:43.577466+0000","last_clean":"2024-09-10T21:16:43.577466+0000","last_became_active":"2024-09-10T21:16:41.423100+0000","last_became_peered":"2024-09-10T21:16:41.423100+0000","last_unstale":"2024-09-10T21:16:43.577466+0000","last_undegraded":"2024-09-10T21:16:43.577466+0000","last_fullsized":"2024-09-10T21:16:43.577466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:08.007186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"91'3","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577588+0000","last_change":"2024-09-10T21:16:40.424166+0000","last_active":"2024-09-10T21:16:43.577588+0000","last_peered":"2024-09-10T21:16:43.577588+0000","last_clean":"2024-09-10T21:16:43.577588+0000","last_became_active":"2024-09-10T21:16:40.423841+0000","last_became_peered":"2024-09-10T21:16:40.423841+0000","last_unstale":"2024-09-10T21:16:43.577588+0000","last_undegraded":"2024-09-10T21:16:43.577588+0000","last_fullsized":"2024-09-10T21:16:43.577588+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:49.846416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409358+0000","last_change":"2024-09-10T21:16:41.422291+0000","last_active":"2024-09-10T21:16:42.409358+0000","last_peered":"2024-09-10T21:16:42.409358+0000","last_clean":"2024-09-10T21:16:42.409358+0000","last_became_active":"2024-09-10T21:16:41.421664+0000","last_became_peered":"2024-09-10T21:16:41.421664+0000","last_unstale":"2024-09-10T21:16:42.409358+0000","last_undegraded":"2024-09-10T21:16:42.409358+0000","last_fullsized":"2024-09-10T21:16:42.409358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:34.519014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"91'6","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.450639+0000","last_change":"2024-09-10T21:16:40.428188+0000","last_active":"2024-09-10T21:16:41.450639+0000","last_peered":"2024-09-10T21:16:41.450639+0000","last_clean":"2024-09-10T21:16:41.450639+0000","last_became_active":"2024-09-10T21:16:40.427871+0000","last_became_peered":"2024-09-10T21:16:40.427871+0000","last_unstale":"2024-09-10T21:16:41.450639+0000","last_undegraded":"2024-09-10T21:16:41.450639+0000","last_fullsized":"2024-09-10T21:16:41.450639+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:16.689109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993884+0000","last_change":"2024-09-10T21:16:41.422770+0000","last_active":"2024-09-10T21:16:42.993884+0000","last_peered":"2024-09-10T21:16:42.993884+0000","last_clean":"2024-09-10T21:16:42.993884+0000","last_became_active":"2024-09-10T21:16:41.422454+0000","last_became_peered":"2024-09-10T21:16:41.422454+0000","last_unstale":"2024-09-10T21:16:42.993884+0000","last_undegraded":"2024-09-10T21:16:42.993884+0000","last_fullsized":"2024-09-10T21:16:42.993884+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:49.093104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418280+0000","last_change":"2024-09-10T21:16:40.424578+0000","last_active":"2024-09-10T21:16:43.418280+0000","last_peered":"2024-09-10T21:16:43.418280+0000","last_clean":"2024-09-10T21:16:43.418280+0000","last_became_active":"2024-09-10T21:16:40.424242+0000","last_became_peered":"2024-09-10T21:16:40.424242+0000","last_unstale":"2024-09-10T21:16:43.418280+0000","last_undegraded":"2024-09-10T21:16:43.418280+0000","last_fullsized":"2024-09-10T21:16:43.418280+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:17.071296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716475+0000","last_change":"2024-09-10T21:16:41.420701+0000","last_active":"2024-09-10T21:16:42.716475+0000","last_peered":"2024-09-10T21:16:42.716475+0000","last_clean":"2024-09-10T21:16:42.716475+0000","last_became_active":"2024-09-10T21:16:41.420312+0000","last_became_peered":"2024-09-10T21:16:41.420312+0000","last_unstale":"2024-09-10T21:16:42.716475+0000","last_undegraded":"2024-09-10T21:16:42.716475+0000","last_fullsized":"2024-09-10T21:16:42.716475+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:56.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993859+0000","last_change":"2024-09-10T21:16:40.431389+0000","last_active":"2024-09-10T21:16:42.993859+0000","last_peered":"2024-09-10T21:16:42.993859+0000","last_clean":"2024-09-10T21:16:42.993859+0000","last_became_active":"2024-09-10T21:16:40.430988+0000","last_became_peered":"2024-09-10T21:16:40.430988+0000","last_unstale":"2024-09-10T21:16:42.993859+0000","last_undegraded":"2024-09-10T21:16:42.993859+0000","last_fullsized":"2024-09-10T21:16:42.993859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:15.821464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577597+0000","last_change":"2024-09-10T21:16:41.434437+0000","last_active":"2024-09-10T21:16:43.577597+0000","last_peered":"2024-09-10T21:16:43.577597+0000","last_clean":"2024-09-10T21:16:43.577597+0000","last_became_active":"2024-09-10T21:16:41.434128+0000","last_became_peered":"2024-09-10T21:16:41.434128+0000","last_unstale":"2024-09-10T21:16:43.577597+0000","last_undegraded":"2024-09-10T21:16:43.577597+0000","last_fullsized":"2024-09-10T21:16:43.577597+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:12.163978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.418653+0000","last_change":"2024-09-10T21:16:40.423517+0000","last_active":"2024-09-10T21:16:41.418653+0000","last_peered":"2024-09-10T21:16:41.418653+0000","last_clean":"2024-09-10T21:16:41.418653+0000","last_became_active":"2024-09-10T21:16:40.423179+0000","last_became_peered":"2024-09-10T21:16:40.423179+0000","last_unstale":"2024-09-10T21:16:41.418653+0000","last_undegraded":"2024-09-10T21:16:41.418653+0000","last_fullsized":"2024-09-10T21:16:41.418653+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:12.096394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716141+0000","last_change":"2024-09-10T21:16:41.434641+0000","last_active":"2024-09-10T21:16:42.716141+0000","last_peered":"2024-09-10T21:16:42.716141+0000","last_clean":"2024-09-10T21:16:42.716141+0000","last_became_active":"2024-09-10T21:16:41.429531+0000","last_became_peered":"2024-09-10T21:16:41.429531+0000","last_unstale":"2024-09-10T21:16:42.716141+0000","last_undegraded":"2024-09-10T21:16:42.716141+0000","last_fullsized":"2024-09-10T21:16:42.716141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:27.207535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"91'8","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994136+0000","last_change":"2024-09-10T21:16:40.422493+0000","last_active":"2024-09-10T21:16:42.994136+0000","last_peered":"2024-09-10T21:16:42.994136+0000","last_clean":"2024-09-10T21:16:42.994136+0000","last_became_active":"2024-09-10T21:16:40.422131+0000","last_became_peered":"2024-09-10T21:16:40.422131+0000","last_unstale":"2024-09-10T21:16:42.994136+0000","last_undegraded":"2024-09-10T21:16:42.994136+0000","last_fullsized":"2024-09-10T21:16:42.994136+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:29.923997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409780+0000","last_change":"2024-09-10T21:16:41.427245+0000","last_active":"2024-09-10T21:16:42.409780+0000","last_peered":"2024-09-10T21:16:42.409780+0000","last_clean":"2024-09-10T21:16:42.409780+0000","last_became_active":"2024-09-10T21:16:41.426624+0000","last_became_peered":"2024-09-10T21:16:41.426624+0000","last_unstale":"2024-09-10T21:16:42.409780+0000","last_undegraded":"2024-09-10T21:16:42.409780+0000","last_fullsized":"2024-09-10T21:16:42.409780+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:17.160075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"91'2","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.442183+0000","last_change":"2024-09-10T21:16:40.428161+0000","last_active":"2024-09-10T21:16:41.442183+0000","last_peered":"2024-09-10T21:16:41.442183+0000","last_clean":"2024-09-10T21:16:41.442183+0000","last_became_active":"2024-09-10T21:16:40.427892+0000","last_became_peered":"2024-09-10T21:16:40.427892+0000","last_unstale":"2024-09-10T21:16:41.442183+0000","last_undegraded":"2024-09-10T21:16:41.442183+0000","last_fullsized":"2024-09-10T21:16:41.442183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:12.829874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409367+0000","last_change":"2024-09-10T21:16:41.427978+0000","last_active":"2024-09-10T21:16:42.409367+0000","last_peered":"2024-09-10T21:16:42.409367+0000","last_clean":"2024-09-10T21:16:42.409367+0000","last_became_active":"2024-09-10T21:16:41.427247+0000","last_became_peered":"2024-09-10T21:16:41.427247+0000","last_unstale":"2024-09-10T21:16:42.409367+0000","last_undegraded":"2024-09-10T21:16:42.409367+0000","last_fullsized":"2024-09-10T21:16:42.409367+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:35.359461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577495+0000","last_change":"2024-09-10T21:16:40.423515+0000","last_active":"2024-09-10T21:16:43.577495+0000","last_peered":"2024-09-10T21:16:43.577495+0000","last_clean":"2024-09-10T21:16:43.577495+0000","last_became_active":"2024-09-10T21:16:40.423214+0000","last_became_peered":"2024-09-10T21:16:40.423214+0000","last_unstale":"2024-09-10T21:16:43.577495+0000","last_undegraded":"2024-09-10T21:16:43.577495+0000","last_fullsized":"2024-09-10T21:16:43.577495+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:52.123893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.441923+0000","last_change":"2024-09-10T21:16:41.438731+0000","last_active":"2024-09-10T21:16:41.441923+0000","last_peered":"2024-09-10T21:16:41.441923+0000","last_clean":"2024-09-10T21:16:41.441923+0000","last_became_active":"2024-09-10T21:16:41.437383+0000","last_became_peered":"2024-09-10T21:16:41.437383+0000","last_unstale":"2024-09-10T21:16:41.441923+0000","last_undegraded":"2024-09-10T21:16:41.441923+0000","last_fullsized":"2024-09-10T21:16:41.441923+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:58.782973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418481+0000","last_change":"2024-09-10T21:16:40.422872+0000","last_active":"2024-09-10T21:16:43.418481+0000","last_peered":"2024-09-10T21:16:43.418481+0000","last_clean":"2024-09-10T21:16:43.418481+0000","last_became_active":"2024-09-10T21:16:40.422557+0000","last_became_peered":"2024-09-10T21:16:40.422557+0000","last_unstale":"2024-09-10T21:16:43.418481+0000","last_undegraded":"2024-09-10T21:16:43.418481+0000","last_fullsized":"2024-09-10T21:16:43.418481+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:00.308807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418131+0000","last_change":"2024-09-10T21:16:41.428480+0000","last_active":"2024-09-10T21:16:43.418131+0000","last_peered":"2024-09-10T21:16:43.418131+0000","last_clean":"2024-09-10T21:16:43.418131+0000","last_became_active":"2024-09-10T21:16:41.428211+0000","last_became_peered":"2024-09-10T21:16:41.428211+0000","last_unstale":"2024-09-10T21:16:43.418131+0000","last_undegraded":"2024-09-10T21:16:43.418131+0000","last_fullsized":"2024-09-10T21:16:43.418131+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:35.128280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.993604+0000","last_change":"2024-09-10T21:16:40.427377+0000","last_active":"2024-09-10T21:16:42.993604+0000","last_peered":"2024-09-10T21:16:42.993604+0000","last_clean":"2024-09-10T21:16:42.993604+0000","last_became_active":"2024-09-10T21:16:40.427084+0000","last_became_peered":"2024-09-10T21:16:40.427084+0000","last_unstale":"2024-09-10T21:16:42.993604+0000","last_undegraded":"2024-09-10T21:16:42.993604+0000","last_fullsized":"2024-09-10T21:16:42.993604+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:47.824551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418289+0000","last_change":"2024-09-10T21:16:41.440826+0000","last_active":"2024-09-10T21:16:43.418289+0000","last_peered":"2024-09-10T21:16:43.418289+0000","last_clean":"2024-09-10T21:16:43.418289+0000","last_became_active":"2024-09-10T21:16:41.440512+0000","last_became_peered":"2024-09-10T21:16:41.440512+0000","last_unstale":"2024-09-10T21:16:43.418289+0000","last_undegraded":"2024-09-10T21:16:43.418289+0000","last_fullsized":"2024-09-10T21:16:43.418289+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:09.617565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.408881+0000","last_change":"2024-09-10T21:16:40.432497+0000","last_active":"2024-09-10T21:16:42.408881+0000","last_peered":"2024-09-10T21:16:42.408881+0000","last_clean":"2024-09-10T21:16:42.408881+0000","last_became_active":"2024-09-10T21:16:40.432182+0000","last_became_peered":"2024-09-10T21:16:40.432182+0000","last_unstale":"2024-09-10T21:16:42.408881+0000","last_undegraded":"2024-09-10T21:16:42.408881+0000","last_fullsized":"2024-09-10T21:16:42.408881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:29.320175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.994269+0000","last_change":"2024-09-10T21:16:41.431642+0000","last_active":"2024-09-10T21:16:42.994269+0000","last_peered":"2024-09-10T21:16:42.994269+0000","last_clean":"2024-09-10T21:16:42.994269+0000","last_became_active":"2024-09-10T21:16:41.431353+0000","last_became_peered":"2024-09-10T21:16:41.431353+0000","last_unstale":"2024-09-10T21:16:42.994269+0000","last_undegraded":"2024-09-10T21:16:42.994269+0000","last_fullsized":"2024-09-10T21:16:42.994269+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:14.550533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409747+0000","last_change":"2024-09-10T21:16:40.428078+0000","last_active":"2024-09-10T21:16:42.409747+0000","last_peered":"2024-09-10T21:16:42.409747+0000","last_clean":"2024-09-10T21:16:42.409747+0000","last_became_active":"2024-09-10T21:16:40.427704+0000","last_became_peered":"2024-09-10T21:16:40.427704+0000","last_unstale":"2024-09-10T21:16:42.409747+0000","last_undegraded":"2024-09-10T21:16:42.409747+0000","last_fullsized":"2024-09-10T21:16:42.409747+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:44.983295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.408958+0000","last_change":"2024-09-10T21:16:41.442032+0000","last_active":"2024-09-10T21:16:42.408958+0000","last_peered":"2024-09-10T21:16:42.408958+0000","last_clean":"2024-09-10T21:16:42.408958+0000","last_became_active":"2024-09-10T21:16:41.441718+0000","last_became_peered":"2024-09-10T21:16:41.441718+0000","last_unstale":"2024-09-10T21:16:42.408958+0000","last_undegraded":"2024-09-10T21:16:42.408958+0000","last_fullsized":"2024-09-10T21:16:42.408958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:12.027369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.715609+0000","last_change":"2024-09-10T21:16:40.420060+0000","last_active":"2024-09-10T21:16:42.715609+0000","last_peered":"2024-09-10T21:16:42.715609+0000","last_clean":"2024-09-10T21:16:42.715609+0000","last_became_active":"2024-09-10T21:16:40.419491+0000","last_became_peered":"2024-09-10T21:16:40.419491+0000","last_unstale":"2024-09-10T21:16:42.715609+0000","last_undegraded":"2024-09-10T21:16:42.715609+0000","last_fullsized":"2024-09-10T21:16:42.715609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:57.278459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418951+0000","last_change":"2024-09-10T21:16:41.430611+0000","last_active":"2024-09-10T21:16:43.418951+0000","last_peered":"2024-09-10T21:16:43.418951+0000","last_clean":"2024-09-10T21:16:43.418951+0000","last_became_active":"2024-09-10T21:16:41.430001+0000","last_became_peered":"2024-09-10T21:16:41.430001+0000","last_unstale":"2024-09-10T21:16:43.418951+0000","last_undegraded":"2024-09-10T21:16:43.418951+0000","last_fullsized":"2024-09-10T21:16:43.418951+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:31.634366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"91'2","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.446581+0000","last_change":"2024-09-10T21:16:40.423506+0000","last_active":"2024-09-10T21:16:41.446581+0000","last_peered":"2024-09-10T21:16:41.446581+0000","last_clean":"2024-09-10T21:16:41.446581+0000","last_became_active":"2024-09-10T21:16:40.422925+0000","last_became_peered":"2024-09-10T21:16:40.422925+0000","last_unstale":"2024-09-10T21:16:41.446581+0000","last_undegraded":"2024-09-10T21:16:41.446581+0000","last_fullsized":"2024-09-10T21:16:41.446581+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:35.124240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.419066+0000","last_change":"2024-09-10T21:16:41.427878+0000","last_active":"2024-09-10T21:16:43.419066+0000","last_peered":"2024-09-10T21:16:43.419066+0000","last_clean":"2024-09-10T21:16:43.419066+0000","last_became_active":"2024-09-10T21:16:41.427616+0000","last_became_peered":"2024-09-10T21:16:41.427616+0000","last_unstale":"2024-09-10T21:16:43.419066+0000","last_undegraded":"2024-09-10T21:16:43.419066+0000","last_fullsized":"2024-09-10T21:16:43.419066+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:59.659550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"92'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.420523+0000","last_change":"2024-09-10T21:16:40.421767+0000","last_active":"2024-09-10T21:16:43.420523+0000","last_peered":"2024-09-10T21:16:43.420523+0000","last_clean":"2024-09-10T21:16:43.420523+0000","last_became_active":"2024-09-10T21:16:40.421481+0000","last_became_peered":"2024-09-10T21:16:40.421481+0000","last_unstale":"2024-09-10T21:16:43.420523+0000","last_undegraded":"2024-09-10T21:16:43.420523+0000","last_fullsized":"2024-09-10T21:16:43.420523+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:13.547786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716038+0000","last_change":"2024-09-10T21:16:41.439815+0000","last_active":"2024-09-10T21:16:42.716038+0000","last_peered":"2024-09-10T21:16:42.716038+0000","last_clean":"2024-09-10T21:16:42.716038+0000","last_became_active":"2024-09-10T21:16:41.439134+0000","last_became_peered":"2024-09-10T21:16:41.439134+0000","last_unstale":"2024-09-10T21:16:42.716038+0000","last_undegraded":"2024-09-10T21:16:42.716038+0000","last_fullsized":"2024-09-10T21:16:42.716038+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:47.005003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418678+0000","last_change":"2024-09-10T21:16:40.409420+0000","last_active":"2024-09-10T21:16:43.418678+0000","last_peered":"2024-09-10T21:16:43.418678+0000","last_clean":"2024-09-10T21:16:43.418678+0000","last_became_active":"2024-09-10T21:16:40.409105+0000","last_became_peered":"2024-09-10T21:16:40.409105+0000","last_unstale":"2024-09-10T21:16:43.418678+0000","last_undegraded":"2024-09-10T21:16:43.418678+0000","last_fullsized":"2024-09-10T21:16:43.418678+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:50.181608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577527+0000","last_change":"2024-09-10T21:16:41.427746+0000","last_active":"2024-09-10T21:16:43.577527+0000","last_peered":"2024-09-10T21:16:43.577527+0000","last_clean":"2024-09-10T21:16:43.577527+0000","last_became_active":"2024-09-10T21:16:41.427457+0000","last_became_peered":"2024-09-10T21:16:41.427457+0000","last_unstale":"2024-09-10T21:16:43.577527+0000","last_undegraded":"2024-09-10T21:16:43.577527+0000","last_fullsized":"2024-09-10T21:16:43.577527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:07.209017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409245+0000","last_change":"2024-09-10T21:16:40.421114+0000","last_active":"2024-09-10T21:16:42.409245+0000","last_peered":"2024-09-10T21:16:42.409245+0000","last_clean":"2024-09-10T21:16:42.409245+0000","last_became_active":"2024-09-10T21:16:40.420628+0000","last_became_peered":"2024-09-10T21:16:40.420628+0000","last_unstale":"2024-09-10T21:16:42.409245+0000","last_undegraded":"2024-09-10T21:16:42.409245+0000","last_fullsized":"2024-09-10T21:16:42.409245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:05.194554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418341+0000","last_change":"2024-09-10T21:16:41.429688+0000","last_active":"2024-09-10T21:16:43.418341+0000","last_peered":"2024-09-10T21:16:43.418341+0000","last_clean":"2024-09-10T21:16:43.418341+0000","last_became_active":"2024-09-10T21:16:41.429399+0000","last_became_peered":"2024-09-10T21:16:41.429399+0000","last_unstale":"2024-09-10T21:16:43.418341+0000","last_undegraded":"2024-09-10T21:16:43.418341+0000","last_fullsized":"2024-09-10T21:16:43.418341+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:44.235771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716548+0000","last_change":"2024-09-10T21:16:40.425176+0000","last_active":"2024-09-10T21:16:42.716548+0000","last_peered":"2024-09-10T21:16:42.716548+0000","last_clean":"2024-09-10T21:16:42.716548+0000","last_became_active":"2024-09-10T21:16:40.424894+0000","last_became_peered":"2024-09-10T21:16:40.424894+0000","last_unstale":"2024-09-10T21:16:42.716548+0000","last_undegraded":"2024-09-10T21:16:42.716548+0000","last_fullsized":"2024-09-10T21:16:42.716548+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:16.920456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.577362+0000","last_change":"2024-09-10T21:16:41.435563+0000","last_active":"2024-09-10T21:16:43.577362+0000","last_peered":"2024-09-10T21:16:43.577362+0000","last_clean":"2024-09-10T21:16:43.577362+0000","last_became_active":"2024-09-10T21:16:41.435299+0000","last_became_peered":"2024-09-10T21:16:41.435299+0000","last_unstale":"2024-09-10T21:16:43.577362+0000","last_undegraded":"2024-09-10T21:16:43.577362+0000","last_fullsized":"2024-09-10T21:16:43.577362+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:04.510513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.408918+0000","last_change":"2024-09-10T21:16:40.416609+0000","last_active":"2024-09-10T21:16:42.408918+0000","last_peered":"2024-09-10T21:16:42.408918+0000","last_clean":"2024-09-10T21:16:42.408918+0000","last_became_active":"2024-09-10T21:16:40.416278+0000","last_became_peered":"2024-09-10T21:16:40.416278+0000","last_unstale":"2024-09-10T21:16:42.408918+0000","last_undegraded":"2024-09-10T21:16:42.408918+0000","last_fullsized":"2024-09-10T21:16:42.408918+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:40.788707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:43.418206+0000","last_change":"2024-09-10T21:16:41.418720+0000","last_active":"2024-09-10T21:16:43.418206+0000","last_peered":"2024-09-10T21:16:43.418206+0000","last_clean":"2024-09-10T21:16:43.418206+0000","last_became_active":"2024-09-10T21:16:41.418184+0000","last_became_peered":"2024-09-10T21:16:41.418184+0000","last_unstale":"2024-09-10T21:16:43.418206+0000","last_undegraded":"2024-09-10T21:16:43.418206+0000","last_fullsized":"2024-09-10T21:16:43.418206+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:26.250271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.409007+0000","last_change":"2024-09-10T21:16:40.429961+0000","last_active":"2024-09-10T21:16:42.409007+0000","last_peered":"2024-09-10T21:16:42.409007+0000","last_clean":"2024-09-10T21:16:42.409007+0000","last_became_active":"2024-09-10T21:16:40.429695+0000","last_became_peered":"2024-09-10T21:16:40.429695+0000","last_unstale":"2024-09-10T21:16:42.409007+0000","last_undegraded":"2024-09-10T21:16:42.409007+0000","last_fullsized":"2024-09-10T21:16:42.409007+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:00.686235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.428138+0000","last_change":"2024-09-10T21:16:41.425291+0000","last_active":"2024-09-10T21:16:41.428138+0000","last_peered":"2024-09-10T21:16:41.428138+0000","last_clean":"2024-09-10T21:16:41.428138+0000","last_became_active":"2024-09-10T21:16:41.424986+0000","last_became_peered":"2024-09-10T21:16:41.424986+0000","last_unstale":"2024-09-10T21:16:41.428138+0000","last_undegraded":"2024-09-10T21:16:41.428138+0000","last_fullsized":"2024-09-10T21:16:41.428138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:53.979110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.715868+0000","last_change":"2024-09-10T21:16:40.419789+0000","last_active":"2024-09-10T21:16:42.715868+0000","last_peered":"2024-09-10T21:16:42.715868+0000","last_clean":"2024-09-10T21:16:42.715868+0000","last_became_active":"2024-09-10T21:16:40.419132+0000","last_became_peered":"2024-09-10T21:16:40.419132+0000","last_unstale":"2024-09-10T21:16:42.715868+0000","last_undegraded":"2024-09-10T21:16:42.715868+0000","last_fullsized":"2024-09-10T21:16:42.715868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:29.923997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:16:42.716296+0000","last_change":"2024-09-10T21:16:41.428232+0000","last_active":"2024-09-10T21:16:42.716296+0000","last_peered":"2024-09-10T21:16:42.716296+0000","last_clean":"2024-09-10T21:16:42.716296+0000","last_became_active":"2024-09-10T21:16:41.427936+0000","last_became_peered":"2024-09-10T21:16:41.427936+0000","last_unstale":"2024-09-10T21:16:42.716296+0000","last_undegraded":"2024-09-10T21:16:42.716296+0000","last_fullsized":"2024-09-10T21:16:42.716296+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:40.394835+0000","last_clean_scrub_stamp":"2024-09-10T21:16:40.394835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:33.343383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"91'1","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.444514+0000","last_change":"2024-09-10T21:16:40.430194+0000","last_active":"2024-09-10T21:16:41.444514+0000","last_peered":"2024-09-10T21:16:41.444514+0000","last_clean":"2024-09-10T21:16:41.444514+0000","last_became_active":"2024-09-10T21:16:40.429293+0000","last_became_peered":"2024-09-10T21:16:40.429293+0000","last_unstale":"2024-09-10T21:16:41.444514+0000","last_undegraded":"2024-09-10T21:16:41.444514+0000","last_fullsized":"2024-09-10T21:16:41.444514+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:16:39.392088+0000","last_clean_scrub_stamp":"2024-09-10T21:16:39.392088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:23.946617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":234,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-10T21:16:41.419435+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:16:41.419435+0000","last_peered":"2024-09-10T21:16:41.419435+0000","last_clean":"2024-09-10T21:16:41.419435+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:16:41.419435+0000","last_undegraded":"2024-09-10T21:16:41.419435+0000","last_fullsized":"2024-09-10T21:16:41.419435+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509520,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36452,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335388,"statfs":{"total":96636764160,"available":96599437312,"internally_reserved":0,"allocated":1130496,"data_stored":481436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509522,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1732608,"data_stored":1067172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509520,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1134592,"data_stored":482018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509519,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":475330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1134592,"data_stored":481506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509520,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":481436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542224,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36456,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335384,"statfs":{"total":96636764160,"available":96599433216,"internally_reserved":0,"allocated":1134592,"data_stored":482018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37048,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334792,"statfs":{"total":96636764160,"available":96598827008,"internally_reserved":0,"allocated":1732608,"data_stored":1067228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:16:45.980 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:16:45.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:16:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.166+0000 7f7647c00640 1 -- 172.21.15.16:0/2998907429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764015f940 msgr2=0x7f764015fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.166+0000 7f7647c00640 1 --2- 172.21.15.16:0/2998907429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764015f940 0x7f764015fdd0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7638009bc0 tx=0x7f763801ca80 comp rx=0 tx=0).stop 2024-09-10T21:16:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.166+0000 7f7647c00640 1 -- 172.21.15.16:0/2998907429 shutdown_connections 2024-09-10T21:16:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.166+0000 7f7647c00640 1 --2- 172.21.15.16:0/2998907429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764015f940 0x7f764015fdd0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.166+0000 7f7647c00640 1 --2- 172.21.15.16:0/2998907429 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76400cd300 0x7f76400d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.166+0000 7f7647c00640 1 -- 172.21.15.16:0/2998907429 >> 172.21.15.16:0/2998907429 conn(0x7f76400d1ce0 msgr2=0x7f76400d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7647c00640 1 -- 172.21.15.16:0/2998907429 shutdown_connections 2024-09-10T21:16:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7647c00640 1 -- 172.21.15.16:0/2998907429 wait complete. 2024-09-10T21:16:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7647c00640 1 Processor -- start 2024-09-10T21:16:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7647c00640 1 -- start start 2024-09-10T21:16:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7647c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76400cd300 0x7f764014f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7647c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f764015bdc0 0x7f764015c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7647c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764015f940 0x7f764015c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7647c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76401691d0 con 0x7f764015f940 2024-09-10T21:16:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7647c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7640169050 con 0x7f764015bdc0 2024-09-10T21:16:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7647c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7640169350 con 0x7f76400cd300 2024-09-10T21:16:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7645c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764015f940 0x7f764015c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7645c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764015f940 0x7f764015c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47734/0 (socket says 172.21.15.16:47734) 2024-09-10T21:16:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7645c00640 1 -- 172.21.15.16:0/1789683837 learned_addr learned my addr 172.21.15.16:0/1789683837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7645c00640 1 -- 172.21.15.16:0/1789683837 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76400cd300 msgr2=0x7f764014f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7645c00640 1 --2- 172.21.15.16:0/1789683837 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76400cd300 0x7f764014f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7645c00640 1 -- 172.21.15.16:0/1789683837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f764015bdc0 msgr2=0x7f764015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7645c00640 1 --2- 172.21.15.16:0/1789683837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f764015bdc0 0x7f764015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7645c00640 1 -- 172.21.15.16:0/1789683837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764006f4e0 con 0x7f764015f940 2024-09-10T21:16:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7645c00640 1 --2- 172.21.15.16:0/1789683837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764015f940 0x7f764015c980 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7638009b10 tx=0x7f7638009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f763cc00640 1 -- 172.21.15.16:0/1789683837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76380bb070 con 0x7f764015f940 2024-09-10T21:16:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f763cc00640 1 -- 172.21.15.16:0/1789683837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7638004d40 con 0x7f764015f940 2024-09-10T21:16:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f763cc00640 1 -- 172.21.15.16:0/1789683837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76380b2040 con 0x7f764015f940 2024-09-10T21:16:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.170+0000 7f7647c00640 1 -- 172.21.15.16:0/1789683837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7640152fb0 con 0x7f764015f940 2024-09-10T21:16:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.174+0000 7f7647c00640 1 -- 172.21.15.16:0/1789683837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7640153570 con 0x7f764015f940 2024-09-10T21:16:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.174+0000 7f7647c00640 1 -- 172.21.15.16:0/1789683837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76400d2800 con 0x7f764015f940 2024-09-10T21:16:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.174+0000 7f763cc00640 1 -- 172.21.15.16:0/1789683837 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f76380b5030 con 0x7f764015f940 2024-09-10T21:16:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.174+0000 7f763cc00640 1 --2- 172.21.15.16:0/1789683837 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76200725e0 0x7f7620074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.174+0000 7f763cc00640 1 -- 172.21.15.16:0/1789683837 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f763812ccf0 con 0x7f764015f940 2024-09-10T21:16:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.174+0000 7f7645200640 1 --2- 172.21.15.16:0/1789683837 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76200725e0 0x7f7620074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.174+0000 7f7645200640 1 --2- 172.21.15.16:0/1789683837 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76200725e0 0x7f7620074aa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f76340046c0 tx=0x7f7634007910 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.178+0000 7f763cc00640 1 -- 172.21.15.16:0/1789683837 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7638016790 con 0x7f764015f940 2024-09-10T21:16:46.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.414+0000 7f7647c00640 1 -- 172.21.15.16:0/1789683837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f76400704a0 con 0x7f764015f940 2024-09-10T21:16:46.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.414+0000 7f763cc00640 1 -- 172.21.15.16:0/1789683837 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v27) ==== 190+0+147 (secure 0 0 0) 0x7f76380f83c0 con 0x7f764015f940 2024-09-10T21:16:46.420 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-10T21:16:46.420 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA== 2024-09-10T21:16:46.420 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-10T21:16:46.420 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-10T21:16:46.420 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:16:46.420 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-10T21:16:46.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.418+0000 7f7647c00640 1 -- 172.21.15.16:0/1789683837 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76200725e0 msgr2=0x7f7620074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:46.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.418+0000 7f7647c00640 1 --2- 172.21.15.16:0/1789683837 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76200725e0 0x7f7620074aa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f76340046c0 tx=0x7f7634007910 comp rx=0 tx=0).stop 2024-09-10T21:16:46.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.418+0000 7f7647c00640 1 -- 172.21.15.16:0/1789683837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764015f940 msgr2=0x7f764015c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:46.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.418+0000 7f7647c00640 1 --2- 172.21.15.16:0/1789683837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764015f940 0x7f764015c980 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7638009b10 tx=0x7f7638009260 comp rx=0 tx=0).stop 2024-09-10T21:16:46.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.418+0000 7f7647c00640 1 -- 172.21.15.16:0/1789683837 shutdown_connections 2024-09-10T21:16:46.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.418+0000 7f7647c00640 1 --2- 172.21.15.16:0/1789683837 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76200725e0 0x7f7620074aa0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.418+0000 7f7647c00640 1 --2- 172.21.15.16:0/1789683837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764015f940 0x7f764015c980 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.418+0000 7f7647c00640 1 --2- 172.21.15.16:0/1789683837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f764015bdc0 0x7f764015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.418+0000 7f7647c00640 1 --2- 172.21.15.16:0/1789683837 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76400cd300 0x7f764014f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.418+0000 7f7647c00640 1 -- 172.21.15.16:0/1789683837 >> 172.21.15.16:0/1789683837 conn(0x7f76400d1ce0 msgr2=0x7f7640166070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:46.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.418+0000 7f7647c00640 1 -- 172.21.15.16:0/1789683837 shutdown_connections 2024-09-10T21:16:46.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.418+0000 7f7647c00640 1 -- 172.21.15.16:0/1789683837 wait complete. 2024-09-10T21:16:46.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:16:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.586+0000 7fc345000640 1 -- 172.21.15.16:0/2737693871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3400b13c0 msgr2=0x7fc340071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.586+0000 7fc345000640 1 --2- 172.21.15.16:0/2737693871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3400b13c0 0x7fc340071520 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc328009900 tx=0x7fc32801c7c0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.586+0000 7fc345000640 1 -- 172.21.15.16:0/2737693871 shutdown_connections 2024-09-10T21:16:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.586+0000 7fc345000640 1 --2- 172.21.15.16:0/2737693871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3400b13c0 0x7fc340071520 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.586+0000 7fc345000640 1 --2- 172.21.15.16:0/2737693871 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc34015f400 0x7fc340070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.586+0000 7fc345000640 1 -- 172.21.15.16:0/2737693871 >> 172.21.15.16:0/2737693871 conn(0x7fc34009b130 msgr2=0x7fc3400d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:46.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.586+0000 7fc345000640 1 -- 172.21.15.16:0/2737693871 shutdown_connections 2024-09-10T21:16:46.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.586+0000 7fc345000640 1 -- 172.21.15.16:0/2737693871 wait complete. 2024-09-10T21:16:46.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc345000640 1 Processor -- start 2024-09-10T21:16:46.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc345000640 1 -- start start 2024-09-10T21:16:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc345000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3400b13c0 0x7fc34006f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc345000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3400cfc80 0x7fc3400b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc345000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34015f400 0x7fc3400b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc345000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc340169770 con 0x7fc34015f400 2024-09-10T21:16:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc345000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3401695f0 con 0x7fc3400b13c0 2024-09-10T21:16:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc345000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc3401698f0 con 0x7fc3400cfc80 2024-09-10T21:16:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc33e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34015f400 0x7fc3400b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc33e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34015f400 0x7fc3400b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47748/0 (socket says 172.21.15.16:47748) 2024-09-10T21:16:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc33e800640 1 -- 172.21.15.16:0/909416713 learned_addr learned my addr 172.21.15.16:0/909416713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc335400640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3400cfc80 0x7fc3400b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc33de00640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3400b13c0 0x7fc34006f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc33e800640 1 -- 172.21.15.16:0/909416713 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3400cfc80 msgr2=0x7fc3400b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc33e800640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3400cfc80 0x7fc3400b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc33e800640 1 -- 172.21.15.16:0/909416713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3400b13c0 msgr2=0x7fc34006f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc33e800640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3400b13c0 0x7fc34006f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc33e800640 1 -- 172.21.15.16:0/909416713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3400b7950 con 0x7fc34015f400 2024-09-10T21:16:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc33e800640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34015f400 0x7fc3400b2920 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc33800c4d0 tx=0x7fc33800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc336a00640 1 -- 172.21.15.16:0/909416713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3380052d0 con 0x7fc34015f400 2024-09-10T21:16:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc336a00640 1 -- 172.21.15.16:0/909416713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc338005470 con 0x7fc34015f400 2024-09-10T21:16:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc336a00640 1 -- 172.21.15.16:0/909416713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc338012870 con 0x7fc34015f400 2024-09-10T21:16:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc345000640 1 -- 172.21.15.16:0/909416713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3400b7670 con 0x7fc34015f400 2024-09-10T21:16:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.590+0000 7fc345000640 1 -- 172.21.15.16:0/909416713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3400b5f80 con 0x7fc34015f400 2024-09-10T21:16:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.594+0000 7fc345000640 1 -- 172.21.15.16:0/909416713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3400d00d0 con 0x7fc34015f400 2024-09-10T21:16:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.594+0000 7fc336a00640 1 -- 172.21.15.16:0/909416713 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc338003600 con 0x7fc34015f400 2024-09-10T21:16:46.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.594+0000 7fc336a00640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc31c0725e0 0x7fc31c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:46.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.594+0000 7fc336a00640 1 -- 172.21.15.16:0/909416713 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fc338092bb0 con 0x7fc34015f400 2024-09-10T21:16:46.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.594+0000 7fc33de00640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc31c0725e0 0x7fc31c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:46.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.594+0000 7fc33de00640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc31c0725e0 0x7fc31c074aa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc330003f70 tx=0x7fc3300095f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.598+0000 7fc336a00640 1 -- 172.21.15.16:0/909416713 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc33805dc40 con 0x7fc34015f400 2024-09-10T21:16:46.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.810+0000 7fc345000640 1 -- 172.21.15.16:0/909416713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3401641c0 con 0x7fc34015f400 2024-09-10T21:16:46.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.810+0000 7fc336a00640 1 -- 172.21.15.16:0/909416713 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5229 (secure 0 0 0) 0x7fc338062670 con 0x7fc34015f400 2024-09-10T21:16:46.814 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:46.814 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":62,"btime":"2024-09-10T21:16:44:430553+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/895621033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":895621033},{"type":"v1","addr":"172.21.15.52:6835","nonce":895621033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6789,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2281471863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2281471863},{"type":"v1","addr":"172.21.15.16:6835","nonce":2281471863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1879915011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1879915011},{"type":"v1","addr":"172.21.15.16:6837","nonce":1879915011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:16:40.823255+0000","modified":"2024-09-10T21:16:43.834820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135725":{"gid":135725,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/585458541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":585458541},{"type":"v1","addr":"172.21.15.52:6834","nonce":585458541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135725,"qdb_cluster":[135725]},"id":7}]} 2024-09-10T21:16:46.814 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 62 2024-09-10T21:16:46.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.814+0000 7fc345000640 1 -- 172.21.15.16:0/909416713 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc31c0725e0 msgr2=0x7fc31c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:46.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.814+0000 7fc345000640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc31c0725e0 0x7fc31c074aa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc330003f70 tx=0x7fc3300095f0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.814+0000 7fc345000640 1 -- 172.21.15.16:0/909416713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34015f400 msgr2=0x7fc3400b2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:46.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.814+0000 7fc345000640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34015f400 0x7fc3400b2920 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc33800c4d0 tx=0x7fc33800c990 comp rx=0 tx=0).stop 2024-09-10T21:16:46.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.814+0000 7fc345000640 1 -- 172.21.15.16:0/909416713 shutdown_connections 2024-09-10T21:16:46.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.814+0000 7fc345000640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc31c0725e0 0x7fc31c074aa0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.814+0000 7fc345000640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34015f400 0x7fc3400b2920 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.814+0000 7fc345000640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3400cfc80 0x7fc3400b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.814+0000 7fc345000640 1 --2- 172.21.15.16:0/909416713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3400b13c0 0x7fc34006f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:46.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.814+0000 7fc345000640 1 -- 172.21.15.16:0/909416713 >> 172.21.15.16:0/909416713 conn(0x7fc34009b130 msgr2=0x7fc3400cd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:46.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.814+0000 7fc345000640 1 -- 172.21.15.16:0/909416713 shutdown_connections 2024-09-10T21:16:46.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:46.814+0000 7fc345000640 1 -- 172.21.15.16:0/909416713 wait complete. 2024-09-10T21:16:47.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:16:40.823255+0000', 'modified': '2024-09-10T21:16:43.834820+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 135725}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_135725': {'gid': 135725, 'name': 'd', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.52:6834/585458541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6832', 'nonce': 585458541}, {'type': 'v1', 'addr': '172.21.15.52:6834', 'nonce': 585458541}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 135725, 'qdb_cluster': [135725]} 2024-09-10T21:16:47.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:16:47.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T21:16:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 -- 172.21.15.16:0/4134931664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2700705e0 msgr2=0x7fc2700ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 --2- 172.21.15.16:0/4134931664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2700705e0 0x7fc2700ce450 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fc2700d0aa0 tx=0x7fc25801c850 comp rx=0 tx=0).stop 2024-09-10T21:16:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 -- 172.21.15.16:0/4134931664 shutdown_connections 2024-09-10T21:16:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 --2- 172.21.15.16:0/4134931664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2700705e0 0x7fc2700ce450 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 --2- 172.21.15.16:0/4134931664 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc27015a270 0x7fc2700700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 -- 172.21.15.16:0/4134931664 >> 172.21.15.16:0/4134931664 conn(0x7fc2700cfb20 msgr2=0x7fc2700cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 -- 172.21.15.16:0/4134931664 shutdown_connections 2024-09-10T21:16:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 -- 172.21.15.16:0/4134931664 wait complete. 2024-09-10T21:16:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 Processor -- start 2024-09-10T21:16:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 -- start start 2024-09-10T21:16:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2700705e0 0x7fc27014e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2700ce990 0x7fc2700b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc27015a270 0x7fc270153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc270167420 con 0x7fc2700ce990 2024-09-10T21:16:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc2700cab00 con 0x7fc2700705e0 2024-09-10T21:16:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc277200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc2701675a0 con 0x7fc27015a270 2024-09-10T21:16:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc267e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2700ce990 0x7fc2700b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc267e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2700ce990 0x7fc2700b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47766/0 (socket says 172.21.15.16:47766) 2024-09-10T21:16:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.390+0000 7fc267e00640 1 -- 172.21.15.16:0/3839729267 learned_addr learned my addr 172.21.15.16:0/3839729267 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc26fe00640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2700705e0 0x7fc27014e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc275200640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc27015a270 0x7fc270153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc267e00640 1 -- 172.21.15.16:0/3839729267 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc27015a270 msgr2=0x7fc270153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc267e00640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc27015a270 0x7fc270153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc267e00640 1 -- 172.21.15.16:0/3839729267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2700705e0 msgr2=0x7fc27014e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc267e00640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2700705e0 0x7fc27014e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc267e00640 1 -- 172.21.15.16:0/3839729267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2700c7600 con 0x7fc2700ce990 2024-09-10T21:16:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc275200640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc27015a270 0x7fc270153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc267e00640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2700ce990 0x7fc2700b50c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc2580b0040 tx=0x7fc2580a56e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc26cc00640 1 -- 172.21.15.16:0/3839729267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2580b2040 con 0x7fc2700ce990 2024-09-10T21:16:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc26cc00640 1 -- 172.21.15.16:0/3839729267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2580044d0 con 0x7fc2700ce990 2024-09-10T21:16:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc26cc00640 1 -- 172.21.15.16:0/3839729267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258005290 con 0x7fc2700ce990 2024-09-10T21:16:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc277200640 1 -- 172.21.15.16:0/3839729267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2701568d0 con 0x7fc2700ce990 2024-09-10T21:16:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc277200640 1 -- 172.21.15.16:0/3839729267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2700c78a0 con 0x7fc2700ce990 2024-09-10T21:16:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.394+0000 7fc26cc00640 1 -- 172.21.15.16:0/3839729267 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc258005430 con 0x7fc2700ce990 2024-09-10T21:16:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc26cc00640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc24c0725e0 0x7fc24c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc26cc00640 1 -- 172.21.15.16:0/3839729267 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fc25812cb90 con 0x7fc2700ce990 2024-09-10T21:16:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 Processor -- start 2024-09-10T21:16:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc26fe00640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc24c0725e0 0x7fc24c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 -- start start 2024-09-10T21:16:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc26fe00640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc24c0725e0 0x7fc24c074aa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc26000ab40 tx=0x7fc26000a250 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 0x55bc57657db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576582f0 0x55bc5760b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc5760c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bc5752c3f0 con 0x55bc576579d0 2024-09-10T21:16:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bc57469b20 con 0x55bc576582f0 2024-09-10T21:16:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55bc57458f50 con 0x55bc5760be10 2024-09-10T21:16:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc26fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 0x55bc57657db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc26fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 0x55bc57657db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47776/0 (socket says 172.21.15.16:47776) 2024-09-10T21:16:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc26fe00640 1 -- 172.21.15.16:0/1668828998 learned_addr learned my addr 172.21.15.16:0/1668828998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc267e00640 1 --2- 172.21.15.16:0/1668828998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576582f0 0x55bc5760b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc275200640 1 --2- 172.21.15.16:0/1668828998 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc5760c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc26fe00640 1 -- 172.21.15.16:0/1668828998 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 msgr2=0x55bc5760c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc26fe00640 1 --2- 172.21.15.16:0/1668828998 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc5760c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc26fe00640 1 -- 172.21.15.16:0/1668828998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576582f0 msgr2=0x55bc5760b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc26fe00640 1 --2- 172.21.15.16:0/1668828998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576582f0 0x55bc5760b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc26fe00640 1 -- 172.21.15.16:0/1668828998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc57420590 con 0x55bc576579d0 2024-09-10T21:16:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc275200640 1 --2- 172.21.15.16:0/1668828998 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc5760c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc26fe00640 1 --2- 172.21.15.16:0/1668828998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 0x55bc57657db0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fc26000ff20 tx=0x7fc260012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8db66e8b517c529 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc264c00640 1 -- 172.21.15.16:0/1668828998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260016070 con 0x55bc576579d0 2024-09-10T21:16:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc264c00640 1 -- 172.21.15.16:0/1668828998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc260025630 con 0x55bc576579d0 2024-09-10T21:16:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc264c00640 1 -- 172.21.15.16:0/1668828998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260025a20 con 0x55bc576579d0 2024-09-10T21:16:47.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 -- 172.21.15.16:0/1668828998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 msgr2=0x55bc57657db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/1668828998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 0x55bc57657db0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fc26000ff20 tx=0x7fc260012000 comp rx=0 tx=0).stop 2024-09-10T21:16:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 -- 172.21.15.16:0/1668828998 shutdown_connections 2024-09-10T21:16:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/1668828998 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc5760c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/1668828998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576582f0 0x55bc5760b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/1668828998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 0x55bc57657db0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 -- 172.21.15.16:0/1668828998 >> 172.21.15.16:0/1668828998 conn(0x55bc574f5130 msgr2=0x55bc5763ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 -- 172.21.15.16:0/1668828998 shutdown_connections 2024-09-10T21:16:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.398+0000 7fc27a6bb000 1 -- 172.21.15.16:0/1668828998 wait complete. 2024-09-10T21:16:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 1 -- start start 2024-09-10T21:16:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 1 Processor -- start 2024-09-10T21:16:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc576386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 0x55bc57638e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576582f0 0x55bc576393b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bc573f35d0 con 0x55bc576579d0 2024-09-10T21:16:47.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bc5745cd60 con 0x55bc576582f0 2024-09-10T21:16:47.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55bc573ff500 con 0x55bc5760be10 2024-09-10T21:16:47.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 10 client.0 _ll_register_callbacks cb 0x55bc57403810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:47.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc267e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 0x55bc57638e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc267e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 0x55bc57638e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47792/0 (socket says 172.21.15.16:47792) 2024-09-10T21:16:47.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc26fe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc576386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc267e00640 1 -- 172.21.15.16:0/2456141372 learned_addr learned my addr 172.21.15.16:0/2456141372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:47.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc275200640 1 --2- 172.21.15.16:0/2456141372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576582f0 0x55bc576393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc267e00640 1 -- 172.21.15.16:0/2456141372 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 msgr2=0x55bc576386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc267e00640 1 --2- 172.21.15.16:0/2456141372 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc576386a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc267e00640 1 -- 172.21.15.16:0/2456141372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576582f0 msgr2=0x55bc576393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc267e00640 1 --2- 172.21.15.16:0/2456141372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576582f0 0x55bc576393b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc267e00640 1 -- 172.21.15.16:0/2456141372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc573f7170 con 0x55bc576579d0 2024-09-10T21:16:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc26fe00640 1 --2- 172.21.15.16:0/2456141372 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc576386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc267e00640 1 --2- 172.21.15.16:0/2456141372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 0x55bc57638e70 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fc25812b020 tx=0x7fc2580f7270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de91a8d2df8d835 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc25d600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:16:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc25d600640 1 -- 172.21.15.16:0/2456141372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258100d40 con 0x55bc576579d0 2024-09-10T21:16:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2456141372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc573f10f0 con 0x55bc576579d0 2024-09-10T21:16:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc25d600640 1 -- 172.21.15.16:0/2456141372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fc25810ec70 con 0x55bc576579d0 2024-09-10T21:16:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc25d600640 1 -- 172.21.15.16:0/2456141372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc2580fc520 con 0x55bc576579d0 2024-09-10T21:16:47.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 10 client.6921 fetch_fsmap learned FSMap version 62 2024-09-10T21:16:47.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2456141372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55bc574ddef0 con 0x55bc576579d0 2024-09-10T21:16:47.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc25d600640 1 -- 172.21.15.16:0/2456141372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55bc573f10f0 con 0x55bc576579d0 2024-09-10T21:16:47.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc25d600640 1 -- 172.21.15.16:0/2456141372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 62) ==== 3502+0+0 (secure 0 0 0) 0x7fc2580f6ca0 con 0x55bc576579d0 2024-09-10T21:16:47.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 10 client.6921 fetch_fsmap finished waiting for FSMap version 62 2024-09-10T21:16:47.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 10 client.6921 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:16:47.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 20 client.6921 populate_metadata read hostname 'smithi016' 2024-09-10T21:16:47.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/2456141372 >> [v2:172.21.15.52:6832/585458541,v1:172.21.15.52:6834/585458541] conn(0x55bc5763a0f0 0x55bc57669c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 4 client.6921 mds_command: new command op to 135725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:16:47.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2456141372 --> [v2:172.21.15.52:6832/585458541,v1:172.21.15.52:6834/585458541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc5746ba70 con 0x55bc5763a0f0 2024-09-10T21:16:47.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.402+0000 7fc26fe00640 1 --2- 172.21.15.16:0/2456141372 >> [v2:172.21.15.52:6832/585458541,v1:172.21.15.52:6834/585458541] conn(0x55bc5763a0f0 0x55bc57669c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc26fe00640 1 --2- 172.21.15.16:0/2456141372 >> [v2:172.21.15.52:6832/585458541,v1:172.21.15.52:6834/585458541] conn(0x55bc5763a0f0 0x55bc57669c90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcda442387f90e3 server_cookie=6e2440f763bb2744 in_seq=0 out_seq=0 2024-09-10T21:16:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc25d600640 10 client.6921 ms_handle_connect on v2:172.21.15.52:6832/585458541 2024-09-10T21:16:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc25d600640 1 -- 172.21.15.16:0/2456141372 <== mds.0 v2:172.21.15.52:6832/585458541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc5746ba70 con 0x55bc5763a0f0 2024-09-10T21:16:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc25d600640 10 client.6921 handle_command_reply: tid=0 2024-09-10T21:16:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc27a6bb000 1 client.6921 shutdown 2024-09-10T21:16:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2456141372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 msgr2=0x55bc57638e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/2456141372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 0x55bc57638e70 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fc25812b020 tx=0x7fc2580f7270 comp rx=0 tx=0).stop 2024-09-10T21:16:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2456141372 shutdown_connections 2024-09-10T21:16:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/2456141372 >> [v2:172.21.15.52:6832/585458541,v1:172.21.15.52:6834/585458541] conn(0x55bc5763a0f0 0x55bc57669c90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/2456141372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576582f0 0x55bc576393b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc25d600640 0 client.6921 ms_handle_reset on v2:172.21.15.52:6832/585458541 2024-09-10T21:16:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/2456141372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576579d0 0x55bc57638e70 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/2456141372 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc576386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2456141372 >> 172.21.15.16:0/2456141372 conn(0x55bc574f5130 msgr2=0x55bc5760afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2456141372 shutdown_connections 2024-09-10T21:16:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2456141372 wait complete. 2024-09-10T21:16:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.406+0000 7fc27a6bb000 20 client.6921 trim_cache size 0 max 16384 2024-09-10T21:16:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 Processor -- start 2024-09-10T21:16:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 -- start start 2024-09-10T21:16:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc5760be10 0x55bc57656dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc576579d0 0x55bc57657310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 0x55bc57669c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc270158fe0 con 0x55bc576582f0 2024-09-10T21:16:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc270158060 con 0x55bc5760be10 2024-09-10T21:16:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc270156360 con 0x55bc576579d0 2024-09-10T21:16:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc275200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 0x55bc57669c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc275200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 0x55bc57669c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47796/0 (socket says 172.21.15.16:47796) 2024-09-10T21:16:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc275200640 1 -- 172.21.15.16:0/1556505554 learned_addr learned my addr 172.21.15.16:0/1556505554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc26fe00640 1 --2- 172.21.15.16:0/1556505554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc5760be10 0x55bc57656dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc267e00640 1 --2- 172.21.15.16:0/1556505554 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc576579d0 0x55bc57657310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc275200640 1 -- 172.21.15.16:0/1556505554 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc576579d0 msgr2=0x55bc57657310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc275200640 1 --2- 172.21.15.16:0/1556505554 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc576579d0 0x55bc57657310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc275200640 1 -- 172.21.15.16:0/1556505554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc5760be10 msgr2=0x55bc57656dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc275200640 1 --2- 172.21.15.16:0/1556505554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc5760be10 0x55bc57656dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc275200640 1 -- 172.21.15.16:0/1556505554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc574b4d90 con 0x55bc576582f0 2024-09-10T21:16:47.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc275200640 1 --2- 172.21.15.16:0/1556505554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 0x55bc57669c90 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc268005550 tx=0x7fc2680136a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bf1c76fcd1eb1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:47.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc25fe00640 1 -- 172.21.15.16:0/1556505554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26801a030 con 0x55bc576582f0 2024-09-10T21:16:47.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc25fe00640 1 -- 172.21.15.16:0/1556505554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc268005640 con 0x55bc576582f0 2024-09-10T21:16:47.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc25fe00640 1 -- 172.21.15.16:0/1556505554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268006b60 con 0x55bc576582f0 2024-09-10T21:16:47.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 -- 172.21.15.16:0/1556505554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 msgr2=0x55bc57669c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/1556505554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 0x55bc57669c90 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc268005550 tx=0x7fc2680136a0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 -- 172.21.15.16:0/1556505554 shutdown_connections 2024-09-10T21:16:47.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/1556505554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 0x55bc57669c90 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/1556505554 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc576579d0 0x55bc57657310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/1556505554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc5760be10 0x55bc57656dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 -- 172.21.15.16:0/1556505554 >> 172.21.15.16:0/1556505554 conn(0x55bc574f5130 msgr2=0x55bc57664fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.426+0000 7fc27a6bb000 1 -- 172.21.15.16:0/1556505554 shutdown_connections 2024-09-10T21:16:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 1 -- 172.21.15.16:0/1556505554 wait complete. 2024-09-10T21:16:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 1 -- start start 2024-09-10T21:16:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 1 Processor -- start 2024-09-10T21:16:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc576cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576579d0 0x55bc576cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 0x55bc576cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bc57424ec0 con 0x55bc576582f0 2024-09-10T21:16:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc270157a10 con 0x55bc576579d0 2024-09-10T21:16:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55bc573f35d0 con 0x55bc5760be10 2024-09-10T21:16:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 10 client.0 _ll_register_callbacks cb 0x7fc2580f38e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc275200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 0x55bc576cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc267e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576579d0 0x55bc576cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc275200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 0x55bc576cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47798/0 (socket says 172.21.15.16:47798) 2024-09-10T21:16:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc275200640 1 -- 172.21.15.16:0/2876106465 learned_addr learned my addr 172.21.15.16:0/2876106465 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc26fe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc576cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc275200640 1 -- 172.21.15.16:0/2876106465 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 msgr2=0x55bc576cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc275200640 1 --2- 172.21.15.16:0/2876106465 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc576cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc275200640 1 -- 172.21.15.16:0/2876106465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576579d0 msgr2=0x55bc576cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc275200640 1 --2- 172.21.15.16:0/2876106465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576579d0 0x55bc576cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc275200640 1 -- 172.21.15.16:0/2876106465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc5741fa90 con 0x55bc576582f0 2024-09-10T21:16:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc267e00640 1 --2- 172.21.15.16:0/2876106465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576579d0 0x55bc576cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc275200640 1 --2- 172.21.15.16:0/2876106465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 0x55bc576cdeb0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x55bc5745b920 tx=0x7fc268007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31cbebd5bed94ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc25d600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:16:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc25d600640 1 -- 172.21.15.16:0/2876106465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268026070 con 0x55bc576582f0 2024-09-10T21:16:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc25d600640 1 -- 172.21.15.16:0/2876106465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fc2680072f0 con 0x55bc576582f0 2024-09-10T21:16:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2876106465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc268005640 con 0x55bc576582f0 2024-09-10T21:16:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc25d600640 1 -- 172.21.15.16:0/2876106465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fc268031050 con 0x55bc576582f0 2024-09-10T21:16:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 10 client.6927 fetch_fsmap learned FSMap version 62 2024-09-10T21:16:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2876106465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55bc573fe050 con 0x55bc576582f0 2024-09-10T21:16:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc25d600640 1 -- 172.21.15.16:0/2876106465 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc268005640 con 0x55bc576582f0 2024-09-10T21:16:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc25d600640 1 -- 172.21.15.16:0/2876106465 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 62) ==== 3502+0+0 (secure 0 0 0) 0x7fc268034020 con 0x55bc576582f0 2024-09-10T21:16:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 10 client.6927 fetch_fsmap finished waiting for FSMap version 62 2024-09-10T21:16:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 10 client.6927 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:16:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 20 client.6927 populate_metadata read hostname 'smithi016' 2024-09-10T21:16:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/2876106465 >> [v2:172.21.15.52:6832/585458541,v1:172.21.15.52:6834/585458541] conn(0x55bc5766c960 0x55bc5766cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 4 client.6927 mds_command: new command op to 135725 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:16:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.430+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2876106465 --> [v2:172.21.15.52:6832/585458541,v1:172.21.15.52:6834/585458541] -- command(tid 0: {"prefix": "status"}) -- 0x55bc57467520 con 0x55bc5766c960 2024-09-10T21:16:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc26fe00640 1 --2- 172.21.15.16:0/2876106465 >> [v2:172.21.15.52:6832/585458541,v1:172.21.15.52:6834/585458541] conn(0x55bc5766c960 0x55bc5766cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc26fe00640 1 --2- 172.21.15.16:0/2876106465 >> [v2:172.21.15.52:6832/585458541,v1:172.21.15.52:6834/585458541] conn(0x55bc5766c960 0x55bc5766cd40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=891dae19d2adb80f server_cookie=be5249309f400b94 in_seq=0 out_seq=0 2024-09-10T21:16:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc25d600640 10 client.6927 ms_handle_connect on v2:172.21.15.52:6832/585458541 2024-09-10T21:16:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc25d600640 1 -- 172.21.15.16:0/2876106465 <== mds.0 v2:172.21.15.52:6832/585458541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55bc57467520 con 0x55bc5766c960 2024-09-10T21:16:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc25d600640 10 client.6927 handle_command_reply: tid=0 2024-09-10T21:16:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc27a6bb000 1 client.6927 shutdown 2024-09-10T21:16:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2876106465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 msgr2=0x55bc576cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/2876106465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 0x55bc576cdeb0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x55bc5745b920 tx=0x7fc268007d30 comp rx=0 tx=0).stop 2024-09-10T21:16:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2876106465 shutdown_connections 2024-09-10T21:16:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/2876106465 >> [v2:172.21.15.52:6832/585458541,v1:172.21.15.52:6834/585458541] conn(0x55bc5766c960 0x55bc5766cd40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/2876106465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc576582f0 0x55bc576cdeb0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/2876106465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc576579d0 0x55bc576cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc25d600640 0 client.6927 ms_handle_reset on v2:172.21.15.52:6832/585458541 2024-09-10T21:16:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc27a6bb000 1 --2- 172.21.15.16:0/2876106465 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55bc5760be10 0x55bc576cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2876106465 >> 172.21.15.16:0/2876106465 conn(0x55bc574f5130 msgr2=0x55bc57656dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2876106465 shutdown_connections 2024-09-10T21:16:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc27a6bb000 1 -- 172.21.15.16:0/2876106465 wait complete. 2024-09-10T21:16:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc27a6bb000 20 client.6927 trim_cache size 0 max 16384 2024-09-10T21:16:47.442 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:16:47.442 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "eb19be60-6d8f-4816-b778-3d7d6698304d", 2024-09-10T21:16:47.442 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-10T21:16:47.442 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 135725, 2024-09-10T21:16:47.442 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-10T21:16:47.443 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-10T21:16:47.443 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-10T21:16:47.443 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.0300085809999997, 2024-09-10T21:16:47.443 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 62, 2024-09-10T21:16:47.443 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 92, 2024-09-10T21:16:47.443 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 92, 2024-09-10T21:16:47.443 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 10.476093693999999 2024-09-10T21:16:47.443 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:16:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc277200640 1 -- 172.21.15.16:0/3839729267 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc24c0725e0 msgr2=0x7fc24c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc277200640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc24c0725e0 0x7fc24c074aa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc26000ab40 tx=0x7fc26000a250 comp rx=0 tx=0).stop 2024-09-10T21:16:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc277200640 1 -- 172.21.15.16:0/3839729267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2700ce990 msgr2=0x7fc2700b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc277200640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2700ce990 0x7fc2700b50c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc2580b0040 tx=0x7fc2580a56e0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc277200640 1 -- 172.21.15.16:0/3839729267 shutdown_connections 2024-09-10T21:16:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc277200640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc24c0725e0 0x7fc24c074aa0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc277200640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc27015a270 0x7fc270153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc277200640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2700ce990 0x7fc2700b50c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc277200640 1 --2- 172.21.15.16:0/3839729267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2700705e0 0x7fc27014e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc277200640 1 -- 172.21.15.16:0/3839729267 >> 172.21.15.16:0/3839729267 conn(0x7fc2700cfb20 msgr2=0x7fc27014f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc277200640 1 -- 172.21.15.16:0/3839729267 shutdown_connections 2024-09-10T21:16:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.434+0000 7fc277200640 1 -- 172.21.15.16:0/3839729267 wait complete. 2024-09-10T21:16:47.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:16:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.602+0000 7fb0cb200640 1 -- 172.21.15.16:0/514413382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c415f450 msgr2=0x7fb0c40cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.602+0000 7fb0cb200640 1 --2- 172.21.15.16:0/514413382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c415f450 0x7fb0c40cfc80 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40b3810 tx=0x7fb0b401f070 comp rx=0 tx=0).stop 2024-09-10T21:16:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.602+0000 7fb0cb200640 1 -- 172.21.15.16:0/514413382 shutdown_connections 2024-09-10T21:16:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.602+0000 7fb0cb200640 1 --2- 172.21.15.16:0/514413382 >> v2:172.21.15.52:3300/0 conn(0x7fb0c40d0890 0x7fb0c40d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.602+0000 7fb0cb200640 1 --2- 172.21.15.16:0/514413382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c415f450 0x7fb0c40cfc80 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.602+0000 7fb0cb200640 1 -- 172.21.15.16:0/514413382 >> 172.21.15.16:0/514413382 conn(0x7fb0c40d1d80 msgr2=0x7fb0c40d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.602+0000 7fb0cb200640 1 -- 172.21.15.16:0/514413382 shutdown_connections 2024-09-10T21:16:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.602+0000 7fb0cb200640 1 -- 172.21.15.16:0/514413382 wait complete. 2024-09-10T21:16:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.602+0000 7fb0cb200640 1 Processor -- start 2024-09-10T21:16:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.602+0000 7fb0cb200640 1 -- start start 2024-09-10T21:16:47.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.602+0000 7fb0cb200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0c40d0890 0x7fb0c41f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.602+0000 7fb0cb200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb0c415f450 0x7fb0c4153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0cb200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c40ceb80 0x7fb0c4155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0cb200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c40b2e30 con 0x7fb0c40ceb80 2024-09-10T21:16:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0cb200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c40b2cb0 con 0x7fb0c40d0890 2024-09-10T21:16:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0cb200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c40b2fb0 con 0x7fb0c415f450 2024-09-10T21:16:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c40ceb80 0x7fb0c4155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c40ceb80 0x7fb0c4155100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47810/0 (socket says 172.21.15.16:47810) 2024-09-10T21:16:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c9200640 1 -- 172.21.15.16:0/1474505728 learned_addr learned my addr 172.21.15.16:0/1474505728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c3e00640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0c40d0890 0x7fb0c41f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0bbe00640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb0c415f450 0x7fb0c4153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c9200640 1 -- 172.21.15.16:0/1474505728 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb0c415f450 msgr2=0x7fb0c4153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c9200640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb0c415f450 0x7fb0c4153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c9200640 1 -- 172.21.15.16:0/1474505728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0c40d0890 msgr2=0x7fb0c41f85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c9200640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0c40d0890 0x7fb0c41f85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c9200640 1 -- 172.21.15.16:0/1474505728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c41f8af0 con 0x7fb0c40ceb80 2024-09-10T21:16:47.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c9200640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c40ceb80 0x7fb0c4155100 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00ea20 tx=0x7fb0bc00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:47.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c0c00640 1 -- 172.21.15.16:0/1474505728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc00cde0 con 0x7fb0c40ceb80 2024-09-10T21:16:47.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c0c00640 1 -- 172.21.15.16:0/1474505728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0bc004c40 con 0x7fb0c40ceb80 2024-09-10T21:16:47.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c0c00640 1 -- 172.21.15.16:0/1474505728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc005280 con 0x7fb0c40ceb80 2024-09-10T21:16:47.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0cb200640 1 -- 172.21.15.16:0/1474505728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c40cdad0 con 0x7fb0c40ceb80 2024-09-10T21:16:47.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0cb200640 1 -- 172.21.15.16:0/1474505728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c40ce090 con 0x7fb0c40ceb80 2024-09-10T21:16:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0cb200640 1 -- 172.21.15.16:0/1474505728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c40b1e50 con 0x7fb0c40ceb80 2024-09-10T21:16:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.606+0000 7fb0c0c00640 1 -- 172.21.15.16:0/1474505728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb0bc00f040 con 0x7fb0c40ceb80 2024-09-10T21:16:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.610+0000 7fb0c0c00640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb0a0072610 0x7fb0a0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.610+0000 7fb0c0c00640 1 -- 172.21.15.16:0/1474505728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fb0bc093830 con 0x7fb0c40ceb80 2024-09-10T21:16:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.610+0000 7fb0c3e00640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb0a0072610 0x7fb0a0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:47.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.610+0000 7fb0c3e00640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb0a0072610 0x7fb0a0074ad0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40b2000 tx=0x7fb0b401d8b0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:47.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.614+0000 7fb0c0c00640 1 -- 172.21.15.16:0/1474505728 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb0c41f8af0 con 0x7fb0c40ceb80 2024-09-10T21:16:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.822+0000 7fb0cb200640 1 -- 172.21.15.16:0/1474505728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb0c41537d0 con 0x7fb0c40ceb80 2024-09-10T21:16:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.822+0000 7fb0c0c00640 1 -- 172.21.15.16:0/1474505728 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) ==== 53+0+73 (secure 0 0 0) 0x7fb0bc016030 con 0x7fb0c40ceb80 2024-09-10T21:16:47.827 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:16:47.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.826+0000 7fb0cb200640 1 -- 172.21.15.16:0/1474505728 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb0a0072610 msgr2=0x7fb0a0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.826+0000 7fb0cb200640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb0a0072610 0x7fb0a0074ad0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40b2000 tx=0x7fb0b401d8b0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.826+0000 7fb0cb200640 1 -- 172.21.15.16:0/1474505728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c40ceb80 msgr2=0x7fb0c4155100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:47.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.826+0000 7fb0cb200640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c40ceb80 0x7fb0c4155100 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00ea20 tx=0x7fb0bc00eee0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.826+0000 7fb0cb200640 1 -- 172.21.15.16:0/1474505728 shutdown_connections 2024-09-10T21:16:47.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.826+0000 7fb0cb200640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb0a0072610 0x7fb0a0074ad0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.826+0000 7fb0cb200640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c40ceb80 0x7fb0c4155100 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.826+0000 7fb0cb200640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb0c415f450 0x7fb0c4153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.826+0000 7fb0cb200640 1 --2- 172.21.15.16:0/1474505728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0c40d0890 0x7fb0c41f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:47.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.826+0000 7fb0cb200640 1 -- 172.21.15.16:0/1474505728 >> 172.21.15.16:0/1474505728 conn(0x7fb0c40d1d80 msgr2=0x7fb0c40cff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:47.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.826+0000 7fb0cb200640 1 -- 172.21.15.16:0/1474505728 shutdown_connections 2024-09-10T21:16:47.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:47.826+0000 7fb0cb200640 1 -- 172.21.15.16:0/1474505728 wait complete. 2024-09-10T21:16:47.848 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:16:47.848 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:16:47.848 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-10T21:16:47.849 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:16:47.849 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:16:47.849 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:16:47.849 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:16:47.849 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:16:47.849 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:16:47.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:47.849 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-10T21:16:47.859 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:16:47.859 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:16:47.859 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:16:47.859 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:47.859 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:16:47.859 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42042sec preferred_lft 42042sec 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:47.860 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:16:47.861 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:16:47.861 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-10T21:16:47.861 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:16:47.861 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:30:fb:74:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:16:47.861 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:16:47.861 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:47.861 INFO:teuthology.orchestra.run.smithi016.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:16:47.861 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:16:47.861 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:16:47.861 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:47.861 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:16:47.861 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:47.861 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:16:47.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:16:47.862 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:16:47.862 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:47.862 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:16:47.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:16:47.862 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:16:47.862 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:47.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:47.862 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:16:47.921 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-10T21:16:47.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:47.924 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-10T21:16:47.976 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-10T21:16:47.976 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:16:47.977 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:16:47.977 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:16:47.977 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:16:47.977 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:16:47.977 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:16:47.977 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:16:47.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:16:47.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:16:47.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:16:47.978 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:16:48.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:16:48.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:16:48.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:48.117 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:16:48.126 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:16:48.126 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:16:48.126 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:16:48.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:48.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-10T21:16:48.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:16:48.191 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:16:48.243 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:16:48.243 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:16:48.243 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:16:48.295 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:16:48.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:48.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:16:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:16:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:16:48.399 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:16:48.399 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:16:48.399 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:16:48.399 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:16:48.399 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:16:48.405 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:16:48.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:48.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:16:48.426 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:16:48.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:16:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:16:48.538 INFO:teuthology.orchestra.run.smithi016.stdout:client135818 2024-09-10T21:16:48.539 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:16:48.539 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:48.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.734+0000 7fdb4f600640 1 -- 172.21.15.16:0/489544560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb30003c70 msgr2=0x7fdb30004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.734+0000 7fdb4f600640 1 --2- 172.21.15.16:0/489544560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb30003c70 0x7fdb30004070 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c001890 tx=0x7fdb3c01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.734+0000 7fdb4d600640 1 -- 172.21.15.16:0/489544560 reap_dead start 2024-09-10T21:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.734+0000 7fdb4f600640 1 -- 172.21.15.16:0/489544560 shutdown_connections 2024-09-10T21:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.734+0000 7fdb4f600640 1 -- 172.21.15.16:0/489544560 >> 172.21.15.16:0/489544560 conn(0x7fdb4809b130 msgr2=0x7fdb480d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.734+0000 7fdb4f600640 1 -- 172.21.15.16:0/489544560 shutdown_connections 2024-09-10T21:16:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.734+0000 7fdb4f600640 1 -- 172.21.15.16:0/489544560 wait complete. 2024-09-10T21:16:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb4f600640 1 Processor -- start 2024-09-10T21:16:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb4f600640 1 -- start start 2024-09-10T21:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb4f600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb30003c70 0x7fdb4815b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb4f600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480cea70 0x7fdb48161920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb4f600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb48161e60 0x7fdb48162310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb4f600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48167f60 con 0x7fdb480cea70 2024-09-10T21:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb4f600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48167de0 con 0x7fdb48161e60 2024-09-10T21:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb4f600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fdb481680e0 con 0x7fdb30003c70 2024-09-10T21:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb47e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480cea70 0x7fdb48161920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb47e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480cea70 0x7fdb48161920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47834/0 (socket says 172.21.15.16:47834) 2024-09-10T21:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb47e00640 1 -- 172.21.15.16:0/974944358 learned_addr learned my addr 172.21.15.16:0/974944358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb4cc00640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb30003c70 0x7fdb4815b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb4d600640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb48161e60 0x7fdb48162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb47e00640 1 -- 172.21.15.16:0/974944358 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb30003c70 msgr2=0x7fdb4815b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb47e00640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb30003c70 0x7fdb4815b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb47e00640 1 -- 172.21.15.16:0/974944358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb48161e60 msgr2=0x7fdb48162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb47e00640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb48161e60 0x7fdb48162310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb47e00640 1 -- 172.21.15.16:0/974944358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4815b770 con 0x7fdb480cea70 2024-09-10T21:16:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb47e00640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480cea70 0x7fdb48161920 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fdb340118e0 tx=0x7fdb34011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb44c00640 1 -- 172.21.15.16:0/974944358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3400f9a0 con 0x7fdb480cea70 2024-09-10T21:16:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb44c00640 1 -- 172.21.15.16:0/974944358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb3400fb40 con 0x7fdb480cea70 2024-09-10T21:16:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb4f600640 1 -- 172.21.15.16:0/974944358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb480b7000 con 0x7fdb480cea70 2024-09-10T21:16:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb44c00640 1 -- 172.21.15.16:0/974944358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34005020 con 0x7fdb480cea70 2024-09-10T21:16:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb4f600640 1 -- 172.21.15.16:0/974944358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb480b75c0 con 0x7fdb480cea70 2024-09-10T21:16:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.738+0000 7fdb4f600640 1 -- 172.21.15.16:0/974944358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb480afac0 con 0x7fdb480cea70 2024-09-10T21:16:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.742+0000 7fdb44c00640 1 -- 172.21.15.16:0/974944358 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdb340051c0 con 0x7fdb480cea70 2024-09-10T21:16:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.742+0000 7fdb44c00640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdb280725e0 0x7fdb28074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.742+0000 7fdb44c00640 1 -- 172.21.15.16:0/974944358 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fdb340940c0 con 0x7fdb480cea70 2024-09-10T21:16:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.742+0000 7fdb4cc00640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdb280725e0 0x7fdb28074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.742+0000 7fdb4cc00640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdb280725e0 0x7fdb28074aa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0a9000 tx=0x7fdb3c019700 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.746+0000 7fdb44c00640 1 -- 172.21.15.16:0/974944358 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdb3405f680 con 0x7fdb480cea70 2024-09-10T21:16:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.954+0000 7fdb4f600640 1 -- 172.21.15.16:0/974944358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdb480cef70 con 0x7fdb480cea70 2024-09-10T21:16:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.954+0000 7fdb44c00640 1 -- 172.21.15.16:0/974944358 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) ==== 53+0+73 (secure 0 0 0) 0x7fdb3405ef80 con 0x7fdb480cea70 2024-09-10T21:16:48.959 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:16:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.958+0000 7fdb4f600640 1 -- 172.21.15.16:0/974944358 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdb280725e0 msgr2=0x7fdb28074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.958+0000 7fdb4f600640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdb280725e0 0x7fdb28074aa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0a9000 tx=0x7fdb3c019700 comp rx=0 tx=0).stop 2024-09-10T21:16:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.958+0000 7fdb4f600640 1 -- 172.21.15.16:0/974944358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480cea70 msgr2=0x7fdb48161920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.958+0000 7fdb4f600640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480cea70 0x7fdb48161920 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fdb340118e0 tx=0x7fdb34011da0 comp rx=0 tx=0).stop 2024-09-10T21:16:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.958+0000 7fdb4f600640 1 -- 172.21.15.16:0/974944358 shutdown_connections 2024-09-10T21:16:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.958+0000 7fdb4f600640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdb280725e0 0x7fdb28074aa0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.958+0000 7fdb4f600640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb48161e60 0x7fdb48162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.958+0000 7fdb4f600640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb480cea70 0x7fdb48161920 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.958+0000 7fdb4f600640 1 --2- 172.21.15.16:0/974944358 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdb30003c70 0x7fdb4815b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.958+0000 7fdb4f600640 1 -- 172.21.15.16:0/974944358 >> 172.21.15.16:0/974944358 conn(0x7fdb4809b130 msgr2=0x7fdb48167450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.958+0000 7fdb4f600640 1 -- 172.21.15.16:0/974944358 shutdown_connections 2024-09-10T21:16:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:48.958+0000 7fdb4f600640 1 -- 172.21.15.16:0/974944358 wait complete. 2024-09-10T21:16:48.980 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:16:48.980 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:16:48.980 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-09-10T21:16:48.980 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T21:16:48.980 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:16:48.980 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:16:48.980 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:16:48.981 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:16:48.981 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:16:48.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:48.981 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-09-10T21:16:48.992 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:16:48.992 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:16:48.992 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:16:48.992 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:48.992 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:16:48.992 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:48.992 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:16:48.992 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:16:48.992 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-09-10T21:16:48.993 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:16:48.993 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:16:48.993 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-09-10T21:16:48.993 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:16:48.993 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-09-10T21:16:48.993 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-09-10T21:16:48.993 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:16:48.993 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42042sec preferred_lft 42042sec 2024-09-10T21:16:48.993 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-09-10T21:16:48.993 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:48.993 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:16:48.993 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-09-10T21:16:48.993 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-09-10T21:16:48.994 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:16:48.994 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:7e:b4:71:a2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:16:48.994 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:16:48.994 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:48.994 INFO:teuthology.orchestra.run.smithi052.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:16:48.994 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:16:48.994 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:16:48.994 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:48.994 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:16:48.994 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:48.994 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:16:48.994 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:16:48.994 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:16:48.995 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:48.995 INFO:teuthology.orchestra.run.smithi052.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:16:48.995 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:16:48.995 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:16:48.995 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:16:48.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:48.996 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:16:49.060 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-09-10T21:16:49.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:49.063 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-09-10T21:16:49.114 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-09-10T21:16:49.114 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:16:49.114 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:16:49.114 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:16:49.115 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:16:49.115 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:16:49.115 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:16:49.115 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:16:49.115 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:16:49.115 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:16:49.115 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:16:49.115 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:16:49.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:16:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:16:49.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:16:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:16:49.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:49.266 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:16:49.275 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:16:49.275 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:16:49.276 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:16:49.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:49.276 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-09-10T21:16:49.340 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:16:49.340 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T21:16:49.392 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:16:49.393 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:16:49.393 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:16:49.444 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T21:16:49.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:49.444 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:16:49.538 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:16:49.538 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:16:49.551 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:16:49.552 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:16:49.552 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:16:49.552 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 1@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:16:49.552 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:16:49.555 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:16:49.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:49.556 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:16:49.576 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:16:49.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:16:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:16:49.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:16:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:16:49.677 INFO:teuthology.orchestra.run.smithi052.stdout:client6942 2024-09-10T21:16:49.678 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:16:49.678 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:49.687 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:16:49.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:16:49.774 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:16:49.846 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:16:49.846 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:16:49.847 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:16:49.847 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:49.856 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:16:49.856 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:49.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:16:50.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.106+0000 7f20a5400640 1 -- 172.21.15.16:0/94381540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20840053d0 msgr2=0x7f208400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:50.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.106+0000 7f20a5400640 1 --2- 172.21.15.16:0/94381540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20840053d0 0x7f208400b9d0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2098001dc0 tx=0x7f209801cb30 comp rx=0 tx=0).stop 2024-09-10T21:16:50.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.106+0000 7f20a5400640 1 -- 172.21.15.16:0/94381540 shutdown_connections 2024-09-10T21:16:50.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.106+0000 7f20a5400640 1 --2- 172.21.15.16:0/94381540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20840053d0 0x7f208400b9d0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.106+0000 7f20a5400640 1 -- 172.21.15.16:0/94381540 >> 172.21.15.16:0/94381540 conn(0x7f20a009b130 msgr2=0x7f20a00d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:50.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.106+0000 7f20a5400640 1 -- 172.21.15.16:0/94381540 shutdown_connections 2024-09-10T21:16:50.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.106+0000 7f20a5400640 1 -- 172.21.15.16:0/94381540 wait complete. 2024-09-10T21:16:50.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f20a5400640 1 Processor -- start 2024-09-10T21:16:50.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f20a5400640 1 -- start start 2024-09-10T21:16:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f20a5400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20840053d0 0x7f20a006f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f20a5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a014be80 0x7f20a00b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f20a5400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f20a00ce200 0x7f20a00ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f20a5400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20a00b1080 con 0x7f20a014be80 2024-09-10T21:16:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f20a5400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f20a00d1260 con 0x7f20840053d0 2024-09-10T21:16:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f20a5400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f20a0070fe0 con 0x7f20a00ce200 2024-09-10T21:16:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f2097e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a014be80 0x7f20a00b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f209e800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20840053d0 0x7f20a006f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f2097e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a014be80 0x7f20a00b55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47850/0 (socket says 172.21.15.16:47850) 2024-09-10T21:16:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f2097e00640 1 -- 172.21.15.16:0/826588361 learned_addr learned my addr 172.21.15.16:0/826588361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f209f200640 1 --2- 172.21.15.16:0/826588361 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f20a00ce200 0x7f20a00ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f2097e00640 1 -- 172.21.15.16:0/826588361 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f20a00ce200 msgr2=0x7f20a00ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f2097e00640 1 --2- 172.21.15.16:0/826588361 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f20a00ce200 0x7f20a00ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f2097e00640 1 -- 172.21.15.16:0/826588361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20840053d0 msgr2=0x7f20a006f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f2097e00640 1 --2- 172.21.15.16:0/826588361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20840053d0 0x7f20a006f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f2097e00640 1 -- 172.21.15.16:0/826588361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a00b53c0 con 0x7f20a014be80 2024-09-10T21:16:50.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f2097e00640 1 --2- 172.21.15.16:0/826588361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a014be80 0x7f20a00b55c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2088006690 tx=0x7f20880039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f2096a00640 1 -- 172.21.15.16:0/826588361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088015070 con 0x7f20a014be80 2024-09-10T21:16:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f2096a00640 1 -- 172.21.15.16:0/826588361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2088011840 con 0x7f20a014be80 2024-09-10T21:16:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f2096a00640 1 -- 172.21.15.16:0/826588361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088011c70 con 0x7f20a014be80 2024-09-10T21:16:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f20a5400640 1 -- 172.21.15.16:0/826588361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a014c300 con 0x7f20a014be80 2024-09-10T21:16:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.110+0000 7f20a5400640 1 -- 172.21.15.16:0/826588361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a00b4e00 con 0x7f20a014be80 2024-09-10T21:16:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.114+0000 7f20a5400640 1 -- 172.21.15.16:0/826588361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a00cfc80 con 0x7f20a014be80 2024-09-10T21:16:50.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.114+0000 7f2096a00640 1 -- 172.21.15.16:0/826588361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f208800f3c0 con 0x7f20a014be80 2024-09-10T21:16:50.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.114+0000 7f2096a00640 1 --2- 172.21.15.16:0/826588361 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f207c072610 0x7f207c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:50.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.114+0000 7f2096a00640 1 -- 172.21.15.16:0/826588361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f2088093c90 con 0x7f20a014be80 2024-09-10T21:16:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.114+0000 7f209e800640 1 --2- 172.21.15.16:0/826588361 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f207c072610 0x7f207c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.114+0000 7f209e800640 1 --2- 172.21.15.16:0/826588361 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f207c072610 0x7f207c074ad0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2090005fd0 tx=0x7f2090005f40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.118+0000 7f2096a00640 1 -- 172.21.15.16:0/826588361 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f20a00b53c0 con 0x7f20a014be80 2024-09-10T21:16:50.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.302+0000 7f20a5400640 1 -- 172.21.15.16:0/826588361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f20a00b2770 con 0x7f20a014be80 2024-09-10T21:16:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.302+0000 7f2096a00640 1 -- 172.21.15.16:0/826588361 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v92) ==== 80+0+0 (secure 0 0 0) 0x7f208805ed10 con 0x7f20a014be80 2024-09-10T21:16:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:16:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.306+0000 7f20a5400640 1 -- 172.21.15.16:0/826588361 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f207c072610 msgr2=0x7f207c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.306+0000 7f20a5400640 1 --2- 172.21.15.16:0/826588361 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f207c072610 0x7f207c074ad0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2090005fd0 tx=0x7f2090005f40 comp rx=0 tx=0).stop 2024-09-10T21:16:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.306+0000 7f20a5400640 1 -- 172.21.15.16:0/826588361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a014be80 msgr2=0x7f20a00b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.306+0000 7f20a5400640 1 --2- 172.21.15.16:0/826588361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a014be80 0x7f20a00b55c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2088006690 tx=0x7f20880039d0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.306+0000 7f20a5400640 1 -- 172.21.15.16:0/826588361 shutdown_connections 2024-09-10T21:16:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.306+0000 7f20a5400640 1 --2- 172.21.15.16:0/826588361 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f207c072610 0x7f207c074ad0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.306+0000 7f20a5400640 1 --2- 172.21.15.16:0/826588361 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f20a00ce200 0x7f20a00ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.306+0000 7f20a5400640 1 --2- 172.21.15.16:0/826588361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a014be80 0x7f20a00b55c0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.306+0000 7f20a5400640 1 --2- 172.21.15.16:0/826588361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20840053d0 0x7f20a006f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.306+0000 7f20a5400640 1 -- 172.21.15.16:0/826588361 >> 172.21.15.16:0/826588361 conn(0x7f20a009b130 msgr2=0x7f20a0156e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.306+0000 7f20a5400640 1 -- 172.21.15.16:0/826588361 shutdown_connections 2024-09-10T21:16:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.306+0000 7f20a5400640 1 -- 172.21.15.16:0/826588361 wait complete. 2024-09-10T21:16:50.329 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:16:50.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:50.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:16:50.356 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:16:50.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:50.356 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:16:50.412 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:16:50.412 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:16:50.412 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:50.423 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:16:50.423 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:50.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:16:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.618+0000 7fe781400640 1 -- 172.21.15.16:0/1447575694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe760004da0 msgr2=0x7fe760005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.618+0000 7fe781400640 1 --2- 172.21.15.16:0/1447575694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe760004da0 0x7fe760005220 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe774009e90 tx=0x7fe77401d5b0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.618+0000 7fe77b200640 1 -- 172.21.15.16:0/1447575694 reap_dead start 2024-09-10T21:16:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.618+0000 7fe781400640 1 -- 172.21.15.16:0/1447575694 shutdown_connections 2024-09-10T21:16:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.618+0000 7fe781400640 1 -- 172.21.15.16:0/1447575694 >> 172.21.15.16:0/1447575694 conn(0x7fe77c0cfb20 msgr2=0x7fe77c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.618+0000 7fe781400640 1 -- 172.21.15.16:0/1447575694 shutdown_connections 2024-09-10T21:16:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.618+0000 7fe781400640 1 -- 172.21.15.16:0/1447575694 wait complete. 2024-09-10T21:16:50.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.618+0000 7fe781400640 1 Processor -- start 2024-09-10T21:16:50.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.618+0000 7fe781400640 1 -- start start 2024-09-10T21:16:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe781400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe760004da0 0x7fe77c0c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe781400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe77c0c6bf0 0x7fe77c0c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe781400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe77c0c8c90 0x7fe77c0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe781400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c163980 con 0x7fe760004da0 2024-09-10T21:16:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe781400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c0cb6c0 con 0x7fe77c0c6bf0 2024-09-10T21:16:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe781400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe77c163b00 con 0x7fe77c0c8c90 2024-09-10T21:16:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe77a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe760004da0 0x7fe77c0c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe77a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe760004da0 0x7fe77c0c2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47874/0 (socket says 172.21.15.16:47874) 2024-09-10T21:16:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe77a800640 1 -- 172.21.15.16:0/2432963468 learned_addr learned my addr 172.21.15.16:0/2432963468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe77b200640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe77c0c8c90 0x7fe77c0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe773e00640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe77c0c6bf0 0x7fe77c0c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe77a800640 1 -- 172.21.15.16:0/2432963468 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe77c0c8c90 msgr2=0x7fe77c0c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe77a800640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe77c0c8c90 0x7fe77c0c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe77a800640 1 -- 172.21.15.16:0/2432963468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe77c0c6bf0 msgr2=0x7fe77c0c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe77a800640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe77c0c6bf0 0x7fe77c0c7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe77a800640 1 -- 172.21.15.16:0/2432963468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77c15aaa0 con 0x7fe760004da0 2024-09-10T21:16:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe77b200640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe77c0c8c90 0x7fe77c0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe773e00640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe77c0c6bf0 0x7fe77c0c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe77a800640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe760004da0 0x7fe77c0c2a90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe76c001440 tx=0x7fe76c001900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe772a00640 1 -- 172.21.15.16:0/2432963468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00f720 con 0x7fe760004da0 2024-09-10T21:16:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe772a00640 1 -- 172.21.15.16:0/2432963468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe76c016020 con 0x7fe760004da0 2024-09-10T21:16:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe772a00640 1 -- 172.21.15.16:0/2432963468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00fb80 con 0x7fe760004da0 2024-09-10T21:16:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe781400640 1 -- 172.21.15.16:0/2432963468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe77c0c9ac0 con 0x7fe760004da0 2024-09-10T21:16:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe781400640 1 -- 172.21.15.16:0/2432963468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe77c0b2ba0 con 0x7fe760004da0 2024-09-10T21:16:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe781400640 1 -- 172.21.15.16:0/2432963468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe77c073520 con 0x7fe760004da0 2024-09-10T21:16:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.622+0000 7fe772a00640 1 -- 172.21.15.16:0/2432963468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe76c00fd20 con 0x7fe760004da0 2024-09-10T21:16:50.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.626+0000 7fe772a00640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe758072610 0x7fe758074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:50.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.626+0000 7fe772a00640 1 -- 172.21.15.16:0/2432963468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fe76c093cd0 con 0x7fe760004da0 2024-09-10T21:16:50.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.626+0000 7fe773e00640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe758072610 0x7fe758074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:50.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.626+0000 7fe773e00640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe758072610 0x7fe758074ad0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe7640036b0 tx=0x7fe764003600 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.630+0000 7fe772a00640 1 -- 172.21.15.16:0/2432963468 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe76c020020 con 0x7fe760004da0 2024-09-10T21:16:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe781400640 1 -- 172.21.15.16:0/2432963468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe77c154300 con 0x7fe760004da0 2024-09-10T21:16:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe772a00640 1 -- 172.21.15.16:0/2432963468 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v92) ==== 80+0+0 (secure 0 0 0) 0x7fe76c05f3a0 con 0x7fe760004da0 2024-09-10T21:16:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:16:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe781400640 1 -- 172.21.15.16:0/2432963468 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe758072610 msgr2=0x7fe758074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe781400640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe758072610 0x7fe758074ad0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe7640036b0 tx=0x7fe764003600 comp rx=0 tx=0).stop 2024-09-10T21:16:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe781400640 1 -- 172.21.15.16:0/2432963468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe760004da0 msgr2=0x7fe77c0c2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe781400640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe760004da0 0x7fe77c0c2a90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe76c001440 tx=0x7fe76c001900 comp rx=0 tx=0).stop 2024-09-10T21:16:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe781400640 1 -- 172.21.15.16:0/2432963468 shutdown_connections 2024-09-10T21:16:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe781400640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe758072610 0x7fe758074ad0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe781400640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe77c0c8c90 0x7fe77c0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe781400640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe77c0c6bf0 0x7fe77c0c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe781400640 1 --2- 172.21.15.16:0/2432963468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe760004da0 0x7fe77c0c2a90 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe781400640 1 -- 172.21.15.16:0/2432963468 >> 172.21.15.16:0/2432963468 conn(0x7fe77c0cfb20 msgr2=0x7fe77c1582d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:50.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe781400640 1 -- 172.21.15.16:0/2432963468 shutdown_connections 2024-09-10T21:16:50.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:50.814+0000 7fe781400640 1 -- 172.21.15.16:0/2432963468 wait complete. 2024-09-10T21:16:50.839 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T21:16:50.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:50.839 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T21:16:50.868 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:16:50.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:16:50.869 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T21:16:50.925 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:16:50.925 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:16:50.925 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:50.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:16:50.934 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:16:50.935 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:16:50.935 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:16:50.980 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:16:50.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:16:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.122+0000 7f9793400640 1 -- 172.21.15.16:0/1895968907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97740052f0 msgr2=0x7f977400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.122+0000 7f9793400640 1 --2- 172.21.15.16:0/1895968907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97740052f0 0x7f977400b9d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f9784009ee0 tx=0x7f978401d5e0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.122+0000 7f9793400640 1 -- 172.21.15.16:0/1895968907 shutdown_connections 2024-09-10T21:16:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.122+0000 7f9793400640 1 --2- 172.21.15.16:0/1895968907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97740052f0 0x7f977400b9d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.122+0000 7f9793400640 1 -- 172.21.15.16:0/1895968907 >> 172.21.15.16:0/1895968907 conn(0x7f978c09b130 msgr2=0x7f978c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.122+0000 7f9793400640 1 -- 172.21.15.16:0/1895968907 shutdown_connections 2024-09-10T21:16:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.122+0000 7f9793400640 1 -- 172.21.15.16:0/1895968907 wait complete. 2024-09-10T21:16:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9793400640 1 Processor -- start 2024-09-10T21:16:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9793400640 1 -- start start 2024-09-10T21:16:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9793400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97740052f0 0x7f978c0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9793400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f978c0c8e50 0x7f978c14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9793400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f978c155690 0x7f978c1f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9793400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f978c071160 con 0x7f978c0c8e50 2024-09-10T21:16:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9793400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f978c070fe0 con 0x7f978c155690 2024-09-10T21:16:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9793400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f978c0712e0 con 0x7f97740052f0 2024-09-10T21:16:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f978be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f978c0c8e50 0x7f978c14bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f978be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f978c0c8e50 0x7f978c14bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47886/0 (socket says 172.21.15.16:47886) 2024-09-10T21:16:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f978be00640 1 -- 172.21.15.16:0/2769065860 learned_addr learned my addr 172.21.15.16:0/2769065860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9790a00640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97740052f0 0x7f978c0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9791400640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f978c155690 0x7f978c1f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f978be00640 1 -- 172.21.15.16:0/2769065860 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97740052f0 msgr2=0x7f978c0cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f978be00640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97740052f0 0x7f978c0cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f978be00640 1 -- 172.21.15.16:0/2769065860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f978c155690 msgr2=0x7f978c1f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f978be00640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f978c155690 0x7f978c1f8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f978be00640 1 -- 172.21.15.16:0/2769065860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c0aef70 con 0x7f978c0c8e50 2024-09-10T21:16:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f978be00640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f978c0c8e50 0x7f978c14bd60 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f97780066c0 tx=0x7f9778001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9788c00640 1 -- 172.21.15.16:0/2769065860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778015020 con 0x7f978c0c8e50 2024-09-10T21:16:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9788c00640 1 -- 172.21.15.16:0/2769065860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9778011750 con 0x7f978c0c8e50 2024-09-10T21:16:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9788c00640 1 -- 172.21.15.16:0/2769065860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778011b60 con 0x7f978c0c8e50 2024-09-10T21:16:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9793400640 1 -- 172.21.15.16:0/2769065860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978c15cb50 con 0x7f978c0c8e50 2024-09-10T21:16:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.126+0000 7f9793400640 1 -- 172.21.15.16:0/2769065860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978c0c9b80 con 0x7f978c0c8e50 2024-09-10T21:16:51.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.130+0000 7f9793400640 1 -- 172.21.15.16:0/2769065860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f978c0d1ad0 con 0x7f978c0c8e50 2024-09-10T21:16:51.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.130+0000 7f9788c00640 1 -- 172.21.15.16:0/2769065860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f977801a400 con 0x7f978c0c8e50 2024-09-10T21:16:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.130+0000 7f9788c00640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f976c0725e0 0x7f976c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.130+0000 7f9788c00640 1 -- 172.21.15.16:0/2769065860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f9778093dc0 con 0x7f978c0c8e50 2024-09-10T21:16:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.130+0000 7f9790a00640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f976c0725e0 0x7f976c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.130+0000 7f9790a00640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f976c0725e0 0x7f976c074aa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f97800012a0 tx=0x7f9780006040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:51.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.134+0000 7f9788c00640 1 -- 172.21.15.16:0/2769065860 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f977801f020 con 0x7f978c0c8e50 2024-09-10T21:16:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.342+0000 7f9793400640 1 -- 172.21.15.16:0/2769065860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f978c0b2d80 con 0x7f978c0c8e50 2024-09-10T21:16:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.342+0000 7f9788c00640 1 -- 172.21.15.16:0/2769065860 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5229 (secure 0 0 0) 0x7f978c15cb50 con 0x7f978c0c8e50 2024-09-10T21:16:51.346 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:51.346 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":62,"btime":"2024-09-10T21:16:44:430553+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/895621033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":895621033},{"type":"v1","addr":"172.21.15.52:6835","nonce":895621033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6789,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2281471863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2281471863},{"type":"v1","addr":"172.21.15.16:6835","nonce":2281471863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1879915011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1879915011},{"type":"v1","addr":"172.21.15.16:6837","nonce":1879915011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:16:40.823255+0000","modified":"2024-09-10T21:16:43.834820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135725":{"gid":135725,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/585458541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":585458541},{"type":"v1","addr":"172.21.15.52:6834","nonce":585458541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135725,"qdb_cluster":[135725]},"id":7}]} 2024-09-10T21:16:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 62 2024-09-10T21:16:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.342+0000 7f9793400640 1 -- 172.21.15.16:0/2769065860 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f976c0725e0 msgr2=0x7f976c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.342+0000 7f9793400640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f976c0725e0 0x7f976c074aa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f97800012a0 tx=0x7f9780006040 comp rx=0 tx=0).stop 2024-09-10T21:16:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.346+0000 7f9793400640 1 -- 172.21.15.16:0/2769065860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f978c0c8e50 msgr2=0x7f978c14bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.346+0000 7f9793400640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f978c0c8e50 0x7f978c14bd60 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f97780066c0 tx=0x7f9778001640 comp rx=0 tx=0).stop 2024-09-10T21:16:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.346+0000 7f9793400640 1 -- 172.21.15.16:0/2769065860 shutdown_connections 2024-09-10T21:16:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.346+0000 7f9793400640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f976c0725e0 0x7f976c074aa0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.346+0000 7f9793400640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f978c155690 0x7f978c1f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.346+0000 7f9793400640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f978c0c8e50 0x7f978c14bd60 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.346+0000 7f9793400640 1 --2- 172.21.15.16:0/2769065860 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f97740052f0 0x7f978c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.346+0000 7f9793400640 1 -- 172.21.15.16:0/2769065860 >> 172.21.15.16:0/2769065860 conn(0x7f978c09b130 msgr2=0x7f978c0ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.346+0000 7f9793400640 1 -- 172.21.15.16:0/2769065860 shutdown_connections 2024-09-10T21:16:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.346+0000 7f9793400640 1 -- 172.21.15.16:0/2769065860 wait complete. 2024-09-10T21:16:51.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:16:51.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.509+0000 7f4cba200640 1 -- 172.21.15.16:0/2891395959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980053a0 msgr2=0x7f4c9800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:51.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.509+0000 7f4cba200640 1 --2- 172.21.15.16:0/2891395959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980053a0 0x7f4c9800b9d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4cac001f50 tx=0x7f4cac01cc40 comp rx=0 tx=0).stop 2024-09-10T21:16:51.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.509+0000 7f4cba200640 1 -- 172.21.15.16:0/2891395959 shutdown_connections 2024-09-10T21:16:51.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.509+0000 7f4cba200640 1 --2- 172.21.15.16:0/2891395959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980053a0 0x7f4c9800b9d0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.509+0000 7f4cba200640 1 -- 172.21.15.16:0/2891395959 >> 172.21.15.16:0/2891395959 conn(0x7f4cb409b130 msgr2=0x7f4cb40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:51.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cba200640 1 -- 172.21.15.16:0/2891395959 shutdown_connections 2024-09-10T21:16:51.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cba200640 1 -- 172.21.15.16:0/2891395959 wait complete. 2024-09-10T21:16:51.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cba200640 1 Processor -- start 2024-09-10T21:16:51.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cba200640 1 -- start start 2024-09-10T21:16:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cba200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980053a0 0x7f4cb40c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cba200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cb40ca630 0x7f4cb41576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cba200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4cb40b6bc0 0x7f4cb40b7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cba200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb40b1030 con 0x7f4c980053a0 2024-09-10T21:16:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cba200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb40d1260 con 0x7f4cb40ca630 2024-09-10T21:16:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cba200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4cb4070fe0 con 0x7f4cb40b6bc0 2024-09-10T21:16:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cb3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980053a0 0x7f4cb40c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cb3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980053a0 0x7f4cb40c86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47914/0 (socket says 172.21.15.16:47914) 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cb3400640 1 -- 172.21.15.16:0/4089613808 learned_addr learned my addr 172.21.15.16:0/4089613808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cabe00640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cb40ca630 0x7f4cb41576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cb3e00640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4cb40b6bc0 0x7f4cb40b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cb3400640 1 -- 172.21.15.16:0/4089613808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4cb40b6bc0 msgr2=0x7f4cb40b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cb3400640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4cb40b6bc0 0x7f4cb40b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cb3400640 1 -- 172.21.15.16:0/4089613808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cb40ca630 msgr2=0x7f4cb41576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cb3400640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cb40ca630 0x7f4cb41576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cb3400640 1 -- 172.21.15.16:0/4089613808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb4158000 con 0x7f4c980053a0 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cb3e00640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4cb40b6bc0 0x7f4cb40b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cabe00640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cb40ca630 0x7f4cb41576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cb3400640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980053a0 0x7f4cb40c86c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40118d0 tx=0x7f4ca4011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cab400640 1 -- 172.21.15.16:0/4089613808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca401e070 con 0x7f4c980053a0 2024-09-10T21:16:51.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cab400640 1 -- 172.21.15.16:0/4089613808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ca400fbb0 con 0x7f4c980053a0 2024-09-10T21:16:51.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.513+0000 7f4cab400640 1 -- 172.21.15.16:0/4089613808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4019410 con 0x7f4c980053a0 2024-09-10T21:16:51.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.517+0000 7f4cba200640 1 -- 172.21.15.16:0/4089613808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb41582e0 con 0x7f4c980053a0 2024-09-10T21:16:51.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.517+0000 7f4cba200640 1 -- 172.21.15.16:0/4089613808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb4157bf0 con 0x7f4c980053a0 2024-09-10T21:16:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.517+0000 7f4cba200640 1 -- 172.21.15.16:0/4089613808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb40d1ad0 con 0x7f4c980053a0 2024-09-10T21:16:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.517+0000 7f4cab400640 1 -- 172.21.15.16:0/4089613808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4ca40195b0 con 0x7f4c980053a0 2024-09-10T21:16:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.517+0000 7f4cab400640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4c90072610 0x7f4c90074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.517+0000 7f4cab400640 1 -- 172.21.15.16:0/4089613808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f4ca4093e00 con 0x7f4c980053a0 2024-09-10T21:16:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.517+0000 7f4cabe00640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4c90072610 0x7f4c90074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.517+0000 7f4cabe00640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4c90072610 0x7f4c90074ad0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c001870 tx=0x7f4c9c0010d0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:51.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.521+0000 7f4cab400640 1 -- 172.21.15.16:0/4089613808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4ca405ed70 con 0x7f4c980053a0 2024-09-10T21:16:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.729+0000 7f4cba200640 1 -- 172.21.15.16:0/4089613808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cb40b28a0 con 0x7f4c980053a0 2024-09-10T21:16:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.729+0000 7f4cab400640 1 -- 172.21.15.16:0/4089613808 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5229 (secure 0 0 0) 0x7f4ca40637a0 con 0x7f4c980053a0 2024-09-10T21:16:51.734 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:51.734 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":62,"btime":"2024-09-10T21:16:44:430553+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/895621033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":895621033},{"type":"v1","addr":"172.21.15.52:6835","nonce":895621033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6789,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2281471863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2281471863},{"type":"v1","addr":"172.21.15.16:6835","nonce":2281471863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1879915011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1879915011},{"type":"v1","addr":"172.21.15.16:6837","nonce":1879915011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:16:40.823255+0000","modified":"2024-09-10T21:16:43.834820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135725":{"gid":135725,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/585458541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":585458541},{"type":"v1","addr":"172.21.15.52:6834","nonce":585458541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135725,"qdb_cluster":[135725]},"id":7}]} 2024-09-10T21:16:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 62 2024-09-10T21:16:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.733+0000 7f4cba200640 1 -- 172.21.15.16:0/4089613808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4c90072610 msgr2=0x7f4c90074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.733+0000 7f4cba200640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4c90072610 0x7f4c90074ad0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c001870 tx=0x7f4c9c0010d0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.733+0000 7f4cba200640 1 -- 172.21.15.16:0/4089613808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980053a0 msgr2=0x7f4cb40c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.733+0000 7f4cba200640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980053a0 0x7f4cb40c86c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40118d0 tx=0x7f4ca4011d90 comp rx=0 tx=0).stop 2024-09-10T21:16:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.733+0000 7f4cba200640 1 -- 172.21.15.16:0/4089613808 shutdown_connections 2024-09-10T21:16:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.733+0000 7f4cba200640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4c90072610 0x7f4c90074ad0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.733+0000 7f4cba200640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4cb40b6bc0 0x7f4cb40b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.733+0000 7f4cba200640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cb40ca630 0x7f4cb41576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.733+0000 7f4cba200640 1 --2- 172.21.15.16:0/4089613808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980053a0 0x7f4cb40c86c0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.733+0000 7f4cba200640 1 -- 172.21.15.16:0/4089613808 >> 172.21.15.16:0/4089613808 conn(0x7f4cb409b130 msgr2=0x7f4cb40b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.733+0000 7f4cba200640 1 -- 172.21.15.16:0/4089613808 shutdown_connections 2024-09-10T21:16:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.733+0000 7f4cba200640 1 -- 172.21.15.16:0/4089613808 wait complete. 2024-09-10T21:16:51.757 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:16:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.897+0000 7f14df400640 1 -- 172.21.15.16:0/3773346786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d815f830 msgr2=0x7f14d80ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.897+0000 7f14df400640 1 --2- 172.21.15.16:0/3773346786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d815f830 0x7f14d80ca170 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009840 tx=0x7f14c401c820 comp rx=0 tx=0).stop 2024-09-10T21:16:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.897+0000 7f14df400640 1 -- 172.21.15.16:0/3773346786 shutdown_connections 2024-09-10T21:16:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.897+0000 7f14df400640 1 --2- 172.21.15.16:0/3773346786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d815f830 0x7f14d80ca170 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.897+0000 7f14df400640 1 -- 172.21.15.16:0/3773346786 >> 172.21.15.16:0/3773346786 conn(0x7f14d80cfb20 msgr2=0x7f14d80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14df400640 1 -- 172.21.15.16:0/3773346786 shutdown_connections 2024-09-10T21:16:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14df400640 1 -- 172.21.15.16:0/3773346786 wait complete. 2024-09-10T21:16:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14df400640 1 Processor -- start 2024-09-10T21:16:51.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14df400640 1 -- start start 2024-09-10T21:16:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14df400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d80ca6b0 0x7f14d81587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14df400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14d81599d0 0x7f14d80caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14df400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d815f830 0x7f14d80cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14df400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8169d60 con 0x7f14d815f830 2024-09-10T21:16:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14df400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8169be0 con 0x7f14d80ca6b0 2024-09-10T21:16:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14df400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f14d8169ee0 con 0x7f14d81599d0 2024-09-10T21:16:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14dd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d815f830 0x7f14d80cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14dd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d815f830 0x7f14d80cb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47920/0 (socket says 172.21.15.16:47920) 2024-09-10T21:16:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14dd400640 1 -- 172.21.15.16:0/1406694696 learned_addr learned my addr 172.21.15.16:0/1406694696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14d7e00640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14d81599d0 0x7f14d80caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14dca00640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d80ca6b0 0x7f14d81587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14dd400640 1 -- 172.21.15.16:0/1406694696 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14d81599d0 msgr2=0x7f14d80caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14dd400640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14d81599d0 0x7f14d80caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14dd400640 1 -- 172.21.15.16:0/1406694696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d80ca6b0 msgr2=0x7f14d81587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14dd400640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d80ca6b0 0x7f14d81587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:51.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14dd400640 1 -- 172.21.15.16:0/1406694696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d80b47d0 con 0x7f14d815f830 2024-09-10T21:16:51.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.901+0000 7f14dd400640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d815f830 0x7f14d80cb020 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f14d000cb30 tx=0x7f14d000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:51.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.905+0000 7f14d4c00640 1 -- 172.21.15.16:0/1406694696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0016020 con 0x7f14d815f830 2024-09-10T21:16:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.905+0000 7f14d4c00640 1 -- 172.21.15.16:0/1406694696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14d000ab40 con 0x7f14d815f830 2024-09-10T21:16:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.905+0000 7f14d4c00640 1 -- 172.21.15.16:0/1406694696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0012420 con 0x7f14d815f830 2024-09-10T21:16:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.905+0000 7f14df400640 1 -- 172.21.15.16:0/1406694696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d80b2b60 con 0x7f14d815f830 2024-09-10T21:16:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.905+0000 7f14df400640 1 -- 172.21.15.16:0/1406694696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d80b4410 con 0x7f14d815f830 2024-09-10T21:16:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.905+0000 7f14df400640 1 -- 172.21.15.16:0/1406694696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d80b2950 con 0x7f14d815f830 2024-09-10T21:16:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.905+0000 7f14d4c00640 1 -- 172.21.15.16:0/1406694696 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f14d00125c0 con 0x7f14d815f830 2024-09-10T21:16:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.905+0000 7f14d4c00640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14b80725e0 0x7f14b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:51.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.909+0000 7f14d4c00640 1 -- 172.21.15.16:0/1406694696 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f14d0093780 con 0x7f14d815f830 2024-09-10T21:16:51.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.909+0000 7f14dca00640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14b80725e0 0x7f14b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:51.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.909+0000 7f14dca00640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14b80725e0 0x7f14b8074aa0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f14cc007fd0 tx=0x7f14cc009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:51.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:51.909+0000 7f14d4c00640 1 -- 172.21.15.16:0/1406694696 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f14d0020020 con 0x7f14d815f830 2024-09-10T21:16:52.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.085+0000 7f14df400640 1 -- 172.21.15.16:0/1406694696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14d80ca170 con 0x7f14d815f830 2024-09-10T21:16:52.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.089+0000 7f14d4c00640 1 -- 172.21.15.16:0/1406694696 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16620 (secure 0 0 0) 0x7f14d005e770 con 0x7f14d815f830 2024-09-10T21:16:52.092 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:52.093 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":92,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:16:42.404534+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:16:38.533164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T21:16:39.781552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:16:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.089+0000 7f14df400640 1 -- 172.21.15.16:0/1406694696 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14b80725e0 msgr2=0x7f14b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.089+0000 7f14df400640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14b80725e0 0x7f14b8074aa0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f14cc007fd0 tx=0x7f14cc009040 comp rx=0 tx=0).stop 2024-09-10T21:16:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.089+0000 7f14df400640 1 -- 172.21.15.16:0/1406694696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d815f830 msgr2=0x7f14d80cb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.089+0000 7f14df400640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d815f830 0x7f14d80cb020 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f14d000cb30 tx=0x7f14d000cc20 comp rx=0 tx=0).stop 2024-09-10T21:16:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.089+0000 7f14df400640 1 -- 172.21.15.16:0/1406694696 shutdown_connections 2024-09-10T21:16:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.089+0000 7f14df400640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14b80725e0 0x7f14b8074aa0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.089+0000 7f14df400640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d815f830 0x7f14d80cb020 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.089+0000 7f14df400640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14d81599d0 0x7f14d80caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.089+0000 7f14df400640 1 --2- 172.21.15.16:0/1406694696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d80ca6b0 0x7f14d81587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.089+0000 7f14df400640 1 -- 172.21.15.16:0/1406694696 >> 172.21.15.16:0/1406694696 conn(0x7f14d80cfb20 msgr2=0x7f14d80ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:52.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.089+0000 7f14df400640 1 -- 172.21.15.16:0/1406694696 shutdown_connections 2024-09-10T21:16:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.089+0000 7f14df400640 1 -- 172.21.15.16:0/1406694696 wait complete. 2024-09-10T21:16:52.115 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:16:52.115 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:16:52.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 -- 172.21.15.16:0/634001984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c40b3200 msgr2=0x7f18c40afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:52.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 --2- 172.21.15.16:0/634001984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c40b3200 0x7f18c40afae0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009840 tx=0x7f18ac01c820 comp rx=0 tx=0).stop 2024-09-10T21:16:52.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 -- 172.21.15.16:0/634001984 shutdown_connections 2024-09-10T21:16:52.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 --2- 172.21.15.16:0/634001984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c40b3200 0x7f18c40afae0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 -- 172.21.15.16:0/634001984 >> 172.21.15.16:0/634001984 conn(0x7f18c40d1d80 msgr2=0x7f18c40d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:52.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 -- 172.21.15.16:0/634001984 shutdown_connections 2024-09-10T21:16:52.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 -- 172.21.15.16:0/634001984 wait complete. 2024-09-10T21:16:52.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 Processor -- start 2024-09-10T21:16:52.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 -- start start 2024-09-10T21:16:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c40d0890 0x7f18c41fb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18c415f450 0x7f18c414be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c415df40 0x7f18c414c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18c40b0730 con 0x7f18c415df40 2024-09-10T21:16:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f18c40b08b0 con 0x7f18c40d0890 2024-09-10T21:16:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c9a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f18c40b0a30 con 0x7f18c415f450 2024-09-10T21:16:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c415df40 0x7f18c414c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c415df40 0x7f18c414c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59986/0 (socket says 172.21.15.16:59986) 2024-09-10T21:16:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.257+0000 7f18c3400640 1 -- 172.21.15.16:0/2862404337 learned_addr learned my addr 172.21.15.16:0/2862404337 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18bbe00640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18c415f450 0x7f18c414be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18c2a00640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c40d0890 0x7f18c41fb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18c3400640 1 -- 172.21.15.16:0/2862404337 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18c415f450 msgr2=0x7f18c414be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18c3400640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18c415f450 0x7f18c414be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18c3400640 1 -- 172.21.15.16:0/2862404337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c40d0890 msgr2=0x7f18c41fb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18c3400640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c40d0890 0x7f18c41fb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18c3400640 1 -- 172.21.15.16:0/2862404337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c41fbf20 con 0x7f18c415df40 2024-09-10T21:16:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18c3400640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c415df40 0x7f18c414c380 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00cb70 tx=0x7f18bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18baa00640 1 -- 172.21.15.16:0/2862404337 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc016020 con 0x7f18c415df40 2024-09-10T21:16:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18baa00640 1 -- 172.21.15.16:0/2862404337 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18bc00ab60 con 0x7f18c415df40 2024-09-10T21:16:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18baa00640 1 -- 172.21.15.16:0/2862404337 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc012420 con 0x7f18c415df40 2024-09-10T21:16:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18c9a00640 1 -- 172.21.15.16:0/2862404337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c40cdad0 con 0x7f18c415df40 2024-09-10T21:16:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18c9a00640 1 -- 172.21.15.16:0/2862404337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c40ce090 con 0x7f18c415df40 2024-09-10T21:16:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18c9a00640 1 -- 172.21.15.16:0/2862404337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c40cfc80 con 0x7f18c415df40 2024-09-10T21:16:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.261+0000 7f18baa00640 1 -- 172.21.15.16:0/2862404337 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f18bc0125c0 con 0x7f18c415df40 2024-09-10T21:16:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.265+0000 7f18baa00640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18a00725e0 0x7f18a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.265+0000 7f18baa00640 1 -- 172.21.15.16:0/2862404337 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f18bc093500 con 0x7f18c415df40 2024-09-10T21:16:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.265+0000 7f18c2a00640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18a00725e0 0x7f18a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.265+0000 7f18c2a00640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18a00725e0 0x7f18a0074aa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f18b4001130 tx=0x7f18b4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:52.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.269+0000 7f18baa00640 1 -- 172.21.15.16:0/2862404337 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f18bc020030 con 0x7f18c415df40 2024-09-10T21:16:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.477+0000 7f18c9a00640 1 -- 172.21.15.16:0/2862404337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18c4074ba0 con 0x7f18c415df40 2024-09-10T21:16:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.477+0000 7f18baa00640 1 -- 172.21.15.16:0/2862404337 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5229 (secure 0 0 0) 0x7f18bc05ebd0 con 0x7f18c415df40 2024-09-10T21:16:52.482 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:52.483 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":62,"btime":"2024-09-10T21:16:44:430553+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/895621033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":895621033},{"type":"v1","addr":"172.21.15.52:6835","nonce":895621033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6789,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2281471863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2281471863},{"type":"v1","addr":"172.21.15.16:6835","nonce":2281471863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1879915011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1879915011},{"type":"v1","addr":"172.21.15.16:6837","nonce":1879915011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:16:40.823255+0000","modified":"2024-09-10T21:16:43.834820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135725":{"gid":135725,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/585458541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":585458541},{"type":"v1","addr":"172.21.15.52:6834","nonce":585458541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135725,"qdb_cluster":[135725]},"id":7}]} 2024-09-10T21:16:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 62 2024-09-10T21:16:52.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.477+0000 7f18c9a00640 1 -- 172.21.15.16:0/2862404337 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18a00725e0 msgr2=0x7f18a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.477+0000 7f18c9a00640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18a00725e0 0x7f18a0074aa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f18b4001130 tx=0x7f18b4009040 comp rx=0 tx=0).stop 2024-09-10T21:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.477+0000 7f18c9a00640 1 -- 172.21.15.16:0/2862404337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c415df40 msgr2=0x7f18c414c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.477+0000 7f18c9a00640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c415df40 0x7f18c414c380 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00cb70 tx=0x7f18bc00cc60 comp rx=0 tx=0).stop 2024-09-10T21:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.481+0000 7f18c9a00640 1 -- 172.21.15.16:0/2862404337 shutdown_connections 2024-09-10T21:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.481+0000 7f18c9a00640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18a00725e0 0x7f18a0074aa0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.481+0000 7f18c9a00640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c415df40 0x7f18c414c380 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.481+0000 7f18c9a00640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f18c415f450 0x7f18c414be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.481+0000 7f18c9a00640 1 --2- 172.21.15.16:0/2862404337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c40d0890 0x7f18c41fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.481+0000 7f18c9a00640 1 -- 172.21.15.16:0/2862404337 >> 172.21.15.16:0/2862404337 conn(0x7f18c40d1d80 msgr2=0x7f18c414e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.481+0000 7f18c9a00640 1 -- 172.21.15.16:0/2862404337 shutdown_connections 2024-09-10T21:16:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.481+0000 7f18c9a00640 1 -- 172.21.15.16:0/2862404337 wait complete. 2024-09-10T21:16:52.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:16:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.645+0000 7f5f17e00640 1 -- 172.21.15.16:0/3243946081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180d2820 msgr2=0x7f5f180b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.645+0000 7f5f17e00640 1 --2- 172.21.15.16:0/3243946081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180d2820 0x7f5f180b1600 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009840 tx=0x7f5f0001c790 comp rx=0 tx=0).stop 2024-09-10T21:16:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.645+0000 7f5f17e00640 1 -- 172.21.15.16:0/3243946081 shutdown_connections 2024-09-10T21:16:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.645+0000 7f5f17e00640 1 --2- 172.21.15.16:0/3243946081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180d2820 0x7f5f180b1600 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.645+0000 7f5f17e00640 1 -- 172.21.15.16:0/3243946081 >> 172.21.15.16:0/3243946081 conn(0x7f5f180d1d00 msgr2=0x7f5f180d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.645+0000 7f5f17e00640 1 -- 172.21.15.16:0/3243946081 shutdown_connections 2024-09-10T21:16:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.645+0000 7f5f17e00640 1 -- 172.21.15.16:0/3243946081 wait complete. 2024-09-10T21:16:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f17e00640 1 Processor -- start 2024-09-10T21:16:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f17e00640 1 -- start start 2024-09-10T21:16:52.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180b1b40 0x7f5f1815af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:52.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f180cfd40 0x7f5f180cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f180d2820 0x7f5f1806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f17e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18169e70 con 0x7f5f180b1b40 2024-09-10T21:16:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f17e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18169cf0 con 0x7f5f180d2820 2024-09-10T21:16:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f17e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5f18169ff0 con 0x7f5f180cfd40 2024-09-10T21:16:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f16a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180b1b40 0x7f5f1815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f16a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180b1b40 0x7f5f1815af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60000/0 (socket says 172.21.15.16:60000) 2024-09-10T21:16:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f16a00640 1 -- 172.21.15.16:0/3543469338 learned_addr learned my addr 172.21.15.16:0/3543469338 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f16a00640 1 -- 172.21.15.16:0/3543469338 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f180cfd40 msgr2=0x7f5f180cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f0fe00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f180cfd40 0x7f5f180cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f17400640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f180d2820 0x7f5f1806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f16a00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f180cfd40 0x7f5f180cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f16a00640 1 -- 172.21.15.16:0/3543469338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f180d2820 msgr2=0x7f5f1806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f16a00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f180d2820 0x7f5f1806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f16a00640 1 -- 172.21.15.16:0/3543469338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f181f7c20 con 0x7f5f180b1b40 2024-09-10T21:16:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f0fe00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f180cfd40 0x7f5f180cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f17400640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f180d2820 0x7f5f1806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f16a00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180b1b40 0x7f5f1815af60 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c500 tx=0x7f5f0800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f0ea00640 1 -- 172.21.15.16:0/3543469338 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08017070 con 0x7f5f180b1b40 2024-09-10T21:16:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f0ea00640 1 -- 172.21.15.16:0/3543469338 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f080053c0 con 0x7f5f180b1b40 2024-09-10T21:16:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f0ea00640 1 -- 172.21.15.16:0/3543469338 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08012890 con 0x7f5f180b1b40 2024-09-10T21:16:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f17e00640 1 -- 172.21.15.16:0/3543469338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f181f7f00 con 0x7f5f180b1b40 2024-09-10T21:16:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f17e00640 1 -- 172.21.15.16:0/3543469338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f180b4b50 con 0x7f5f180b1b40 2024-09-10T21:16:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.649+0000 7f5f17e00640 1 -- 172.21.15.16:0/3543469338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f181f89c0 con 0x7f5f180b1b40 2024-09-10T21:16:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.653+0000 7f5f0ea00640 1 -- 172.21.15.16:0/3543469338 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5f08003600 con 0x7f5f180b1b40 2024-09-10T21:16:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.653+0000 7f5f0ea00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5ef40725e0 0x7f5ef4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.653+0000 7f5f0ea00640 1 -- 172.21.15.16:0/3543469338 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f5f08092b30 con 0x7f5f180b1b40 2024-09-10T21:16:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.653+0000 7f5f0fe00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5ef40725e0 0x7f5ef4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.653+0000 7f5f0fe00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5ef40725e0 0x7f5ef4074aa0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5f000af000 tx=0x7f5f0001afd0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:52.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.657+0000 7f5f0ea00640 1 -- 172.21.15.16:0/3543469338 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f08021080 con 0x7f5f180b1b40 2024-09-10T21:16:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.865+0000 7f5f17e00640 1 -- 172.21.15.16:0/3543469338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f180b1600 con 0x7f5f180b1b40 2024-09-10T21:16:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.865+0000 7f5f0ea00640 1 -- 172.21.15.16:0/3543469338 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5229 (secure 0 0 0) 0x7f5f0805e200 con 0x7f5f180b1b40 2024-09-10T21:16:52.872 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:52.872 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":62,"btime":"2024-09-10T21:16:44:430553+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/895621033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":895621033},{"type":"v1","addr":"172.21.15.52:6835","nonce":895621033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6789,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2281471863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2281471863},{"type":"v1","addr":"172.21.15.16:6835","nonce":2281471863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1879915011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1879915011},{"type":"v1","addr":"172.21.15.16:6837","nonce":1879915011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:16:40.823255+0000","modified":"2024-09-10T21:16:43.834820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135725":{"gid":135725,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/585458541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":585458541},{"type":"v1","addr":"172.21.15.52:6834","nonce":585458541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135725,"qdb_cluster":[135725]},"id":7}]} 2024-09-10T21:16:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 62 2024-09-10T21:16:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.869+0000 7f5f17e00640 1 -- 172.21.15.16:0/3543469338 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5ef40725e0 msgr2=0x7f5ef4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.869+0000 7f5f17e00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5ef40725e0 0x7f5ef4074aa0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5f000af000 tx=0x7f5f0001afd0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.869+0000 7f5f17e00640 1 -- 172.21.15.16:0/3543469338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180b1b40 msgr2=0x7f5f1815af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.869+0000 7f5f17e00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180b1b40 0x7f5f1815af60 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c500 tx=0x7f5f0800c9c0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.869+0000 7f5f17e00640 1 -- 172.21.15.16:0/3543469338 shutdown_connections 2024-09-10T21:16:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.869+0000 7f5f17e00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5ef40725e0 0x7f5ef4074aa0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.869+0000 7f5f17e00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f180d2820 0x7f5f1806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.869+0000 7f5f17e00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5f180cfd40 0x7f5f180cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.869+0000 7f5f17e00640 1 --2- 172.21.15.16:0/3543469338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f180b1b40 0x7f5f1815af60 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.869+0000 7f5f17e00640 1 -- 172.21.15.16:0/3543469338 >> 172.21.15.16:0/3543469338 conn(0x7f5f180d1d00 msgr2=0x7f5f1815f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.869+0000 7f5f17e00640 1 -- 172.21.15.16:0/3543469338 shutdown_connections 2024-09-10T21:16:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:52.869+0000 7f5f17e00640 1 -- 172.21.15.16:0/3543469338 wait complete. 2024-09-10T21:16:52.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:16:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 -- 172.21.15.16:0/1579745000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d80b3200 msgr2=0x7f07d80afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 --2- 172.21.15.16:0/1579745000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d80b3200 0x7f07d80afae0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009840 tx=0x7f07c001c7c0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 -- 172.21.15.16:0/1579745000 shutdown_connections 2024-09-10T21:16:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 --2- 172.21.15.16:0/1579745000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d80b3200 0x7f07d80afae0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 --2- 172.21.15.16:0/1579745000 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d80d0890 0x7f07d80b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 -- 172.21.15.16:0/1579745000 >> 172.21.15.16:0/1579745000 conn(0x7f07d80d1d80 msgr2=0x7f07d80d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 -- 172.21.15.16:0/1579745000 shutdown_connections 2024-09-10T21:16:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 -- 172.21.15.16:0/1579745000 wait complete. 2024-09-10T21:16:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 Processor -- start 2024-09-10T21:16:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 -- start start 2024-09-10T21:16:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d80d0890 0x7f07d806f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d815f450 0x7f07d806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d81fb040 0x7f07d81565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07d8168d50 con 0x7f07d81fb040 2024-09-10T21:16:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07d8168bd0 con 0x7f07d80d0890 2024-09-10T21:16:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07dd200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f07d8168ed0 con 0x7f07d815f450 2024-09-10T21:16:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07d6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d81fb040 0x7f07d81565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07d6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d81fb040 0x7f07d81565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60014/0 (socket says 172.21.15.16:60014) 2024-09-10T21:16:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07d6800640 1 -- 172.21.15.16:0/2956898021 learned_addr learned my addr 172.21.15.16:0/2956898021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07cfe00640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d815f450 0x7f07d806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.037+0000 7f07d5e00640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d80d0890 0x7f07d806f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07d6800640 1 -- 172.21.15.16:0/2956898021 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d815f450 msgr2=0x7f07d806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07d6800640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d815f450 0x7f07d806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07d6800640 1 -- 172.21.15.16:0/2956898021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d80d0890 msgr2=0x7f07d806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07d6800640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d80d0890 0x7f07d806f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07d6800640 1 -- 172.21.15.16:0/2956898021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d81fb590 con 0x7f07d81fb040 2024-09-10T21:16:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07cfe00640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d815f450 0x7f07d806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07d6800640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d81fb040 0x7f07d81565b0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f07d000daa0 tx=0x7f07d000df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07ce000640 1 -- 172.21.15.16:0/2956898021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0017070 con 0x7f07d81fb040 2024-09-10T21:16:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07ce000640 1 -- 172.21.15.16:0/2956898021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07d0004b30 con 0x7f07d81fb040 2024-09-10T21:16:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07ce000640 1 -- 172.21.15.16:0/2956898021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0005020 con 0x7f07d81fb040 2024-09-10T21:16:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07dd200640 1 -- 172.21.15.16:0/2956898021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d81fb870 con 0x7f07d81fb040 2024-09-10T21:16:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07dd200640 1 -- 172.21.15.16:0/2956898021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d81fbe30 con 0x7f07d81fb040 2024-09-10T21:16:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07dd200640 1 -- 172.21.15.16:0/2956898021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d80cfc80 con 0x7f07d81fb040 2024-09-10T21:16:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07ce000640 1 -- 172.21.15.16:0/2956898021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f07d00051c0 con 0x7f07d81fb040 2024-09-10T21:16:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07ce000640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07b40725e0 0x7f07b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.041+0000 7f07ce000640 1 -- 172.21.15.16:0/2956898021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f07d0093820 con 0x7f07d81fb040 2024-09-10T21:16:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.045+0000 7f07d5e00640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07b40725e0 0x7f07b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.045+0000 7f07d5e00640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07b40725e0 0x7f07b4074aa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f07d80af7d0 tx=0x7f07c8009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.045+0000 7f07ce000640 1 -- 172.21.15.16:0/2956898021 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f07d0020070 con 0x7f07d81fb040 2024-09-10T21:16:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.221+0000 7f07dd200640 1 -- 172.21.15.16:0/2956898021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07d8154570 con 0x7f07d81fb040 2024-09-10T21:16:53.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.221+0000 7f07ce000640 1 -- 172.21.15.16:0/2956898021 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16620 (secure 0 0 0) 0x7f07d005ee60 con 0x7f07d81fb040 2024-09-10T21:16:53.227 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:53.227 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":92,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:16:42.404534+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:16:38.533164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T21:16:39.781552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:16:53.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.225+0000 7f07dd200640 1 -- 172.21.15.16:0/2956898021 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07b40725e0 msgr2=0x7f07b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:53.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.225+0000 7f07dd200640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07b40725e0 0x7f07b4074aa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f07d80af7d0 tx=0x7f07c8009040 comp rx=0 tx=0).stop 2024-09-10T21:16:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.225+0000 7f07dd200640 1 -- 172.21.15.16:0/2956898021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d81fb040 msgr2=0x7f07d81565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.225+0000 7f07dd200640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d81fb040 0x7f07d81565b0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f07d000daa0 tx=0x7f07d000df60 comp rx=0 tx=0).stop 2024-09-10T21:16:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.225+0000 7f07dd200640 1 -- 172.21.15.16:0/2956898021 shutdown_connections 2024-09-10T21:16:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.225+0000 7f07dd200640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07b40725e0 0x7f07b4074aa0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.225+0000 7f07dd200640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07d81fb040 0x7f07d81565b0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.225+0000 7f07dd200640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07d815f450 0x7f07d806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.225+0000 7f07dd200640 1 --2- 172.21.15.16:0/2956898021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d80d0890 0x7f07d806f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.225+0000 7f07dd200640 1 -- 172.21.15.16:0/2956898021 >> 172.21.15.16:0/2956898021 conn(0x7f07d80d1d80 msgr2=0x7f07d8071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.225+0000 7f07dd200640 1 -- 172.21.15.16:0/2956898021 shutdown_connections 2024-09-10T21:16:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.225+0000 7f07dd200640 1 -- 172.21.15.16:0/2956898021 wait complete. 2024-09-10T21:16:53.249 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:16:53.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.393+0000 7f7d71e00640 1 -- 172.21.15.16:0/4174757920 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d6c0b13c0 msgr2=0x7f7d6c0afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:53.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.393+0000 7f7d71e00640 1 --2- 172.21.15.16:0/4174757920 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d6c0b13c0 0x7f7d6c0afc90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009bf0 tx=0x7f7d6401cac0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.393+0000 7f7d71e00640 1 -- 172.21.15.16:0/4174757920 shutdown_connections 2024-09-10T21:16:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.393+0000 7f7d71e00640 1 --2- 172.21.15.16:0/4174757920 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d6c0b13c0 0x7f7d6c0afc90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.393+0000 7f7d71e00640 1 -- 172.21.15.16:0/4174757920 >> 172.21.15.16:0/4174757920 conn(0x7f7d6c09b130 msgr2=0x7f7d6c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.393+0000 7f7d71e00640 1 -- 172.21.15.16:0/4174757920 shutdown_connections 2024-09-10T21:16:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.393+0000 7f7d71e00640 1 -- 172.21.15.16:0/4174757920 wait complete. 2024-09-10T21:16:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.393+0000 7f7d71e00640 1 Processor -- start 2024-09-10T21:16:53.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.393+0000 7f7d71e00640 1 -- start start 2024-09-10T21:16:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d71e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d6c0716f0 0x7f7d6c153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d71e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d6c0b13c0 0x7f7d6c1585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d71e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15f400 0x7f7d6c0b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d71e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c16a030 con 0x7f7d6c15f400 2024-09-10T21:16:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d71e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c169eb0 con 0x7f7d6c0b13c0 2024-09-10T21:16:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d71e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6c16a1b0 con 0x7f7d6c0716f0 2024-09-10T21:16:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d6b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15f400 0x7f7d6c0b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d6b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15f400 0x7f7d6c0b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60024/0 (socket says 172.21.15.16:60024) 2024-09-10T21:16:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d6b400640 1 -- 172.21.15.16:0/2588674426 learned_addr learned my addr 172.21.15.16:0/2588674426 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d6aa00640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d6c0716f0 0x7f7d6c153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d63e00640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d6c0b13c0 0x7f7d6c1585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d6b400640 1 -- 172.21.15.16:0/2588674426 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d6c0716f0 msgr2=0x7f7d6c153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d6b400640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d6c0716f0 0x7f7d6c153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d6b400640 1 -- 172.21.15.16:0/2588674426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d6c0b13c0 msgr2=0x7f7d6c1585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d6b400640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d6c0b13c0 0x7f7d6c1585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d6b400640 1 -- 172.21.15.16:0/2588674426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c0cece0 con 0x7f7d6c15f400 2024-09-10T21:16:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d6b400640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15f400 0x7f7d6c0b27f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7d6401b170 tx=0x7f7d64009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d63400640 1 -- 172.21.15.16:0/2588674426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d640bb070 con 0x7f7d6c15f400 2024-09-10T21:16:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d63400640 1 -- 172.21.15.16:0/2588674426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d64004d40 con 0x7f7d6c15f400 2024-09-10T21:16:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d71e00640 1 -- 172.21.15.16:0/2588674426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c0cdc40 con 0x7f7d6c15f400 2024-09-10T21:16:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d63400640 1 -- 172.21.15.16:0/2588674426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d640b2040 con 0x7f7d6c15f400 2024-09-10T21:16:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d71e00640 1 -- 172.21.15.16:0/2588674426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c0ce200 con 0x7f7d6c15f400 2024-09-10T21:16:53.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d71e00640 1 -- 172.21.15.16:0/2588674426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6c070fe0 con 0x7f7d6c15f400 2024-09-10T21:16:53.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.397+0000 7f7d63400640 1 -- 172.21.15.16:0/2588674426 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7d640b5030 con 0x7f7d6c15f400 2024-09-10T21:16:53.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.401+0000 7f7d63400640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7d480725e0 0x7f7d48074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:53.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.401+0000 7f7d63400640 1 -- 172.21.15.16:0/2588674426 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f7d6412cf10 con 0x7f7d6c15f400 2024-09-10T21:16:53.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.401+0000 7f7d6aa00640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7d480725e0 0x7f7d48074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:53.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.401+0000 7f7d6aa00640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7d480725e0 0x7f7d48074aa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c007940 tx=0x7f7d5c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.405+0000 7f7d63400640 1 -- 172.21.15.16:0/2588674426 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7d640167d0 con 0x7f7d6c15f400 2024-09-10T21:16:53.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.613+0000 7f7d71e00640 1 -- 172.21.15.16:0/2588674426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7d6c0761e0 con 0x7f7d6c15f400 2024-09-10T21:16:53.620 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-10T21:16:53.617+0000 7ff429400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:16:53.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.617+0000 7f7d63400640 1 -- 172.21.15.16:0/2588674426 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) ==== 161+0+0 (secure 0 0 0) 0x7f7d640f8550 con 0x7f7d6c15f400 2024-09-10T21:16:53.621 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:16:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.621+0000 7f7d71e00640 1 -- 172.21.15.16:0/2588674426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7d480725e0 msgr2=0x7f7d48074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.621+0000 7f7d71e00640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7d480725e0 0x7f7d48074aa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c007940 tx=0x7f7d5c009040 comp rx=0 tx=0).stop 2024-09-10T21:16:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.621+0000 7f7d71e00640 1 -- 172.21.15.16:0/2588674426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15f400 msgr2=0x7f7d6c0b27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.621+0000 7f7d71e00640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15f400 0x7f7d6c0b27f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7d6401b170 tx=0x7f7d64009200 comp rx=0 tx=0).stop 2024-09-10T21:16:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.621+0000 7f7d71e00640 1 -- 172.21.15.16:0/2588674426 shutdown_connections 2024-09-10T21:16:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.621+0000 7f7d71e00640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7d480725e0 0x7f7d48074aa0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.621+0000 7f7d71e00640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c15f400 0x7f7d6c0b27f0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.621+0000 7f7d71e00640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d6c0b13c0 0x7f7d6c1585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.621+0000 7f7d71e00640 1 --2- 172.21.15.16:0/2588674426 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d6c0716f0 0x7f7d6c153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.621+0000 7f7d71e00640 1 -- 172.21.15.16:0/2588674426 >> 172.21.15.16:0/2588674426 conn(0x7f7d6c09b130 msgr2=0x7f7d6c0d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.621+0000 7f7d71e00640 1 -- 172.21.15.16:0/2588674426 shutdown_connections 2024-09-10T21:16:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.621+0000 7f7d71e00640 1 -- 172.21.15.16:0/2588674426 wait complete. 2024-09-10T21:16:53.632 INFO:tasks.ceph.mds.d.smithi052.stderr: -2180> 2024-09-10T21:16:43.389+0000 7f1ef1600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:16:53.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:16:53.681 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:16:53.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.789+0000 7f871f800640 1 -- 172.21.15.16:0/503988093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87180d0210 msgr2=0x7f87180ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:53.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.789+0000 7f871f800640 1 --2- 172.21.15.16:0/503988093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87180d0210 0x7f87180ceca0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f87180cda10 tx=0x7f870401f090 comp rx=0 tx=0).stop 2024-09-10T21:16:53.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.789+0000 7f871f800640 1 -- 172.21.15.16:0/503988093 shutdown_connections 2024-09-10T21:16:53.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.789+0000 7f871f800640 1 --2- 172.21.15.16:0/503988093 >> v2:172.21.15.52:3300/0 conn(0x7f87180ce040 0x7f87180b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.789+0000 7f871f800640 1 --2- 172.21.15.16:0/503988093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87180d0210 0x7f87180ceca0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.789+0000 7f871f800640 1 -- 172.21.15.16:0/503988093 >> 172.21.15.16:0/503988093 conn(0x7f87180d1df0 msgr2=0x7f87180d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:53.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.789+0000 7f871f800640 1 -- 172.21.15.16:0/503988093 shutdown_connections 2024-09-10T21:16:53.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.789+0000 7f871f800640 1 -- 172.21.15.16:0/503988093 wait complete. 2024-09-10T21:16:53.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.789+0000 7f871f800640 1 Processor -- start 2024-09-10T21:16:53.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.789+0000 7f871f800640 1 -- start start 2024-09-10T21:16:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871f800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87180ce040 0x7f871815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871f800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f87180cfe30 0x7f87181f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871f800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87180d0210 0x7f87181f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871f800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87180cf1e0 con 0x7f87180d0210 2024-09-10T21:16:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871f800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f87180b14f0 con 0x7f87180ce040 2024-09-10T21:16:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871f800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f87180cf360 con 0x7f87180cfe30 2024-09-10T21:16:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87180d0210 0x7f87181f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87180d0210 0x7f87181f8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60034/0 (socket says 172.21.15.16:60034) 2024-09-10T21:16:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871d800640 1 -- 172.21.15.16:0/3994520248 learned_addr learned my addr 172.21.15.16:0/3994520248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f8717e00640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f87180cfe30 0x7f87181f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871ce00640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87180ce040 0x7f871815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871d800640 1 -- 172.21.15.16:0/3994520248 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f87180cfe30 msgr2=0x7f87181f8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871d800640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f87180cfe30 0x7f87181f8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871d800640 1 -- 172.21.15.16:0/3994520248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87180ce040 msgr2=0x7f871815efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871d800640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87180ce040 0x7f871815efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871d800640 1 -- 172.21.15.16:0/3994520248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87180b5120 con 0x7f87180d0210 2024-09-10T21:16:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871d800640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87180d0210 0x7f87181f8540 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f871000eb20 tx=0x7f871000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f8714c00640 1 -- 172.21.15.16:0/3994520248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710004b30 con 0x7f87180d0210 2024-09-10T21:16:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f8714c00640 1 -- 172.21.15.16:0/3994520248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8710004cf0 con 0x7f87180d0210 2024-09-10T21:16:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871f800640 1 -- 172.21.15.16:0/3994520248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87181f8dd0 con 0x7f87180d0210 2024-09-10T21:16:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f8714c00640 1 -- 172.21.15.16:0/3994520248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710005280 con 0x7f87180d0210 2024-09-10T21:16:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871f800640 1 -- 172.21.15.16:0/3994520248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87180c83e0 con 0x7f87180d0210 2024-09-10T21:16:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f871f800640 1 -- 172.21.15.16:0/3994520248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87180b1710 con 0x7f87180d0210 2024-09-10T21:16:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.793+0000 7f8714c00640 1 -- 172.21.15.16:0/3994520248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8710005420 con 0x7f87180d0210 2024-09-10T21:16:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.797+0000 7f8714c00640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f86f80725e0 0x7f86f8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.797+0000 7f8714c00640 1 -- 172.21.15.16:0/3994520248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f8710093ea0 con 0x7f87180d0210 2024-09-10T21:16:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.797+0000 7f871ce00640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f86f80725e0 0x7f86f8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.797+0000 7f871ce00640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f86f80725e0 0x7f86f8074aa0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f870c007fd0 tx=0x7f870c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:53.801+0000 7f8714c00640 1 -- 172.21.15.16:0/3994520248 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f871005ee40 con 0x7f87180d0210 2024-09-10T21:16:53.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:16:53.994 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:16:54.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.009+0000 7f871f800640 1 -- 172.21.15.16:0/3994520248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f87180b52f0 con 0x7f87180d0210 2024-09-10T21:16:54.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.617+0000 7f8714c00640 1 -- 172.21.15.16:0/3994520248 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) ==== 104+0+0 (secure 0 0 0) 0x7f8710063870 con 0x7f87180d0210 2024-09-10T21:16:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.621+0000 7f871f800640 1 -- 172.21.15.16:0/3994520248 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f86f80725e0 msgr2=0x7f86f8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.621+0000 7f871f800640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f86f80725e0 0x7f86f8074aa0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f870c007fd0 tx=0x7f870c009040 comp rx=0 tx=0).stop 2024-09-10T21:16:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.621+0000 7f871f800640 1 -- 172.21.15.16:0/3994520248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87180d0210 msgr2=0x7f87181f8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:54.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.621+0000 7f871f800640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87180d0210 0x7f87181f8540 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f871000eb20 tx=0x7f871000c2d0 comp rx=0 tx=0).stop 2024-09-10T21:16:54.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.621+0000 7f871f800640 1 -- 172.21.15.16:0/3994520248 shutdown_connections 2024-09-10T21:16:54.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.621+0000 7f871f800640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f86f80725e0 0x7f86f8074aa0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:54.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.621+0000 7f871f800640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87180d0210 0x7f87181f8540 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:54.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.621+0000 7f871f800640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f87180cfe30 0x7f87181f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:54.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.621+0000 7f871f800640 1 --2- 172.21.15.16:0/3994520248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87180ce040 0x7f871815efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:54.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.621+0000 7f871f800640 1 -- 172.21.15.16:0/3994520248 >> 172.21.15.16:0/3994520248 conn(0x7f87180d1df0 msgr2=0x7f8718070960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:54.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.621+0000 7f871f800640 1 -- 172.21.15.16:0/3994520248 shutdown_connections 2024-09-10T21:16:54.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.621+0000 7f871f800640 1 -- 172.21.15.16:0/3994520248 wait complete. 2024-09-10T21:16:54.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:16:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.797+0000 7f4f3b400640 1 -- 172.21.15.16:0/1775155910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f34073bc0 msgr2=0x7f4f34073fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.797+0000 7f4f3b400640 1 --2- 172.21.15.16:0/1775155910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f34073bc0 0x7f4f34073fa0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009840 tx=0x7f4f2801c760 comp rx=0 tx=0).stop 2024-09-10T21:16:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.797+0000 7f4f3b400640 1 -- 172.21.15.16:0/1775155910 shutdown_connections 2024-09-10T21:16:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.797+0000 7f4f3b400640 1 --2- 172.21.15.16:0/1775155910 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4f340b2b10 0x7f4f3415a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.797+0000 7f4f3b400640 1 --2- 172.21.15.16:0/1775155910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f34073bc0 0x7f4f34073fa0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.797+0000 7f4f3b400640 1 -- 172.21.15.16:0/1775155910 >> 172.21.15.16:0/1775155910 conn(0x7f4f3406d1e0 msgr2=0x7f4f340b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.797+0000 7f4f3b400640 1 -- 172.21.15.16:0/1775155910 shutdown_connections 2024-09-10T21:16:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.797+0000 7f4f3b400640 1 -- 172.21.15.16:0/1775155910 wait complete. 2024-09-10T21:16:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.797+0000 7f4f3b400640 1 Processor -- start 2024-09-10T21:16:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.797+0000 7f4f3b400640 1 -- start start 2024-09-10T21:16:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f3b400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f34073bc0 0x7f4f341f9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f3b400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4f340b2b10 0x7f4f34163940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f3b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f340cda80 0x7f4f3406e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f3b400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f341517a0 con 0x7f4f340cda80 2024-09-10T21:16:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f3b400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f34151620 con 0x7f4f34073bc0 2024-09-10T21:16:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f3b400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4f340b1f20 con 0x7f4f340b2b10 2024-09-10T21:16:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f39400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f340cda80 0x7f4f3406e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f39400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f340cda80 0x7f4f3406e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60046/0 (socket says 172.21.15.16:60046) 2024-09-10T21:16:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f39400640 1 -- 172.21.15.16:0/3792271839 learned_addr learned my addr 172.21.15.16:0/3792271839 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f33e00640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4f340b2b10 0x7f4f34163940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f38a00640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f34073bc0 0x7f4f341f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f39400640 1 -- 172.21.15.16:0/3792271839 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4f340b2b10 msgr2=0x7f4f34163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f39400640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4f340b2b10 0x7f4f34163940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f39400640 1 -- 172.21.15.16:0/3792271839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f34073bc0 msgr2=0x7f4f341f9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f39400640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f34073bc0 0x7f4f341f9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f39400640 1 -- 172.21.15.16:0/3792271839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f341544e0 con 0x7f4f340cda80 2024-09-10T21:16:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f39400640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f340cda80 0x7f4f3406e490 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00cce0 tx=0x7f4f2c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f30c00640 1 -- 172.21.15.16:0/3792271839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c00f450 con 0x7f4f340cda80 2024-09-10T21:16:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f30c00640 1 -- 172.21.15.16:0/3792271839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f2c00f700 con 0x7f4f340cda80 2024-09-10T21:16:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f30c00640 1 -- 172.21.15.16:0/3792271839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c00fb50 con 0x7f4f340cda80 2024-09-10T21:16:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f3b400640 1 -- 172.21.15.16:0/3792271839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f341547c0 con 0x7f4f340cda80 2024-09-10T21:16:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f3b400640 1 -- 172.21.15.16:0/3792271839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f341f66e0 con 0x7f4f340cda80 2024-09-10T21:16:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.801+0000 7f4f3b400640 1 -- 172.21.15.16:0/3792271839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f340b24f0 con 0x7f4f340cda80 2024-09-10T21:16:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.805+0000 7f4f30c00640 1 -- 172.21.15.16:0/3792271839 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4f2c0213f0 con 0x7f4f340cda80 2024-09-10T21:16:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.805+0000 7f4f30c00640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4f14072610 0x7f4f14074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.805+0000 7f4f30c00640 1 -- 172.21.15.16:0/3792271839 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f4f2c093f40 con 0x7f4f340cda80 2024-09-10T21:16:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.805+0000 7f4f38a00640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4f14072610 0x7f4f14074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.805+0000 7f4f38a00640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4f14072610 0x7f4f14074ad0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4f280af000 tx=0x7f4f2801afa0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:54.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.809+0000 7f4f30c00640 1 -- 172.21.15.16:0/3792271839 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4f2c014030 con 0x7f4f340cda80 2024-09-10T21:16:54.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:54.989+0000 7f4f3b400640 1 -- 172.21.15.16:0/3792271839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4f34069af0 con 0x7f4f340cda80 2024-09-10T21:16:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.621+0000 7f4f30c00640 1 -- 172.21.15.16:0/3792271839 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) ==== 181+0+0 (secure 0 0 0) 0x7f4f2c05f520 con 0x7f4f340cda80 2024-09-10T21:16:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:16:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.625+0000 7f4f3b400640 1 -- 172.21.15.16:0/3792271839 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4f14072610 msgr2=0x7f4f14074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.625+0000 7f4f3b400640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4f14072610 0x7f4f14074ad0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4f280af000 tx=0x7f4f2801afa0 comp rx=0 tx=0).stop 2024-09-10T21:16:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.625+0000 7f4f3b400640 1 -- 172.21.15.16:0/3792271839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f340cda80 msgr2=0x7f4f3406e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.625+0000 7f4f3b400640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f340cda80 0x7f4f3406e490 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00cce0 tx=0x7f4f2c00a620 comp rx=0 tx=0).stop 2024-09-10T21:16:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.625+0000 7f4f3b400640 1 -- 172.21.15.16:0/3792271839 shutdown_connections 2024-09-10T21:16:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.625+0000 7f4f3b400640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4f14072610 0x7f4f14074ad0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.625+0000 7f4f3b400640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f340cda80 0x7f4f3406e490 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.625+0000 7f4f3b400640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4f340b2b10 0x7f4f34163940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.625+0000 7f4f3b400640 1 --2- 172.21.15.16:0/3792271839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f34073bc0 0x7f4f341f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.625+0000 7f4f3b400640 1 -- 172.21.15.16:0/3792271839 >> 172.21.15.16:0/3792271839 conn(0x7f4f3406d1e0 msgr2=0x7f4f340b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.625+0000 7f4f3b400640 1 -- 172.21.15.16:0/3792271839 shutdown_connections 2024-09-10T21:16:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.625+0000 7f4f3b400640 1 -- 172.21.15.16:0/3792271839 wait complete. 2024-09-10T21:16:55.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:16:55.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.805+0000 7f35e6400640 1 -- 172.21.15.16:0/1322135120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35e0143860 msgr2=0x7f35e015da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.805+0000 7f35e6400640 1 --2- 172.21.15.16:0/1322135120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35e0143860 0x7f35e015da90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009900 tx=0x7f35c801c8b0 comp rx=0 tx=0).stop 2024-09-10T21:16:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.805+0000 7f35e6400640 1 -- 172.21.15.16:0/1322135120 shutdown_connections 2024-09-10T21:16:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.805+0000 7f35e6400640 1 --2- 172.21.15.16:0/1322135120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35e0143860 0x7f35e015da90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.805+0000 7f35e6400640 1 --2- 172.21.15.16:0/1322135120 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f35e0143480 0x7f35e0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.805+0000 7f35e6400640 1 -- 172.21.15.16:0/1322135120 >> 172.21.15.16:0/1322135120 conn(0x7f35e006d1e0 msgr2=0x7f35e00b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.805+0000 7f35e6400640 1 -- 172.21.15.16:0/1322135120 shutdown_connections 2024-09-10T21:16:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.805+0000 7f35e6400640 1 -- 172.21.15.16:0/1322135120 wait complete. 2024-09-10T21:16:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35e6400640 1 Processor -- start 2024-09-10T21:16:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35e6400640 1 -- start start 2024-09-10T21:16:55.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35e6400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e0143480 0x7f35e015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35e6400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f35e0143860 0x7f35e00aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35e6400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35e015acb0 0x7f35e0162e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35e6400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35e015b2f0 con 0x7f35e0143480 2024-09-10T21:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35e6400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0070ad0 con 0x7f35e015acb0 2024-09-10T21:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35e6400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f35e015b470 con 0x7f35e0143860 2024-09-10T21:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35df400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e0143480 0x7f35e015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35df400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e0143480 0x7f35e015cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60060/0 (socket says 172.21.15.16:60060) 2024-09-10T21:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35df400640 1 -- 172.21.15.16:0/93913043 learned_addr learned my addr 172.21.15.16:0/93913043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35d7e00640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f35e0143860 0x7f35e00aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35dfe00640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35e015acb0 0x7f35e0162e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35df400640 1 -- 172.21.15.16:0/93913043 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f35e0143860 msgr2=0x7f35e00aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35df400640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f35e0143860 0x7f35e00aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35df400640 1 -- 172.21.15.16:0/93913043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35e015acb0 msgr2=0x7f35e0162e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35df400640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35e015acb0 0x7f35e0162e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35df400640 1 -- 172.21.15.16:0/93913043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e015ba90 con 0x7f35e0143480 2024-09-10T21:16:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35df400640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e0143480 0x7f35e015cb30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009750 tx=0x7f35d0009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35d7400640 1 -- 172.21.15.16:0/93913043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0018030 con 0x7f35e0143480 2024-09-10T21:16:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35d7400640 1 -- 172.21.15.16:0/93913043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35d000e8c0 con 0x7f35e0143480 2024-09-10T21:16:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35d7400640 1 -- 172.21.15.16:0/93913043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d000ed10 con 0x7f35e0143480 2024-09-10T21:16:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35e6400640 1 -- 172.21.15.16:0/93913043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e015bd70 con 0x7f35e0143480 2024-09-10T21:16:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35e6400640 1 -- 172.21.15.16:0/93913043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e015c330 con 0x7f35e0143480 2024-09-10T21:16:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.809+0000 7f35e6400640 1 -- 172.21.15.16:0/93913043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e0073b30 con 0x7f35e0143480 2024-09-10T21:16:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.813+0000 7f35d7400640 1 -- 172.21.15.16:0/93913043 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f35d001d410 con 0x7f35e0143480 2024-09-10T21:16:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.813+0000 7f35d7400640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f35bc0725e0 0x7f35bc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.813+0000 7f35d7400640 1 -- 172.21.15.16:0/93913043 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5357+0+0 (secure 0 0 0) 0x7f35d0093410 con 0x7f35e0143480 2024-09-10T21:16:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.813+0000 7f35d7e00640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f35bc0725e0 0x7f35bc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.813+0000 7f35d7e00640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f35bc0725e0 0x7f35bc074aa0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f35c80af000 tx=0x7f35c801b0f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.817+0000 7f35d7400640 1 -- 172.21.15.16:0/93913043 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f35d0022070 con 0x7f35e0143480 2024-09-10T21:16:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:55.989+0000 7f35e6400640 1 -- 172.21.15.16:0/93913043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f35e00b12b0 con 0x7f35e0143480 2024-09-10T21:16:56.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.621+0000 7f35d7400640 1 -- 172.21.15.16:0/93913043 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) ==== 169+0+0 (secure 0 0 0) 0x7f35d0091e20 con 0x7f35e0143480 2024-09-10T21:16:56.627 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-10T21:16:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.625+0000 7f35e6400640 1 -- 172.21.15.16:0/93913043 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f35bc0725e0 msgr2=0x7f35bc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.625+0000 7f35e6400640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f35bc0725e0 0x7f35bc074aa0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f35c80af000 tx=0x7f35c801b0f0 comp rx=0 tx=0).stop 2024-09-10T21:16:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.625+0000 7f35e6400640 1 -- 172.21.15.16:0/93913043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e0143480 msgr2=0x7f35e015cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.625+0000 7f35e6400640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e0143480 0x7f35e015cb30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009750 tx=0x7f35d0009c10 comp rx=0 tx=0).stop 2024-09-10T21:16:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.625+0000 7f35e6400640 1 -- 172.21.15.16:0/93913043 shutdown_connections 2024-09-10T21:16:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.625+0000 7f35e6400640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f35bc0725e0 0x7f35bc074aa0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.625+0000 7f35e6400640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35e015acb0 0x7f35e0162e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.625+0000 7f35e6400640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f35e0143860 0x7f35e00aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.625+0000 7f35e6400640 1 --2- 172.21.15.16:0/93913043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e0143480 0x7f35e015cb30 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.625+0000 7f35e6400640 1 -- 172.21.15.16:0/93913043 >> 172.21.15.16:0/93913043 conn(0x7f35e006d1e0 msgr2=0x7f35e00b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.625+0000 7f35e6400640 1 -- 172.21.15.16:0/93913043 shutdown_connections 2024-09-10T21:16:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.625+0000 7f35e6400640 1 -- 172.21.15.16:0/93913043 wait complete. 2024-09-10T21:16:56.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:16:56.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 -- 172.21.15.16:0/2306820457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a9006e640 msgr2=0x7f3a9015c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:56.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 --2- 172.21.15.16:0/2306820457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a9006e640 0x7f3a9015c990 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f3a9006f7a0 tx=0x7f3a7c01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:16:56.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 -- 172.21.15.16:0/2306820457 shutdown_connections 2024-09-10T21:16:56.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 --2- 172.21.15.16:0/2306820457 >> v2:172.21.15.52:3300/0 conn(0x7f3a9015ced0 0x7f3a900ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:56.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 --2- 172.21.15.16:0/2306820457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a9006e640 0x7f3a9015c990 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:56.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 --2- 172.21.15.16:0/2306820457 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3a900b5050 0x7f3a9015f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:56.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 -- 172.21.15.16:0/2306820457 >> 172.21.15.16:0/2306820457 conn(0x7f3a900cfb20 msgr2=0x7f3a900cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:56.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 -- 172.21.15.16:0/2306820457 shutdown_connections 2024-09-10T21:16:56.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 -- 172.21.15.16:0/2306820457 wait complete. 2024-09-10T21:16:56.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 Processor -- start 2024-09-10T21:16:56.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 -- start start 2024-09-10T21:16:56.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9006e640 0x7f3a90155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:56.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a900b5050 0x7f3a90155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3a9015ced0 0x7f3a901f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a90074250 con 0x7f3a900b5050 2024-09-10T21:16:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a900740d0 con 0x7f3a9006e640 2024-09-10T21:16:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a98600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3a900743d0 con 0x7f3a9015ced0 2024-09-10T21:16:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a95200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a900b5050 0x7f3a90155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a95200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a900b5050 0x7f3a90155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60088/0 (socket says 172.21.15.16:60088) 2024-09-10T21:16:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a95200640 1 -- 172.21.15.16:0/2341313671 learned_addr learned my addr 172.21.15.16:0/2341313671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a96600640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3a9015ced0 0x7f3a901f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.805+0000 7f3a95c00640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9006e640 0x7f3a90155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a95200640 1 -- 172.21.15.16:0/2341313671 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3a9015ced0 msgr2=0x7f3a901f8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a95200640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3a9015ced0 0x7f3a901f8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a95200640 1 -- 172.21.15.16:0/2341313671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9006e640 msgr2=0x7f3a90155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a95200640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9006e640 0x7f3a90155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a95200640 1 -- 172.21.15.16:0/2341313671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a901f9a80 con 0x7f3a900b5050 2024-09-10T21:16:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a95200640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a900b5050 0x7f3a90155170 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c01af70 tx=0x7f3a7c0a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a8d600640 1 -- 172.21.15.16:0/2341313671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0b2040 con 0x7f3a900b5050 2024-09-10T21:16:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a8d600640 1 -- 172.21.15.16:0/2341313671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a7c004490 con 0x7f3a900b5050 2024-09-10T21:16:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a8d600640 1 -- 172.21.15.16:0/2341313671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c005290 con 0x7f3a900b5050 2024-09-10T21:16:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a98600640 1 -- 172.21.15.16:0/2341313671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a90156920 con 0x7f3a900b5050 2024-09-10T21:16:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a98600640 1 -- 172.21.15.16:0/2341313671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a900b3d20 con 0x7f3a900b5050 2024-09-10T21:16:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a98600640 1 -- 172.21.15.16:0/2341313671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a90074850 con 0x7f3a900b5050 2024-09-10T21:16:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a8d600640 1 -- 172.21.15.16:0/2341313671 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3a90072ec0 con 0x7f3a900b5050 2024-09-10T21:16:56.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a8d600640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3a700725e0 0x7f3a70074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:56.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.809+0000 7f3a8d600640 1 -- 172.21.15.16:0/2341313671 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f3a7c12c110 con 0x7f3a900b5050 2024-09-10T21:16:56.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.813+0000 7f3a95c00640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3a700725e0 0x7f3a70074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:56.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.813+0000 7f3a95c00640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3a700725e0 0x7f3a70074aa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009be0 tx=0x7f3a840091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:56.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.813+0000 7f3a8d600640 1 -- 172.21.15.16:0/2341313671 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3a7c0164b0 con 0x7f3a900b5050 2024-09-10T21:16:56.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.989+0000 7f3a98600640 1 -- 172.21.15.16:0/2341313671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f3a9015bc20 con 0x7f3a900b5050 2024-09-10T21:16:56.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.993+0000 7f3a8d600640 1 -- 172.21.15.16:0/2341313671 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v46) ==== 71+0+0 (secure 0 0 0) 0x7f3a7c100070 con 0x7f3a900b5050 2024-09-10T21:16:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.993+0000 7f3a98600640 1 -- 172.21.15.16:0/2341313671 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3a700725e0 msgr2=0x7f3a70074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.993+0000 7f3a98600640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3a700725e0 0x7f3a70074aa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009be0 tx=0x7f3a840091e0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.993+0000 7f3a98600640 1 -- 172.21.15.16:0/2341313671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a900b5050 msgr2=0x7f3a90155170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.993+0000 7f3a98600640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a900b5050 0x7f3a90155170 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c01af70 tx=0x7f3a7c0a5650 comp rx=0 tx=0).stop 2024-09-10T21:16:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.993+0000 7f3a98600640 1 -- 172.21.15.16:0/2341313671 shutdown_connections 2024-09-10T21:16:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.993+0000 7f3a98600640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3a700725e0 0x7f3a70074aa0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.993+0000 7f3a98600640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3a9015ced0 0x7f3a901f8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.993+0000 7f3a98600640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a900b5050 0x7f3a90155170 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.993+0000 7f3a98600640 1 --2- 172.21.15.16:0/2341313671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a9006e640 0x7f3a90155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.993+0000 7f3a98600640 1 -- 172.21.15.16:0/2341313671 >> 172.21.15.16:0/2341313671 conn(0x7f3a900cfb20 msgr2=0x7f3a900cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:57.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.997+0000 7f3a98600640 1 -- 172.21.15.16:0/2341313671 shutdown_connections 2024-09-10T21:16:57.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:56.997+0000 7f3a98600640 1 -- 172.21.15.16:0/2341313671 wait complete. 2024-09-10T21:16:57.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2024-09-10T21:16:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 -- 172.21.15.16:0/186512184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604003a60 msgr2=0x7f9604003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 --2- 172.21.15.16:0/186512184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604003a60 0x7f9604003e80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f9610009b60 tx=0x7f961001d280 comp rx=0 tx=0).stop 2024-09-10T21:16:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f961f200640 1 -- 172.21.15.16:0/186512184 reap_dead start 2024-09-10T21:16:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 -- 172.21.15.16:0/186512184 shutdown_connections 2024-09-10T21:16:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 -- 172.21.15.16:0/186512184 >> 172.21.15.16:0/186512184 conn(0x7f96200cfb20 msgr2=0x7f96200cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 -- 172.21.15.16:0/186512184 shutdown_connections 2024-09-10T21:16:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 -- 172.21.15.16:0/186512184 wait complete. 2024-09-10T21:16:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 Processor -- start 2024-09-10T21:16:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 -- start start 2024-09-10T21:16:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604003a60 0x7f96200c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f962014ec50 0x7f96201f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f96201f6a70 0x7f96201f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96200ca510 con 0x7f9604003a60 2024-09-10T21:16:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f962006fc80 con 0x7f962014ec50 2024-09-10T21:16:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f9625400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f96200ca690 con 0x7f96201f6a70 2024-09-10T21:16:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f961e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604003a60 0x7f96200c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f961e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604003a60 0x7f96200c6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60102/0 (socket says 172.21.15.16:60102) 2024-09-10T21:16:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.165+0000 7f961e800640 1 -- 172.21.15.16:0/1082386754 learned_addr learned my addr 172.21.15.16:0/1082386754 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f9617e00640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f962014ec50 0x7f96201f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f961f200640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f96201f6a70 0x7f96201f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f961e800640 1 -- 172.21.15.16:0/1082386754 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f96201f6a70 msgr2=0x7f96201f6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f961e800640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f96201f6a70 0x7f96201f6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f961e800640 1 -- 172.21.15.16:0/1082386754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f962014ec50 msgr2=0x7f96201f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f961e800640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f962014ec50 0x7f96201f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f961e800640 1 -- 172.21.15.16:0/1082386754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96200c69c0 con 0x7f9604003a60 2024-09-10T21:16:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f9617e00640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f962014ec50 0x7f96201f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f961e800640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604003a60 0x7f96200c6250 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f961001d0b0 tx=0x7f96100055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f9616a00640 1 -- 172.21.15.16:0/1082386754 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96100b1040 con 0x7f9604003a60 2024-09-10T21:16:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f9616a00640 1 -- 172.21.15.16:0/1082386754 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96100ac400 con 0x7f9604003a60 2024-09-10T21:16:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f9616a00640 1 -- 172.21.15.16:0/1082386754 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610004290 con 0x7f9604003a60 2024-09-10T21:16:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f9625400640 1 -- 172.21.15.16:0/1082386754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96200c5240 con 0x7f9604003a60 2024-09-10T21:16:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f9625400640 1 -- 172.21.15.16:0/1082386754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96200c5800 con 0x7f9604003a60 2024-09-10T21:16:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f9625400640 1 -- 172.21.15.16:0/1082386754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96200b39a0 con 0x7f9604003a60 2024-09-10T21:16:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f9616a00640 1 -- 172.21.15.16:0/1082386754 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f96100046d0 con 0x7f9604003a60 2024-09-10T21:16:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.169+0000 7f9616a00640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f95fc0725e0 0x7f95fc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.173+0000 7f9616a00640 1 -- 172.21.15.16:0/1082386754 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f961012d730 con 0x7f9604003a60 2024-09-10T21:16:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.173+0000 7f9617e00640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f95fc0725e0 0x7f95fc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.173+0000 7f9617e00640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f95fc0725e0 0x7f95fc074aa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9608001250 tx=0x7f9608001120 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.177+0000 7f9616a00640 1 -- 172.21.15.16:0/1082386754 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f96100c4020 con 0x7f9604003a60 2024-09-10T21:16:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.409+0000 7f9625400640 1 -- 172.21.15.16:0/1082386754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) -- 0x7f9620072d40 con 0x7f9604003a60 2024-09-10T21:16:57.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.633+0000 7f9616a00640 1 -- 172.21.15.16:0/1082386754 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v322) ==== 150+0+0 (secure 0 0 0) 0x7f9610016fc0 con 0x7f9604003a60 2024-09-10T21:16:57.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.637+0000 7f9625400640 1 -- 172.21.15.16:0/1082386754 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f95fc0725e0 msgr2=0x7f95fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:57.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.637+0000 7f9625400640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f95fc0725e0 0x7f95fc074aa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9608001250 tx=0x7f9608001120 comp rx=0 tx=0).stop 2024-09-10T21:16:57.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.637+0000 7f9625400640 1 -- 172.21.15.16:0/1082386754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604003a60 msgr2=0x7f96200c6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:57.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.637+0000 7f9625400640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604003a60 0x7f96200c6250 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f961001d0b0 tx=0x7f96100055d0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.637+0000 7f9625400640 1 -- 172.21.15.16:0/1082386754 shutdown_connections 2024-09-10T21:16:57.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.637+0000 7f9625400640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f95fc0725e0 0x7f95fc074aa0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.637+0000 7f9625400640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f96201f6a70 0x7f96201f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.637+0000 7f9625400640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f962014ec50 0x7f96201f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.637+0000 7f9625400640 1 --2- 172.21.15.16:0/1082386754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604003a60 0x7f96200c6250 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.637+0000 7f9625400640 1 -- 172.21.15.16:0/1082386754 >> 172.21.15.16:0/1082386754 conn(0x7f96200cfb20 msgr2=0x7f9620150180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:57.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.637+0000 7f9625400640 1 -- 172.21.15.16:0/1082386754 shutdown_connections 2024-09-10T21:16:57.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.637+0000 7f9625400640 1 -- 172.21.15.16:0/1082386754 wait complete. 2024-09-10T21:16:57.660 INFO:tasks.cephfs_test_runner:When a client does not advance its oldest tid, the MDS should notice that ... skipped 'Require FUSE client to inject client release failure' 2024-09-10T21:16:57.660 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-10T21:16:57.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:16:57.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 -- 172.21.15.16:0/3687711820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72640cfb60 msgr2=0x7f72640cfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 --2- 172.21.15.16:0/3687711820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72640cfb60 0x7f72640cfff0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f726406dae0 tx=0x7f724c01c800 comp rx=0 tx=0).stop 2024-09-10T21:16:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 -- 172.21.15.16:0/3687711820 shutdown_connections 2024-09-10T21:16:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 --2- 172.21.15.16:0/3687711820 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f72640d0530 0x7f72640ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 --2- 172.21.15.16:0/3687711820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72640cfb60 0x7f72640cfff0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 --2- 172.21.15.16:0/3687711820 >> v2:172.21.15.52:3300/0 conn(0x7f726415f350 0x7f72640cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 -- 172.21.15.16:0/3687711820 >> 172.21.15.16:0/3687711820 conn(0x7f72640d1d80 msgr2=0x7f72640d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 -- 172.21.15.16:0/3687711820 shutdown_connections 2024-09-10T21:16:57.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 -- 172.21.15.16:0/3687711820 wait complete. 2024-09-10T21:16:57.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 Processor -- start 2024-09-10T21:16:57.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 -- start start 2024-09-10T21:16:57.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f72640cfb60 0x7f72641f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:57.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72640d0530 0x7f726414eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:57.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726415f350 0x7f72641fa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:57.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72640af210 con 0x7f726415f350 2024-09-10T21:16:57.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f72640af090 con 0x7f72640d0530 2024-09-10T21:16:57.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7268a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f72640af390 con 0x7f72640cfb60 2024-09-10T21:16:57.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7262800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726415f350 0x7f72641fa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:57.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7262800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726415f350 0x7f72641fa6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60120/0 (socket says 172.21.15.16:60120) 2024-09-10T21:16:57.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7262800640 1 -- 172.21.15.16:0/3398465245 learned_addr learned my addr 172.21.15.16:0/3398465245 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:57.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f725be00640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72640d0530 0x7f726414eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:57.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7261e00640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f72640cfb60 0x7f72641f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:57.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7262800640 1 -- 172.21.15.16:0/3398465245 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f72640cfb60 msgr2=0x7f72641f7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:57.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7262800640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f72640cfb60 0x7f72641f7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7262800640 1 -- 172.21.15.16:0/3398465245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72640d0530 msgr2=0x7f726414eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:57.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7262800640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72640d0530 0x7f726414eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:57.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.805+0000 7f7262800640 1 -- 172.21.15.16:0/3398465245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72641f8070 con 0x7f726415f350 2024-09-10T21:16:57.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.809+0000 7f7262800640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726415f350 0x7f72641fa6f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f725c00e990 tx=0x7f725c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:57.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.809+0000 7f725a000640 1 -- 172.21.15.16:0/3398465245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c004d30 con 0x7f726415f350 2024-09-10T21:16:57.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.809+0000 7f725a000640 1 -- 172.21.15.16:0/3398465245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f725c00cb00 con 0x7f726415f350 2024-09-10T21:16:57.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.809+0000 7f725a000640 1 -- 172.21.15.16:0/3398465245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c005020 con 0x7f726415f350 2024-09-10T21:16:57.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.809+0000 7f7268a00640 1 -- 172.21.15.16:0/3398465245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72641fb140 con 0x7f726415f350 2024-09-10T21:16:57.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.809+0000 7f7268a00640 1 -- 172.21.15.16:0/3398465245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72641f7330 con 0x7f726415f350 2024-09-10T21:16:57.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.809+0000 7f7268a00640 1 -- 172.21.15.16:0/3398465245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72640ce500 con 0x7f726415f350 2024-09-10T21:16:57.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.809+0000 7f725a000640 1 -- 172.21.15.16:0/3398465245 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f725c00f040 con 0x7f726415f350 2024-09-10T21:16:57.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.809+0000 7f725a000640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72400725e0 0x7f7240074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:57.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.809+0000 7f725a000640 1 -- 172.21.15.16:0/3398465245 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f725c0934b0 con 0x7f726415f350 2024-09-10T21:16:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.809+0000 7f7261e00640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72400725e0 0x7f7240074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.813+0000 7f7261e00640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72400725e0 0x7f7240074aa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f725400bb50 tx=0x7f725400b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.813+0000 7f725a000640 1 -- 172.21.15.16:0/3398465245 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f725c016030 con 0x7f726415f350 2024-09-10T21:16:57.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.989+0000 7f7268a00640 1 -- 172.21.15.16:0/3398465245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7264076250 con 0x7f726415f350 2024-09-10T21:16:57.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.993+0000 7f725a000640 1 -- 172.21.15.16:0/3398465245 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+12983 (secure 0 0 0) 0x7f725c05e830 con 0x7f726415f350 2024-09-10T21:16:57.996 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:16:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":95,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:16:56.624268+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6834/585458541":"2024-09-11T21:16:53.618876+0000","172.21.15.52:6832/585458541":"2024-09-11T21:16:53.618876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:16:58.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.993+0000 7f7268a00640 1 -- 172.21.15.16:0/3398465245 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72400725e0 msgr2=0x7f7240074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.993+0000 7f7268a00640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72400725e0 0x7f7240074aa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f725400bb50 tx=0x7f725400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.993+0000 7f7268a00640 1 -- 172.21.15.16:0/3398465245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726415f350 msgr2=0x7f72641fa6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.993+0000 7f7268a00640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726415f350 0x7f72641fa6f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f725c00e990 tx=0x7f725c00ee50 comp rx=0 tx=0).stop 2024-09-10T21:16:58.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.993+0000 7f7268a00640 1 -- 172.21.15.16:0/3398465245 shutdown_connections 2024-09-10T21:16:58.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.993+0000 7f7268a00640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72400725e0 0x7f7240074aa0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.993+0000 7f7268a00640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726415f350 0x7f72641fa6f0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.993+0000 7f7268a00640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72640d0530 0x7f726414eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.993+0000 7f7268a00640 1 --2- 172.21.15.16:0/3398465245 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f72640cfb60 0x7f72641f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.993+0000 7f7268a00640 1 -- 172.21.15.16:0/3398465245 >> 172.21.15.16:0/3398465245 conn(0x7f72640d1d80 msgr2=0x7f72640b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:58.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.993+0000 7f7268a00640 1 -- 172.21.15.16:0/3398465245 shutdown_connections 2024-09-10T21:16:58.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:57.993+0000 7f7268a00640 1 -- 172.21.15.16:0/3398465245 wait complete. 2024-09-10T21:16:58.018 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:10:17.493770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:16:58.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:16:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.169+0000 7f56ae600640 1 -- 172.21.15.16:0/1792740029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f568c003fe0 msgr2=0x7f568c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.169+0000 7f56ae600640 1 --2- 172.21.15.16:0/1792740029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f568c003fe0 0x7f568c004810 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f5690009b60 tx=0x7f569001d300 comp rx=0 tx=0).stop 2024-09-10T21:16:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.169+0000 7f56a7e00640 1 -- 172.21.15.16:0/1792740029 reap_dead start 2024-09-10T21:16:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.169+0000 7f56ae600640 1 -- 172.21.15.16:0/1792740029 shutdown_connections 2024-09-10T21:16:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.169+0000 7f56ae600640 1 -- 172.21.15.16:0/1792740029 >> 172.21.15.16:0/1792740029 conn(0x7f56a80d1df0 msgr2=0x7f56a80d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.169+0000 7f56ae600640 1 -- 172.21.15.16:0/1792740029 shutdown_connections 2024-09-10T21:16:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.169+0000 7f56ae600640 1 -- 172.21.15.16:0/1792740029 wait complete. 2024-09-10T21:16:58.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.169+0000 7f56ae600640 1 Processor -- start 2024-09-10T21:16:58.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.169+0000 7f56ae600640 1 -- start start 2024-09-10T21:16:58.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f56ae600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f568c003fe0 0x7f56a806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f56ae600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a81f5220 0x7f56a81582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f56ae600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a80cb8c0 0x7f56a80c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f56ae600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56a80b12e0 con 0x7f56a81f5220 2024-09-10T21:16:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f56ae600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f56a80d2b20 con 0x7f56a80cb8c0 2024-09-10T21:16:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f56ae600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f56a80b1460 con 0x7f568c003fe0 2024-09-10T21:16:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f569fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a81f5220 0x7f56a81582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f569fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a81f5220 0x7f56a81582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60144/0 (socket says 172.21.15.16:60144) 2024-09-10T21:16:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f569fe00640 1 -- 172.21.15.16:0/4204047669 learned_addr learned my addr 172.21.15.16:0/4204047669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f56a7400640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f568c003fe0 0x7f56a806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f56a7e00640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a80cb8c0 0x7f56a80c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f569fe00640 1 -- 172.21.15.16:0/4204047669 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f568c003fe0 msgr2=0x7f56a806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f569fe00640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f568c003fe0 0x7f56a806de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f569fe00640 1 -- 172.21.15.16:0/4204047669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a80cb8c0 msgr2=0x7f56a80c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f569fe00640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a80cb8c0 0x7f56a80c9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f569fe00640 1 -- 172.21.15.16:0/4204047669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a80b5d10 con 0x7f56a81f5220 2024-09-10T21:16:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f569fe00640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a81f5220 0x7f56a81582e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f569001d130 tx=0x7f56900055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f569f400640 1 -- 172.21.15.16:0/4204047669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56900b3040 con 0x7f56a81f5220 2024-09-10T21:16:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f569f400640 1 -- 172.21.15.16:0/4204047669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56900bd070 con 0x7f56a81f5220 2024-09-10T21:16:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f569f400640 1 -- 172.21.15.16:0/4204047669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690009730 con 0x7f56a81f5220 2024-09-10T21:16:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f56ae600640 1 -- 172.21.15.16:0/4204047669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a80b5ff0 con 0x7f56a81f5220 2024-09-10T21:16:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f56ae600640 1 -- 172.21.15.16:0/4204047669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a80b65b0 con 0x7f56a81f5220 2024-09-10T21:16:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.173+0000 7f56ae600640 1 -- 172.21.15.16:0/4204047669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a81608e0 con 0x7f56a81f5220 2024-09-10T21:16:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.177+0000 7f569f400640 1 -- 172.21.15.16:0/4204047669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f56900046d0 con 0x7f56a81f5220 2024-09-10T21:16:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.177+0000 7f569f400640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5684072610 0x7f5684074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.177+0000 7f569f400640 1 -- 172.21.15.16:0/4204047669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f569012dd10 con 0x7f56a81f5220 2024-09-10T21:16:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.177+0000 7f56a7400640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5684072610 0x7f5684074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.177+0000 7f56a7400640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5684072610 0x7f5684074ad0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5698004610 tx=0x7f569800b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.181+0000 7f569f400640 1 -- 172.21.15.16:0/4204047669 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5690133020 con 0x7f56a81f5220 2024-09-10T21:16:58.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f56ae600640 1 -- 172.21.15.16:0/4204047669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56a80cc290 con 0x7f56a81f5220 2024-09-10T21:16:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f569f400640 1 -- 172.21.15.16:0/4204047669 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f5690016f80 con 0x7f56a81f5220 2024-09-10T21:16:58.357 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-10T21:16:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f56ae600640 1 -- 172.21.15.16:0/4204047669 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5684072610 msgr2=0x7f5684074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f56ae600640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5684072610 0x7f5684074ad0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5698004610 tx=0x7f569800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f56ae600640 1 -- 172.21.15.16:0/4204047669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a81f5220 msgr2=0x7f56a81582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f56ae600640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a81f5220 0x7f56a81582e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f569001d130 tx=0x7f56900055b0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f56ae600640 1 -- 172.21.15.16:0/4204047669 shutdown_connections 2024-09-10T21:16:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f56ae600640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5684072610 0x7f5684074ad0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f56ae600640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a80cb8c0 0x7f56a80c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f56ae600640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a81f5220 0x7f56a81582e0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f56ae600640 1 --2- 172.21.15.16:0/4204047669 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f568c003fe0 0x7f56a806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f56ae600640 1 -- 172.21.15.16:0/4204047669 >> 172.21.15.16:0/4204047669 conn(0x7f56a80d1df0 msgr2=0x7f56a80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f56ae600640 1 -- 172.21.15.16:0/4204047669 shutdown_connections 2024-09-10T21:16:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.353+0000 7f56ae600640 1 -- 172.21.15.16:0/4204047669 wait complete. 2024-09-10T21:16:58.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2024-09-10T21:16:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 -- 172.21.15.16:0/2851747241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc1599d0 msgr2=0x7f95cc15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 --2- 172.21.15.16:0/2851747241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc1599d0 0x7f95cc15f830 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f95cc06fd80 tx=0x7f95bc01c830 comp rx=0 tx=0).stop 2024-09-10T21:16:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 -- 172.21.15.16:0/2851747241 shutdown_connections 2024-09-10T21:16:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 --2- 172.21.15.16:0/2851747241 >> v2:172.21.15.52:3300/0 conn(0x7f95cc06e640 0x7f95cc0ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 --2- 172.21.15.16:0/2851747241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc1599d0 0x7f95cc15f830 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 -- 172.21.15.16:0/2851747241 >> 172.21.15.16:0/2851747241 conn(0x7f95cc0cfb20 msgr2=0x7f95cc0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 -- 172.21.15.16:0/2851747241 shutdown_connections 2024-09-10T21:16:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 -- 172.21.15.16:0/2851747241 wait complete. 2024-09-10T21:16:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 Processor -- start 2024-09-10T21:16:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 -- start start 2024-09-10T21:16:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f95cc06e640 0x7f95cc0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0ca6b0 0x7f95cc147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95cc1599d0 0x7f95cc148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc168bd0 con 0x7f95cc0ca6b0 2024-09-10T21:16:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc073060 con 0x7f95cc1599d0 2024-09-10T21:16:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95d1600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f95cc168d50 con 0x7f95cc06e640 2024-09-10T21:16:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95c2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0ca6b0 0x7f95cc147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95c2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0ca6b0 0x7f95cc147b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60148/0 (socket says 172.21.15.16:60148) 2024-09-10T21:16:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.521+0000 7f95c2000640 1 -- 172.21.15.16:0/1165962353 learned_addr learned my addr 172.21.15.16:0/1165962353 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95caa00640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f95cc06e640 0x7f95cc0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95cb400640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95cc1599d0 0x7f95cc148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95c2000640 1 -- 172.21.15.16:0/1165962353 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f95cc06e640 msgr2=0x7f95cc0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95c2000640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f95cc06e640 0x7f95cc0c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95c2000640 1 -- 172.21.15.16:0/1165962353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95cc1599d0 msgr2=0x7f95cc148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95c2000640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95cc1599d0 0x7f95cc148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95c2000640 1 -- 172.21.15.16:0/1165962353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc0c6bf0 con 0x7f95cc0ca6b0 2024-09-10T21:16:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95cb400640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95cc1599d0 0x7f95cc148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95c2000640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0ca6b0 0x7f95cc147b10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f95b400d950 tx=0x7f95b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95c3400640 1 -- 172.21.15.16:0/1165962353 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4017070 con 0x7f95cc0ca6b0 2024-09-10T21:16:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95c3400640 1 -- 172.21.15.16:0/1165962353 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95b4009d20 con 0x7f95cc0ca6b0 2024-09-10T21:16:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95c3400640 1 -- 172.21.15.16:0/1165962353 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b40106c0 con 0x7f95cc0ca6b0 2024-09-10T21:16:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95d1600640 1 -- 172.21.15.16:0/1165962353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95cc0c43f0 con 0x7f95cc0ca6b0 2024-09-10T21:16:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95d1600640 1 -- 172.21.15.16:0/1165962353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95cc0c49b0 con 0x7f95cc0ca6b0 2024-09-10T21:16:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95d1600640 1 -- 172.21.15.16:0/1165962353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95cc0ca170 con 0x7f95cc0ca6b0 2024-09-10T21:16:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.525+0000 7f95c3400640 1 -- 172.21.15.16:0/1165962353 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f95b4010860 con 0x7f95cc0ca6b0 2024-09-10T21:16:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.529+0000 7f95c3400640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f95a80725e0 0x7f95a8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.529+0000 7f95c3400640 1 -- 172.21.15.16:0/1165962353 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f95b4093290 con 0x7f95cc0ca6b0 2024-09-10T21:16:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.529+0000 7f95caa00640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f95a80725e0 0x7f95a8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.529+0000 7f95caa00640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f95a80725e0 0x7f95a8074aa0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0af000 tx=0x7f95bc0a7040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:58.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.533+0000 7f95c3400640 1 -- 172.21.15.16:0/1165962353 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f95b405e610 con 0x7f95cc0ca6b0 2024-09-10T21:16:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:58.769+0000 7f95d1600640 1 -- 172.21.15.16:0/1165962353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) -- 0x7f95cc0b4960 con 0x7f95cc0ca6b0 2024-09-10T21:16:59.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.641+0000 7f95c3400640 1 -- 172.21.15.16:0/1165962353 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v324) ==== 146+0+0 (secure 0 0 0) 0x7f95b4063040 con 0x7f95cc0ca6b0 2024-09-10T21:16:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.645+0000 7f95d1600640 1 -- 172.21.15.16:0/1165962353 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f95a80725e0 msgr2=0x7f95a8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.645+0000 7f95d1600640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f95a80725e0 0x7f95a8074aa0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0af000 tx=0x7f95bc0a7040 comp rx=0 tx=0).stop 2024-09-10T21:16:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.645+0000 7f95d1600640 1 -- 172.21.15.16:0/1165962353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0ca6b0 msgr2=0x7f95cc147b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.645+0000 7f95d1600640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0ca6b0 0x7f95cc147b10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f95b400d950 tx=0x7f95b400de10 comp rx=0 tx=0).stop 2024-09-10T21:16:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.645+0000 7f95d1600640 1 -- 172.21.15.16:0/1165962353 shutdown_connections 2024-09-10T21:16:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.645+0000 7f95d1600640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f95a80725e0 0x7f95a8074aa0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.645+0000 7f95d1600640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95cc1599d0 0x7f95cc148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.645+0000 7f95d1600640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0ca6b0 0x7f95cc147b10 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.645+0000 7f95d1600640 1 --2- 172.21.15.16:0/1165962353 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f95cc06e640 0x7f95cc0c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.645+0000 7f95d1600640 1 -- 172.21.15.16:0/1165962353 >> 172.21.15.16:0/1165962353 conn(0x7f95cc0cfb20 msgr2=0x7f95cc15e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.645+0000 7f95d1600640 1 -- 172.21.15.16:0/1165962353 shutdown_connections 2024-09-10T21:16:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.645+0000 7f95d1600640 1 -- 172.21.15.16:0/1165962353 wait complete. 2024-09-10T21:16:59.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:16:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 -- 172.21.15.16:0/376221738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14300cfd40 msgr2=0x7f14300d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 --2- 172.21.15.16:0/376221738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14300cfd40 0x7f14300d0120 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f1420009840 tx=0x7f142001c790 comp rx=0 tx=0).stop 2024-09-10T21:16:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 -- 172.21.15.16:0/376221738 shutdown_connections 2024-09-10T21:16:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 --2- 172.21.15.16:0/376221738 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14300d2820 0x7f143015fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 --2- 172.21.15.16:0/376221738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14300cfd40 0x7f14300d0120 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 -- 172.21.15.16:0/376221738 >> 172.21.15.16:0/376221738 conn(0x7f14300d1d00 msgr2=0x7f14300d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 -- 172.21.15.16:0/376221738 shutdown_connections 2024-09-10T21:16:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 -- 172.21.15.16:0/376221738 wait complete. 2024-09-10T21:16:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 Processor -- start 2024-09-10T21:16:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 -- start start 2024-09-10T21:16:59.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14300cfd40 0x7f1430158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:59.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14300d2820 0x7f1430156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:59.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430160120 0x7f14300b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:59.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.821+0000 7f142f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1430169360 con 0x7f1430160120 2024-09-10T21:16:59.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14301691e0 con 0x7f14300cfd40 2024-09-10T21:16:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142f200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f14301694e0 con 0x7f14300d2820 2024-09-10T21:16:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430160120 0x7f14300b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430160120 0x7f14300b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60158/0 (socket says 172.21.15.16:60158) 2024-09-10T21:16:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142e800640 1 -- 172.21.15.16:0/4251568523 learned_addr learned my addr 172.21.15.16:0/4251568523 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:16:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142de00640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14300cfd40 0x7f1430158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f1427e00640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14300d2820 0x7f1430156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142e800640 1 -- 172.21.15.16:0/4251568523 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14300d2820 msgr2=0x7f1430156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142e800640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14300d2820 0x7f1430156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142e800640 1 -- 172.21.15.16:0/4251568523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14300cfd40 msgr2=0x7f1430158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142e800640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14300cfd40 0x7f1430158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142e800640 1 -- 172.21.15.16:0/4251568523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14301573f0 con 0x7f1430160120 2024-09-10T21:16:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f1427e00640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14300d2820 0x7f1430156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142de00640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14300cfd40 0x7f1430158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:59.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142e800640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430160120 0x7f14300b3780 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f142800cae0 tx=0x7f142800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:59.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f1426000640 1 -- 172.21.15.16:0/4251568523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428016020 con 0x7f1430160120 2024-09-10T21:16:59.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f1426000640 1 -- 172.21.15.16:0/4251568523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f142800ab40 con 0x7f1430160120 2024-09-10T21:16:59.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f1426000640 1 -- 172.21.15.16:0/4251568523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142800ce20 con 0x7f1430160120 2024-09-10T21:16:59.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142f200640 1 -- 172.21.15.16:0/4251568523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14300b6a10 con 0x7f1430160120 2024-09-10T21:16:59.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142f200640 1 -- 172.21.15.16:0/4251568523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1430157b00 con 0x7f1430160120 2024-09-10T21:16:59.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f142f200640 1 -- 172.21.15.16:0/4251568523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14301578f0 con 0x7f1430160120 2024-09-10T21:16:59.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.825+0000 7f1426000640 1 -- 172.21.15.16:0/4251568523 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f142801b400 con 0x7f1430160120 2024-09-10T21:16:59.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.829+0000 7f1426000640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f140c072610 0x7f140c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:59.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.829+0000 7f1426000640 1 -- 172.21.15.16:0/4251568523 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f1428093390 con 0x7f1430160120 2024-09-10T21:16:59.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.829+0000 7f142de00640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f140c072610 0x7f140c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:59.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.829+0000 7f142de00640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f140c072610 0x7f140c074ad0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f14200af000 tx=0x7f142001afd0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:59.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:16:59.833+0000 7f1426000640 1 -- 172.21.15.16:0/4251568523 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1428020020 con 0x7f1430160120 2024-09-10T21:17:00.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.005+0000 7f142f200640 1 -- 172.21.15.16:0/4251568523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f14300d0120 con 0x7f1430160120 2024-09-10T21:17:00.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.005+0000 7f1426000640 1 -- 172.21.15.16:0/4251568523 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v46) ==== 59+0+1998 (secure 0 0 0) 0x7f142805ee00 con 0x7f1430160120 2024-09-10T21:17:00.012 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:17:00.012 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:17:00.012 INFO:teuthology.orchestra.run.smithi016.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:17:00.012 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:17:00.012 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4226.427518 2024-09-10T21:17:00.012 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5426.620519 2024-09-10T21:17:00.012 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4255.210203 2024-09-10T21:17:00.012 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4255.734871 2024-09-10T21:17:00.012 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4192.833008 2024-09-10T21:17:00.012 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4136.940759 2024-09-10T21:17:00.012 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4217.384701 2024-09-10T21:17:00.013 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5220.337972 2024-09-10T21:17:00.013 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-10T21:17:00.013 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:17:00.013 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-10T21:17:00.013 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-10T21:17:00.013 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:17:00.013 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:17:00.013 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-10T21:17:00.013 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:17:00.013 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:17:00.013 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:17:00.014 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-10T21:17:00.014 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-10T21:17:00.014 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-10T21:17:00.014 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:17:00.014 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:17:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.009+0000 7f142f200640 1 -- 172.21.15.16:0/4251568523 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f140c072610 msgr2=0x7f140c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.009+0000 7f142f200640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f140c072610 0x7f140c074ad0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f14200af000 tx=0x7f142001afd0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.009+0000 7f142f200640 1 -- 172.21.15.16:0/4251568523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430160120 msgr2=0x7f14300b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.009+0000 7f142f200640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430160120 0x7f14300b3780 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f142800cae0 tx=0x7f142800cbd0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.009+0000 7f142f200640 1 -- 172.21.15.16:0/4251568523 shutdown_connections 2024-09-10T21:17:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.009+0000 7f142f200640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f140c072610 0x7f140c074ad0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.009+0000 7f142f200640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430160120 0x7f14300b3780 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.009+0000 7f142f200640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14300d2820 0x7f1430156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.009+0000 7f142f200640 1 --2- 172.21.15.16:0/4251568523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14300cfd40 0x7f1430158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.009+0000 7f142f200640 1 -- 172.21.15.16:0/4251568523 >> 172.21.15.16:0/4251568523 conn(0x7f14300d1d00 msgr2=0x7f14300cd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.009+0000 7f142f200640 1 -- 172.21.15.16:0/4251568523 shutdown_connections 2024-09-10T21:17:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.009+0000 7f142f200640 1 -- 172.21.15.16:0/4251568523 wait complete. 2024-09-10T21:17:00.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:17:00.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.177+0000 7ff96e600640 1 -- 172.21.15.16:0/2687207519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c003a40 msgr2=0x7ff94c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.177+0000 7ff96e600640 1 --2- 172.21.15.16:0/2687207519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c003a40 0x7ff94c003e20 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff958009b60 tx=0x7ff95801d270 comp rx=0 tx=0).stop 2024-09-10T21:17:00.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.177+0000 7ff967e00640 1 -- 172.21.15.16:0/2687207519 reap_dead start 2024-09-10T21:17:00.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff96e600640 1 -- 172.21.15.16:0/2687207519 shutdown_connections 2024-09-10T21:17:00.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff96e600640 1 -- 172.21.15.16:0/2687207519 >> 172.21.15.16:0/2687207519 conn(0x7ff9680d1ce0 msgr2=0x7ff9680d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:00.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff96e600640 1 -- 172.21.15.16:0/2687207519 shutdown_connections 2024-09-10T21:17:00.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff96e600640 1 -- 172.21.15.16:0/2687207519 wait complete. 2024-09-10T21:17:00.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff96e600640 1 Processor -- start 2024-09-10T21:17:00.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff96e600640 1 -- start start 2024-09-10T21:17:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff96e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c003a40 0x7ff96806f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff96e600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff96806f010 0x7ff9680cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff96e600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff9680c6fa0 0x7ff9680c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff96e600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9681602d0 con 0x7ff94c003a40 2024-09-10T21:17:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff96e600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff968160150 con 0x7ff96806f010 2024-09-10T21:17:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff96e600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff968168490 con 0x7ff9680c6fa0 2024-09-10T21:17:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff967400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c003a40 0x7ff96806f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff967400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c003a40 0x7ff96806f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60184/0 (socket says 172.21.15.16:60184) 2024-09-10T21:17:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff967400640 1 -- 172.21.15.16:0/1401313655 learned_addr learned my addr 172.21.15.16:0/1401313655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff95ea00640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff96806f010 0x7ff9680cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:00.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff967e00640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff9680c6fa0 0x7ff9680c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:00.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff967400640 1 -- 172.21.15.16:0/1401313655 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff9680c6fa0 msgr2=0x7ff9680c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff967400640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff9680c6fa0 0x7ff9680c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff967400640 1 -- 172.21.15.16:0/1401313655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff96806f010 msgr2=0x7ff9680cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff967400640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff96806f010 0x7ff9680cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff967400640 1 -- 172.21.15.16:0/1401313655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96806f560 con 0x7ff94c003a40 2024-09-10T21:17:00.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff967e00640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff9680c6fa0 0x7ff9680c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:00.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff967400640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c003a40 0x7ff96806f9a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff9580af040 tx=0x7ff958005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:00.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.181+0000 7ff95fe00640 1 -- 172.21.15.16:0/1401313655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9580b1040 con 0x7ff94c003a40 2024-09-10T21:17:00.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.185+0000 7ff95fe00640 1 -- 172.21.15.16:0/1401313655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9580bb070 con 0x7ff94c003a40 2024-09-10T21:17:00.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.185+0000 7ff95fe00640 1 -- 172.21.15.16:0/1401313655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958009730 con 0x7ff94c003a40 2024-09-10T21:17:00.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.185+0000 7ff96e600640 1 -- 172.21.15.16:0/1401313655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9681fb580 con 0x7ff94c003a40 2024-09-10T21:17:00.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.185+0000 7ff96e600640 1 -- 172.21.15.16:0/1401313655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9681fbb40 con 0x7ff94c003a40 2024-09-10T21:17:00.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.185+0000 7ff96e600640 1 -- 172.21.15.16:0/1401313655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9680b4b10 con 0x7ff94c003a40 2024-09-10T21:17:00.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.185+0000 7ff95fe00640 1 -- 172.21.15.16:0/1401313655 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff9580046d0 con 0x7ff94c003a40 2024-09-10T21:17:00.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.185+0000 7ff95fe00640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff944072610 0x7ff944074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:00.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.185+0000 7ff95fe00640 1 -- 172.21.15.16:0/1401313655 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7ff95812dba0 con 0x7ff94c003a40 2024-09-10T21:17:00.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.185+0000 7ff95ea00640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff944072610 0x7ff944074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:00.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.189+0000 7ff95ea00640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff944072610 0x7ff944074ad0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff9680cbe60 tx=0x7ff950001e80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:00.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.189+0000 7ff95fe00640 1 -- 172.21.15.16:0/1401313655 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff958016f80 con 0x7ff94c003a40 2024-09-10T21:17:00.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.365+0000 7ff96e600640 1 -- 172.21.15.16:0/1401313655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff9680704a0 con 0x7ff94c003a40 2024-09-10T21:17:00.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.369+0000 7ff95fe00640 1 -- 172.21.15.16:0/1401313655 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v47)=0 v47) ==== 122+0+0 (secure 0 0 0) 0x7ff9580f8e90 con 0x7ff94c003a40 2024-09-10T21:17:00.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.369+0000 7ff96e600640 1 -- 172.21.15.16:0/1401313655 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff944072610 msgr2=0x7ff944074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.369+0000 7ff96e600640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff944072610 0x7ff944074ad0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff9680cbe60 tx=0x7ff950001e80 comp rx=0 tx=0).stop 2024-09-10T21:17:00.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.369+0000 7ff96e600640 1 -- 172.21.15.16:0/1401313655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c003a40 msgr2=0x7ff96806f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.369+0000 7ff96e600640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c003a40 0x7ff96806f9a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff9580af040 tx=0x7ff958005590 comp rx=0 tx=0).stop 2024-09-10T21:17:00.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.369+0000 7ff96e600640 1 -- 172.21.15.16:0/1401313655 shutdown_connections 2024-09-10T21:17:00.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.369+0000 7ff96e600640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff944072610 0x7ff944074ad0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.369+0000 7ff96e600640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff9680c6fa0 0x7ff9680c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.369+0000 7ff96e600640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff96806f010 0x7ff9680cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.369+0000 7ff96e600640 1 --2- 172.21.15.16:0/1401313655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c003a40 0x7ff96806f9a0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.369+0000 7ff96e600640 1 -- 172.21.15.16:0/1401313655 >> 172.21.15.16:0/1401313655 conn(0x7ff9680d1ce0 msgr2=0x7ff9680b5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:00.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.369+0000 7ff96e600640 1 -- 172.21.15.16:0/1401313655 shutdown_connections 2024-09-10T21:17:00.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.369+0000 7ff96e600640 1 -- 172.21.15.16:0/1401313655 wait complete. 2024-09-10T21:17:00.395 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:17:00.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:17:00.466 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:17:00.546 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:17:00.547 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:17:00.547 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:00.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:17:00.556 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:17:00.556 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:00.565 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:17:00.566 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:17:00.566 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:00.611 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:17:00.612 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:17:00.612 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:00.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:17:00.621 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:17:00.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.797+0000 7f4d63200640 1 -- 172.21.15.16:0/171172895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d40004810 msgr2=0x7f4d40004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.797+0000 7f4d63200640 1 --2- 172.21.15.16:0/171172895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d40004810 0x7f4d40004c90 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009b60 tx=0x7f4d4401d250 comp rx=0 tx=0).stop 2024-09-10T21:17:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.797+0000 7f4d61200640 1 -- 172.21.15.16:0/171172895 reap_dead start 2024-09-10T21:17:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.797+0000 7f4d63200640 1 -- 172.21.15.16:0/171172895 shutdown_connections 2024-09-10T21:17:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.797+0000 7f4d63200640 1 -- 172.21.15.16:0/171172895 >> 172.21.15.16:0/171172895 conn(0x7f4d5c0cfb20 msgr2=0x7f4d5c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.797+0000 7f4d63200640 1 -- 172.21.15.16:0/171172895 shutdown_connections 2024-09-10T21:17:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.797+0000 7f4d63200640 1 -- 172.21.15.16:0/171172895 wait complete. 2024-09-10T21:17:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.797+0000 7f4d63200640 1 Processor -- start 2024-09-10T21:17:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d63200640 1 -- start start 2024-09-10T21:17:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d63200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d40004810 0x7f4d5c152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d63200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c1530a0 0x7f4d5c1e4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d63200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d5c1f8260 0x7f4d5c1e91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d63200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c070430 con 0x7f4d5c1530a0 2024-09-10T21:17:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d63200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c0702b0 con 0x7f4d40004810 2024-09-10T21:17:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d63200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4d5c06e640 con 0x7f4d5c1f8260 2024-09-10T21:17:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d53e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c1530a0 0x7f4d5c1e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d53e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c1530a0 0x7f4d5c1e4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60204/0 (socket says 172.21.15.16:60204) 2024-09-10T21:17:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d53e00640 1 -- 172.21.15.16:0/452508373 learned_addr learned my addr 172.21.15.16:0/452508373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d5be00640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d40004810 0x7f4d5c152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d61200640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d5c1f8260 0x7f4d5c1e91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d53e00640 1 -- 172.21.15.16:0/452508373 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d5c1f8260 msgr2=0x7f4d5c1e91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d53e00640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d5c1f8260 0x7f4d5c1e91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d53e00640 1 -- 172.21.15.16:0/452508373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d40004810 msgr2=0x7f4d5c152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d53e00640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d40004810 0x7f4d5c152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d53e00640 1 -- 172.21.15.16:0/452508373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5c152ea0 con 0x7f4d5c1530a0 2024-09-10T21:17:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d5be00640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d40004810 0x7f4d5c152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d61200640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d5c1f8260 0x7f4d5c1e91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d53e00640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c1530a0 0x7f4d5c1e4650 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4d4401d080 tx=0x7f4d44005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d58c00640 1 -- 172.21.15.16:0/452508373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d440ba020 con 0x7f4d5c1530a0 2024-09-10T21:17:00.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d58c00640 1 -- 172.21.15.16:0/452508373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d440ac8a0 con 0x7f4d5c1530a0 2024-09-10T21:17:00.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d58c00640 1 -- 172.21.15.16:0/452508373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d440b1040 con 0x7f4d5c1530a0 2024-09-10T21:17:00.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d63200640 1 -- 172.21.15.16:0/452508373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5c1e4b90 con 0x7f4d5c1530a0 2024-09-10T21:17:00.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d63200640 1 -- 172.21.15.16:0/452508373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d5c1e5110 con 0x7f4d5c1530a0 2024-09-10T21:17:00.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.801+0000 7f4d63200640 1 -- 172.21.15.16:0/452508373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d5c06eba0 con 0x7f4d5c1530a0 2024-09-10T21:17:00.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.805+0000 7f4d58c00640 1 -- 172.21.15.16:0/452508373 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4d440046d0 con 0x7f4d5c1530a0 2024-09-10T21:17:00.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.805+0000 7f4d58c00640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4d38072610 0x7f4d38074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:00.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.805+0000 7f4d58c00640 1 -- 172.21.15.16:0/452508373 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f4d4412e0e0 con 0x7f4d5c1530a0 2024-09-10T21:17:00.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.805+0000 7f4d5be00640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4d38072610 0x7f4d38074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:00.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.805+0000 7f4d5be00640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4d38072610 0x7f4d38074ad0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c001020 tx=0x7f4d4c006730 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:00.809+0000 7f4d58c00640 1 -- 172.21.15.16:0/452508373 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4d440c4030 con 0x7f4d5c1530a0 2024-09-10T21:17:01.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.017+0000 7f4d63200640 1 -- 172.21.15.16:0/452508373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d5c0b5aa0 con 0x7f4d5c1530a0 2024-09-10T21:17:01.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.017+0000 7f4d58c00640 1 -- 172.21.15.16:0/452508373 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+3908 (secure 0 0 0) 0x7f4d5c152ea0 con 0x7f4d5c1530a0 2024-09-10T21:17:01.023 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:01.023 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":64,"btime":"2024-09-10T21:16:54:620758+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/895621033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":895621033},{"type":"v1","addr":"172.21.15.52:6835","nonce":895621033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2675529240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2675529240},{"type":"v1","addr":"172.21.15.52:6834","nonce":2675529240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6789,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2281471863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2281471863},{"type":"v1","addr":"172.21.15.16:6835","nonce":2281471863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1879915011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1879915011},{"type":"v1","addr":"172.21.15.16:6837","nonce":1879915011}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[]} 2024-09-10T21:17:01.023 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 64 2024-09-10T21:17:01.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.021+0000 7f4d63200640 1 -- 172.21.15.16:0/452508373 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4d38072610 msgr2=0x7f4d38074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:01.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.021+0000 7f4d63200640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4d38072610 0x7f4d38074ad0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c001020 tx=0x7f4d4c006730 comp rx=0 tx=0).stop 2024-09-10T21:17:01.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.021+0000 7f4d63200640 1 -- 172.21.15.16:0/452508373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c1530a0 msgr2=0x7f4d5c1e4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:01.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.021+0000 7f4d63200640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c1530a0 0x7f4d5c1e4650 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4d4401d080 tx=0x7f4d44005620 comp rx=0 tx=0).stop 2024-09-10T21:17:01.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.021+0000 7f4d63200640 1 -- 172.21.15.16:0/452508373 shutdown_connections 2024-09-10T21:17:01.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.021+0000 7f4d63200640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4d38072610 0x7f4d38074ad0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.021+0000 7f4d63200640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4d5c1f8260 0x7f4d5c1e91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.021+0000 7f4d63200640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c1530a0 0x7f4d5c1e4650 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.021+0000 7f4d63200640 1 --2- 172.21.15.16:0/452508373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d40004810 0x7f4d5c152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.021+0000 7f4d63200640 1 -- 172.21.15.16:0/452508373 >> 172.21.15.16:0/452508373 conn(0x7f4d5c0cfb20 msgr2=0x7f4d5c0b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:01.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.021+0000 7f4d63200640 1 -- 172.21.15.16:0/452508373 shutdown_connections 2024-09-10T21:17:01.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.021+0000 7f4d63200640 1 -- 172.21.15.16:0/452508373 wait complete. 2024-09-10T21:17:01.045 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:17:01.045 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:17:01.045 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:17:01.045 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:17:01.045 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:17:01.046 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:17:01.046 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:17:01.046 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:17:01.046 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:17:01.046 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:17:01.046 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:17:01.046 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:17:01.047 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:17:01.047 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:17:01.047 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:17:01.047 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:17:01.059 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:17:01.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:17:01.060 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:17:01.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:17:01.062 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:17:01.062 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:17:01.063 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:17:01.064 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:17:01.105 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:17:01.106 DEBUG:teuthology.parallel:result is None 2024-09-10T21:17:01.107 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:17:01.107 DEBUG:teuthology.parallel:result is None 2024-09-10T21:17:01.108 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:17:01.108 DEBUG:teuthology.parallel:result is None 2024-09-10T21:17:01.109 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:17:01.109 DEBUG:teuthology.parallel:result is None 2024-09-10T21:17:01.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:17:01.192 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-10T21:17:01.194 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-10T21:17:01.201 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:17:01.201 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:17:01.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.257+0000 7faf98200640 1 -- 172.21.15.16:0/4222648345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf900cfb60 msgr2=0x7faf900cfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:01.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.257+0000 7faf98200640 1 --2- 172.21.15.16:0/4222648345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf900cfb60 0x7faf900cfff0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7faf9006dae0 tx=0x7faf7c01c800 comp rx=0 tx=0).stop 2024-09-10T21:17:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.257+0000 7faf98200640 1 -- 172.21.15.16:0/4222648345 shutdown_connections 2024-09-10T21:17:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.257+0000 7faf98200640 1 --2- 172.21.15.16:0/4222648345 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf900d0530 0x7faf900ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.257+0000 7faf98200640 1 --2- 172.21.15.16:0/4222648345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf900cfb60 0x7faf900cfff0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.257+0000 7faf98200640 1 --2- 172.21.15.16:0/4222648345 >> v2:172.21.15.52:3300/0 conn(0x7faf9015f350 0x7faf900cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.257+0000 7faf98200640 1 -- 172.21.15.16:0/4222648345 >> 172.21.15.16:0/4222648345 conn(0x7faf900d1d80 msgr2=0x7faf900d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.257+0000 7faf98200640 1 -- 172.21.15.16:0/4222648345 shutdown_connections 2024-09-10T21:17:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.257+0000 7faf98200640 1 -- 172.21.15.16:0/4222648345 wait complete. 2024-09-10T21:17:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf98200640 1 Processor -- start 2024-09-10T21:17:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf98200640 1 -- start start 2024-09-10T21:17:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf98200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf900cfb60 0x7faf9014ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf98200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf900d0530 0x7faf9014e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf98200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9015f350 0x7faf9015c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf98200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf900af210 con 0x7faf9015f350 2024-09-10T21:17:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf98200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faf900af090 con 0x7faf900cfb60 2024-09-10T21:17:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf98200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7faf900af390 con 0x7faf900d0530 2024-09-10T21:17:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf96200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9015f350 0x7faf9015c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf95800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf900cfb60 0x7faf9014ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf95800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf900cfb60 0x7faf9014ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:37976/0 (socket says 172.21.15.16:37976) 2024-09-10T21:17:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf95800640 1 -- 172.21.15.16:0/467351586 learned_addr learned my addr 172.21.15.16:0/467351586 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf94e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf900d0530 0x7faf9014e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf95800640 1 -- 172.21.15.16:0/467351586 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf900d0530 msgr2=0x7faf9014e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf95800640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf900d0530 0x7faf9014e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf95800640 1 -- 172.21.15.16:0/467351586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9015f350 msgr2=0x7faf9015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf95800640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9015f350 0x7faf9015c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf95800640 1 -- 172.21.15.16:0/467351586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9014e8a0 con 0x7faf900cfb60 2024-09-10T21:17:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf94e00640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf900d0530 0x7faf9014e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf96200640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9015f350 0x7faf9015c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf95800640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf900cfb60 0x7faf9014ef20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faf8400d750 tx=0x7faf8400dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf8d600640 1 -- 172.21.15.16:0/467351586 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84017070 con 0x7faf900cfb60 2024-09-10T21:17:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf8d600640 1 -- 172.21.15.16:0/467351586 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf84009d20 con 0x7faf900cfb60 2024-09-10T21:17:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf8d600640 1 -- 172.21.15.16:0/467351586 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8400de60 con 0x7faf900cfb60 2024-09-10T21:17:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf98200640 1 -- 172.21.15.16:0/467351586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9014ec40 con 0x7faf900cfb60 2024-09-10T21:17:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf98200640 1 -- 172.21.15.16:0/467351586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf900c7c70 con 0x7faf900cfb60 2024-09-10T21:17:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.261+0000 7faf98200640 1 -- 172.21.15.16:0/467351586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf900c7780 con 0x7faf900cfb60 2024-09-10T21:17:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.265+0000 7faf8d600640 1 -- 172.21.15.16:0/467351586 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faf84004280 con 0x7faf900cfb60 2024-09-10T21:17:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.265+0000 7faf8d600640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faf70072610 0x7faf70074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.265+0000 7faf8d600640 1 -- 172.21.15.16:0/467351586 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7faf840937e0 con 0x7faf900cfb60 2024-09-10T21:17:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.265+0000 7faf94e00640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faf70072610 0x7faf70074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.265+0000 7faf94e00640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faf70072610 0x7faf70074ad0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0af000 tx=0x7faf7c01b040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.269+0000 7faf8d600640 1 -- 172.21.15.16:0/467351586 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faf84020020 con 0x7faf900cfb60 2024-09-10T21:17:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.453+0000 7faf98200640 1 -- 172.21.15.16:0/467351586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7faf900b0680 con 0x7faf900cfb60 2024-09-10T21:17:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.657+0000 7faf8d600640 1 -- 172.21.15.16:0/467351586 <== mon.1 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) ==== 97+0+0 (secure 0 0 0) 0x7faf9014e8a0 con 0x7faf900cfb60 2024-09-10T21:17:01.662 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-10T21:17:01.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.661+0000 7faf98200640 1 -- 172.21.15.16:0/467351586 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faf70072610 msgr2=0x7faf70074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:01.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.661+0000 7faf98200640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faf70072610 0x7faf70074ad0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0af000 tx=0x7faf7c01b040 comp rx=0 tx=0).stop 2024-09-10T21:17:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.661+0000 7faf98200640 1 -- 172.21.15.16:0/467351586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf900cfb60 msgr2=0x7faf9014ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.661+0000 7faf98200640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf900cfb60 0x7faf9014ef20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faf8400d750 tx=0x7faf8400dc10 comp rx=0 tx=0).stop 2024-09-10T21:17:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.661+0000 7faf98200640 1 -- 172.21.15.16:0/467351586 shutdown_connections 2024-09-10T21:17:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.661+0000 7faf98200640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faf70072610 0x7faf70074ad0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.661+0000 7faf98200640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9015f350 0x7faf9015c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.661+0000 7faf98200640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faf900d0530 0x7faf9014e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.661+0000 7faf98200640 1 --2- 172.21.15.16:0/467351586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf900cfb60 0x7faf9014ef20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.661+0000 7faf98200640 1 -- 172.21.15.16:0/467351586 >> 172.21.15.16:0/467351586 conn(0x7faf900d1d80 msgr2=0x7faf90158d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.661+0000 7faf98200640 1 -- 172.21.15.16:0/467351586 shutdown_connections 2024-09-10T21:17:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.661+0000 7faf98200640 1 -- 172.21.15.16:0/467351586 wait complete. 2024-09-10T21:17:01.688 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:17:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.829+0000 7f8003e00640 1 -- 172.21.15.16:0/4265172404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80040d2820 msgr2=0x7f800415fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.829+0000 7f8003e00640 1 --2- 172.21.15.16:0/4265172404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80040d2820 0x7f800415fbe0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009900 tx=0x7f7fec01c850 comp rx=0 tx=0).stop 2024-09-10T21:17:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.829+0000 7f8003e00640 1 -- 172.21.15.16:0/4265172404 shutdown_connections 2024-09-10T21:17:01.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.829+0000 7f8003e00640 1 --2- 172.21.15.16:0/4265172404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80040d2820 0x7f800415fbe0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.829+0000 7f8003e00640 1 --2- 172.21.15.16:0/4265172404 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80040cfd40 0x7f80040d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.829+0000 7f8003e00640 1 -- 172.21.15.16:0/4265172404 >> 172.21.15.16:0/4265172404 conn(0x7f80040d1d00 msgr2=0x7f80040d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:01.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.829+0000 7f8003e00640 1 -- 172.21.15.16:0/4265172404 shutdown_connections 2024-09-10T21:17:01.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.829+0000 7f8003e00640 1 -- 172.21.15.16:0/4265172404 wait complete. 2024-09-10T21:17:01.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.829+0000 7f8003e00640 1 Processor -- start 2024-09-10T21:17:01.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003e00640 1 -- start start 2024-09-10T21:17:01.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80040cfd40 0x7f80040b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:01.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80040d2820 0x7f8004157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:01.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004160120 0x7f80040b6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:01.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8004169360 con 0x7f8004160120 2024-09-10T21:17:01.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f80041691e0 con 0x7f80040d2820 2024-09-10T21:17:01.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f80041694e0 con 0x7f80040cfd40 2024-09-10T21:17:01.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004160120 0x7f80040b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:01.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f7ffbe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80040d2820 0x7f8004157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:01.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8002a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80040cfd40 0x7f80040b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:01.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004160120 0x7f80040b6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60272/0 (socket says 172.21.15.16:60272) 2024-09-10T21:17:01.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003400640 1 -- 172.21.15.16:0/494191579 learned_addr learned my addr 172.21.15.16:0/494191579 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003400640 1 -- 172.21.15.16:0/494191579 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80040cfd40 msgr2=0x7f80040b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003400640 1 --2- 172.21.15.16:0/494191579 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80040cfd40 0x7f80040b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003400640 1 -- 172.21.15.16:0/494191579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80040d2820 msgr2=0x7f8004157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003400640 1 --2- 172.21.15.16:0/494191579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80040d2820 0x7f8004157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003400640 1 -- 172.21.15.16:0/494191579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8004157c10 con 0x7f8004160120 2024-09-10T21:17:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003400640 1 --2- 172.21.15.16:0/494191579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004160120 0x7f80040b6950 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c6e0 tx=0x7f7ffc00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f7ffaa00640 1 -- 172.21.15.16:0/494191579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc003b80 con 0x7f8004160120 2024-09-10T21:17:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f7ffaa00640 1 -- 172.21.15.16:0/494191579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ffc003d40 con 0x7f8004160120 2024-09-10T21:17:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f7ffaa00640 1 -- 172.21.15.16:0/494191579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc005330 con 0x7f8004160120 2024-09-10T21:17:01.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003e00640 1 -- 172.21.15.16:0/494191579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8004157ef0 con 0x7f8004160120 2024-09-10T21:17:01.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003e00640 1 -- 172.21.15.16:0/494191579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80041584b0 con 0x7f8004160120 2024-09-10T21:17:01.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.833+0000 7f8003e00640 1 -- 172.21.15.16:0/494191579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80040d0120 con 0x7f8004160120 2024-09-10T21:17:01.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.837+0000 7f7ffaa00640 1 -- 172.21.15.16:0/494191579 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7ffc0054d0 con 0x7f8004160120 2024-09-10T21:17:01.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.837+0000 7f7ffaa00640 1 --2- 172.21.15.16:0/494191579 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fe00725e0 0x7f7fe0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:01.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.837+0000 7f7ffaa00640 1 -- 172.21.15.16:0/494191579 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f7ffc092c80 con 0x7f8004160120 2024-09-10T21:17:01.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.837+0000 7f8002a00640 1 --2- 172.21.15.16:0/494191579 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fe00725e0 0x7f7fe0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:01.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.837+0000 7f8002a00640 1 --2- 172.21.15.16:0/494191579 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fe00725e0 0x7f7fe0074aa0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4004630 tx=0x7f7ff400b390 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:01.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:01.841+0000 7f7ffaa00640 1 -- 172.21.15.16:0/494191579 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7ffc05df30 con 0x7f8004160120 2024-09-10T21:17:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.073+0000 7f8003e00640 1 -- 172.21.15.16:0/494191579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f80040b3b90 con 0x7f8004160120 2024-09-10T21:17:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.077+0000 7f7ffaa00640 1 -- 172.21.15.16:0/494191579 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v27) ==== 80+0+5304 (secure 0 0 0) 0x7f7ffc062960 con 0x7f8004160120 2024-09-10T21:17:02.080 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:02.080 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:17:02.080 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-10T21:17:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-10T21:17:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmhg+lJBAAWjuYW75BGo2dmyAUi7s9+A==", 2024-09-10T21:17:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:17:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:17:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-10T21:17:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmQr7+MBAAOzXhWIirBOoi/7wKnrFgmg==", 2024-09-10T21:17:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:17:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:17:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-10T21:17:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmxxoALBAAgkmw91/VP1Uv2WMZsJKAyA==", 2024-09-10T21:17:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:17:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:17:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-10T21:17:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmA4tzOBAAiS+8n+Gokz6yXapVf2ecYg==", 2024-09-10T21:17:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:17:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:17:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-10T21:17:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHteBmRoahLRAA32PsUkqtGkeI7tMFRwas5Q==", 2024-09-10T21:17:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-10T21:17:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCLteBmGmEMBhAAr1yQEf/NkiN4+ezAQ9VR2A==", 2024-09-10T21:17:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-10T21:17:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCOteBmmMuAHBAAt+G/KApsZymjTLRSmyRbdw==", 2024-09-10T21:17:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-10T21:17:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCRteBmoQfNMBAA0vnIqlYoI13KNAAfNmJg3w==", 2024-09-10T21:17:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-10T21:17:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCWteBmc/7lDxAADlSi4UpcFJVNdp6YTkpdSw==", 2024-09-10T21:17:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-10T21:17:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZteBmBZ1LJhAAPcbZa1LQu4kNbSsJ/8Z81A==", 2024-09-10T21:17:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-10T21:17:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCcteBm7/czOxAAqREnY84+LF1N9dRXOhzz6g==", 2024-09-10T21:17:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-10T21:17:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgteBmG2J8FRAAB+cMFCnf9wJg1R3RD5SUJw==", 2024-09-10T21:17:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-10T21:17:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA==", 2024-09-10T21:17:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-10T21:17:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:17:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-10T21:17:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmuEm4ChAAe08QvhwIcN8ZHa3AoVcYww==", 2024-09-10T21:17:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:17:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:17:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:17:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-10T21:17:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBm50o1BxAAIFbVxVqU4rU8eKYpuZjTHA==", 2024-09-10T21:17:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:17:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:17:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:17:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-10T21:17:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmxtu0EBAAmN7+07lDiwjsIC/pNYUlSw==", 2024-09-10T21:17:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:17:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:17:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:17:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-10T21:17:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmfqgYCBAAm00tBe45phGieCcp1q46cw==", 2024-09-10T21:17:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:17:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:17:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:17:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-10T21:17:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmMVABIBAAoB72mPyXyUO6Y6k+fMqQQg==", 2024-09-10T21:17:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:17:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:17:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-10T21:17:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmeyYxHBAAvBkpTzjPQKAOlHzI1jQ46g==", 2024-09-10T21:17:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:17:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:17:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-10T21:17:02.097 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:17:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.077+0000 7f8003e00640 1 -- 172.21.15.16:0/494191579 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fe00725e0 msgr2=0x7f7fe0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.077+0000 7f8003e00640 1 --2- 172.21.15.16:0/494191579 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fe00725e0 0x7f7fe0074aa0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4004630 tx=0x7f7ff400b390 comp rx=0 tx=0).stop 2024-09-10T21:17:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.077+0000 7f8003e00640 1 -- 172.21.15.16:0/494191579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004160120 msgr2=0x7f80040b6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.077+0000 7f8003e00640 1 --2- 172.21.15.16:0/494191579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004160120 0x7f80040b6950 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c6e0 tx=0x7f7ffc00cba0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.077+0000 7f8003e00640 1 -- 172.21.15.16:0/494191579 shutdown_connections 2024-09-10T21:17:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.077+0000 7f8003e00640 1 --2- 172.21.15.16:0/494191579 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fe00725e0 0x7f7fe0074aa0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.077+0000 7f8003e00640 1 --2- 172.21.15.16:0/494191579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8004160120 0x7f80040b6950 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.077+0000 7f8003e00640 1 --2- 172.21.15.16:0/494191579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80040d2820 0x7f8004157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.077+0000 7f8003e00640 1 --2- 172.21.15.16:0/494191579 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f80040cfd40 0x7f80040b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.077+0000 7f8003e00640 1 -- 172.21.15.16:0/494191579 >> 172.21.15.16:0/494191579 conn(0x7f80040d1d00 msgr2=0x7f80040cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.077+0000 7f8003e00640 1 -- 172.21.15.16:0/494191579 shutdown_connections 2024-09-10T21:17:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.077+0000 7f8003e00640 1 -- 172.21.15.16:0/494191579 wait complete. 2024-09-10T21:17:02.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:17:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 -- 172.21.15.16:0/237943926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f279806e640 msgr2=0x7f27980ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 --2- 172.21.15.16:0/237943926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f279806e640 0x7f27980ca170 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f279806daf0 tx=0x7f278401c830 comp rx=0 tx=0).stop 2024-09-10T21:17:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 -- 172.21.15.16:0/237943926 shutdown_connections 2024-09-10T21:17:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 --2- 172.21.15.16:0/237943926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f279806e640 0x7f27980ca170 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 --2- 172.21.15.16:0/237943926 >> v2:172.21.15.52:3300/0 conn(0x7f27981599d0 0x7f279815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 -- 172.21.15.16:0/237943926 >> 172.21.15.16:0/237943926 conn(0x7f27980cfb20 msgr2=0x7f27980cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 -- 172.21.15.16:0/237943926 shutdown_connections 2024-09-10T21:17:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 -- 172.21.15.16:0/237943926 wait complete. 2024-09-10T21:17:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 Processor -- start 2024-09-10T21:17:02.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 -- start start 2024-09-10T21:17:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f279806e640 0x7f27980c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f27980ca6b0 0x7f27980c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27981599d0 0x7f279815b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2798168bd0 con 0x7f279806e640 2024-09-10T21:17:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2798073000 con 0x7f27981599d0 2024-09-10T21:17:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f27a0000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2798168d50 con 0x7f27980ca6b0 2024-09-10T21:17:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f279d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f279806e640 0x7f27980c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f279d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f279806e640 0x7f27980c2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54222/0 (socket says 172.21.15.16:54222) 2024-09-10T21:17:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f279d600640 1 -- 172.21.15.16:0/1970149187 learned_addr learned my addr 172.21.15.16:0/1970149187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f279d600640 1 -- 172.21.15.16:0/1970149187 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f27980ca6b0 msgr2=0x7f27980c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f279cc00640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f27980ca6b0 0x7f27980c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f279e000640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27981599d0 0x7f279815b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f279d600640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f27980ca6b0 0x7f27980c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f279d600640 1 -- 172.21.15.16:0/1970149187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27981599d0 msgr2=0x7f279815b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f279d600640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27981599d0 0x7f279815b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f279d600640 1 -- 172.21.15.16:0/1970149187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27980c3040 con 0x7f279806e640 2024-09-10T21:17:02.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f279e000640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27981599d0 0x7f279815b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:02.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.245+0000 7f279d600640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f279806e640 0x7f27980c2b00 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f278c00d750 tx=0x7f278c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:02.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.249+0000 7f2795600640 1 -- 172.21.15.16:0/1970149187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c017070 con 0x7f279806e640 2024-09-10T21:17:02.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.249+0000 7f2795600640 1 -- 172.21.15.16:0/1970149187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f278c009d20 con 0x7f279806e640 2024-09-10T21:17:02.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.249+0000 7f2795600640 1 -- 172.21.15.16:0/1970149187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c00de60 con 0x7f279806e640 2024-09-10T21:17:02.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.249+0000 7f27a0000640 1 -- 172.21.15.16:0/1970149187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279815a170 con 0x7f279806e640 2024-09-10T21:17:02.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.249+0000 7f27a0000640 1 -- 172.21.15.16:0/1970149187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279815a6f0 con 0x7f279806e640 2024-09-10T21:17:02.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.249+0000 7f27a0000640 1 -- 172.21.15.16:0/1970149187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27980ca170 con 0x7f279806e640 2024-09-10T21:17:02.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.249+0000 7f2795600640 1 -- 172.21.15.16:0/1970149187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f278c004280 con 0x7f279806e640 2024-09-10T21:17:02.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.249+0000 7f2795600640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2778072610 0x7f2778074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.249+0000 7f2795600640 1 -- 172.21.15.16:0/1970149187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f278c093400 con 0x7f279806e640 2024-09-10T21:17:02.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.249+0000 7f279cc00640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2778072610 0x7f2778074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.253+0000 7f279cc00640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2778072610 0x7f2778074ad0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f27840af000 tx=0x7f278401b070 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:02.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.253+0000 7f2795600640 1 -- 172.21.15.16:0/1970149187 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f278c020020 con 0x7f279806e640 2024-09-10T21:17:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f27a0000640 1 -- 172.21.15.16:0/1970149187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f279815e480 con 0x7f279806e640 2024-09-10T21:17:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f2795600640 1 -- 172.21.15.16:0/1970149187 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) ==== 84+0+65 (secure 0 0 0) 0x7f27980c3040 con 0x7f279806e640 2024-09-10T21:17:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:02.433 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-10T21:17:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:02.433 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-10T21:17:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-10T21:17:02.434 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:02.434 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-10T21:17:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f27a0000640 1 -- 172.21.15.16:0/1970149187 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2778072610 msgr2=0x7f2778074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f27a0000640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2778072610 0x7f2778074ad0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f27840af000 tx=0x7f278401b070 comp rx=0 tx=0).stop 2024-09-10T21:17:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f27a0000640 1 -- 172.21.15.16:0/1970149187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f279806e640 msgr2=0x7f27980c2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f27a0000640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f279806e640 0x7f27980c2b00 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f278c00d750 tx=0x7f278c00dc10 comp rx=0 tx=0).stop 2024-09-10T21:17:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f27a0000640 1 -- 172.21.15.16:0/1970149187 shutdown_connections 2024-09-10T21:17:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f27a0000640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2778072610 0x7f2778074ad0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f27a0000640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27981599d0 0x7f279815b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f27a0000640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f27980ca6b0 0x7f27980c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f27a0000640 1 --2- 172.21.15.16:0/1970149187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f279806e640 0x7f27980c2b00 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f27a0000640 1 -- 172.21.15.16:0/1970149187 >> 172.21.15.16:0/1970149187 conn(0x7f27980cfb20 msgr2=0x7f279815e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f27a0000640 1 -- 172.21.15.16:0/1970149187 shutdown_connections 2024-09-10T21:17:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.429+0000 7f27a0000640 1 -- 172.21.15.16:0/1970149187 wait complete. 2024-09-10T21:17:02.454 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:17:02.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:17:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.601+0000 7f1129a00640 1 -- 172.21.15.16:0/985558227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11240cef60 msgr2=0x7f11240cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.601+0000 7f1129a00640 1 --2- 172.21.15.16:0/985558227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11240cef60 0x7f11240cc9e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f11240d0ab0 tx=0x7f110c01c810 comp rx=0 tx=0).stop 2024-09-10T21:17:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.601+0000 7f1129a00640 1 -- 172.21.15.16:0/985558227 shutdown_connections 2024-09-10T21:17:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.601+0000 7f1129a00640 1 --2- 172.21.15.16:0/985558227 >> v2:172.21.15.52:3300/0 conn(0x7f11240b2a70 0x7f11240ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.601+0000 7f1129a00640 1 --2- 172.21.15.16:0/985558227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11240cef60 0x7f11240cc9e0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.601+0000 7f1129a00640 1 -- 172.21.15.16:0/985558227 >> 172.21.15.16:0/985558227 conn(0x7f11240d1e40 msgr2=0x7f11240d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.601+0000 7f1129a00640 1 -- 172.21.15.16:0/985558227 shutdown_connections 2024-09-10T21:17:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.601+0000 7f1129a00640 1 -- 172.21.15.16:0/985558227 wait complete. 2024-09-10T21:17:02.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.601+0000 7f1129a00640 1 Processor -- start 2024-09-10T21:17:02.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.601+0000 7f1129a00640 1 -- start start 2024-09-10T21:17:02.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1129a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f11240b2a70 0x7f11240b6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1129a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11240cef60 0x7f11240b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1129a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112415bcf0 0x7f112414eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1129a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1124169160 con 0x7f112415bcf0 2024-09-10T21:17:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1129a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1124168fe0 con 0x7f11240cef60 2024-09-10T21:17:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1129a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f11241692e0 con 0x7f11240b2a70 2024-09-10T21:17:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1123400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112415bcf0 0x7f112414eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1123400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112415bcf0 0x7f112414eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54230/0 (socket says 172.21.15.16:54230) 2024-09-10T21:17:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1123400640 1 -- 172.21.15.16:0/210984844 learned_addr learned my addr 172.21.15.16:0/210984844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f111a000640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11240cef60 0x7f11240b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1122a00640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f11240b2a70 0x7f11240b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1123400640 1 -- 172.21.15.16:0/210984844 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f11240b2a70 msgr2=0x7f11240b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1123400640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f11240b2a70 0x7f11240b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1123400640 1 -- 172.21.15.16:0/210984844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11240cef60 msgr2=0x7f11240b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1123400640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11240cef60 0x7f11240b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1123400640 1 -- 172.21.15.16:0/210984844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11240b7420 con 0x7f112415bcf0 2024-09-10T21:17:02.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1122a00640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f11240b2a70 0x7f11240b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:02.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1123400640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112415bcf0 0x7f112414eee0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f111c00eb50 tx=0x7f111c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:02.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f111b400640 1 -- 172.21.15.16:0/210984844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c004d30 con 0x7f112415bcf0 2024-09-10T21:17:02.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f111b400640 1 -- 172.21.15.16:0/210984844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f111c00ee50 con 0x7f112415bcf0 2024-09-10T21:17:02.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f111b400640 1 -- 172.21.15.16:0/210984844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c005280 con 0x7f112415bcf0 2024-09-10T21:17:02.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1129a00640 1 -- 172.21.15.16:0/210984844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11240b76f0 con 0x7f112415bcf0 2024-09-10T21:17:02.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1129a00640 1 -- 172.21.15.16:0/210984844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11240b5d60 con 0x7f112415bcf0 2024-09-10T21:17:02.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.605+0000 7f1129a00640 1 -- 172.21.15.16:0/210984844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11240ce850 con 0x7f112415bcf0 2024-09-10T21:17:02.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.609+0000 7f111b400640 1 -- 172.21.15.16:0/210984844 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f111c005420 con 0x7f112415bcf0 2024-09-10T21:17:02.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.609+0000 7f111b400640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11000725e0 0x7f1100074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.609+0000 7f111b400640 1 -- 172.21.15.16:0/210984844 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f111c093bd0 con 0x7f112415bcf0 2024-09-10T21:17:02.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.609+0000 7f1122a00640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11000725e0 0x7f1100074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.609+0000 7f1122a00640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11000725e0 0x7f1100074aa0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f11140016b0 tx=0x7f1114009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:02.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.613+0000 7f111b400640 1 -- 172.21.15.16:0/210984844 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f111c05f5f0 con 0x7f112415bcf0 2024-09-10T21:17:02.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:02.789+0000 7f1129a00640 1 -- 172.21.15.16:0/210984844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f11240761e0 con 0x7f112415bcf0 2024-09-10T21:17:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.665+0000 7f111b400640 1 -- 172.21.15.16:0/210984844 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) ==== 138+0+0 (secure 0 0 0) 0x7f111c05eef0 con 0x7f112415bcf0 2024-09-10T21:17:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-10T21:17:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.669+0000 7f1129a00640 1 -- 172.21.15.16:0/210984844 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11000725e0 msgr2=0x7f1100074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.669+0000 7f1129a00640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11000725e0 0x7f1100074aa0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f11140016b0 tx=0x7f1114009040 comp rx=0 tx=0).stop 2024-09-10T21:17:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.669+0000 7f1129a00640 1 -- 172.21.15.16:0/210984844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112415bcf0 msgr2=0x7f112414eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.669+0000 7f1129a00640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112415bcf0 0x7f112414eee0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f111c00eb50 tx=0x7f111c00c6a0 comp rx=0 tx=0).stop 2024-09-10T21:17:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.669+0000 7f1129a00640 1 -- 172.21.15.16:0/210984844 shutdown_connections 2024-09-10T21:17:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.669+0000 7f1129a00640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f11000725e0 0x7f1100074aa0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.669+0000 7f1129a00640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112415bcf0 0x7f112414eee0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.669+0000 7f1129a00640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11240cef60 0x7f11240b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.669+0000 7f1129a00640 1 --2- 172.21.15.16:0/210984844 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f11240b2a70 0x7f11240b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.669+0000 7f1129a00640 1 -- 172.21.15.16:0/210984844 >> 172.21.15.16:0/210984844 conn(0x7f11240d1e40 msgr2=0x7f11241558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.669+0000 7f1129a00640 1 -- 172.21.15.16:0/210984844 shutdown_connections 2024-09-10T21:17:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.669+0000 7f1129a00640 1 -- 172.21.15.16:0/210984844 wait complete. 2024-09-10T21:17:03.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:17:03.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.857+0000 7f9116200640 1 -- 172.21.15.16:0/3158982154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100cfc10 msgr2=0x7f91100cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.857+0000 7f9116200640 1 --2- 172.21.15.16:0/3158982154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100cfc10 0x7f91100cec90 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009900 tx=0x7f90fc01c7e0 comp rx=0 tx=0).stop 2024-09-10T21:17:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.857+0000 7f9116200640 1 -- 172.21.15.16:0/3158982154 shutdown_connections 2024-09-10T21:17:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.857+0000 7f9116200640 1 --2- 172.21.15.16:0/3158982154 >> v2:172.21.15.52:3300/0 conn(0x7f9110163870 0x7f9110165c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.857+0000 7f9116200640 1 --2- 172.21.15.16:0/3158982154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100cfc10 0x7f91100cec90 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.857+0000 7f9116200640 1 -- 172.21.15.16:0/3158982154 >> 172.21.15.16:0/3158982154 conn(0x7f91100d1b70 msgr2=0x7f91100d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.857+0000 7f9116200640 1 -- 172.21.15.16:0/3158982154 shutdown_connections 2024-09-10T21:17:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.857+0000 7f9116200640 1 -- 172.21.15.16:0/3158982154 wait complete. 2024-09-10T21:17:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.857+0000 7f9116200640 1 Processor -- start 2024-09-10T21:17:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.857+0000 7f9116200640 1 -- start start 2024-09-10T21:17:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9116200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100cfc10 0x7f911014bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9116200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f911015be70 0x7f911015aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9116200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9110163870 0x7f911015b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9116200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9110168d50 con 0x7f91100cfc10 2024-09-10T21:17:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9116200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9110168bd0 con 0x7f911015be70 2024-09-10T21:17:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9116200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9110168ed0 con 0x7f9110163870 2024-09-10T21:17:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9114e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100cfc10 0x7f911014bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9114e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100cfc10 0x7f911014bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54250/0 (socket says 172.21.15.16:54250) 2024-09-10T21:17:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9114e00640 1 -- 172.21.15.16:0/508282533 learned_addr learned my addr 172.21.15.16:0/508282533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f910fe00640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f911015be70 0x7f911015aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9115800640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9110163870 0x7f911015b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9114e00640 1 -- 172.21.15.16:0/508282533 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9110163870 msgr2=0x7f911015b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9114e00640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9110163870 0x7f911015b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9114e00640 1 -- 172.21.15.16:0/508282533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f911015be70 msgr2=0x7f911015aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9114e00640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f911015be70 0x7f911015aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9114e00640 1 -- 172.21.15.16:0/508282533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9110159530 con 0x7f91100cfc10 2024-09-10T21:17:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9114e00640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100cfc10 0x7f911014bf10 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f910400c530 tx=0x7f910400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f910cc00640 1 -- 172.21.15.16:0/508282533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104017070 con 0x7f91100cfc10 2024-09-10T21:17:03.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f910cc00640 1 -- 172.21.15.16:0/508282533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91040053c0 con 0x7f91100cfc10 2024-09-10T21:17:03.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f910cc00640 1 -- 172.21.15.16:0/508282533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104012890 con 0x7f91100cfc10 2024-09-10T21:17:03.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9116200640 1 -- 172.21.15.16:0/508282533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9110159810 con 0x7f91100cfc10 2024-09-10T21:17:03.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9116200640 1 -- 172.21.15.16:0/508282533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9110159dd0 con 0x7f91100cfc10 2024-09-10T21:17:03.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f9116200640 1 -- 172.21.15.16:0/508282533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91100cea60 con 0x7f91100cfc10 2024-09-10T21:17:03.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.861+0000 7f910cc00640 1 -- 172.21.15.16:0/508282533 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9104003600 con 0x7f91100cfc10 2024-09-10T21:17:03.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.865+0000 7f910cc00640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90f00725e0 0x7f90f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:03.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.865+0000 7f910cc00640 1 -- 172.21.15.16:0/508282533 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5227+0+0 (secure 0 0 0) 0x7f91040926a0 con 0x7f91100cfc10 2024-09-10T21:17:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.865+0000 7f910fe00640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90f00725e0 0x7f90f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.865+0000 7f910fe00640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90f00725e0 0x7f90f0074aa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0af000 tx=0x7f90fc01b020 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:03.869+0000 7f910cc00640 1 -- 172.21.15.16:0/508282533 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f910405d870 con 0x7f91100cfc10 2024-09-10T21:17:04.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.045+0000 7f9116200640 1 -- 172.21.15.16:0/508282533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f911014c450 con 0x7f91100cfc10 2024-09-10T21:17:04.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.669+0000 7f910cc00640 1 -- 172.21.15.16:0/508282533 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) ==== 170+0+0 (secure 0 0 0) 0x7f91040622a0 con 0x7f91100cfc10 2024-09-10T21:17:04.674 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-10T21:17:04.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.673+0000 7f9116200640 1 -- 172.21.15.16:0/508282533 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90f00725e0 msgr2=0x7f90f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.673+0000 7f9116200640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90f00725e0 0x7f90f0074aa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0af000 tx=0x7f90fc01b020 comp rx=0 tx=0).stop 2024-09-10T21:17:04.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.673+0000 7f9116200640 1 -- 172.21.15.16:0/508282533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100cfc10 msgr2=0x7f911014bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.673+0000 7f9116200640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100cfc10 0x7f911014bf10 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f910400c530 tx=0x7f910400c9f0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.673+0000 7f9116200640 1 -- 172.21.15.16:0/508282533 shutdown_connections 2024-09-10T21:17:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.677+0000 7f9116200640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f90f00725e0 0x7f90f0074aa0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.677+0000 7f9116200640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9110163870 0x7f911015b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.677+0000 7f9116200640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f911015be70 0x7f911015aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.677+0000 7f9116200640 1 --2- 172.21.15.16:0/508282533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100cfc10 0x7f911014bf10 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.677+0000 7f9116200640 1 -- 172.21.15.16:0/508282533 >> 172.21.15.16:0/508282533 conn(0x7f91100d1b70 msgr2=0x7f9110163c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.677+0000 7f9116200640 1 -- 172.21.15.16:0/508282533 shutdown_connections 2024-09-10T21:17:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.677+0000 7f9116200640 1 -- 172.21.15.16:0/508282533 wait complete. 2024-09-10T21:17:04.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:17:04.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 -- 172.21.15.16:0/2454439552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf00d0210 msgr2=0x7fabf00ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 --2- 172.21.15.16:0/2454439552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf00d0210 0x7fabf00ceca0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009930 tx=0x7fabdc01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 -- 172.21.15.16:0/2454439552 shutdown_connections 2024-09-10T21:17:04.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 --2- 172.21.15.16:0/2454439552 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fabf00ce040 0x7fabf00b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 --2- 172.21.15.16:0/2454439552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf00d0210 0x7fabf00ceca0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 -- 172.21.15.16:0/2454439552 >> 172.21.15.16:0/2454439552 conn(0x7fabf00d1df0 msgr2=0x7fabf00d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:04.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 -- 172.21.15.16:0/2454439552 shutdown_connections 2024-09-10T21:17:04.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 -- 172.21.15.16:0/2454439552 wait complete. 2024-09-10T21:17:04.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 Processor -- start 2024-09-10T21:17:04.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 -- start start 2024-09-10T21:17:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf00ce040 0x7fabf00c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf00cfe30 0x7fabf00c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fabf00d0210 0x7fabf01f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabf00b1460 con 0x7fabf00cfe30 2024-09-10T21:17:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fabf00b12e0 con 0x7fabf00ce040 2024-09-10T21:17:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.869+0000 7fabf8000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fabf00cf1e0 con 0x7fabf00d0210 2024-09-10T21:17:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf00cfe30 0x7fabf00c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf00cfe30 0x7fabf00c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54262/0 (socket says 172.21.15.16:54262) 2024-09-10T21:17:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf4c00640 1 -- 172.21.15.16:0/2497766543 learned_addr learned my addr 172.21.15.16:0/2497766543 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf5600640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf00ce040 0x7fabf00c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf6000640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fabf00d0210 0x7fabf01f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf4c00640 1 -- 172.21.15.16:0/2497766543 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fabf00d0210 msgr2=0x7fabf01f85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf4c00640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fabf00d0210 0x7fabf01f85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf4c00640 1 -- 172.21.15.16:0/2497766543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf00ce040 msgr2=0x7fabf00c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf4c00640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf00ce040 0x7fabf00c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf4c00640 1 -- 172.21.15.16:0/2497766543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf00c7830 con 0x7fabf00cfe30 2024-09-10T21:17:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf4c00640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf00cfe30 0x7fabf00c5b80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b3040 tx=0x7fabdc005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabed600640 1 -- 172.21.15.16:0/2497766543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0b5040 con 0x7fabf00cfe30 2024-09-10T21:17:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabed600640 1 -- 172.21.15.16:0/2497766543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabdc003600 con 0x7fabf00cfe30 2024-09-10T21:17:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabed600640 1 -- 172.21.15.16:0/2497766543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc003a50 con 0x7fabf00cfe30 2024-09-10T21:17:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf8000640 1 -- 172.21.15.16:0/2497766543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf01f58e0 con 0x7fabf00cfe30 2024-09-10T21:17:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf8000640 1 -- 172.21.15.16:0/2497766543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf00c7db0 con 0x7fabf00cfe30 2024-09-10T21:17:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.873+0000 7fabf8000640 1 -- 172.21.15.16:0/2497766543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf00b1710 con 0x7fabf00cfe30 2024-09-10T21:17:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.877+0000 7fabed600640 1 -- 172.21.15.16:0/2497766543 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fabdc0b8050 con 0x7fabf00cfe30 2024-09-10T21:17:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.877+0000 7fabed600640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fabd00725e0 0x7fabd0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.877+0000 7fabed600640 1 -- 172.21.15.16:0/2497766543 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5609+0+0 (secure 0 0 0) 0x7fabdc1309b0 con 0x7fabf00cfe30 2024-09-10T21:17:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.877+0000 7fabf5600640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fabd00725e0 0x7fabd0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.877+0000 7fabf5600640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fabd00725e0 0x7fabd0074aa0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fabf015d7f0 tx=0x7fabe4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:04.881+0000 7fabed600640 1 -- 172.21.15.16:0/2497766543 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fabdc0c8020 con 0x7fabf00cfe30 2024-09-10T21:17:05.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.089+0000 7fabf8000640 1 -- 172.21.15.16:0/2497766543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fabf00b3110 con 0x7fabf00cfe30 2024-09-10T21:17:05.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.701+0000 7fabed600640 1 -- 172.21.15.16:0/2497766543 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v67) ==== 174+0+0 (secure 0 0 0) 0x7fabdc018c30 con 0x7fabf00cfe30 2024-09-10T21:17:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 16 and data pool 17 2024-09-10T21:17:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.713+0000 7fabf8000640 1 -- 172.21.15.16:0/2497766543 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fabd00725e0 msgr2=0x7fabd0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.713+0000 7fabf8000640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fabd00725e0 0x7fabd0074aa0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fabf015d7f0 tx=0x7fabe4009040 comp rx=0 tx=0).stop 2024-09-10T21:17:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.713+0000 7fabf8000640 1 -- 172.21.15.16:0/2497766543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf00cfe30 msgr2=0x7fabf00c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.713+0000 7fabf8000640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf00cfe30 0x7fabf00c5b80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b3040 tx=0x7fabdc005760 comp rx=0 tx=0).stop 2024-09-10T21:17:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.713+0000 7fabf8000640 1 -- 172.21.15.16:0/2497766543 shutdown_connections 2024-09-10T21:17:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.713+0000 7fabf8000640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fabd00725e0 0x7fabd0074aa0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.713+0000 7fabf8000640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fabf00d0210 0x7fabf01f85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.713+0000 7fabf8000640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf00cfe30 0x7fabf00c5b80 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.713+0000 7fabf8000640 1 --2- 172.21.15.16:0/2497766543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabf00ce040 0x7fabf00c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.713+0000 7fabf8000640 1 -- 172.21.15.16:0/2497766543 >> 172.21.15.16:0/2497766543 conn(0x7fabf00d1df0 msgr2=0x7fabf00aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.713+0000 7fabf8000640 1 -- 172.21.15.16:0/2497766543 shutdown_connections 2024-09-10T21:17:05.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.713+0000 7fabf8000640 1 -- 172.21.15.16:0/2497766543 wait complete. 2024-09-10T21:17:05.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:17:05.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.893+0000 7f6f12a00640 1 -- 172.21.15.16:0/2538059006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c0cc420 msgr2=0x7f6f0c0cbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.893+0000 7f6f12a00640 1 --2- 172.21.15.16:0/2538059006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c0cc420 0x7f6f0c0cbc90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0b6b60 tx=0x7f6f0401ccc0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.893+0000 7f6f12a00640 1 -- 172.21.15.16:0/2538059006 shutdown_connections 2024-09-10T21:17:05.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.893+0000 7f6f12a00640 1 --2- 172.21.15.16:0/2538059006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c0cc420 0x7f6f0c0cbc90 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.893+0000 7f6f12a00640 1 --2- 172.21.15.16:0/2538059006 >> v2:172.21.15.52:3300/0 conn(0x7f6f0c15be00 0x7f6f0c070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.893+0000 7f6f12a00640 1 -- 172.21.15.16:0/2538059006 >> 172.21.15.16:0/2538059006 conn(0x7f6f0c0d0c30 msgr2=0x7f6f0c0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:05.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.893+0000 7f6f12a00640 1 -- 172.21.15.16:0/2538059006 shutdown_connections 2024-09-10T21:17:05.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.893+0000 7f6f12a00640 1 -- 172.21.15.16:0/2538059006 wait complete. 2024-09-10T21:17:05.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f12a00640 1 Processor -- start 2024-09-10T21:17:05.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f12a00640 1 -- start start 2024-09-10T21:17:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f12a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f0c0cc420 0x7f6f0c156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f12a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f0c0cc960 0x7f6f0c1f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f12a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c0b1c10 0x7f6f0c0b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f12a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c168de0 con 0x7f6f0c0b1c10 2024-09-10T21:17:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f12a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c168c60 con 0x7f6f0c0cc960 2024-09-10T21:17:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f12a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0c168f60 con 0x7f6f0c0cc420 2024-09-10T21:17:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c0b1c10 0x7f6f0c0b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f03e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f0c0cc960 0x7f6f0c1f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c0b1c10 0x7f6f0c0b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54276/0 (socket says 172.21.15.16:54276) 2024-09-10T21:17:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f10a00640 1 -- 172.21.15.16:0/1003787825 learned_addr learned my addr 172.21.15.16:0/1003787825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f0be00640 1 --2- 172.21.15.16:0/1003787825 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f0c0cc420 0x7f6f0c156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f10a00640 1 -- 172.21.15.16:0/1003787825 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f0c0cc420 msgr2=0x7f6f0c156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f10a00640 1 --2- 172.21.15.16:0/1003787825 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f0c0cc420 0x7f6f0c156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f10a00640 1 -- 172.21.15.16:0/1003787825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f0c0cc960 msgr2=0x7f6f0c1f84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f10a00640 1 --2- 172.21.15.16:0/1003787825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f0c0cc960 0x7f6f0c1f84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f10a00640 1 -- 172.21.15.16:0/1003787825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c1f8a00 con 0x7f6f0c0b1c10 2024-09-10T21:17:05.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f10a00640 1 --2- 172.21.15.16:0/1003787825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c0b1c10 0x7f6f0c0b20c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6f0401b370 tx=0x7f6f04003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f08c00640 1 -- 172.21.15.16:0/1003787825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f040bb070 con 0x7f6f0c0b1c10 2024-09-10T21:17:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f08c00640 1 -- 172.21.15.16:0/1003787825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f04004e00 con 0x7f6f0c0b1c10 2024-09-10T21:17:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f08c00640 1 -- 172.21.15.16:0/1003787825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f040b2040 con 0x7f6f0c0b1c10 2024-09-10T21:17:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f12a00640 1 -- 172.21.15.16:0/1003787825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0c0b4fb0 con 0x7f6f0c0b1c10 2024-09-10T21:17:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.897+0000 7f6f12a00640 1 -- 172.21.15.16:0/1003787825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0c0b5570 con 0x7f6f0c0b1c10 2024-09-10T21:17:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.901+0000 7f6f12a00640 1 -- 172.21.15.16:0/1003787825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0c0cbc90 con 0x7f6f0c0b1c10 2024-09-10T21:17:05.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.901+0000 7f6f08c00640 1 -- 172.21.15.16:0/1003787825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6f040b5030 con 0x7f6f0c0b1c10 2024-09-10T21:17:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.901+0000 7f6f08c00640 1 --2- 172.21.15.16:0/1003787825 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6ee80725e0 0x7f6ee8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.901+0000 7f6f08c00640 1 -- 172.21.15.16:0/1003787825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f6f0412cfe0 con 0x7f6f0c0b1c10 2024-09-10T21:17:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.901+0000 7f6f0be00640 1 --2- 172.21.15.16:0/1003787825 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6ee80725e0 0x7f6ee8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.905+0000 7f6f0be00640 1 --2- 172.21.15.16:0/1003787825 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6ee80725e0 0x7f6ee8074aa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6efc007fd0 tx=0x7f6efc007e40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:05.905+0000 7f6f08c00640 1 -- 172.21.15.16:0/1003787825 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6f0c1f8a00 con 0x7f6f0c0b1c10 2024-09-10T21:17:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.081+0000 7f6f12a00640 1 -- 172.21.15.16:0/1003787825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f0c0b6bb0 con 0x7f6f0c0b1c10 2024-09-10T21:17:06.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.081+0000 7f6f08c00640 1 -- 172.21.15.16:0/1003787825 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16708 (secure 0 0 0) 0x7f6f04016970 con 0x7f6f0c0b1c10 2024-09-10T21:17:06.086 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:06.086 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":102,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:17:05.094319+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:17:02.802106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T21:17:04.057780+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:17:06.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.085+0000 7f6f12a00640 1 -- 172.21.15.16:0/1003787825 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6ee80725e0 msgr2=0x7f6ee8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:06.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.085+0000 7f6f12a00640 1 --2- 172.21.15.16:0/1003787825 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6ee80725e0 0x7f6ee8074aa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6efc007fd0 tx=0x7f6efc007e40 comp rx=0 tx=0).stop 2024-09-10T21:17:06.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.085+0000 7f6f12a00640 1 -- 172.21.15.16:0/1003787825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c0b1c10 msgr2=0x7f6f0c0b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:06.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.085+0000 7f6f12a00640 1 --2- 172.21.15.16:0/1003787825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c0b1c10 0x7f6f0c0b20c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6f0401b370 tx=0x7f6f04003f60 comp rx=0 tx=0).stop 2024-09-10T21:17:06.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.085+0000 7f6f12a00640 1 -- 172.21.15.16:0/1003787825 shutdown_connections 2024-09-10T21:17:06.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.085+0000 7f6f12a00640 1 --2- 172.21.15.16:0/1003787825 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6ee80725e0 0x7f6ee8074aa0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.085+0000 7f6f12a00640 1 --2- 172.21.15.16:0/1003787825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f0c0b1c10 0x7f6f0c0b20c0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.085+0000 7f6f12a00640 1 --2- 172.21.15.16:0/1003787825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f0c0cc960 0x7f6f0c1f84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.085+0000 7f6f12a00640 1 --2- 172.21.15.16:0/1003787825 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f0c0cc420 0x7f6f0c156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.085+0000 7f6f12a00640 1 -- 172.21.15.16:0/1003787825 >> 172.21.15.16:0/1003787825 conn(0x7f6f0c0d0c30 msgr2=0x7f6f0c159ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:06.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.085+0000 7f6f12a00640 1 -- 172.21.15.16:0/1003787825 shutdown_connections 2024-09-10T21:17:06.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.085+0000 7f6f12a00640 1 -- 172.21.15.16:0/1003787825 wait complete. 2024-09-10T21:17:06.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:17:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.249+0000 7f2b0f800640 1 -- 172.21.15.16:0/1438847969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b080ca170 msgr2=0x7f2b080ca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.249+0000 7f2b0f800640 1 --2- 172.21.15.16:0/1438847969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b080ca170 0x7f2b080ca550 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f2b080af550 tx=0x7f2af801c890 comp rx=0 tx=0).stop 2024-09-10T21:17:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.249+0000 7f2b0f800640 1 -- 172.21.15.16:0/1438847969 shutdown_connections 2024-09-10T21:17:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.249+0000 7f2b0f800640 1 --2- 172.21.15.16:0/1438847969 >> v2:172.21.15.52:3300/0 conn(0x7f2b080cf370 0x7f2b080ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.249+0000 7f2b0f800640 1 --2- 172.21.15.16:0/1438847969 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b080cd6f0 0x7f2b080cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.249+0000 7f2b0f800640 1 --2- 172.21.15.16:0/1438847969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b080ca170 0x7f2b080ca550 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.249+0000 7f2b0f800640 1 -- 172.21.15.16:0/1438847969 >> 172.21.15.16:0/1438847969 conn(0x7f2b080cfb20 msgr2=0x7f2b080cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.249+0000 7f2b0f800640 1 -- 172.21.15.16:0/1438847969 shutdown_connections 2024-09-10T21:17:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.249+0000 7f2b0f800640 1 -- 172.21.15.16:0/1438847969 wait complete. 2024-09-10T21:17:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0f800640 1 Processor -- start 2024-09-10T21:17:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0f800640 1 -- start start 2024-09-10T21:17:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0f800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b080ca170 0x7f2b080b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0f800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b080cd6f0 0x7f2b0815c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0f800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b080cf370 0x7f2b0815cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0f800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b08073040 con 0x7f2b080cd6f0 2024-09-10T21:17:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0f800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b08072ec0 con 0x7f2b080cf370 2024-09-10T21:17:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0f800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2b080731c0 con 0x7f2b080ca170 2024-09-10T21:17:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2affe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b080cd6f0 0x7f2b0815c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2affe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b080cd6f0 0x7f2b0815c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54286/0 (socket says 172.21.15.16:54286) 2024-09-10T21:17:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2affe00640 1 -- 172.21.15.16:0/1872250008 learned_addr learned my addr 172.21.15.16:0/1872250008 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0ce00640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b080ca170 0x7f2b080b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0d800640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b080cf370 0x7f2b0815cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2affe00640 1 -- 172.21.15.16:0/1872250008 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b080ca170 msgr2=0x7f2b080b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2affe00640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b080ca170 0x7f2b080b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2affe00640 1 -- 172.21.15.16:0/1872250008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b080cf370 msgr2=0x7f2b0815cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2affe00640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b080cf370 0x7f2b0815cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2affe00640 1 -- 172.21.15.16:0/1872250008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b081fbc50 con 0x7f2b080cd6f0 2024-09-10T21:17:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0d800640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b080cf370 0x7f2b0815cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2affe00640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b080cd6f0 0x7f2b0815c6a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2af00058d0 tx=0x7f2af000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b05600640 1 -- 172.21.15.16:0/1872250008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0017070 con 0x7f2b080cd6f0 2024-09-10T21:17:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b05600640 1 -- 172.21.15.16:0/1872250008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2af0009b10 con 0x7f2b080cd6f0 2024-09-10T21:17:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b05600640 1 -- 172.21.15.16:0/1872250008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0010460 con 0x7f2b080cd6f0 2024-09-10T21:17:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0f800640 1 -- 172.21.15.16:0/1872250008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b081fbf30 con 0x7f2b080cd6f0 2024-09-10T21:17:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0f800640 1 -- 172.21.15.16:0/1872250008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b081fc4f0 con 0x7f2b080cd6f0 2024-09-10T21:17:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.253+0000 7f2b0f800640 1 -- 172.21.15.16:0/1872250008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b080ca550 con 0x7f2b080cd6f0 2024-09-10T21:17:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.257+0000 7f2b05600640 1 -- 172.21.15.16:0/1872250008 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2af0010810 con 0x7f2b080cd6f0 2024-09-10T21:17:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.257+0000 7f2b05600640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2ae40725e0 0x7f2ae4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.257+0000 7f2b05600640 1 -- 172.21.15.16:0/1872250008 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f2af0093820 con 0x7f2b080cd6f0 2024-09-10T21:17:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.257+0000 7f2b0ce00640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2ae40725e0 0x7f2ae4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:06.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.257+0000 7f2b0ce00640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2ae40725e0 0x7f2ae4074aa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2af80af000 tx=0x7f2af801b0d0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.261+0000 7f2b05600640 1 -- 172.21.15.16:0/1872250008 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2af005e8a0 con 0x7f2b080cd6f0 2024-09-10T21:17:06.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.437+0000 7f2b0f800640 1 -- 172.21.15.16:0/1872250008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b080b5aa0 con 0x7f2b080cd6f0 2024-09-10T21:17:06.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.437+0000 7f2b05600640 1 -- 172.21.15.16:0/1872250008 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16708 (secure 0 0 0) 0x7f2af00632d0 con 0x7f2b080cd6f0 2024-09-10T21:17:06.441 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:06.441 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":102,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:17:05.094319+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:17:02.802106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T21:17:04.057780+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:17:06.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.437+0000 7f2b0f800640 1 -- 172.21.15.16:0/1872250008 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2ae40725e0 msgr2=0x7f2ae4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:06.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.437+0000 7f2b0f800640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2ae40725e0 0x7f2ae4074aa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2af80af000 tx=0x7f2af801b0d0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.437+0000 7f2b0f800640 1 -- 172.21.15.16:0/1872250008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b080cd6f0 msgr2=0x7f2b0815c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:06.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.437+0000 7f2b0f800640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b080cd6f0 0x7f2b0815c6a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2af00058d0 tx=0x7f2af000db90 comp rx=0 tx=0).stop 2024-09-10T21:17:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.437+0000 7f2b0f800640 1 -- 172.21.15.16:0/1872250008 shutdown_connections 2024-09-10T21:17:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.437+0000 7f2b0f800640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2ae40725e0 0x7f2ae4074aa0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.437+0000 7f2b0f800640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b080cf370 0x7f2b0815cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.437+0000 7f2b0f800640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b080cd6f0 0x7f2b0815c6a0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.437+0000 7f2b0f800640 1 --2- 172.21.15.16:0/1872250008 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b080ca170 0x7f2b080b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.437+0000 7f2b0f800640 1 -- 172.21.15.16:0/1872250008 >> 172.21.15.16:0/1872250008 conn(0x7f2b080cfb20 msgr2=0x7f2b080b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.441+0000 7f2b0f800640 1 -- 172.21.15.16:0/1872250008 shutdown_connections 2024-09-10T21:17:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.441+0000 7f2b0f800640 1 -- 172.21.15.16:0/1872250008 wait complete. 2024-09-10T21:17:06.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:17:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.609+0000 7f19a6c00640 1 -- 172.21.15.16:0/4080298251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a0151210 msgr2=0x7f19a00726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.609+0000 7f19a6c00640 1 --2- 172.21.15.16:0/4080298251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a0151210 0x7f19a00726b0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f19a0074b60 tx=0x7f199001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.609+0000 7f19a6c00640 1 -- 172.21.15.16:0/4080298251 shutdown_connections 2024-09-10T21:17:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.609+0000 7f19a6c00640 1 --2- 172.21.15.16:0/4080298251 >> v2:172.21.15.52:3300/0 conn(0x7f19a0072bf0 0x7f19a015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.609+0000 7f19a6c00640 1 --2- 172.21.15.16:0/4080298251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a0151210 0x7f19a00726b0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.609+0000 7f19a6c00640 1 -- 172.21.15.16:0/4080298251 >> 172.21.15.16:0/4080298251 conn(0x7f19a00b16a0 msgr2=0x7f19a00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.609+0000 7f19a6c00640 1 -- 172.21.15.16:0/4080298251 shutdown_connections 2024-09-10T21:17:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.609+0000 7f19a6c00640 1 -- 172.21.15.16:0/4080298251 wait complete. 2024-09-10T21:17:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.609+0000 7f19a6c00640 1 Processor -- start 2024-09-10T21:17:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.609+0000 7f19a6c00640 1 -- start start 2024-09-10T21:17:06.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a6c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f19a0072bf0 0x7f19a013f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:06.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a6c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19a0151210 0x7f19a013c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:06.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a6c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a015e2c0 0x7f19a01fb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:06.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a6c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19a015b280 con 0x7f19a015e2c0 2024-09-10T21:17:06.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a6c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f19a015b100 con 0x7f19a0151210 2024-09-10T21:17:06.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a6c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f19a015b400 con 0x7f19a0072bf0 2024-09-10T21:17:06.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a015e2c0 0x7f19a01fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:06.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a015e2c0 0x7f19a01fb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54308/0 (socket says 172.21.15.16:54308) 2024-09-10T21:17:06.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a4c00640 1 -- 172.21.15.16:0/3336852404 learned_addr learned my addr 172.21.15.16:0/3336852404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:06.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f199fe00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f19a0072bf0 0x7f19a013f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:06.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f1997e00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19a0151210 0x7f19a013c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:06.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a4c00640 1 -- 172.21.15.16:0/3336852404 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f19a0072bf0 msgr2=0x7f19a013f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:06.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a4c00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f19a0072bf0 0x7f19a013f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a4c00640 1 -- 172.21.15.16:0/3336852404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19a0151210 msgr2=0x7f19a013c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:06.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a4c00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19a0151210 0x7f19a013c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:06.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a4c00640 1 -- 172.21.15.16:0/3336852404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a00a9780 con 0x7f19a015e2c0 2024-09-10T21:17:06.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a4c00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a015e2c0 0x7f19a01fb220 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f199800c730 tx=0x7f199800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:06.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f199cc00640 1 -- 172.21.15.16:0/3336852404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998003ba0 con 0x7f19a015e2c0 2024-09-10T21:17:06.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f199cc00640 1 -- 172.21.15.16:0/3336852404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1998003d60 con 0x7f19a015e2c0 2024-09-10T21:17:06.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a6c00640 1 -- 172.21.15.16:0/3336852404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a00a9a60 con 0x7f19a015e2c0 2024-09-10T21:17:06.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f199cc00640 1 -- 172.21.15.16:0/3336852404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199800aa50 con 0x7f19a015e2c0 2024-09-10T21:17:06.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a6c00640 1 -- 172.21.15.16:0/3336852404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a013c9e0 con 0x7f19a015e2c0 2024-09-10T21:17:06.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f19a6c00640 1 -- 172.21.15.16:0/3336852404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a00a9ca0 con 0x7f19a015e2c0 2024-09-10T21:17:06.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.613+0000 7f199cc00640 1 -- 172.21.15.16:0/3336852404 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f199800abf0 con 0x7f19a015e2c0 2024-09-10T21:17:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.617+0000 7f199cc00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f197c0725e0 0x7f197c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.617+0000 7f199cc00640 1 -- 172.21.15.16:0/3336852404 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f1998092f00 con 0x7f19a015e2c0 2024-09-10T21:17:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.617+0000 7f199fe00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f197c0725e0 0x7f197c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.617+0000 7f199fe00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f197c0725e0 0x7f197c074aa0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f19900af000 tx=0x7f199001b100 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.621+0000 7f199cc00640 1 -- 172.21.15.16:0/3336852404 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f199805def0 con 0x7f19a015e2c0 2024-09-10T21:17:06.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:06.833+0000 7f19a6c00640 1 -- 172.21.15.16:0/3336852404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f19a015b8c0 con 0x7f19a015e2c0 2024-09-10T21:17:07.670 INFO:tasks.ceph.mds.b.smithi052.stderr:2024-09-10T21:17:07.664+0000 7ff1b6400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:17:07.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.713+0000 7f199cc00640 1 -- 172.21.15.16:0/3336852404 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) ==== 118+0+0 (secure 0 0 0) 0x7f1998062920 con 0x7f19a015e2c0 2024-09-10T21:17:07.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.717+0000 7f19a6c00640 1 -- 172.21.15.16:0/3336852404 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f197c0725e0 msgr2=0x7f197c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.717+0000 7f19a6c00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f197c0725e0 0x7f197c074aa0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f19900af000 tx=0x7f199001b100 comp rx=0 tx=0).stop 2024-09-10T21:17:07.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.717+0000 7f19a6c00640 1 -- 172.21.15.16:0/3336852404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a015e2c0 msgr2=0x7f19a01fb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.717+0000 7f19a6c00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a015e2c0 0x7f19a01fb220 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f199800c730 tx=0x7f199800cbf0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.717+0000 7f19a6c00640 1 -- 172.21.15.16:0/3336852404 shutdown_connections 2024-09-10T21:17:07.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.717+0000 7f19a6c00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f197c0725e0 0x7f197c074aa0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.717+0000 7f19a6c00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a015e2c0 0x7f19a01fb220 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.717+0000 7f19a6c00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19a0151210 0x7f19a013c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.717+0000 7f19a6c00640 1 --2- 172.21.15.16:0/3336852404 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f19a0072bf0 0x7f19a013f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.717+0000 7f19a6c00640 1 -- 172.21.15.16:0/3336852404 >> 172.21.15.16:0/3336852404 conn(0x7f19a00b16a0 msgr2=0x7f19a00afa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:07.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.717+0000 7f19a6c00640 1 -- 172.21.15.16:0/3336852404 shutdown_connections 2024-09-10T21:17:07.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.717+0000 7f19a6c00640 1 -- 172.21.15.16:0/3336852404 wait complete. 2024-09-10T21:17:07.744 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:17:07.744 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:17:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.893+0000 7fb060800640 1 -- 172.21.15.16:0/4203624216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb058160050 msgr2=0x7fb0580d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.893+0000 7fb060800640 1 --2- 172.21.15.16:0/4203624216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb058160050 0x7fb0580d2800 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb054009c80 tx=0x7fb05401cc80 comp rx=0 tx=0).stop 2024-09-10T21:17:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.893+0000 7fb060800640 1 -- 172.21.15.16:0/4203624216 shutdown_connections 2024-09-10T21:17:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.893+0000 7fb060800640 1 --2- 172.21.15.16:0/4203624216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb058160050 0x7fb0580d2800 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.893+0000 7fb060800640 1 -- 172.21.15.16:0/4203624216 >> 172.21.15.16:0/4203624216 conn(0x7fb0580d1ce0 msgr2=0x7fb0580d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.893+0000 7fb060800640 1 -- 172.21.15.16:0/4203624216 shutdown_connections 2024-09-10T21:17:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.893+0000 7fb060800640 1 -- 172.21.15.16:0/4203624216 wait complete. 2024-09-10T21:17:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb060800640 1 Processor -- start 2024-09-10T21:17:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb060800640 1 -- start start 2024-09-10T21:17:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb060800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0580cd300 0x7fb058155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb060800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb05815bdc0 0x7fb058164f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb060800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb058160050 0x7fb05806e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb060800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb05816a300 con 0x7fb058160050 2024-09-10T21:17:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb060800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb05816a180 con 0x7fb0580cd300 2024-09-10T21:17:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb060800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb05816a480 con 0x7fb05815bdc0 2024-09-10T21:17:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb05e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb058160050 0x7fb05806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb05e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb058160050 0x7fb05806e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54314/0 (socket says 172.21.15.16:54314) 2024-09-10T21:17:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb05e800640 1 -- 172.21.15.16:0/609904395 learned_addr learned my addr 172.21.15.16:0/609904395 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb05e800640 1 -- 172.21.15.16:0/609904395 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb05815bdc0 msgr2=0x7fb058164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb05e800640 1 --2- 172.21.15.16:0/609904395 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb05815bdc0 0x7fb058164f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb05e800640 1 -- 172.21.15.16:0/609904395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0580cd300 msgr2=0x7fb058155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb05e800640 1 --2- 172.21.15.16:0/609904395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0580cd300 0x7fb058155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb05e800640 1 -- 172.21.15.16:0/609904395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05806f560 con 0x7fb058160050 2024-09-10T21:17:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb05e800640 1 --2- 172.21.15.16:0/609904395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb058160050 0x7fb05806e390 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb05401a8b0 tx=0x7fb054009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:07.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb052000640 1 -- 172.21.15.16:0/609904395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540b2040 con 0x7fb058160050 2024-09-10T21:17:07.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb052000640 1 -- 172.21.15.16:0/609904395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb054004220 con 0x7fb058160050 2024-09-10T21:17:07.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb052000640 1 -- 172.21.15.16:0/609904395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054004670 con 0x7fb058160050 2024-09-10T21:17:07.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb060800640 1 -- 172.21.15.16:0/609904395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05806f840 con 0x7fb058160050 2024-09-10T21:17:07.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.897+0000 7fb060800640 1 -- 172.21.15.16:0/609904395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0581654a0 con 0x7fb058160050 2024-09-10T21:17:07.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.901+0000 7fb060800640 1 -- 172.21.15.16:0/609904395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0580d2800 con 0x7fb058160050 2024-09-10T21:17:07.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.901+0000 7fb052000640 1 -- 172.21.15.16:0/609904395 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb0540b5070 con 0x7fb058160050 2024-09-10T21:17:07.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.901+0000 7fb052000640 1 --2- 172.21.15.16:0/609904395 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb03c0725e0 0x7fb03c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.901+0000 7fb052000640 1 -- 172.21.15.16:0/609904395 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fb05412d600 con 0x7fb058160050 2024-09-10T21:17:07.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.901+0000 7fb05de00640 1 --2- 172.21.15.16:0/609904395 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb03c0725e0 0x7fb03c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.901+0000 7fb05de00640 1 --2- 172.21.15.16:0/609904395 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb03c0725e0 0x7fb03c074aa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb04c000f70 tx=0x7fb04c0079c0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:07.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:07.905+0000 7fb052000640 1 -- 172.21.15.16:0/609904395 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb054016990 con 0x7fb058160050 2024-09-10T21:17:08.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.117+0000 7fb060800640 1 -- 172.21.15.16:0/609904395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb0580704a0 con 0x7fb058160050 2024-09-10T21:17:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.717+0000 7fb052000640 1 -- 172.21.15.16:0/609904395 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) ==== 122+0+0 (secure 0 0 0) 0x7fb0540f8cd0 con 0x7fb058160050 2024-09-10T21:17:08.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.717+0000 7fb060800640 1 -- 172.21.15.16:0/609904395 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb03c0725e0 msgr2=0x7fb03c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.717+0000 7fb060800640 1 --2- 172.21.15.16:0/609904395 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb03c0725e0 0x7fb03c074aa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb04c000f70 tx=0x7fb04c0079c0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.717+0000 7fb060800640 1 -- 172.21.15.16:0/609904395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb058160050 msgr2=0x7fb05806e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.717+0000 7fb060800640 1 --2- 172.21.15.16:0/609904395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb058160050 0x7fb05806e390 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb05401a8b0 tx=0x7fb054009320 comp rx=0 tx=0).stop 2024-09-10T21:17:08.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.717+0000 7fb060800640 1 -- 172.21.15.16:0/609904395 shutdown_connections 2024-09-10T21:17:08.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.717+0000 7fb060800640 1 --2- 172.21.15.16:0/609904395 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb03c0725e0 0x7fb03c074aa0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.717+0000 7fb060800640 1 --2- 172.21.15.16:0/609904395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb058160050 0x7fb05806e390 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.717+0000 7fb060800640 1 --2- 172.21.15.16:0/609904395 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb05815bdc0 0x7fb058164f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.717+0000 7fb060800640 1 --2- 172.21.15.16:0/609904395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0580cd300 0x7fb058155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.717+0000 7fb060800640 1 -- 172.21.15.16:0/609904395 >> 172.21.15.16:0/609904395 conn(0x7fb0580d1ce0 msgr2=0x7fb0580d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:08.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.717+0000 7fb060800640 1 -- 172.21.15.16:0/609904395 shutdown_connections 2024-09-10T21:17:08.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.717+0000 7fb060800640 1 -- 172.21.15.16:0/609904395 wait complete. 2024-09-10T21:17:08.749 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:17:08.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.897+0000 7f6d2c000640 1 -- 172.21.15.16:0/396780971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d0c0039f0 msgr2=0x7f6d0c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.897+0000 7f6d2c000640 1 --2- 172.21.15.16:0/396780971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d0c0039f0 0x7f6d0c003dd0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6d18006f90 tx=0x7f6d1801cf20 comp rx=0 tx=0).stop 2024-09-10T21:17:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.897+0000 7f6d2a000640 1 -- 172.21.15.16:0/396780971 reap_dead start 2024-09-10T21:17:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.897+0000 7f6d2c000640 1 -- 172.21.15.16:0/396780971 shutdown_connections 2024-09-10T21:17:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.897+0000 7f6d2c000640 1 -- 172.21.15.16:0/396780971 >> 172.21.15.16:0/396780971 conn(0x7f6d240cfb20 msgr2=0x7f6d240cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.897+0000 7f6d2c000640 1 -- 172.21.15.16:0/396780971 shutdown_connections 2024-09-10T21:17:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.897+0000 7f6d2c000640 1 -- 172.21.15.16:0/396780971 wait complete. 2024-09-10T21:17:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d2c000640 1 Processor -- start 2024-09-10T21:17:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d2c000640 1 -- start start 2024-09-10T21:17:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d2c000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d0c0039f0 0x7f6d241522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d2c000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d2415e510 0x7f6d24157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d2c000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6d2415a8d0 0x7f6d2415ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d2c000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24072fe0 con 0x7f6d2415e510 2024-09-10T21:17:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d2c000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24072e60 con 0x7f6d0c0039f0 2024-09-10T21:17:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d2c000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6d2415f1b0 con 0x7f6d2415a8d0 2024-09-10T21:17:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d28c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d2415e510 0x7f6d24157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d28c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d2415e510 0x7f6d24157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54330/0 (socket says 172.21.15.16:54330) 2024-09-10T21:17:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d28c00640 1 -- 172.21.15.16:0/3554700746 learned_addr learned my addr 172.21.15.16:0/3554700746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d2a000640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6d2415a8d0 0x7f6d2415ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d29600640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d0c0039f0 0x7f6d241522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d28c00640 1 -- 172.21.15.16:0/3554700746 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6d2415a8d0 msgr2=0x7f6d2415ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d28c00640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6d2415a8d0 0x7f6d2415ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d28c00640 1 -- 172.21.15.16:0/3554700746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d0c0039f0 msgr2=0x7f6d241522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d28c00640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d0c0039f0 0x7f6d241522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d28c00640 1 -- 172.21.15.16:0/3554700746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d24157fc0 con 0x7f6d2415e510 2024-09-10T21:17:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d28c00640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d2415e510 0x7f6d24157870 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6d10003eb0 tx=0x7f6d10001740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:08.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d21600640 1 -- 172.21.15.16:0/3554700746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10015070 con 0x7f6d2415e510 2024-09-10T21:17:08.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d21600640 1 -- 172.21.15.16:0/3554700746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d10011900 con 0x7f6d2415e510 2024-09-10T21:17:08.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d2c000640 1 -- 172.21.15.16:0/3554700746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d241582a0 con 0x7f6d2415e510 2024-09-10T21:17:08.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d21600640 1 -- 172.21.15.16:0/3554700746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10011d50 con 0x7f6d2415e510 2024-09-10T21:17:08.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d2c000640 1 -- 172.21.15.16:0/3554700746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d24158860 con 0x7f6d2415e510 2024-09-10T21:17:08.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.901+0000 7f6d2c000640 1 -- 172.21.15.16:0/3554700746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d240ce610 con 0x7f6d2415e510 2024-09-10T21:17:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.905+0000 7f6d21600640 1 -- 172.21.15.16:0/3554700746 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6d100103c0 con 0x7f6d2415e510 2024-09-10T21:17:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.905+0000 7f6d21600640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d04072610 0x7f6d04074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.905+0000 7f6d21600640 1 -- 172.21.15.16:0/3554700746 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f6d10094310 con 0x7f6d2415e510 2024-09-10T21:17:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.905+0000 7f6d29600640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d04072610 0x7f6d04074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.905+0000 7f6d29600640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d04072610 0x7f6d04074ad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6d180ab000 tx=0x7f6d18019d60 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:08.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:08.909+0000 7f6d21600640 1 -- 172.21.15.16:0/3554700746 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6d1005f8d0 con 0x7f6d2415e510 2024-09-10T21:17:09.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.117+0000 7f6d2c000640 1 -- 172.21.15.16:0/3554700746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d240744a0 con 0x7f6d2415e510 2024-09-10T21:17:09.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.117+0000 7f6d21600640 1 -- 172.21.15.16:0/3554700746 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5235 (secure 0 0 0) 0x7f6d1005f1d0 con 0x7f6d2415e510 2024-09-10T21:17:09.122 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:09.122 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"btime":"2024-09-10T21:17:08:717481+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3765085271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765085271},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765085271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":7104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4198210360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":4198210360},{"type":"v1","addr":"172.21.15.16:6837","nonce":4198210360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2954517211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2954517211},{"type":"v1","addr":"172.21.15.52:6835","nonce":2954517211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:05.094283+0000","modified":"2024-09-10T21:17:08.119983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135926":{"gid":135926,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1108687782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108687782},{"type":"v1","addr":"172.21.15.52:6834","nonce":1108687782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135926,"qdb_cluster":[135926]},"id":8}]} 2024-09-10T21:17:09.122 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 71 2024-09-10T21:17:09.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.121+0000 7f6d2c000640 1 -- 172.21.15.16:0/3554700746 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d04072610 msgr2=0x7f6d04074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:09.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.121+0000 7f6d2c000640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d04072610 0x7f6d04074ad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6d180ab000 tx=0x7f6d18019d60 comp rx=0 tx=0).stop 2024-09-10T21:17:09.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.121+0000 7f6d2c000640 1 -- 172.21.15.16:0/3554700746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d2415e510 msgr2=0x7f6d24157870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:09.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.121+0000 7f6d2c000640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d2415e510 0x7f6d24157870 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6d10003eb0 tx=0x7f6d10001740 comp rx=0 tx=0).stop 2024-09-10T21:17:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.121+0000 7f6d2c000640 1 -- 172.21.15.16:0/3554700746 shutdown_connections 2024-09-10T21:17:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.121+0000 7f6d2c000640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6d04072610 0x7f6d04074ad0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.121+0000 7f6d2c000640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6d2415a8d0 0x7f6d2415ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.121+0000 7f6d2c000640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d2415e510 0x7f6d24157870 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.121+0000 7f6d2c000640 1 --2- 172.21.15.16:0/3554700746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d0c0039f0 0x7f6d241522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.121+0000 7f6d2c000640 1 -- 172.21.15.16:0/3554700746 >> 172.21.15.16:0/3554700746 conn(0x7f6d240cfb20 msgr2=0x7f6d241596f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.121+0000 7f6d2c000640 1 -- 172.21.15.16:0/3554700746 shutdown_connections 2024-09-10T21:17:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.121+0000 7f6d2c000640 1 -- 172.21.15.16:0/3554700746 wait complete. 2024-09-10T21:17:09.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:17:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 -- 172.21.15.16:0/3981116921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc003a10 msgr2=0x7f4acc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 --2- 172.21.15.16:0/3981116921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc003a10 0x7f4acc003e30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009b60 tx=0x7f4ad801d280 comp rx=0 tx=0).stop 2024-09-10T21:17:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aecc00640 1 -- 172.21.15.16:0/3981116921 reap_dead start 2024-09-10T21:17:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 -- 172.21.15.16:0/3981116921 shutdown_connections 2024-09-10T21:17:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 -- 172.21.15.16:0/3981116921 >> 172.21.15.16:0/3981116921 conn(0x7f4ae80cfb20 msgr2=0x7f4ae80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 -- 172.21.15.16:0/3981116921 shutdown_connections 2024-09-10T21:17:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 -- 172.21.15.16:0/3981116921 wait complete. 2024-09-10T21:17:09.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 Processor -- start 2024-09-10T21:17:09.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 -- start start 2024-09-10T21:17:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4acc003a10 0x7f4ae80c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae81f9040 0x7f4ae81f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:09.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ae815d560 0x7f4ae815da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae8072b20 con 0x7f4ae81f9040 2024-09-10T21:17:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae80729a0 con 0x7f4ae815d560 2024-09-10T21:17:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aeec00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae8072ca0 con 0x7f4acc003a10 2024-09-10T21:17:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4adfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae81f9040 0x7f4ae81f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4adfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae81f9040 0x7f4ae81f94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54354/0 (socket says 172.21.15.16:54354) 2024-09-10T21:17:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4adfe00640 1 -- 172.21.15.16:0/1023587570 learned_addr learned my addr 172.21.15.16:0/1023587570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4ae7e00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4acc003a10 0x7f4ae80c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4aecc00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ae815d560 0x7f4ae815da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4adfe00640 1 -- 172.21.15.16:0/1023587570 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4acc003a10 msgr2=0x7f4ae80c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4adfe00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4acc003a10 0x7f4ae80c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4adfe00640 1 -- 172.21.15.16:0/1023587570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ae815d560 msgr2=0x7f4ae815da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:09.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4adfe00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ae815d560 0x7f4ae815da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.289+0000 7f4adfe00640 1 -- 172.21.15.16:0/1023587570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae8147280 con 0x7f4ae81f9040 2024-09-10T21:17:09.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.293+0000 7f4adfe00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae81f9040 0x7f4ae81f94c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000e590 tx=0x7f4ad000ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:09.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.293+0000 7f4ae4c00640 1 -- 172.21.15.16:0/1023587570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0015070 con 0x7f4ae81f9040 2024-09-10T21:17:09.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.293+0000 7f4ae4c00640 1 -- 172.21.15.16:0/1023587570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ad0010980 con 0x7f4ae81f9040 2024-09-10T21:17:09.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.293+0000 7f4ae4c00640 1 -- 172.21.15.16:0/1023587570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0010dd0 con 0x7f4ae81f9040 2024-09-10T21:17:09.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.293+0000 7f4aeec00640 1 -- 172.21.15.16:0/1023587570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae8147550 con 0x7f4ae81f9040 2024-09-10T21:17:09.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.293+0000 7f4aeec00640 1 -- 172.21.15.16:0/1023587570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae8147b10 con 0x7f4ae81f9040 2024-09-10T21:17:09.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.293+0000 7f4aeec00640 1 -- 172.21.15.16:0/1023587570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae80b5aa0 con 0x7f4ae81f9040 2024-09-10T21:17:09.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.293+0000 7f4ae4c00640 1 -- 172.21.15.16:0/1023587570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4ad0011460 con 0x7f4ae81f9040 2024-09-10T21:17:09.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.293+0000 7f4ae4c00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ac4072610 0x7f4ac4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:09.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.293+0000 7f4ae4c00640 1 -- 172.21.15.16:0/1023587570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f4ad0093790 con 0x7f4ae81f9040 2024-09-10T21:17:09.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.293+0000 7f4ae7e00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ac4072610 0x7f4ac4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:09.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.297+0000 7f4ae7e00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ac4072610 0x7f4ac4074ad0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80ae000 tx=0x7f4acc005840 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:09.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.297+0000 7f4ae4c00640 1 -- 172.21.15.16:0/1023587570 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4ad005e810 con 0x7f4ae81f9040 2024-09-10T21:17:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.477+0000 7f4aeec00640 1 -- 172.21.15.16:0/1023587570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ae8075d40 con 0x7f4ae81f9040 2024-09-10T21:17:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.477+0000 7f4ae4c00640 1 -- 172.21.15.16:0/1023587570 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16695 (secure 0 0 0) 0x7f4ad0063240 con 0x7f4ae81f9040 2024-09-10T21:17:09.483 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:09.483 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":103,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:17:06.703527+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:17:02.802106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T21:17:04.057780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:17:09.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.481+0000 7f4aeec00640 1 -- 172.21.15.16:0/1023587570 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ac4072610 msgr2=0x7f4ac4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:09.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.481+0000 7f4aeec00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ac4072610 0x7f4ac4074ad0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80ae000 tx=0x7f4acc005840 comp rx=0 tx=0).stop 2024-09-10T21:17:09.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.481+0000 7f4aeec00640 1 -- 172.21.15.16:0/1023587570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae81f9040 msgr2=0x7f4ae81f94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.481+0000 7f4aeec00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae81f9040 0x7f4ae81f94c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000e590 tx=0x7f4ad000ea50 comp rx=0 tx=0).stop 2024-09-10T21:17:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.481+0000 7f4aeec00640 1 -- 172.21.15.16:0/1023587570 shutdown_connections 2024-09-10T21:17:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.481+0000 7f4aeec00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4ac4072610 0x7f4ac4074ad0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.481+0000 7f4aeec00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ae815d560 0x7f4ae815da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.481+0000 7f4aeec00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae81f9040 0x7f4ae81f94c0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.481+0000 7f4aeec00640 1 --2- 172.21.15.16:0/1023587570 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4acc003a10 0x7f4ae80c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.481+0000 7f4aeec00640 1 -- 172.21.15.16:0/1023587570 >> 172.21.15.16:0/1023587570 conn(0x7f4ae80cfb20 msgr2=0x7f4ae8156fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.481+0000 7f4aeec00640 1 -- 172.21.15.16:0/1023587570 shutdown_connections 2024-09-10T21:17:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.481+0000 7f4aeec00640 1 -- 172.21.15.16:0/1023587570 wait complete. 2024-09-10T21:17:09.504 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:17:09.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:17:09.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 -- 172.21.15.16:0/4014016697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b00b0050 msgr2=0x7f16b006cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:09.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 --2- 172.21.15.16:0/4014016697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b00b0050 0x7f16b006cbe0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009c80 tx=0x7f16a801cda0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 -- 172.21.15.16:0/4014016697 shutdown_connections 2024-09-10T21:17:09.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 --2- 172.21.15.16:0/4014016697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b00b0050 0x7f16b006cbe0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 --2- 172.21.15.16:0/4014016697 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f16b0152a10 0x7f16b00afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 -- 172.21.15.16:0/4014016697 >> 172.21.15.16:0/4014016697 conn(0x7f16b00d0a60 msgr2=0x7f16b00d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 -- 172.21.15.16:0/4014016697 shutdown_connections 2024-09-10T21:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 -- 172.21.15.16:0/4014016697 wait complete. 2024-09-10T21:17:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 Processor -- start 2024-09-10T21:17:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 -- start start 2024-09-10T21:17:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f16b00b0050 0x7f16b00ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16b0152630 0x7f16b014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0152a10 0x7f16b00c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16b00b5fe0 con 0x7f16b0152a10 2024-09-10T21:17:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f16b00b5e60 con 0x7f16b0152630 2024-09-10T21:17:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16b4a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f16b00b6160 con 0x7f16b00b0050 2024-09-10T21:17:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16af200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0152a10 0x7f16b00c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16af200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0152a10 0x7f16b00c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54382/0 (socket says 172.21.15.16:54382) 2024-09-10T21:17:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.649+0000 7f16af200640 1 -- 172.21.15.16:0/2312571117 learned_addr learned my addr 172.21.15.16:0/2312571117 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:09.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16a7e00640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16b0152630 0x7f16b014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:09.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16af200640 1 -- 172.21.15.16:0/2312571117 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f16b00b0050 msgr2=0x7f16b00ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:09.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16af200640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f16b00b0050 0x7f16b00ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16af200640 1 -- 172.21.15.16:0/2312571117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16b0152630 msgr2=0x7f16b014fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:09.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16af200640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16b0152630 0x7f16b014fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16af200640 1 -- 172.21.15.16:0/2312571117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b01f8290 con 0x7f16b0152a10 2024-09-10T21:17:09.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16a7e00640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16b0152630 0x7f16b014fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:09.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16af200640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0152a10 0x7f16b00c8290 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009bd0 tx=0x7f16a8009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:09.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16a6a00640 1 -- 172.21.15.16:0/2312571117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a80bb070 con 0x7f16b0152a10 2024-09-10T21:17:09.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16a6a00640 1 -- 172.21.15.16:0/2312571117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16a8004d10 con 0x7f16b0152a10 2024-09-10T21:17:09.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16a6a00640 1 -- 172.21.15.16:0/2312571117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a80b2040 con 0x7f16b0152a10 2024-09-10T21:17:09.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16b4a00640 1 -- 172.21.15.16:0/2312571117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b01f8570 con 0x7f16b0152a10 2024-09-10T21:17:09.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16b4a00640 1 -- 172.21.15.16:0/2312571117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b01f8b30 con 0x7f16b0152a10 2024-09-10T21:17:09.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16b4a00640 1 -- 172.21.15.16:0/2312571117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b006cbe0 con 0x7f16b0152a10 2024-09-10T21:17:09.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16a6a00640 1 -- 172.21.15.16:0/2312571117 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f16a80b5030 con 0x7f16b0152a10 2024-09-10T21:17:09.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16a6a00640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f168c0725e0 0x7f168c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:09.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.653+0000 7f16a6a00640 1 -- 172.21.15.16:0/2312571117 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f16a812cc60 con 0x7f16b0152a10 2024-09-10T21:17:09.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.657+0000 7f16ae800640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f168c0725e0 0x7f168c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:09.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.657+0000 7f16ae800640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f168c0725e0 0x7f168c074aa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f16a0000f70 tx=0x7f16a00079c0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:09.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.661+0000 7f16a6a00640 1 -- 172.21.15.16:0/2312571117 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f16a8016990 con 0x7f16b0152a10 2024-09-10T21:17:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.829+0000 7f16b4a00640 1 -- 172.21.15.16:0/2312571117 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16b00b1a40 con 0x7f168c0725e0 2024-09-10T21:17:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.837+0000 7f16a6a00640 1 -- 172.21.15.16:0/2312571117 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252465 (secure 0 0 0) 0x7f16b00b1a40 con 0x7f168c0725e0 2024-09-10T21:17:09.842 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:17:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.837+0000 7f16b4a00640 1 -- 172.21.15.16:0/2312571117 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f168c0725e0 msgr2=0x7f168c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.841+0000 7f16b4a00640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f168c0725e0 0x7f168c074aa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f16a0000f70 tx=0x7f16a00079c0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.841+0000 7f16b4a00640 1 -- 172.21.15.16:0/2312571117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0152a10 msgr2=0x7f16b00c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.841+0000 7f16b4a00640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0152a10 0x7f16b00c8290 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009bd0 tx=0x7f16a8009390 comp rx=0 tx=0).stop 2024-09-10T21:17:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.841+0000 7f16b4a00640 1 -- 172.21.15.16:0/2312571117 shutdown_connections 2024-09-10T21:17:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.841+0000 7f16b4a00640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f168c0725e0 0x7f168c074aa0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.841+0000 7f16b4a00640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0152a10 0x7f16b00c8290 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.841+0000 7f16b4a00640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16b0152630 0x7f16b014fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.841+0000 7f16b4a00640 1 --2- 172.21.15.16:0/2312571117 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f16b00b0050 0x7f16b00ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.841+0000 7f16b4a00640 1 -- 172.21.15.16:0/2312571117 >> 172.21.15.16:0/2312571117 conn(0x7f16b00d0a60 msgr2=0x7f16b00d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.841+0000 7f16b4a00640 1 -- 172.21.15.16:0/2312571117 shutdown_connections 2024-09-10T21:17:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:09.841+0000 7f16b4a00640 1 -- 172.21.15.16:0/2312571117 wait complete. 2024-09-10T21:17:09.864 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-10T21:17:09.549619+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277332,"kb_used_data":11176,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754697388,"statfs":{"total":773094113280,"available":772810125312,"internally_reserved":0,"allocated":11444224,"data_stored":5542644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.879329"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.715634+0000","last_change":"2024-09-10T21:17:05.723715+0000","last_active":"2024-09-10T21:17:06.715634+0000","last_peered":"2024-09-10T21:17:06.715634+0000","last_clean":"2024-09-10T21:17:06.715634+0000","last_became_active":"2024-09-10T21:17:05.723373+0000","last_became_peered":"2024-09-10T21:17:05.723373+0000","last_unstale":"2024-09-10T21:17:06.715634+0000","last_undegraded":"2024-09-10T21:17:06.715634+0000","last_fullsized":"2024-09-10T21:17:06.715634+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:30.298078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713600+0000","last_change":"2024-09-10T21:17:05.733972+0000","last_active":"2024-09-10T21:17:06.713600+0000","last_peered":"2024-09-10T21:17:06.713600+0000","last_clean":"2024-09-10T21:17:06.713600+0000","last_became_active":"2024-09-10T21:17:05.733615+0000","last_became_peered":"2024-09-10T21:17:05.733615+0000","last_unstale":"2024-09-10T21:17:06.713600+0000","last_undegraded":"2024-09-10T21:17:06.713600+0000","last_fullsized":"2024-09-10T21:17:06.713600+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:19.466316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789795+0000","last_change":"2024-09-10T21:17:05.715198+0000","last_active":"2024-09-10T21:17:06.789795+0000","last_peered":"2024-09-10T21:17:06.789795+0000","last_clean":"2024-09-10T21:17:06.789795+0000","last_became_active":"2024-09-10T21:17:05.714916+0000","last_became_peered":"2024-09-10T21:17:05.714916+0000","last_unstale":"2024-09-10T21:17:06.789795+0000","last_undegraded":"2024-09-10T21:17:06.789795+0000","last_fullsized":"2024-09-10T21:17:06.789795+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:52.527664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790024+0000","last_change":"2024-09-10T21:17:05.727547+0000","last_active":"2024-09-10T21:17:06.790024+0000","last_peered":"2024-09-10T21:17:06.790024+0000","last_clean":"2024-09-10T21:17:06.790024+0000","last_became_active":"2024-09-10T21:17:05.727277+0000","last_became_peered":"2024-09-10T21:17:05.727277+0000","last_unstale":"2024-09-10T21:17:06.790024+0000","last_undegraded":"2024-09-10T21:17:06.790024+0000","last_fullsized":"2024-09-10T21:17:06.790024+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:44.442642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.788970+0000","last_change":"2024-09-10T21:17:05.718700+0000","last_active":"2024-09-10T21:17:06.788970+0000","last_peered":"2024-09-10T21:17:06.788970+0000","last_clean":"2024-09-10T21:17:06.788970+0000","last_became_active":"2024-09-10T21:17:05.718269+0000","last_became_peered":"2024-09-10T21:17:05.718269+0000","last_unstale":"2024-09-10T21:17:06.788970+0000","last_undegraded":"2024-09-10T21:17:06.788970+0000","last_fullsized":"2024-09-10T21:17:06.788970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:52.527664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714174+0000","last_change":"2024-09-10T21:17:05.717650+0000","last_active":"2024-09-10T21:17:06.714174+0000","last_peered":"2024-09-10T21:17:06.714174+0000","last_clean":"2024-09-10T21:17:06.714174+0000","last_became_active":"2024-09-10T21:17:05.717297+0000","last_became_peered":"2024-09-10T21:17:05.717297+0000","last_unstale":"2024-09-10T21:17:06.714174+0000","last_undegraded":"2024-09-10T21:17:06.714174+0000","last_fullsized":"2024-09-10T21:17:06.714174+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:30:08.977859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725391+0000","last_change":"2024-09-10T21:17:05.714048+0000","last_active":"2024-09-10T21:17:06.725391+0000","last_peered":"2024-09-10T21:17:06.725391+0000","last_clean":"2024-09-10T21:17:06.725391+0000","last_became_active":"2024-09-10T21:17:05.713661+0000","last_became_peered":"2024-09-10T21:17:05.713661+0000","last_unstale":"2024-09-10T21:17:06.725391+0000","last_undegraded":"2024-09-10T21:17:06.725391+0000","last_fullsized":"2024-09-10T21:17:06.725391+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:22.153232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725466+0000","last_change":"2024-09-10T21:17:05.707400+0000","last_active":"2024-09-10T21:17:06.725466+0000","last_peered":"2024-09-10T21:17:06.725466+0000","last_clean":"2024-09-10T21:17:06.725466+0000","last_became_active":"2024-09-10T21:17:05.707158+0000","last_became_peered":"2024-09-10T21:17:05.707158+0000","last_unstale":"2024-09-10T21:17:06.725466+0000","last_undegraded":"2024-09-10T21:17:06.725466+0000","last_fullsized":"2024-09-10T21:17:06.725466+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:38.211010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.715661+0000","last_change":"2024-09-10T21:17:05.737208+0000","last_active":"2024-09-10T21:17:06.715661+0000","last_peered":"2024-09-10T21:17:06.715661+0000","last_clean":"2024-09-10T21:17:06.715661+0000","last_became_active":"2024-09-10T21:17:05.736688+0000","last_became_peered":"2024-09-10T21:17:05.736688+0000","last_unstale":"2024-09-10T21:17:06.715661+0000","last_undegraded":"2024-09-10T21:17:06.715661+0000","last_fullsized":"2024-09-10T21:17:06.715661+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:46.092359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.788884+0000","last_change":"2024-09-10T21:17:05.713658+0000","last_active":"2024-09-10T21:17:06.788884+0000","last_peered":"2024-09-10T21:17:06.788884+0000","last_clean":"2024-09-10T21:17:06.788884+0000","last_became_active":"2024-09-10T21:17:05.713360+0000","last_became_peered":"2024-09-10T21:17:05.713360+0000","last_unstale":"2024-09-10T21:17:06.788884+0000","last_undegraded":"2024-09-10T21:17:06.788884+0000","last_fullsized":"2024-09-10T21:17:06.788884+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:03.138945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725789+0000","last_change":"2024-09-10T21:17:05.714421+0000","last_active":"2024-09-10T21:17:06.725789+0000","last_peered":"2024-09-10T21:17:06.725789+0000","last_clean":"2024-09-10T21:17:06.725789+0000","last_became_active":"2024-09-10T21:17:05.714157+0000","last_became_peered":"2024-09-10T21:17:05.714157+0000","last_unstale":"2024-09-10T21:17:06.725789+0000","last_undegraded":"2024-09-10T21:17:06.725789+0000","last_fullsized":"2024-09-10T21:17:06.725789+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:38.483591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789008+0000","last_change":"2024-09-10T21:17:05.716690+0000","last_active":"2024-09-10T21:17:06.789008+0000","last_peered":"2024-09-10T21:17:06.789008+0000","last_clean":"2024-09-10T21:17:06.789008+0000","last_became_active":"2024-09-10T21:17:05.715825+0000","last_became_peered":"2024-09-10T21:17:05.715825+0000","last_unstale":"2024-09-10T21:17:06.789008+0000","last_undegraded":"2024-09-10T21:17:06.789008+0000","last_fullsized":"2024-09-10T21:17:06.789008+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:09.530403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713711+0000","last_change":"2024-09-10T21:17:05.709524+0000","last_active":"2024-09-10T21:17:06.713711+0000","last_peered":"2024-09-10T21:17:06.713711+0000","last_clean":"2024-09-10T21:17:06.713711+0000","last_became_active":"2024-09-10T21:17:05.709225+0000","last_became_peered":"2024-09-10T21:17:05.709225+0000","last_unstale":"2024-09-10T21:17:06.713711+0000","last_undegraded":"2024-09-10T21:17:06.713711+0000","last_fullsized":"2024-09-10T21:17:06.713711+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:33.711790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790485+0000","last_change":"2024-09-10T21:17:05.721167+0000","last_active":"2024-09-10T21:17:06.790485+0000","last_peered":"2024-09-10T21:17:06.790485+0000","last_clean":"2024-09-10T21:17:06.790485+0000","last_became_active":"2024-09-10T21:17:05.719003+0000","last_became_peered":"2024-09-10T21:17:05.719003+0000","last_unstale":"2024-09-10T21:17:06.790485+0000","last_undegraded":"2024-09-10T21:17:06.790485+0000","last_fullsized":"2024-09-10T21:17:06.790485+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:20.034188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713774+0000","last_change":"2024-09-10T21:17:05.726933+0000","last_active":"2024-09-10T21:17:06.713774+0000","last_peered":"2024-09-10T21:17:06.713774+0000","last_clean":"2024-09-10T21:17:06.713774+0000","last_became_active":"2024-09-10T21:17:05.726509+0000","last_became_peered":"2024-09-10T21:17:05.726509+0000","last_unstale":"2024-09-10T21:17:06.713774+0000","last_undegraded":"2024-09-10T21:17:06.713774+0000","last_fullsized":"2024-09-10T21:17:06.713774+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:05.782281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789392+0000","last_change":"2024-09-10T21:17:05.705180+0000","last_active":"2024-09-10T21:17:06.789392+0000","last_peered":"2024-09-10T21:17:06.789392+0000","last_clean":"2024-09-10T21:17:06.789392+0000","last_became_active":"2024-09-10T21:17:05.704804+0000","last_became_peered":"2024-09-10T21:17:05.704804+0000","last_unstale":"2024-09-10T21:17:06.789392+0000","last_undegraded":"2024-09-10T21:17:06.789392+0000","last_fullsized":"2024-09-10T21:17:06.789392+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:08.888392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790064+0000","last_change":"2024-09-10T21:17:05.721489+0000","last_active":"2024-09-10T21:17:06.790064+0000","last_peered":"2024-09-10T21:17:06.790064+0000","last_clean":"2024-09-10T21:17:06.790064+0000","last_became_active":"2024-09-10T21:17:05.721186+0000","last_became_peered":"2024-09-10T21:17:05.721186+0000","last_unstale":"2024-09-10T21:17:06.790064+0000","last_undegraded":"2024-09-10T21:17:06.790064+0000","last_fullsized":"2024-09-10T21:17:06.790064+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:06.574126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725646+0000","last_change":"2024-09-10T21:17:05.714193+0000","last_active":"2024-09-10T21:17:06.725646+0000","last_peered":"2024-09-10T21:17:06.725646+0000","last_clean":"2024-09-10T21:17:06.725646+0000","last_became_active":"2024-09-10T21:17:05.713896+0000","last_became_peered":"2024-09-10T21:17:05.713896+0000","last_unstale":"2024-09-10T21:17:06.725646+0000","last_undegraded":"2024-09-10T21:17:06.725646+0000","last_fullsized":"2024-09-10T21:17:06.725646+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:49.649607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790055+0000","last_change":"2024-09-10T21:17:05.712173+0000","last_active":"2024-09-10T21:17:06.790055+0000","last_peered":"2024-09-10T21:17:06.790055+0000","last_clean":"2024-09-10T21:17:06.790055+0000","last_became_active":"2024-09-10T21:17:05.711738+0000","last_became_peered":"2024-09-10T21:17:05.711738+0000","last_unstale":"2024-09-10T21:17:06.790055+0000","last_undegraded":"2024-09-10T21:17:06.790055+0000","last_fullsized":"2024-09-10T21:17:06.790055+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:53.783562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789630+0000","last_change":"2024-09-10T21:17:05.734524+0000","last_active":"2024-09-10T21:17:06.789630+0000","last_peered":"2024-09-10T21:17:06.789630+0000","last_clean":"2024-09-10T21:17:06.789630+0000","last_became_active":"2024-09-10T21:17:05.733939+0000","last_became_peered":"2024-09-10T21:17:05.733939+0000","last_unstale":"2024-09-10T21:17:06.789630+0000","last_undegraded":"2024-09-10T21:17:06.789630+0000","last_fullsized":"2024-09-10T21:17:06.789630+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:58.198912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725845+0000","last_change":"2024-09-10T21:17:05.736637+0000","last_active":"2024-09-10T21:17:06.725845+0000","last_peered":"2024-09-10T21:17:06.725845+0000","last_clean":"2024-09-10T21:17:06.725845+0000","last_became_active":"2024-09-10T21:17:05.736266+0000","last_became_peered":"2024-09-10T21:17:05.736266+0000","last_unstale":"2024-09-10T21:17:06.725845+0000","last_undegraded":"2024-09-10T21:17:06.725845+0000","last_fullsized":"2024-09-10T21:17:06.725845+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:53.932483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714371+0000","last_change":"2024-09-10T21:17:05.729126+0000","last_active":"2024-09-10T21:17:06.714371+0000","last_peered":"2024-09-10T21:17:06.714371+0000","last_clean":"2024-09-10T21:17:06.714371+0000","last_became_active":"2024-09-10T21:17:05.725414+0000","last_became_peered":"2024-09-10T21:17:05.725414+0000","last_unstale":"2024-09-10T21:17:06.714371+0000","last_undegraded":"2024-09-10T21:17:06.714371+0000","last_fullsized":"2024-09-10T21:17:06.714371+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:48.942190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790676+0000","last_change":"2024-09-10T21:17:05.721470+0000","last_active":"2024-09-10T21:17:06.790676+0000","last_peered":"2024-09-10T21:17:06.790676+0000","last_clean":"2024-09-10T21:17:06.790676+0000","last_became_active":"2024-09-10T21:17:05.720747+0000","last_became_peered":"2024-09-10T21:17:05.720747+0000","last_unstale":"2024-09-10T21:17:06.790676+0000","last_undegraded":"2024-09-10T21:17:06.790676+0000","last_fullsized":"2024-09-10T21:17:06.790676+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:48.842455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789318+0000","last_change":"2024-09-10T21:17:05.720308+0000","last_active":"2024-09-10T21:17:06.789318+0000","last_peered":"2024-09-10T21:17:06.789318+0000","last_clean":"2024-09-10T21:17:06.789318+0000","last_became_active":"2024-09-10T21:17:05.718512+0000","last_became_peered":"2024-09-10T21:17:05.718512+0000","last_unstale":"2024-09-10T21:17:06.789318+0000","last_undegraded":"2024-09-10T21:17:06.789318+0000","last_fullsized":"2024-09-10T21:17:06.789318+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:35.531854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725544+0000","last_change":"2024-09-10T21:17:05.706902+0000","last_active":"2024-09-10T21:17:06.725544+0000","last_peered":"2024-09-10T21:17:06.725544+0000","last_clean":"2024-09-10T21:17:06.725544+0000","last_became_active":"2024-09-10T21:17:05.706640+0000","last_became_peered":"2024-09-10T21:17:05.706640+0000","last_unstale":"2024-09-10T21:17:06.725544+0000","last_undegraded":"2024-09-10T21:17:06.725544+0000","last_fullsized":"2024-09-10T21:17:06.725544+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:53.848905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714340+0000","last_change":"2024-09-10T21:17:05.713706+0000","last_active":"2024-09-10T21:17:06.714340+0000","last_peered":"2024-09-10T21:17:06.714340+0000","last_clean":"2024-09-10T21:17:06.714340+0000","last_became_active":"2024-09-10T21:17:05.713119+0000","last_became_peered":"2024-09-10T21:17:05.713119+0000","last_unstale":"2024-09-10T21:17:06.714340+0000","last_undegraded":"2024-09-10T21:17:06.714340+0000","last_fullsized":"2024-09-10T21:17:06.714340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:39.485778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790131+0000","last_change":"2024-09-10T21:17:05.732360+0000","last_active":"2024-09-10T21:17:06.790131+0000","last_peered":"2024-09-10T21:17:06.790131+0000","last_clean":"2024-09-10T21:17:06.790131+0000","last_became_active":"2024-09-10T21:17:05.732077+0000","last_became_peered":"2024-09-10T21:17:05.732077+0000","last_unstale":"2024-09-10T21:17:06.790131+0000","last_undegraded":"2024-09-10T21:17:06.790131+0000","last_fullsized":"2024-09-10T21:17:06.790131+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:16.869649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714221+0000","last_change":"2024-09-10T21:17:05.734177+0000","last_active":"2024-09-10T21:17:06.714221+0000","last_peered":"2024-09-10T21:17:06.714221+0000","last_clean":"2024-09-10T21:17:06.714221+0000","last_became_active":"2024-09-10T21:17:05.733726+0000","last_became_peered":"2024-09-10T21:17:05.733726+0000","last_unstale":"2024-09-10T21:17:06.714221+0000","last_undegraded":"2024-09-10T21:17:06.714221+0000","last_fullsized":"2024-09-10T21:17:06.714221+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:19.607889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790804+0000","last_change":"2024-09-10T21:17:05.723406+0000","last_active":"2024-09-10T21:17:06.790804+0000","last_peered":"2024-09-10T21:17:06.790804+0000","last_clean":"2024-09-10T21:17:06.790804+0000","last_became_active":"2024-09-10T21:17:05.723069+0000","last_became_peered":"2024-09-10T21:17:05.723069+0000","last_unstale":"2024-09-10T21:17:06.790804+0000","last_undegraded":"2024-09-10T21:17:06.790804+0000","last_fullsized":"2024-09-10T21:17:06.790804+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:56.946550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725616+0000","last_change":"2024-09-10T21:17:05.719422+0000","last_active":"2024-09-10T21:17:06.725616+0000","last_peered":"2024-09-10T21:17:06.725616+0000","last_clean":"2024-09-10T21:17:06.725616+0000","last_became_active":"2024-09-10T21:17:05.719133+0000","last_became_peered":"2024-09-10T21:17:05.719133+0000","last_unstale":"2024-09-10T21:17:06.725616+0000","last_undegraded":"2024-09-10T21:17:06.725616+0000","last_fullsized":"2024-09-10T21:17:06.725616+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:35.727546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714060+0000","last_change":"2024-09-10T21:17:05.719248+0000","last_active":"2024-09-10T21:17:06.714060+0000","last_peered":"2024-09-10T21:17:06.714060+0000","last_clean":"2024-09-10T21:17:06.714060+0000","last_became_active":"2024-09-10T21:17:05.716913+0000","last_became_peered":"2024-09-10T21:17:05.716913+0000","last_unstale":"2024-09-10T21:17:06.714060+0000","last_undegraded":"2024-09-10T21:17:06.714060+0000","last_fullsized":"2024-09-10T21:17:06.714060+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:07.613681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.788834+0000","last_change":"2024-09-10T21:17:05.720448+0000","last_active":"2024-09-10T21:17:06.788834+0000","last_peered":"2024-09-10T21:17:06.788834+0000","last_clean":"2024-09-10T21:17:06.788834+0000","last_became_active":"2024-09-10T21:17:05.719681+0000","last_became_peered":"2024-09-10T21:17:05.719681+0000","last_unstale":"2024-09-10T21:17:06.788834+0000","last_undegraded":"2024-09-10T21:17:06.788834+0000","last_fullsized":"2024-09-10T21:17:06.788834+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:04.596230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.715598+0000","last_change":"2024-09-10T21:17:05.734329+0000","last_active":"2024-09-10T21:17:06.715598+0000","last_peered":"2024-09-10T21:17:06.715598+0000","last_clean":"2024-09-10T21:17:06.715598+0000","last_became_active":"2024-09-10T21:17:05.734053+0000","last_became_peered":"2024-09-10T21:17:05.734053+0000","last_unstale":"2024-09-10T21:17:06.715598+0000","last_undegraded":"2024-09-10T21:17:06.715598+0000","last_fullsized":"2024-09-10T21:17:06.715598+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:21.802637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"102'4","reported_seq":19,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790537+0000","last_change":"2024-09-10T21:17:04.695565+0000","last_active":"2024-09-10T21:17:06.790537+0000","last_peered":"2024-09-10T21:17:06.790537+0000","last_clean":"2024-09-10T21:17:06.790537+0000","last_became_active":"2024-09-10T21:17:04.695246+0000","last_became_peered":"2024-09-10T21:17:04.695246+0000","last_unstale":"2024-09-10T21:17:06.790537+0000","last_undegraded":"2024-09-10T21:17:06.790537+0000","last_fullsized":"2024-09-10T21:17:06.790537+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:09.828134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790642+0000","last_change":"2024-09-10T21:17:05.716779+0000","last_active":"2024-09-10T21:17:06.790642+0000","last_peered":"2024-09-10T21:17:06.790642+0000","last_clean":"2024-09-10T21:17:06.790642+0000","last_became_active":"2024-09-10T21:17:05.716496+0000","last_became_peered":"2024-09-10T21:17:05.716496+0000","last_unstale":"2024-09-10T21:17:06.790642+0000","last_undegraded":"2024-09-10T21:17:06.790642+0000","last_fullsized":"2024-09-10T21:17:06.790642+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:41.153323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713797+0000","last_change":"2024-09-10T21:17:04.682244+0000","last_active":"2024-09-10T21:17:06.713797+0000","last_peered":"2024-09-10T21:17:06.713797+0000","last_clean":"2024-09-10T21:17:06.713797+0000","last_became_active":"2024-09-10T21:17:04.681930+0000","last_became_peered":"2024-09-10T21:17:04.681930+0000","last_unstale":"2024-09-10T21:17:06.713797+0000","last_undegraded":"2024-09-10T21:17:06.713797+0000","last_fullsized":"2024-09-10T21:17:06.713797+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:47.085742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713970+0000","last_change":"2024-09-10T21:17:05.716573+0000","last_active":"2024-09-10T21:17:06.713970+0000","last_peered":"2024-09-10T21:17:06.713970+0000","last_clean":"2024-09-10T21:17:06.713970+0000","last_became_active":"2024-09-10T21:17:05.715945+0000","last_became_peered":"2024-09-10T21:17:05.715945+0000","last_unstale":"2024-09-10T21:17:06.713970+0000","last_undegraded":"2024-09-10T21:17:06.713970+0000","last_fullsized":"2024-09-10T21:17:06.713970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:45.800320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789759+0000","last_change":"2024-09-10T21:17:04.695949+0000","last_active":"2024-09-10T21:17:06.789759+0000","last_peered":"2024-09-10T21:17:06.789759+0000","last_clean":"2024-09-10T21:17:06.789759+0000","last_became_active":"2024-09-10T21:17:04.695583+0000","last_became_peered":"2024-09-10T21:17:04.695583+0000","last_unstale":"2024-09-10T21:17:06.789759+0000","last_undegraded":"2024-09-10T21:17:06.789759+0000","last_fullsized":"2024-09-10T21:17:06.789759+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:25.757676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789925+0000","last_change":"2024-09-10T21:17:05.717252+0000","last_active":"2024-09-10T21:17:06.789925+0000","last_peered":"2024-09-10T21:17:06.789925+0000","last_clean":"2024-09-10T21:17:06.789925+0000","last_became_active":"2024-09-10T21:17:05.716873+0000","last_became_peered":"2024-09-10T21:17:05.716873+0000","last_unstale":"2024-09-10T21:17:06.789925+0000","last_undegraded":"2024-09-10T21:17:06.789925+0000","last_fullsized":"2024-09-10T21:17:06.789925+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:09.530403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790021+0000","last_change":"2024-09-10T21:17:04.699016+0000","last_active":"2024-09-10T21:17:06.790021+0000","last_peered":"2024-09-10T21:17:06.790021+0000","last_clean":"2024-09-10T21:17:06.790021+0000","last_became_active":"2024-09-10T21:17:04.698458+0000","last_became_peered":"2024-09-10T21:17:04.698458+0000","last_unstale":"2024-09-10T21:17:06.790021+0000","last_undegraded":"2024-09-10T21:17:06.790021+0000","last_fullsized":"2024-09-10T21:17:06.790021+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:14.614359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789741+0000","last_change":"2024-09-10T21:17:05.717045+0000","last_active":"2024-09-10T21:17:06.789741+0000","last_peered":"2024-09-10T21:17:06.789741+0000","last_clean":"2024-09-10T21:17:06.789741+0000","last_became_active":"2024-09-10T21:17:05.716767+0000","last_became_peered":"2024-09-10T21:17:05.716767+0000","last_unstale":"2024-09-10T21:17:06.789741+0000","last_undegraded":"2024-09-10T21:17:06.789741+0000","last_fullsized":"2024-09-10T21:17:06.789741+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:53.452886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789606+0000","last_change":"2024-09-10T21:17:04.682093+0000","last_active":"2024-09-10T21:17:06.789606+0000","last_peered":"2024-09-10T21:17:06.789606+0000","last_clean":"2024-09-10T21:17:06.789606+0000","last_became_active":"2024-09-10T21:17:04.681799+0000","last_became_peered":"2024-09-10T21:17:04.681799+0000","last_unstale":"2024-09-10T21:17:06.789606+0000","last_undegraded":"2024-09-10T21:17:06.789606+0000","last_fullsized":"2024-09-10T21:17:06.789606+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:07.226948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725758+0000","last_change":"2024-09-10T21:17:05.704156+0000","last_active":"2024-09-10T21:17:06.725758+0000","last_peered":"2024-09-10T21:17:06.725758+0000","last_clean":"2024-09-10T21:17:06.725758+0000","last_became_active":"2024-09-10T21:17:05.703648+0000","last_became_peered":"2024-09-10T21:17:05.703648+0000","last_unstale":"2024-09-10T21:17:06.725758+0000","last_undegraded":"2024-09-10T21:17:06.725758+0000","last_fullsized":"2024-09-10T21:17:06.725758+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:57.792968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714528+0000","last_change":"2024-09-10T21:17:04.699682+0000","last_active":"2024-09-10T21:17:06.714528+0000","last_peered":"2024-09-10T21:17:06.714528+0000","last_clean":"2024-09-10T21:17:06.714528+0000","last_became_active":"2024-09-10T21:17:04.699406+0000","last_became_peered":"2024-09-10T21:17:04.699406+0000","last_unstale":"2024-09-10T21:17:06.714528+0000","last_undegraded":"2024-09-10T21:17:06.714528+0000","last_fullsized":"2024-09-10T21:17:06.714528+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:39.848996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789058+0000","last_change":"2024-09-10T21:17:05.720729+0000","last_active":"2024-09-10T21:17:06.789058+0000","last_peered":"2024-09-10T21:17:06.789058+0000","last_clean":"2024-09-10T21:17:06.789058+0000","last_became_active":"2024-09-10T21:17:05.720084+0000","last_became_peered":"2024-09-10T21:17:05.720084+0000","last_unstale":"2024-09-10T21:17:06.789058+0000","last_undegraded":"2024-09-10T21:17:06.789058+0000","last_fullsized":"2024-09-10T21:17:06.789058+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:37.107915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725459+0000","last_change":"2024-09-10T21:17:04.693503+0000","last_active":"2024-09-10T21:17:06.725459+0000","last_peered":"2024-09-10T21:17:06.725459+0000","last_clean":"2024-09-10T21:17:06.725459+0000","last_became_active":"2024-09-10T21:17:04.693071+0000","last_became_peered":"2024-09-10T21:17:04.693071+0000","last_unstale":"2024-09-10T21:17:06.725459+0000","last_undegraded":"2024-09-10T21:17:06.725459+0000","last_fullsized":"2024-09-10T21:17:06.725459+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:03.176796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725397+0000","last_change":"2024-09-10T21:17:05.705534+0000","last_active":"2024-09-10T21:17:06.725397+0000","last_peered":"2024-09-10T21:17:06.725397+0000","last_clean":"2024-09-10T21:17:06.725397+0000","last_became_active":"2024-09-10T21:17:05.705226+0000","last_became_peered":"2024-09-10T21:17:05.705226+0000","last_unstale":"2024-09-10T21:17:06.725397+0000","last_undegraded":"2024-09-10T21:17:06.725397+0000","last_fullsized":"2024-09-10T21:17:06.725397+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:50.189589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789169+0000","last_change":"2024-09-10T21:17:04.698131+0000","last_active":"2024-09-10T21:17:06.789169+0000","last_peered":"2024-09-10T21:17:06.789169+0000","last_clean":"2024-09-10T21:17:06.789169+0000","last_became_active":"2024-09-10T21:17:04.697827+0000","last_became_peered":"2024-09-10T21:17:04.697827+0000","last_unstale":"2024-09-10T21:17:06.789169+0000","last_undegraded":"2024-09-10T21:17:06.789169+0000","last_fullsized":"2024-09-10T21:17:06.789169+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:14.346425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713807+0000","last_change":"2024-09-10T21:17:05.713532+0000","last_active":"2024-09-10T21:17:06.713807+0000","last_peered":"2024-09-10T21:17:06.713807+0000","last_clean":"2024-09-10T21:17:06.713807+0000","last_became_active":"2024-09-10T21:17:05.713097+0000","last_became_peered":"2024-09-10T21:17:05.713097+0000","last_unstale":"2024-09-10T21:17:06.713807+0000","last_undegraded":"2024-09-10T21:17:06.713807+0000","last_fullsized":"2024-09-10T21:17:06.713807+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:58.222857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789927+0000","last_change":"2024-09-10T21:17:04.698054+0000","last_active":"2024-09-10T21:17:06.789927+0000","last_peered":"2024-09-10T21:17:06.789927+0000","last_clean":"2024-09-10T21:17:06.789927+0000","last_became_active":"2024-09-10T21:17:04.697481+0000","last_became_peered":"2024-09-10T21:17:04.697481+0000","last_unstale":"2024-09-10T21:17:06.789927+0000","last_undegraded":"2024-09-10T21:17:06.789927+0000","last_fullsized":"2024-09-10T21:17:06.789927+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:49.932598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713901+0000","last_change":"2024-09-10T21:17:05.709326+0000","last_active":"2024-09-10T21:17:06.713901+0000","last_peered":"2024-09-10T21:17:06.713901+0000","last_clean":"2024-09-10T21:17:06.713901+0000","last_became_active":"2024-09-10T21:17:05.708989+0000","last_became_peered":"2024-09-10T21:17:05.708989+0000","last_unstale":"2024-09-10T21:17:06.713901+0000","last_undegraded":"2024-09-10T21:17:06.713901+0000","last_fullsized":"2024-09-10T21:17:06.713901+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:26.915328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.788840+0000","last_change":"2024-09-10T21:17:04.696842+0000","last_active":"2024-09-10T21:17:06.788840+0000","last_peered":"2024-09-10T21:17:06.788840+0000","last_clean":"2024-09-10T21:17:06.788840+0000","last_became_active":"2024-09-10T21:17:04.696540+0000","last_became_peered":"2024-09-10T21:17:04.696540+0000","last_unstale":"2024-09-10T21:17:06.788840+0000","last_undegraded":"2024-09-10T21:17:06.788840+0000","last_fullsized":"2024-09-10T21:17:06.788840+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:06.998463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714266+0000","last_change":"2024-09-10T21:17:05.717559+0000","last_active":"2024-09-10T21:17:06.714266+0000","last_peered":"2024-09-10T21:17:06.714266+0000","last_clean":"2024-09-10T21:17:06.714266+0000","last_became_active":"2024-09-10T21:17:05.717208+0000","last_became_peered":"2024-09-10T21:17:05.717208+0000","last_unstale":"2024-09-10T21:17:06.714266+0000","last_undegraded":"2024-09-10T21:17:06.714266+0000","last_fullsized":"2024-09-10T21:17:06.714266+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:47.158495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"102'3","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789286+0000","last_change":"2024-09-10T21:17:04.703239+0000","last_active":"2024-09-10T21:17:06.789286+0000","last_peered":"2024-09-10T21:17:06.789286+0000","last_clean":"2024-09-10T21:17:06.789286+0000","last_became_active":"2024-09-10T21:17:04.702940+0000","last_became_peered":"2024-09-10T21:17:04.702940+0000","last_unstale":"2024-09-10T21:17:06.789286+0000","last_undegraded":"2024-09-10T21:17:06.789286+0000","last_fullsized":"2024-09-10T21:17:06.789286+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:57.976080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.726002+0000","last_change":"2024-09-10T21:17:05.715752+0000","last_active":"2024-09-10T21:17:06.726002+0000","last_peered":"2024-09-10T21:17:06.726002+0000","last_clean":"2024-09-10T21:17:06.726002+0000","last_became_active":"2024-09-10T21:17:05.715497+0000","last_became_peered":"2024-09-10T21:17:05.715497+0000","last_unstale":"2024-09-10T21:17:06.726002+0000","last_undegraded":"2024-09-10T21:17:06.726002+0000","last_fullsized":"2024-09-10T21:17:06.726002+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:19.793726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"102'6","reported_seq":21,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790131+0000","last_change":"2024-09-10T21:17:04.698722+0000","last_active":"2024-09-10T21:17:06.790131+0000","last_peered":"2024-09-10T21:17:06.790131+0000","last_clean":"2024-09-10T21:17:06.790131+0000","last_became_active":"2024-09-10T21:17:04.698079+0000","last_became_peered":"2024-09-10T21:17:04.698079+0000","last_unstale":"2024-09-10T21:17:06.790131+0000","last_undegraded":"2024-09-10T21:17:06.790131+0000","last_fullsized":"2024-09-10T21:17:06.790131+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:15.763202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789323+0000","last_change":"2024-09-10T21:17:05.706059+0000","last_active":"2024-09-10T21:17:06.789323+0000","last_peered":"2024-09-10T21:17:06.789323+0000","last_clean":"2024-09-10T21:17:06.789323+0000","last_became_active":"2024-09-10T21:17:05.704630+0000","last_became_peered":"2024-09-10T21:17:05.704630+0000","last_unstale":"2024-09-10T21:17:06.789323+0000","last_undegraded":"2024-09-10T21:17:06.789323+0000","last_fullsized":"2024-09-10T21:17:06.789323+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:06.574126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789109+0000","last_change":"2024-09-10T21:17:04.694226+0000","last_active":"2024-09-10T21:17:06.789109+0000","last_peered":"2024-09-10T21:17:06.789109+0000","last_clean":"2024-09-10T21:17:06.789109+0000","last_became_active":"2024-09-10T21:17:04.693900+0000","last_became_peered":"2024-09-10T21:17:04.693900+0000","last_unstale":"2024-09-10T21:17:06.789109+0000","last_undegraded":"2024-09-10T21:17:06.789109+0000","last_fullsized":"2024-09-10T21:17:06.789109+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:02.475825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790362+0000","last_change":"2024-09-10T21:17:05.733006+0000","last_active":"2024-09-10T21:17:06.790362+0000","last_peered":"2024-09-10T21:17:06.790362+0000","last_clean":"2024-09-10T21:17:06.790362+0000","last_became_active":"2024-09-10T21:17:05.731901+0000","last_became_peered":"2024-09-10T21:17:05.731901+0000","last_unstale":"2024-09-10T21:17:06.790362+0000","last_undegraded":"2024-09-10T21:17:06.790362+0000","last_fullsized":"2024-09-10T21:17:06.790362+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:31.110568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789809+0000","last_change":"2024-09-10T21:17:04.692932+0000","last_active":"2024-09-10T21:17:06.789809+0000","last_peered":"2024-09-10T21:17:06.789809+0000","last_clean":"2024-09-10T21:17:06.789809+0000","last_became_active":"2024-09-10T21:17:04.692669+0000","last_became_peered":"2024-09-10T21:17:04.692669+0000","last_unstale":"2024-09-10T21:17:06.789809+0000","last_undegraded":"2024-09-10T21:17:06.789809+0000","last_fullsized":"2024-09-10T21:17:06.789809+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:36:46.857369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789207+0000","last_change":"2024-09-10T21:17:05.721097+0000","last_active":"2024-09-10T21:17:06.789207+0000","last_peered":"2024-09-10T21:17:06.789207+0000","last_clean":"2024-09-10T21:17:06.789207+0000","last_became_active":"2024-09-10T21:17:05.720810+0000","last_became_peered":"2024-09-10T21:17:05.720810+0000","last_unstale":"2024-09-10T21:17:06.789207+0000","last_undegraded":"2024-09-10T21:17:06.789207+0000","last_fullsized":"2024-09-10T21:17:06.789207+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:25.693651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"102'8","reported_seq":23,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713499+0000","last_change":"2024-09-10T21:17:04.696549+0000","last_active":"2024-09-10T21:17:06.713499+0000","last_peered":"2024-09-10T21:17:06.713499+0000","last_clean":"2024-09-10T21:17:06.713499+0000","last_became_active":"2024-09-10T21:17:04.696267+0000","last_became_peered":"2024-09-10T21:17:04.696267+0000","last_unstale":"2024-09-10T21:17:06.713499+0000","last_undegraded":"2024-09-10T21:17:06.713499+0000","last_fullsized":"2024-09-10T21:17:06.713499+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:31.579988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.715589+0000","last_change":"2024-09-10T21:17:05.722933+0000","last_active":"2024-09-10T21:17:06.715589+0000","last_peered":"2024-09-10T21:17:06.715589+0000","last_clean":"2024-09-10T21:17:06.715589+0000","last_became_active":"2024-09-10T21:17:05.722661+0000","last_became_peered":"2024-09-10T21:17:05.722661+0000","last_unstale":"2024-09-10T21:17:06.715589+0000","last_undegraded":"2024-09-10T21:17:06.715589+0000","last_fullsized":"2024-09-10T21:17:06.715589+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:44.442642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714136+0000","last_change":"2024-09-10T21:17:04.684959+0000","last_active":"2024-09-10T21:17:06.714136+0000","last_peered":"2024-09-10T21:17:06.714136+0000","last_clean":"2024-09-10T21:17:06.714136+0000","last_became_active":"2024-09-10T21:17:04.684651+0000","last_became_peered":"2024-09-10T21:17:04.684651+0000","last_unstale":"2024-09-10T21:17:06.714136+0000","last_undegraded":"2024-09-10T21:17:06.714136+0000","last_fullsized":"2024-09-10T21:17:06.714136+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:47.179182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.715851+0000","last_change":"2024-09-10T21:17:05.736938+0000","last_active":"2024-09-10T21:17:06.715851+0000","last_peered":"2024-09-10T21:17:06.715851+0000","last_clean":"2024-09-10T21:17:06.715851+0000","last_became_active":"2024-09-10T21:17:05.736336+0000","last_became_peered":"2024-09-10T21:17:05.736336+0000","last_unstale":"2024-09-10T21:17:06.715851+0000","last_undegraded":"2024-09-10T21:17:06.715851+0000","last_fullsized":"2024-09-10T21:17:06.715851+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:34.951525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789558+0000","last_change":"2024-09-10T21:17:04.694123+0000","last_active":"2024-09-10T21:17:06.789558+0000","last_peered":"2024-09-10T21:17:06.789558+0000","last_clean":"2024-09-10T21:17:06.789558+0000","last_became_active":"2024-09-10T21:17:04.693785+0000","last_became_peered":"2024-09-10T21:17:04.693785+0000","last_unstale":"2024-09-10T21:17:06.789558+0000","last_undegraded":"2024-09-10T21:17:06.789558+0000","last_fullsized":"2024-09-10T21:17:06.789558+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:35.975151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725553+0000","last_change":"2024-09-10T21:17:05.713293+0000","last_active":"2024-09-10T21:17:06.725553+0000","last_peered":"2024-09-10T21:17:06.725553+0000","last_clean":"2024-09-10T21:17:06.725553+0000","last_became_active":"2024-09-10T21:17:05.713015+0000","last_became_peered":"2024-09-10T21:17:05.713015+0000","last_unstale":"2024-09-10T21:17:06.725553+0000","last_undegraded":"2024-09-10T21:17:06.725553+0000","last_fullsized":"2024-09-10T21:17:06.725553+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:53.163112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790678+0000","last_change":"2024-09-10T21:17:04.703676+0000","last_active":"2024-09-10T21:17:06.790678+0000","last_peered":"2024-09-10T21:17:06.790678+0000","last_clean":"2024-09-10T21:17:06.790678+0000","last_became_active":"2024-09-10T21:17:04.703280+0000","last_became_peered":"2024-09-10T21:17:04.703280+0000","last_unstale":"2024-09-10T21:17:06.790678+0000","last_undegraded":"2024-09-10T21:17:06.790678+0000","last_fullsized":"2024-09-10T21:17:06.790678+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:39.837301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790746+0000","last_change":"2024-09-10T21:17:05.720913+0000","last_active":"2024-09-10T21:17:06.790746+0000","last_peered":"2024-09-10T21:17:06.790746+0000","last_clean":"2024-09-10T21:17:06.790746+0000","last_became_active":"2024-09-10T21:17:05.720633+0000","last_became_peered":"2024-09-10T21:17:05.720633+0000","last_unstale":"2024-09-10T21:17:06.790746+0000","last_undegraded":"2024-09-10T21:17:06.790746+0000","last_fullsized":"2024-09-10T21:17:06.790746+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:36:16.108624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790629+0000","last_change":"2024-09-10T21:17:04.703781+0000","last_active":"2024-09-10T21:17:06.790629+0000","last_peered":"2024-09-10T21:17:06.790629+0000","last_clean":"2024-09-10T21:17:06.790629+0000","last_became_active":"2024-09-10T21:17:04.703444+0000","last_became_peered":"2024-09-10T21:17:04.703444+0000","last_unstale":"2024-09-10T21:17:06.790629+0000","last_undegraded":"2024-09-10T21:17:06.790629+0000","last_fullsized":"2024-09-10T21:17:06.790629+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:45.230767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789949+0000","last_change":"2024-09-10T21:17:05.711925+0000","last_active":"2024-09-10T21:17:06.789949+0000","last_peered":"2024-09-10T21:17:06.789949+0000","last_clean":"2024-09-10T21:17:06.789949+0000","last_became_active":"2024-09-10T21:17:05.711624+0000","last_became_peered":"2024-09-10T21:17:05.711624+0000","last_unstale":"2024-09-10T21:17:06.789949+0000","last_undegraded":"2024-09-10T21:17:06.789949+0000","last_fullsized":"2024-09-10T21:17:06.789949+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:37.107915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789176+0000","last_change":"2024-09-10T21:17:04.696347+0000","last_active":"2024-09-10T21:17:06.789176+0000","last_peered":"2024-09-10T21:17:06.789176+0000","last_clean":"2024-09-10T21:17:06.789176+0000","last_became_active":"2024-09-10T21:17:04.695362+0000","last_became_peered":"2024-09-10T21:17:04.695362+0000","last_unstale":"2024-09-10T21:17:06.789176+0000","last_undegraded":"2024-09-10T21:17:06.789176+0000","last_fullsized":"2024-09-10T21:17:06.789176+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:04.933492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714898+0000","last_change":"2024-09-10T21:17:05.715049+0000","last_active":"2024-09-10T21:17:06.714898+0000","last_peered":"2024-09-10T21:17:06.714898+0000","last_clean":"2024-09-10T21:17:06.714898+0000","last_became_active":"2024-09-10T21:17:05.714759+0000","last_became_peered":"2024-09-10T21:17:05.714759+0000","last_unstale":"2024-09-10T21:17:06.714898+0000","last_undegraded":"2024-09-10T21:17:06.714898+0000","last_fullsized":"2024-09-10T21:17:06.714898+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:17.265724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714565+0000","last_change":"2024-09-10T21:17:04.702725+0000","last_active":"2024-09-10T21:17:06.714565+0000","last_peered":"2024-09-10T21:17:06.714565+0000","last_clean":"2024-09-10T21:17:06.714565+0000","last_became_active":"2024-09-10T21:17:04.702394+0000","last_became_peered":"2024-09-10T21:17:04.702394+0000","last_unstale":"2024-09-10T21:17:06.714565+0000","last_undegraded":"2024-09-10T21:17:06.714565+0000","last_fullsized":"2024-09-10T21:17:06.714565+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:52.701234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790293+0000","last_change":"2024-09-10T21:17:05.725270+0000","last_active":"2024-09-10T21:17:06.790293+0000","last_peered":"2024-09-10T21:17:06.790293+0000","last_clean":"2024-09-10T21:17:06.790293+0000","last_became_active":"2024-09-10T21:17:05.724923+0000","last_became_peered":"2024-09-10T21:17:05.724923+0000","last_unstale":"2024-09-10T21:17:06.790293+0000","last_undegraded":"2024-09-10T21:17:06.790293+0000","last_fullsized":"2024-09-10T21:17:06.790293+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:10.496133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789360+0000","last_change":"2024-09-10T21:17:04.702256+0000","last_active":"2024-09-10T21:17:06.789360+0000","last_peered":"2024-09-10T21:17:06.789360+0000","last_clean":"2024-09-10T21:17:06.789360+0000","last_became_active":"2024-09-10T21:17:04.701928+0000","last_became_peered":"2024-09-10T21:17:04.701928+0000","last_unstale":"2024-09-10T21:17:06.789360+0000","last_undegraded":"2024-09-10T21:17:06.789360+0000","last_fullsized":"2024-09-10T21:17:06.789360+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:58.395259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.788713+0000","last_change":"2024-09-10T21:17:05.728741+0000","last_active":"2024-09-10T21:17:06.788713+0000","last_peered":"2024-09-10T21:17:06.788713+0000","last_clean":"2024-09-10T21:17:06.788713+0000","last_became_active":"2024-09-10T21:17:05.728272+0000","last_became_peered":"2024-09-10T21:17:05.728272+0000","last_unstale":"2024-09-10T21:17:06.788713+0000","last_undegraded":"2024-09-10T21:17:06.788713+0000","last_fullsized":"2024-09-10T21:17:06.788713+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:01.867268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789309+0000","last_change":"2024-09-10T21:17:04.691733+0000","last_active":"2024-09-10T21:17:06.789309+0000","last_peered":"2024-09-10T21:17:06.789309+0000","last_clean":"2024-09-10T21:17:06.789309+0000","last_became_active":"2024-09-10T21:17:04.691416+0000","last_became_peered":"2024-09-10T21:17:04.691416+0000","last_unstale":"2024-09-10T21:17:06.789309+0000","last_undegraded":"2024-09-10T21:17:06.789309+0000","last_fullsized":"2024-09-10T21:17:06.789309+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:32.884545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790432+0000","last_change":"2024-09-10T21:17:05.726304+0000","last_active":"2024-09-10T21:17:06.790432+0000","last_peered":"2024-09-10T21:17:06.790432+0000","last_clean":"2024-09-10T21:17:06.790432+0000","last_became_active":"2024-09-10T21:17:05.725913+0000","last_became_peered":"2024-09-10T21:17:05.725913+0000","last_unstale":"2024-09-10T21:17:06.790432+0000","last_undegraded":"2024-09-10T21:17:06.790432+0000","last_fullsized":"2024-09-10T21:17:06.790432+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:48.032460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789998+0000","last_change":"2024-09-10T21:17:04.697705+0000","last_active":"2024-09-10T21:17:06.789998+0000","last_peered":"2024-09-10T21:17:06.789998+0000","last_clean":"2024-09-10T21:17:06.789998+0000","last_became_active":"2024-09-10T21:17:04.697166+0000","last_became_peered":"2024-09-10T21:17:04.697166+0000","last_unstale":"2024-09-10T21:17:06.789998+0000","last_undegraded":"2024-09-10T21:17:06.789998+0000","last_fullsized":"2024-09-10T21:17:06.789998+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:09:54.646678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714058+0000","last_change":"2024-09-10T21:17:05.713757+0000","last_active":"2024-09-10T21:17:06.714058+0000","last_peered":"2024-09-10T21:17:06.714058+0000","last_clean":"2024-09-10T21:17:06.714058+0000","last_became_active":"2024-09-10T21:17:05.713071+0000","last_became_peered":"2024-09-10T21:17:05.713071+0000","last_unstale":"2024-09-10T21:17:06.714058+0000","last_undegraded":"2024-09-10T21:17:06.714058+0000","last_fullsized":"2024-09-10T21:17:06.714058+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:45.777851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"103'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725726+0000","last_change":"2024-09-10T21:17:04.688907+0000","last_active":"2024-09-10T21:17:06.725726+0000","last_peered":"2024-09-10T21:17:06.725726+0000","last_clean":"2024-09-10T21:17:06.725726+0000","last_became_active":"2024-09-10T21:17:04.688614+0000","last_became_peered":"2024-09-10T21:17:04.688614+0000","last_unstale":"2024-09-10T21:17:06.725726+0000","last_undegraded":"2024-09-10T21:17:06.725726+0000","last_fullsized":"2024-09-10T21:17:06.725726+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:02.464966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.715053+0000","last_change":"2024-09-10T21:17:05.724980+0000","last_active":"2024-09-10T21:17:06.715053+0000","last_peered":"2024-09-10T21:17:06.715053+0000","last_clean":"2024-09-10T21:17:06.715053+0000","last_became_active":"2024-09-10T21:17:05.724702+0000","last_became_peered":"2024-09-10T21:17:05.724702+0000","last_unstale":"2024-09-10T21:17:06.715053+0000","last_undegraded":"2024-09-10T21:17:06.715053+0000","last_fullsized":"2024-09-10T21:17:06.715053+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:55.403243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789931+0000","last_change":"2024-09-10T21:17:04.686893+0000","last_active":"2024-09-10T21:17:06.789931+0000","last_peered":"2024-09-10T21:17:06.789931+0000","last_clean":"2024-09-10T21:17:06.789931+0000","last_became_active":"2024-09-10T21:17:04.686614+0000","last_became_peered":"2024-09-10T21:17:04.686614+0000","last_unstale":"2024-09-10T21:17:06.789931+0000","last_undegraded":"2024-09-10T21:17:06.789931+0000","last_fullsized":"2024-09-10T21:17:06.789931+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:54.901346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790613+0000","last_change":"2024-09-10T21:17:05.736758+0000","last_active":"2024-09-10T21:17:06.790613+0000","last_peered":"2024-09-10T21:17:06.790613+0000","last_clean":"2024-09-10T21:17:06.790613+0000","last_became_active":"2024-09-10T21:17:05.736377+0000","last_became_peered":"2024-09-10T21:17:05.736377+0000","last_unstale":"2024-09-10T21:17:06.790613+0000","last_undegraded":"2024-09-10T21:17:06.790613+0000","last_fullsized":"2024-09-10T21:17:06.790613+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:37.932179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790178+0000","last_change":"2024-09-10T21:17:04.687679+0000","last_active":"2024-09-10T21:17:06.790178+0000","last_peered":"2024-09-10T21:17:06.790178+0000","last_clean":"2024-09-10T21:17:06.790178+0000","last_became_active":"2024-09-10T21:17:04.687377+0000","last_became_peered":"2024-09-10T21:17:04.687377+0000","last_unstale":"2024-09-10T21:17:06.790178+0000","last_undegraded":"2024-09-10T21:17:06.790178+0000","last_fullsized":"2024-09-10T21:17:06.790178+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:55.233907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789705+0000","last_change":"2024-09-10T21:17:05.719517+0000","last_active":"2024-09-10T21:17:06.789705+0000","last_peered":"2024-09-10T21:17:06.789705+0000","last_clean":"2024-09-10T21:17:06.789705+0000","last_became_active":"2024-09-10T21:17:05.719246+0000","last_became_peered":"2024-09-10T21:17:05.719246+0000","last_unstale":"2024-09-10T21:17:06.789705+0000","last_undegraded":"2024-09-10T21:17:06.789705+0000","last_fullsized":"2024-09-10T21:17:06.789705+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:03.138945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725801+0000","last_change":"2024-09-10T21:17:04.690559+0000","last_active":"2024-09-10T21:17:06.725801+0000","last_peered":"2024-09-10T21:17:06.725801+0000","last_clean":"2024-09-10T21:17:06.725801+0000","last_became_active":"2024-09-10T21:17:04.690280+0000","last_became_peered":"2024-09-10T21:17:04.690280+0000","last_unstale":"2024-09-10T21:17:06.725801+0000","last_undegraded":"2024-09-10T21:17:06.725801+0000","last_fullsized":"2024-09-10T21:17:06.725801+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:18.613917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790886+0000","last_change":"2024-09-10T21:17:05.725184+0000","last_active":"2024-09-10T21:17:06.790886+0000","last_peered":"2024-09-10T21:17:06.790886+0000","last_clean":"2024-09-10T21:17:06.790886+0000","last_became_active":"2024-09-10T21:17:05.724849+0000","last_became_peered":"2024-09-10T21:17:05.724849+0000","last_unstale":"2024-09-10T21:17:06.790886+0000","last_undegraded":"2024-09-10T21:17:06.790886+0000","last_fullsized":"2024-09-10T21:17:06.790886+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:22.263637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790563+0000","last_change":"2024-09-10T21:17:04.705593+0000","last_active":"2024-09-10T21:17:06.790563+0000","last_peered":"2024-09-10T21:17:06.790563+0000","last_clean":"2024-09-10T21:17:06.790563+0000","last_became_active":"2024-09-10T21:17:04.705280+0000","last_became_peered":"2024-09-10T21:17:04.705280+0000","last_unstale":"2024-09-10T21:17:06.790563+0000","last_undegraded":"2024-09-10T21:17:06.790563+0000","last_fullsized":"2024-09-10T21:17:06.790563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:24.189910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790383+0000","last_change":"2024-09-10T21:17:05.717785+0000","last_active":"2024-09-10T21:17:06.790383+0000","last_peered":"2024-09-10T21:17:06.790383+0000","last_clean":"2024-09-10T21:17:06.790383+0000","last_became_active":"2024-09-10T21:17:05.717478+0000","last_became_peered":"2024-09-10T21:17:05.717478+0000","last_unstale":"2024-09-10T21:17:06.790383+0000","last_undegraded":"2024-09-10T21:17:06.790383+0000","last_fullsized":"2024-09-10T21:17:06.790383+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:24.730712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714390+0000","last_change":"2024-09-10T21:17:04.698439+0000","last_active":"2024-09-10T21:17:06.714390+0000","last_peered":"2024-09-10T21:17:06.714390+0000","last_clean":"2024-09-10T21:17:06.714390+0000","last_became_active":"2024-09-10T21:17:04.697954+0000","last_became_peered":"2024-09-10T21:17:04.697954+0000","last_unstale":"2024-09-10T21:17:06.714390+0000","last_undegraded":"2024-09-10T21:17:06.714390+0000","last_fullsized":"2024-09-10T21:17:06.714390+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:58.728826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713776+0000","last_change":"2024-09-10T21:17:05.724864+0000","last_active":"2024-09-10T21:17:06.713776+0000","last_peered":"2024-09-10T21:17:06.713776+0000","last_clean":"2024-09-10T21:17:06.713776+0000","last_became_active":"2024-09-10T21:17:05.724517+0000","last_became_peered":"2024-09-10T21:17:05.724517+0000","last_unstale":"2024-09-10T21:17:06.713776+0000","last_undegraded":"2024-09-10T21:17:06.713776+0000","last_fullsized":"2024-09-10T21:17:06.713776+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:08.115164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713654+0000","last_change":"2024-09-10T21:17:04.683757+0000","last_active":"2024-09-10T21:17:06.713654+0000","last_peered":"2024-09-10T21:17:06.713654+0000","last_clean":"2024-09-10T21:17:06.713654+0000","last_became_active":"2024-09-10T21:17:04.683246+0000","last_became_peered":"2024-09-10T21:17:04.683246+0000","last_unstale":"2024-09-10T21:17:06.713654+0000","last_undegraded":"2024-09-10T21:17:06.713654+0000","last_fullsized":"2024-09-10T21:17:06.713654+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:20.795469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789230+0000","last_change":"2024-09-10T21:17:05.722935+0000","last_active":"2024-09-10T21:17:06.789230+0000","last_peered":"2024-09-10T21:17:06.789230+0000","last_clean":"2024-09-10T21:17:06.789230+0000","last_became_active":"2024-09-10T21:17:05.718877+0000","last_became_peered":"2024-09-10T21:17:05.718877+0000","last_unstale":"2024-09-10T21:17:06.789230+0000","last_undegraded":"2024-09-10T21:17:06.789230+0000","last_fullsized":"2024-09-10T21:17:06.789230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:07.613681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713332+0000","last_change":"2024-09-10T21:17:04.689707+0000","last_active":"2024-09-10T21:17:06.713332+0000","last_peered":"2024-09-10T21:17:06.713332+0000","last_clean":"2024-09-10T21:17:06.713332+0000","last_became_active":"2024-09-10T21:17:04.689349+0000","last_became_peered":"2024-09-10T21:17:04.689349+0000","last_unstale":"2024-09-10T21:17:06.713332+0000","last_undegraded":"2024-09-10T21:17:06.713332+0000","last_fullsized":"2024-09-10T21:17:06.713332+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:55.356058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":255,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789446+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:17:06.789446+0000","last_peered":"2024-09-10T21:17:06.789446+0000","last_clean":"2024-09-10T21:17:06.789446+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:17:06.789446+0000","last_undegraded":"2024-09-10T21:17:06.789446+0000","last_fullsized":"2024-09-10T21:17:06.789446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509525,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36592,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335248,"statfs":{"total":96636764160,"available":96599293952,"internally_reserved":0,"allocated":1273856,"data_stored":544636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509527,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1875968,"data_stored":1135096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509525,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36608,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335232,"statfs":{"total":96636764160,"available":96599277568,"internally_reserved":0,"allocated":1290240,"data_stored":546338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509524,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1273856,"data_stored":544636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509526,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1286144,"data_stored":545274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509525,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1282048,"data_stored":545288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542229,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1294336,"data_stored":546372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542228,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37172,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334668,"statfs":{"total":96636764160,"available":96598700032,"internally_reserved":0,"allocated":1867776,"data_stored":1135004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:17:09.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:17:10.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.053+0000 7fa67d400640 1 -- 172.21.15.16:0/3047636553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6780d0890 msgr2=0x7fa6780b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:10.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.053+0000 7fa67d400640 1 --2- 172.21.15.16:0/3047636553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6780d0890 0x7fa6780b01f0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa670009c80 tx=0x7fa67001cda0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.053+0000 7fa67d400640 1 -- 172.21.15.16:0/3047636553 shutdown_connections 2024-09-10T21:17:10.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.053+0000 7fa67d400640 1 --2- 172.21.15.16:0/3047636553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6780d0890 0x7fa6780b01f0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.053+0000 7fa67d400640 1 -- 172.21.15.16:0/3047636553 >> 172.21.15.16:0/3047636553 conn(0x7fa6780d1d80 msgr2=0x7fa6780d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:10.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.053+0000 7fa67d400640 1 -- 172.21.15.16:0/3047636553 shutdown_connections 2024-09-10T21:17:10.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.053+0000 7fa67d400640 1 -- 172.21.15.16:0/3047636553 wait complete. 2024-09-10T21:17:10.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.053+0000 7fa67d400640 1 Processor -- start 2024-09-10T21:17:10.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.053+0000 7fa67d400640 1 -- start start 2024-09-10T21:17:10.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.053+0000 7fa67d400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6780d0890 0x7fa678147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:10.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.053+0000 7fa67d400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa67815f450 0x7fa6780c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:10.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.053+0000 7fa67d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6780c58b0 0x7fa6780b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:10.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa67d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6780b08b0 con 0x7fa6780c58b0 2024-09-10T21:17:10.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa67d400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa6780b0730 con 0x7fa67815f450 2024-09-10T21:17:10.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa67d400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa6780b0a30 con 0x7fa6780d0890 2024-09-10T21:17:10.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa677200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6780c58b0 0x7fa6780b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:10.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa677200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6780c58b0 0x7fa6780b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54394/0 (socket says 172.21.15.16:54394) 2024-09-10T21:17:10.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa677200640 1 -- 172.21.15.16:0/1064425178 learned_addr learned my addr 172.21.15.16:0/1064425178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:10.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa66de00640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa67815f450 0x7fa6780c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:10.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa676800640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6780d0890 0x7fa678147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:10.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa677200640 1 -- 172.21.15.16:0/1064425178 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6780d0890 msgr2=0x7fa678147eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:10.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa677200640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6780d0890 0x7fa678147eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa677200640 1 -- 172.21.15.16:0/1064425178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa67815f450 msgr2=0x7fa6780c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:10.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa677200640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa67815f450 0x7fa6780c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa677200640 1 -- 172.21.15.16:0/1064425178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6780c5e20 con 0x7fa6780c58b0 2024-09-10T21:17:10.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa677200640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6780c58b0 0x7fa6780b3d90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa67001a9d0 tx=0x7fa670004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:10.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa66f400640 1 -- 172.21.15.16:0/1064425178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6700bb070 con 0x7fa6780c58b0 2024-09-10T21:17:10.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa66f400640 1 -- 172.21.15.16:0/1064425178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa670004a50 con 0x7fa6780c58b0 2024-09-10T21:17:10.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa67d400640 1 -- 172.21.15.16:0/1064425178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6780c6100 con 0x7fa6780c58b0 2024-09-10T21:17:10.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa66f400640 1 -- 172.21.15.16:0/1064425178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6700b2040 con 0x7fa6780c58b0 2024-09-10T21:17:10.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa67d400640 1 -- 172.21.15.16:0/1064425178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6780c69a0 con 0x7fa6780c58b0 2024-09-10T21:17:10.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa67d400640 1 -- 172.21.15.16:0/1064425178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6780c64f0 con 0x7fa6780c58b0 2024-09-10T21:17:10.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.057+0000 7fa66f400640 1 -- 172.21.15.16:0/1064425178 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa6700b5030 con 0x7fa6780c58b0 2024-09-10T21:17:10.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.061+0000 7fa66f400640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa6540725e0 0x7fa654074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:10.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.061+0000 7fa66f400640 1 -- 172.21.15.16:0/1064425178 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fa67012cfb0 con 0x7fa6780c58b0 2024-09-10T21:17:10.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.061+0000 7fa676800640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa6540725e0 0x7fa654074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:10.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.061+0000 7fa676800640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa6540725e0 0x7fa654074aa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa668007fd0 tx=0x7fa668009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:10.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.065+0000 7fa66f400640 1 -- 172.21.15.16:0/1064425178 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa670016990 con 0x7fa6780c58b0 2024-09-10T21:17:10.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.233+0000 7fa67d400640 1 -- 172.21.15.16:0/1064425178 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6780cf760 con 0x7fa6540725e0 2024-09-10T21:17:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.237+0000 7fa66f400640 1 -- 172.21.15.16:0/1064425178 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252465 (secure 0 0 0) 0x7fa6780cf760 con 0x7fa6540725e0 2024-09-10T21:17:10.245 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:10.246 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.241+0000 7fa67d400640 1 -- 172.21.15.16:0/1064425178 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa6540725e0 msgr2=0x7fa654074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.241+0000 7fa67d400640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa6540725e0 0x7fa654074aa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa668007fd0 tx=0x7fa668009040 comp rx=0 tx=0).stop 2024-09-10T21:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.241+0000 7fa67d400640 1 -- 172.21.15.16:0/1064425178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6780c58b0 msgr2=0x7fa6780b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.241+0000 7fa67d400640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6780c58b0 0x7fa6780b3d90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa67001a9d0 tx=0x7fa670004270 comp rx=0 tx=0).stop 2024-09-10T21:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.241+0000 7fa67d400640 1 -- 172.21.15.16:0/1064425178 shutdown_connections 2024-09-10T21:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.241+0000 7fa67d400640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa6540725e0 0x7fa654074aa0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.241+0000 7fa67d400640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6780c58b0 0x7fa6780b3d90 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.241+0000 7fa67d400640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa67815f450 0x7fa6780c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.241+0000 7fa67d400640 1 --2- 172.21.15.16:0/1064425178 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6780d0890 0x7fa678147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.241+0000 7fa67d400640 1 -- 172.21.15.16:0/1064425178 >> 172.21.15.16:0/1064425178 conn(0x7fa6780d1d80 msgr2=0x7fa6780721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.241+0000 7fa67d400640 1 -- 172.21.15.16:0/1064425178 shutdown_connections 2024-09-10T21:17:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.241+0000 7fa67d400640 1 -- 172.21.15.16:0/1064425178 wait complete. 2024-09-10T21:17:10.268 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-10T21:17:09.549619+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277332,"kb_used_data":11176,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754697388,"statfs":{"total":773094113280,"available":772810125312,"internally_reserved":0,"allocated":11444224,"data_stored":5542644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.879329"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.715634+0000","last_change":"2024-09-10T21:17:05.723715+0000","last_active":"2024-09-10T21:17:06.715634+0000","last_peered":"2024-09-10T21:17:06.715634+0000","last_clean":"2024-09-10T21:17:06.715634+0000","last_became_active":"2024-09-10T21:17:05.723373+0000","last_became_peered":"2024-09-10T21:17:05.723373+0000","last_unstale":"2024-09-10T21:17:06.715634+0000","last_undegraded":"2024-09-10T21:17:06.715634+0000","last_fullsized":"2024-09-10T21:17:06.715634+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:30.298078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713600+0000","last_change":"2024-09-10T21:17:05.733972+0000","last_active":"2024-09-10T21:17:06.713600+0000","last_peered":"2024-09-10T21:17:06.713600+0000","last_clean":"2024-09-10T21:17:06.713600+0000","last_became_active":"2024-09-10T21:17:05.733615+0000","last_became_peered":"2024-09-10T21:17:05.733615+0000","last_unstale":"2024-09-10T21:17:06.713600+0000","last_undegraded":"2024-09-10T21:17:06.713600+0000","last_fullsized":"2024-09-10T21:17:06.713600+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:19.466316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789795+0000","last_change":"2024-09-10T21:17:05.715198+0000","last_active":"2024-09-10T21:17:06.789795+0000","last_peered":"2024-09-10T21:17:06.789795+0000","last_clean":"2024-09-10T21:17:06.789795+0000","last_became_active":"2024-09-10T21:17:05.714916+0000","last_became_peered":"2024-09-10T21:17:05.714916+0000","last_unstale":"2024-09-10T21:17:06.789795+0000","last_undegraded":"2024-09-10T21:17:06.789795+0000","last_fullsized":"2024-09-10T21:17:06.789795+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:52.527664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790024+0000","last_change":"2024-09-10T21:17:05.727547+0000","last_active":"2024-09-10T21:17:06.790024+0000","last_peered":"2024-09-10T21:17:06.790024+0000","last_clean":"2024-09-10T21:17:06.790024+0000","last_became_active":"2024-09-10T21:17:05.727277+0000","last_became_peered":"2024-09-10T21:17:05.727277+0000","last_unstale":"2024-09-10T21:17:06.790024+0000","last_undegraded":"2024-09-10T21:17:06.790024+0000","last_fullsized":"2024-09-10T21:17:06.790024+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:44.442642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.788970+0000","last_change":"2024-09-10T21:17:05.718700+0000","last_active":"2024-09-10T21:17:06.788970+0000","last_peered":"2024-09-10T21:17:06.788970+0000","last_clean":"2024-09-10T21:17:06.788970+0000","last_became_active":"2024-09-10T21:17:05.718269+0000","last_became_peered":"2024-09-10T21:17:05.718269+0000","last_unstale":"2024-09-10T21:17:06.788970+0000","last_undegraded":"2024-09-10T21:17:06.788970+0000","last_fullsized":"2024-09-10T21:17:06.788970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:52.527664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714174+0000","last_change":"2024-09-10T21:17:05.717650+0000","last_active":"2024-09-10T21:17:06.714174+0000","last_peered":"2024-09-10T21:17:06.714174+0000","last_clean":"2024-09-10T21:17:06.714174+0000","last_became_active":"2024-09-10T21:17:05.717297+0000","last_became_peered":"2024-09-10T21:17:05.717297+0000","last_unstale":"2024-09-10T21:17:06.714174+0000","last_undegraded":"2024-09-10T21:17:06.714174+0000","last_fullsized":"2024-09-10T21:17:06.714174+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:30:08.977859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725391+0000","last_change":"2024-09-10T21:17:05.714048+0000","last_active":"2024-09-10T21:17:06.725391+0000","last_peered":"2024-09-10T21:17:06.725391+0000","last_clean":"2024-09-10T21:17:06.725391+0000","last_became_active":"2024-09-10T21:17:05.713661+0000","last_became_peered":"2024-09-10T21:17:05.713661+0000","last_unstale":"2024-09-10T21:17:06.725391+0000","last_undegraded":"2024-09-10T21:17:06.725391+0000","last_fullsized":"2024-09-10T21:17:06.725391+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:22.153232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725466+0000","last_change":"2024-09-10T21:17:05.707400+0000","last_active":"2024-09-10T21:17:06.725466+0000","last_peered":"2024-09-10T21:17:06.725466+0000","last_clean":"2024-09-10T21:17:06.725466+0000","last_became_active":"2024-09-10T21:17:05.707158+0000","last_became_peered":"2024-09-10T21:17:05.707158+0000","last_unstale":"2024-09-10T21:17:06.725466+0000","last_undegraded":"2024-09-10T21:17:06.725466+0000","last_fullsized":"2024-09-10T21:17:06.725466+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:38.211010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.715661+0000","last_change":"2024-09-10T21:17:05.737208+0000","last_active":"2024-09-10T21:17:06.715661+0000","last_peered":"2024-09-10T21:17:06.715661+0000","last_clean":"2024-09-10T21:17:06.715661+0000","last_became_active":"2024-09-10T21:17:05.736688+0000","last_became_peered":"2024-09-10T21:17:05.736688+0000","last_unstale":"2024-09-10T21:17:06.715661+0000","last_undegraded":"2024-09-10T21:17:06.715661+0000","last_fullsized":"2024-09-10T21:17:06.715661+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:46.092359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.788884+0000","last_change":"2024-09-10T21:17:05.713658+0000","last_active":"2024-09-10T21:17:06.788884+0000","last_peered":"2024-09-10T21:17:06.788884+0000","last_clean":"2024-09-10T21:17:06.788884+0000","last_became_active":"2024-09-10T21:17:05.713360+0000","last_became_peered":"2024-09-10T21:17:05.713360+0000","last_unstale":"2024-09-10T21:17:06.788884+0000","last_undegraded":"2024-09-10T21:17:06.788884+0000","last_fullsized":"2024-09-10T21:17:06.788884+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:03.138945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725789+0000","last_change":"2024-09-10T21:17:05.714421+0000","last_active":"2024-09-10T21:17:06.725789+0000","last_peered":"2024-09-10T21:17:06.725789+0000","last_clean":"2024-09-10T21:17:06.725789+0000","last_became_active":"2024-09-10T21:17:05.714157+0000","last_became_peered":"2024-09-10T21:17:05.714157+0000","last_unstale":"2024-09-10T21:17:06.725789+0000","last_undegraded":"2024-09-10T21:17:06.725789+0000","last_fullsized":"2024-09-10T21:17:06.725789+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:38.483591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789008+0000","last_change":"2024-09-10T21:17:05.716690+0000","last_active":"2024-09-10T21:17:06.789008+0000","last_peered":"2024-09-10T21:17:06.789008+0000","last_clean":"2024-09-10T21:17:06.789008+0000","last_became_active":"2024-09-10T21:17:05.715825+0000","last_became_peered":"2024-09-10T21:17:05.715825+0000","last_unstale":"2024-09-10T21:17:06.789008+0000","last_undegraded":"2024-09-10T21:17:06.789008+0000","last_fullsized":"2024-09-10T21:17:06.789008+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:09.530403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713711+0000","last_change":"2024-09-10T21:17:05.709524+0000","last_active":"2024-09-10T21:17:06.713711+0000","last_peered":"2024-09-10T21:17:06.713711+0000","last_clean":"2024-09-10T21:17:06.713711+0000","last_became_active":"2024-09-10T21:17:05.709225+0000","last_became_peered":"2024-09-10T21:17:05.709225+0000","last_unstale":"2024-09-10T21:17:06.713711+0000","last_undegraded":"2024-09-10T21:17:06.713711+0000","last_fullsized":"2024-09-10T21:17:06.713711+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:33.711790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790485+0000","last_change":"2024-09-10T21:17:05.721167+0000","last_active":"2024-09-10T21:17:06.790485+0000","last_peered":"2024-09-10T21:17:06.790485+0000","last_clean":"2024-09-10T21:17:06.790485+0000","last_became_active":"2024-09-10T21:17:05.719003+0000","last_became_peered":"2024-09-10T21:17:05.719003+0000","last_unstale":"2024-09-10T21:17:06.790485+0000","last_undegraded":"2024-09-10T21:17:06.790485+0000","last_fullsized":"2024-09-10T21:17:06.790485+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:20.034188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713774+0000","last_change":"2024-09-10T21:17:05.726933+0000","last_active":"2024-09-10T21:17:06.713774+0000","last_peered":"2024-09-10T21:17:06.713774+0000","last_clean":"2024-09-10T21:17:06.713774+0000","last_became_active":"2024-09-10T21:17:05.726509+0000","last_became_peered":"2024-09-10T21:17:05.726509+0000","last_unstale":"2024-09-10T21:17:06.713774+0000","last_undegraded":"2024-09-10T21:17:06.713774+0000","last_fullsized":"2024-09-10T21:17:06.713774+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:05.782281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789392+0000","last_change":"2024-09-10T21:17:05.705180+0000","last_active":"2024-09-10T21:17:06.789392+0000","last_peered":"2024-09-10T21:17:06.789392+0000","last_clean":"2024-09-10T21:17:06.789392+0000","last_became_active":"2024-09-10T21:17:05.704804+0000","last_became_peered":"2024-09-10T21:17:05.704804+0000","last_unstale":"2024-09-10T21:17:06.789392+0000","last_undegraded":"2024-09-10T21:17:06.789392+0000","last_fullsized":"2024-09-10T21:17:06.789392+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:08.888392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790064+0000","last_change":"2024-09-10T21:17:05.721489+0000","last_active":"2024-09-10T21:17:06.790064+0000","last_peered":"2024-09-10T21:17:06.790064+0000","last_clean":"2024-09-10T21:17:06.790064+0000","last_became_active":"2024-09-10T21:17:05.721186+0000","last_became_peered":"2024-09-10T21:17:05.721186+0000","last_unstale":"2024-09-10T21:17:06.790064+0000","last_undegraded":"2024-09-10T21:17:06.790064+0000","last_fullsized":"2024-09-10T21:17:06.790064+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:06.574126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725646+0000","last_change":"2024-09-10T21:17:05.714193+0000","last_active":"2024-09-10T21:17:06.725646+0000","last_peered":"2024-09-10T21:17:06.725646+0000","last_clean":"2024-09-10T21:17:06.725646+0000","last_became_active":"2024-09-10T21:17:05.713896+0000","last_became_peered":"2024-09-10T21:17:05.713896+0000","last_unstale":"2024-09-10T21:17:06.725646+0000","last_undegraded":"2024-09-10T21:17:06.725646+0000","last_fullsized":"2024-09-10T21:17:06.725646+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:49.649607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790055+0000","last_change":"2024-09-10T21:17:05.712173+0000","last_active":"2024-09-10T21:17:06.790055+0000","last_peered":"2024-09-10T21:17:06.790055+0000","last_clean":"2024-09-10T21:17:06.790055+0000","last_became_active":"2024-09-10T21:17:05.711738+0000","last_became_peered":"2024-09-10T21:17:05.711738+0000","last_unstale":"2024-09-10T21:17:06.790055+0000","last_undegraded":"2024-09-10T21:17:06.790055+0000","last_fullsized":"2024-09-10T21:17:06.790055+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:53.783562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789630+0000","last_change":"2024-09-10T21:17:05.734524+0000","last_active":"2024-09-10T21:17:06.789630+0000","last_peered":"2024-09-10T21:17:06.789630+0000","last_clean":"2024-09-10T21:17:06.789630+0000","last_became_active":"2024-09-10T21:17:05.733939+0000","last_became_peered":"2024-09-10T21:17:05.733939+0000","last_unstale":"2024-09-10T21:17:06.789630+0000","last_undegraded":"2024-09-10T21:17:06.789630+0000","last_fullsized":"2024-09-10T21:17:06.789630+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:58.198912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725845+0000","last_change":"2024-09-10T21:17:05.736637+0000","last_active":"2024-09-10T21:17:06.725845+0000","last_peered":"2024-09-10T21:17:06.725845+0000","last_clean":"2024-09-10T21:17:06.725845+0000","last_became_active":"2024-09-10T21:17:05.736266+0000","last_became_peered":"2024-09-10T21:17:05.736266+0000","last_unstale":"2024-09-10T21:17:06.725845+0000","last_undegraded":"2024-09-10T21:17:06.725845+0000","last_fullsized":"2024-09-10T21:17:06.725845+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:53.932483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714371+0000","last_change":"2024-09-10T21:17:05.729126+0000","last_active":"2024-09-10T21:17:06.714371+0000","last_peered":"2024-09-10T21:17:06.714371+0000","last_clean":"2024-09-10T21:17:06.714371+0000","last_became_active":"2024-09-10T21:17:05.725414+0000","last_became_peered":"2024-09-10T21:17:05.725414+0000","last_unstale":"2024-09-10T21:17:06.714371+0000","last_undegraded":"2024-09-10T21:17:06.714371+0000","last_fullsized":"2024-09-10T21:17:06.714371+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:48.942190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790676+0000","last_change":"2024-09-10T21:17:05.721470+0000","last_active":"2024-09-10T21:17:06.790676+0000","last_peered":"2024-09-10T21:17:06.790676+0000","last_clean":"2024-09-10T21:17:06.790676+0000","last_became_active":"2024-09-10T21:17:05.720747+0000","last_became_peered":"2024-09-10T21:17:05.720747+0000","last_unstale":"2024-09-10T21:17:06.790676+0000","last_undegraded":"2024-09-10T21:17:06.790676+0000","last_fullsized":"2024-09-10T21:17:06.790676+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:48.842455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789318+0000","last_change":"2024-09-10T21:17:05.720308+0000","last_active":"2024-09-10T21:17:06.789318+0000","last_peered":"2024-09-10T21:17:06.789318+0000","last_clean":"2024-09-10T21:17:06.789318+0000","last_became_active":"2024-09-10T21:17:05.718512+0000","last_became_peered":"2024-09-10T21:17:05.718512+0000","last_unstale":"2024-09-10T21:17:06.789318+0000","last_undegraded":"2024-09-10T21:17:06.789318+0000","last_fullsized":"2024-09-10T21:17:06.789318+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:35.531854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725544+0000","last_change":"2024-09-10T21:17:05.706902+0000","last_active":"2024-09-10T21:17:06.725544+0000","last_peered":"2024-09-10T21:17:06.725544+0000","last_clean":"2024-09-10T21:17:06.725544+0000","last_became_active":"2024-09-10T21:17:05.706640+0000","last_became_peered":"2024-09-10T21:17:05.706640+0000","last_unstale":"2024-09-10T21:17:06.725544+0000","last_undegraded":"2024-09-10T21:17:06.725544+0000","last_fullsized":"2024-09-10T21:17:06.725544+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:53.848905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714340+0000","last_change":"2024-09-10T21:17:05.713706+0000","last_active":"2024-09-10T21:17:06.714340+0000","last_peered":"2024-09-10T21:17:06.714340+0000","last_clean":"2024-09-10T21:17:06.714340+0000","last_became_active":"2024-09-10T21:17:05.713119+0000","last_became_peered":"2024-09-10T21:17:05.713119+0000","last_unstale":"2024-09-10T21:17:06.714340+0000","last_undegraded":"2024-09-10T21:17:06.714340+0000","last_fullsized":"2024-09-10T21:17:06.714340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:39.485778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790131+0000","last_change":"2024-09-10T21:17:05.732360+0000","last_active":"2024-09-10T21:17:06.790131+0000","last_peered":"2024-09-10T21:17:06.790131+0000","last_clean":"2024-09-10T21:17:06.790131+0000","last_became_active":"2024-09-10T21:17:05.732077+0000","last_became_peered":"2024-09-10T21:17:05.732077+0000","last_unstale":"2024-09-10T21:17:06.790131+0000","last_undegraded":"2024-09-10T21:17:06.790131+0000","last_fullsized":"2024-09-10T21:17:06.790131+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:16.869649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714221+0000","last_change":"2024-09-10T21:17:05.734177+0000","last_active":"2024-09-10T21:17:06.714221+0000","last_peered":"2024-09-10T21:17:06.714221+0000","last_clean":"2024-09-10T21:17:06.714221+0000","last_became_active":"2024-09-10T21:17:05.733726+0000","last_became_peered":"2024-09-10T21:17:05.733726+0000","last_unstale":"2024-09-10T21:17:06.714221+0000","last_undegraded":"2024-09-10T21:17:06.714221+0000","last_fullsized":"2024-09-10T21:17:06.714221+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:19.607889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790804+0000","last_change":"2024-09-10T21:17:05.723406+0000","last_active":"2024-09-10T21:17:06.790804+0000","last_peered":"2024-09-10T21:17:06.790804+0000","last_clean":"2024-09-10T21:17:06.790804+0000","last_became_active":"2024-09-10T21:17:05.723069+0000","last_became_peered":"2024-09-10T21:17:05.723069+0000","last_unstale":"2024-09-10T21:17:06.790804+0000","last_undegraded":"2024-09-10T21:17:06.790804+0000","last_fullsized":"2024-09-10T21:17:06.790804+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:56.946550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725616+0000","last_change":"2024-09-10T21:17:05.719422+0000","last_active":"2024-09-10T21:17:06.725616+0000","last_peered":"2024-09-10T21:17:06.725616+0000","last_clean":"2024-09-10T21:17:06.725616+0000","last_became_active":"2024-09-10T21:17:05.719133+0000","last_became_peered":"2024-09-10T21:17:05.719133+0000","last_unstale":"2024-09-10T21:17:06.725616+0000","last_undegraded":"2024-09-10T21:17:06.725616+0000","last_fullsized":"2024-09-10T21:17:06.725616+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:35.727546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714060+0000","last_change":"2024-09-10T21:17:05.719248+0000","last_active":"2024-09-10T21:17:06.714060+0000","last_peered":"2024-09-10T21:17:06.714060+0000","last_clean":"2024-09-10T21:17:06.714060+0000","last_became_active":"2024-09-10T21:17:05.716913+0000","last_became_peered":"2024-09-10T21:17:05.716913+0000","last_unstale":"2024-09-10T21:17:06.714060+0000","last_undegraded":"2024-09-10T21:17:06.714060+0000","last_fullsized":"2024-09-10T21:17:06.714060+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:07.613681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.788834+0000","last_change":"2024-09-10T21:17:05.720448+0000","last_active":"2024-09-10T21:17:06.788834+0000","last_peered":"2024-09-10T21:17:06.788834+0000","last_clean":"2024-09-10T21:17:06.788834+0000","last_became_active":"2024-09-10T21:17:05.719681+0000","last_became_peered":"2024-09-10T21:17:05.719681+0000","last_unstale":"2024-09-10T21:17:06.788834+0000","last_undegraded":"2024-09-10T21:17:06.788834+0000","last_fullsized":"2024-09-10T21:17:06.788834+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:04.596230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.715598+0000","last_change":"2024-09-10T21:17:05.734329+0000","last_active":"2024-09-10T21:17:06.715598+0000","last_peered":"2024-09-10T21:17:06.715598+0000","last_clean":"2024-09-10T21:17:06.715598+0000","last_became_active":"2024-09-10T21:17:05.734053+0000","last_became_peered":"2024-09-10T21:17:05.734053+0000","last_unstale":"2024-09-10T21:17:06.715598+0000","last_undegraded":"2024-09-10T21:17:06.715598+0000","last_fullsized":"2024-09-10T21:17:06.715598+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:21.802637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"102'4","reported_seq":19,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790537+0000","last_change":"2024-09-10T21:17:04.695565+0000","last_active":"2024-09-10T21:17:06.790537+0000","last_peered":"2024-09-10T21:17:06.790537+0000","last_clean":"2024-09-10T21:17:06.790537+0000","last_became_active":"2024-09-10T21:17:04.695246+0000","last_became_peered":"2024-09-10T21:17:04.695246+0000","last_unstale":"2024-09-10T21:17:06.790537+0000","last_undegraded":"2024-09-10T21:17:06.790537+0000","last_fullsized":"2024-09-10T21:17:06.790537+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:09.828134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790642+0000","last_change":"2024-09-10T21:17:05.716779+0000","last_active":"2024-09-10T21:17:06.790642+0000","last_peered":"2024-09-10T21:17:06.790642+0000","last_clean":"2024-09-10T21:17:06.790642+0000","last_became_active":"2024-09-10T21:17:05.716496+0000","last_became_peered":"2024-09-10T21:17:05.716496+0000","last_unstale":"2024-09-10T21:17:06.790642+0000","last_undegraded":"2024-09-10T21:17:06.790642+0000","last_fullsized":"2024-09-10T21:17:06.790642+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:41.153323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713797+0000","last_change":"2024-09-10T21:17:04.682244+0000","last_active":"2024-09-10T21:17:06.713797+0000","last_peered":"2024-09-10T21:17:06.713797+0000","last_clean":"2024-09-10T21:17:06.713797+0000","last_became_active":"2024-09-10T21:17:04.681930+0000","last_became_peered":"2024-09-10T21:17:04.681930+0000","last_unstale":"2024-09-10T21:17:06.713797+0000","last_undegraded":"2024-09-10T21:17:06.713797+0000","last_fullsized":"2024-09-10T21:17:06.713797+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:47.085742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713970+0000","last_change":"2024-09-10T21:17:05.716573+0000","last_active":"2024-09-10T21:17:06.713970+0000","last_peered":"2024-09-10T21:17:06.713970+0000","last_clean":"2024-09-10T21:17:06.713970+0000","last_became_active":"2024-09-10T21:17:05.715945+0000","last_became_peered":"2024-09-10T21:17:05.715945+0000","last_unstale":"2024-09-10T21:17:06.713970+0000","last_undegraded":"2024-09-10T21:17:06.713970+0000","last_fullsized":"2024-09-10T21:17:06.713970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:45.800320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789759+0000","last_change":"2024-09-10T21:17:04.695949+0000","last_active":"2024-09-10T21:17:06.789759+0000","last_peered":"2024-09-10T21:17:06.789759+0000","last_clean":"2024-09-10T21:17:06.789759+0000","last_became_active":"2024-09-10T21:17:04.695583+0000","last_became_peered":"2024-09-10T21:17:04.695583+0000","last_unstale":"2024-09-10T21:17:06.789759+0000","last_undegraded":"2024-09-10T21:17:06.789759+0000","last_fullsized":"2024-09-10T21:17:06.789759+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:25.757676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789925+0000","last_change":"2024-09-10T21:17:05.717252+0000","last_active":"2024-09-10T21:17:06.789925+0000","last_peered":"2024-09-10T21:17:06.789925+0000","last_clean":"2024-09-10T21:17:06.789925+0000","last_became_active":"2024-09-10T21:17:05.716873+0000","last_became_peered":"2024-09-10T21:17:05.716873+0000","last_unstale":"2024-09-10T21:17:06.789925+0000","last_undegraded":"2024-09-10T21:17:06.789925+0000","last_fullsized":"2024-09-10T21:17:06.789925+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:09.530403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790021+0000","last_change":"2024-09-10T21:17:04.699016+0000","last_active":"2024-09-10T21:17:06.790021+0000","last_peered":"2024-09-10T21:17:06.790021+0000","last_clean":"2024-09-10T21:17:06.790021+0000","last_became_active":"2024-09-10T21:17:04.698458+0000","last_became_peered":"2024-09-10T21:17:04.698458+0000","last_unstale":"2024-09-10T21:17:06.790021+0000","last_undegraded":"2024-09-10T21:17:06.790021+0000","last_fullsized":"2024-09-10T21:17:06.790021+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:14.614359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789741+0000","last_change":"2024-09-10T21:17:05.717045+0000","last_active":"2024-09-10T21:17:06.789741+0000","last_peered":"2024-09-10T21:17:06.789741+0000","last_clean":"2024-09-10T21:17:06.789741+0000","last_became_active":"2024-09-10T21:17:05.716767+0000","last_became_peered":"2024-09-10T21:17:05.716767+0000","last_unstale":"2024-09-10T21:17:06.789741+0000","last_undegraded":"2024-09-10T21:17:06.789741+0000","last_fullsized":"2024-09-10T21:17:06.789741+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:53.452886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789606+0000","last_change":"2024-09-10T21:17:04.682093+0000","last_active":"2024-09-10T21:17:06.789606+0000","last_peered":"2024-09-10T21:17:06.789606+0000","last_clean":"2024-09-10T21:17:06.789606+0000","last_became_active":"2024-09-10T21:17:04.681799+0000","last_became_peered":"2024-09-10T21:17:04.681799+0000","last_unstale":"2024-09-10T21:17:06.789606+0000","last_undegraded":"2024-09-10T21:17:06.789606+0000","last_fullsized":"2024-09-10T21:17:06.789606+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:07.226948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725758+0000","last_change":"2024-09-10T21:17:05.704156+0000","last_active":"2024-09-10T21:17:06.725758+0000","last_peered":"2024-09-10T21:17:06.725758+0000","last_clean":"2024-09-10T21:17:06.725758+0000","last_became_active":"2024-09-10T21:17:05.703648+0000","last_became_peered":"2024-09-10T21:17:05.703648+0000","last_unstale":"2024-09-10T21:17:06.725758+0000","last_undegraded":"2024-09-10T21:17:06.725758+0000","last_fullsized":"2024-09-10T21:17:06.725758+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:57.792968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714528+0000","last_change":"2024-09-10T21:17:04.699682+0000","last_active":"2024-09-10T21:17:06.714528+0000","last_peered":"2024-09-10T21:17:06.714528+0000","last_clean":"2024-09-10T21:17:06.714528+0000","last_became_active":"2024-09-10T21:17:04.699406+0000","last_became_peered":"2024-09-10T21:17:04.699406+0000","last_unstale":"2024-09-10T21:17:06.714528+0000","last_undegraded":"2024-09-10T21:17:06.714528+0000","last_fullsized":"2024-09-10T21:17:06.714528+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:39.848996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789058+0000","last_change":"2024-09-10T21:17:05.720729+0000","last_active":"2024-09-10T21:17:06.789058+0000","last_peered":"2024-09-10T21:17:06.789058+0000","last_clean":"2024-09-10T21:17:06.789058+0000","last_became_active":"2024-09-10T21:17:05.720084+0000","last_became_peered":"2024-09-10T21:17:05.720084+0000","last_unstale":"2024-09-10T21:17:06.789058+0000","last_undegraded":"2024-09-10T21:17:06.789058+0000","last_fullsized":"2024-09-10T21:17:06.789058+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:37.107915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725459+0000","last_change":"2024-09-10T21:17:04.693503+0000","last_active":"2024-09-10T21:17:06.725459+0000","last_peered":"2024-09-10T21:17:06.725459+0000","last_clean":"2024-09-10T21:17:06.725459+0000","last_became_active":"2024-09-10T21:17:04.693071+0000","last_became_peered":"2024-09-10T21:17:04.693071+0000","last_unstale":"2024-09-10T21:17:06.725459+0000","last_undegraded":"2024-09-10T21:17:06.725459+0000","last_fullsized":"2024-09-10T21:17:06.725459+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:03.176796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725397+0000","last_change":"2024-09-10T21:17:05.705534+0000","last_active":"2024-09-10T21:17:06.725397+0000","last_peered":"2024-09-10T21:17:06.725397+0000","last_clean":"2024-09-10T21:17:06.725397+0000","last_became_active":"2024-09-10T21:17:05.705226+0000","last_became_peered":"2024-09-10T21:17:05.705226+0000","last_unstale":"2024-09-10T21:17:06.725397+0000","last_undegraded":"2024-09-10T21:17:06.725397+0000","last_fullsized":"2024-09-10T21:17:06.725397+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:50.189589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789169+0000","last_change":"2024-09-10T21:17:04.698131+0000","last_active":"2024-09-10T21:17:06.789169+0000","last_peered":"2024-09-10T21:17:06.789169+0000","last_clean":"2024-09-10T21:17:06.789169+0000","last_became_active":"2024-09-10T21:17:04.697827+0000","last_became_peered":"2024-09-10T21:17:04.697827+0000","last_unstale":"2024-09-10T21:17:06.789169+0000","last_undegraded":"2024-09-10T21:17:06.789169+0000","last_fullsized":"2024-09-10T21:17:06.789169+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:14.346425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713807+0000","last_change":"2024-09-10T21:17:05.713532+0000","last_active":"2024-09-10T21:17:06.713807+0000","last_peered":"2024-09-10T21:17:06.713807+0000","last_clean":"2024-09-10T21:17:06.713807+0000","last_became_active":"2024-09-10T21:17:05.713097+0000","last_became_peered":"2024-09-10T21:17:05.713097+0000","last_unstale":"2024-09-10T21:17:06.713807+0000","last_undegraded":"2024-09-10T21:17:06.713807+0000","last_fullsized":"2024-09-10T21:17:06.713807+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:58.222857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789927+0000","last_change":"2024-09-10T21:17:04.698054+0000","last_active":"2024-09-10T21:17:06.789927+0000","last_peered":"2024-09-10T21:17:06.789927+0000","last_clean":"2024-09-10T21:17:06.789927+0000","last_became_active":"2024-09-10T21:17:04.697481+0000","last_became_peered":"2024-09-10T21:17:04.697481+0000","last_unstale":"2024-09-10T21:17:06.789927+0000","last_undegraded":"2024-09-10T21:17:06.789927+0000","last_fullsized":"2024-09-10T21:17:06.789927+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:49.932598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713901+0000","last_change":"2024-09-10T21:17:05.709326+0000","last_active":"2024-09-10T21:17:06.713901+0000","last_peered":"2024-09-10T21:17:06.713901+0000","last_clean":"2024-09-10T21:17:06.713901+0000","last_became_active":"2024-09-10T21:17:05.708989+0000","last_became_peered":"2024-09-10T21:17:05.708989+0000","last_unstale":"2024-09-10T21:17:06.713901+0000","last_undegraded":"2024-09-10T21:17:06.713901+0000","last_fullsized":"2024-09-10T21:17:06.713901+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:26.915328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.788840+0000","last_change":"2024-09-10T21:17:04.696842+0000","last_active":"2024-09-10T21:17:06.788840+0000","last_peered":"2024-09-10T21:17:06.788840+0000","last_clean":"2024-09-10T21:17:06.788840+0000","last_became_active":"2024-09-10T21:17:04.696540+0000","last_became_peered":"2024-09-10T21:17:04.696540+0000","last_unstale":"2024-09-10T21:17:06.788840+0000","last_undegraded":"2024-09-10T21:17:06.788840+0000","last_fullsized":"2024-09-10T21:17:06.788840+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:06.998463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714266+0000","last_change":"2024-09-10T21:17:05.717559+0000","last_active":"2024-09-10T21:17:06.714266+0000","last_peered":"2024-09-10T21:17:06.714266+0000","last_clean":"2024-09-10T21:17:06.714266+0000","last_became_active":"2024-09-10T21:17:05.717208+0000","last_became_peered":"2024-09-10T21:17:05.717208+0000","last_unstale":"2024-09-10T21:17:06.714266+0000","last_undegraded":"2024-09-10T21:17:06.714266+0000","last_fullsized":"2024-09-10T21:17:06.714266+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:47.158495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"102'3","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789286+0000","last_change":"2024-09-10T21:17:04.703239+0000","last_active":"2024-09-10T21:17:06.789286+0000","last_peered":"2024-09-10T21:17:06.789286+0000","last_clean":"2024-09-10T21:17:06.789286+0000","last_became_active":"2024-09-10T21:17:04.702940+0000","last_became_peered":"2024-09-10T21:17:04.702940+0000","last_unstale":"2024-09-10T21:17:06.789286+0000","last_undegraded":"2024-09-10T21:17:06.789286+0000","last_fullsized":"2024-09-10T21:17:06.789286+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:57.976080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.726002+0000","last_change":"2024-09-10T21:17:05.715752+0000","last_active":"2024-09-10T21:17:06.726002+0000","last_peered":"2024-09-10T21:17:06.726002+0000","last_clean":"2024-09-10T21:17:06.726002+0000","last_became_active":"2024-09-10T21:17:05.715497+0000","last_became_peered":"2024-09-10T21:17:05.715497+0000","last_unstale":"2024-09-10T21:17:06.726002+0000","last_undegraded":"2024-09-10T21:17:06.726002+0000","last_fullsized":"2024-09-10T21:17:06.726002+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:19.793726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"102'6","reported_seq":21,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790131+0000","last_change":"2024-09-10T21:17:04.698722+0000","last_active":"2024-09-10T21:17:06.790131+0000","last_peered":"2024-09-10T21:17:06.790131+0000","last_clean":"2024-09-10T21:17:06.790131+0000","last_became_active":"2024-09-10T21:17:04.698079+0000","last_became_peered":"2024-09-10T21:17:04.698079+0000","last_unstale":"2024-09-10T21:17:06.790131+0000","last_undegraded":"2024-09-10T21:17:06.790131+0000","last_fullsized":"2024-09-10T21:17:06.790131+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:15.763202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789323+0000","last_change":"2024-09-10T21:17:05.706059+0000","last_active":"2024-09-10T21:17:06.789323+0000","last_peered":"2024-09-10T21:17:06.789323+0000","last_clean":"2024-09-10T21:17:06.789323+0000","last_became_active":"2024-09-10T21:17:05.704630+0000","last_became_peered":"2024-09-10T21:17:05.704630+0000","last_unstale":"2024-09-10T21:17:06.789323+0000","last_undegraded":"2024-09-10T21:17:06.789323+0000","last_fullsized":"2024-09-10T21:17:06.789323+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:06.574126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789109+0000","last_change":"2024-09-10T21:17:04.694226+0000","last_active":"2024-09-10T21:17:06.789109+0000","last_peered":"2024-09-10T21:17:06.789109+0000","last_clean":"2024-09-10T21:17:06.789109+0000","last_became_active":"2024-09-10T21:17:04.693900+0000","last_became_peered":"2024-09-10T21:17:04.693900+0000","last_unstale":"2024-09-10T21:17:06.789109+0000","last_undegraded":"2024-09-10T21:17:06.789109+0000","last_fullsized":"2024-09-10T21:17:06.789109+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:02.475825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790362+0000","last_change":"2024-09-10T21:17:05.733006+0000","last_active":"2024-09-10T21:17:06.790362+0000","last_peered":"2024-09-10T21:17:06.790362+0000","last_clean":"2024-09-10T21:17:06.790362+0000","last_became_active":"2024-09-10T21:17:05.731901+0000","last_became_peered":"2024-09-10T21:17:05.731901+0000","last_unstale":"2024-09-10T21:17:06.790362+0000","last_undegraded":"2024-09-10T21:17:06.790362+0000","last_fullsized":"2024-09-10T21:17:06.790362+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:31.110568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789809+0000","last_change":"2024-09-10T21:17:04.692932+0000","last_active":"2024-09-10T21:17:06.789809+0000","last_peered":"2024-09-10T21:17:06.789809+0000","last_clean":"2024-09-10T21:17:06.789809+0000","last_became_active":"2024-09-10T21:17:04.692669+0000","last_became_peered":"2024-09-10T21:17:04.692669+0000","last_unstale":"2024-09-10T21:17:06.789809+0000","last_undegraded":"2024-09-10T21:17:06.789809+0000","last_fullsized":"2024-09-10T21:17:06.789809+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:36:46.857369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789207+0000","last_change":"2024-09-10T21:17:05.721097+0000","last_active":"2024-09-10T21:17:06.789207+0000","last_peered":"2024-09-10T21:17:06.789207+0000","last_clean":"2024-09-10T21:17:06.789207+0000","last_became_active":"2024-09-10T21:17:05.720810+0000","last_became_peered":"2024-09-10T21:17:05.720810+0000","last_unstale":"2024-09-10T21:17:06.789207+0000","last_undegraded":"2024-09-10T21:17:06.789207+0000","last_fullsized":"2024-09-10T21:17:06.789207+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:25.693651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"102'8","reported_seq":23,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713499+0000","last_change":"2024-09-10T21:17:04.696549+0000","last_active":"2024-09-10T21:17:06.713499+0000","last_peered":"2024-09-10T21:17:06.713499+0000","last_clean":"2024-09-10T21:17:06.713499+0000","last_became_active":"2024-09-10T21:17:04.696267+0000","last_became_peered":"2024-09-10T21:17:04.696267+0000","last_unstale":"2024-09-10T21:17:06.713499+0000","last_undegraded":"2024-09-10T21:17:06.713499+0000","last_fullsized":"2024-09-10T21:17:06.713499+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:31.579988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.715589+0000","last_change":"2024-09-10T21:17:05.722933+0000","last_active":"2024-09-10T21:17:06.715589+0000","last_peered":"2024-09-10T21:17:06.715589+0000","last_clean":"2024-09-10T21:17:06.715589+0000","last_became_active":"2024-09-10T21:17:05.722661+0000","last_became_peered":"2024-09-10T21:17:05.722661+0000","last_unstale":"2024-09-10T21:17:06.715589+0000","last_undegraded":"2024-09-10T21:17:06.715589+0000","last_fullsized":"2024-09-10T21:17:06.715589+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:44.442642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714136+0000","last_change":"2024-09-10T21:17:04.684959+0000","last_active":"2024-09-10T21:17:06.714136+0000","last_peered":"2024-09-10T21:17:06.714136+0000","last_clean":"2024-09-10T21:17:06.714136+0000","last_became_active":"2024-09-10T21:17:04.684651+0000","last_became_peered":"2024-09-10T21:17:04.684651+0000","last_unstale":"2024-09-10T21:17:06.714136+0000","last_undegraded":"2024-09-10T21:17:06.714136+0000","last_fullsized":"2024-09-10T21:17:06.714136+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:47.179182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.715851+0000","last_change":"2024-09-10T21:17:05.736938+0000","last_active":"2024-09-10T21:17:06.715851+0000","last_peered":"2024-09-10T21:17:06.715851+0000","last_clean":"2024-09-10T21:17:06.715851+0000","last_became_active":"2024-09-10T21:17:05.736336+0000","last_became_peered":"2024-09-10T21:17:05.736336+0000","last_unstale":"2024-09-10T21:17:06.715851+0000","last_undegraded":"2024-09-10T21:17:06.715851+0000","last_fullsized":"2024-09-10T21:17:06.715851+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:34.951525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789558+0000","last_change":"2024-09-10T21:17:04.694123+0000","last_active":"2024-09-10T21:17:06.789558+0000","last_peered":"2024-09-10T21:17:06.789558+0000","last_clean":"2024-09-10T21:17:06.789558+0000","last_became_active":"2024-09-10T21:17:04.693785+0000","last_became_peered":"2024-09-10T21:17:04.693785+0000","last_unstale":"2024-09-10T21:17:06.789558+0000","last_undegraded":"2024-09-10T21:17:06.789558+0000","last_fullsized":"2024-09-10T21:17:06.789558+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:35.975151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725553+0000","last_change":"2024-09-10T21:17:05.713293+0000","last_active":"2024-09-10T21:17:06.725553+0000","last_peered":"2024-09-10T21:17:06.725553+0000","last_clean":"2024-09-10T21:17:06.725553+0000","last_became_active":"2024-09-10T21:17:05.713015+0000","last_became_peered":"2024-09-10T21:17:05.713015+0000","last_unstale":"2024-09-10T21:17:06.725553+0000","last_undegraded":"2024-09-10T21:17:06.725553+0000","last_fullsized":"2024-09-10T21:17:06.725553+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:53.163112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790678+0000","last_change":"2024-09-10T21:17:04.703676+0000","last_active":"2024-09-10T21:17:06.790678+0000","last_peered":"2024-09-10T21:17:06.790678+0000","last_clean":"2024-09-10T21:17:06.790678+0000","last_became_active":"2024-09-10T21:17:04.703280+0000","last_became_peered":"2024-09-10T21:17:04.703280+0000","last_unstale":"2024-09-10T21:17:06.790678+0000","last_undegraded":"2024-09-10T21:17:06.790678+0000","last_fullsized":"2024-09-10T21:17:06.790678+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:39.837301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790746+0000","last_change":"2024-09-10T21:17:05.720913+0000","last_active":"2024-09-10T21:17:06.790746+0000","last_peered":"2024-09-10T21:17:06.790746+0000","last_clean":"2024-09-10T21:17:06.790746+0000","last_became_active":"2024-09-10T21:17:05.720633+0000","last_became_peered":"2024-09-10T21:17:05.720633+0000","last_unstale":"2024-09-10T21:17:06.790746+0000","last_undegraded":"2024-09-10T21:17:06.790746+0000","last_fullsized":"2024-09-10T21:17:06.790746+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:36:16.108624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790629+0000","last_change":"2024-09-10T21:17:04.703781+0000","last_active":"2024-09-10T21:17:06.790629+0000","last_peered":"2024-09-10T21:17:06.790629+0000","last_clean":"2024-09-10T21:17:06.790629+0000","last_became_active":"2024-09-10T21:17:04.703444+0000","last_became_peered":"2024-09-10T21:17:04.703444+0000","last_unstale":"2024-09-10T21:17:06.790629+0000","last_undegraded":"2024-09-10T21:17:06.790629+0000","last_fullsized":"2024-09-10T21:17:06.790629+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:45.230767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789949+0000","last_change":"2024-09-10T21:17:05.711925+0000","last_active":"2024-09-10T21:17:06.789949+0000","last_peered":"2024-09-10T21:17:06.789949+0000","last_clean":"2024-09-10T21:17:06.789949+0000","last_became_active":"2024-09-10T21:17:05.711624+0000","last_became_peered":"2024-09-10T21:17:05.711624+0000","last_unstale":"2024-09-10T21:17:06.789949+0000","last_undegraded":"2024-09-10T21:17:06.789949+0000","last_fullsized":"2024-09-10T21:17:06.789949+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:37.107915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789176+0000","last_change":"2024-09-10T21:17:04.696347+0000","last_active":"2024-09-10T21:17:06.789176+0000","last_peered":"2024-09-10T21:17:06.789176+0000","last_clean":"2024-09-10T21:17:06.789176+0000","last_became_active":"2024-09-10T21:17:04.695362+0000","last_became_peered":"2024-09-10T21:17:04.695362+0000","last_unstale":"2024-09-10T21:17:06.789176+0000","last_undegraded":"2024-09-10T21:17:06.789176+0000","last_fullsized":"2024-09-10T21:17:06.789176+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:04.933492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714898+0000","last_change":"2024-09-10T21:17:05.715049+0000","last_active":"2024-09-10T21:17:06.714898+0000","last_peered":"2024-09-10T21:17:06.714898+0000","last_clean":"2024-09-10T21:17:06.714898+0000","last_became_active":"2024-09-10T21:17:05.714759+0000","last_became_peered":"2024-09-10T21:17:05.714759+0000","last_unstale":"2024-09-10T21:17:06.714898+0000","last_undegraded":"2024-09-10T21:17:06.714898+0000","last_fullsized":"2024-09-10T21:17:06.714898+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:17.265724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714565+0000","last_change":"2024-09-10T21:17:04.702725+0000","last_active":"2024-09-10T21:17:06.714565+0000","last_peered":"2024-09-10T21:17:06.714565+0000","last_clean":"2024-09-10T21:17:06.714565+0000","last_became_active":"2024-09-10T21:17:04.702394+0000","last_became_peered":"2024-09-10T21:17:04.702394+0000","last_unstale":"2024-09-10T21:17:06.714565+0000","last_undegraded":"2024-09-10T21:17:06.714565+0000","last_fullsized":"2024-09-10T21:17:06.714565+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:52.701234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790293+0000","last_change":"2024-09-10T21:17:05.725270+0000","last_active":"2024-09-10T21:17:06.790293+0000","last_peered":"2024-09-10T21:17:06.790293+0000","last_clean":"2024-09-10T21:17:06.790293+0000","last_became_active":"2024-09-10T21:17:05.724923+0000","last_became_peered":"2024-09-10T21:17:05.724923+0000","last_unstale":"2024-09-10T21:17:06.790293+0000","last_undegraded":"2024-09-10T21:17:06.790293+0000","last_fullsized":"2024-09-10T21:17:06.790293+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:10.496133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789360+0000","last_change":"2024-09-10T21:17:04.702256+0000","last_active":"2024-09-10T21:17:06.789360+0000","last_peered":"2024-09-10T21:17:06.789360+0000","last_clean":"2024-09-10T21:17:06.789360+0000","last_became_active":"2024-09-10T21:17:04.701928+0000","last_became_peered":"2024-09-10T21:17:04.701928+0000","last_unstale":"2024-09-10T21:17:06.789360+0000","last_undegraded":"2024-09-10T21:17:06.789360+0000","last_fullsized":"2024-09-10T21:17:06.789360+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:58.395259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.788713+0000","last_change":"2024-09-10T21:17:05.728741+0000","last_active":"2024-09-10T21:17:06.788713+0000","last_peered":"2024-09-10T21:17:06.788713+0000","last_clean":"2024-09-10T21:17:06.788713+0000","last_became_active":"2024-09-10T21:17:05.728272+0000","last_became_peered":"2024-09-10T21:17:05.728272+0000","last_unstale":"2024-09-10T21:17:06.788713+0000","last_undegraded":"2024-09-10T21:17:06.788713+0000","last_fullsized":"2024-09-10T21:17:06.788713+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:01.867268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789309+0000","last_change":"2024-09-10T21:17:04.691733+0000","last_active":"2024-09-10T21:17:06.789309+0000","last_peered":"2024-09-10T21:17:06.789309+0000","last_clean":"2024-09-10T21:17:06.789309+0000","last_became_active":"2024-09-10T21:17:04.691416+0000","last_became_peered":"2024-09-10T21:17:04.691416+0000","last_unstale":"2024-09-10T21:17:06.789309+0000","last_undegraded":"2024-09-10T21:17:06.789309+0000","last_fullsized":"2024-09-10T21:17:06.789309+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:32.884545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790432+0000","last_change":"2024-09-10T21:17:05.726304+0000","last_active":"2024-09-10T21:17:06.790432+0000","last_peered":"2024-09-10T21:17:06.790432+0000","last_clean":"2024-09-10T21:17:06.790432+0000","last_became_active":"2024-09-10T21:17:05.725913+0000","last_became_peered":"2024-09-10T21:17:05.725913+0000","last_unstale":"2024-09-10T21:17:06.790432+0000","last_undegraded":"2024-09-10T21:17:06.790432+0000","last_fullsized":"2024-09-10T21:17:06.790432+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:48.032460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789998+0000","last_change":"2024-09-10T21:17:04.697705+0000","last_active":"2024-09-10T21:17:06.789998+0000","last_peered":"2024-09-10T21:17:06.789998+0000","last_clean":"2024-09-10T21:17:06.789998+0000","last_became_active":"2024-09-10T21:17:04.697166+0000","last_became_peered":"2024-09-10T21:17:04.697166+0000","last_unstale":"2024-09-10T21:17:06.789998+0000","last_undegraded":"2024-09-10T21:17:06.789998+0000","last_fullsized":"2024-09-10T21:17:06.789998+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:09:54.646678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714058+0000","last_change":"2024-09-10T21:17:05.713757+0000","last_active":"2024-09-10T21:17:06.714058+0000","last_peered":"2024-09-10T21:17:06.714058+0000","last_clean":"2024-09-10T21:17:06.714058+0000","last_became_active":"2024-09-10T21:17:05.713071+0000","last_became_peered":"2024-09-10T21:17:05.713071+0000","last_unstale":"2024-09-10T21:17:06.714058+0000","last_undegraded":"2024-09-10T21:17:06.714058+0000","last_fullsized":"2024-09-10T21:17:06.714058+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:45.777851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"103'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725726+0000","last_change":"2024-09-10T21:17:04.688907+0000","last_active":"2024-09-10T21:17:06.725726+0000","last_peered":"2024-09-10T21:17:06.725726+0000","last_clean":"2024-09-10T21:17:06.725726+0000","last_became_active":"2024-09-10T21:17:04.688614+0000","last_became_peered":"2024-09-10T21:17:04.688614+0000","last_unstale":"2024-09-10T21:17:06.725726+0000","last_undegraded":"2024-09-10T21:17:06.725726+0000","last_fullsized":"2024-09-10T21:17:06.725726+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:02.464966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.715053+0000","last_change":"2024-09-10T21:17:05.724980+0000","last_active":"2024-09-10T21:17:06.715053+0000","last_peered":"2024-09-10T21:17:06.715053+0000","last_clean":"2024-09-10T21:17:06.715053+0000","last_became_active":"2024-09-10T21:17:05.724702+0000","last_became_peered":"2024-09-10T21:17:05.724702+0000","last_unstale":"2024-09-10T21:17:06.715053+0000","last_undegraded":"2024-09-10T21:17:06.715053+0000","last_fullsized":"2024-09-10T21:17:06.715053+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:55.403243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789931+0000","last_change":"2024-09-10T21:17:04.686893+0000","last_active":"2024-09-10T21:17:06.789931+0000","last_peered":"2024-09-10T21:17:06.789931+0000","last_clean":"2024-09-10T21:17:06.789931+0000","last_became_active":"2024-09-10T21:17:04.686614+0000","last_became_peered":"2024-09-10T21:17:04.686614+0000","last_unstale":"2024-09-10T21:17:06.789931+0000","last_undegraded":"2024-09-10T21:17:06.789931+0000","last_fullsized":"2024-09-10T21:17:06.789931+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:54.901346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790613+0000","last_change":"2024-09-10T21:17:05.736758+0000","last_active":"2024-09-10T21:17:06.790613+0000","last_peered":"2024-09-10T21:17:06.790613+0000","last_clean":"2024-09-10T21:17:06.790613+0000","last_became_active":"2024-09-10T21:17:05.736377+0000","last_became_peered":"2024-09-10T21:17:05.736377+0000","last_unstale":"2024-09-10T21:17:06.790613+0000","last_undegraded":"2024-09-10T21:17:06.790613+0000","last_fullsized":"2024-09-10T21:17:06.790613+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:37.932179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790178+0000","last_change":"2024-09-10T21:17:04.687679+0000","last_active":"2024-09-10T21:17:06.790178+0000","last_peered":"2024-09-10T21:17:06.790178+0000","last_clean":"2024-09-10T21:17:06.790178+0000","last_became_active":"2024-09-10T21:17:04.687377+0000","last_became_peered":"2024-09-10T21:17:04.687377+0000","last_unstale":"2024-09-10T21:17:06.790178+0000","last_undegraded":"2024-09-10T21:17:06.790178+0000","last_fullsized":"2024-09-10T21:17:06.790178+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:55.233907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789705+0000","last_change":"2024-09-10T21:17:05.719517+0000","last_active":"2024-09-10T21:17:06.789705+0000","last_peered":"2024-09-10T21:17:06.789705+0000","last_clean":"2024-09-10T21:17:06.789705+0000","last_became_active":"2024-09-10T21:17:05.719246+0000","last_became_peered":"2024-09-10T21:17:05.719246+0000","last_unstale":"2024-09-10T21:17:06.789705+0000","last_undegraded":"2024-09-10T21:17:06.789705+0000","last_fullsized":"2024-09-10T21:17:06.789705+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:03.138945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.725801+0000","last_change":"2024-09-10T21:17:04.690559+0000","last_active":"2024-09-10T21:17:06.725801+0000","last_peered":"2024-09-10T21:17:06.725801+0000","last_clean":"2024-09-10T21:17:06.725801+0000","last_became_active":"2024-09-10T21:17:04.690280+0000","last_became_peered":"2024-09-10T21:17:04.690280+0000","last_unstale":"2024-09-10T21:17:06.725801+0000","last_undegraded":"2024-09-10T21:17:06.725801+0000","last_fullsized":"2024-09-10T21:17:06.725801+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:18.613917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790886+0000","last_change":"2024-09-10T21:17:05.725184+0000","last_active":"2024-09-10T21:17:06.790886+0000","last_peered":"2024-09-10T21:17:06.790886+0000","last_clean":"2024-09-10T21:17:06.790886+0000","last_became_active":"2024-09-10T21:17:05.724849+0000","last_became_peered":"2024-09-10T21:17:05.724849+0000","last_unstale":"2024-09-10T21:17:06.790886+0000","last_undegraded":"2024-09-10T21:17:06.790886+0000","last_fullsized":"2024-09-10T21:17:06.790886+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:22.263637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790563+0000","last_change":"2024-09-10T21:17:04.705593+0000","last_active":"2024-09-10T21:17:06.790563+0000","last_peered":"2024-09-10T21:17:06.790563+0000","last_clean":"2024-09-10T21:17:06.790563+0000","last_became_active":"2024-09-10T21:17:04.705280+0000","last_became_peered":"2024-09-10T21:17:04.705280+0000","last_unstale":"2024-09-10T21:17:06.790563+0000","last_undegraded":"2024-09-10T21:17:06.790563+0000","last_fullsized":"2024-09-10T21:17:06.790563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:24.189910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.790383+0000","last_change":"2024-09-10T21:17:05.717785+0000","last_active":"2024-09-10T21:17:06.790383+0000","last_peered":"2024-09-10T21:17:06.790383+0000","last_clean":"2024-09-10T21:17:06.790383+0000","last_became_active":"2024-09-10T21:17:05.717478+0000","last_became_peered":"2024-09-10T21:17:05.717478+0000","last_unstale":"2024-09-10T21:17:06.790383+0000","last_undegraded":"2024-09-10T21:17:06.790383+0000","last_fullsized":"2024-09-10T21:17:06.790383+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:24.730712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.714390+0000","last_change":"2024-09-10T21:17:04.698439+0000","last_active":"2024-09-10T21:17:06.714390+0000","last_peered":"2024-09-10T21:17:06.714390+0000","last_clean":"2024-09-10T21:17:06.714390+0000","last_became_active":"2024-09-10T21:17:04.697954+0000","last_became_peered":"2024-09-10T21:17:04.697954+0000","last_unstale":"2024-09-10T21:17:06.714390+0000","last_undegraded":"2024-09-10T21:17:06.714390+0000","last_fullsized":"2024-09-10T21:17:06.714390+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:58.728826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713776+0000","last_change":"2024-09-10T21:17:05.724864+0000","last_active":"2024-09-10T21:17:06.713776+0000","last_peered":"2024-09-10T21:17:06.713776+0000","last_clean":"2024-09-10T21:17:06.713776+0000","last_became_active":"2024-09-10T21:17:05.724517+0000","last_became_peered":"2024-09-10T21:17:05.724517+0000","last_unstale":"2024-09-10T21:17:06.713776+0000","last_undegraded":"2024-09-10T21:17:06.713776+0000","last_fullsized":"2024-09-10T21:17:06.713776+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:08.115164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713654+0000","last_change":"2024-09-10T21:17:04.683757+0000","last_active":"2024-09-10T21:17:06.713654+0000","last_peered":"2024-09-10T21:17:06.713654+0000","last_clean":"2024-09-10T21:17:06.713654+0000","last_became_active":"2024-09-10T21:17:04.683246+0000","last_became_peered":"2024-09-10T21:17:04.683246+0000","last_unstale":"2024-09-10T21:17:06.713654+0000","last_undegraded":"2024-09-10T21:17:06.713654+0000","last_fullsized":"2024-09-10T21:17:06.713654+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:20.795469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789230+0000","last_change":"2024-09-10T21:17:05.722935+0000","last_active":"2024-09-10T21:17:06.789230+0000","last_peered":"2024-09-10T21:17:06.789230+0000","last_clean":"2024-09-10T21:17:06.789230+0000","last_became_active":"2024-09-10T21:17:05.718877+0000","last_became_peered":"2024-09-10T21:17:05.718877+0000","last_unstale":"2024-09-10T21:17:06.789230+0000","last_undegraded":"2024-09-10T21:17:06.789230+0000","last_fullsized":"2024-09-10T21:17:06.789230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:04.668983+0000","last_clean_scrub_stamp":"2024-09-10T21:17:04.668983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:07.613681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.713332+0000","last_change":"2024-09-10T21:17:04.689707+0000","last_active":"2024-09-10T21:17:06.713332+0000","last_peered":"2024-09-10T21:17:06.713332+0000","last_clean":"2024-09-10T21:17:06.713332+0000","last_became_active":"2024-09-10T21:17:04.689349+0000","last_became_peered":"2024-09-10T21:17:04.689349+0000","last_unstale":"2024-09-10T21:17:06.713332+0000","last_undegraded":"2024-09-10T21:17:06.713332+0000","last_fullsized":"2024-09-10T21:17:06.713332+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:03.661814+0000","last_clean_scrub_stamp":"2024-09-10T21:17:03.661814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:55.356058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":255,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T21:17:06.789446+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:17:06.789446+0000","last_peered":"2024-09-10T21:17:06.789446+0000","last_clean":"2024-09-10T21:17:06.789446+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:17:06.789446+0000","last_undegraded":"2024-09-10T21:17:06.789446+0000","last_fullsized":"2024-09-10T21:17:06.789446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509525,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36592,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335248,"statfs":{"total":96636764160,"available":96599293952,"internally_reserved":0,"allocated":1273856,"data_stored":544636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509527,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1875968,"data_stored":1135096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509525,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36608,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335232,"statfs":{"total":96636764160,"available":96599277568,"internally_reserved":0,"allocated":1290240,"data_stored":546338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509524,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1273856,"data_stored":544636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509526,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1286144,"data_stored":545274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509525,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1282048,"data_stored":545288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542229,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1294336,"data_stored":546372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542228,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37172,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334668,"statfs":{"total":96636764160,"available":96598700032,"internally_reserved":0,"allocated":1867776,"data_stored":1135004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:17:10.272 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:17:10.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:17:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 -- 172.21.15.16:0/3037907693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f88004810 msgr2=0x7f9f88004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 --2- 172.21.15.16:0/3037907693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f88004810 0x7f9f88004c90 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c006f70 tx=0x7f9f8c01ce50 comp rx=0 tx=0).stop 2024-09-10T21:17:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 -- 172.21.15.16:0/3037907693 shutdown_connections 2024-09-10T21:17:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 --2- 172.21.15.16:0/3037907693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f88004810 0x7f9f88004c90 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 -- 172.21.15.16:0/3037907693 >> 172.21.15.16:0/3037907693 conn(0x7f9fa00d1b70 msgr2=0x7f9fa00d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 -- 172.21.15.16:0/3037907693 shutdown_connections 2024-09-10T21:17:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 -- 172.21.15.16:0/3037907693 wait complete. 2024-09-10T21:17:10.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 Processor -- start 2024-09-10T21:17:10.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 -- start start 2024-09-10T21:17:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f88004810 0x7f9fa009d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa00ca420 0x7f9fa01f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa00ca8a0 0x7f9fa01f9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa00cf120 con 0x7f9f88004810 2024-09-10T21:17:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa00cefa0 con 0x7f9fa00ca420 2024-09-10T21:17:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.465+0000 7f9fa7000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa00cc5d0 con 0x7f9fa00ca8a0 2024-09-10T21:17:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f88004810 0x7f9fa009d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f88004810 0x7f9fa009d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54422/0 (socket says 172.21.15.16:54422) 2024-09-10T21:17:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa5c00640 1 -- 172.21.15.16:0/541910370 learned_addr learned my addr 172.21.15.16:0/541910370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa5200640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa00ca420 0x7f9fa01f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa6600640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa00ca8a0 0x7f9fa01f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa5c00640 1 -- 172.21.15.16:0/541910370 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa00ca8a0 msgr2=0x7f9fa01f9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa5c00640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa00ca8a0 0x7f9fa01f9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa5c00640 1 -- 172.21.15.16:0/541910370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa00ca420 msgr2=0x7f9fa01f8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa5c00640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa00ca420 0x7f9fa01f8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa5c00640 1 -- 172.21.15.16:0/541910370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa01f9880 con 0x7f9f88004810 2024-09-10T21:17:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa5c00640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f88004810 0x7f9fa009d100 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9f94003e30 tx=0x7f9f94001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:10.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9f9d600640 1 -- 172.21.15.16:0/541910370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94015070 con 0x7f9f88004810 2024-09-10T21:17:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9f9d600640 1 -- 172.21.15.16:0/541910370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f94011920 con 0x7f9f88004810 2024-09-10T21:17:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9f9d600640 1 -- 172.21.15.16:0/541910370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94011d70 con 0x7f9f88004810 2024-09-10T21:17:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa7000640 1 -- 172.21.15.16:0/541910370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa00b0b70 con 0x7f9f88004810 2024-09-10T21:17:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa7000640 1 -- 172.21.15.16:0/541910370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa00b1130 con 0x7f9f88004810 2024-09-10T21:17:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9fa7000640 1 -- 172.21.15.16:0/541910370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa00ccb30 con 0x7f9f88004810 2024-09-10T21:17:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.469+0000 7f9f9d600640 1 -- 172.21.15.16:0/541910370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9f940103c0 con 0x7f9f88004810 2024-09-10T21:17:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.473+0000 7f9f9d600640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9f80072610 0x7f9f80074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.473+0000 7f9f9d600640 1 -- 172.21.15.16:0/541910370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f9f940942d0 con 0x7f9f88004810 2024-09-10T21:17:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.473+0000 7f9fa5200640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9f80072610 0x7f9f80074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.473+0000 7f9fa5200640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9f80072610 0x7f9f80074ad0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0ac000 tx=0x7f9f8c005040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.477+0000 7f9f9d600640 1 -- 172.21.15.16:0/541910370 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9f9405f2c0 con 0x7f9f88004810 2024-09-10T21:17:10.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.713+0000 7f9fa7000640 1 -- 172.21.15.16:0/541910370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9fa00ce360 con 0x7f9f88004810 2024-09-10T21:17:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.717+0000 7f9f9d600640 1 -- 172.21.15.16:0/541910370 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v28) ==== 190+0+147 (secure 0 0 0) 0x7f9f94063cf0 con 0x7f9f88004810 2024-09-10T21:17:10.721 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-10T21:17:10.721 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA== 2024-09-10T21:17:10.721 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-10T21:17:10.721 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-10T21:17:10.721 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:17:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-10T21:17:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.717+0000 7f9fa7000640 1 -- 172.21.15.16:0/541910370 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9f80072610 msgr2=0x7f9f80074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.717+0000 7f9fa7000640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9f80072610 0x7f9f80074ad0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0ac000 tx=0x7f9f8c005040 comp rx=0 tx=0).stop 2024-09-10T21:17:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.717+0000 7f9fa7000640 1 -- 172.21.15.16:0/541910370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f88004810 msgr2=0x7f9fa009d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.717+0000 7f9fa7000640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f88004810 0x7f9fa009d100 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9f94003e30 tx=0x7f9f94001810 comp rx=0 tx=0).stop 2024-09-10T21:17:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.717+0000 7f9fa7000640 1 -- 172.21.15.16:0/541910370 shutdown_connections 2024-09-10T21:17:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.717+0000 7f9fa7000640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9f80072610 0x7f9f80074ad0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.717+0000 7f9fa7000640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fa00ca8a0 0x7f9fa01f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.717+0000 7f9fa7000640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa00ca420 0x7f9fa01f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.717+0000 7f9fa7000640 1 --2- 172.21.15.16:0/541910370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f88004810 0x7f9fa009d100 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.717+0000 7f9fa7000640 1 -- 172.21.15.16:0/541910370 >> 172.21.15.16:0/541910370 conn(0x7f9fa00d1b70 msgr2=0x7f9fa00ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.717+0000 7f9fa7000640 1 -- 172.21.15.16:0/541910370 shutdown_connections 2024-09-10T21:17:10.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.717+0000 7f9fa7000640 1 -- 172.21.15.16:0/541910370 wait complete. 2024-09-10T21:17:10.745 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.889+0000 7fd277600640 1 -- 172.21.15.16:0/1326294860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2701599d0 msgr2=0x7fd27015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.889+0000 7fd277600640 1 --2- 172.21.15.16:0/1326294860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2701599d0 0x7fd27015f830 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fd27006fd80 tx=0x7fd26401c830 comp rx=0 tx=0).stop 2024-09-10T21:17:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.889+0000 7fd277600640 1 -- 172.21.15.16:0/1326294860 shutdown_connections 2024-09-10T21:17:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.889+0000 7fd277600640 1 --2- 172.21.15.16:0/1326294860 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd2700ca6b0 0x7fd2700726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.889+0000 7fd277600640 1 --2- 172.21.15.16:0/1326294860 >> v2:172.21.15.52:3300/0 conn(0x7fd27006e640 0x7fd2700ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.889+0000 7fd277600640 1 --2- 172.21.15.16:0/1326294860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2701599d0 0x7fd27015f830 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.889+0000 7fd277600640 1 -- 172.21.15.16:0/1326294860 >> 172.21.15.16:0/1326294860 conn(0x7fd2700cfb20 msgr2=0x7fd2700cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.889+0000 7fd277600640 1 -- 172.21.15.16:0/1326294860 shutdown_connections 2024-09-10T21:17:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.889+0000 7fd277600640 1 -- 172.21.15.16:0/1326294860 wait complete. 2024-09-10T21:17:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.889+0000 7fd277600640 1 Processor -- start 2024-09-10T21:17:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.889+0000 7fd277600640 1 -- start start 2024-09-10T21:17:10.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd277600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd27006e640 0x7fd270156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd277600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2700ca6b0 0x7fd27015b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd277600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd2701599d0 0x7fd27015b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd277600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd270074200 con 0x7fd27006e640 2024-09-10T21:17:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd277600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd270074080 con 0x7fd2700ca6b0 2024-09-10T21:17:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd277600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd270074380 con 0x7fd2701599d0 2024-09-10T21:17:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd274c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd27006e640 0x7fd270156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd274c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd27006e640 0x7fd270156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54434/0 (socket says 172.21.15.16:54434) 2024-09-10T21:17:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd274c00640 1 -- 172.21.15.16:0/209204459 learned_addr learned my addr 172.21.15.16:0/209204459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd26fe00640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2700ca6b0 0x7fd27015b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd275600640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd2701599d0 0x7fd27015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd274c00640 1 -- 172.21.15.16:0/209204459 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd2701599d0 msgr2=0x7fd27015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd274c00640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd2701599d0 0x7fd27015b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd274c00640 1 -- 172.21.15.16:0/209204459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2700ca6b0 msgr2=0x7fd27015b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd274c00640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2700ca6b0 0x7fd27015b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd274c00640 1 -- 172.21.15.16:0/209204459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2700b4660 con 0x7fd27006e640 2024-09-10T21:17:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd274c00640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd27006e640 0x7fd270156000 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd2640b0040 tx=0x7fd2640a6750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd26cc00640 1 -- 172.21.15.16:0/209204459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264004340 con 0x7fd27006e640 2024-09-10T21:17:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd26cc00640 1 -- 172.21.15.16:0/209204459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd264005020 con 0x7fd27006e640 2024-09-10T21:17:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd277600640 1 -- 172.21.15.16:0/209204459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2700b4940 con 0x7fd27006e640 2024-09-10T21:17:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd26cc00640 1 -- 172.21.15.16:0/209204459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264005470 con 0x7fd27006e640 2024-09-10T21:17:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd277600640 1 -- 172.21.15.16:0/209204459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd270155800 con 0x7fd27006e640 2024-09-10T21:17:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd277600640 1 -- 172.21.15.16:0/209204459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2700c6d80 con 0x7fd27006e640 2024-09-10T21:17:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.893+0000 7fd26cc00640 1 -- 172.21.15.16:0/209204459 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd2640046d0 con 0x7fd27006e640 2024-09-10T21:17:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.897+0000 7fd26cc00640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd2500725e0 0x7fd250074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.897+0000 7fd26cc00640 1 -- 172.21.15.16:0/209204459 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fd26412d560 con 0x7fd27006e640 2024-09-10T21:17:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.897+0000 7fd26fe00640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd2500725e0 0x7fd250074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.897+0000 7fd26fe00640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd2500725e0 0x7fd250074aa0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd270151730 tx=0x7fd25c00b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:10.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:10.901+0000 7fd26cc00640 1 -- 172.21.15.16:0/209204459 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd2640164b0 con 0x7fd27006e640 2024-09-10T21:17:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.105+0000 7fd277600640 1 -- 172.21.15.16:0/209204459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2700748e0 con 0x7fd27006e640 2024-09-10T21:17:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.105+0000 7fd26cc00640 1 -- 172.21.15.16:0/209204459 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5235 (secure 0 0 0) 0x7fd2640f8bf0 con 0x7fd27006e640 2024-09-10T21:17:11.110 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:11.111 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"btime":"2024-09-10T21:17:08:717481+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3765085271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765085271},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765085271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":7104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4198210360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":4198210360},{"type":"v1","addr":"172.21.15.16:6837","nonce":4198210360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2954517211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2954517211},{"type":"v1","addr":"172.21.15.52:6835","nonce":2954517211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:05.094283+0000","modified":"2024-09-10T21:17:08.119983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135926":{"gid":135926,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1108687782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108687782},{"type":"v1","addr":"172.21.15.52:6834","nonce":1108687782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135926,"qdb_cluster":[135926]},"id":8}]} 2024-09-10T21:17:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 71 2024-09-10T21:17:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.109+0000 7fd277600640 1 -- 172.21.15.16:0/209204459 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd2500725e0 msgr2=0x7fd250074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.109+0000 7fd277600640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd2500725e0 0x7fd250074aa0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd270151730 tx=0x7fd25c00b2e0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.109+0000 7fd277600640 1 -- 172.21.15.16:0/209204459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd27006e640 msgr2=0x7fd270156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.109+0000 7fd277600640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd27006e640 0x7fd270156000 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd2640b0040 tx=0x7fd2640a6750 comp rx=0 tx=0).stop 2024-09-10T21:17:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.109+0000 7fd277600640 1 -- 172.21.15.16:0/209204459 shutdown_connections 2024-09-10T21:17:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.109+0000 7fd277600640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd2500725e0 0x7fd250074aa0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.109+0000 7fd277600640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd2701599d0 0x7fd27015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.109+0000 7fd277600640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2700ca6b0 0x7fd27015b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.109+0000 7fd277600640 1 --2- 172.21.15.16:0/209204459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd27006e640 0x7fd270156000 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.109+0000 7fd277600640 1 -- 172.21.15.16:0/209204459 >> 172.21.15.16:0/209204459 conn(0x7fd2700cfb20 msgr2=0x7fd2700cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.109+0000 7fd277600640 1 -- 172.21.15.16:0/209204459 shutdown_connections 2024-09-10T21:17:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.109+0000 7fd277600640 1 -- 172.21.15.16:0/209204459 wait complete. 2024-09-10T21:17:11.535 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:17:05.094283+0000', 'modified': '2024-09-10T21:17:08.119983+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 135926}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_135926': {'gid': 135926, 'name': 'b', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.52:6834/1108687782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6832', 'nonce': 1108687782}, {'type': 'v1', 'addr': '172.21.15.52:6834', 'nonce': 1108687782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 135926, 'qdb_cluster': [135926]} 2024-09-10T21:17:11.536 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:17:11.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T21:17:11.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.685+0000 7fa55de00640 1 -- 172.21.15.16:0/1341302475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa55806eb80 msgr2=0x7fa5580700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.685+0000 7fa55de00640 1 --2- 172.21.15.16:0/1341302475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa55806eb80 0x7fa5580700a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa540009840 tx=0x7fa54001c820 comp rx=0 tx=0).stop 2024-09-10T21:17:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.685+0000 7fa55de00640 1 -- 172.21.15.16:0/1341302475 shutdown_connections 2024-09-10T21:17:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.685+0000 7fa55de00640 1 --2- 172.21.15.16:0/1341302475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa55806eb80 0x7fa5580700a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.685+0000 7fa55de00640 1 --2- 172.21.15.16:0/1341302475 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa55815a270 0x7fa55806e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.685+0000 7fa55de00640 1 -- 172.21.15.16:0/1341302475 >> 172.21.15.16:0/1341302475 conn(0x7fa5580cfb20 msgr2=0x7fa5580cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.685+0000 7fa55de00640 1 -- 172.21.15.16:0/1341302475 shutdown_connections 2024-09-10T21:17:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.685+0000 7fa55de00640 1 -- 172.21.15.16:0/1341302475 wait complete. 2024-09-10T21:17:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa55de00640 1 Processor -- start 2024-09-10T21:17:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa55de00640 1 -- start start 2024-09-10T21:17:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa55de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa55806eb80 0x7fa5580b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa55de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5580705e0 0x7fa5580c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa55de00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa55815a270 0x7fa5580c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa55de00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa558072ed0 con 0x7fa5580705e0 2024-09-10T21:17:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa55de00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa558072d50 con 0x7fa55806eb80 2024-09-10T21:17:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa55de00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa558073050 con 0x7fa55815a270 2024-09-10T21:17:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa54fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5580705e0 0x7fa5580c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa54fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5580705e0 0x7fa5580c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54446/0 (socket says 172.21.15.16:54446) 2024-09-10T21:17:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa54fe00640 1 -- 172.21.15.16:0/2683793143 learned_addr learned my addr 172.21.15.16:0/2683793143 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa556a00640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa55806eb80 0x7fa5580b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa557400640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa55815a270 0x7fa5580c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa54fe00640 1 -- 172.21.15.16:0/2683793143 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa55815a270 msgr2=0x7fa5580c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa54fe00640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa55815a270 0x7fa5580c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa54fe00640 1 -- 172.21.15.16:0/2683793143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa55806eb80 msgr2=0x7fa5580b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa54fe00640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa55806eb80 0x7fa5580b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa54fe00640 1 -- 172.21.15.16:0/2683793143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55814cc10 con 0x7fa5580705e0 2024-09-10T21:17:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa556a00640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa55806eb80 0x7fa5580b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa54fe00640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5580705e0 0x7fa5580c8dc0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa540009790 tx=0x7fa540005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa54f400640 1 -- 172.21.15.16:0/2683793143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5400b2040 con 0x7fa5580705e0 2024-09-10T21:17:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa54f400640 1 -- 172.21.15.16:0/2683793143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa540004320 con 0x7fa5580705e0 2024-09-10T21:17:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa55de00640 1 -- 172.21.15.16:0/2683793143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55814cef0 con 0x7fa5580705e0 2024-09-10T21:17:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa54f400640 1 -- 172.21.15.16:0/2683793143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5400044f0 con 0x7fa5580705e0 2024-09-10T21:17:11.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.689+0000 7fa55de00640 1 -- 172.21.15.16:0/2683793143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55814d4b0 con 0x7fa5580705e0 2024-09-10T21:17:11.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa54f400640 1 -- 172.21.15.16:0/2683793143 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa5400b5030 con 0x7fa5580705e0 2024-09-10T21:17:11.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa561157000 1 Processor -- start 2024-09-10T21:17:11.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa54f400640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5340725e0 0x7fa534074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa561157000 1 -- start start 2024-09-10T21:17:11.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa54f400640 1 -- 172.21.15.16:0/2683793143 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fa54012c990 con 0x7fa5580705e0 2024-09-10T21:17:11.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa557400640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5340725e0 0x7fa534074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa557400640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5340725e0 0x7fa534074aa0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa5580709e0 tx=0x7fa550009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa561157000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8949d0 0x55ad9d894db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa561157000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 0x55ad9d8488d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa561157000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d848e10 0x55ad9d8491f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa561157000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d7693f0 con 0x55ad9d8952f0 2024-09-10T21:17:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa561157000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d6a6b20 con 0x55ad9d8949d0 2024-09-10T21:17:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa561157000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55ad9d695f50 con 0x55ad9d848e10 2024-09-10T21:17:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.693+0000 7fa54fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 0x55ad9d8488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa54fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 0x55ad9d8488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54462/0 (socket says 172.21.15.16:54462) 2024-09-10T21:17:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa54fe00640 1 -- 172.21.15.16:0/2357138873 learned_addr learned my addr 172.21.15.16:0/2357138873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa556a00640 1 --2- 172.21.15.16:0/2357138873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8949d0 0x55ad9d894db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa557400640 1 --2- 172.21.15.16:0/2357138873 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d848e10 0x55ad9d8491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa54fe00640 1 -- 172.21.15.16:0/2357138873 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d848e10 msgr2=0x55ad9d8491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa54fe00640 1 --2- 172.21.15.16:0/2357138873 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d848e10 0x55ad9d8491f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa54fe00640 1 -- 172.21.15.16:0/2357138873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8949d0 msgr2=0x55ad9d894db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa54fe00640 1 --2- 172.21.15.16:0/2357138873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8949d0 0x55ad9d894db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa54fe00640 1 -- 172.21.15.16:0/2357138873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad9d65d590 con 0x55ad9d8952f0 2024-09-10T21:17:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa557400640 1 --2- 172.21.15.16:0/2357138873 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d848e10 0x55ad9d8491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa54fe00640 1 --2- 172.21.15.16:0/2357138873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 0x55ad9d8488d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa5400f7590 tx=0x7fa5400f72a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeea359ae924906a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa547e00640 1 -- 172.21.15.16:0/2357138873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5400f7890 con 0x55ad9d8952f0 2024-09-10T21:17:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa547e00640 1 -- 172.21.15.16:0/2357138873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5400fc370 con 0x55ad9d8952f0 2024-09-10T21:17:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa547e00640 1 -- 172.21.15.16:0/2357138873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540100d90 con 0x55ad9d8952f0 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 -- 172.21.15.16:0/2357138873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 msgr2=0x55ad9d8488d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 --2- 172.21.15.16:0/2357138873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 0x55ad9d8488d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa5400f7590 tx=0x7fa5400f72a0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 -- 172.21.15.16:0/2357138873 shutdown_connections 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 --2- 172.21.15.16:0/2357138873 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d848e10 0x55ad9d8491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 --2- 172.21.15.16:0/2357138873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 0x55ad9d8488d0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 --2- 172.21.15.16:0/2357138873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8949d0 0x55ad9d894db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 -- 172.21.15.16:0/2357138873 >> 172.21.15.16:0/2357138873 conn(0x55ad9d732130 msgr2=0x55ad9d877e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 -- 172.21.15.16:0/2357138873 shutdown_connections 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 -- 172.21.15.16:0/2357138873 wait complete. 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 -- start start 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 Processor -- start 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 0x55ad9d84a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 0x55ad9d84a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8952f0 0x55ad9d84acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d6305d0 con 0x55ad9d848e10 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d699d60 con 0x55ad9d8952f0 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55ad9d63c500 con 0x55ad9d8949d0 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa561157000 10 client.0 _ll_register_callbacks cb 0x55ad9d640810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa556a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 0x55ad9d84a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa556a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 0x55ad9d84a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54474/0 (socket says 172.21.15.16:54474) 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa556a00640 1 -- 172.21.15.16:0/2700509381 learned_addr learned my addr 172.21.15.16:0/2700509381 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa557400640 1 --2- 172.21.15.16:0/2700509381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8952f0 0x55ad9d84acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.697+0000 7fa54fe00640 1 --2- 172.21.15.16:0/2700509381 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 0x55ad9d84a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa556a00640 1 -- 172.21.15.16:0/2700509381 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 msgr2=0x55ad9d84a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa556a00640 1 --2- 172.21.15.16:0/2700509381 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 0x55ad9d84a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa556a00640 1 -- 172.21.15.16:0/2700509381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8952f0 msgr2=0x55ad9d84acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa556a00640 1 --2- 172.21.15.16:0/2700509381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8952f0 0x55ad9d84acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa556a00640 1 -- 172.21.15.16:0/2700509381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9d634170 con 0x55ad9d848e10 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa556a00640 1 --2- 172.21.15.16:0/2700509381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 0x55ad9d84a240 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa548013790 tx=0x7fa548013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46b947f3419f835 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa544c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa544c00640 1 -- 172.21.15.16:0/2700509381 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548020060 con 0x55ad9d848e10 2024-09-10T21:17:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa544c00640 1 -- 172.21.15.16:0/2700509381 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fa548011dd0 con 0x55ad9d848e10 2024-09-10T21:17:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 1 -- 172.21.15.16:0/2700509381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ad9d62e0f0 con 0x55ad9d848e10 2024-09-10T21:17:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa544c00640 1 -- 172.21.15.16:0/2700509381 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fa548026050 con 0x55ad9d848e10 2024-09-10T21:17:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 10 client.7224 fetch_fsmap learned FSMap version 71 2024-09-10T21:17:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 1 -- 172.21.15.16:0/2700509381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55ad9d71aef0 con 0x55ad9d848e10 2024-09-10T21:17:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa544c00640 1 -- 172.21.15.16:0/2700509381 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ad9d62e0f0 con 0x55ad9d848e10 2024-09-10T21:17:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa544c00640 1 -- 172.21.15.16:0/2700509381 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7fa548029020 con 0x55ad9d848e10 2024-09-10T21:17:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 10 client.7224 fetch_fsmap finished waiting for FSMap version 71 2024-09-10T21:17:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 10 client.7224 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:17:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 20 client.7224 populate_metadata read hostname 'smithi016' 2024-09-10T21:17:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 1 --2- 172.21.15.16:0/2700509381 >> [v2:172.21.15.52:6832/1108687782,v1:172.21.15.52:6834/1108687782] conn(0x55ad9d8a3c70 0x55ad9d8a6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 4 client.7224 mds_command: new command op to 135926 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 1 -- 172.21.15.16:0/2700509381 --> [v2:172.21.15.52:6832/1108687782,v1:172.21.15.52:6834/1108687782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad9d6a8a70 con 0x55ad9d8a3c70 2024-09-10T21:17:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa54fe00640 1 --2- 172.21.15.16:0/2700509381 >> [v2:172.21.15.52:6832/1108687782,v1:172.21.15.52:6834/1108687782] conn(0x55ad9d8a3c70 0x55ad9d8a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa54fe00640 1 --2- 172.21.15.16:0/2700509381 >> [v2:172.21.15.52:6832/1108687782,v1:172.21.15.52:6834/1108687782] conn(0x55ad9d8a3c70 0x55ad9d8a6060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a9e3e088ba767d server_cookie=938efe14d35c526 in_seq=0 out_seq=0 2024-09-10T21:17:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa544c00640 10 client.7224 ms_handle_connect on v2:172.21.15.52:6832/1108687782 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa544c00640 1 -- 172.21.15.16:0/2700509381 <== mds.0 v2:172.21.15.52:6832/1108687782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ad9d6a8a70 con 0x55ad9d8a3c70 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa544c00640 10 client.7224 handle_command_reply: tid=0 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 1 client.7224 shutdown 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 1 -- 172.21.15.16:0/2700509381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 msgr2=0x55ad9d84a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 1 --2- 172.21.15.16:0/2700509381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 0x55ad9d84a240 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa548013790 tx=0x7fa548013c50 comp rx=0 tx=0).stop 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 1 -- 172.21.15.16:0/2700509381 shutdown_connections 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 1 --2- 172.21.15.16:0/2700509381 >> [v2:172.21.15.52:6832/1108687782,v1:172.21.15.52:6834/1108687782] conn(0x55ad9d8a3c70 0x55ad9d8a6060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 1 --2- 172.21.15.16:0/2700509381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8952f0 0x55ad9d84acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 1 --2- 172.21.15.16:0/2700509381 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 0x55ad9d84a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 1 --2- 172.21.15.16:0/2700509381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 0x55ad9d84a240 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa544c00640 0 client.7224 ms_handle_reset on v2:172.21.15.52:6832/1108687782 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.701+0000 7fa561157000 1 -- 172.21.15.16:0/2700509381 >> 172.21.15.16:0/2700509381 conn(0x55ad9d732130 msgr2=0x55ad9d847fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.705+0000 7fa561157000 1 -- 172.21.15.16:0/2700509381 shutdown_connections 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.705+0000 7fa561157000 1 -- 172.21.15.16:0/2700509381 wait complete. 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.705+0000 7fa561157000 20 client.7224 trim_cache size 0 max 16384 2024-09-10T21:17:11.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.721+0000 7fa561157000 1 Processor -- start 2024-09-10T21:17:11.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.721+0000 7fa561157000 1 -- start start 2024-09-10T21:17:11.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.721+0000 7fa561157000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 0x55ad9d906fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.721+0000 7fa561157000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 0x55ad9d9074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.721+0000 7fa561157000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8952f0 0x55ad9d907a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.721+0000 7fa561157000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5581fc250 con 0x55ad9d848e10 2024-09-10T21:17:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.721+0000 7fa561157000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa558202a20 con 0x55ad9d8952f0 2024-09-10T21:17:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.721+0000 7fa561157000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55ad9d661ec0 con 0x55ad9d8949d0 2024-09-10T21:17:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.721+0000 7fa556a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 0x55ad9d906fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa556a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 0x55ad9d906fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54480/0 (socket says 172.21.15.16:54480) 2024-09-10T21:17:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa556a00640 1 -- 172.21.15.16:0/978141594 learned_addr learned my addr 172.21.15.16:0/978141594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa557400640 1 --2- 172.21.15.16:0/978141594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8952f0 0x55ad9d907a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa54fe00640 1 --2- 172.21.15.16:0/978141594 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 0x55ad9d9074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa556a00640 1 -- 172.21.15.16:0/978141594 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 msgr2=0x55ad9d9074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa556a00640 1 --2- 172.21.15.16:0/978141594 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 0x55ad9d9074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa556a00640 1 -- 172.21.15.16:0/978141594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8952f0 msgr2=0x55ad9d907a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa556a00640 1 --2- 172.21.15.16:0/978141594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8952f0 0x55ad9d907a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa556a00640 1 -- 172.21.15.16:0/978141594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad9d6f1d90 con 0x55ad9d848e10 2024-09-10T21:17:11.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa557400640 1 --2- 172.21.15.16:0/978141594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8952f0 0x55ad9d907a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:11.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa556a00640 1 --2- 172.21.15.16:0/978141594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 0x55ad9d906fb0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa548013760 tx=0x7fa5480097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27e93539dfd777 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa547400640 1 -- 172.21.15.16:0/978141594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548011bd0 con 0x55ad9d848e10 2024-09-10T21:17:11.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa547400640 1 -- 172.21.15.16:0/978141594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa548000e50 con 0x55ad9d848e10 2024-09-10T21:17:11.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa547400640 1 -- 172.21.15.16:0/978141594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54800fc40 con 0x55ad9d848e10 2024-09-10T21:17:11.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 -- 172.21.15.16:0/978141594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 msgr2=0x55ad9d906fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 --2- 172.21.15.16:0/978141594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 0x55ad9d906fb0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa548013760 tx=0x7fa5480097b0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 -- 172.21.15.16:0/978141594 shutdown_connections 2024-09-10T21:17:11.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 --2- 172.21.15.16:0/978141594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d8952f0 0x55ad9d907a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 --2- 172.21.15.16:0/978141594 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 0x55ad9d9074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 --2- 172.21.15.16:0/978141594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d848e10 0x55ad9d906fb0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 -- 172.21.15.16:0/978141594 >> 172.21.15.16:0/978141594 conn(0x55ad9d732130 msgr2=0x55ad9d8a6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 -- 172.21.15.16:0/978141594 shutdown_connections 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 -- 172.21.15.16:0/978141594 wait complete. 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 -- start start 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 Processor -- start 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d848e10 0x55ad9d8ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 0x55ad9d8ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 0x55ad9d8efa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d6a58f0 con 0x55ad9d8952f0 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d71aa60 con 0x55ad9d848e10 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55ad9d699070 con 0x55ad9d8949d0 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa561157000 10 client.0 _ll_register_callbacks cb 0x7fa548009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa557400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 0x55ad9d8efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa557400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 0x55ad9d8efa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54496/0 (socket says 172.21.15.16:54496) 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa557400640 1 -- 172.21.15.16:0/2093957274 learned_addr learned my addr 172.21.15.16:0/2093957274 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa556a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d848e10 0x55ad9d8ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.725+0000 7fa54fe00640 1 --2- 172.21.15.16:0/2093957274 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 0x55ad9d8ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa557400640 1 -- 172.21.15.16:0/2093957274 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 msgr2=0x55ad9d8ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa557400640 1 --2- 172.21.15.16:0/2093957274 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 0x55ad9d8ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa557400640 1 -- 172.21.15.16:0/2093957274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d848e10 msgr2=0x55ad9d8ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa557400640 1 --2- 172.21.15.16:0/2093957274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d848e10 0x55ad9d8ab160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa557400640 1 -- 172.21.15.16:0/2093957274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9d65ca90 con 0x55ad9d8952f0 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa556a00640 1 --2- 172.21.15.16:0/2093957274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d848e10 0x55ad9d8ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa557400640 1 --2- 172.21.15.16:0/2093957274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 0x55ad9d8efa20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa550015760 tx=0x7fa550011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd4d27127386dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa544c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa544c00640 1 -- 172.21.15.16:0/2093957274 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550027070 con 0x55ad9d8952f0 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa544c00640 1 -- 172.21.15.16:0/2093957274 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fa5500136b0 con 0x55ad9d8952f0 2024-09-10T21:17:11.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 -- 172.21.15.16:0/2093957274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa548000e50 con 0x55ad9d8952f0 2024-09-10T21:17:11.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa544c00640 1 -- 172.21.15.16:0/2093957274 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fa550032050 con 0x55ad9d8952f0 2024-09-10T21:17:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 10 client.7230 fetch_fsmap learned FSMap version 71 2024-09-10T21:17:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 -- 172.21.15.16:0/2093957274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55ad9d63b050 con 0x55ad9d8952f0 2024-09-10T21:17:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa544c00640 1 -- 172.21.15.16:0/2093957274 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa548000e50 con 0x55ad9d8952f0 2024-09-10T21:17:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa544c00640 1 -- 172.21.15.16:0/2093957274 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7fa550035020 con 0x55ad9d8952f0 2024-09-10T21:17:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 10 client.7230 fetch_fsmap finished waiting for FSMap version 71 2024-09-10T21:17:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 10 client.7230 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:17:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 20 client.7230 populate_metadata read hostname 'smithi016' 2024-09-10T21:17:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 --2- 172.21.15.16:0/2093957274 >> [v2:172.21.15.52:6832/1108687782,v1:172.21.15.52:6834/1108687782] conn(0x55ad9d90b200 0x55ad9d90b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 4 client.7230 mds_command: new command op to 135926 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:17:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 -- 172.21.15.16:0/2093957274 --> [v2:172.21.15.52:6832/1108687782,v1:172.21.15.52:6834/1108687782] -- command(tid 0: {"prefix": "status"}) -- 0x55ad9d6a4520 con 0x55ad9d90b200 2024-09-10T21:17:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa556a00640 1 --2- 172.21.15.16:0/2093957274 >> [v2:172.21.15.52:6832/1108687782,v1:172.21.15.52:6834/1108687782] conn(0x55ad9d90b200 0x55ad9d90b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa556a00640 1 --2- 172.21.15.16:0/2093957274 >> [v2:172.21.15.52:6832/1108687782,v1:172.21.15.52:6834/1108687782] conn(0x55ad9d90b200 0x55ad9d90b5e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13961fb27b9dd6d7 server_cookie=b77757721a416655 in_seq=0 out_seq=0 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa544c00640 10 client.7230 ms_handle_connect on v2:172.21.15.52:6832/1108687782 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa544c00640 1 -- 172.21.15.16:0/2093957274 <== mds.0 v2:172.21.15.52:6832/1108687782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55ad9d6a4520 con 0x55ad9d90b200 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa544c00640 10 client.7230 handle_command_reply: tid=0 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 client.7230 shutdown 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 -- 172.21.15.16:0/2093957274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 msgr2=0x55ad9d8efa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 --2- 172.21.15.16:0/2093957274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 0x55ad9d8efa20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa550015760 tx=0x7fa550011000 comp rx=0 tx=0).stop 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 -- 172.21.15.16:0/2093957274 shutdown_connections 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 --2- 172.21.15.16:0/2093957274 >> [v2:172.21.15.52:6832/1108687782,v1:172.21.15.52:6834/1108687782] conn(0x55ad9d90b200 0x55ad9d90b5e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 --2- 172.21.15.16:0/2093957274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9d8952f0 0x55ad9d8efa20 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 --2- 172.21.15.16:0/2093957274 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55ad9d8949d0 0x55ad9d8ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 --2- 172.21.15.16:0/2093957274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad9d848e10 0x55ad9d8ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa544c00640 0 client.7230 ms_handle_reset on v2:172.21.15.52:6832/1108687782 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 -- 172.21.15.16:0/2093957274 >> 172.21.15.16:0/2093957274 conn(0x55ad9d732130 msgr2=0x55ad9d8a6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 -- 172.21.15.16:0/2093957274 shutdown_connections 2024-09-10T21:17:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 1 -- 172.21.15.16:0/2093957274 wait complete. 2024-09-10T21:17:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa561157000 20 client.7230 trim_cache size 0 max 16384 2024-09-10T21:17:11.739 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:17:11.739 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "eb19be60-6d8f-4816-b778-3d7d6698304d", 2024-09-10T21:17:11.739 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-10T21:17:11.739 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 135926, 2024-09-10T21:17:11.739 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-10T21:17:11.739 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-10T21:17:11.739 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-10T21:17:11.739 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.0243221370000004, 2024-09-10T21:17:11.739 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 71, 2024-09-10T21:17:11.739 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 104, 2024-09-10T21:17:11.740 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 103, 2024-09-10T21:17:11.740 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 10.530042637999999 2024-09-10T21:17:11.740 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:17:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa55de00640 1 -- 172.21.15.16:0/2683793143 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5340725e0 msgr2=0x7fa534074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa55de00640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5340725e0 0x7fa534074aa0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa5580709e0 tx=0x7fa550009040 comp rx=0 tx=0).stop 2024-09-10T21:17:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa55de00640 1 -- 172.21.15.16:0/2683793143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5580705e0 msgr2=0x7fa5580c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.729+0000 7fa55de00640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5580705e0 0x7fa5580c8dc0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa540009790 tx=0x7fa540005dd0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.733+0000 7fa55de00640 1 -- 172.21.15.16:0/2683793143 shutdown_connections 2024-09-10T21:17:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.733+0000 7fa55de00640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5340725e0 0x7fa534074aa0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.733+0000 7fa55de00640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa55815a270 0x7fa5580c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.733+0000 7fa55de00640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5580705e0 0x7fa5580c8dc0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.733+0000 7fa55de00640 1 --2- 172.21.15.16:0/2683793143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa55806eb80 0x7fa5580b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.733+0000 7fa55de00640 1 -- 172.21.15.16:0/2683793143 >> 172.21.15.16:0/2683793143 conn(0x7fa5580cfb20 msgr2=0x7fa5580b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.733+0000 7fa55de00640 1 -- 172.21.15.16:0/2683793143 shutdown_connections 2024-09-10T21:17:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.733+0000 7fa55de00640 1 -- 172.21.15.16:0/2683793143 wait complete. 2024-09-10T21:17:11.759 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:17:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.901+0000 7f8dfc400640 1 -- 172.21.15.16:0/3334692482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8df40b0590 msgr2=0x7f8df4071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.901+0000 7f8dfc400640 1 --2- 172.21.15.16:0/3334692482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8df40b0590 0x7f8df4071520 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8df4073290 tx=0x7f8dec01cc60 comp rx=0 tx=0).stop 2024-09-10T21:17:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.901+0000 7f8dfc400640 1 -- 172.21.15.16:0/3334692482 shutdown_connections 2024-09-10T21:17:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.901+0000 7f8dfc400640 1 --2- 172.21.15.16:0/3334692482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8df40b0590 0x7f8df4071520 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.901+0000 7f8dfc400640 1 --2- 172.21.15.16:0/3334692482 >> v2:172.21.15.52:3300/0 conn(0x7f8df415f400 0x7f8df4070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.901+0000 7f8dfc400640 1 -- 172.21.15.16:0/3334692482 >> 172.21.15.16:0/3334692482 conn(0x7f8df409b130 msgr2=0x7f8df40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.901+0000 7f8dfc400640 1 -- 172.21.15.16:0/3334692482 shutdown_connections 2024-09-10T21:17:11.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.901+0000 7f8dfc400640 1 -- 172.21.15.16:0/3334692482 wait complete. 2024-09-10T21:17:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8dfc400640 1 Processor -- start 2024-09-10T21:17:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8dfc400640 1 -- start start 2024-09-10T21:17:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8dfc400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8df40b0590 0x7f8df41f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8dfc400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df40b13c0 0x7f8df40c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8dfc400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8df415f400 0x7f8df41f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8dfc400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8df4168f60 con 0x7f8df40b0590 2024-09-10T21:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8dfc400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8df4168de0 con 0x7f8df40b13c0 2024-09-10T21:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8dfc400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8df41690e0 con 0x7f8df415f400 2024-09-10T21:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8df9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8df40b0590 0x7f8df41f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8df9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8df40b0590 0x7f8df41f6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54514/0 (socket says 172.21.15.16:54514) 2024-09-10T21:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8df9a00640 1 -- 172.21.15.16:0/1127212397 learned_addr learned my addr 172.21.15.16:0/1127212397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8df9000640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df40b13c0 0x7f8df40c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8dfa400640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8df415f400 0x7f8df41f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8df9a00640 1 -- 172.21.15.16:0/1127212397 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8df415f400 msgr2=0x7f8df41f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8df9a00640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8df415f400 0x7f8df41f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8df9a00640 1 -- 172.21.15.16:0/1127212397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df40b13c0 msgr2=0x7f8df40c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8df9a00640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df40b13c0 0x7f8df40c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8df9a00640 1 -- 172.21.15.16:0/1127212397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df40c6e70 con 0x7f8df40b0590 2024-09-10T21:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8dfa400640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8df415f400 0x7f8df41f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8df9a00640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8df40b0590 0x7f8df41f6000 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f8de800d700 tx=0x7f8de800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8df1600640 1 -- 172.21.15.16:0/1127212397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de8017070 con 0x7f8df40b0590 2024-09-10T21:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8df1600640 1 -- 172.21.15.16:0/1127212397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8de8009d20 con 0x7f8df40b0590 2024-09-10T21:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8df1600640 1 -- 172.21.15.16:0/1127212397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de80106c0 con 0x7f8df40b0590 2024-09-10T21:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8dfc400640 1 -- 172.21.15.16:0/1127212397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df41f8f50 con 0x7f8df40b0590 2024-09-10T21:17:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8dfc400640 1 -- 172.21.15.16:0/1127212397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df41f94b0 con 0x7f8df40b0590 2024-09-10T21:17:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.905+0000 7f8dfc400640 1 -- 172.21.15.16:0/1127212397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df40cfc80 con 0x7f8df40b0590 2024-09-10T21:17:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.909+0000 7f8df1600640 1 -- 172.21.15.16:0/1127212397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8de8010860 con 0x7f8df40b0590 2024-09-10T21:17:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.909+0000 7f8df1600640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8dd4072610 0x7f8dd4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.909+0000 7f8df1600640 1 -- 172.21.15.16:0/1127212397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f8de80936d0 con 0x7f8df40b0590 2024-09-10T21:17:11.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.909+0000 7f8df9000640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8dd4072610 0x7f8dd4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.909+0000 7f8df9000640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8dd4072610 0x7f8dd4074ad0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8df40cd4e0 tx=0x7f8de0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:11.913+0000 7f8df1600640 1 -- 172.21.15.16:0/1127212397 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8df41f8f50 con 0x7f8df40b0590 2024-09-10T21:17:12.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.121+0000 7f8dfc400640 1 -- 172.21.15.16:0/1127212397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8df415bbe0 con 0x7f8df40b0590 2024-09-10T21:17:12.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.121+0000 7f8df1600640 1 -- 172.21.15.16:0/1127212397 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) ==== 53+0+73 (secure 0 0 0) 0x7f8de805e680 con 0x7f8df40b0590 2024-09-10T21:17:12.127 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:17:12.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.125+0000 7f8dfc400640 1 -- 172.21.15.16:0/1127212397 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8dd4072610 msgr2=0x7f8dd4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:12.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.125+0000 7f8dfc400640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8dd4072610 0x7f8dd4074ad0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8df40cd4e0 tx=0x7f8de0009040 comp rx=0 tx=0).stop 2024-09-10T21:17:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.125+0000 7f8dfc400640 1 -- 172.21.15.16:0/1127212397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8df40b0590 msgr2=0x7f8df41f6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.125+0000 7f8dfc400640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8df40b0590 0x7f8df41f6000 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f8de800d700 tx=0x7f8de800dbc0 comp rx=0 tx=0).stop 2024-09-10T21:17:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.125+0000 7f8dfc400640 1 -- 172.21.15.16:0/1127212397 shutdown_connections 2024-09-10T21:17:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.125+0000 7f8dfc400640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8dd4072610 0x7f8dd4074ad0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.125+0000 7f8dfc400640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8df415f400 0x7f8df41f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.125+0000 7f8dfc400640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df40b13c0 0x7f8df40c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.125+0000 7f8dfc400640 1 --2- 172.21.15.16:0/1127212397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8df40b0590 0x7f8df41f6000 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.125+0000 7f8dfc400640 1 -- 172.21.15.16:0/1127212397 >> 172.21.15.16:0/1127212397 conn(0x7f8df409b130 msgr2=0x7f8df41578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.125+0000 7f8dfc400640 1 -- 172.21.15.16:0/1127212397 shutdown_connections 2024-09-10T21:17:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12.125+0000 7f8dfc400640 1 -- 172.21.15.16:0/1127212397 wait complete. 2024-09-10T21:17:12.149 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:17:12.149 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:17:12.149 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-10T21:17:12.149 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:17:12.149 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:17:12.149 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:17:12.149 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:17:12.150 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:17:12.150 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:17:12.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:12.150 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-10T21:17:12.159 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:17:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:17:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:17:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:17:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:12.160 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:17:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-10T21:17:12.160 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:17:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-10T21:17:12.160 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:17:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-10T21:17:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:17:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42017sec preferred_lft 42017sec 2024-09-10T21:17:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-10T21:17:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:12.161 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:17:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-10T21:17:12.161 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:17:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:30:fb:74:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:17:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:12.161 INFO:teuthology.orchestra.run.smithi016.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:17:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:17:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:17:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:12.162 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:17:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:17:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:17:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:12.162 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:17:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:17:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:17:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:12.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:12.163 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:17:12.225 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-10T21:17:12.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:12.227 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-10T21:17:12.276 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-10T21:17:12.276 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:17:12.276 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:17:12.277 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:17:12.277 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:17:12.277 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:17:12.277 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:17:12.277 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:17:12.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:17:12.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:17:12.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:17:12.277 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:17:12.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:17:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:17:12.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:12.414 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:17:12.463 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:17:12.463 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:17:12.464 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:17:12.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:12.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-10T21:17:12.527 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:17:12.527 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:17:12.583 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:17:12.583 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:17:12.583 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:17:12.639 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:17:12.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:12.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:17:12.729 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:17:12.729 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:17:12.743 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:17:12.743 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:17:12.743 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:17:12.743 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:17:12.743 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:17:12.747 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:17:12.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:12.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:17:12.768 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:17:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:17:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:17:12.882 INFO:teuthology.orchestra.run.smithi016.stdout:client136013 2024-09-10T21:17:12.882 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:17:12.882 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:12.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:17:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.077+0000 7fba1d800640 1 -- 172.21.15.16:0/3692294485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba18152630 msgr2=0x7fba18152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.077+0000 7fba1d800640 1 --2- 172.21.15.16:0/3692294485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba18152630 0x7fba18152a10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fba08009840 tx=0x7fba0801c820 comp rx=0 tx=0).stop 2024-09-10T21:17:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.077+0000 7fba1d800640 1 -- 172.21.15.16:0/3692294485 shutdown_connections 2024-09-10T21:17:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.077+0000 7fba1d800640 1 --2- 172.21.15.16:0/3692294485 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba1815fc70 0x7fba18160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.077+0000 7fba1d800640 1 --2- 172.21.15.16:0/3692294485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba18152630 0x7fba18152a10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.077+0000 7fba1d800640 1 -- 172.21.15.16:0/3692294485 >> 172.21.15.16:0/3692294485 conn(0x7fba180d0a60 msgr2=0x7fba180d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.077+0000 7fba1d800640 1 -- 172.21.15.16:0/3692294485 shutdown_connections 2024-09-10T21:17:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.077+0000 7fba1d800640 1 -- 172.21.15.16:0/3692294485 wait complete. 2024-09-10T21:17:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba1d800640 1 Processor -- start 2024-09-10T21:17:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba1d800640 1 -- start start 2024-09-10T21:17:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba1d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180afb10 0x7fba180ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba1d800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba18152630 0x7fba180ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba1d800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba1815fc70 0x7fba18150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba1d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba180b58f0 con 0x7fba180afb10 2024-09-10T21:17:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba1d800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fba180b0320 con 0x7fba18152630 2024-09-10T21:17:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba1d800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fba180b5a70 con 0x7fba1815fc70 2024-09-10T21:17:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba17e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180afb10 0x7fba180ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba17e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180afb10 0x7fba180ca130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47230/0 (socket says 172.21.15.16:47230) 2024-09-10T21:17:13.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba17e00640 1 -- 172.21.15.16:0/2050735195 learned_addr learned my addr 172.21.15.16:0/2050735195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:13.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba1ce00640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba1815fc70 0x7fba18150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba17400640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba18152630 0x7fba180ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba17e00640 1 -- 172.21.15.16:0/2050735195 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba1815fc70 msgr2=0x7fba18150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba17e00640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba1815fc70 0x7fba18150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba17e00640 1 -- 172.21.15.16:0/2050735195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba18152630 msgr2=0x7fba180ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba17e00640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba18152630 0x7fba180ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba17e00640 1 -- 172.21.15.16:0/2050735195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1814ad30 con 0x7fba180afb10 2024-09-10T21:17:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba17e00640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180afb10 0x7fba180ca130 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fba08009790 tx=0x7fba080a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba0fe00640 1 -- 172.21.15.16:0/2050735195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba080b2040 con 0x7fba180afb10 2024-09-10T21:17:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba0fe00640 1 -- 172.21.15.16:0/2050735195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba08004430 con 0x7fba180afb10 2024-09-10T21:17:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba0fe00640 1 -- 172.21.15.16:0/2050735195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08005280 con 0x7fba180afb10 2024-09-10T21:17:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba1d800640 1 -- 172.21.15.16:0/2050735195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba180c6cf0 con 0x7fba180afb10 2024-09-10T21:17:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba1d800640 1 -- 172.21.15.16:0/2050735195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1814b310 con 0x7fba180afb10 2024-09-10T21:17:13.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.081+0000 7fba1d800640 1 -- 172.21.15.16:0/2050735195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba18152a10 con 0x7fba180afb10 2024-09-10T21:17:13.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.085+0000 7fba0fe00640 1 -- 172.21.15.16:0/2050735195 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fba080b5070 con 0x7fba180afb10 2024-09-10T21:17:13.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.085+0000 7fba0fe00640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9f80725e0 0x7fb9f8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.085+0000 7fba0fe00640 1 -- 172.21.15.16:0/2050735195 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7fba0812c920 con 0x7fba180afb10 2024-09-10T21:17:13.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.085+0000 7fba17400640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9f80725e0 0x7fb9f8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.085+0000 7fba17400640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9f80725e0 0x7fb9f8074aa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fba040096c0 tx=0x7fba04009260 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:13.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.089+0000 7fba0fe00640 1 -- 172.21.15.16:0/2050735195 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fba08016410 con 0x7fba180afb10 2024-09-10T21:17:13.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.297+0000 7fba1d800640 1 -- 172.21.15.16:0/2050735195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fba180af8f0 con 0x7fba180afb10 2024-09-10T21:17:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.297+0000 7fba0fe00640 1 -- 172.21.15.16:0/2050735195 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) ==== 53+0+73 (secure 0 0 0) 0x7fba080f78d0 con 0x7fba180afb10 2024-09-10T21:17:13.303 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:17:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.301+0000 7fba1d800640 1 -- 172.21.15.16:0/2050735195 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9f80725e0 msgr2=0x7fb9f8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.301+0000 7fba1d800640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9f80725e0 0x7fb9f8074aa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fba040096c0 tx=0x7fba04009260 comp rx=0 tx=0).stop 2024-09-10T21:17:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.301+0000 7fba1d800640 1 -- 172.21.15.16:0/2050735195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180afb10 msgr2=0x7fba180ca130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.301+0000 7fba1d800640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180afb10 0x7fba180ca130 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fba08009790 tx=0x7fba080a5700 comp rx=0 tx=0).stop 2024-09-10T21:17:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.301+0000 7fba1d800640 1 -- 172.21.15.16:0/2050735195 shutdown_connections 2024-09-10T21:17:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.301+0000 7fba1d800640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9f80725e0 0x7fb9f8074aa0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.301+0000 7fba1d800640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba1815fc70 0x7fba18150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.301+0000 7fba1d800640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba18152630 0x7fba180ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.301+0000 7fba1d800640 1 --2- 172.21.15.16:0/2050735195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180afb10 0x7fba180ca130 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.301+0000 7fba1d800640 1 -- 172.21.15.16:0/2050735195 >> 172.21.15.16:0/2050735195 conn(0x7fba180d0a60 msgr2=0x7fba1815d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.301+0000 7fba1d800640 1 -- 172.21.15.16:0/2050735195 shutdown_connections 2024-09-10T21:17:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:13.301+0000 7fba1d800640 1 -- 172.21.15.16:0/2050735195 wait complete. 2024-09-10T21:17:13.324 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:17:13.324 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:17:13.325 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-09-10T21:17:13.325 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T21:17:13.325 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:17:13.325 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:17:13.325 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:17:13.325 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:17:13.325 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:17:13.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:13.325 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-09-10T21:17:13.336 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:17:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:17:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:17:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:17:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:13.336 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:17:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-09-10T21:17:13.336 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:17:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-09-10T21:17:13.337 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:17:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-09-10T21:17:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:17:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42017sec preferred_lft 42017sec 2024-09-10T21:17:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-09-10T21:17:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:13.337 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:17:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-09-10T21:17:13.337 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:17:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:7e:b4:71:a2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:17:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:13.338 INFO:teuthology.orchestra.run.smithi052.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:17:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:17:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:17:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:13.338 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:17:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:17:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:17:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:13.338 INFO:teuthology.orchestra.run.smithi052.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:17:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:17:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:17:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:13.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:13.339 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:17:13.404 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-09-10T21:17:13.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:13.406 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-09-10T21:17:13.462 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-09-10T21:17:13.462 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:17:13.462 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:17:13.462 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:17:13.462 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:17:13.462 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:17:13.462 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:17:13.463 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:17:13.463 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:17:13.463 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:17:13.463 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:17:13.463 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:17:13.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:17:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:17:13.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:17:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:13.611 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:17:13.621 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:13.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-09-10T21:17:13.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:17:13.688 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T21:17:13.740 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:17:13.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:17:13.740 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:17:13.795 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T21:17:13.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:13.796 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:17:13.889 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:17:13.889 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:17:13.904 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:17:13.904 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:17:13.904 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:17:13.904 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 1@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:17:13.904 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:17:13.908 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:17:13.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:13.908 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:17:13.928 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:17:14.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:17:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:17:14.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:17:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:17:14.033 INFO:teuthology.orchestra.run.smithi052.stdout:client7245 2024-09-10T21:17:14.033 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:17:14.033 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:14.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-09-10T21:17:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.193+0000 7f2cb6000640 1 -- 172.21.15.16:0/4106377440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb00cf1e0 msgr2=0x7f2cb00af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.193+0000 7f2cb6000640 1 --2- 172.21.15.16:0/4106377440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb00cf1e0 0x7f2cb00af050 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009c80 tx=0x7f2ca801cd70 comp rx=0 tx=0).stop 2024-09-10T21:17:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.193+0000 7f2cb6000640 1 -- 172.21.15.16:0/4106377440 shutdown_connections 2024-09-10T21:17:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.193+0000 7f2cb6000640 1 --2- 172.21.15.16:0/4106377440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb00cf1e0 0x7f2cb00af050 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.193+0000 7f2cb6000640 1 --2- 172.21.15.16:0/4106377440 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2cb00ce8c0 0x7f2cb00ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.193+0000 7f2cb6000640 1 -- 172.21.15.16:0/4106377440 >> 172.21.15.16:0/4106377440 conn(0x7f2cb00d1c70 msgr2=0x7f2cb00d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.193+0000 7f2cb6000640 1 -- 172.21.15.16:0/4106377440 shutdown_connections 2024-09-10T21:17:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.193+0000 7f2cb6000640 1 -- 172.21.15.16:0/4106377440 wait complete. 2024-09-10T21:17:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.193+0000 7f2cb6000640 1 Processor -- start 2024-09-10T21:17:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.193+0000 7f2cb6000640 1 -- start start 2024-09-10T21:17:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.193+0000 7f2cb6000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2cb00b0980 0x7f2cb01fbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.193+0000 7f2cb6000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cb00ce8c0 0x7f2cb00b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.193+0000 7f2cb6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb00cf1e0 0x7f2cb0071b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb6000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0169340 con 0x7f2cb00cf1e0 2024-09-10T21:17:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb6000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb01691c0 con 0x7f2cb00ce8c0 2024-09-10T21:17:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb6000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb01694c0 con 0x7f2cb00b0980 2024-09-10T21:17:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb5600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb00cf1e0 0x7f2cb0071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb5600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb00cf1e0 0x7f2cb0071b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47250/0 (socket says 172.21.15.16:47250) 2024-09-10T21:17:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb5600640 1 -- 172.21.15.16:0/3303585466 learned_addr learned my addr 172.21.15.16:0/3303585466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb4c00640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2cb00b0980 0x7f2cb01fbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cb00ce8c0 0x7f2cb00b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb5600640 1 -- 172.21.15.16:0/3303585466 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2cb00b0980 msgr2=0x7f2cb01fbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb5600640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2cb00b0980 0x7f2cb01fbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb5600640 1 -- 172.21.15.16:0/3303585466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cb00ce8c0 msgr2=0x7f2cb00b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb5600640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cb00ce8c0 0x7f2cb00b7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb5600640 1 -- 172.21.15.16:0/3303585466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb00cbf80 con 0x7f2cb00cf1e0 2024-09-10T21:17:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2ca7e00640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cb00ce8c0 0x7f2cb00b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb5600640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb00cf1e0 0x7f2cb0071b00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009bd0 tx=0x7f2ca8003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cad600640 1 -- 172.21.15.16:0/3303585466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80bb070 con 0x7f2cb00cf1e0 2024-09-10T21:17:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cad600640 1 -- 172.21.15.16:0/3303585466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ca8004b40 con 0x7f2cb00cf1e0 2024-09-10T21:17:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cad600640 1 -- 172.21.15.16:0/3303585466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80b2040 con 0x7f2cb00cf1e0 2024-09-10T21:17:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb6000640 1 -- 172.21.15.16:0/3303585466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb01586c0 con 0x7f2cb00cf1e0 2024-09-10T21:17:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb6000640 1 -- 172.21.15.16:0/3303585466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb0158c80 con 0x7f2cb00cf1e0 2024-09-10T21:17:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cb6000640 1 -- 172.21.15.16:0/3303585466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb00ceca0 con 0x7f2cb00cf1e0 2024-09-10T21:17:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.197+0000 7f2cad600640 1 -- 172.21.15.16:0/3303585466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2ca80b5030 con 0x7f2cb00cf1e0 2024-09-10T21:17:14.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.201+0000 7f2cad600640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2c8c0725e0 0x7f2c8c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:14.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.201+0000 7f2cad600640 1 -- 172.21.15.16:0/3303585466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f2ca812ccb0 con 0x7f2cb00cf1e0 2024-09-10T21:17:14.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.201+0000 7f2cb4c00640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2c8c0725e0 0x7f2c8c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:14.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.201+0000 7f2cb4c00640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2c8c0725e0 0x7f2c8c074aa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009be0 tx=0x7f2ca00091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.205+0000 7f2cad600640 1 -- 172.21.15.16:0/3303585466 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2ca8016990 con 0x7f2cb00cf1e0 2024-09-10T21:17:14.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.381+0000 7f2cb6000640 1 -- 172.21.15.16:0/3303585466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f2cb00b7a10 con 0x7f2cb00cf1e0 2024-09-10T21:17:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.385+0000 7f2cad600640 1 -- 172.21.15.16:0/3303585466 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v48)=0 v48) ==== 121+0+0 (secure 0 0 0) 0x7f2ca80f7cf0 con 0x7f2cb00cf1e0 2024-09-10T21:17:14.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.385+0000 7f2cb6000640 1 -- 172.21.15.16:0/3303585466 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2c8c0725e0 msgr2=0x7f2c8c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:14.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.385+0000 7f2cb6000640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2c8c0725e0 0x7f2c8c074aa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009be0 tx=0x7f2ca00091e0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.385+0000 7f2cb6000640 1 -- 172.21.15.16:0/3303585466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb00cf1e0 msgr2=0x7f2cb0071b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:14.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.385+0000 7f2cb6000640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb00cf1e0 0x7f2cb0071b00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009bd0 tx=0x7f2ca8003f40 comp rx=0 tx=0).stop 2024-09-10T21:17:14.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.385+0000 7f2cb6000640 1 -- 172.21.15.16:0/3303585466 shutdown_connections 2024-09-10T21:17:14.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.385+0000 7f2cb6000640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2c8c0725e0 0x7f2c8c074aa0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.385+0000 7f2cb6000640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb00cf1e0 0x7f2cb0071b00 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.385+0000 7f2cb6000640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cb00ce8c0 0x7f2cb00b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.385+0000 7f2cb6000640 1 --2- 172.21.15.16:0/3303585466 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2cb00b0980 0x7f2cb01fbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.385+0000 7f2cb6000640 1 -- 172.21.15.16:0/3303585466 >> 172.21.15.16:0/3303585466 conn(0x7f2cb00d1c70 msgr2=0x7f2cb00b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:14.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.385+0000 7f2cb6000640 1 -- 172.21.15.16:0/3303585466 shutdown_connections 2024-09-10T21:17:14.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.385+0000 7f2cb6000640 1 -- 172.21.15.16:0/3303585466 wait complete. 2024-09-10T21:17:14.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 400 2024-09-10T21:17:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 -- 172.21.15.16:0/948595412 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa5c40cca00 msgr2=0x7fa5c40ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 --2- 172.21.15.16:0/948595412 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa5c40cca00 0x7fa5c40ce340 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009870 tx=0x7fa5b001c6f0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 -- 172.21.15.16:0/948595412 shutdown_connections 2024-09-10T21:17:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 --2- 172.21.15.16:0/948595412 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa5c40cca00 0x7fa5c40ce340 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 -- 172.21.15.16:0/948595412 >> 172.21.15.16:0/948595412 conn(0x7fa5c40d0390 msgr2=0x7fa5c40d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 -- 172.21.15.16:0/948595412 shutdown_connections 2024-09-10T21:17:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 -- 172.21.15.16:0/948595412 wait complete. 2024-09-10T21:17:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 Processor -- start 2024-09-10T21:17:14.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 -- start start 2024-09-10T21:17:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c40b04a0 0x7fa5c40993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c40cca00 0x7fa5c415ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa5c40cdf60 0x7fa5c415d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c40cc6a0 con 0x7fa5c40b04a0 2024-09-10T21:17:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c40cea90 con 0x7fa5c40cca00 2024-09-10T21:17:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.557+0000 7fa5cc600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c41696a0 con 0x7fa5c40cdf60 2024-09-10T21:17:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c40b04a0 0x7fa5c40993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c40b04a0 0x7fa5c40993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47260/0 (socket says 172.21.15.16:47260) 2024-09-10T21:17:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c9c00640 1 -- 172.21.15.16:0/1396238735 learned_addr learned my addr 172.21.15.16:0/1396238735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c9200640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c40cca00 0x7fa5c415ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5ca600640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa5c40cdf60 0x7fa5c415d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c9c00640 1 -- 172.21.15.16:0/1396238735 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa5c40cdf60 msgr2=0x7fa5c415d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c9c00640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa5c40cdf60 0x7fa5c415d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c9c00640 1 -- 172.21.15.16:0/1396238735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c40cca00 msgr2=0x7fa5c415ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c9c00640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c40cca00 0x7fa5c415ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c9c00640 1 -- 172.21.15.16:0/1396238735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c415bb90 con 0x7fa5c40b04a0 2024-09-10T21:17:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c9c00640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c40b04a0 0x7fa5c40993a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c6b0 tx=0x7fa5b800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c1600640 1 -- 172.21.15.16:0/1396238735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8016020 con 0x7fa5c40b04a0 2024-09-10T21:17:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c1600640 1 -- 172.21.15.16:0/1396238735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5b800a930 con 0x7fa5c40b04a0 2024-09-10T21:17:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5cc600640 1 -- 172.21.15.16:0/1396238735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c415be70 con 0x7fa5c40b04a0 2024-09-10T21:17:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c1600640 1 -- 172.21.15.16:0/1396238735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b800ad80 con 0x7fa5c40b04a0 2024-09-10T21:17:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5cc600640 1 -- 172.21.15.16:0/1396238735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c415c430 con 0x7fa5c40b04a0 2024-09-10T21:17:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5cc600640 1 -- 172.21.15.16:0/1396238735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c40ce340 con 0x7fa5c40b04a0 2024-09-10T21:17:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.561+0000 7fa5c1600640 1 -- 172.21.15.16:0/1396238735 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa5b8012420 con 0x7fa5c40b04a0 2024-09-10T21:17:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.565+0000 7fa5c1600640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5a40725e0 0x7fa5a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:14.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.565+0000 7fa5c1600640 1 -- 172.21.15.16:0/1396238735 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7fa5b8093670 con 0x7fa5c40b04a0 2024-09-10T21:17:14.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.565+0000 7fa5c9200640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5a40725e0 0x7fa5a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:14.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.565+0000 7fa5c9200640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5a40725e0 0x7fa5a4074aa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00af000 tx=0x7fa5b00b4040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:14.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.569+0000 7fa5c1600640 1 -- 172.21.15.16:0/1396238735 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa5b8020020 con 0x7fa5c40b04a0 2024-09-10T21:17:14.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.741+0000 7fa5cc600640 1 -- 172.21.15.16:0/1396238735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7fa5c406ec50 con 0x7fa5c40b04a0 2024-09-10T21:17:14.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.745+0000 7fa5c1600640 1 -- 172.21.15.16:0/1396238735 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v49)=0 v49) ==== 119+0+0 (secure 0 0 0) 0x7fa5b805e620 con 0x7fa5c40b04a0 2024-09-10T21:17:14.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.749+0000 7fa5cc600640 1 -- 172.21.15.16:0/1396238735 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5a40725e0 msgr2=0x7fa5a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.749+0000 7fa5cc600640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5a40725e0 0x7fa5a4074aa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00af000 tx=0x7fa5b00b4040 comp rx=0 tx=0).stop 2024-09-10T21:17:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.749+0000 7fa5cc600640 1 -- 172.21.15.16:0/1396238735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c40b04a0 msgr2=0x7fa5c40993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.749+0000 7fa5cc600640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c40b04a0 0x7fa5c40993a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c6b0 tx=0x7fa5b800cb70 comp rx=0 tx=0).stop 2024-09-10T21:17:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.749+0000 7fa5cc600640 1 -- 172.21.15.16:0/1396238735 shutdown_connections 2024-09-10T21:17:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.749+0000 7fa5cc600640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5a40725e0 0x7fa5a4074aa0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.749+0000 7fa5cc600640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa5c40cdf60 0x7fa5c415d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.749+0000 7fa5cc600640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c40cca00 0x7fa5c415ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.749+0000 7fa5cc600640 1 --2- 172.21.15.16:0/1396238735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c40b04a0 0x7fa5c40993a0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.749+0000 7fa5cc600640 1 -- 172.21.15.16:0/1396238735 >> 172.21.15.16:0/1396238735 conn(0x7fa5c40d0390 msgr2=0x7fa5c40cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.749+0000 7fa5cc600640 1 -- 172.21.15.16:0/1396238735 shutdown_connections 2024-09-10T21:17:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.749+0000 7fa5cc600640 1 -- 172.21.15.16:0/1396238735 wait complete. 2024-09-10T21:17:14.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 800 2024-09-10T21:17:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 -- 172.21.15.16:0/2354740523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c005220 msgr2=0x7f242c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 --2- 172.21.15.16:0/2354740523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c005220 0x7f242c00b9d0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2440009ee0 tx=0x7f244001d630 comp rx=0 tx=0).stop 2024-09-10T21:17:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 -- 172.21.15.16:0/2354740523 shutdown_connections 2024-09-10T21:17:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 --2- 172.21.15.16:0/2354740523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c005220 0x7f242c00b9d0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 -- 172.21.15.16:0/2354740523 >> 172.21.15.16:0/2354740523 conn(0x7f24480d1df0 msgr2=0x7f24480d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 -- 172.21.15.16:0/2354740523 shutdown_connections 2024-09-10T21:17:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 -- 172.21.15.16:0/2354740523 wait complete. 2024-09-10T21:17:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 Processor -- start 2024-09-10T21:17:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 -- start start 2024-09-10T21:17:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c005220 0x7f24481e5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f24480c51d0 0x7f24481e6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244806eef0 0x7f24481f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24480d2a90 con 0x7f242c005220 2024-09-10T21:17:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f24480d2910 con 0x7f244806eef0 2024-09-10T21:17:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f244e400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f24480b12e0 con 0x7f24480c51d0 2024-09-10T21:17:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f2447400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c005220 0x7f24481e5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f2447400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c005220 0x7f24481e5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47276/0 (socket says 172.21.15.16:47276) 2024-09-10T21:17:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f2447400640 1 -- 172.21.15.16:0/500441470 learned_addr learned my addr 172.21.15.16:0/500441470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f243fe00640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f24480c51d0 0x7f24481e6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f2447e00640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244806eef0 0x7f24481f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.921+0000 7f2447400640 1 -- 172.21.15.16:0/500441470 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f24480c51d0 msgr2=0x7f24481e6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f2447400640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f24480c51d0 0x7f24481e6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f2447400640 1 -- 172.21.15.16:0/500441470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244806eef0 msgr2=0x7f24481f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f2447400640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244806eef0 0x7f24481f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f2447400640 1 -- 172.21.15.16:0/500441470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24481e5140 con 0x7f242c005220 2024-09-10T21:17:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f2447400640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c005220 0x7f24481e5f10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f2438003eb0 tx=0x7f2438001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f243f400640 1 -- 172.21.15.16:0/500441470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438015070 con 0x7f242c005220 2024-09-10T21:17:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f243f400640 1 -- 172.21.15.16:0/500441470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2438011840 con 0x7f242c005220 2024-09-10T21:17:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f243f400640 1 -- 172.21.15.16:0/500441470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438011c70 con 0x7f242c005220 2024-09-10T21:17:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f244e400640 1 -- 172.21.15.16:0/500441470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24481e5420 con 0x7f242c005220 2024-09-10T21:17:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f244e400640 1 -- 172.21.15.16:0/500441470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24481e59e0 con 0x7f242c005220 2024-09-10T21:17:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f244e400640 1 -- 172.21.15.16:0/500441470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24480ce590 con 0x7f242c005220 2024-09-10T21:17:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f243f400640 1 -- 172.21.15.16:0/500441470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f24380103c0 con 0x7f242c005220 2024-09-10T21:17:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f243f400640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2424072610 0x7f2424074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.925+0000 7f243f400640 1 -- 172.21.15.16:0/500441470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f2438094140 con 0x7f242c005220 2024-09-10T21:17:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.929+0000 7f243fe00640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2424072610 0x7f2424074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.929+0000 7f243fe00640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2424072610 0x7f2424074ad0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2430001250 tx=0x7f2430001120 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:14.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:14.929+0000 7f243f400640 1 -- 172.21.15.16:0/500441470 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f243805f7d0 con 0x7f242c005220 2024-09-10T21:17:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.105+0000 7f244e400640 1 -- 172.21.15.16:0/500441470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7f24480712a0 con 0x7f242c005220 2024-09-10T21:17:15.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.109+0000 7f243f400640 1 -- 172.21.15.16:0/500441470 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v50)=0 v50) ==== 128+0+0 (secure 0 0 0) 0x7f243805f0d0 con 0x7f242c005220 2024-09-10T21:17:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.113+0000 7f244e400640 1 -- 172.21.15.16:0/500441470 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2424072610 msgr2=0x7f2424074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.113+0000 7f244e400640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2424072610 0x7f2424074ad0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2430001250 tx=0x7f2430001120 comp rx=0 tx=0).stop 2024-09-10T21:17:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.113+0000 7f244e400640 1 -- 172.21.15.16:0/500441470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c005220 msgr2=0x7f24481e5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.113+0000 7f244e400640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c005220 0x7f24481e5f10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f2438003eb0 tx=0x7f2438001640 comp rx=0 tx=0).stop 2024-09-10T21:17:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.113+0000 7f244e400640 1 -- 172.21.15.16:0/500441470 shutdown_connections 2024-09-10T21:17:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.113+0000 7f244e400640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2424072610 0x7f2424074ad0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.113+0000 7f244e400640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f244806eef0 0x7f24481f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.113+0000 7f244e400640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f24480c51d0 0x7f24481e6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.113+0000 7f244e400640 1 --2- 172.21.15.16:0/500441470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c005220 0x7f24481e5f10 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.113+0000 7f244e400640 1 -- 172.21.15.16:0/500441470 >> 172.21.15.16:0/500441470 conn(0x7f24480d1df0 msgr2=0x7f24480b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.113+0000 7f244e400640 1 -- 172.21.15.16:0/500441470 shutdown_connections 2024-09-10T21:17:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.113+0000 7f244e400640 1 -- 172.21.15.16:0/500441470 wait complete. 2024-09-10T21:17:15.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-10T21:17:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 -- 172.21.15.16:0/2389562940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4c0d0210 msgr2=0x7f7a4c0ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 --2- 172.21.15.16:0/2389562940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4c0d0210 0x7f7a4c0ceca0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009930 tx=0x7f7a3401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 -- 172.21.15.16:0/2389562940 shutdown_connections 2024-09-10T21:17:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 --2- 172.21.15.16:0/2389562940 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a4c0ce040 0x7f7a4c0b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 --2- 172.21.15.16:0/2389562940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4c0d0210 0x7f7a4c0ceca0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 -- 172.21.15.16:0/2389562940 >> 172.21.15.16:0/2389562940 conn(0x7f7a4c0d1df0 msgr2=0x7f7a4c0d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 -- 172.21.15.16:0/2389562940 shutdown_connections 2024-09-10T21:17:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 -- 172.21.15.16:0/2389562940 wait complete. 2024-09-10T21:17:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 Processor -- start 2024-09-10T21:17:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 -- start start 2024-09-10T21:17:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a4c0ce040 0x7f7a4c14cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a4c0cfe30 0x7f7a4c149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4c0d0210 0x7f7a4c1f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c0d2b00 con 0x7f7a4c0d0210 2024-09-10T21:17:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c0b14f0 con 0x7f7a4c0ce040 2024-09-10T21:17:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a52a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4c0cf1e0 con 0x7f7a4c0cfe30 2024-09-10T21:17:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a50a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4c0d0210 0x7f7a4c1f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a50a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4c0d0210 0x7f7a4c1f8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47306/0 (socket says 172.21.15.16:47306) 2024-09-10T21:17:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a50a00640 1 -- 172.21.15.16:0/3972696615 learned_addr learned my addr 172.21.15.16:0/3972696615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a4be00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a4c0ce040 0x7f7a4c14cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a43e00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a4c0cfe30 0x7f7a4c149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a50a00640 1 -- 172.21.15.16:0/3972696615 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a4c0cfe30 msgr2=0x7f7a4c149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a50a00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a4c0cfe30 0x7f7a4c149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a50a00640 1 -- 172.21.15.16:0/3972696615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a4c0ce040 msgr2=0x7f7a4c14cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a50a00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a4c0ce040 0x7f7a4c14cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a50a00640 1 -- 172.21.15.16:0/3972696615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c1f8ef0 con 0x7f7a4c0d0210 2024-09-10T21:17:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a43e00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a4c0cfe30 0x7f7a4c149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.281+0000 7f7a4be00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a4c0ce040 0x7f7a4c14cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.285+0000 7f7a50a00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4c0d0210 0x7f7a4c1f8380 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400eb20 tx=0x7f7a4400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.285+0000 7f7a48c00640 1 -- 172.21.15.16:0/3972696615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44004b30 con 0x7f7a4c0d0210 2024-09-10T21:17:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.285+0000 7f7a48c00640 1 -- 172.21.15.16:0/3972696615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a44004cf0 con 0x7f7a4c0d0210 2024-09-10T21:17:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.285+0000 7f7a52a00640 1 -- 172.21.15.16:0/3972696615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4c1f91d0 con 0x7f7a4c0d0210 2024-09-10T21:17:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.285+0000 7f7a48c00640 1 -- 172.21.15.16:0/3972696615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44005280 con 0x7f7a4c0d0210 2024-09-10T21:17:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.285+0000 7f7a52a00640 1 -- 172.21.15.16:0/3972696615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4c14c530 con 0x7f7a4c0d0210 2024-09-10T21:17:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.285+0000 7f7a52a00640 1 -- 172.21.15.16:0/3972696615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a4c0b1710 con 0x7f7a4c0d0210 2024-09-10T21:17:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.285+0000 7f7a48c00640 1 -- 172.21.15.16:0/3972696615 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7a44005420 con 0x7f7a4c0d0210 2024-09-10T21:17:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.285+0000 7f7a48c00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a280725e0 0x7f7a28074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.285+0000 7f7a48c00640 1 -- 172.21.15.16:0/3972696615 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f7a44094060 con 0x7f7a4c0d0210 2024-09-10T21:17:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.285+0000 7f7a4be00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a280725e0 0x7f7a28074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.289+0000 7f7a4be00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a280725e0 0x7f7a28074aa0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c001750 tx=0x7f7a3c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.289+0000 7f7a48c00640 1 -- 172.21.15.16:0/3972696615 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7a4405f6f0 con 0x7f7a4c0d0210 2024-09-10T21:17:15.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.465+0000 7f7a52a00640 1 -- 172.21.15.16:0/3972696615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7f7a4c0afa30 con 0x7f7a4c0d0210 2024-09-10T21:17:15.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.465+0000 7f7a48c00640 1 -- 172.21.15.16:0/3972696615 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v50) ==== 106+0+4 (secure 0 0 0) 0x7f7a4405eff0 con 0x7f7a4c0d0210 2024-09-10T21:17:15.470 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-10T21:17:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.469+0000 7f7a52a00640 1 -- 172.21.15.16:0/3972696615 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a280725e0 msgr2=0x7f7a28074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.469+0000 7f7a52a00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a280725e0 0x7f7a28074aa0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c001750 tx=0x7f7a3c009040 comp rx=0 tx=0).stop 2024-09-10T21:17:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.469+0000 7f7a52a00640 1 -- 172.21.15.16:0/3972696615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4c0d0210 msgr2=0x7f7a4c1f8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.469+0000 7f7a52a00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4c0d0210 0x7f7a4c1f8380 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400eb20 tx=0x7f7a4400c2d0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.469+0000 7f7a52a00640 1 -- 172.21.15.16:0/3972696615 shutdown_connections 2024-09-10T21:17:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.469+0000 7f7a52a00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7a280725e0 0x7f7a28074aa0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.469+0000 7f7a52a00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4c0d0210 0x7f7a4c1f8380 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.469+0000 7f7a52a00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a4c0cfe30 0x7f7a4c149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.469+0000 7f7a52a00640 1 --2- 172.21.15.16:0/3972696615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a4c0ce040 0x7f7a4c14cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.469+0000 7f7a52a00640 1 -- 172.21.15.16:0/3972696615 >> 172.21.15.16:0/3972696615 conn(0x7f7a4c0d1df0 msgr2=0x7f7a4c0ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.469+0000 7f7a52a00640 1 -- 172.21.15.16:0/3972696615 shutdown_connections 2024-09-10T21:17:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.469+0000 7f7a52a00640 1 -- 172.21.15.16:0/3972696615 wait complete. 2024-09-10T21:17:15.493 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-09-10T21:17:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.637+0000 7f2fa4a00640 1 -- 172.21.15.16:0/3525692831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa00cbb00 msgr2=0x7f2fa00cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.637+0000 7f2fa4a00640 1 --2- 172.21.15.16:0/3525692831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa00cbb00 0x7f2fa00cf3e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2fa006fc20 tx=0x7f2f9001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 -- 172.21.15.16:0/3525692831 shutdown_connections 2024-09-10T21:17:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 --2- 172.21.15.16:0/3525692831 >> v2:172.21.15.52:3300/0 conn(0x7f2fa006f0e0 0x7f2fa006f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 --2- 172.21.15.16:0/3525692831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa00cbb00 0x7f2fa00cf3e0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 -- 172.21.15.16:0/3525692831 >> 172.21.15.16:0/3525692831 conn(0x7f2fa00b16a0 msgr2=0x7f2fa00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 -- 172.21.15.16:0/3525692831 shutdown_connections 2024-09-10T21:17:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 -- 172.21.15.16:0/3525692831 wait complete. 2024-09-10T21:17:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 Processor -- start 2024-09-10T21:17:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 -- start start 2024-09-10T21:17:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa006f0e0 0x7f2fa0139ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa00cbb00 0x7f2fa0159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fa00cbee0 0x7f2fa015a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa0168d00 con 0x7f2fa006f0e0 2024-09-10T21:17:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa0168b80 con 0x7f2fa00cbb00 2024-09-10T21:17:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2fa4a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa0168e80 con 0x7f2fa00cbee0 2024-09-10T21:17:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2f9de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa006f0e0 0x7f2fa0139ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2f9de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa006f0e0 0x7f2fa0139ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47324/0 (socket says 172.21.15.16:47324) 2024-09-10T21:17:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2f9de00640 1 -- 172.21.15.16:0/561905991 learned_addr learned my addr 172.21.15.16:0/561905991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2f97e00640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa00cbb00 0x7f2fa0159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2f9e800640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fa00cbee0 0x7f2fa015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2f9de00640 1 -- 172.21.15.16:0/561905991 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fa00cbee0 msgr2=0x7f2fa015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2f9de00640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fa00cbee0 0x7f2fa015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2f9de00640 1 -- 172.21.15.16:0/561905991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa00cbb00 msgr2=0x7f2fa0159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2f9de00640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa00cbb00 0x7f2fa0159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2f9de00640 1 -- 172.21.15.16:0/561905991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa01f9410 con 0x7f2fa006f0e0 2024-09-10T21:17:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2f9e800640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fa00cbee0 0x7f2fa015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2f97e00640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa00cbb00 0x7f2fa0159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.641+0000 7f2f9de00640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa006f0e0 0x7f2fa0139ff0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2f900b0040 tx=0x7f2f900a5600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.645+0000 7f2f96000640 1 -- 172.21.15.16:0/561905991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f900b2040 con 0x7f2fa006f0e0 2024-09-10T21:17:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.645+0000 7f2f96000640 1 -- 172.21.15.16:0/561905991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f90004470 con 0x7f2fa006f0e0 2024-09-10T21:17:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.645+0000 7f2f96000640 1 -- 172.21.15.16:0/561905991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90005280 con 0x7f2fa006f0e0 2024-09-10T21:17:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.645+0000 7f2fa4a00640 1 -- 172.21.15.16:0/561905991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa0139510 con 0x7f2fa006f0e0 2024-09-10T21:17:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.645+0000 7f2fa4a00640 1 -- 172.21.15.16:0/561905991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa013a530 con 0x7f2fa006f0e0 2024-09-10T21:17:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.645+0000 7f2fa4a00640 1 -- 172.21.15.16:0/561905991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa006f550 con 0x7f2fa006f0e0 2024-09-10T21:17:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.645+0000 7f2f96000640 1 -- 172.21.15.16:0/561905991 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2f90005420 con 0x7f2fa006f0e0 2024-09-10T21:17:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.649+0000 7f2f96000640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2f7c0725e0 0x7f2f7c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.649+0000 7f2f96000640 1 -- 172.21.15.16:0/561905991 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f2f9012ce00 con 0x7f2fa006f0e0 2024-09-10T21:17:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.649+0000 7f2f97e00640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2f7c0725e0 0x7f2f7c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.649+0000 7f2f97e00640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2f7c0725e0 0x7f2f7c074aa0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0166890 tx=0x7f2f8800b2c0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.653+0000 7f2f96000640 1 -- 172.21.15.16:0/561905991 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2f900164b0 con 0x7f2fa006f0e0 2024-09-10T21:17:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.825+0000 7f2fa4a00640 1 -- 172.21.15.16:0/561905991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f2fa0074020 con 0x7f2fa006f0e0 2024-09-10T21:17:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.829+0000 7f2f96000640 1 -- 172.21.15.16:0/561905991 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v51)=0 v51) ==== 124+0+0 (secure 0 0 0) 0x7f2f900f8410 con 0x7f2fa006f0e0 2024-09-10T21:17:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.833+0000 7f2fa4a00640 1 -- 172.21.15.16:0/561905991 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2f7c0725e0 msgr2=0x7f2f7c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.833+0000 7f2fa4a00640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2f7c0725e0 0x7f2f7c074aa0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0166890 tx=0x7f2f8800b2c0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.833+0000 7f2fa4a00640 1 -- 172.21.15.16:0/561905991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa006f0e0 msgr2=0x7f2fa0139ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.833+0000 7f2fa4a00640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa006f0e0 0x7f2fa0139ff0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2f900b0040 tx=0x7f2f900a5600 comp rx=0 tx=0).stop 2024-09-10T21:17:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.833+0000 7f2fa4a00640 1 -- 172.21.15.16:0/561905991 shutdown_connections 2024-09-10T21:17:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.833+0000 7f2fa4a00640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2f7c0725e0 0x7f2f7c074aa0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.833+0000 7f2fa4a00640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fa00cbee0 0x7f2fa015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.833+0000 7f2fa4a00640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa00cbb00 0x7f2fa0159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.833+0000 7f2fa4a00640 1 --2- 172.21.15.16:0/561905991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa006f0e0 0x7f2fa0139ff0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.833+0000 7f2fa4a00640 1 -- 172.21.15.16:0/561905991 >> 172.21.15.16:0/561905991 conn(0x7f2fa00b16a0 msgr2=0x7f2fa015b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.833+0000 7f2fa4a00640 1 -- 172.21.15.16:0/561905991 shutdown_connections 2024-09-10T21:17:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:15.833+0000 7f2fa4a00640 1 -- 172.21.15.16:0/561905991 wait complete. 2024-09-10T21:17:15.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-09-10T21:17:16.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.001+0000 7faef4c00640 1 -- 172.21.15.16:0/1328087181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faef015e050 msgr2=0x7faef00d0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:16.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.001+0000 7faef4c00640 1 --2- 172.21.15.16:0/1328087181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faef015e050 0x7faef00d0030 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7faef0073750 tx=0x7faee001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.001+0000 7faef4c00640 1 -- 172.21.15.16:0/1328087181 shutdown_connections 2024-09-10T21:17:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.001+0000 7faef4c00640 1 --2- 172.21.15.16:0/1328087181 >> v2:172.21.15.52:3300/0 conn(0x7faef00b0070 0x7faef00cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.001+0000 7faef4c00640 1 --2- 172.21.15.16:0/1328087181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faef015e050 0x7faef00d0030 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.001+0000 7faef4c00640 1 -- 172.21.15.16:0/1328087181 >> 172.21.15.16:0/1328087181 conn(0x7faef00cd6a0 msgr2=0x7faef00cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.001+0000 7faef4c00640 1 -- 172.21.15.16:0/1328087181 shutdown_connections 2024-09-10T21:17:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.001+0000 7faef4c00640 1 -- 172.21.15.16:0/1328087181 wait complete. 2024-09-10T21:17:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faef4c00640 1 Processor -- start 2024-09-10T21:17:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faef4c00640 1 -- start start 2024-09-10T21:17:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faef4c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faef00b0070 0x7faef01537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faef4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faef00cb8b0 0x7faef00b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faef4c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef015e050 0x7faef00b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faef4c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faef016b390 con 0x7faef00cb8b0 2024-09-10T21:17:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faef4c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faef0074a90 con 0x7faef015e050 2024-09-10T21:17:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faef4c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7faef016b510 con 0x7faef00b0070 2024-09-10T21:17:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faee7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faef00cb8b0 0x7faef00b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faee7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faef00cb8b0 0x7faef00b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47348/0 (socket says 172.21.15.16:47348) 2024-09-10T21:17:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faee7e00640 1 -- 172.21.15.16:0/1603193587 learned_addr learned my addr 172.21.15.16:0/1603193587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faeee800640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef015e050 0x7faef00b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faeede00640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faef00b0070 0x7faef01537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faee7e00640 1 -- 172.21.15.16:0/1603193587 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faef00b0070 msgr2=0x7faef01537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faee7e00640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faef00b0070 0x7faef01537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faee7e00640 1 -- 172.21.15.16:0/1603193587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef015e050 msgr2=0x7faef00b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faee7e00640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef015e050 0x7faef00b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faee7e00640 1 -- 172.21.15.16:0/1603193587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef0154110 con 0x7faef00cb8b0 2024-09-10T21:17:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faee7e00640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faef00cb8b0 0x7faef00b4560 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7faed800ea20 tx=0x7faed800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faee6000640 1 -- 172.21.15.16:0/1603193587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed800cd80 con 0x7faef00cb8b0 2024-09-10T21:17:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faee6000640 1 -- 172.21.15.16:0/1603193587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faed8010430 con 0x7faef00cb8b0 2024-09-10T21:17:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faee6000640 1 -- 172.21.15.16:0/1603193587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8010860 con 0x7faef00cb8b0 2024-09-10T21:17:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faef4c00640 1 -- 172.21.15.16:0/1603193587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef01550e0 con 0x7faef00cb8b0 2024-09-10T21:17:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faef4c00640 1 -- 172.21.15.16:0/1603193587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef0157530 con 0x7faef00cb8b0 2024-09-10T21:17:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.005+0000 7faef4c00640 1 -- 172.21.15.16:0/1603193587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef00cb370 con 0x7faef00cb8b0 2024-09-10T21:17:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.009+0000 7faee6000640 1 -- 172.21.15.16:0/1603193587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faed8010a00 con 0x7faef00cb8b0 2024-09-10T21:17:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.009+0000 7faee6000640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faecc072610 0x7faecc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.009+0000 7faee6000640 1 -- 172.21.15.16:0/1603193587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7faed8093e40 con 0x7faef00cb8b0 2024-09-10T21:17:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.009+0000 7faeede00640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faecc072610 0x7faecc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.009+0000 7faeede00640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faecc072610 0x7faecc074ad0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7faee00af000 tx=0x7faee001b100 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:16.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.013+0000 7faee6000640 1 -- 172.21.15.16:0/1603193587 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faed805f4d0 con 0x7faef00cb8b0 2024-09-10T21:17:16.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faef4c00640 1 -- 172.21.15.16:0/1603193587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7faef00cc150 con 0x7faef00cb8b0 2024-09-10T21:17:16.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faee6000640 1 -- 172.21.15.16:0/1603193587 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v51) ==== 106+0+8 (secure 0 0 0) 0x7faed805edd0 con 0x7faef00cb8b0 2024-09-10T21:17:16.193 INFO:teuthology.orchestra.run.smithi016.stdout:1048576 2024-09-10T21:17:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faef4c00640 1 -- 172.21.15.16:0/1603193587 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faecc072610 msgr2=0x7faecc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faef4c00640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faecc072610 0x7faecc074ad0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7faee00af000 tx=0x7faee001b100 comp rx=0 tx=0).stop 2024-09-10T21:17:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faef4c00640 1 -- 172.21.15.16:0/1603193587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faef00cb8b0 msgr2=0x7faef00b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faef4c00640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faef00cb8b0 0x7faef00b4560 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7faed800ea20 tx=0x7faed800eee0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faef4c00640 1 -- 172.21.15.16:0/1603193587 shutdown_connections 2024-09-10T21:17:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faef4c00640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faecc072610 0x7faecc074ad0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faef4c00640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef015e050 0x7faef00b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faef4c00640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faef00cb8b0 0x7faef00b4560 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faef4c00640 1 --2- 172.21.15.16:0/1603193587 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faef00b0070 0x7faef01537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faef4c00640 1 -- 172.21.15.16:0/1603193587 >> 172.21.15.16:0/1603193587 conn(0x7faef00cd6a0 msgr2=0x7faef00ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:16.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faef4c00640 1 -- 172.21.15.16:0/1603193587 shutdown_connections 2024-09-10T21:17:16.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.189+0000 7faef4c00640 1 -- 172.21.15.16:0/1603193587 wait complete. 2024-09-10T21:17:16.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-09-10T21:17:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2591355629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3a4003a60 msgr2=0x7ff3a4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 --2- 172.21.15.16:0/2591355629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3a4003a60 0x7ff3a4003e80 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009b60 tx=0x7ff3b001d270 comp rx=0 tx=0).stop 2024-09-10T21:17:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2591355629 shutdown_connections 2024-09-10T21:17:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 --2- 172.21.15.16:0/2591355629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3a4003a60 0x7ff3a4003e80 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2591355629 >> 172.21.15.16:0/2591355629 conn(0x7ff3c00d1df0 msgr2=0x7ff3c00d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:16.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2591355629 shutdown_connections 2024-09-10T21:17:16.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2591355629 wait complete. 2024-09-10T21:17:16.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 Processor -- start 2024-09-10T21:17:16.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 -- start start 2024-09-10T21:17:16.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3a4003a60 0x7ff3c00cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:16.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3c0155f90 0x7ff3c00c4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:16.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c00c4d80 0x7ff3c00aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:16.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c00d2ad0 con 0x7ff3c00c4d80 2024-09-10T21:17:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c00b14f0 con 0x7ff3a4003a60 2024-09-10T21:17:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.361+0000 7ff3c5a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c00ce590 con 0x7ff3c0155f90 2024-09-10T21:17:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3bf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c00c4d80 0x7ff3c00aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3bea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3a4003a60 0x7ff3c00cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3b7e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3c0155f90 0x7ff3c00c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3bf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c00c4d80 0x7ff3c00aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47362/0 (socket says 172.21.15.16:47362) 2024-09-10T21:17:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3bf400640 1 -- 172.21.15.16:0/2764094350 learned_addr learned my addr 172.21.15.16:0/2764094350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3bf400640 1 -- 172.21.15.16:0/2764094350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3c0155f90 msgr2=0x7ff3c00c4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3bf400640 1 --2- 172.21.15.16:0/2764094350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3c0155f90 0x7ff3c00c4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3bf400640 1 -- 172.21.15.16:0/2764094350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3a4003a60 msgr2=0x7ff3c00cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3bf400640 1 --2- 172.21.15.16:0/2764094350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3a4003a60 0x7ff3c00cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3bf400640 1 -- 172.21.15.16:0/2764094350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c00ca5a0 con 0x7ff3c00c4d80 2024-09-10T21:17:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3bf400640 1 --2- 172.21.15.16:0/2764094350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c00c4d80 0x7ff3c00aff50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00b7e70 tx=0x7ff3b80019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3b6a00640 1 -- 172.21.15.16:0/2764094350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80107d0 con 0x7ff3c00c4d80 2024-09-10T21:17:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3b6a00640 1 -- 172.21.15.16:0/2764094350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3b8016070 con 0x7ff3c00c4d80 2024-09-10T21:17:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3b6a00640 1 -- 172.21.15.16:0/2764094350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8010c80 con 0x7ff3c00c4d80 2024-09-10T21:17:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2764094350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c0156410 con 0x7ff3c00c4d80 2024-09-10T21:17:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2764094350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c00cab20 con 0x7ff3c00c4d80 2024-09-10T21:17:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2764094350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c00ceaa0 con 0x7ff3c00c4d80 2024-09-10T21:17:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.365+0000 7ff3b6a00640 1 -- 172.21.15.16:0/2764094350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff3b801b460 con 0x7ff3c00c4d80 2024-09-10T21:17:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.369+0000 7ff3b6a00640 1 --2- 172.21.15.16:0/2764094350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff39c072610 0x7ff39c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.369+0000 7ff3b6a00640 1 -- 172.21.15.16:0/2764094350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7ff3b8093a10 con 0x7ff3c00c4d80 2024-09-10T21:17:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.369+0000 7ff3bea00640 1 --2- 172.21.15.16:0/2764094350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff39c072610 0x7ff39c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.369+0000 7ff3bea00640 1 --2- 172.21.15.16:0/2764094350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff39c072610 0x7ff39c074ad0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00ae000 tx=0x7ff3b001a010 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:16.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.373+0000 7ff3b6a00640 1 -- 172.21.15.16:0/2764094350 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff3b8010e20 con 0x7ff3c00c4d80 2024-09-10T21:17:16.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.545+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2764094350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7ff3c00b1d60 con 0x7ff3c00c4d80 2024-09-10T21:17:16.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.545+0000 7ff3b6a00640 1 -- 172.21.15.16:0/2764094350 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v51) ==== 112+0+10 (secure 0 0 0) 0x7ff3b805e9c0 con 0x7ff3c00c4d80 2024-09-10T21:17:16.550 INFO:teuthology.orchestra.run.smithi016.stdout:60.000000 2024-09-10T21:17:16.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.545+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2764094350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff39c072610 msgr2=0x7ff39c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:16.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.545+0000 7ff3c5a00640 1 --2- 172.21.15.16:0/2764094350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff39c072610 0x7ff39c074ad0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00ae000 tx=0x7ff3b001a010 comp rx=0 tx=0).stop 2024-09-10T21:17:16.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.545+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2764094350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c00c4d80 msgr2=0x7ff3c00aff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:16.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.545+0000 7ff3c5a00640 1 --2- 172.21.15.16:0/2764094350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c00c4d80 0x7ff3c00aff50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00b7e70 tx=0x7ff3b80019c0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.549+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2764094350 shutdown_connections 2024-09-10T21:17:16.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.549+0000 7ff3c5a00640 1 --2- 172.21.15.16:0/2764094350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff39c072610 0x7ff39c074ad0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.549+0000 7ff3c5a00640 1 --2- 172.21.15.16:0/2764094350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3c00c4d80 0x7ff3c00aff50 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.549+0000 7ff3c5a00640 1 --2- 172.21.15.16:0/2764094350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff3c0155f90 0x7ff3c00c4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.549+0000 7ff3c5a00640 1 --2- 172.21.15.16:0/2764094350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3a4003a60 0x7ff3c00cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.549+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2764094350 >> 172.21.15.16:0/2764094350 conn(0x7ff3c00d1df0 msgr2=0x7ff3c00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:16.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.549+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2764094350 shutdown_connections 2024-09-10T21:17:16.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.549+0000 7ff3c5a00640 1 -- 172.21.15.16:0/2764094350 wait complete. 2024-09-10T21:17:16.570 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:17:16.570 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:16.579 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:17:16.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:17:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:17:16.685 INFO:teuthology.orchestra.run.smithi016.stdout:client136013 2024-09-10T21:17:16.685 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:17:16.685 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:16.694 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> import fcntl 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> import time 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> n = 800 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> if not os.path.exists(abs_path): 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(abs_path) 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:17:16.695 DEBUG:teuthology.orchestra.run.smithi016:> handles = [] 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, n): 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> fname = "file_"+str(i) 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> path = os.path.join(abs_path, fname) 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> handles.append(open(path, '"'"'w'"'"')) 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> print("waiting with handles open", file=sys.stderr) 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> while True: 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> print("open_n_background: keeping files open", file=sys.stderr) 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> if os.read(0, 4096) == b"": 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> break 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> except BlockingIOError: 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> pass 2024-09-10T21:17:16.696 DEBUG:teuthology.orchestra.run.smithi016:> time.sleep(2) 2024-09-10T21:17:16.697 DEBUG:teuthology.orchestra.run.smithi016:> print("stdin closed, goodbye!", file=sys.stderr) 2024-09-10T21:17:16.697 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-10T21:17:16.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:16.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.889+0000 7f3fb3200640 1 -- 172.21.15.16:0/3404157028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fac15f400 msgr2=0x7f3fac070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:16.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.889+0000 7f3fb3200640 1 --2- 172.21.15.16:0/3404157028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fac15f400 0x7f3fac070fe0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c005fd0 tx=0x7f3f9c01c700 comp rx=0 tx=0).stop 2024-09-10T21:17:16.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.889+0000 7f3fb3200640 1 -- 172.21.15.16:0/3404157028 shutdown_connections 2024-09-10T21:17:16.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.889+0000 7f3fb3200640 1 --2- 172.21.15.16:0/3404157028 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3fac0b0590 0x7f3fac071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.889+0000 7f3fb3200640 1 --2- 172.21.15.16:0/3404157028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fac15f400 0x7f3fac070fe0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.889+0000 7f3fb3200640 1 -- 172.21.15.16:0/3404157028 >> 172.21.15.16:0/3404157028 conn(0x7f3fac09b130 msgr2=0x7f3fac0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:16.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.889+0000 7f3fb3200640 1 -- 172.21.15.16:0/3404157028 shutdown_connections 2024-09-10T21:17:16.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.889+0000 7f3fb3200640 1 -- 172.21.15.16:0/3404157028 wait complete. 2024-09-10T21:17:16.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.889+0000 7f3fb3200640 1 Processor -- start 2024-09-10T21:17:16.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.889+0000 7f3fb3200640 1 -- start start 2024-09-10T21:17:16.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fb3200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3fac0b0590 0x7f3fac153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:16.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fb3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fac0b13c0 0x7f3fac0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:16.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fb3200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fac15f400 0x7f3fac0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fb3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac168f60 con 0x7f3fac0b13c0 2024-09-10T21:17:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fb3200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac168de0 con 0x7f3fac15f400 2024-09-10T21:17:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fb3200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3fac1690e0 con 0x7f3fac0b0590 2024-09-10T21:17:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fac0b13c0 0x7f3fac0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fac0b13c0 0x7f3fac0b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47374/0 (socket says 172.21.15.16:47374) 2024-09-10T21:17:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fab400640 1 -- 172.21.15.16:0/1455158085 learned_addr learned my addr 172.21.15.16:0/1455158085 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fabe00640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3fac0b0590 0x7f3fac153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fb1200640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fac15f400 0x7f3fac0cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fab400640 1 -- 172.21.15.16:0/1455158085 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3fac0b0590 msgr2=0x7f3fac153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fab400640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3fac0b0590 0x7f3fac153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fab400640 1 -- 172.21.15.16:0/1455158085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fac15f400 msgr2=0x7f3fac0cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fab400640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fac15f400 0x7f3fac0cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:16.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fab400640 1 -- 172.21.15.16:0/1455158085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fac0cdff0 con 0x7f3fac0b13c0 2024-09-10T21:17:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fab400640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fac0b13c0 0x7f3fac0b5150 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800c490 tx=0x7f3f9800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fa3e00640 1 -- 172.21.15.16:0/1455158085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98011790 con 0x7f3fac0b13c0 2024-09-10T21:17:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fa3e00640 1 -- 172.21.15.16:0/1455158085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f98017070 con 0x7f3fac0b13c0 2024-09-10T21:17:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fb3200640 1 -- 172.21.15.16:0/1455158085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fac15c100 con 0x7f3fac0b13c0 2024-09-10T21:17:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fa3e00640 1 -- 172.21.15.16:0/1455158085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98011ce0 con 0x7f3fac0b13c0 2024-09-10T21:17:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fb3200640 1 -- 172.21.15.16:0/1455158085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fac15c6c0 con 0x7f3fac0b13c0 2024-09-10T21:17:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.893+0000 7f3fb3200640 1 -- 172.21.15.16:0/1455158085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fac0cfc80 con 0x7f3fac0b13c0 2024-09-10T21:17:16.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.897+0000 7f3fa3e00640 1 -- 172.21.15.16:0/1455158085 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3f9801c460 con 0x7f3fac0b13c0 2024-09-10T21:17:16.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.897+0000 7f3fa3e00640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3f8c0725e0 0x7f3f8c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:16.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.897+0000 7f3fa3e00640 1 -- 172.21.15.16:0/1455158085 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f3f98092f70 con 0x7f3fac0b13c0 2024-09-10T21:17:16.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.897+0000 7f3fabe00640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3f8c0725e0 0x7f3f8c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:16.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.897+0000 7f3fabe00640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3f8c0725e0 0x7f3f8c074aa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0ae000 tx=0x7f3f9c0b2040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:16.901+0000 7f3fa3e00640 1 -- 172.21.15.16:0/1455158085 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3f9805e6c0 con 0x7f3fac0b13c0 2024-09-10T21:17:17.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.125+0000 7f3fb3200640 1 -- 172.21.15.16:0/1455158085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fac15e820 con 0x7f3fac0b13c0 2024-09-10T21:17:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.125+0000 7f3fa3e00640 1 -- 172.21.15.16:0/1455158085 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5235 (secure 0 0 0) 0x7f3f9805dfc0 con 0x7f3fac0b13c0 2024-09-10T21:17:17.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:17.129 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"btime":"2024-09-10T21:17:08:717481+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3765085271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765085271},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765085271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":7104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4198210360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":4198210360},{"type":"v1","addr":"172.21.15.16:6837","nonce":4198210360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2954517211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2954517211},{"type":"v1","addr":"172.21.15.52:6835","nonce":2954517211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:05.094283+0000","modified":"2024-09-10T21:17:08.119983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135926":{"gid":135926,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1108687782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108687782},{"type":"v1","addr":"172.21.15.52:6834","nonce":1108687782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135926,"qdb_cluster":[135926]},"id":8}]} 2024-09-10T21:17:17.130 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 71 2024-09-10T21:17:17.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.125+0000 7f3fb3200640 1 -- 172.21.15.16:0/1455158085 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3f8c0725e0 msgr2=0x7f3f8c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.125+0000 7f3fb3200640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3f8c0725e0 0x7f3f8c074aa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0ae000 tx=0x7f3f9c0b2040 comp rx=0 tx=0).stop 2024-09-10T21:17:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.125+0000 7f3fb3200640 1 -- 172.21.15.16:0/1455158085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fac0b13c0 msgr2=0x7f3fac0b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.125+0000 7f3fb3200640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fac0b13c0 0x7f3fac0b5150 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800c490 tx=0x7f3f9800c890 comp rx=0 tx=0).stop 2024-09-10T21:17:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.125+0000 7f3fb3200640 1 -- 172.21.15.16:0/1455158085 shutdown_connections 2024-09-10T21:17:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.125+0000 7f3fb3200640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3f8c0725e0 0x7f3f8c074aa0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.125+0000 7f3fb3200640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fac15f400 0x7f3fac0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.125+0000 7f3fb3200640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fac0b13c0 0x7f3fac0b5150 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.125+0000 7f3fb3200640 1 --2- 172.21.15.16:0/1455158085 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3fac0b0590 0x7f3fac153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.125+0000 7f3fb3200640 1 -- 172.21.15.16:0/1455158085 >> 172.21.15.16:0/1455158085 conn(0x7f3fac09b130 msgr2=0x7f3fac15aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.125+0000 7f3fb3200640 1 -- 172.21.15.16:0/1455158085 shutdown_connections 2024-09-10T21:17:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:17.129+0000 7f3fb3200640 1 -- 172.21.15.16:0/1455158085 wait complete. 2024-09-10T21:17:17.152 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-10T21:17:17.323 INFO:teuthology.orchestra.run.smithi052.stdout:[{"id":7245,"entity":{"name":{"type":"client","num":7245},"addr":{"type":"any","addr":"172.21.15.52:0","nonce":2089774086}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":3.4079765059999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.989856850681512,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7245 172.21.15.52:0/2089774086","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi052","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":136013,"entity":{"name":{"type":"client","num":136013},"addr":{"type":"any","addr":"172.21.15.16:0","nonce":1871709369}},"state":"open","num_leases":1,"num_caps":434,"request_load_avg":457,"uptime":4.5679685269999997,"requests_in_flight":432,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":868.49222649296132,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.136013 172.21.15.16:0/1871709369","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-10T21:17:17.324 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi052", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.52:0", "nonce": 2089774086, "type": "any" }, "name": { "num": 7245, "type": "client" } }, "id": 7245, "inst": "client.7245 172.21.15.52:0/2089774086", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.989856850681512 }, "state": "open", "uptime": 3.407976506 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 1871709369, "type": "any" }, "name": { "num": 136013, "type": "client" } }, "id": 136013, "inst": "client.136013 172.21.15.16:0/1871709369", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 434, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 457, "requests_in_flight": 432, "session_cache_liveness": { "halflife": 300, "value": 868.4922264929613 }, "state": "open", "uptime": 4.567968527 } ] 2024-09-10T21:17:17.324 DEBUG:tasks.ceph_test_case:wait_until_equal: 434 != 802, waiting (timeout=600)... 2024-09-10T21:17:17.914 INFO:teuthology.orchestra.run.smithi016.stderr:waiting with handles open 2024-09-10T21:17:17.914 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:17:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:17:21.918 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:17:22.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:22.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 -- 172.21.15.16:0/2953039349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd240ca170 msgr2=0x7fdd240ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 --2- 172.21.15.16:0/2953039349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd240ca170 0x7fdd240ca620 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00ad90 tx=0x7fdd1c01dc20 comp rx=0 tx=0).stop 2024-09-10T21:17:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 -- 172.21.15.16:0/2953039349 shutdown_connections 2024-09-10T21:17:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 --2- 172.21.15.16:0/2953039349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd240ca170 0x7fdd240ca620 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 --2- 172.21.15.16:0/2953039349 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdd2415a270 0x7fdd240ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 -- 172.21.15.16:0/2953039349 >> 172.21.15.16:0/2953039349 conn(0x7fdd240cfb20 msgr2=0x7fdd240cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 -- 172.21.15.16:0/2953039349 shutdown_connections 2024-09-10T21:17:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 -- 172.21.15.16:0/2953039349 wait complete. 2024-09-10T21:17:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 Processor -- start 2024-09-10T21:17:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 -- start start 2024-09-10T21:17:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd240ca170 0x7fdd240c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdd240d06b0 0x7fdd2413ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd2415a270 0x7fdd241551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24072c80 con 0x7fdd240ca170 2024-09-10T21:17:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24072b00 con 0x7fdd2415a270 2024-09-10T21:17:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd29200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fdd24072e00 con 0x7fdd240d06b0 2024-09-10T21:17:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd21e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd240ca170 0x7fdd240c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd21e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd240ca170 0x7fdd240c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55042/0 (socket says 172.21.15.16:55042) 2024-09-10T21:17:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.481+0000 7fdd21e00640 1 -- 172.21.15.16:0/2528193122 learned_addr learned my addr 172.21.15.16:0/2528193122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd22800640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd2415a270 0x7fdd241551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd1be00640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdd240d06b0 0x7fdd2413ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd21e00640 1 -- 172.21.15.16:0/2528193122 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdd240d06b0 msgr2=0x7fdd2413ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd21e00640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdd240d06b0 0x7fdd2413ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd21e00640 1 -- 172.21.15.16:0/2528193122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd2415a270 msgr2=0x7fdd241551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd21e00640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd2415a270 0x7fdd241551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd21e00640 1 -- 172.21.15.16:0/2528193122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd240c5d30 con 0x7fdd240ca170 2024-09-10T21:17:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd21e00640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd240ca170 0x7fdd240c9490 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400e7a0 tx=0x7fdd1400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd1a000640 1 -- 172.21.15.16:0/2528193122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1400ccc0 con 0x7fdd240ca170 2024-09-10T21:17:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd1a000640 1 -- 172.21.15.16:0/2528193122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd14010430 con 0x7fdd240ca170 2024-09-10T21:17:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd1a000640 1 -- 172.21.15.16:0/2528193122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14010860 con 0x7fdd240ca170 2024-09-10T21:17:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd29200640 1 -- 172.21.15.16:0/2528193122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd240c6010 con 0x7fdd240ca170 2024-09-10T21:17:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd29200640 1 -- 172.21.15.16:0/2528193122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd24151450 con 0x7fdd240ca170 2024-09-10T21:17:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd29200640 1 -- 172.21.15.16:0/2528193122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd240ca620 con 0x7fdd240ca170 2024-09-10T21:17:22.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.485+0000 7fdd1a000640 1 -- 172.21.15.16:0/2528193122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdd14010a00 con 0x7fdd240ca170 2024-09-10T21:17:22.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.489+0000 7fdd1a000640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd00072610 0x7fdd00074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:22.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.489+0000 7fdd1a000640 1 -- 172.21.15.16:0/2528193122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7fdd14093c90 con 0x7fdd240ca170 2024-09-10T21:17:22.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.489+0000 7fdd1be00640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd00072610 0x7fdd00074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.489+0000 7fdd1be00640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd00072610 0x7fdd00074ad0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c000f70 tx=0x7fdd0c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:22.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.493+0000 7fdd1a000640 1 -- 172.21.15.16:0/2528193122 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdd1405ec40 con 0x7fdd240ca170 2024-09-10T21:17:22.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.701+0000 7fdd29200640 1 -- 172.21.15.16:0/2528193122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd240ce930 con 0x7fdd240ca170 2024-09-10T21:17:22.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.701+0000 7fdd1a000640 1 -- 172.21.15.16:0/2528193122 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5235 (secure 0 0 0) 0x7fdd14063670 con 0x7fdd240ca170 2024-09-10T21:17:22.706 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:22.706 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"btime":"2024-09-10T21:17:21:843659+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3765085271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765085271},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765085271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":7104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4198210360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":4198210360},{"type":"v1","addr":"172.21.15.16:6837","nonce":4198210360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2954517211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2954517211},{"type":"v1","addr":"172.21.15.52:6835","nonce":2954517211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:05.094283+0000","modified":"2024-09-10T21:17:21.746067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135926":{"gid":135926,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":7,"addr":"172.21.15.52:6834/1108687782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108687782},{"type":"v1","addr":"172.21.15.52:6834","nonce":1108687782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135926,"qdb_cluster":[135926]},"id":8}]} 2024-09-10T21:17:22.707 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 73 2024-09-10T21:17:22.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.705+0000 7fdd29200640 1 -- 172.21.15.16:0/2528193122 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd00072610 msgr2=0x7fdd00074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:22.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.705+0000 7fdd29200640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd00072610 0x7fdd00074ad0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c000f70 tx=0x7fdd0c009040 comp rx=0 tx=0).stop 2024-09-10T21:17:22.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.705+0000 7fdd29200640 1 -- 172.21.15.16:0/2528193122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd240ca170 msgr2=0x7fdd240c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:22.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.705+0000 7fdd29200640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd240ca170 0x7fdd240c9490 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400e7a0 tx=0x7fdd1400ec60 comp rx=0 tx=0).stop 2024-09-10T21:17:22.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.705+0000 7fdd29200640 1 -- 172.21.15.16:0/2528193122 shutdown_connections 2024-09-10T21:17:22.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.705+0000 7fdd29200640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd00072610 0x7fdd00074ad0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:22.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.705+0000 7fdd29200640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd2415a270 0x7fdd241551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:22.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.705+0000 7fdd29200640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdd240d06b0 0x7fdd2413ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:22.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.705+0000 7fdd29200640 1 --2- 172.21.15.16:0/2528193122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd240ca170 0x7fdd240c9490 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:22.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.705+0000 7fdd29200640 1 -- 172.21.15.16:0/2528193122 >> 172.21.15.16:0/2528193122 conn(0x7fdd240cfb20 msgr2=0x7fdd240cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:22.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.705+0000 7fdd29200640 1 -- 172.21.15.16:0/2528193122 shutdown_connections 2024-09-10T21:17:22.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:22.705+0000 7fdd29200640 1 -- 172.21.15.16:0/2528193122 wait complete. 2024-09-10T21:17:22.728 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-10T21:17:22.907 INFO:teuthology.orchestra.run.smithi052.stdout:[{"id":7245,"entity":{"name":{"type":"client","num":7245},"addr":{"type":"any","addr":"172.21.15.52:0","nonce":2089774086}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":8.9919378420000005,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9515304134269202,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7245 172.21.15.52:0/2089774086","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi052","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":136013,"entity":{"name":{"type":"client","num":136013},"addr":{"type":"any","addr":"172.21.15.16:0","nonce":1871709369}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":775,"uptime":10.151929862999999,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":2327.9502931774819,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":943.61567379580833,"halflife":1.5},"recall_caps_throttle2o":{"value":431.30174781615113,"halflife":0.5},"session_cache_liveness":{"value":1584.4899679492544,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":530,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.136013 172.21.15.16:0/1871709369","completed_requests":[{"tid":530,"created_ino":"0x100000003e9"},{"tid":531,"created_ino":"0x100000003ea"},{"tid":532,"created_ino":"0x100000003eb"},{"tid":533,"created_ino":"0x100000003ec"},{"tid":534,"created_ino":"0x100000003ed"},{"tid":535,"created_ino":"0x100000003ee"},{"tid":536,"created_ino":"0x100000003ef"},{"tid":537,"created_ino":"0x100000003f0"},{"tid":538,"created_ino":"0x100000003f1"},{"tid":539,"created_ino":"0x100000003f2"},{"tid":540,"created_ino":"0x100000003f3"},{"tid":541,"created_ino":"0x100000003f4"},{"tid":542,"created_ino":"0x100000003f5"},{"tid":543,"created_ino":"0x100000003f6"},{"tid":544,"created_ino":"0x100000003f7"},{"tid":545,"created_ino":"0x100000003f8"},{"tid":546,"created_ino":"0x100000003f9"},{"tid":547,"created_ino":"0x100000003fa"},{"tid":548,"created_ino":"0x100000003fb"},{"tid":549,"created_ino":"0x100000003fc"},{"tid":550,"created_ino":"0x100000003fd"},{"tid":551,"created_ino":"0x100000003fe"},{"tid":552,"created_ino":"0x100000003ff"},{"tid":553,"created_ino":"0x10000000400"},{"tid":554,"created_ino":"0x10000000401"},{"tid":555,"created_ino":"0x10000000402"},{"tid":556,"created_ino":"0x10000000403"},{"tid":557,"created_ino":"0x10000000404"},{"tid":558,"created_ino":"0x10000000405"},{"tid":559,"created_ino":"0x10000000406"},{"tid":560,"created_ino":"0x10000000407"},{"tid":561,"created_ino":"0x10000000408"},{"tid":562,"created_ino":"0x10000000409"},{"tid":563,"created_ino":"0x1000000040a"},{"tid":564,"created_ino":"0x1000000040b"},{"tid":565,"created_ino":"0x1000000040c"},{"tid":566,"created_ino":"0x1000000040d"},{"tid":567,"created_ino":"0x1000000040e"},{"tid":568,"created_ino":"0x1000000040f"},{"tid":569,"created_ino":"0x10000000410"},{"tid":570,"created_ino":"0x10000000411"},{"tid":571,"created_ino":"0x10000000412"},{"tid":572,"created_ino":"0x10000000413"},{"tid":573,"created_ino":"0x10000000414"},{"tid":574,"created_ino":"0x10000000415"},{"tid":575,"created_ino":"0x10000000416"},{"tid":576,"created_ino":"0x10000000417"},{"tid":577,"created_ino":"0x10000000418"},{"tid":578,"created_ino":"0x10000000419"},{"tid":579,"created_ino":"0x1000000041a"},{"tid":580,"created_ino":"0x1000000041b"},{"tid":581,"created_ino":"0x1000000041c"},{"tid":582,"created_ino":"0x1000000041d"},{"tid":583,"created_ino":"0x1000000041e"},{"tid":584,"created_ino":"0x1000000041f"},{"tid":585,"created_ino":"0x10000000420"},{"tid":586,"created_ino":"0x10000000421"},{"tid":587,"created_ino":"0x10000000422"},{"tid":588,"created_ino":"0x10000000423"},{"tid":589,"created_ino":"0x10000000424"},{"tid":590,"created_ino":"0x10000000425"},{"tid":591,"created_ino":"0x10000000426"},{"tid":592,"created_ino":"0x10000000427"},{"tid":593,"created_ino":"0x10000000428"},{"tid":594,"created_ino":"0x10000000429"},{"tid":595,"created_ino":"0x1000000042a"},{"tid":596,"created_ino":"0x1000000042b"},{"tid":597,"created_ino":"0x1000000042c"},{"tid":598,"created_ino":"0x1000000042d"},{"tid":599,"created_ino":"0x1000000042e"},{"tid":600,"created_ino":"0x1000000042f"},{"tid":601,"created_ino":"0x10000000430"},{"tid":602,"created_ino":"0x10000000431"},{"tid":603,"created_ino":"0x10000000432"},{"tid":604,"created_ino":"0x10000000433"},{"tid":605,"created_ino":"0x10000000434"},{"tid":606,"created_ino":"0x10000000435"},{"tid":607,"created_ino":"0x10000000436"},{"tid":608,"created_ino":"0x10000000437"},{"tid":609,"created_ino":"0x10000000438"},{"tid":610,"created_ino":"0x10000000439"},{"tid":611,"created_ino":"0x1000000043a"},{"tid":612,"created_ino":"0x1000000043b"},{"tid":613,"created_ino":"0x1000000043c"},{"tid":614,"created_ino":"0x1000000043d"},{"tid":615,"created_ino":"0x1000000043e"},{"tid":616,"created_ino":"0x1000000043f"},{"tid":617,"created_ino":"0x10000000440"},{"tid":618,"created_ino":"0x10000000441"},{"tid":619,"created_ino":"0x10000000442"},{"tid":620,"created_ino":"0x10000000443"},{"tid":621,"created_ino":"0x10000000444"},{"tid":622,"created_ino":"0x10000000445"},{"tid":623,"created_ino":"0x10000000446"},{"tid":624,"created_ino":"0x10000000447"},{"tid":625,"created_ino":"0x10000000448"},{"tid":626,"created_ino":"0x10000000449"},{"tid":627,"created_ino":"0x1000000044a"},{"tid":628,"created_ino":"0x1000000044b"},{"tid":629,"created_ino":"0x1000000044c"},{"tid":630,"created_ino":"0x1000000044d"},{"tid":631,"created_ino":"0x1000000044e"},{"tid":632,"created_ino":"0x1000000044f"},{"tid":633,"created_ino":"0x10000000450"},{"tid":634,"created_ino":"0x10000000451"},{"tid":635,"created_ino":"0x10000000452"},{"tid":636,"created_ino":"0x10000000453"},{"tid":637,"created_ino":"0x10000000454"},{"tid":638,"created_ino":"0x10000000455"},{"tid":639,"created_ino":"0x10000000456"},{"tid":640,"created_ino":"0x10000000457"},{"tid":641,"created_ino":"0x10000000458"},{"tid":642,"created_ino":"0x10000000459"},{"tid":643,"created_ino":"0x1000000045a"},{"tid":644,"created_ino":"0x1000000045b"},{"tid":645,"created_ino":"0x1000000045c"},{"tid":646,"created_ino":"0x1000000045d"},{"tid":647,"created_ino":"0x1000000045e"},{"tid":648,"created_ino":"0x1000000045f"},{"tid":649,"created_ino":"0x10000000460"},{"tid":650,"created_ino":"0x10000000461"},{"tid":651,"created_ino":"0x10000000462"},{"tid":652,"created_ino":"0x10000000463"},{"tid":653,"created_ino":"0x10000000464"},{"tid":654,"created_ino":"0x10000000465"},{"tid":655,"created_ino":"0x10000000466"},{"tid":656,"created_ino":"0x10000000467"},{"tid":657,"created_ino":"0x10000000468"},{"tid":658,"created_ino":"0x10000000469"},{"tid":659,"created_ino":"0x1000000046a"},{"tid":660,"created_ino":"0x1000000046b"},{"tid":661,"created_ino":"0x1000000046c"},{"tid":662,"created_ino":"0x1000000046d"},{"tid":663,"created_ino":"0x1000000046e"},{"tid":664,"created_ino":"0x1000000046f"},{"tid":665,"created_ino":"0x10000000470"},{"tid":666,"created_ino":"0x10000000471"},{"tid":667,"created_ino":"0x10000000472"},{"tid":668,"created_ino":"0x10000000473"},{"tid":669,"created_ino":"0x10000000474"},{"tid":670,"created_ino":"0x10000000475"},{"tid":671,"created_ino":"0x10000000476"},{"tid":672,"created_ino":"0x10000000477"},{"tid":673,"created_ino":"0x10000000478"},{"tid":674,"created_ino":"0x10000000479"},{"tid":675,"created_ino":"0x1000000047a"},{"tid":676,"created_ino":"0x1000000047b"},{"tid":677,"created_ino":"0x1000000047c"},{"tid":678,"created_ino":"0x1000000047d"},{"tid":679,"created_ino":"0x1000000047e"},{"tid":680,"created_ino":"0x1000000047f"},{"tid":681,"created_ino":"0x10000000480"},{"tid":682,"created_ino":"0x10000000481"},{"tid":683,"created_ino":"0x10000000482"},{"tid":684,"created_ino":"0x10000000483"},{"tid":685,"created_ino":"0x10000000484"},{"tid":686,"created_ino":"0x10000000485"},{"tid":687,"created_ino":"0x10000000486"},{"tid":688,"created_ino":"0x10000000487"},{"tid":689,"created_ino":"0x10000000488"},{"tid":690,"created_ino":"0x10000000489"},{"tid":691,"created_ino":"0x1000000048a"},{"tid":692,"created_ino":"0x1000000048b"},{"tid":693,"created_ino":"0x1000000048c"},{"tid":694,"created_ino":"0x1000000048d"},{"tid":695,"created_ino":"0x1000000048e"},{"tid":696,"created_ino":"0x1000000048f"},{"tid":697,"created_ino":"0x10000000490"},{"tid":698,"created_ino":"0x10000000491"},{"tid":699,"created_ino":"0x10000000492"},{"tid":700,"created_ino":"0x10000000493"},{"tid":701,"created_ino":"0x10000000494"},{"tid":702,"created_ino":"0x10000000495"},{"tid":703,"created_ino":"0x10000000496"},{"tid":704,"created_ino":"0x10000000497"},{"tid":705,"created_ino":"0x10000000498"},{"tid":706,"created_ino":"0x10000000499"},{"tid":707,"created_ino":"0x1000000049a"},{"tid":708,"created_ino":"0x1000000049b"},{"tid":709,"created_ino":"0x1000000049c"},{"tid":710,"created_ino":"0x1000000049d"},{"tid":711,"created_ino":"0x1000000049e"},{"tid":712,"created_ino":"0x1000000049f"},{"tid":713,"created_ino":"0x100000004a0"},{"tid":714,"created_ino":"0x100000004a1"},{"tid":715,"created_ino":"0x100000004a2"},{"tid":716,"created_ino":"0x100000004a3"},{"tid":717,"created_ino":"0x100000004a4"},{"tid":718,"created_ino":"0x100000004a5"},{"tid":719,"created_ino":"0x100000004a6"},{"tid":720,"created_ino":"0x100000004a7"},{"tid":721,"created_ino":"0x100000004a8"},{"tid":722,"created_ino":"0x100000004a9"},{"tid":723,"created_ino":"0x100000004aa"},{"tid":724,"created_ino":"0x100000004ab"},{"tid":725,"created_ino":"0x100000004ac"},{"tid":726,"created_ino":"0x100000004ad"},{"tid":727,"created_ino":"0x100000004ae"},{"tid":728,"created_ino":"0x100000004af"},{"tid":729,"created_ino":"0x100000004b0"},{"tid":730,"created_ino":"0x100000004b1"},{"tid":731,"created_ino":"0x100000004b2"},{"tid":732,"created_ino":"0x100000004b3"},{"tid":733,"created_ino":"0x100000004b4"},{"tid":734,"created_ino":"0x100000004b5"},{"tid":735,"created_ino":"0x100000004b6"},{"tid":736,"created_ino":"0x100000004b7"},{"tid":737,"created_ino":"0x100000004b8"},{"tid":738,"created_ino":"0x100000004b9"},{"tid":739,"created_ino":"0x100000004ba"},{"tid":740,"created_ino":"0x100000004bb"},{"tid":741,"created_ino":"0x100000004bc"},{"tid":742,"created_ino":"0x100000004bd"},{"tid":743,"created_ino":"0x100000004be"},{"tid":744,"created_ino":"0x100000004bf"},{"tid":745,"created_ino":"0x100000004c0"},{"tid":746,"created_ino":"0x100000004c1"},{"tid":747,"created_ino":"0x100000004c2"},{"tid":748,"created_ino":"0x100000004c3"},{"tid":749,"created_ino":"0x100000004c4"},{"tid":750,"created_ino":"0x100000004c5"},{"tid":751,"created_ino":"0x100000004c6"},{"tid":752,"created_ino":"0x100000004c7"},{"tid":753,"created_ino":"0x100000004c8"},{"tid":754,"created_ino":"0x100000004c9"},{"tid":755,"created_ino":"0x100000004ca"},{"tid":756,"created_ino":"0x100000004cb"},{"tid":757,"created_ino":"0x100000004cc"},{"tid":758,"created_ino":"0x100000004cd"},{"tid":759,"created_ino":"0x100000004ce"},{"tid":760,"created_ino":"0x100000004cf"},{"tid":761,"created_ino":"0x100000004d0"},{"tid":762,"created_ino":"0x100000004d1"},{"tid":763,"created_ino":"0x100000004d2"},{"tid":764,"created_ino":"0x100000004d3"},{"tid":765,"created_ino":"0x100000004d4"},{"tid":766,"created_ino":"0x100000004d5"},{"tid":767,"created_ino":"0x100000004d6"},{"tid":768,"created_ino":"0x100000004d7"},{"tid":769,"created_ino":"0x100000004d8"},{"tid":770,"created_ino":"0x100000004d9"},{"tid":771,"created_ino":"0x100000004da"},{"tid":772,"created_ino":"0x100000004db"},{"tid":773,"created_ino":"0x100000004dc"},{"tid":774,"created_ino":"0x100000004dd"},{"tid":775,"created_ino":"0x100000004de"},{"tid":776,"created_ino":"0x100000004df"},{"tid":777,"created_ino":"0x100000004e0"},{"tid":778,"created_ino":"0x100000004e1"},{"tid":779,"created_ino":"0x100000004e2"},{"tid":780,"created_ino":"0x100000004e3"},{"tid":781,"created_ino":"0x100000004e4"},{"tid":782,"created_ino":"0x100000004e5"},{"tid":783,"created_ino":"0x100000004e6"},{"tid":784,"created_ino":"0x100000004e7"},{"tid":785,"created_ino":"0x100000004e8"},{"tid":786,"created_ino":"0x100000004e9"},{"tid":787,"created_ino":"0x100000004ea"},{"tid":788,"created_ino":"0x100000004eb"},{"tid":789,"created_ino":"0x100000004ec"},{"tid":790,"created_ino":"0x100000004ed"},{"tid":791,"created_ino":"0x100000004ee"},{"tid":792,"created_ino":"0x100000004ef"},{"tid":793,"created_ino":"0x100000004f0"},{"tid":794,"created_ino":"0x100000004f1"},{"tid":795,"created_ino":"0x100000004f2"},{"tid":796,"created_ino":"0x100000004f3"},{"tid":797,"created_ino":"0x100000004f4"},{"tid":798,"created_ino":"0x100000004f5"},{"tid":799,"created_ino":"0x100000004f6"},{"tid":800,"created_ino":"0x100000004f7"},{"tid":801,"created_ino":"0x100000004f8"},{"tid":802,"created_ino":"0x100000004f9"},{"tid":803,"created_ino":"0x100000004fa"},{"tid":804,"created_ino":"0x100000004fb"},{"tid":805,"created_ino":"0x100000004fc"},{"tid":806,"created_ino":"0x100000004fd"},{"tid":807,"created_ino":"0x100000004fe"},{"tid":808,"created_ino":"0x100000004ff"},{"tid":809,"created_ino":"0x10000000500"},{"tid":810,"created_ino":"0x10000000501"},{"tid":811,"created_ino":"0x10000000502"},{"tid":812,"created_ino":"0x10000000503"},{"tid":813,"created_ino":"0x10000000504"},{"tid":814,"created_ino":"0x10000000505"},{"tid":815,"created_ino":"0x10000000506"},{"tid":816,"created_ino":"0x10000000507"},{"tid":817,"created_ino":"0x10000000508"},{"tid":818,"created_ino":"0x10000000509"},{"tid":819,"created_ino":"0x1000000050a"},{"tid":820,"created_ino":"0x1000000050b"},{"tid":821,"created_ino":"0x1000000050c"},{"tid":822,"created_ino":"0x1000000050d"},{"tid":823,"created_ino":"0x1000000050e"},{"tid":824,"created_ino":"0x1000000050f"},{"tid":825,"created_ino":"0x10000000510"},{"tid":826,"created_ino":"0x10000000511"},{"tid":827,"created_ino":"0x10000000512"},{"tid":828,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-10T21:17:22.909 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi052", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.52:0", "nonce": 2089774086, "type": "any" }, "name": { "num": 7245, "type": "client" } }, "id": 7245, "inst": "client.7245 172.21.15.52:0/2089774086", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9515304134269202 }, "state": "open", "uptime": 8.991937842 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 530 }, { "created_ino": "0x100000003ea", "tid": 531 }, { "created_ino": "0x100000003eb", "tid": 532 }, { "created_ino": "0x100000003ec", "tid": 533 }, { "created_ino": "0x100000003ed", "tid": 534 }, { "created_ino": "0x100000003ee", "tid": 535 }, { "created_ino": "0x100000003ef", "tid": 536 }, { "created_ino": "0x100000003f0", "tid": 537 }, { "created_ino": "0x100000003f1", "tid": 538 }, { "created_ino": "0x100000003f2", "tid": 539 }, { "created_ino": "0x100000003f3", "tid": 540 }, { "created_ino": "0x100000003f4", "tid": 541 }, { "created_ino": "0x100000003f5", "tid": 542 }, { "created_ino": "0x100000003f6", "tid": 543 }, { "created_ino": "0x100000003f7", "tid": 544 }, { "created_ino": "0x100000003f8", "tid": 545 }, { "created_ino": "0x100000003f9", "tid": 546 }, { "created_ino": "0x100000003fa", "tid": 547 }, { "created_ino": "0x100000003fb", "tid": 548 }, { "created_ino": "0x100000003fc", "tid": 549 }, { "created_ino": "0x100000003fd", "tid": 550 }, { "created_ino": "0x100000003fe", "tid": 551 }, { "created_ino": "0x100000003ff", "tid": 552 }, { "created_ino": "0x10000000400", "tid": 553 }, { "created_ino": "0x10000000401", "tid": 554 }, { "created_ino": "0x10000000402", "tid": 555 }, { "created_ino": "0x10000000403", "tid": 556 }, { "created_ino": "0x10000000404", "tid": 557 }, { "created_ino": "0x10000000405", "tid": 558 }, { "created_ino": "0x10000000406", "tid": 559 }, { "created_ino": "0x10000000407", "tid": 560 }, { "created_ino": "0x10000000408", "tid": 561 }, { "created_ino": "0x10000000409", "tid": 562 }, { "created_ino": "0x1000000040a", "tid": 563 }, { "created_ino": "0x1000000040b", "tid": 564 }, { "created_ino": "0x1000000040c", "tid": 565 }, { "created_ino": "0x1000000040d", "tid": 566 }, { "created_ino": "0x1000000040e", "tid": 567 }, { "created_ino": "0x1000000040f", "tid": 568 }, { "created_ino": "0x10000000410", "tid": 569 }, { "created_ino": "0x10000000411", "tid": 570 }, { "created_ino": "0x10000000412", "tid": 571 }, { "created_ino": "0x10000000413", "tid": 572 }, { "created_ino": "0x10000000414", "tid": 573 }, { "created_ino": "0x10000000415", "tid": 574 }, { "created_ino": "0x10000000416", "tid": 575 }, { "created_ino": "0x10000000417", "tid": 576 }, { "created_ino": "0x10000000418", "tid": 577 }, { "created_ino": "0x10000000419", "tid": 578 }, { "created_ino": "0x1000000041a", "tid": 579 }, { "created_ino": "0x1000000041b", "tid": 580 }, { "created_ino": "0x1000000041c", "tid": 581 }, { "created_ino": "0x1000000041d", "tid": 582 }, { "created_ino": "0x1000000041e", "tid": 583 }, { "created_ino": "0x1000000041f", "tid": 584 }, { "created_ino": "0x10000000420", "tid": 585 }, { "created_ino": "0x10000000421", "tid": 586 }, { "created_ino": "0x10000000422", "tid": 587 }, { "created_ino": "0x10000000423", "tid": 588 }, { "created_ino": "0x10000000424", "tid": 589 }, { "created_ino": "0x10000000425", "tid": 590 }, { "created_ino": "0x10000000426", "tid": 591 }, { "created_ino": "0x10000000427", "tid": 592 }, { "created_ino": "0x10000000428", "tid": 593 }, { "created_ino": "0x10000000429", "tid": 594 }, { "created_ino": "0x1000000042a", "tid": 595 }, { "created_ino": "0x1000000042b", "tid": 596 }, { "created_ino": "0x1000000042c", "tid": 597 }, { "created_ino": "0x1000000042d", "tid": 598 }, { "created_ino": "0x1000000042e", "tid": 599 }, { "created_ino": "0x1000000042f", "tid": 600 }, { "created_ino": "0x10000000430", "tid": 601 }, { "created_ino": "0x10000000431", "tid": 602 }, { "created_ino": "0x10000000432", "tid": 603 }, { "created_ino": "0x10000000433", "tid": 604 }, { "created_ino": "0x10000000434", "tid": 605 }, { "created_ino": "0x10000000435", "tid": 606 }, { "created_ino": "0x10000000436", "tid": 607 }, { "created_ino": "0x10000000437", "tid": 608 }, { "created_ino": "0x10000000438", "tid": 609 }, { "created_ino": "0x10000000439", "tid": 610 }, { "created_ino": "0x1000000043a", "tid": 611 }, { "created_ino": "0x1000000043b", "tid": 612 }, { "created_ino": "0x1000000043c", "tid": 613 }, { "created_ino": "0x1000000043d", "tid": 614 }, { "created_ino": "0x1000000043e", "tid": 615 }, { "created_ino": "0x1000000043f", "tid": 616 }, { "created_ino": "0x10000000440", "tid": 617 }, { "created_ino": "0x10000000441", "tid": 618 }, { "created_ino": "0x10000000442", "tid": 619 }, { "created_ino": "0x10000000443", "tid": 620 }, { "created_ino": "0x10000000444", "tid": 621 }, { "created_ino": "0x10000000445", "tid": 622 }, { "created_ino": "0x10000000446", "tid": 623 }, { "created_ino": "0x10000000447", "tid": 624 }, { "created_ino": "0x10000000448", "tid": 625 }, { "created_ino": "0x10000000449", "tid": 626 }, { "created_ino": "0x1000000044a", "tid": 627 }, { "created_ino": "0x1000000044b", "tid": 628 }, { "created_ino": "0x1000000044c", "tid": 629 }, { "created_ino": "0x1000000044d", "tid": 630 }, { "created_ino": "0x1000000044e", "tid": 631 }, { "created_ino": "0x1000000044f", "tid": 632 }, { "created_ino": "0x10000000450", "tid": 633 }, { "created_ino": "0x10000000451", "tid": 634 }, { "created_ino": "0x10000000452", "tid": 635 }, { "created_ino": "0x10000000453", "tid": 636 }, { "created_ino": "0x10000000454", "tid": 637 }, { "created_ino": "0x10000000455", "tid": 638 }, { "created_ino": "0x10000000456", "tid": 639 }, { "created_ino": "0x10000000457", "tid": 640 }, { "created_ino": "0x10000000458", "tid": 641 }, { "created_ino": "0x10000000459", "tid": 642 }, { "created_ino": "0x1000000045a", "tid": 643 }, { "created_ino": "0x1000000045b", "tid": 644 }, { "created_ino": "0x1000000045c", "tid": 645 }, { "created_ino": "0x1000000045d", "tid": 646 }, { "created_ino": "0x1000000045e", "tid": 647 }, { "created_ino": "0x1000000045f", "tid": 648 }, { "created_ino": "0x10000000460", "tid": 649 }, { "created_ino": "0x10000000461", "tid": 650 }, { "created_ino": "0x10000000462", "tid": 651 }, { "created_ino": "0x10000000463", "tid": 652 }, { "created_ino": "0x10000000464", "tid": 653 }, { "created_ino": "0x10000000465", "tid": 654 }, { "created_ino": "0x10000000466", "tid": 655 }, { "created_ino": "0x10000000467", "tid": 656 }, { "created_ino": "0x10000000468", "tid": 657 }, { "created_ino": "0x10000000469", "tid": 658 }, { "created_ino": "0x1000000046a", "tid": 659 }, { "created_ino": "0x1000000046b", "tid": 660 }, { "created_ino": "0x1000000046c", "tid": 661 }, { "created_ino": "0x1000000046d", "tid": 662 }, { "created_ino": "0x1000000046e", "tid": 663 }, { "created_ino": "0x1000000046f", "tid": 664 }, { "created_ino": "0x10000000470", "tid": 665 }, { "created_ino": "0x10000000471", "tid": 666 }, { "created_ino": "0x10000000472", "tid": 667 }, { "created_ino": "0x10000000473", "tid": 668 }, { "created_ino": "0x10000000474", "tid": 669 }, { "created_ino": "0x10000000475", "tid": 670 }, { "created_ino": "0x10000000476", "tid": 671 }, { "created_ino": "0x10000000477", "tid": 672 }, { "created_ino": "0x10000000478", "tid": 673 }, { "created_ino": "0x10000000479", "tid": 674 }, { "created_ino": "0x1000000047a", "tid": 675 }, { "created_ino": "0x1000000047b", "tid": 676 }, { "created_ino": "0x1000000047c", "tid": 677 }, { "created_ino": "0x1000000047d", "tid": 678 }, { "created_ino": "0x1000000047e", "tid": 679 }, { "created_ino": "0x1000000047f", "tid": 680 }, { "created_ino": "0x10000000480", "tid": 681 }, { "created_ino": "0x10000000481", "tid": 682 }, { "created_ino": "0x10000000482", "tid": 683 }, { "created_ino": "0x10000000483", "tid": 684 }, { "created_ino": "0x10000000484", "tid": 685 }, { "created_ino": "0x10000000485", "tid": 686 }, { "created_ino": "0x10000000486", "tid": 687 }, { "created_ino": "0x10000000487", "tid": 688 }, { "created_ino": "0x10000000488", "tid": 689 }, { "created_ino": "0x10000000489", "tid": 690 }, { "created_ino": "0x1000000048a", "tid": 691 }, { "created_ino": "0x1000000048b", "tid": 692 }, { "created_ino": "0x1000000048c", "tid": 693 }, { "created_ino": "0x1000000048d", "tid": 694 }, { "created_ino": "0x1000000048e", "tid": 695 }, { "created_ino": "0x1000000048f", "tid": 696 }, { "created_ino": "0x10000000490", "tid": 697 }, { "created_ino": "0x10000000491", "tid": 698 }, { "created_ino": "0x10000000492", "tid": 699 }, { "created_ino": "0x10000000493", "tid": 700 }, { "created_ino": "0x10000000494", "tid": 701 }, { "created_ino": "0x10000000495", "tid": 702 }, { "created_ino": "0x10000000496", "tid": 703 }, { "created_ino": "0x10000000497", "tid": 704 }, { "created_ino": "0x10000000498", "tid": 705 }, { "created_ino": "0x10000000499", "tid": 706 }, { "created_ino": "0x1000000049a", "tid": 707 }, { "created_ino": "0x1000000049b", "tid": 708 }, { "created_ino": "0x1000000049c", "tid": 709 }, { "created_ino": "0x1000000049d", "tid": 710 }, { "created_ino": "0x1000000049e", "tid": 711 }, { "created_ino": "0x1000000049f", "tid": 712 }, { "created_ino": "0x100000004a0", "tid": 713 }, { "created_ino": "0x100000004a1", "tid": 714 }, { "created_ino": "0x100000004a2", "tid": 715 }, { "created_ino": "0x100000004a3", "tid": 716 }, { "created_ino": "0x100000004a4", "tid": 717 }, { "created_ino": "0x100000004a5", "tid": 718 }, { "created_ino": "0x100000004a6", "tid": 719 }, { "created_ino": "0x100000004a7", "tid": 720 }, { "created_ino": "0x100000004a8", "tid": 721 }, { "created_ino": "0x100000004a9", "tid": 722 }, { "created_ino": "0x100000004aa", "tid": 723 }, { "created_ino": "0x100000004ab", "tid": 724 }, { "created_ino": "0x100000004ac", "tid": 725 }, { "created_ino": "0x100000004ad", "tid": 726 }, { "created_ino": "0x100000004ae", "tid": 727 }, { "created_ino": "0x100000004af", "tid": 728 }, { "created_ino": "0x100000004b0", "tid": 729 }, { "created_ino": "0x100000004b1", "tid": 730 }, { "created_ino": "0x100000004b2", "tid": 731 }, { "created_ino": "0x100000004b3", "tid": 732 }, { "created_ino": "0x100000004b4", "tid": 733 }, { "created_ino": "0x100000004b5", "tid": 734 }, { "created_ino": "0x100000004b6", "tid": 735 }, { "created_ino": "0x100000004b7", "tid": 736 }, { "created_ino": "0x100000004b8", "tid": 737 }, { "created_ino": "0x100000004b9", "tid": 738 }, { "created_ino": "0x100000004ba", "tid": 739 }, { "created_ino": "0x100000004bb", "tid": 740 }, { "created_ino": "0x100000004bc", "tid": 741 }, { "created_ino": "0x100000004bd", "tid": 742 }, { "created_ino": "0x100000004be", "tid": 743 }, { "created_ino": "0x100000004bf", "tid": 744 }, { "created_ino": "0x100000004c0", "tid": 745 }, { "created_ino": "0x100000004c1", "tid": 746 }, { "created_ino": "0x100000004c2", "tid": 747 }, { "created_ino": "0x100000004c3", "tid": 748 }, { "created_ino": "0x100000004c4", "tid": 749 }, { "created_ino": "0x100000004c5", "tid": 750 }, { "created_ino": "0x100000004c6", "tid": 751 }, { "created_ino": "0x100000004c7", "tid": 752 }, { "created_ino": "0x100000004c8", "tid": 753 }, { "created_ino": "0x100000004c9", "tid": 754 }, { "created_ino": "0x100000004ca", "tid": 755 }, { "created_ino": "0x100000004cb", "tid": 756 }, { "created_ino": "0x100000004cc", "tid": 757 }, { "created_ino": "0x100000004cd", "tid": 758 }, { "created_ino": "0x100000004ce", "tid": 759 }, { "created_ino": "0x100000004cf", "tid": 760 }, { "created_ino": "0x100000004d0", "tid": 761 }, { "created_ino": "0x100000004d1", "tid": 762 }, { "created_ino": "0x100000004d2", "tid": 763 }, { "created_ino": "0x100000004d3", "tid": 764 }, { "created_ino": "0x100000004d4", "tid": 765 }, { "created_ino": "0x100000004d5", "tid": 766 }, { "created_ino": "0x100000004d6", "tid": 767 }, { "created_ino": "0x100000004d7", "tid": 768 }, { "created_ino": "0x100000004d8", "tid": 769 }, { "created_ino": "0x100000004d9", "tid": 770 }, { "created_ino": "0x100000004da", "tid": 771 }, { "created_ino": "0x100000004db", "tid": 772 }, { "created_ino": "0x100000004dc", "tid": 773 }, { "created_ino": "0x100000004dd", "tid": 774 }, { "created_ino": "0x100000004de", "tid": 775 }, { "created_ino": "0x100000004df", "tid": 776 }, { "created_ino": "0x100000004e0", "tid": 777 }, { "created_ino": "0x100000004e1", "tid": 778 }, { "created_ino": "0x100000004e2", "tid": 779 }, { "created_ino": "0x100000004e3", "tid": 780 }, { "created_ino": "0x100000004e4", "tid": 781 }, { "created_ino": "0x100000004e5", "tid": 782 }, { "created_ino": "0x100000004e6", "tid": 783 }, { "created_ino": "0x100000004e7", "tid": 784 }, { "created_ino": "0x100000004e8", "tid": 785 }, { "created_ino": "0x100000004e9", "tid": 786 }, { "created_ino": "0x100000004ea", "tid": 787 }, { "created_ino": "0x100000004eb", "tid": 788 }, { "created_ino": "0x100000004ec", "tid": 789 }, { "created_ino": "0x100000004ed", "tid": 790 }, { "created_ino": "0x100000004ee", "tid": 791 }, { "created_ino": "0x100000004ef", "tid": 792 }, { "created_ino": "0x100000004f0", "tid": 793 }, { "created_ino": "0x100000004f1", "tid": 794 }, { "created_ino": "0x100000004f2", "tid": 795 }, { "created_ino": "0x100000004f3", "tid": 796 }, { "created_ino": "0x100000004f4", "tid": 797 }, { "created_ino": "0x100000004f5", "tid": 798 }, { "created_ino": "0x100000004f6", "tid": 799 }, { "created_ino": "0x100000004f7", "tid": 800 }, { "created_ino": "0x100000004f8", "tid": 801 }, { "created_ino": "0x100000004f9", "tid": 802 }, { "created_ino": "0x100000004fa", "tid": 803 }, { "created_ino": "0x100000004fb", "tid": 804 }, { "created_ino": "0x100000004fc", "tid": 805 }, { "created_ino": "0x100000004fd", "tid": 806 }, { "created_ino": "0x100000004fe", "tid": 807 }, { "created_ino": "0x100000004ff", "tid": 808 }, { "created_ino": "0x10000000500", "tid": 809 }, { "created_ino": "0x10000000501", "tid": 810 }, { "created_ino": "0x10000000502", "tid": 811 }, { "created_ino": "0x10000000503", "tid": 812 }, { "created_ino": "0x10000000504", "tid": 813 }, { "created_ino": "0x10000000505", "tid": 814 }, { "created_ino": "0x10000000506", "tid": 815 }, { "created_ino": "0x10000000507", "tid": 816 }, { "created_ino": "0x10000000508", "tid": 817 }, { "created_ino": "0x10000000509", "tid": 818 }, { "created_ino": "0x1000000050a", "tid": 819 }, { "created_ino": "0x1000000050b", "tid": 820 }, { "created_ino": "0x1000000050c", "tid": 821 }, { "created_ino": "0x1000000050d", "tid": 822 }, { "created_ino": "0x1000000050e", "tid": 823 }, { "created_ino": "0x1000000050f", "tid": 824 }, { "created_ino": "0x10000000510", "tid": 825 }, { "created_ino": "0x10000000511", "tid": 826 }, { "created_ino": "0x10000000512", "tid": 827 }, { "created_ino": "0x10000000513", "tid": 828 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 1871709369, "type": "any" }, "name": { "num": 136013, "type": "client" } }, "id": 136013, "inst": "client.136013 172.21.15.16:0/1871709369", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 530, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 2327.950293177482 }, "recall_caps_throttle": { "halflife": 1.5, "value": 943.6156737958083 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 431.30174781615113 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 775, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1584.4899679492544 }, "state": "open", "uptime": 10.151929863 } ] 2024-09-10T21:17:22.909 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-09-10T21:17:22.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:17:23.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 -- 172.21.15.16:0/1365784738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50b815f450 msgr2=0x7f50b80cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:23.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 --2- 172.21.15.16:0/1365784738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50b815f450 0x7f50b80cfc80 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f50b8073820 tx=0x7f50a801f070 comp rx=0 tx=0).stop 2024-09-10T21:17:23.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 -- 172.21.15.16:0/1365784738 shutdown_connections 2024-09-10T21:17:23.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 --2- 172.21.15.16:0/1365784738 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50b80d0890 0x7f50b80d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 --2- 172.21.15.16:0/1365784738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50b815f450 0x7f50b80cfc80 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 -- 172.21.15.16:0/1365784738 >> 172.21.15.16:0/1365784738 conn(0x7f50b80d1d80 msgr2=0x7f50b80d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:23.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 -- 172.21.15.16:0/1365784738 shutdown_connections 2024-09-10T21:17:23.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 -- 172.21.15.16:0/1365784738 wait complete. 2024-09-10T21:17:23.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 Processor -- start 2024-09-10T21:17:23.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 -- start start 2024-09-10T21:17:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50b80d0890 0x7f50b814d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50b815f450 0x7f50b80b7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b80b78d0 0x7f50b80c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50b80b2e30 con 0x7f50b815f450 2024-09-10T21:17:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f50b80b2cb0 con 0x7f50b80b78d0 2024-09-10T21:17:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50bd200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f50b80b2fb0 con 0x7f50b80d0890 2024-09-10T21:17:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50b815f450 0x7f50b80b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50b815f450 0x7f50b80b7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55054/0 (socket says 172.21.15.16:55054) 2024-09-10T21:17:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50afe00640 1 -- 172.21.15.16:0/965284930 learned_addr learned my addr 172.21.15.16:0/965284930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50b6800640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b80b78d0 0x7f50b80c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50b5e00640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50b80d0890 0x7f50b814d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:23.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50afe00640 1 -- 172.21.15.16:0/965284930 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50b80d0890 msgr2=0x7f50b814d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:23.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50afe00640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50b80d0890 0x7f50b814d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50afe00640 1 -- 172.21.15.16:0/965284930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b80b78d0 msgr2=0x7f50b80c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50afe00640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b80b78d0 0x7f50b80c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50afe00640 1 -- 172.21.15.16:0/965284930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b80b6330 con 0x7f50b815f450 2024-09-10T21:17:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.053+0000 7f50afe00640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50b815f450 0x7f50b80b7390 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f50a000c490 tx=0x7f50a0001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.057+0000 7f50ae000640 1 -- 172.21.15.16:0/965284930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0017070 con 0x7f50b815f450 2024-09-10T21:17:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.057+0000 7f50ae000640 1 -- 172.21.15.16:0/965284930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50a0003600 con 0x7f50b815f450 2024-09-10T21:17:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.057+0000 7f50bd200640 1 -- 172.21.15.16:0/965284930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b80b65d0 con 0x7f50b815f450 2024-09-10T21:17:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.057+0000 7f50ae000640 1 -- 172.21.15.16:0/965284930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0003a90 con 0x7f50b815f450 2024-09-10T21:17:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.057+0000 7f50bd200640 1 -- 172.21.15.16:0/965284930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b80b6b90 con 0x7f50b815f450 2024-09-10T21:17:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.057+0000 7f50bd200640 1 -- 172.21.15.16:0/965284930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b80b1e50 con 0x7f50b815f450 2024-09-10T21:17:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.057+0000 7f50ae000640 1 -- 172.21.15.16:0/965284930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f50a0003c30 con 0x7f50b815f450 2024-09-10T21:17:23.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.057+0000 7f50ae000640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50940725e0 0x7f5094074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:23.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.061+0000 7f50ae000640 1 -- 172.21.15.16:0/965284930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f50a0092910 con 0x7f50b815f450 2024-09-10T21:17:23.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.061+0000 7f50b5e00640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50940725e0 0x7f5094074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:23.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.061+0000 7f50b5e00640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50940725e0 0x7f5094074aa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f50a80b2000 tx=0x7f50a801d910 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.061+0000 7f50ae000640 1 -- 172.21.15.16:0/965284930 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f50a005d8c0 con 0x7f50b815f450 2024-09-10T21:17:23.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50bd200640 1 -- 172.21.15.16:0/965284930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f50b80b1160 con 0x7f50b815f450 2024-09-10T21:17:23.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50ae000640 1 -- 172.21.15.16:0/965284930 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f50a00622f0 con 0x7f50b815f450 2024-09-10T21:17:23.281 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:23.281 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:17:23.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50bd200640 1 -- 172.21.15.16:0/965284930 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50940725e0 msgr2=0x7f5094074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:23.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50bd200640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50940725e0 0x7f5094074aa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f50a80b2000 tx=0x7f50a801d910 comp rx=0 tx=0).stop 2024-09-10T21:17:23.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50bd200640 1 -- 172.21.15.16:0/965284930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50b815f450 msgr2=0x7f50b80b7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:23.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50bd200640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50b815f450 0x7f50b80b7390 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f50a000c490 tx=0x7f50a0001e80 comp rx=0 tx=0).stop 2024-09-10T21:17:23.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50bd200640 1 -- 172.21.15.16:0/965284930 shutdown_connections 2024-09-10T21:17:23.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50bd200640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f50940725e0 0x7f5094074aa0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50bd200640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b80b78d0 0x7f50b80c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50bd200640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50b815f450 0x7f50b80b7390 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50bd200640 1 --2- 172.21.15.16:0/965284930 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f50b80d0890 0x7f50b814d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50bd200640 1 -- 172.21.15.16:0/965284930 >> 172.21.15.16:0/965284930 conn(0x7f50b80d1d80 msgr2=0x7f50b806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50bd200640 1 -- 172.21.15.16:0/965284930 shutdown_connections 2024-09-10T21:17:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.277+0000 7f50bd200640 1 -- 172.21.15.16:0/965284930 wait complete. 2024-09-10T21:17:23.301 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:17:23.302 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-09-10T21:17:23.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:17:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 -- 172.21.15.16:0/748105328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e8004810 msgr2=0x7f69e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 --2- 172.21.15.16:0/748105328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e8004810 0x7f69e8004c90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6a00069e00 tx=0x7f69ec01cde0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a06200640 1 -- 172.21.15.16:0/748105328 reap_dead start 2024-09-10T21:17:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 -- 172.21.15.16:0/748105328 shutdown_connections 2024-09-10T21:17:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 -- 172.21.15.16:0/748105328 >> 172.21.15.16:0/748105328 conn(0x7f6a0009b130 msgr2=0x7f6a000d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 -- 172.21.15.16:0/748105328 shutdown_connections 2024-09-10T21:17:23.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 -- 172.21.15.16:0/748105328 wait complete. 2024-09-10T21:17:23.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 Processor -- start 2024-09-10T21:17:23.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 -- start start 2024-09-10T21:17:23.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e8004810 0x7f6a0006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:23.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0006f700 0x7f6a00159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:23.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a00159f60 0x7f6a001fb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:23.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a00167de0 con 0x7f69e8004810 2024-09-10T21:17:23.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a00167f60 con 0x7f6a0006f700 2024-09-10T21:17:23.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a08200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6a001680e0 con 0x7f6a00159f60 2024-09-10T21:17:23.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a05800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e8004810 0x7f6a0006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:23.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a05800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e8004810 0x7f6a0006f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55060/0 (socket says 172.21.15.16:55060) 2024-09-10T21:17:23.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a05800640 1 -- 172.21.15.16:0/1810774952 learned_addr learned my addr 172.21.15.16:0/1810774952 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a04e00640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0006f700 0x7f6a00159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a06200640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a00159f60 0x7f6a001fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a05800640 1 -- 172.21.15.16:0/1810774952 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a00159f60 msgr2=0x7f6a001fb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a05800640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a00159f60 0x7f6a001fb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a05800640 1 -- 172.21.15.16:0/1810774952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0006f700 msgr2=0x7f6a00159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a05800640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0006f700 0x7f6a00159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a05800640 1 -- 172.21.15.16:0/1810774952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0006df40 con 0x7f69e8004810 2024-09-10T21:17:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f6a05800640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e8004810 0x7f6a0006f1c0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f69f4010a70 tx=0x7f69f4010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:23.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.445+0000 7f69fd600640 1 -- 172.21.15.16:0/1810774952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f400ee00 con 0x7f69e8004810 2024-09-10T21:17:23.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.449+0000 7f69fd600640 1 -- 172.21.15.16:0/1810774952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69f4005030 con 0x7f69e8004810 2024-09-10T21:17:23.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.449+0000 7f69fd600640 1 -- 172.21.15.16:0/1810774952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4005460 con 0x7f69e8004810 2024-09-10T21:17:23.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.449+0000 7f6a08200640 1 -- 172.21.15.16:0/1810774952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0006e220 con 0x7f69e8004810 2024-09-10T21:17:23.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.449+0000 7f6a08200640 1 -- 172.21.15.16:0/1810774952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0006e7e0 con 0x7f69e8004810 2024-09-10T21:17:23.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.449+0000 7f6a08200640 1 -- 172.21.15.16:0/1810774952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0006fae0 con 0x7f69e8004810 2024-09-10T21:17:23.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.449+0000 7f69fd600640 1 -- 172.21.15.16:0/1810774952 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f69f4023030 con 0x7f69e8004810 2024-09-10T21:17:23.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.449+0000 7f69fd600640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69e0072610 0x7f69e0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:23.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.449+0000 7f69fd600640 1 -- 172.21.15.16:0/1810774952 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f69f4093d70 con 0x7f69e8004810 2024-09-10T21:17:23.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.449+0000 7f6a04e00640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69e0072610 0x7f69e0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:23.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.453+0000 7f6a04e00640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69e0072610 0x7f69e0074ad0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0ab000 tx=0x7f69ec019c20 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.453+0000 7f69fd600640 1 -- 172.21.15.16:0/1810774952 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f69f405edb0 con 0x7f69e8004810 2024-09-10T21:17:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f6a08200640 1 -- 172.21.15.16:0/1810774952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6a0015a340 con 0x7f69e8004810 2024-09-10T21:17:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f69fd600640 1 -- 172.21.15.16:0/1810774952 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f69f40637e0 con 0x7f69e8004810 2024-09-10T21:17:23.669 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:23.669 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:17:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f6a08200640 1 -- 172.21.15.16:0/1810774952 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69e0072610 msgr2=0x7f69e0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f6a08200640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69e0072610 0x7f69e0074ad0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0ab000 tx=0x7f69ec019c20 comp rx=0 tx=0).stop 2024-09-10T21:17:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f6a08200640 1 -- 172.21.15.16:0/1810774952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e8004810 msgr2=0x7f6a0006f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f6a08200640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e8004810 0x7f6a0006f1c0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f69f4010a70 tx=0x7f69f4010f30 comp rx=0 tx=0).stop 2024-09-10T21:17:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f6a08200640 1 -- 172.21.15.16:0/1810774952 shutdown_connections 2024-09-10T21:17:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f6a08200640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69e0072610 0x7f69e0074ad0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f6a08200640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a00159f60 0x7f6a001fb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f6a08200640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0006f700 0x7f6a00159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f6a08200640 1 --2- 172.21.15.16:0/1810774952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e8004810 0x7f6a0006f1c0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f6a08200640 1 -- 172.21.15.16:0/1810774952 >> 172.21.15.16:0/1810774952 conn(0x7f6a0009b130 msgr2=0x7f6a000aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f6a08200640 1 -- 172.21.15.16:0/1810774952 shutdown_connections 2024-09-10T21:17:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23.665+0000 7f6a08200640 1 -- 172.21.15.16:0/1810774952 wait complete. 2024-09-10T21:17:23.690 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:17:23.691 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-09-10T21:17:23.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-10T21:17:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:17:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:stdin closed, goodbye! 2024-09-10T21:17:23.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:17:23.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:24.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:17:24.058 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:17:24.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.137+0000 7f69d8400640 1 -- 172.21.15.16:0/3042184110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b8004d50 msgr2=0x7f69b80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:24.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.137+0000 7f69d8400640 1 --2- 172.21.15.16:0/3042184110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b8004d50 0x7f69b80051d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f69d00c6ba0 tx=0x7f69c801f9e0 comp rx=0 tx=0).stop 2024-09-10T21:17:24.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.137+0000 7f69d6400640 1 -- 172.21.15.16:0/3042184110 reap_dead start 2024-09-10T21:17:24.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.137+0000 7f69d8400640 1 -- 172.21.15.16:0/3042184110 shutdown_connections 2024-09-10T21:17:24.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.137+0000 7f69d8400640 1 -- 172.21.15.16:0/3042184110 >> 172.21.15.16:0/3042184110 conn(0x7f69d00cfb20 msgr2=0x7f69d00cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:24.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.137+0000 7f69d8400640 1 -- 172.21.15.16:0/3042184110 shutdown_connections 2024-09-10T21:17:24.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.137+0000 7f69d8400640 1 -- 172.21.15.16:0/3042184110 wait complete. 2024-09-10T21:17:24.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.137+0000 7f69d8400640 1 Processor -- start 2024-09-10T21:17:24.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.137+0000 7f69d8400640 1 -- start start 2024-09-10T21:17:24.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d8400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b8004d50 0x7f69d00ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:24.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d8400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f69d01f4710 0x7f69d0073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:24.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d8400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d00c9560 0x7f69d01580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:24.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d8400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0070430 con 0x7f69b8004d50 2024-09-10T21:17:24.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d8400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f69d00702b0 con 0x7f69d00c9560 2024-09-10T21:17:24.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d8400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f69d006e640 con 0x7f69d01f4710 2024-09-10T21:17:24.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b8004d50 0x7f69d00ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:24.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b8004d50 0x7f69d00ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55080/0 (socket says 172.21.15.16:55080) 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d5a00640 1 -- 172.21.15.16:0/2843103320 learned_addr learned my addr 172.21.15.16:0/2843103320 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d5000640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f69d01f4710 0x7f69d0073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d6400640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d00c9560 0x7f69d01580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d5a00640 1 -- 172.21.15.16:0/2843103320 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f69d01f4710 msgr2=0x7f69d0073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d5a00640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f69d01f4710 0x7f69d0073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d5a00640 1 -- 172.21.15.16:0/2843103320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d00c9560 msgr2=0x7f69d01580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d5a00640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d00c9560 0x7f69d01580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d5a00640 1 -- 172.21.15.16:0/2843103320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d0072500 con 0x7f69b8004d50 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d6400640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d00c9560 0x7f69d01580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d5a00640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b8004d50 0x7f69d00ad850 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f69c4003f20 tx=0x7f69c4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69cd600640 1 -- 172.21.15.16:0/2843103320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4011640 con 0x7f69b8004d50 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69cd600640 1 -- 172.21.15.16:0/2843103320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69c4016020 con 0x7f69b8004d50 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69cd600640 1 -- 172.21.15.16:0/2843103320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4011a40 con 0x7f69b8004d50 2024-09-10T21:17:24.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d8400640 1 -- 172.21.15.16:0/2843103320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d01f4c80 con 0x7f69b8004d50 2024-09-10T21:17:24.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d8400640 1 -- 172.21.15.16:0/2843103320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d01f5240 con 0x7f69b8004d50 2024-09-10T21:17:24.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69d8400640 1 -- 172.21.15.16:0/2843103320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69d006eba0 con 0x7f69b8004d50 2024-09-10T21:17:24.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.141+0000 7f69cd600640 1 -- 172.21.15.16:0/2843103320 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f69c4011be0 con 0x7f69b8004d50 2024-09-10T21:17:24.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.145+0000 7f69cd600640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69b00725e0 0x7f69b0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:24.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.145+0000 7f69cd600640 1 -- 172.21.15.16:0/2843103320 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f69c4093e90 con 0x7f69b8004d50 2024-09-10T21:17:24.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.145+0000 7f69d5000640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69b00725e0 0x7f69b0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:24.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.145+0000 7f69d5000640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69b00725e0 0x7f69b0074aa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f69d0073b30 tx=0x7f69bc00a2f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.149+0000 7f69cd600640 1 -- 172.21.15.16:0/2843103320 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f69c4020020 con 0x7f69b8004d50 2024-09-10T21:17:24.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.357+0000 7f69d8400640 1 -- 172.21.15.16:0/2843103320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69d01f55e0 con 0x7f69b8004d50 2024-09-10T21:17:24.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.357+0000 7f69cd600640 1 -- 172.21.15.16:0/2843103320 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5235 (secure 0 0 0) 0x7f69c4095020 con 0x7f69b8004d50 2024-09-10T21:17:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:24.362 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"btime":"2024-09-10T21:17:21:843659+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3765085271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765085271},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765085271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":7104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4198210360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":4198210360},{"type":"v1","addr":"172.21.15.16:6837","nonce":4198210360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2954517211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2954517211},{"type":"v1","addr":"172.21.15.52:6835","nonce":2954517211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:05.094283+0000","modified":"2024-09-10T21:17:21.746067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135926":{"gid":135926,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":7,"addr":"172.21.15.52:6834/1108687782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108687782},{"type":"v1","addr":"172.21.15.52:6834","nonce":1108687782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135926,"qdb_cluster":[135926]},"id":8}]} 2024-09-10T21:17:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 73 2024-09-10T21:17:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.361+0000 7f69d8400640 1 -- 172.21.15.16:0/2843103320 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69b00725e0 msgr2=0x7f69b0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.361+0000 7f69d8400640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69b00725e0 0x7f69b0074aa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f69d0073b30 tx=0x7f69bc00a2f0 comp rx=0 tx=0).stop 2024-09-10T21:17:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.361+0000 7f69d8400640 1 -- 172.21.15.16:0/2843103320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b8004d50 msgr2=0x7f69d00ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.361+0000 7f69d8400640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b8004d50 0x7f69d00ad850 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f69c4003f20 tx=0x7f69c4001640 comp rx=0 tx=0).stop 2024-09-10T21:17:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.361+0000 7f69d8400640 1 -- 172.21.15.16:0/2843103320 shutdown_connections 2024-09-10T21:17:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.361+0000 7f69d8400640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69b00725e0 0x7f69b0074aa0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.361+0000 7f69d8400640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69d00c9560 0x7f69d01580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.361+0000 7f69d8400640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f69d01f4710 0x7f69d0073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.361+0000 7f69d8400640 1 --2- 172.21.15.16:0/2843103320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b8004d50 0x7f69d00ad850 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.361+0000 7f69d8400640 1 -- 172.21.15.16:0/2843103320 >> 172.21.15.16:0/2843103320 conn(0x7f69d00cfb20 msgr2=0x7f69d00cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.361+0000 7f69d8400640 1 -- 172.21.15.16:0/2843103320 shutdown_connections 2024-09-10T21:17:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.361+0000 7f69d8400640 1 -- 172.21.15.16:0/2843103320 wait complete. 2024-09-10T21:17:24.386 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-10T21:17:24.563 INFO:teuthology.orchestra.run.smithi052.stdout:[{"id":7245,"entity":{"name":{"type":"client","num":7245},"addr":{"type":"any","addr":"172.21.15.52:0","nonce":2089774086}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":10.647926332999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9402590043764429,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7245 172.21.15.52:0/2089774086","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi052","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":136013,"entity":{"name":{"type":"client","num":136013},"addr":{"type":"any","addr":"172.21.15.16:0","nonce":1871709369}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":760,"uptime":11.807918354,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":2680.1218450301794,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":714.35922530100027,"halflife":1.5},"recall_caps_throttle2o":{"value":173.92520850204335,"halflife":0.5},"session_cache_liveness":{"value":1578.4390614487188,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":530,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.136013 172.21.15.16:0/1871709369","completed_requests":[{"tid":530,"created_ino":"0x100000003e9"},{"tid":531,"created_ino":"0x100000003ea"},{"tid":532,"created_ino":"0x100000003eb"},{"tid":533,"created_ino":"0x100000003ec"},{"tid":534,"created_ino":"0x100000003ed"},{"tid":535,"created_ino":"0x100000003ee"},{"tid":536,"created_ino":"0x100000003ef"},{"tid":537,"created_ino":"0x100000003f0"},{"tid":538,"created_ino":"0x100000003f1"},{"tid":539,"created_ino":"0x100000003f2"},{"tid":540,"created_ino":"0x100000003f3"},{"tid":541,"created_ino":"0x100000003f4"},{"tid":542,"created_ino":"0x100000003f5"},{"tid":543,"created_ino":"0x100000003f6"},{"tid":544,"created_ino":"0x100000003f7"},{"tid":545,"created_ino":"0x100000003f8"},{"tid":546,"created_ino":"0x100000003f9"},{"tid":547,"created_ino":"0x100000003fa"},{"tid":548,"created_ino":"0x100000003fb"},{"tid":549,"created_ino":"0x100000003fc"},{"tid":550,"created_ino":"0x100000003fd"},{"tid":551,"created_ino":"0x100000003fe"},{"tid":552,"created_ino":"0x100000003ff"},{"tid":553,"created_ino":"0x10000000400"},{"tid":554,"created_ino":"0x10000000401"},{"tid":555,"created_ino":"0x10000000402"},{"tid":556,"created_ino":"0x10000000403"},{"tid":557,"created_ino":"0x10000000404"},{"tid":558,"created_ino":"0x10000000405"},{"tid":559,"created_ino":"0x10000000406"},{"tid":560,"created_ino":"0x10000000407"},{"tid":561,"created_ino":"0x10000000408"},{"tid":562,"created_ino":"0x10000000409"},{"tid":563,"created_ino":"0x1000000040a"},{"tid":564,"created_ino":"0x1000000040b"},{"tid":565,"created_ino":"0x1000000040c"},{"tid":566,"created_ino":"0x1000000040d"},{"tid":567,"created_ino":"0x1000000040e"},{"tid":568,"created_ino":"0x1000000040f"},{"tid":569,"created_ino":"0x10000000410"},{"tid":570,"created_ino":"0x10000000411"},{"tid":571,"created_ino":"0x10000000412"},{"tid":572,"created_ino":"0x10000000413"},{"tid":573,"created_ino":"0x10000000414"},{"tid":574,"created_ino":"0x10000000415"},{"tid":575,"created_ino":"0x10000000416"},{"tid":576,"created_ino":"0x10000000417"},{"tid":577,"created_ino":"0x10000000418"},{"tid":578,"created_ino":"0x10000000419"},{"tid":579,"created_ino":"0x1000000041a"},{"tid":580,"created_ino":"0x1000000041b"},{"tid":581,"created_ino":"0x1000000041c"},{"tid":582,"created_ino":"0x1000000041d"},{"tid":583,"created_ino":"0x1000000041e"},{"tid":584,"created_ino":"0x1000000041f"},{"tid":585,"created_ino":"0x10000000420"},{"tid":586,"created_ino":"0x10000000421"},{"tid":587,"created_ino":"0x10000000422"},{"tid":588,"created_ino":"0x10000000423"},{"tid":589,"created_ino":"0x10000000424"},{"tid":590,"created_ino":"0x10000000425"},{"tid":591,"created_ino":"0x10000000426"},{"tid":592,"created_ino":"0x10000000427"},{"tid":593,"created_ino":"0x10000000428"},{"tid":594,"created_ino":"0x10000000429"},{"tid":595,"created_ino":"0x1000000042a"},{"tid":596,"created_ino":"0x1000000042b"},{"tid":597,"created_ino":"0x1000000042c"},{"tid":598,"created_ino":"0x1000000042d"},{"tid":599,"created_ino":"0x1000000042e"},{"tid":600,"created_ino":"0x1000000042f"},{"tid":601,"created_ino":"0x10000000430"},{"tid":602,"created_ino":"0x10000000431"},{"tid":603,"created_ino":"0x10000000432"},{"tid":604,"created_ino":"0x10000000433"},{"tid":605,"created_ino":"0x10000000434"},{"tid":606,"created_ino":"0x10000000435"},{"tid":607,"created_ino":"0x10000000436"},{"tid":608,"created_ino":"0x10000000437"},{"tid":609,"created_ino":"0x10000000438"},{"tid":610,"created_ino":"0x10000000439"},{"tid":611,"created_ino":"0x1000000043a"},{"tid":612,"created_ino":"0x1000000043b"},{"tid":613,"created_ino":"0x1000000043c"},{"tid":614,"created_ino":"0x1000000043d"},{"tid":615,"created_ino":"0x1000000043e"},{"tid":616,"created_ino":"0x1000000043f"},{"tid":617,"created_ino":"0x10000000440"},{"tid":618,"created_ino":"0x10000000441"},{"tid":619,"created_ino":"0x10000000442"},{"tid":620,"created_ino":"0x10000000443"},{"tid":621,"created_ino":"0x10000000444"},{"tid":622,"created_ino":"0x10000000445"},{"tid":623,"created_ino":"0x10000000446"},{"tid":624,"created_ino":"0x10000000447"},{"tid":625,"created_ino":"0x10000000448"},{"tid":626,"created_ino":"0x10000000449"},{"tid":627,"created_ino":"0x1000000044a"},{"tid":628,"created_ino":"0x1000000044b"},{"tid":629,"created_ino":"0x1000000044c"},{"tid":630,"created_ino":"0x1000000044d"},{"tid":631,"created_ino":"0x1000000044e"},{"tid":632,"created_ino":"0x1000000044f"},{"tid":633,"created_ino":"0x10000000450"},{"tid":634,"created_ino":"0x10000000451"},{"tid":635,"created_ino":"0x10000000452"},{"tid":636,"created_ino":"0x10000000453"},{"tid":637,"created_ino":"0x10000000454"},{"tid":638,"created_ino":"0x10000000455"},{"tid":639,"created_ino":"0x10000000456"},{"tid":640,"created_ino":"0x10000000457"},{"tid":641,"created_ino":"0x10000000458"},{"tid":642,"created_ino":"0x10000000459"},{"tid":643,"created_ino":"0x1000000045a"},{"tid":644,"created_ino":"0x1000000045b"},{"tid":645,"created_ino":"0x1000000045c"},{"tid":646,"created_ino":"0x1000000045d"},{"tid":647,"created_ino":"0x1000000045e"},{"tid":648,"created_ino":"0x1000000045f"},{"tid":649,"created_ino":"0x10000000460"},{"tid":650,"created_ino":"0x10000000461"},{"tid":651,"created_ino":"0x10000000462"},{"tid":652,"created_ino":"0x10000000463"},{"tid":653,"created_ino":"0x10000000464"},{"tid":654,"created_ino":"0x10000000465"},{"tid":655,"created_ino":"0x10000000466"},{"tid":656,"created_ino":"0x10000000467"},{"tid":657,"created_ino":"0x10000000468"},{"tid":658,"created_ino":"0x10000000469"},{"tid":659,"created_ino":"0x1000000046a"},{"tid":660,"created_ino":"0x1000000046b"},{"tid":661,"created_ino":"0x1000000046c"},{"tid":662,"created_ino":"0x1000000046d"},{"tid":663,"created_ino":"0x1000000046e"},{"tid":664,"created_ino":"0x1000000046f"},{"tid":665,"created_ino":"0x10000000470"},{"tid":666,"created_ino":"0x10000000471"},{"tid":667,"created_ino":"0x10000000472"},{"tid":668,"created_ino":"0x10000000473"},{"tid":669,"created_ino":"0x10000000474"},{"tid":670,"created_ino":"0x10000000475"},{"tid":671,"created_ino":"0x10000000476"},{"tid":672,"created_ino":"0x10000000477"},{"tid":673,"created_ino":"0x10000000478"},{"tid":674,"created_ino":"0x10000000479"},{"tid":675,"created_ino":"0x1000000047a"},{"tid":676,"created_ino":"0x1000000047b"},{"tid":677,"created_ino":"0x1000000047c"},{"tid":678,"created_ino":"0x1000000047d"},{"tid":679,"created_ino":"0x1000000047e"},{"tid":680,"created_ino":"0x1000000047f"},{"tid":681,"created_ino":"0x10000000480"},{"tid":682,"created_ino":"0x10000000481"},{"tid":683,"created_ino":"0x10000000482"},{"tid":684,"created_ino":"0x10000000483"},{"tid":685,"created_ino":"0x10000000484"},{"tid":686,"created_ino":"0x10000000485"},{"tid":687,"created_ino":"0x10000000486"},{"tid":688,"created_ino":"0x10000000487"},{"tid":689,"created_ino":"0x10000000488"},{"tid":690,"created_ino":"0x10000000489"},{"tid":691,"created_ino":"0x1000000048a"},{"tid":692,"created_ino":"0x1000000048b"},{"tid":693,"created_ino":"0x1000000048c"},{"tid":694,"created_ino":"0x1000000048d"},{"tid":695,"created_ino":"0x1000000048e"},{"tid":696,"created_ino":"0x1000000048f"},{"tid":697,"created_ino":"0x10000000490"},{"tid":698,"created_ino":"0x10000000491"},{"tid":699,"created_ino":"0x10000000492"},{"tid":700,"created_ino":"0x10000000493"},{"tid":701,"created_ino":"0x10000000494"},{"tid":702,"created_ino":"0x10000000495"},{"tid":703,"created_ino":"0x10000000496"},{"tid":704,"created_ino":"0x10000000497"},{"tid":705,"created_ino":"0x10000000498"},{"tid":706,"created_ino":"0x10000000499"},{"tid":707,"created_ino":"0x1000000049a"},{"tid":708,"created_ino":"0x1000000049b"},{"tid":709,"created_ino":"0x1000000049c"},{"tid":710,"created_ino":"0x1000000049d"},{"tid":711,"created_ino":"0x1000000049e"},{"tid":712,"created_ino":"0x1000000049f"},{"tid":713,"created_ino":"0x100000004a0"},{"tid":714,"created_ino":"0x100000004a1"},{"tid":715,"created_ino":"0x100000004a2"},{"tid":716,"created_ino":"0x100000004a3"},{"tid":717,"created_ino":"0x100000004a4"},{"tid":718,"created_ino":"0x100000004a5"},{"tid":719,"created_ino":"0x100000004a6"},{"tid":720,"created_ino":"0x100000004a7"},{"tid":721,"created_ino":"0x100000004a8"},{"tid":722,"created_ino":"0x100000004a9"},{"tid":723,"created_ino":"0x100000004aa"},{"tid":724,"created_ino":"0x100000004ab"},{"tid":725,"created_ino":"0x100000004ac"},{"tid":726,"created_ino":"0x100000004ad"},{"tid":727,"created_ino":"0x100000004ae"},{"tid":728,"created_ino":"0x100000004af"},{"tid":729,"created_ino":"0x100000004b0"},{"tid":730,"created_ino":"0x100000004b1"},{"tid":731,"created_ino":"0x100000004b2"},{"tid":732,"created_ino":"0x100000004b3"},{"tid":733,"created_ino":"0x100000004b4"},{"tid":734,"created_ino":"0x100000004b5"},{"tid":735,"created_ino":"0x100000004b6"},{"tid":736,"created_ino":"0x100000004b7"},{"tid":737,"created_ino":"0x100000004b8"},{"tid":738,"created_ino":"0x100000004b9"},{"tid":739,"created_ino":"0x100000004ba"},{"tid":740,"created_ino":"0x100000004bb"},{"tid":741,"created_ino":"0x100000004bc"},{"tid":742,"created_ino":"0x100000004bd"},{"tid":743,"created_ino":"0x100000004be"},{"tid":744,"created_ino":"0x100000004bf"},{"tid":745,"created_ino":"0x100000004c0"},{"tid":746,"created_ino":"0x100000004c1"},{"tid":747,"created_ino":"0x100000004c2"},{"tid":748,"created_ino":"0x100000004c3"},{"tid":749,"created_ino":"0x100000004c4"},{"tid":750,"created_ino":"0x100000004c5"},{"tid":751,"created_ino":"0x100000004c6"},{"tid":752,"created_ino":"0x100000004c7"},{"tid":753,"created_ino":"0x100000004c8"},{"tid":754,"created_ino":"0x100000004c9"},{"tid":755,"created_ino":"0x100000004ca"},{"tid":756,"created_ino":"0x100000004cb"},{"tid":757,"created_ino":"0x100000004cc"},{"tid":758,"created_ino":"0x100000004cd"},{"tid":759,"created_ino":"0x100000004ce"},{"tid":760,"created_ino":"0x100000004cf"},{"tid":761,"created_ino":"0x100000004d0"},{"tid":762,"created_ino":"0x100000004d1"},{"tid":763,"created_ino":"0x100000004d2"},{"tid":764,"created_ino":"0x100000004d3"},{"tid":765,"created_ino":"0x100000004d4"},{"tid":766,"created_ino":"0x100000004d5"},{"tid":767,"created_ino":"0x100000004d6"},{"tid":768,"created_ino":"0x100000004d7"},{"tid":769,"created_ino":"0x100000004d8"},{"tid":770,"created_ino":"0x100000004d9"},{"tid":771,"created_ino":"0x100000004da"},{"tid":772,"created_ino":"0x100000004db"},{"tid":773,"created_ino":"0x100000004dc"},{"tid":774,"created_ino":"0x100000004dd"},{"tid":775,"created_ino":"0x100000004de"},{"tid":776,"created_ino":"0x100000004df"},{"tid":777,"created_ino":"0x100000004e0"},{"tid":778,"created_ino":"0x100000004e1"},{"tid":779,"created_ino":"0x100000004e2"},{"tid":780,"created_ino":"0x100000004e3"},{"tid":781,"created_ino":"0x100000004e4"},{"tid":782,"created_ino":"0x100000004e5"},{"tid":783,"created_ino":"0x100000004e6"},{"tid":784,"created_ino":"0x100000004e7"},{"tid":785,"created_ino":"0x100000004e8"},{"tid":786,"created_ino":"0x100000004e9"},{"tid":787,"created_ino":"0x100000004ea"},{"tid":788,"created_ino":"0x100000004eb"},{"tid":789,"created_ino":"0x100000004ec"},{"tid":790,"created_ino":"0x100000004ed"},{"tid":791,"created_ino":"0x100000004ee"},{"tid":792,"created_ino":"0x100000004ef"},{"tid":793,"created_ino":"0x100000004f0"},{"tid":794,"created_ino":"0x100000004f1"},{"tid":795,"created_ino":"0x100000004f2"},{"tid":796,"created_ino":"0x100000004f3"},{"tid":797,"created_ino":"0x100000004f4"},{"tid":798,"created_ino":"0x100000004f5"},{"tid":799,"created_ino":"0x100000004f6"},{"tid":800,"created_ino":"0x100000004f7"},{"tid":801,"created_ino":"0x100000004f8"},{"tid":802,"created_ino":"0x100000004f9"},{"tid":803,"created_ino":"0x100000004fa"},{"tid":804,"created_ino":"0x100000004fb"},{"tid":805,"created_ino":"0x100000004fc"},{"tid":806,"created_ino":"0x100000004fd"},{"tid":807,"created_ino":"0x100000004fe"},{"tid":808,"created_ino":"0x100000004ff"},{"tid":809,"created_ino":"0x10000000500"},{"tid":810,"created_ino":"0x10000000501"},{"tid":811,"created_ino":"0x10000000502"},{"tid":812,"created_ino":"0x10000000503"},{"tid":813,"created_ino":"0x10000000504"},{"tid":814,"created_ino":"0x10000000505"},{"tid":815,"created_ino":"0x10000000506"},{"tid":816,"created_ino":"0x10000000507"},{"tid":817,"created_ino":"0x10000000508"},{"tid":818,"created_ino":"0x10000000509"},{"tid":819,"created_ino":"0x1000000050a"},{"tid":820,"created_ino":"0x1000000050b"},{"tid":821,"created_ino":"0x1000000050c"},{"tid":822,"created_ino":"0x1000000050d"},{"tid":823,"created_ino":"0x1000000050e"},{"tid":824,"created_ino":"0x1000000050f"},{"tid":825,"created_ino":"0x10000000510"},{"tid":826,"created_ino":"0x10000000511"},{"tid":827,"created_ino":"0x10000000512"},{"tid":828,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-10T21:17:24.566 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi052", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.52:0", "nonce": 2089774086, "type": "any" }, "name": { "num": 7245, "type": "client" } }, "id": 7245, "inst": "client.7245 172.21.15.52:0/2089774086", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.940259004376443 }, "state": "open", "uptime": 10.647926333 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 530 }, { "created_ino": "0x100000003ea", "tid": 531 }, { "created_ino": "0x100000003eb", "tid": 532 }, { "created_ino": "0x100000003ec", "tid": 533 }, { "created_ino": "0x100000003ed", "tid": 534 }, { "created_ino": "0x100000003ee", "tid": 535 }, { "created_ino": "0x100000003ef", "tid": 536 }, { "created_ino": "0x100000003f0", "tid": 537 }, { "created_ino": "0x100000003f1", "tid": 538 }, { "created_ino": "0x100000003f2", "tid": 539 }, { "created_ino": "0x100000003f3", "tid": 540 }, { "created_ino": "0x100000003f4", "tid": 541 }, { "created_ino": "0x100000003f5", "tid": 542 }, { "created_ino": "0x100000003f6", "tid": 543 }, { "created_ino": "0x100000003f7", "tid": 544 }, { "created_ino": "0x100000003f8", "tid": 545 }, { "created_ino": "0x100000003f9", "tid": 546 }, { "created_ino": "0x100000003fa", "tid": 547 }, { "created_ino": "0x100000003fb", "tid": 548 }, { "created_ino": "0x100000003fc", "tid": 549 }, { "created_ino": "0x100000003fd", "tid": 550 }, { "created_ino": "0x100000003fe", "tid": 551 }, { "created_ino": "0x100000003ff", "tid": 552 }, { "created_ino": "0x10000000400", "tid": 553 }, { "created_ino": "0x10000000401", "tid": 554 }, { "created_ino": "0x10000000402", "tid": 555 }, { "created_ino": "0x10000000403", "tid": 556 }, { "created_ino": "0x10000000404", "tid": 557 }, { "created_ino": "0x10000000405", "tid": 558 }, { "created_ino": "0x10000000406", "tid": 559 }, { "created_ino": "0x10000000407", "tid": 560 }, { "created_ino": "0x10000000408", "tid": 561 }, { "created_ino": "0x10000000409", "tid": 562 }, { "created_ino": "0x1000000040a", "tid": 563 }, { "created_ino": "0x1000000040b", "tid": 564 }, { "created_ino": "0x1000000040c", "tid": 565 }, { "created_ino": "0x1000000040d", "tid": 566 }, { "created_ino": "0x1000000040e", "tid": 567 }, { "created_ino": "0x1000000040f", "tid": 568 }, { "created_ino": "0x10000000410", "tid": 569 }, { "created_ino": "0x10000000411", "tid": 570 }, { "created_ino": "0x10000000412", "tid": 571 }, { "created_ino": "0x10000000413", "tid": 572 }, { "created_ino": "0x10000000414", "tid": 573 }, { "created_ino": "0x10000000415", "tid": 574 }, { "created_ino": "0x10000000416", "tid": 575 }, { "created_ino": "0x10000000417", "tid": 576 }, { "created_ino": "0x10000000418", "tid": 577 }, { "created_ino": "0x10000000419", "tid": 578 }, { "created_ino": "0x1000000041a", "tid": 579 }, { "created_ino": "0x1000000041b", "tid": 580 }, { "created_ino": "0x1000000041c", "tid": 581 }, { "created_ino": "0x1000000041d", "tid": 582 }, { "created_ino": "0x1000000041e", "tid": 583 }, { "created_ino": "0x1000000041f", "tid": 584 }, { "created_ino": "0x10000000420", "tid": 585 }, { "created_ino": "0x10000000421", "tid": 586 }, { "created_ino": "0x10000000422", "tid": 587 }, { "created_ino": "0x10000000423", "tid": 588 }, { "created_ino": "0x10000000424", "tid": 589 }, { "created_ino": "0x10000000425", "tid": 590 }, { "created_ino": "0x10000000426", "tid": 591 }, { "created_ino": "0x10000000427", "tid": 592 }, { "created_ino": "0x10000000428", "tid": 593 }, { "created_ino": "0x10000000429", "tid": 594 }, { "created_ino": "0x1000000042a", "tid": 595 }, { "created_ino": "0x1000000042b", "tid": 596 }, { "created_ino": "0x1000000042c", "tid": 597 }, { "created_ino": "0x1000000042d", "tid": 598 }, { "created_ino": "0x1000000042e", "tid": 599 }, { "created_ino": "0x1000000042f", "tid": 600 }, { "created_ino": "0x10000000430", "tid": 601 }, { "created_ino": "0x10000000431", "tid": 602 }, { "created_ino": "0x10000000432", "tid": 603 }, { "created_ino": "0x10000000433", "tid": 604 }, { "created_ino": "0x10000000434", "tid": 605 }, { "created_ino": "0x10000000435", "tid": 606 }, { "created_ino": "0x10000000436", "tid": 607 }, { "created_ino": "0x10000000437", "tid": 608 }, { "created_ino": "0x10000000438", "tid": 609 }, { "created_ino": "0x10000000439", "tid": 610 }, { "created_ino": "0x1000000043a", "tid": 611 }, { "created_ino": "0x1000000043b", "tid": 612 }, { "created_ino": "0x1000000043c", "tid": 613 }, { "created_ino": "0x1000000043d", "tid": 614 }, { "created_ino": "0x1000000043e", "tid": 615 }, { "created_ino": "0x1000000043f", "tid": 616 }, { "created_ino": "0x10000000440", "tid": 617 }, { "created_ino": "0x10000000441", "tid": 618 }, { "created_ino": "0x10000000442", "tid": 619 }, { "created_ino": "0x10000000443", "tid": 620 }, { "created_ino": "0x10000000444", "tid": 621 }, { "created_ino": "0x10000000445", "tid": 622 }, { "created_ino": "0x10000000446", "tid": 623 }, { "created_ino": "0x10000000447", "tid": 624 }, { "created_ino": "0x10000000448", "tid": 625 }, { "created_ino": "0x10000000449", "tid": 626 }, { "created_ino": "0x1000000044a", "tid": 627 }, { "created_ino": "0x1000000044b", "tid": 628 }, { "created_ino": "0x1000000044c", "tid": 629 }, { "created_ino": "0x1000000044d", "tid": 630 }, { "created_ino": "0x1000000044e", "tid": 631 }, { "created_ino": "0x1000000044f", "tid": 632 }, { "created_ino": "0x10000000450", "tid": 633 }, { "created_ino": "0x10000000451", "tid": 634 }, { "created_ino": "0x10000000452", "tid": 635 }, { "created_ino": "0x10000000453", "tid": 636 }, { "created_ino": "0x10000000454", "tid": 637 }, { "created_ino": "0x10000000455", "tid": 638 }, { "created_ino": "0x10000000456", "tid": 639 }, { "created_ino": "0x10000000457", "tid": 640 }, { "created_ino": "0x10000000458", "tid": 641 }, { "created_ino": "0x10000000459", "tid": 642 }, { "created_ino": "0x1000000045a", "tid": 643 }, { "created_ino": "0x1000000045b", "tid": 644 }, { "created_ino": "0x1000000045c", "tid": 645 }, { "created_ino": "0x1000000045d", "tid": 646 }, { "created_ino": "0x1000000045e", "tid": 647 }, { "created_ino": "0x1000000045f", "tid": 648 }, { "created_ino": "0x10000000460", "tid": 649 }, { "created_ino": "0x10000000461", "tid": 650 }, { "created_ino": "0x10000000462", "tid": 651 }, { "created_ino": "0x10000000463", "tid": 652 }, { "created_ino": "0x10000000464", "tid": 653 }, { "created_ino": "0x10000000465", "tid": 654 }, { "created_ino": "0x10000000466", "tid": 655 }, { "created_ino": "0x10000000467", "tid": 656 }, { "created_ino": "0x10000000468", "tid": 657 }, { "created_ino": "0x10000000469", "tid": 658 }, { "created_ino": "0x1000000046a", "tid": 659 }, { "created_ino": "0x1000000046b", "tid": 660 }, { "created_ino": "0x1000000046c", "tid": 661 }, { "created_ino": "0x1000000046d", "tid": 662 }, { "created_ino": "0x1000000046e", "tid": 663 }, { "created_ino": "0x1000000046f", "tid": 664 }, { "created_ino": "0x10000000470", "tid": 665 }, { "created_ino": "0x10000000471", "tid": 666 }, { "created_ino": "0x10000000472", "tid": 667 }, { "created_ino": "0x10000000473", "tid": 668 }, { "created_ino": "0x10000000474", "tid": 669 }, { "created_ino": "0x10000000475", "tid": 670 }, { "created_ino": "0x10000000476", "tid": 671 }, { "created_ino": "0x10000000477", "tid": 672 }, { "created_ino": "0x10000000478", "tid": 673 }, { "created_ino": "0x10000000479", "tid": 674 }, { "created_ino": "0x1000000047a", "tid": 675 }, { "created_ino": "0x1000000047b", "tid": 676 }, { "created_ino": "0x1000000047c", "tid": 677 }, { "created_ino": "0x1000000047d", "tid": 678 }, { "created_ino": "0x1000000047e", "tid": 679 }, { "created_ino": "0x1000000047f", "tid": 680 }, { "created_ino": "0x10000000480", "tid": 681 }, { "created_ino": "0x10000000481", "tid": 682 }, { "created_ino": "0x10000000482", "tid": 683 }, { "created_ino": "0x10000000483", "tid": 684 }, { "created_ino": "0x10000000484", "tid": 685 }, { "created_ino": "0x10000000485", "tid": 686 }, { "created_ino": "0x10000000486", "tid": 687 }, { "created_ino": "0x10000000487", "tid": 688 }, { "created_ino": "0x10000000488", "tid": 689 }, { "created_ino": "0x10000000489", "tid": 690 }, { "created_ino": "0x1000000048a", "tid": 691 }, { "created_ino": "0x1000000048b", "tid": 692 }, { "created_ino": "0x1000000048c", "tid": 693 }, { "created_ino": "0x1000000048d", "tid": 694 }, { "created_ino": "0x1000000048e", "tid": 695 }, { "created_ino": "0x1000000048f", "tid": 696 }, { "created_ino": "0x10000000490", "tid": 697 }, { "created_ino": "0x10000000491", "tid": 698 }, { "created_ino": "0x10000000492", "tid": 699 }, { "created_ino": "0x10000000493", "tid": 700 }, { "created_ino": "0x10000000494", "tid": 701 }, { "created_ino": "0x10000000495", "tid": 702 }, { "created_ino": "0x10000000496", "tid": 703 }, { "created_ino": "0x10000000497", "tid": 704 }, { "created_ino": "0x10000000498", "tid": 705 }, { "created_ino": "0x10000000499", "tid": 706 }, { "created_ino": "0x1000000049a", "tid": 707 }, { "created_ino": "0x1000000049b", "tid": 708 }, { "created_ino": "0x1000000049c", "tid": 709 }, { "created_ino": "0x1000000049d", "tid": 710 }, { "created_ino": "0x1000000049e", "tid": 711 }, { "created_ino": "0x1000000049f", "tid": 712 }, { "created_ino": "0x100000004a0", "tid": 713 }, { "created_ino": "0x100000004a1", "tid": 714 }, { "created_ino": "0x100000004a2", "tid": 715 }, { "created_ino": "0x100000004a3", "tid": 716 }, { "created_ino": "0x100000004a4", "tid": 717 }, { "created_ino": "0x100000004a5", "tid": 718 }, { "created_ino": "0x100000004a6", "tid": 719 }, { "created_ino": "0x100000004a7", "tid": 720 }, { "created_ino": "0x100000004a8", "tid": 721 }, { "created_ino": "0x100000004a9", "tid": 722 }, { "created_ino": "0x100000004aa", "tid": 723 }, { "created_ino": "0x100000004ab", "tid": 724 }, { "created_ino": "0x100000004ac", "tid": 725 }, { "created_ino": "0x100000004ad", "tid": 726 }, { "created_ino": "0x100000004ae", "tid": 727 }, { "created_ino": "0x100000004af", "tid": 728 }, { "created_ino": "0x100000004b0", "tid": 729 }, { "created_ino": "0x100000004b1", "tid": 730 }, { "created_ino": "0x100000004b2", "tid": 731 }, { "created_ino": "0x100000004b3", "tid": 732 }, { "created_ino": "0x100000004b4", "tid": 733 }, { "created_ino": "0x100000004b5", "tid": 734 }, { "created_ino": "0x100000004b6", "tid": 735 }, { "created_ino": "0x100000004b7", "tid": 736 }, { "created_ino": "0x100000004b8", "tid": 737 }, { "created_ino": "0x100000004b9", "tid": 738 }, { "created_ino": "0x100000004ba", "tid": 739 }, { "created_ino": "0x100000004bb", "tid": 740 }, { "created_ino": "0x100000004bc", "tid": 741 }, { "created_ino": "0x100000004bd", "tid": 742 }, { "created_ino": "0x100000004be", "tid": 743 }, { "created_ino": "0x100000004bf", "tid": 744 }, { "created_ino": "0x100000004c0", "tid": 745 }, { "created_ino": "0x100000004c1", "tid": 746 }, { "created_ino": "0x100000004c2", "tid": 747 }, { "created_ino": "0x100000004c3", "tid": 748 }, { "created_ino": "0x100000004c4", "tid": 749 }, { "created_ino": "0x100000004c5", "tid": 750 }, { "created_ino": "0x100000004c6", "tid": 751 }, { "created_ino": "0x100000004c7", "tid": 752 }, { "created_ino": "0x100000004c8", "tid": 753 }, { "created_ino": "0x100000004c9", "tid": 754 }, { "created_ino": "0x100000004ca", "tid": 755 }, { "created_ino": "0x100000004cb", "tid": 756 }, { "created_ino": "0x100000004cc", "tid": 757 }, { "created_ino": "0x100000004cd", "tid": 758 }, { "created_ino": "0x100000004ce", "tid": 759 }, { "created_ino": "0x100000004cf", "tid": 760 }, { "created_ino": "0x100000004d0", "tid": 761 }, { "created_ino": "0x100000004d1", "tid": 762 }, { "created_ino": "0x100000004d2", "tid": 763 }, { "created_ino": "0x100000004d3", "tid": 764 }, { "created_ino": "0x100000004d4", "tid": 765 }, { "created_ino": "0x100000004d5", "tid": 766 }, { "created_ino": "0x100000004d6", "tid": 767 }, { "created_ino": "0x100000004d7", "tid": 768 }, { "created_ino": "0x100000004d8", "tid": 769 }, { "created_ino": "0x100000004d9", "tid": 770 }, { "created_ino": "0x100000004da", "tid": 771 }, { "created_ino": "0x100000004db", "tid": 772 }, { "created_ino": "0x100000004dc", "tid": 773 }, { "created_ino": "0x100000004dd", "tid": 774 }, { "created_ino": "0x100000004de", "tid": 775 }, { "created_ino": "0x100000004df", "tid": 776 }, { "created_ino": "0x100000004e0", "tid": 777 }, { "created_ino": "0x100000004e1", "tid": 778 }, { "created_ino": "0x100000004e2", "tid": 779 }, { "created_ino": "0x100000004e3", "tid": 780 }, { "created_ino": "0x100000004e4", "tid": 781 }, { "created_ino": "0x100000004e5", "tid": 782 }, { "created_ino": "0x100000004e6", "tid": 783 }, { "created_ino": "0x100000004e7", "tid": 784 }, { "created_ino": "0x100000004e8", "tid": 785 }, { "created_ino": "0x100000004e9", "tid": 786 }, { "created_ino": "0x100000004ea", "tid": 787 }, { "created_ino": "0x100000004eb", "tid": 788 }, { "created_ino": "0x100000004ec", "tid": 789 }, { "created_ino": "0x100000004ed", "tid": 790 }, { "created_ino": "0x100000004ee", "tid": 791 }, { "created_ino": "0x100000004ef", "tid": 792 }, { "created_ino": "0x100000004f0", "tid": 793 }, { "created_ino": "0x100000004f1", "tid": 794 }, { "created_ino": "0x100000004f2", "tid": 795 }, { "created_ino": "0x100000004f3", "tid": 796 }, { "created_ino": "0x100000004f4", "tid": 797 }, { "created_ino": "0x100000004f5", "tid": 798 }, { "created_ino": "0x100000004f6", "tid": 799 }, { "created_ino": "0x100000004f7", "tid": 800 }, { "created_ino": "0x100000004f8", "tid": 801 }, { "created_ino": "0x100000004f9", "tid": 802 }, { "created_ino": "0x100000004fa", "tid": 803 }, { "created_ino": "0x100000004fb", "tid": 804 }, { "created_ino": "0x100000004fc", "tid": 805 }, { "created_ino": "0x100000004fd", "tid": 806 }, { "created_ino": "0x100000004fe", "tid": 807 }, { "created_ino": "0x100000004ff", "tid": 808 }, { "created_ino": "0x10000000500", "tid": 809 }, { "created_ino": "0x10000000501", "tid": 810 }, { "created_ino": "0x10000000502", "tid": 811 }, { "created_ino": "0x10000000503", "tid": 812 }, { "created_ino": "0x10000000504", "tid": 813 }, { "created_ino": "0x10000000505", "tid": 814 }, { "created_ino": "0x10000000506", "tid": 815 }, { "created_ino": "0x10000000507", "tid": 816 }, { "created_ino": "0x10000000508", "tid": 817 }, { "created_ino": "0x10000000509", "tid": 818 }, { "created_ino": "0x1000000050a", "tid": 819 }, { "created_ino": "0x1000000050b", "tid": 820 }, { "created_ino": "0x1000000050c", "tid": 821 }, { "created_ino": "0x1000000050d", "tid": 822 }, { "created_ino": "0x1000000050e", "tid": 823 }, { "created_ino": "0x1000000050f", "tid": 824 }, { "created_ino": "0x10000000510", "tid": 825 }, { "created_ino": "0x10000000511", "tid": 826 }, { "created_ino": "0x10000000512", "tid": 827 }, { "created_ino": "0x10000000513", "tid": 828 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 1871709369, "type": "any" }, "name": { "num": 136013, "type": "client" } }, "id": 136013, "inst": "client.136013 172.21.15.16:0/1871709369", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 530, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 2680.1218450301794 }, "recall_caps_throttle": { "halflife": 1.5, "value": 714.3592253010003 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 173.92520850204335 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 760, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1578.4390614487188 }, "state": "open", "uptime": 11.807918354 } ] 2024-09-10T21:17:24.566 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:17:24.566 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:17:24.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:17:24.654 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:17:24.742 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:17:24.742 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:17:24.742 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 800\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi016') 2024-09-10T21:17:24.743 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:17:24.743 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:24.752 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:17:24.752 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:24.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:17:25.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 -- 172.21.15.16:0/4079238173 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10bc15a270 msgr2=0x7f10bc0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:25.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 --2- 172.21.15.16:0/4079238173 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10bc15a270 0x7f10bc0700a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00a980 tx=0x7f10ac01db00 comp rx=0 tx=0).stop 2024-09-10T21:17:25.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 -- 172.21.15.16:0/4079238173 shutdown_connections 2024-09-10T21:17:25.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 --2- 172.21.15.16:0/4079238173 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10bc15a270 0x7f10bc0700a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 -- 172.21.15.16:0/4079238173 >> 172.21.15.16:0/4079238173 conn(0x7f10bc0cfb20 msgr2=0x7f10bc0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:25.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 -- 172.21.15.16:0/4079238173 shutdown_connections 2024-09-10T21:17:25.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 -- 172.21.15.16:0/4079238173 wait complete. 2024-09-10T21:17:25.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 Processor -- start 2024-09-10T21:17:25.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 -- start start 2024-09-10T21:17:25.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10bc0705e0 0x7f10bc1fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:25.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc0ca170 0x7f10bc157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:25.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10bc15a270 0x7f10bc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc1686c0 con 0x7f10bc0705e0 2024-09-10T21:17:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc168540 con 0x7f10bc0ca170 2024-09-10T21:17:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10c1600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f10bc168840 con 0x7f10bc15a270 2024-09-10T21:17:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10baa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10bc0705e0 0x7f10bc1fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10baa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10bc0705e0 0x7f10bc1fc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55086/0 (socket says 172.21.15.16:55086) 2024-09-10T21:17:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10baa00640 1 -- 172.21.15.16:0/4145198644 learned_addr learned my addr 172.21.15.16:0/4145198644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10ba000640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc0ca170 0x7f10bc157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10bb400640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10bc15a270 0x7f10bc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10baa00640 1 -- 172.21.15.16:0/4145198644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10bc15a270 msgr2=0x7f10bc0b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10baa00640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10bc15a270 0x7f10bc0b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10baa00640 1 -- 172.21.15.16:0/4145198644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc0ca170 msgr2=0x7f10bc157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10baa00640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc0ca170 0x7f10bc157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10baa00640 1 -- 172.21.15.16:0/4145198644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc0b0ee0 con 0x7f10bc0705e0 2024-09-10T21:17:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:24.997+0000 7f10bb400640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10bc15a270 0x7f10bc0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.001+0000 7f10baa00640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10bc0705e0 0x7f10bc1fc0d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f10ac01a710 tx=0x7f10ac005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.001+0000 7f10b2a00640 1 -- 172.21.15.16:0/4145198644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0be070 con 0x7f10bc0705e0 2024-09-10T21:17:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.001+0000 7f10b2a00640 1 -- 172.21.15.16:0/4145198644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10ac0038f0 con 0x7f10bc0705e0 2024-09-10T21:17:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.001+0000 7f10c1600640 1 -- 172.21.15.16:0/4145198644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10bc1fb610 con 0x7f10bc0705e0 2024-09-10T21:17:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.001+0000 7f10b2a00640 1 -- 172.21.15.16:0/4145198644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0b3040 con 0x7f10bc0705e0 2024-09-10T21:17:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.001+0000 7f10c1600640 1 -- 172.21.15.16:0/4145198644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10bc15abd0 con 0x7f10bc0705e0 2024-09-10T21:17:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.001+0000 7f10c1600640 1 -- 172.21.15.16:0/4145198644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10bc0700a0 con 0x7f10bc0705e0 2024-09-10T21:17:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.001+0000 7f10b2a00640 1 -- 172.21.15.16:0/4145198644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f10ac00b040 con 0x7f10bc0705e0 2024-09-10T21:17:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.001+0000 7f10b2a00640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f109c0725e0 0x7f109c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.001+0000 7f10b2a00640 1 -- 172.21.15.16:0/4145198644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f10ac12d890 con 0x7f10bc0705e0 2024-09-10T21:17:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.001+0000 7f10ba000640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f109c0725e0 0x7f109c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.005+0000 7f10ba000640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f109c0725e0 0x7f109c074aa0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f10bc1fb570 tx=0x7f10a8001440 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.005+0000 7f10b2a00640 1 -- 172.21.15.16:0/4145198644 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f10ac017780 con 0x7f10bc0705e0 2024-09-10T21:17:25.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.193+0000 7f10c1600640 1 -- 172.21.15.16:0/4145198644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f10bc0b5aa0 con 0x7f10bc0705e0 2024-09-10T21:17:25.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.193+0000 7f10b2a00640 1 -- 172.21.15.16:0/4145198644 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7f10ac0f88e0 con 0x7f10bc0705e0 2024-09-10T21:17:25.198 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:17:25.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.193+0000 7f10c1600640 1 -- 172.21.15.16:0/4145198644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f109c0725e0 msgr2=0x7f109c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:25.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.193+0000 7f10c1600640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f109c0725e0 0x7f109c074aa0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f10bc1fb570 tx=0x7f10a8001440 comp rx=0 tx=0).stop 2024-09-10T21:17:25.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.193+0000 7f10c1600640 1 -- 172.21.15.16:0/4145198644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10bc0705e0 msgr2=0x7f10bc1fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:25.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.193+0000 7f10c1600640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10bc0705e0 0x7f10bc1fc0d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f10ac01a710 tx=0x7f10ac005760 comp rx=0 tx=0).stop 2024-09-10T21:17:25.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.197+0000 7f10c1600640 1 -- 172.21.15.16:0/4145198644 shutdown_connections 2024-09-10T21:17:25.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.197+0000 7f10c1600640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f109c0725e0 0x7f109c074aa0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.197+0000 7f10c1600640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f10bc15a270 0x7f10bc0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.197+0000 7f10c1600640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc0ca170 0x7f10bc157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.197+0000 7f10c1600640 1 --2- 172.21.15.16:0/4145198644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10bc0705e0 0x7f10bc1fc0d0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.197+0000 7f10c1600640 1 -- 172.21.15.16:0/4145198644 >> 172.21.15.16:0/4145198644 conn(0x7f10bc0cfb20 msgr2=0x7f10bc0c55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.197+0000 7f10c1600640 1 -- 172.21.15.16:0/4145198644 shutdown_connections 2024-09-10T21:17:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.197+0000 7f10c1600640 1 -- 172.21.15.16:0/4145198644 wait complete. 2024-09-10T21:17:25.219 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:17:25.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:25.219 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:17:25.429 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:17:25.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:25.430 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:17:25.439 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:17:25.440 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:17:25.440 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:25.449 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:17:25.449 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:25.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:17:25.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.645+0000 7f54be600640 1 -- 172.21.15.16:0/2061466714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b80d03c0 msgr2=0x7f54b815fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:25.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.645+0000 7f54be600640 1 --2- 172.21.15.16:0/2061466714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b80d03c0 0x7f54b815fe00 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f54b8070400 tx=0x7f54a001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.645+0000 7f54be600640 1 -- 172.21.15.16:0/2061466714 shutdown_connections 2024-09-10T21:17:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.645+0000 7f54be600640 1 --2- 172.21.15.16:0/2061466714 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f54b80b2860 0x7f54b8160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.645+0000 7f54be600640 1 --2- 172.21.15.16:0/2061466714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b80d03c0 0x7f54b815fe00 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.645+0000 7f54be600640 1 --2- 172.21.15.16:0/2061466714 >> v2:172.21.15.52:3300/0 conn(0x7f54b80cffe0 0x7f54b80ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.645+0000 7f54be600640 1 -- 172.21.15.16:0/2061466714 >> 172.21.15.16:0/2061466714 conn(0x7f54b80d1d80 msgr2=0x7f54b80d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.645+0000 7f54be600640 1 -- 172.21.15.16:0/2061466714 shutdown_connections 2024-09-10T21:17:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.645+0000 7f54be600640 1 -- 172.21.15.16:0/2061466714 wait complete. 2024-09-10T21:17:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.645+0000 7f54be600640 1 Processor -- start 2024-09-10T21:17:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54be600640 1 -- start start 2024-09-10T21:17:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54be600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f54b80b2860 0x7f54b80c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54be600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b80cffe0 0x7f54b815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54be600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b80d03c0 0x7f54b81f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54be600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54b80cca10 con 0x7f54b80d03c0 2024-09-10T21:17:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54be600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54b80cc890 con 0x7f54b80cffe0 2024-09-10T21:17:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54be600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f54b80ccb90 con 0x7f54b80b2860 2024-09-10T21:17:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54b7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b80d03c0 0x7f54b81f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54b7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b80d03c0 0x7f54b81f79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55110/0 (socket says 172.21.15.16:55110) 2024-09-10T21:17:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54b7400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f54b80b2860 0x7f54b80c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54b7e00640 1 -- 172.21.15.16:0/2919483508 learned_addr learned my addr 172.21.15.16:0/2919483508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54afe00640 1 --2- 172.21.15.16:0/2919483508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b80cffe0 0x7f54b815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54b7e00640 1 -- 172.21.15.16:0/2919483508 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f54b80b2860 msgr2=0x7f54b80c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54b7e00640 1 --2- 172.21.15.16:0/2919483508 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f54b80b2860 0x7f54b80c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54b7e00640 1 -- 172.21.15.16:0/2919483508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b80cffe0 msgr2=0x7f54b815b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54b7e00640 1 --2- 172.21.15.16:0/2919483508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b80cffe0 0x7f54b815b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54b7e00640 1 -- 172.21.15.16:0/2919483508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b806fc80 con 0x7f54b80d03c0 2024-09-10T21:17:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54b7e00640 1 --2- 172.21.15.16:0/2919483508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b80d03c0 0x7f54b81f79e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f54b000eab0 tx=0x7f54b000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54af400640 1 -- 172.21.15.16:0/2919483508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0004b30 con 0x7f54b80d03c0 2024-09-10T21:17:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54af400640 1 -- 172.21.15.16:0/2919483508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54b0004dc0 con 0x7f54b80d03c0 2024-09-10T21:17:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54af400640 1 -- 172.21.15.16:0/2919483508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0005280 con 0x7f54b80d03c0 2024-09-10T21:17:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54be600640 1 -- 172.21.15.16:0/2919483508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b80b7460 con 0x7f54b80d03c0 2024-09-10T21:17:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54be600640 1 -- 172.21.15.16:0/2919483508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b80b7700 con 0x7f54b80d03c0 2024-09-10T21:17:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.649+0000 7f54be600640 1 -- 172.21.15.16:0/2919483508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b815eb40 con 0x7f54b80d03c0 2024-09-10T21:17:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.653+0000 7f54af400640 1 -- 172.21.15.16:0/2919483508 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f54b0020030 con 0x7f54b80d03c0 2024-09-10T21:17:25.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.653+0000 7f54af400640 1 --2- 172.21.15.16:0/2919483508 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54940725e0 0x7f5494074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:25.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.653+0000 7f54af400640 1 -- 172.21.15.16:0/2919483508 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f54b0093d70 con 0x7f54b80d03c0 2024-09-10T21:17:25.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.653+0000 7f54b7400640 1 --2- 172.21.15.16:0/2919483508 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54940725e0 0x7f5494074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.653+0000 7f54b7400640 1 --2- 172.21.15.16:0/2919483508 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54940725e0 0x7f5494074aa0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f54a8005d90 tx=0x7f54a800b390 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.657+0000 7f54af400640 1 -- 172.21.15.16:0/2919483508 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f54b005edb0 con 0x7f54b80d03c0 2024-09-10T21:17:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54be600640 1 -- 172.21.15.16:0/2919483508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f54b80cf950 con 0x7f54b80d03c0 2024-09-10T21:17:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54af400640 1 -- 172.21.15.16:0/2919483508 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7f54b00637e0 con 0x7f54b80d03c0 2024-09-10T21:17:25.848 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:17:25.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54be600640 1 -- 172.21.15.16:0/2919483508 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54940725e0 msgr2=0x7f5494074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54be600640 1 --2- 172.21.15.16:0/2919483508 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54940725e0 0x7f5494074aa0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f54a8005d90 tx=0x7f54a800b390 comp rx=0 tx=0).stop 2024-09-10T21:17:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54be600640 1 -- 172.21.15.16:0/2919483508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b80d03c0 msgr2=0x7f54b81f79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54be600640 1 --2- 172.21.15.16:0/2919483508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b80d03c0 0x7f54b81f79e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f54b000eab0 tx=0x7f54b000ef70 comp rx=0 tx=0).stop 2024-09-10T21:17:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54be600640 1 -- 172.21.15.16:0/2919483508 shutdown_connections 2024-09-10T21:17:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54be600640 1 --2- 172.21.15.16:0/2919483508 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f54940725e0 0x7f5494074aa0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54be600640 1 --2- 172.21.15.16:0/2919483508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b80d03c0 0x7f54b81f79e0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54be600640 1 --2- 172.21.15.16:0/2919483508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b80cffe0 0x7f54b815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54be600640 1 --2- 172.21.15.16:0/2919483508 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f54b80b2860 0x7f54b80c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54be600640 1 -- 172.21.15.16:0/2919483508 >> 172.21.15.16:0/2919483508 conn(0x7f54b80d1d80 msgr2=0x7f54b8072220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54be600640 1 -- 172.21.15.16:0/2919483508 shutdown_connections 2024-09-10T21:17:25.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:25.845+0000 7f54be600640 1 -- 172.21.15.16:0/2919483508 wait complete. 2024-09-10T21:17:25.871 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T21:17:25.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:25.871 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T21:17:25.897 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:17:25.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:25.897 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T21:17:25.952 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:17:25.952 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:17:25.952 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:25.962 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:17:25.962 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:17:25.962 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:17:25.962 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:26.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:17:26.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.145+0000 7f4049e00640 1 -- 172.21.15.16:0/3242694015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40440716f0 msgr2=0x7f40440af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.145+0000 7f4049e00640 1 --2- 172.21.15.16:0/3242694015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40440716f0 0x7f40440af580 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f402c009840 tx=0x7f402c01c700 comp rx=0 tx=0).stop 2024-09-10T21:17:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.145+0000 7f4049e00640 1 -- 172.21.15.16:0/3242694015 shutdown_connections 2024-09-10T21:17:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.145+0000 7f4049e00640 1 --2- 172.21.15.16:0/3242694015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40440716f0 0x7f40440af580 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.145+0000 7f4049e00640 1 -- 172.21.15.16:0/3242694015 >> 172.21.15.16:0/3242694015 conn(0x7f404409b130 msgr2=0x7f40440d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.145+0000 7f4049e00640 1 -- 172.21.15.16:0/3242694015 shutdown_connections 2024-09-10T21:17:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.145+0000 7f4049e00640 1 -- 172.21.15.16:0/3242694015 wait complete. 2024-09-10T21:17:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.145+0000 7f4049e00640 1 Processor -- start 2024-09-10T21:17:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.145+0000 7f4049e00640 1 -- start start 2024-09-10T21:17:26.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4049e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40440716f0 0x7f40441570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:26.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4049e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40440b13c0 0x7f404415de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:26.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4049e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f404415f400 0x7f40440b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:26.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4049e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f404416a020 con 0x7f404415f400 2024-09-10T21:17:26.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4049e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4044169ea0 con 0x7f40440b13c0 2024-09-10T21:17:26.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4049e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f404416a1a0 con 0x7f40440716f0 2024-09-10T21:17:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4043400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f404415f400 0x7f40440b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4043400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f404415f400 0x7f40440b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55126/0 (socket says 172.21.15.16:55126) 2024-09-10T21:17:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4043400640 1 -- 172.21.15.16:0/1491950637 learned_addr learned my addr 172.21.15.16:0/1491950637 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f403be00640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40440b13c0 0x7f404415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4042a00640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40440716f0 0x7f40441570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4043400640 1 -- 172.21.15.16:0/1491950637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40440716f0 msgr2=0x7f40441570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4043400640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40440716f0 0x7f40441570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4043400640 1 -- 172.21.15.16:0/1491950637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40440b13c0 msgr2=0x7f404415de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4043400640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40440b13c0 0x7f404415de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4043400640 1 -- 172.21.15.16:0/1491950637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40440b67b0 con 0x7f404415f400 2024-09-10T21:17:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4043400640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f404415f400 0x7f40440b4bb0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f403c00cbc0 tx=0x7f403c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f403b400640 1 -- 172.21.15.16:0/1491950637 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c016020 con 0x7f404415f400 2024-09-10T21:17:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f403b400640 1 -- 172.21.15.16:0/1491950637 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f403c00ab00 con 0x7f404415f400 2024-09-10T21:17:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4049e00640 1 -- 172.21.15.16:0/1491950637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40440cb5b0 con 0x7f404415f400 2024-09-10T21:17:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f403b400640 1 -- 172.21.15.16:0/1491950637 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c012420 con 0x7f404415f400 2024-09-10T21:17:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4049e00640 1 -- 172.21.15.16:0/1491950637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40440b6a30 con 0x7f404415f400 2024-09-10T21:17:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.149+0000 7f4049e00640 1 -- 172.21.15.16:0/1491950637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40440cb9a0 con 0x7f404415f400 2024-09-10T21:17:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.153+0000 7f403b400640 1 -- 172.21.15.16:0/1491950637 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f403c0125c0 con 0x7f404415f400 2024-09-10T21:17:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.153+0000 7f403b400640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40200725e0 0x7f4020074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.153+0000 7f403b400640 1 -- 172.21.15.16:0/1491950637 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f403c0938b0 con 0x7f404415f400 2024-09-10T21:17:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.153+0000 7f4042a00640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40200725e0 0x7f4020074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.153+0000 7f4042a00640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40200725e0 0x7f4020074aa0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f404415eae0 tx=0x7f4034009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.157+0000 7f403b400640 1 -- 172.21.15.16:0/1491950637 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f403c020020 con 0x7f404415f400 2024-09-10T21:17:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.365+0000 7f4049e00640 1 -- 172.21.15.16:0/1491950637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4044070fe0 con 0x7f404415f400 2024-09-10T21:17:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.365+0000 7f403b400640 1 -- 172.21.15.16:0/1491950637 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5235 (secure 0 0 0) 0x7f40440b67b0 con 0x7f404415f400 2024-09-10T21:17:26.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:26.371 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"btime":"2024-09-10T21:17:21:843659+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3765085271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765085271},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765085271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":7104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4198210360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":4198210360},{"type":"v1","addr":"172.21.15.16:6837","nonce":4198210360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2954517211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2954517211},{"type":"v1","addr":"172.21.15.52:6835","nonce":2954517211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:05.094283+0000","modified":"2024-09-10T21:17:21.746067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135926":{"gid":135926,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":7,"addr":"172.21.15.52:6834/1108687782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108687782},{"type":"v1","addr":"172.21.15.52:6834","nonce":1108687782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135926,"qdb_cluster":[135926]},"id":8}]} 2024-09-10T21:17:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 73 2024-09-10T21:17:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.369+0000 7f4049e00640 1 -- 172.21.15.16:0/1491950637 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40200725e0 msgr2=0x7f4020074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.369+0000 7f4049e00640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40200725e0 0x7f4020074aa0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f404415eae0 tx=0x7f4034009040 comp rx=0 tx=0).stop 2024-09-10T21:17:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.369+0000 7f4049e00640 1 -- 172.21.15.16:0/1491950637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f404415f400 msgr2=0x7f40440b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.369+0000 7f4049e00640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f404415f400 0x7f40440b4bb0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f403c00cbc0 tx=0x7f403c00ccb0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.369+0000 7f4049e00640 1 -- 172.21.15.16:0/1491950637 shutdown_connections 2024-09-10T21:17:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.369+0000 7f4049e00640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40200725e0 0x7f4020074aa0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.369+0000 7f4049e00640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f404415f400 0x7f40440b4bb0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.369+0000 7f4049e00640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40440b13c0 0x7f404415de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.369+0000 7f4049e00640 1 --2- 172.21.15.16:0/1491950637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40440716f0 0x7f40441570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.369+0000 7f4049e00640 1 -- 172.21.15.16:0/1491950637 >> 172.21.15.16:0/1491950637 conn(0x7f404409b130 msgr2=0x7f404406e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:26.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.369+0000 7f4049e00640 1 -- 172.21.15.16:0/1491950637 shutdown_connections 2024-09-10T21:17:26.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.369+0000 7f4049e00640 1 -- 172.21.15.16:0/1491950637 wait complete. 2024-09-10T21:17:26.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:26.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 -- 172.21.15.16:0/2394765554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e240d2820 msgr2=0x7f9e2415fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:26.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 --2- 172.21.15.16:0/2394765554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e240d2820 0x7f9e2415fbe0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009900 tx=0x7f9e0c01c850 comp rx=0 tx=0).stop 2024-09-10T21:17:26.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 -- 172.21.15.16:0/2394765554 shutdown_connections 2024-09-10T21:17:26.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 --2- 172.21.15.16:0/2394765554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e240d2820 0x7f9e2415fbe0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 --2- 172.21.15.16:0/2394765554 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e240cfd40 0x7f9e240d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 -- 172.21.15.16:0/2394765554 >> 172.21.15.16:0/2394765554 conn(0x7f9e240d1d00 msgr2=0x7f9e240d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:26.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 -- 172.21.15.16:0/2394765554 shutdown_connections 2024-09-10T21:17:26.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 -- 172.21.15.16:0/2394765554 wait complete. 2024-09-10T21:17:26.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 Processor -- start 2024-09-10T21:17:26.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 -- start start 2024-09-10T21:17:26.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e240cfd40 0x7f9e240c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:26.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e240d2820 0x7f9e240b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:26.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e24160120 0x7f9e2415b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e24169360 con 0x7f9e24160120 2024-09-10T21:17:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9e241691e0 con 0x7f9e240cfd40 2024-09-10T21:17:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9e241694e0 con 0x7f9e240d2820 2024-09-10T21:17:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e24160120 0x7f9e2415b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e23e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e240cfd40 0x7f9e240c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e24160120 0x7f9e2415b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55142/0 (socket says 172.21.15.16:55142) 2024-09-10T21:17:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e29000640 1 -- 172.21.15.16:0/1026723347 learned_addr learned my addr 172.21.15.16:0/1026723347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.537+0000 7f9e1b400640 1 --2- 172.21.15.16:0/1026723347 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e240d2820 0x7f9e240b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e29000640 1 -- 172.21.15.16:0/1026723347 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e240d2820 msgr2=0x7f9e240b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e29000640 1 --2- 172.21.15.16:0/1026723347 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e240d2820 0x7f9e240b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e29000640 1 -- 172.21.15.16:0/1026723347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e240cfd40 msgr2=0x7f9e240c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e29000640 1 --2- 172.21.15.16:0/1026723347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e240cfd40 0x7f9e240c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e29000640 1 -- 172.21.15.16:0/1026723347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e240c8b80 con 0x7f9e24160120 2024-09-10T21:17:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e29000640 1 --2- 172.21.15.16:0/1026723347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e24160120 0x7f9e2415b800 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00c6b0 tx=0x7f9e1c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e20c00640 1 -- 172.21.15.16:0/1026723347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c003b80 con 0x7f9e24160120 2024-09-10T21:17:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e20c00640 1 -- 172.21.15.16:0/1026723347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e1c003d40 con 0x7f9e24160120 2024-09-10T21:17:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e29a00640 1 -- 172.21.15.16:0/1026723347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e241529b0 con 0x7f9e24160120 2024-09-10T21:17:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e20c00640 1 -- 172.21.15.16:0/1026723347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c005330 con 0x7f9e24160120 2024-09-10T21:17:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e29a00640 1 -- 172.21.15.16:0/1026723347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e24152f70 con 0x7f9e24160120 2024-09-10T21:17:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e29a00640 1 -- 172.21.15.16:0/1026723347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e240d0120 con 0x7f9e24160120 2024-09-10T21:17:26.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e20c00640 1 -- 172.21.15.16:0/1026723347 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9e1c0054d0 con 0x7f9e24160120 2024-09-10T21:17:26.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e20c00640 1 --2- 172.21.15.16:0/1026723347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e00072610 0x7f9e00074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:26.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.541+0000 7f9e20c00640 1 -- 172.21.15.16:0/1026723347 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f9e1c092e10 con 0x7f9e24160120 2024-09-10T21:17:26.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.545+0000 7f9e23e00640 1 --2- 172.21.15.16:0/1026723347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e00072610 0x7f9e00074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:26.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.545+0000 7f9e23e00640 1 --2- 172.21.15.16:0/1026723347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e00072610 0x7f9e00074ad0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9e14005d90 tx=0x7f9e1400b390 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.545+0000 7f9e20c00640 1 -- 172.21.15.16:0/1026723347 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9e1c096030 con 0x7f9e24160120 2024-09-10T21:17:26.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.753+0000 7f9e29a00640 1 -- 172.21.15.16:0/1026723347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e241598d0 con 0x7f9e24160120 2024-09-10T21:17:26.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.753+0000 7f9e20c00640 1 -- 172.21.15.16:0/1026723347 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5235 (secure 0 0 0) 0x7f9e1c05de50 con 0x7f9e24160120 2024-09-10T21:17:26.758 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:26.758 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"btime":"2024-09-10T21:17:21:843659+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3765085271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765085271},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765085271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":7104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4198210360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":4198210360},{"type":"v1","addr":"172.21.15.16:6837","nonce":4198210360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2954517211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2954517211},{"type":"v1","addr":"172.21.15.52:6835","nonce":2954517211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:05.094283+0000","modified":"2024-09-10T21:17:21.746067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135926":{"gid":135926,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":7,"addr":"172.21.15.52:6834/1108687782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108687782},{"type":"v1","addr":"172.21.15.52:6834","nonce":1108687782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135926,"qdb_cluster":[135926]},"id":8}]} 2024-09-10T21:17:26.759 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 73 2024-09-10T21:17:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.757+0000 7f9e29a00640 1 -- 172.21.15.16:0/1026723347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e00072610 msgr2=0x7f9e00074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.757+0000 7f9e29a00640 1 --2- 172.21.15.16:0/1026723347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e00072610 0x7f9e00074ad0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9e14005d90 tx=0x7f9e1400b390 comp rx=0 tx=0).stop 2024-09-10T21:17:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.757+0000 7f9e29a00640 1 -- 172.21.15.16:0/1026723347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e24160120 msgr2=0x7f9e2415b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.757+0000 7f9e29a00640 1 --2- 172.21.15.16:0/1026723347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e24160120 0x7f9e2415b800 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00c6b0 tx=0x7f9e1c00cb70 comp rx=0 tx=0).stop 2024-09-10T21:17:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.757+0000 7f9e29a00640 1 -- 172.21.15.16:0/1026723347 shutdown_connections 2024-09-10T21:17:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.757+0000 7f9e29a00640 1 --2- 172.21.15.16:0/1026723347 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9e00072610 0x7f9e00074ad0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.757+0000 7f9e29a00640 1 --2- 172.21.15.16:0/1026723347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e24160120 0x7f9e2415b800 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.757+0000 7f9e29a00640 1 --2- 172.21.15.16:0/1026723347 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9e240d2820 0x7f9e240b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.757+0000 7f9e29a00640 1 --2- 172.21.15.16:0/1026723347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e240cfd40 0x7f9e240c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.757+0000 7f9e29a00640 1 -- 172.21.15.16:0/1026723347 >> 172.21.15.16:0/1026723347 conn(0x7f9e240d1d00 msgr2=0x7f9e240d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.757+0000 7f9e29a00640 1 -- 172.21.15.16:0/1026723347 shutdown_connections 2024-09-10T21:17:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.757+0000 7f9e29a00640 1 -- 172.21.15.16:0/1026723347 wait complete. 2024-09-10T21:17:26.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:17:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.921+0000 7fed2ba00640 1 -- 172.21.15.16:0/843675824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed240d0890 msgr2=0x7fed240b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.921+0000 7fed2ba00640 1 --2- 172.21.15.16:0/843675824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed240d0890 0x7fed240b1410 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fed24073d10 tx=0x7fed1c01f580 comp rx=0 tx=0).stop 2024-09-10T21:17:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.921+0000 7fed2ba00640 1 -- 172.21.15.16:0/843675824 shutdown_connections 2024-09-10T21:17:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.921+0000 7fed2ba00640 1 --2- 172.21.15.16:0/843675824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed240d0890 0x7fed240b1410 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.921+0000 7fed2ba00640 1 --2- 172.21.15.16:0/843675824 >> v2:172.21.15.52:3300/0 conn(0x7fed240b3200 0x7fed240d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.921+0000 7fed2ba00640 1 --2- 172.21.15.16:0/843675824 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed2415f450 0x7fed240cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.921+0000 7fed2ba00640 1 -- 172.21.15.16:0/843675824 >> 172.21.15.16:0/843675824 conn(0x7fed240d1d80 msgr2=0x7fed240d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.921+0000 7fed2ba00640 1 -- 172.21.15.16:0/843675824 shutdown_connections 2024-09-10T21:17:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.921+0000 7fed2ba00640 1 -- 172.21.15.16:0/843675824 wait complete. 2024-09-10T21:17:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed2ba00640 1 Processor -- start 2024-09-10T21:17:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed2ba00640 1 -- start start 2024-09-10T21:17:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed2ba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed240d0890 0x7fed240cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed2ba00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed2415f450 0x7fed24154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed2ba00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed24155380 0x7fed240cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed2ba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed240b1950 con 0x7fed240d0890 2024-09-10T21:17:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed2ba00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fed240b2f80 con 0x7fed24155380 2024-09-10T21:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed2ba00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fed240b1ad0 con 0x7fed2415f450 2024-09-10T21:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed29000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed240d0890 0x7fed240cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed29000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed240d0890 0x7fed240cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55172/0 (socket says 172.21.15.16:55172) 2024-09-10T21:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed29000640 1 -- 172.21.15.16:0/3924122792 learned_addr learned my addr 172.21.15.16:0/3924122792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed29a00640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed24155380 0x7fed240cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed1be00640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed2415f450 0x7fed24154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed29000640 1 -- 172.21.15.16:0/3924122792 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed2415f450 msgr2=0x7fed24154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed29000640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed2415f450 0x7fed24154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed29000640 1 -- 172.21.15.16:0/3924122792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed24155380 msgr2=0x7fed240cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed29000640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed24155380 0x7fed240cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed29000640 1 -- 172.21.15.16:0/3924122792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed240ce8d0 con 0x7fed240d0890 2024-09-10T21:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed29a00640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed24155380 0x7fed240cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:26.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed29000640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed240d0890 0x7fed240cf2a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fed1400d780 tx=0x7fed1400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed21600640 1 -- 172.21.15.16:0/3924122792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14017070 con 0x7fed240d0890 2024-09-10T21:17:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed21600640 1 -- 172.21.15.16:0/3924122792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed1400de90 con 0x7fed240d0890 2024-09-10T21:17:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed21600640 1 -- 172.21.15.16:0/3924122792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14005020 con 0x7fed240d0890 2024-09-10T21:17:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed2ba00640 1 -- 172.21.15.16:0/3924122792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed241f2d70 con 0x7fed240d0890 2024-09-10T21:17:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.925+0000 7fed2ba00640 1 -- 172.21.15.16:0/3924122792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed241f3330 con 0x7fed240d0890 2024-09-10T21:17:26.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.929+0000 7fed2ba00640 1 -- 172.21.15.16:0/3924122792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed240b1fd0 con 0x7fed240d0890 2024-09-10T21:17:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.929+0000 7fed21600640 1 -- 172.21.15.16:0/3924122792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fed140051c0 con 0x7fed240d0890 2024-09-10T21:17:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.929+0000 7fed21600640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed000725e0 0x7fed00074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.929+0000 7fed21600640 1 -- 172.21.15.16:0/3924122792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7fed140933f0 con 0x7fed240d0890 2024-09-10T21:17:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.929+0000 7fed1be00640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed000725e0 0x7fed00074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.929+0000 7fed1be00640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed000725e0 0x7fed00074aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fed240728b0 tx=0x7fed0c00a4e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:26.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:26.933+0000 7fed21600640 1 -- 172.21.15.16:0/3924122792 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fed1405e9f0 con 0x7fed240d0890 2024-09-10T21:17:27.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.109+0000 7fed2ba00640 1 -- 172.21.15.16:0/3924122792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed240c55e0 con 0x7fed240d0890 2024-09-10T21:17:27.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.109+0000 7fed21600640 1 -- 172.21.15.16:0/3924122792 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16781 (secure 0 0 0) 0x7fed14067110 con 0x7fed240d0890 2024-09-10T21:17:27.114 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:27.115 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":105,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:17:11.735725+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:17:02.802106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T21:17:04.057780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":4,"to":3}]},{"pgid":"17.37","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:17:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.113+0000 7fed2ba00640 1 -- 172.21.15.16:0/3924122792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed000725e0 msgr2=0x7fed00074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.113+0000 7fed2ba00640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed000725e0 0x7fed00074aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fed240728b0 tx=0x7fed0c00a4e0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.113+0000 7fed2ba00640 1 -- 172.21.15.16:0/3924122792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed240d0890 msgr2=0x7fed240cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.113+0000 7fed2ba00640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed240d0890 0x7fed240cf2a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fed1400d780 tx=0x7fed1400dc40 comp rx=0 tx=0).stop 2024-09-10T21:17:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.113+0000 7fed2ba00640 1 -- 172.21.15.16:0/3924122792 shutdown_connections 2024-09-10T21:17:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.113+0000 7fed2ba00640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed000725e0 0x7fed00074aa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.113+0000 7fed2ba00640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed24155380 0x7fed240cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.113+0000 7fed2ba00640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed2415f450 0x7fed24154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.113+0000 7fed2ba00640 1 --2- 172.21.15.16:0/3924122792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed240d0890 0x7fed240cf2a0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.113+0000 7fed2ba00640 1 -- 172.21.15.16:0/3924122792 >> 172.21.15.16:0/3924122792 conn(0x7fed240d1d80 msgr2=0x7fed24071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:27.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.113+0000 7fed2ba00640 1 -- 172.21.15.16:0/3924122792 shutdown_connections 2024-09-10T21:17:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.113+0000 7fed2ba00640 1 -- 172.21.15.16:0/3924122792 wait complete. 2024-09-10T21:17:27.136 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:17:27.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 -- 172.21.15.16:0/3893353107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c004290 msgr2=0x7f674c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 --2- 172.21.15.16:0/3893353107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c004290 0x7f674c004810 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f6750009b60 tx=0x7f675001d270 comp rx=0 tx=0).stop 2024-09-10T21:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f6769e00640 1 -- 172.21.15.16:0/3893353107 reap_dead start 2024-09-10T21:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 -- 172.21.15.16:0/3893353107 shutdown_connections 2024-09-10T21:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 -- 172.21.15.16:0/3893353107 >> 172.21.15.16:0/3893353107 conn(0x7f67640d1d00 msgr2=0x7f67640d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 -- 172.21.15.16:0/3893353107 shutdown_connections 2024-09-10T21:17:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 -- 172.21.15.16:0/3893353107 wait complete. 2024-09-10T21:17:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 Processor -- start 2024-09-10T21:17:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 -- start start 2024-09-10T21:17:27.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c004290 0x7f676415af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:27.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67640b5540 0x7f67640b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:27.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67640cc610 0x7f67640cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:27.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6764167b50 con 0x7f674c004290 2024-09-10T21:17:27.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f67641679d0 con 0x7f67640cc610 2024-09-10T21:17:27.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f676a800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6764167cd0 con 0x7f67640b5540 2024-09-10T21:17:27.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f6769400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c004290 0x7f676415af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f6769400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c004290 0x7f676415af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55188/0 (socket says 172.21.15.16:55188) 2024-09-10T21:17:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f6769400640 1 -- 172.21.15.16:0/3745390845 learned_addr learned my addr 172.21.15.16:0/3745390845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f6769e00640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67640cc610 0x7f67640cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f6768a00640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67640b5540 0x7f67640b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f6769400640 1 -- 172.21.15.16:0/3745390845 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67640b5540 msgr2=0x7f67640b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f6769400640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67640b5540 0x7f67640b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f6769400640 1 -- 172.21.15.16:0/3745390845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67640cc610 msgr2=0x7f67640cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f6769400640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67640cc610 0x7f67640cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f6769400640 1 -- 172.21.15.16:0/3745390845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67640ca020 con 0x7f674c004290 2024-09-10T21:17:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f6769400640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c004290 0x7f676415af80 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f675800e5f0 tx=0x7f675800eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.281+0000 7f6761600640 1 -- 172.21.15.16:0/3745390845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758015070 con 0x7f674c004290 2024-09-10T21:17:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.285+0000 7f6761600640 1 -- 172.21.15.16:0/3745390845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6758010980 con 0x7f674c004290 2024-09-10T21:17:27.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.285+0000 7f6761600640 1 -- 172.21.15.16:0/3745390845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758010dd0 con 0x7f674c004290 2024-09-10T21:17:27.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.285+0000 7f676a800640 1 -- 172.21.15.16:0/3745390845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67640ca2e0 con 0x7f674c004290 2024-09-10T21:17:27.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.285+0000 7f676a800640 1 -- 172.21.15.16:0/3745390845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67640ca8a0 con 0x7f674c004290 2024-09-10T21:17:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.285+0000 7f676a800640 1 -- 172.21.15.16:0/3745390845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67640caaa0 con 0x7f674c004290 2024-09-10T21:17:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.285+0000 7f6761600640 1 -- 172.21.15.16:0/3745390845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6758011460 con 0x7f674c004290 2024-09-10T21:17:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.285+0000 7f6761600640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6744072610 0x7f6744074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.285+0000 7f6761600640 1 -- 172.21.15.16:0/3745390845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f6758093900 con 0x7f674c004290 2024-09-10T21:17:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.285+0000 7f6768a00640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6744072610 0x7f6744074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.285+0000 7f6768a00640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6744072610 0x7f6744074ad0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f67500ae000 tx=0x7f675001a010 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:27.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.289+0000 7f6761600640 1 -- 172.21.15.16:0/3745390845 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f67640caaa0 con 0x7f674c004290 2024-09-10T21:17:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.497+0000 7f676a800640 1 -- 172.21.15.16:0/3745390845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67640b1f80 con 0x7f674c004290 2024-09-10T21:17:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.497+0000 7f6761600640 1 -- 172.21.15.16:0/3745390845 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5235 (secure 0 0 0) 0x7f675805ef90 con 0x7f674c004290 2024-09-10T21:17:27.502 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:27.502 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"btime":"2024-09-10T21:17:21:843659+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3765085271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765085271},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765085271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":7104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4198210360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":4198210360},{"type":"v1","addr":"172.21.15.16:6837","nonce":4198210360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2954517211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2954517211},{"type":"v1","addr":"172.21.15.52:6835","nonce":2954517211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:05.094283+0000","modified":"2024-09-10T21:17:21.746067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135926":{"gid":135926,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":7,"addr":"172.21.15.52:6834/1108687782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108687782},{"type":"v1","addr":"172.21.15.52:6834","nonce":1108687782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135926,"qdb_cluster":[135926]},"id":8}]} 2024-09-10T21:17:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 73 2024-09-10T21:17:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.497+0000 7f676a800640 1 -- 172.21.15.16:0/3745390845 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6744072610 msgr2=0x7f6744074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.497+0000 7f676a800640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6744072610 0x7f6744074ad0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f67500ae000 tx=0x7f675001a010 comp rx=0 tx=0).stop 2024-09-10T21:17:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.497+0000 7f676a800640 1 -- 172.21.15.16:0/3745390845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c004290 msgr2=0x7f676415af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.497+0000 7f676a800640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c004290 0x7f676415af80 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f675800e5f0 tx=0x7f675800eab0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.497+0000 7f676a800640 1 -- 172.21.15.16:0/3745390845 shutdown_connections 2024-09-10T21:17:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.497+0000 7f676a800640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6744072610 0x7f6744074ad0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.497+0000 7f676a800640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67640cc610 0x7f67640cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.497+0000 7f676a800640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f67640b5540 0x7f67640b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.497+0000 7f676a800640 1 --2- 172.21.15.16:0/3745390845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674c004290 0x7f676415af80 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.497+0000 7f676a800640 1 -- 172.21.15.16:0/3745390845 >> 172.21.15.16:0/3745390845 conn(0x7f67640d1d00 msgr2=0x7f6764166710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.501+0000 7f676a800640 1 -- 172.21.15.16:0/3745390845 shutdown_connections 2024-09-10T21:17:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.501+0000 7f676a800640 1 -- 172.21.15.16:0/3745390845 wait complete. 2024-09-10T21:17:27.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.665+0000 7efe87000640 1 -- 172.21.15.16:0/993806560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68004da0 msgr2=0x7efe68005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.665+0000 7efe87000640 1 --2- 172.21.15.16:0/993806560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68004da0 0x7efe68005200 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7efe78009ee0 tx=0x7efe7801d5f0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.665+0000 7efe86600640 1 -- 172.21.15.16:0/993806560 reap_dead start 2024-09-10T21:17:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.665+0000 7efe87000640 1 -- 172.21.15.16:0/993806560 shutdown_connections 2024-09-10T21:17:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.665+0000 7efe87000640 1 -- 172.21.15.16:0/993806560 >> 172.21.15.16:0/993806560 conn(0x7efe800d1d00 msgr2=0x7efe800d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.665+0000 7efe87000640 1 -- 172.21.15.16:0/993806560 shutdown_connections 2024-09-10T21:17:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.665+0000 7efe87000640 1 -- 172.21.15.16:0/993806560 wait complete. 2024-09-10T21:17:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe87000640 1 Processor -- start 2024-09-10T21:17:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe87000640 1 -- start start 2024-09-10T21:17:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe87000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68004da0 0x7efe800c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe87000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe800b6df0 0x7efe801f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe87000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe8015c5e0 0x7efe800c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe87000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe800b2420 con 0x7efe68004da0 2024-09-10T21:17:27.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe87000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe800b22a0 con 0x7efe8015c5e0 2024-09-10T21:17:27.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe87000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7efe800b25a0 con 0x7efe800b6df0 2024-09-10T21:17:27.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe85c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68004da0 0x7efe800c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:27.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe85c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68004da0 0x7efe800c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55200/0 (socket says 172.21.15.16:55200) 2024-09-10T21:17:27.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe85c00640 1 -- 172.21.15.16:0/3081007004 learned_addr learned my addr 172.21.15.16:0/3081007004 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe86600640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe8015c5e0 0x7efe800c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe85200640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe800b6df0 0x7efe801f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe85c00640 1 -- 172.21.15.16:0/3081007004 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe800b6df0 msgr2=0x7efe801f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe85c00640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe800b6df0 0x7efe801f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe85c00640 1 -- 172.21.15.16:0/3081007004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe8015c5e0 msgr2=0x7efe800c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe85c00640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe8015c5e0 0x7efe800c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe85c00640 1 -- 172.21.15.16:0/3081007004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe800c58d0 con 0x7efe68004da0 2024-09-10T21:17:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe85200640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe800b6df0 0x7efe801f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe85c00640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68004da0 0x7efe800c6620 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7efe74004fd0 tx=0x7efe7400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe7d600640 1 -- 172.21.15.16:0/3081007004 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74015070 con 0x7efe68004da0 2024-09-10T21:17:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe7d600640 1 -- 172.21.15.16:0/3081007004 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe740109a0 con 0x7efe68004da0 2024-09-10T21:17:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe7d600640 1 -- 172.21.15.16:0/3081007004 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74010df0 con 0x7efe68004da0 2024-09-10T21:17:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe87000640 1 -- 172.21.15.16:0/3081007004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe800c5b40 con 0x7efe68004da0 2024-09-10T21:17:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe87000640 1 -- 172.21.15.16:0/3081007004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe800c6100 con 0x7efe68004da0 2024-09-10T21:17:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.669+0000 7efe87000640 1 -- 172.21.15.16:0/3081007004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe800d0120 con 0x7efe68004da0 2024-09-10T21:17:27.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.673+0000 7efe7d600640 1 -- 172.21.15.16:0/3081007004 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efe74011460 con 0x7efe68004da0 2024-09-10T21:17:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.673+0000 7efe7d600640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efe600725e0 0x7efe60074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.673+0000 7efe7d600640 1 -- 172.21.15.16:0/3081007004 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7efe74093920 con 0x7efe68004da0 2024-09-10T21:17:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.673+0000 7efe85200640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efe600725e0 0x7efe60074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.673+0000 7efe85200640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efe600725e0 0x7efe60074aa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00afd0 tx=0x7efe6c006960 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:27.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.677+0000 7efe7d600640 1 -- 172.21.15.16:0/3081007004 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efe7405ef20 con 0x7efe68004da0 2024-09-10T21:17:27.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.885+0000 7efe87000640 1 -- 172.21.15.16:0/3081007004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe800cf120 con 0x7efe68004da0 2024-09-10T21:17:27.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.885+0000 7efe7d600640 1 -- 172.21.15.16:0/3081007004 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5235 (secure 0 0 0) 0x7efe7405e820 con 0x7efe68004da0 2024-09-10T21:17:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:27.890 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"btime":"2024-09-10T21:17:21:843659+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3765085271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765085271},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765085271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":7104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4198210360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":4198210360},{"type":"v1","addr":"172.21.15.16:6837","nonce":4198210360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2954517211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2954517211},{"type":"v1","addr":"172.21.15.52:6835","nonce":2954517211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:05.094283+0000","modified":"2024-09-10T21:17:21.746067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":135926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_135926":{"gid":135926,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":7,"addr":"172.21.15.52:6834/1108687782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108687782},{"type":"v1","addr":"172.21.15.52:6834","nonce":1108687782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":135926,"qdb_cluster":[135926]},"id":8}]} 2024-09-10T21:17:27.890 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 73 2024-09-10T21:17:27.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.885+0000 7efe87000640 1 -- 172.21.15.16:0/3081007004 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efe600725e0 msgr2=0x7efe60074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.885+0000 7efe87000640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efe600725e0 0x7efe60074aa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00afd0 tx=0x7efe6c006960 comp rx=0 tx=0).stop 2024-09-10T21:17:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.885+0000 7efe87000640 1 -- 172.21.15.16:0/3081007004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68004da0 msgr2=0x7efe800c6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.885+0000 7efe87000640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68004da0 0x7efe800c6620 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7efe74004fd0 tx=0x7efe7400ea20 comp rx=0 tx=0).stop 2024-09-10T21:17:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.885+0000 7efe87000640 1 -- 172.21.15.16:0/3081007004 shutdown_connections 2024-09-10T21:17:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.885+0000 7efe87000640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efe600725e0 0x7efe60074aa0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.885+0000 7efe87000640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe8015c5e0 0x7efe800c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.889+0000 7efe87000640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efe800b6df0 0x7efe801f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.889+0000 7efe87000640 1 --2- 172.21.15.16:0/3081007004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68004da0 0x7efe800c6620 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.889+0000 7efe87000640 1 -- 172.21.15.16:0/3081007004 >> 172.21.15.16:0/3081007004 conn(0x7efe800d1d00 msgr2=0x7efe80071740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.889+0000 7efe87000640 1 -- 172.21.15.16:0/3081007004 shutdown_connections 2024-09-10T21:17:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:27.889+0000 7efe87000640 1 -- 172.21.15.16:0/3081007004 wait complete. 2024-09-10T21:17:27.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:17:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.053+0000 7f6705a00640 1 -- 172.21.15.16:0/4153076048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67000b0590 msgr2=0x7f6700071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.053+0000 7f6705a00640 1 --2- 172.21.15.16:0/4153076048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67000b0590 0x7f6700071520 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009bc0 tx=0x7f66f801caa0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.053+0000 7f6705a00640 1 -- 172.21.15.16:0/4153076048 shutdown_connections 2024-09-10T21:17:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.053+0000 7f6705a00640 1 --2- 172.21.15.16:0/4153076048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67000b0590 0x7f6700071520 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.053+0000 7f6705a00640 1 --2- 172.21.15.16:0/4153076048 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f670015f400 0x7f6700070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.053+0000 7f6705a00640 1 -- 172.21.15.16:0/4153076048 >> 172.21.15.16:0/4153076048 conn(0x7f670009b130 msgr2=0x7f67000d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.053+0000 7f6705a00640 1 -- 172.21.15.16:0/4153076048 shutdown_connections 2024-09-10T21:17:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.053+0000 7f6705a00640 1 -- 172.21.15.16:0/4153076048 wait complete. 2024-09-10T21:17:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f6705a00640 1 Processor -- start 2024-09-10T21:17:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f6705a00640 1 -- start start 2024-09-10T21:17:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f6705a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67000b0590 0x7f67001570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f6705a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67000b13c0 0x7f670015d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f6705a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f670015f400 0x7f670015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f6705a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6700168f60 con 0x7f67000b0590 2024-09-10T21:17:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f6705a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6700168de0 con 0x7f67000b13c0 2024-09-10T21:17:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f6705a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f67001690e0 con 0x7f670015f400 2024-09-10T21:17:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f66fea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67000b0590 0x7f67001570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f66fea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67000b0590 0x7f67001570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55220/0 (socket says 172.21.15.16:55220) 2024-09-10T21:17:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f66fea00640 1 -- 172.21.15.16:0/129446087 learned_addr learned my addr 172.21.15.16:0/129446087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f66fea00640 1 -- 172.21.15.16:0/129446087 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f670015f400 msgr2=0x7f670015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f66fea00640 1 --2- 172.21.15.16:0/129446087 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f670015f400 0x7f670015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f66fea00640 1 -- 172.21.15.16:0/129446087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67000b13c0 msgr2=0x7f670015d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f66fea00640 1 --2- 172.21.15.16:0/129446087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67000b13c0 0x7f670015d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f66fea00640 1 -- 172.21.15.16:0/129446087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67001575e0 con 0x7f67000b0590 2024-09-10T21:17:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f66fea00640 1 --2- 172.21.15.16:0/129446087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67000b0590 0x7f67001570a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f66f000d700 tx=0x7f66f000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f66f7400640 1 -- 172.21.15.16:0/129446087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0017070 con 0x7f67000b0590 2024-09-10T21:17:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f66f7400640 1 -- 172.21.15.16:0/129446087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66f0009d20 con 0x7f67000b0590 2024-09-10T21:17:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f6705a00640 1 -- 172.21.15.16:0/129446087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670015c360 con 0x7f67000b0590 2024-09-10T21:17:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f66f7400640 1 -- 172.21.15.16:0/129446087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f00106c0 con 0x7f67000b0590 2024-09-10T21:17:28.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.057+0000 7f6705a00640 1 -- 172.21.15.16:0/129446087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6700158f50 con 0x7f67000b0590 2024-09-10T21:17:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.061+0000 7f6705a00640 1 -- 172.21.15.16:0/129446087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67000cfc80 con 0x7f67000b0590 2024-09-10T21:17:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.061+0000 7f66f7400640 1 -- 172.21.15.16:0/129446087 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f66f0010860 con 0x7f67000b0590 2024-09-10T21:17:28.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.061+0000 7f66f7400640 1 --2- 172.21.15.16:0/129446087 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66dc072610 0x7f66dc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:28.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.061+0000 7f66f7400640 1 -- 172.21.15.16:0/129446087 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f66f0093750 con 0x7f67000b0590 2024-09-10T21:17:28.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.061+0000 7f66f6000640 1 --2- 172.21.15.16:0/129446087 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66dc072610 0x7f66dc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:28.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.061+0000 7f66f6000640 1 --2- 172.21.15.16:0/129446087 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66dc072610 0x7f66dc074ad0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f66e8007fd0 tx=0x7f66e80078e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.065+0000 7f66f7400640 1 -- 172.21.15.16:0/129446087 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f66f005ed50 con 0x7f67000b0590 2024-09-10T21:17:28.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.241+0000 7f6705a00640 1 -- 172.21.15.16:0/129446087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f670006de00 con 0x7f67000b0590 2024-09-10T21:17:28.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.241+0000 7f66f7400640 1 -- 172.21.15.16:0/129446087 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16781 (secure 0 0 0) 0x7f66f005e650 con 0x7f67000b0590 2024-09-10T21:17:28.245 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:28.245 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":105,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:17:11.735725+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:17:02.802106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T21:17:04.057780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":4,"to":3}]},{"pgid":"17.37","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:17:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.241+0000 7f6705a00640 1 -- 172.21.15.16:0/129446087 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66dc072610 msgr2=0x7f66dc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.241+0000 7f6705a00640 1 --2- 172.21.15.16:0/129446087 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66dc072610 0x7f66dc074ad0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f66e8007fd0 tx=0x7f66e80078e0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.241+0000 7f6705a00640 1 -- 172.21.15.16:0/129446087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67000b0590 msgr2=0x7f67001570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.241+0000 7f6705a00640 1 --2- 172.21.15.16:0/129446087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67000b0590 0x7f67001570a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f66f000d700 tx=0x7f66f000dbc0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.241+0000 7f6705a00640 1 -- 172.21.15.16:0/129446087 shutdown_connections 2024-09-10T21:17:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.241+0000 7f6705a00640 1 --2- 172.21.15.16:0/129446087 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66dc072610 0x7f66dc074ad0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.241+0000 7f6705a00640 1 --2- 172.21.15.16:0/129446087 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f670015f400 0x7f670015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.241+0000 7f6705a00640 1 --2- 172.21.15.16:0/129446087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67000b13c0 0x7f670015d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.241+0000 7f6705a00640 1 --2- 172.21.15.16:0/129446087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67000b0590 0x7f67001570a0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.241+0000 7f6705a00640 1 -- 172.21.15.16:0/129446087 >> 172.21.15.16:0/129446087 conn(0x7f670009b130 msgr2=0x7f67000b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.245+0000 7f6705a00640 1 -- 172.21.15.16:0/129446087 shutdown_connections 2024-09-10T21:17:28.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.245+0000 7f6705a00640 1 -- 172.21.15.16:0/129446087 wait complete. 2024-09-10T21:17:28.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:17:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.409+0000 7f8ce6600640 1 -- 172.21.15.16:0/49927304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cc4003ab0 msgr2=0x7f8cc4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.409+0000 7f8ce6600640 1 --2- 172.21.15.16:0/49927304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cc4003ab0 0x7f8cc4003ed0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009b60 tx=0x7f8cd001d270 comp rx=0 tx=0).stop 2024-09-10T21:17:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 -- 172.21.15.16:0/49927304 shutdown_connections 2024-09-10T21:17:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 --2- 172.21.15.16:0/49927304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cc4003ab0 0x7f8cc4003ed0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 -- 172.21.15.16:0/49927304 >> 172.21.15.16:0/49927304 conn(0x7f8ce00d03a0 msgr2=0x7f8ce00d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 -- 172.21.15.16:0/49927304 shutdown_connections 2024-09-10T21:17:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 -- 172.21.15.16:0/49927304 wait complete. 2024-09-10T21:17:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 Processor -- start 2024-09-10T21:17:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 -- start start 2024-09-10T21:17:28.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cc4003ab0 0x7f8ce0099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce0154400 0x7f8ce01f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8ce01f49a0 0x7f8ce01e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce00b02b0 con 0x7f8ce0154400 2024-09-10T21:17:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce00b0130 con 0x7f8cc4003ab0 2024-09-10T21:17:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce00b0430 con 0x7f8ce01f49a0 2024-09-10T21:17:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cd7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce0154400 0x7f8ce01f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cd7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce0154400 0x7f8ce01f4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55244/0 (socket says 172.21.15.16:55244) 2024-09-10T21:17:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cd7e00640 1 -- 172.21.15.16:0/2344230612 learned_addr learned my addr 172.21.15.16:0/2344230612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cdf400640 1 --2- 172.21.15.16:0/2344230612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cc4003ab0 0x7f8ce0099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cdfe00640 1 --2- 172.21.15.16:0/2344230612 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8ce01f49a0 0x7f8ce01e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cd7e00640 1 -- 172.21.15.16:0/2344230612 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8ce01f49a0 msgr2=0x7f8ce01e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cd7e00640 1 --2- 172.21.15.16:0/2344230612 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8ce01f49a0 0x7f8ce01e7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cd7e00640 1 -- 172.21.15.16:0/2344230612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cc4003ab0 msgr2=0x7f8ce0099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cd7e00640 1 --2- 172.21.15.16:0/2344230612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cc4003ab0 0x7f8ce0099a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cd7e00640 1 -- 172.21.15.16:0/2344230612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce015c0f0 con 0x7f8ce0154400 2024-09-10T21:17:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cd7e00640 1 --2- 172.21.15.16:0/2344230612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce0154400 0x7f8ce01f4460 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8001250 tx=0x7f8cc800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cd7400640 1 -- 172.21.15.16:0/2344230612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8004440 con 0x7f8ce0154400 2024-09-10T21:17:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cd7400640 1 -- 172.21.15.16:0/2344230612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cc8016070 con 0x7f8ce0154400 2024-09-10T21:17:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8cd7400640 1 -- 172.21.15.16:0/2344230612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800ca20 con 0x7f8ce0154400 2024-09-10T21:17:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.413+0000 7f8ce6600640 1 -- 172.21.15.16:0/2344230612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce00b3350 con 0x7f8ce0154400 2024-09-10T21:17:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.417+0000 7f8ce6600640 1 -- 172.21.15.16:0/2344230612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce00b3610 con 0x7f8ce0154400 2024-09-10T21:17:28.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.417+0000 7f8ce6600640 1 -- 172.21.15.16:0/2344230612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce00b4ba0 con 0x7f8ce0154400 2024-09-10T21:17:28.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.417+0000 7f8cd7400640 1 -- 172.21.15.16:0/2344230612 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8cc800cbc0 con 0x7f8ce0154400 2024-09-10T21:17:28.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.417+0000 7f8cd7400640 1 --2- 172.21.15.16:0/2344230612 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8cbc072610 0x7f8cbc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:28.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.417+0000 7f8cd7400640 1 -- 172.21.15.16:0/2344230612 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7f8cc80936c0 con 0x7f8ce0154400 2024-09-10T21:17:28.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.417+0000 7f8cdf400640 1 --2- 172.21.15.16:0/2344230612 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8cbc072610 0x7f8cbc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:28.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.421+0000 7f8cdf400640 1 --2- 172.21.15.16:0/2344230612 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8cbc072610 0x7f8cbc074ad0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00ae000 tx=0x7f8cd001a010 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:28.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.421+0000 7f8cd7400640 1 -- 172.21.15.16:0/2344230612 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8cc805e700 con 0x7f8ce0154400 2024-09-10T21:17:28.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.633+0000 7f8ce6600640 1 -- 172.21.15.16:0/2344230612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8ce00b0990 con 0x7f8ce0154400 2024-09-10T21:17:28.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.633+0000 7f8cd7400640 1 -- 172.21.15.16:0/2344230612 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v73) ==== 316+0+0 (secure 0 0 0) 0x7f8cc8063130 con 0x7f8ce0154400 2024-09-10T21:17:28.637 INFO:teuthology.orchestra.run.smithi016.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-09-10T21:17:28.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:17:28.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-09-10T21:17:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.793+0000 7fe334200640 1 -- 172.21.15.16:0/1320971783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32c0ce040 msgr2=0x7fe32c0b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.793+0000 7fe334200640 1 --2- 172.21.15.16:0/1320971783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32c0ce040 0x7fe32c0b1710 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe324009c80 tx=0x7fe32401f5a0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.793+0000 7fe334200640 1 -- 172.21.15.16:0/1320971783 shutdown_connections 2024-09-10T21:17:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.793+0000 7fe334200640 1 --2- 172.21.15.16:0/1320971783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32c0ce040 0x7fe32c0b1710 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.793+0000 7fe334200640 1 --2- 172.21.15.16:0/1320971783 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe32c0d0210 0x7fe32c0ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.793+0000 7fe334200640 1 -- 172.21.15.16:0/1320971783 >> 172.21.15.16:0/1320971783 conn(0x7fe32c0d1df0 msgr2=0x7fe32c0d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.793+0000 7fe334200640 1 -- 172.21.15.16:0/1320971783 shutdown_connections 2024-09-10T21:17:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.793+0000 7fe334200640 1 -- 172.21.15.16:0/1320971783 wait complete. 2024-09-10T21:17:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.793+0000 7fe334200640 1 Processor -- start 2024-09-10T21:17:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.793+0000 7fe334200640 1 -- start start 2024-09-10T21:17:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe334200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32c0ce040 0x7fe32c1f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe334200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe32c0cfe30 0x7fe32c0c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe334200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe32c0d0210 0x7fe32c0c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe334200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c0d2ad0 con 0x7fe32c0ce040 2024-09-10T21:17:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe334200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c0b14f0 con 0x7fe32c0cfe30 2024-09-10T21:17:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe334200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe32c0cf1e0 con 0x7fe32c0d0210 2024-09-10T21:17:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe331800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32c0ce040 0x7fe32c1f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe331800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32c0ce040 0x7fe32c1f2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55264/0 (socket says 172.21.15.16:55264) 2024-09-10T21:17:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe331800640 1 -- 172.21.15.16:0/3141345551 learned_addr learned my addr 172.21.15.16:0/3141345551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe330e00640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe32c0cfe30 0x7fe32c0c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe332200640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe32c0d0210 0x7fe32c0c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe331800640 1 -- 172.21.15.16:0/3141345551 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe32c0d0210 msgr2=0x7fe32c0c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe331800640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe32c0d0210 0x7fe32c0c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe331800640 1 -- 172.21.15.16:0/3141345551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe32c0cfe30 msgr2=0x7fe32c0c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe331800640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe32c0cfe30 0x7fe32c0c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe331800640 1 -- 172.21.15.16:0/3141345551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32c0b5f60 con 0x7fe32c0ce040 2024-09-10T21:17:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe331800640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32c0ce040 0x7fe32c1f2400 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe32000c500 tx=0x7fe32000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe329600640 1 -- 172.21.15.16:0/3141345551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320017070 con 0x7fe32c0ce040 2024-09-10T21:17:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe329600640 1 -- 172.21.15.16:0/3141345551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3200053c0 con 0x7fe32c0ce040 2024-09-10T21:17:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe329600640 1 -- 172.21.15.16:0/3141345551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320012890 con 0x7fe32c0ce040 2024-09-10T21:17:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe334200640 1 -- 172.21.15.16:0/3141345551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32c0b6240 con 0x7fe32c0ce040 2024-09-10T21:17:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe334200640 1 -- 172.21.15.16:0/3141345551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe32c0b6800 con 0x7fe32c0ce040 2024-09-10T21:17:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe334200640 1 -- 172.21.15.16:0/3141345551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe32c0b1710 con 0x7fe32c0ce040 2024-09-10T21:17:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.797+0000 7fe329600640 1 -- 172.21.15.16:0/3141345551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe320003600 con 0x7fe32c0ce040 2024-09-10T21:17:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.801+0000 7fe329600640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe30c0725e0 0x7fe30c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.801+0000 7fe329600640 1 -- 172.21.15.16:0/3141345551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5767+0+0 (secure 0 0 0) 0x7fe3200928f0 con 0x7fe32c0ce040 2024-09-10T21:17:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.801+0000 7fe330e00640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe30c0725e0 0x7fe30c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.801+0000 7fe330e00640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe30c0725e0 0x7fe30c074aa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe3180058d0 tx=0x7fe318005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:28.805+0000 7fe329600640 1 -- 172.21.15.16:0/3141345551 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe32005df00 con 0x7fe32c0ce040 2024-09-10T21:17:29.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.013+0000 7fe334200640 1 -- 172.21.15.16:0/3141345551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe32c0b69d0 con 0x7fe32c0ce040 2024-09-10T21:17:29.019 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-10T21:17:29.013+0000 7ff429400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:17:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.017+0000 7fe329600640 1 -- 172.21.15.16:0/3141345551 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) ==== 191+0+0 (secure 0 0 0) 0x7fe32005d800 con 0x7fe32c0ce040 2024-09-10T21:17:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:17:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.021+0000 7fe334200640 1 -- 172.21.15.16:0/3141345551 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe30c0725e0 msgr2=0x7fe30c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.021+0000 7fe334200640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe30c0725e0 0x7fe30c074aa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe3180058d0 tx=0x7fe318005860 comp rx=0 tx=0).stop 2024-09-10T21:17:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.021+0000 7fe334200640 1 -- 172.21.15.16:0/3141345551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32c0ce040 msgr2=0x7fe32c1f2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.021+0000 7fe334200640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32c0ce040 0x7fe32c1f2400 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe32000c500 tx=0x7fe32000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:17:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.021+0000 7fe334200640 1 -- 172.21.15.16:0/3141345551 shutdown_connections 2024-09-10T21:17:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.021+0000 7fe334200640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe30c0725e0 0x7fe30c074aa0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.021+0000 7fe334200640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe32c0d0210 0x7fe32c0c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.021+0000 7fe334200640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe32c0cfe30 0x7fe32c0c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.021+0000 7fe334200640 1 --2- 172.21.15.16:0/3141345551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32c0ce040 0x7fe32c1f2400 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.021+0000 7fe334200640 1 -- 172.21.15.16:0/3141345551 >> 172.21.15.16:0/3141345551 conn(0x7fe32c0d1df0 msgr2=0x7fe32c0b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.021+0000 7fe334200640 1 -- 172.21.15.16:0/3141345551 shutdown_connections 2024-09-10T21:17:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.021+0000 7fe334200640 1 -- 172.21.15.16:0/3141345551 wait complete. 2024-09-10T21:17:29.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:17:29.097 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:17:29.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.189+0000 7ff6d9600640 1 -- 172.21.15.16:0/3344208151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d415ad80 msgr2=0x7ff6d40b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:29.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.189+0000 7ff6d9600640 1 --2- 172.21.15.16:0/3344208151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d415ad80 0x7ff6d40b33b0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009900 tx=0x7ff6bc01c820 comp rx=0 tx=0).stop 2024-09-10T21:17:29.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.189+0000 7ff6d9600640 1 -- 172.21.15.16:0/3344208151 shutdown_connections 2024-09-10T21:17:29.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.189+0000 7ff6d9600640 1 --2- 172.21.15.16:0/3344208151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d415ad80 0x7ff6d40b33b0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:29.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.189+0000 7ff6d9600640 1 --2- 172.21.15.16:0/3344208151 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6d415a9a0 0x7ff6d40b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:29.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.189+0000 7ff6d9600640 1 -- 172.21.15.16:0/3344208151 >> 172.21.15.16:0/3344208151 conn(0x7ff6d40b16a0 msgr2=0x7ff6d40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:29.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.189+0000 7ff6d9600640 1 -- 172.21.15.16:0/3344208151 shutdown_connections 2024-09-10T21:17:29.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.189+0000 7ff6d9600640 1 -- 172.21.15.16:0/3344208151 wait complete. 2024-09-10T21:17:29.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.189+0000 7ff6d9600640 1 Processor -- start 2024-09-10T21:17:29.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.189+0000 7ff6d9600640 1 -- start start 2024-09-10T21:17:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d415a9a0 0x7ff6d4147170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d9600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6d415ad80 0x7ff6d41f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d9600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6d415eff0 0x7ff6d41506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d9600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d4167b00 con 0x7ff6d415a9a0 2024-09-10T21:17:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d9600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d4167980 con 0x7ff6d415eff0 2024-09-10T21:17:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d9600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d4167c80 con 0x7ff6d415ad80 2024-09-10T21:17:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d415a9a0 0x7ff6d4147170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d415a9a0 0x7ff6d4147170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55290/0 (socket says 172.21.15.16:55290) 2024-09-10T21:17:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d2a00640 1 -- 172.21.15.16:0/2484199133 learned_addr learned my addr 172.21.15.16:0/2484199133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6ca000640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6d415ad80 0x7ff6d41f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d3400640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6d415eff0 0x7ff6d41506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d2a00640 1 -- 172.21.15.16:0/2484199133 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6d415ad80 msgr2=0x7ff6d41f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d2a00640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6d415ad80 0x7ff6d41f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d2a00640 1 -- 172.21.15.16:0/2484199133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6d415eff0 msgr2=0x7ff6d41506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d2a00640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6d415eff0 0x7ff6d41506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d2a00640 1 -- 172.21.15.16:0/2484199133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d406fb80 con 0x7ff6d415a9a0 2024-09-10T21:17:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d3400640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6d415eff0 0x7ff6d41506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d2a00640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d415a9a0 0x7ff6d4147170 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400d980 tx=0x7ff6c400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6cb400640 1 -- 172.21.15.16:0/2484199133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4017070 con 0x7ff6d415a9a0 2024-09-10T21:17:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6cb400640 1 -- 172.21.15.16:0/2484199133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6c4009d60 con 0x7ff6d415a9a0 2024-09-10T21:17:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6cb400640 1 -- 172.21.15.16:0/2484199133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4005280 con 0x7ff6d415a9a0 2024-09-10T21:17:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d9600640 1 -- 172.21.15.16:0/2484199133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d4147c10 con 0x7ff6d415a9a0 2024-09-10T21:17:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d9600640 1 -- 172.21.15.16:0/2484199133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d41f5e50 con 0x7ff6d415a9a0 2024-09-10T21:17:29.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6d9600640 1 -- 172.21.15.16:0/2484199133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d40b33b0 con 0x7ff6d415a9a0 2024-09-10T21:17:29.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.193+0000 7ff6cb400640 1 -- 172.21.15.16:0/2484199133 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff6c4005420 con 0x7ff6d415a9a0 2024-09-10T21:17:29.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.197+0000 7ff6cb400640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6b00725e0 0x7ff6b0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:29.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.197+0000 7ff6cb400640 1 -- 172.21.15.16:0/2484199133 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5853+0+0 (secure 0 0 0) 0x7ff6c4093510 con 0x7ff6d415a9a0 2024-09-10T21:17:29.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.197+0000 7ff6ca000640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6b00725e0 0x7ff6b0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.197+0000 7ff6ca000640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6b00725e0 0x7ff6b0074aa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0af000 tx=0x7ff6bc01b060 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.201+0000 7ff6cb400640 1 -- 172.21.15.16:0/2484199133 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff6c405e470 con 0x7ff6d415a9a0 2024-09-10T21:17:29.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:29.413+0000 7ff6d9600640 1 -- 172.21.15.16:0/2484199133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff6d415e560 con 0x7ff6d415a9a0 2024-09-10T21:17:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.021+0000 7ff6cb400640 1 -- 172.21.15.16:0/2484199133 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) ==== 104+0+0 (secure 0 0 0) 0x7ff6c4062ea0 con 0x7ff6d415a9a0 2024-09-10T21:17:30.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.025+0000 7ff6d9600640 1 -- 172.21.15.16:0/2484199133 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6b00725e0 msgr2=0x7ff6b0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:30.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.025+0000 7ff6d9600640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6b00725e0 0x7ff6b0074aa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0af000 tx=0x7ff6bc01b060 comp rx=0 tx=0).stop 2024-09-10T21:17:30.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.025+0000 7ff6d9600640 1 -- 172.21.15.16:0/2484199133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d415a9a0 msgr2=0x7ff6d4147170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:30.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.025+0000 7ff6d9600640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d415a9a0 0x7ff6d4147170 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400d980 tx=0x7ff6c400de40 comp rx=0 tx=0).stop 2024-09-10T21:17:30.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.025+0000 7ff6d9600640 1 -- 172.21.15.16:0/2484199133 shutdown_connections 2024-09-10T21:17:30.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.025+0000 7ff6d9600640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6b00725e0 0x7ff6b0074aa0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:30.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.025+0000 7ff6d9600640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6d415eff0 0x7ff6d41506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:30.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.025+0000 7ff6d9600640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6d415ad80 0x7ff6d41f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:30.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.025+0000 7ff6d9600640 1 --2- 172.21.15.16:0/2484199133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d415a9a0 0x7ff6d4147170 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:30.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.025+0000 7ff6d9600640 1 -- 172.21.15.16:0/2484199133 >> 172.21.15.16:0/2484199133 conn(0x7ff6d40b16a0 msgr2=0x7ff6d40b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:30.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.025+0000 7ff6d9600640 1 -- 172.21.15.16:0/2484199133 shutdown_connections 2024-09-10T21:17:30.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.025+0000 7ff6d9600640 1 -- 172.21.15.16:0/2484199133 wait complete. 2024-09-10T21:17:30.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:17:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.201+0000 7f4829c00640 1 -- 172.21.15.16:0/209369451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482415f4b0 msgr2=0x7f48240cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.201+0000 7f4829c00640 1 --2- 172.21.15.16:0/209369451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482415f4b0 0x7f48240cfce0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4814009840 tx=0x7f481401c760 comp rx=0 tx=0).stop 2024-09-10T21:17:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.201+0000 7f4829c00640 1 -- 172.21.15.16:0/209369451 shutdown_connections 2024-09-10T21:17:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.201+0000 7f4829c00640 1 --2- 172.21.15.16:0/209369451 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48240d2900 0x7f48240d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.201+0000 7f4829c00640 1 --2- 172.21.15.16:0/209369451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482415f4b0 0x7f48240cfce0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.201+0000 7f4829c00640 1 -- 172.21.15.16:0/209369451 >> 172.21.15.16:0/209369451 conn(0x7f48240d1de0 msgr2=0x7f48240d21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.201+0000 7f4829c00640 1 -- 172.21.15.16:0/209369451 shutdown_connections 2024-09-10T21:17:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.201+0000 7f4829c00640 1 -- 172.21.15.16:0/209369451 wait complete. 2024-09-10T21:17:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4829c00640 1 Processor -- start 2024-09-10T21:17:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4829c00640 1 -- start start 2024-09-10T21:17:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4829c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48240d0760 0x7f482414cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4829c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48240d2900 0x7f482406ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4829c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482415f4b0 0x7f482415b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4829c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48241692e0 con 0x7f48240d0760 2024-09-10T21:17:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4829c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4824169160 con 0x7f482415f4b0 2024-09-10T21:17:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4829c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4824169460 con 0x7f48240d2900 2024-09-10T21:17:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4822a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48240d0760 0x7f482414cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4822a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48240d0760 0x7f482414cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55298/0 (socket says 172.21.15.16:55298) 2024-09-10T21:17:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4822a00640 1 -- 172.21.15.16:0/4133044887 learned_addr learned my addr 172.21.15.16:0/4133044887 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f481be00640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48240d2900 0x7f482406ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4823400640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482415f4b0 0x7f482415b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4822a00640 1 -- 172.21.15.16:0/4133044887 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48240d2900 msgr2=0x7f482406ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4822a00640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48240d2900 0x7f482406ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4822a00640 1 -- 172.21.15.16:0/4133044887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482415f4b0 msgr2=0x7f482415b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4822a00640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482415f4b0 0x7f482415b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4822a00640 1 -- 172.21.15.16:0/4133044887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48240b1770 con 0x7f48240d0760 2024-09-10T21:17:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4822a00640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48240d0760 0x7f482414cdc0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f4814009790 tx=0x7f4814005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f481aa00640 1 -- 172.21.15.16:0/4133044887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48140b2040 con 0x7f48240d0760 2024-09-10T21:17:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f481aa00640 1 -- 172.21.15.16:0/4133044887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48140043e0 con 0x7f48240d0760 2024-09-10T21:17:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f481aa00640 1 -- 172.21.15.16:0/4133044887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48140052d0 con 0x7f48240d0760 2024-09-10T21:17:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4829c00640 1 -- 172.21.15.16:0/4133044887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482414d450 con 0x7f48240d0760 2024-09-10T21:17:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4829c00640 1 -- 172.21.15.16:0/4133044887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f482414c850 con 0x7f48240d0760 2024-09-10T21:17:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.205+0000 7f4829c00640 1 -- 172.21.15.16:0/4133044887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48240b13a0 con 0x7f48240d0760 2024-09-10T21:17:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.209+0000 7f481aa00640 1 -- 172.21.15.16:0/4133044887 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f48140b5070 con 0x7f48240d0760 2024-09-10T21:17:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.209+0000 7f481aa00640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f48000725e0 0x7f4800074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.209+0000 7f481aa00640 1 -- 172.21.15.16:0/4133044887 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5853+0+0 (secure 0 0 0) 0x7f481412cc30 con 0x7f48240d0760 2024-09-10T21:17:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.209+0000 7f481be00640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f48000725e0 0x7f4800074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.209+0000 7f481be00640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f48000725e0 0x7f4800074aa0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f480c009600 tx=0x7f480c0091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:30.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.213+0000 7f481aa00640 1 -- 172.21.15.16:0/4133044887 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f48140bd060 con 0x7f48240d0760 2024-09-10T21:17:30.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:30.389+0000 7f4829c00640 1 -- 172.21.15.16:0/4133044887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4824150760 con 0x7f48240d0760 2024-09-10T21:17:31.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.025+0000 7f481aa00640 1 -- 172.21.15.16:0/4133044887 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) ==== 181+0+0 (secure 0 0 0) 0x7f4814016410 con 0x7f48240d0760 2024-09-10T21:17:31.029 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:17:31.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.029+0000 7f4829c00640 1 -- 172.21.15.16:0/4133044887 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f48000725e0 msgr2=0x7f4800074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:31.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.029+0000 7f4829c00640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f48000725e0 0x7f4800074aa0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f480c009600 tx=0x7f480c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:17:31.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.029+0000 7f4829c00640 1 -- 172.21.15.16:0/4133044887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48240d0760 msgr2=0x7f482414cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:31.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.029+0000 7f4829c00640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48240d0760 0x7f482414cdc0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f4814009790 tx=0x7f4814005db0 comp rx=0 tx=0).stop 2024-09-10T21:17:31.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.029+0000 7f4829c00640 1 -- 172.21.15.16:0/4133044887 shutdown_connections 2024-09-10T21:17:31.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.029+0000 7f4829c00640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f48000725e0 0x7f4800074aa0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:31.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.029+0000 7f4829c00640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f482415f4b0 0x7f482415b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:31.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.029+0000 7f4829c00640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f48240d2900 0x7f482406ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:31.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.029+0000 7f4829c00640 1 --2- 172.21.15.16:0/4133044887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48240d0760 0x7f482414cdc0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:31.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.029+0000 7f4829c00640 1 -- 172.21.15.16:0/4133044887 >> 172.21.15.16:0/4133044887 conn(0x7f48240d1de0 msgr2=0x7f482415d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:31.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.029+0000 7f4829c00640 1 -- 172.21.15.16:0/4133044887 shutdown_connections 2024-09-10T21:17:31.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.029+0000 7f4829c00640 1 -- 172.21.15.16:0/4133044887 wait complete. 2024-09-10T21:17:31.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:17:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.205+0000 7ff6f1600640 1 -- 172.21.15.16:0/2019462001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec0b3920 msgr2=0x7ff6ec161f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.205+0000 7ff6f1600640 1 --2- 172.21.15.16:0/2019462001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec0b3920 0x7ff6ec161f00 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0b3100 tx=0x7ff6dc01c740 comp rx=0 tx=0).stop 2024-09-10T21:17:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.205+0000 7ff6f1600640 1 -- 172.21.15.16:0/2019462001 shutdown_connections 2024-09-10T21:17:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.205+0000 7ff6f1600640 1 --2- 172.21.15.16:0/2019462001 >> v2:172.21.15.52:3300/0 conn(0x7ff6ec0b4bb0 0x7ff6ec1681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.205+0000 7ff6f1600640 1 --2- 172.21.15.16:0/2019462001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec0b3920 0x7ff6ec161f00 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.205+0000 7ff6f1600640 1 -- 172.21.15.16:0/2019462001 >> 172.21.15.16:0/2019462001 conn(0x7ff6ec0d3e10 msgr2=0x7ff6ec0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.205+0000 7ff6f1600640 1 -- 172.21.15.16:0/2019462001 shutdown_connections 2024-09-10T21:17:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.205+0000 7ff6f1600640 1 -- 172.21.15.16:0/2019462001 wait complete. 2024-09-10T21:17:31.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.205+0000 7ff6f1600640 1 Processor -- start 2024-09-10T21:17:31.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.205+0000 7ff6f1600640 1 -- start start 2024-09-10T21:17:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f1600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6ec0b3920 0x7ff6ec0b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f1600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6ec0b4bb0 0x7ff6ec14f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec0d1bf0 0x7ff6ec14ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f1600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec16b0a0 con 0x7ff6ec0d1bf0 2024-09-10T21:17:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f1600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec16af20 con 0x7ff6ec0b3920 2024-09-10T21:17:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f1600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ec16b220 con 0x7ff6ec0b4bb0 2024-09-10T21:17:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec0d1bf0 0x7ff6ec14ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec0d1bf0 0x7ff6ec14ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55320/0 (socket says 172.21.15.16:55320) 2024-09-10T21:17:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f0c00640 1 -- 172.21.15.16:0/1416084108 learned_addr learned my addr 172.21.15.16:0/1416084108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:31.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6ebe00640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6ec0b3920 0x7ff6ec0b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:31.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6eb400640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6ec0b4bb0 0x7ff6ec14f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:31.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f0c00640 1 -- 172.21.15.16:0/1416084108 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6ec0b4bb0 msgr2=0x7ff6ec14f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:31.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f0c00640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6ec0b4bb0 0x7ff6ec14f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:31.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f0c00640 1 -- 172.21.15.16:0/1416084108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6ec0b3920 msgr2=0x7ff6ec0b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:31.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f0c00640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6ec0b3920 0x7ff6ec0b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:31.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f0c00640 1 -- 172.21.15.16:0/1416084108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ec1fb4c0 con 0x7ff6ec0d1bf0 2024-09-10T21:17:31.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f0c00640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec0d1bf0 0x7ff6ec14ffb0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400ea50 tx=0x7ff6e400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:31.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6e3e00640 1 -- 172.21.15.16:0/1416084108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400cd60 con 0x7ff6ec0d1bf0 2024-09-10T21:17:31.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6e3e00640 1 -- 172.21.15.16:0/1416084108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6e4004b30 con 0x7ff6ec0d1bf0 2024-09-10T21:17:31.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6e3e00640 1 -- 172.21.15.16:0/1416084108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4005020 con 0x7ff6ec0d1bf0 2024-09-10T21:17:31.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f1600640 1 -- 172.21.15.16:0/1416084108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ec1fb7a0 con 0x7ff6ec0d1bf0 2024-09-10T21:17:31.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f1600640 1 -- 172.21.15.16:0/1416084108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ec1fbd60 con 0x7ff6ec0d1bf0 2024-09-10T21:17:31.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6f1600640 1 -- 172.21.15.16:0/1416084108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6ec161f00 con 0x7ff6ec0d1bf0 2024-09-10T21:17:31.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.209+0000 7ff6e3e00640 1 -- 172.21.15.16:0/1416084108 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff6e400f040 con 0x7ff6ec0d1bf0 2024-09-10T21:17:31.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.213+0000 7ff6e3e00640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6cc0725e0 0x7ff6cc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:31.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.213+0000 7ff6e3e00640 1 -- 172.21.15.16:0/1416084108 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5386+0+0 (secure 0 0 0) 0x7ff6e4093730 con 0x7ff6ec0d1bf0 2024-09-10T21:17:31.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.213+0000 7ff6ebe00640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6cc0725e0 0x7ff6cc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:31.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.213+0000 7ff6ebe00640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6cc0725e0 0x7ff6cc074aa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0af000 tx=0x7ff6dc01af80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:31.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.217+0000 7ff6e3e00640 1 -- 172.21.15.16:0/1416084108 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff6e4016030 con 0x7ff6ec0d1bf0 2024-09-10T21:17:31.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:31.393+0000 7ff6f1600640 1 -- 172.21.15.16:0/1416084108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff6ec1fbfa0 con 0x7ff6ec0d1bf0 2024-09-10T21:17:32.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.029+0000 7ff6e3e00640 1 -- 172.21.15.16:0/1416084108 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) ==== 169+0+0 (secure 0 0 0) 0x7ff6e405e8f0 con 0x7ff6ec0d1bf0 2024-09-10T21:17:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-10T21:17:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.033+0000 7ff6f1600640 1 -- 172.21.15.16:0/1416084108 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6cc0725e0 msgr2=0x7ff6cc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.033+0000 7ff6f1600640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6cc0725e0 0x7ff6cc074aa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0af000 tx=0x7ff6dc01af80 comp rx=0 tx=0).stop 2024-09-10T21:17:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.033+0000 7ff6f1600640 1 -- 172.21.15.16:0/1416084108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec0d1bf0 msgr2=0x7ff6ec14ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.033+0000 7ff6f1600640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec0d1bf0 0x7ff6ec14ffb0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400ea50 tx=0x7ff6e400ef10 comp rx=0 tx=0).stop 2024-09-10T21:17:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.037+0000 7ff6f1600640 1 -- 172.21.15.16:0/1416084108 shutdown_connections 2024-09-10T21:17:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.037+0000 7ff6f1600640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff6cc0725e0 0x7ff6cc074aa0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.037+0000 7ff6f1600640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6ec0d1bf0 0x7ff6ec14ffb0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.037+0000 7ff6f1600640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff6ec0b4bb0 0x7ff6ec14f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.037+0000 7ff6f1600640 1 --2- 172.21.15.16:0/1416084108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6ec0b3920 0x7ff6ec0b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.037+0000 7ff6f1600640 1 -- 172.21.15.16:0/1416084108 >> 172.21.15.16:0/1416084108 conn(0x7ff6ec0d3e10 msgr2=0x7ff6ec0cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.037+0000 7ff6f1600640 1 -- 172.21.15.16:0/1416084108 shutdown_connections 2024-09-10T21:17:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.037+0000 7ff6f1600640 1 -- 172.21.15.16:0/1416084108 wait complete. 2024-09-10T21:17:32.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.209+0000 7f675ca00640 1 -- 172.21.15.16:0/1138620289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6758155000 msgr2=0x7f675815bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.209+0000 7f675ca00640 1 --2- 172.21.15.16:0/1138620289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6758155000 0x7f675815bbb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f6748009900 tx=0x7f674801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.209+0000 7f675ca00640 1 -- 172.21.15.16:0/1138620289 shutdown_connections 2024-09-10T21:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.209+0000 7f675ca00640 1 --2- 172.21.15.16:0/1138620289 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6758072d40 0x7f675806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.209+0000 7f675ca00640 1 --2- 172.21.15.16:0/1138620289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6758155000 0x7f675815bbb0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.209+0000 7f675ca00640 1 -- 172.21.15.16:0/1138620289 >> 172.21.15.16:0/1138620289 conn(0x7f67580cfb20 msgr2=0x7f67580cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f675ca00640 1 -- 172.21.15.16:0/1138620289 shutdown_connections 2024-09-10T21:17:32.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f675ca00640 1 -- 172.21.15.16:0/1138620289 wait complete. 2024-09-10T21:17:32.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f675ca00640 1 Processor -- start 2024-09-10T21:17:32.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f675ca00640 1 -- start start 2024-09-10T21:17:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f675ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6758070520 0x7f67581f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f675ca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6758072d40 0x7f675815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f675ca00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6758155000 0x7f67580cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f675ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6758167b20 con 0x7f6758070520 2024-09-10T21:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f675ca00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f67581679a0 con 0x7f6758072d40 2024-09-10T21:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f675ca00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6758167ca0 con 0x7f6758155000 2024-09-10T21:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f6755e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6758070520 0x7f67581f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f6755e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6758070520 0x7f67581f7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37800/0 (socket says 172.21.15.16:37800) 2024-09-10T21:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f6755e00640 1 -- 172.21.15.16:0/3756319005 learned_addr learned my addr 172.21.15.16:0/3756319005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f674fe00640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6758072d40 0x7f675815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f6756800640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6758155000 0x7f67580cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f6755e00640 1 -- 172.21.15.16:0/3756319005 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6758155000 msgr2=0x7f67580cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f6755e00640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6758155000 0x7f67580cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f6755e00640 1 -- 172.21.15.16:0/3756319005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6758072d40 msgr2=0x7f675815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f6755e00640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6758072d40 0x7f675815dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f6755e00640 1 -- 172.21.15.16:0/3756319005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675815e420 con 0x7f6758070520 2024-09-10T21:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f6755e00640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6758070520 0x7f67581f7c00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f6748009850 tx=0x7f67480a5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f674e000640 1 -- 172.21.15.16:0/3756319005 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67480b2040 con 0x7f6758070520 2024-09-10T21:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f674e000640 1 -- 172.21.15.16:0/3756319005 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67480043b0 con 0x7f6758070520 2024-09-10T21:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f674e000640 1 -- 172.21.15.16:0/3756319005 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748005270 con 0x7f6758070520 2024-09-10T21:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f675ca00640 1 -- 172.21.15.16:0/3756319005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6758163bc0 con 0x7f6758070520 2024-09-10T21:17:32.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.213+0000 7f675ca00640 1 -- 172.21.15.16:0/3756319005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6758164180 con 0x7f6758070520 2024-09-10T21:17:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.217+0000 7f675ca00640 1 -- 172.21.15.16:0/3756319005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6758164790 con 0x7f6758070520 2024-09-10T21:17:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.217+0000 7f674e000640 1 -- 172.21.15.16:0/3756319005 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f67480bf060 con 0x7f6758070520 2024-09-10T21:17:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.217+0000 7f674e000640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67340725e0 0x7f6734074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.217+0000 7f674e000640 1 -- 172.21.15.16:0/3756319005 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f674812bf20 con 0x7f6758070520 2024-09-10T21:17:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.217+0000 7f674fe00640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67340725e0 0x7f6734074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.217+0000 7f674fe00640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67340725e0 0x7f6734074aa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f67400058d0 tx=0x7f6740005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:32.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.221+0000 7f674e000640 1 -- 172.21.15.16:0/3756319005 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f675815e420 con 0x7f6758070520 2024-09-10T21:17:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.393+0000 7f675ca00640 1 -- 172.21.15.16:0/3756319005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f675806ffe0 con 0x7f6758070520 2024-09-10T21:17:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.401+0000 7f674e000640 1 -- 172.21.15.16:0/3756319005 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v52) ==== 71+0+0 (secure 0 0 0) 0x7f67480164d0 con 0x7f6758070520 2024-09-10T21:17:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.401+0000 7f675ca00640 1 -- 172.21.15.16:0/3756319005 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67340725e0 msgr2=0x7f6734074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.401+0000 7f675ca00640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67340725e0 0x7f6734074aa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f67400058d0 tx=0x7f6740005860 comp rx=0 tx=0).stop 2024-09-10T21:17:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.401+0000 7f675ca00640 1 -- 172.21.15.16:0/3756319005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6758070520 msgr2=0x7f67581f7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.401+0000 7f675ca00640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6758070520 0x7f67581f7c00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f6748009850 tx=0x7f67480a5550 comp rx=0 tx=0).stop 2024-09-10T21:17:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.401+0000 7f675ca00640 1 -- 172.21.15.16:0/3756319005 shutdown_connections 2024-09-10T21:17:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.401+0000 7f675ca00640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67340725e0 0x7f6734074aa0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.401+0000 7f675ca00640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6758155000 0x7f67580cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.401+0000 7f675ca00640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6758072d40 0x7f675815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.401+0000 7f675ca00640 1 --2- 172.21.15.16:0/3756319005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6758070520 0x7f67581f7c00 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.401+0000 7f675ca00640 1 -- 172.21.15.16:0/3756319005 >> 172.21.15.16:0/3756319005 conn(0x7f67580cfb20 msgr2=0x7f6758154230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.401+0000 7f675ca00640 1 -- 172.21.15.16:0/3756319005 shutdown_connections 2024-09-10T21:17:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.401+0000 7f675ca00640 1 -- 172.21.15.16:0/3756319005 wait complete. 2024-09-10T21:17:32.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2024-09-10T21:17:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.573+0000 7fcd5be00640 1 -- 172.21.15.16:0/2271268289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd5c070090 msgr2=0x7fcd5c15b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.573+0000 7fcd5be00640 1 --2- 172.21.15.16:0/2271268289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd5c070090 0x7fcd5c15b100 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400acd0 tx=0x7fcd5401dde0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.573+0000 7fcd5be00640 1 -- 172.21.15.16:0/2271268289 shutdown_connections 2024-09-10T21:17:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.573+0000 7fcd5be00640 1 --2- 172.21.15.16:0/2271268289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd5c070090 0x7fcd5c15b100 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.573+0000 7fcd5be00640 1 -- 172.21.15.16:0/2271268289 >> 172.21.15.16:0/2271268289 conn(0x7fcd5c0cc810 msgr2=0x7fcd5c0ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.573+0000 7fcd5be00640 1 -- 172.21.15.16:0/2271268289 shutdown_connections 2024-09-10T21:17:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.573+0000 7fcd5be00640 1 -- 172.21.15.16:0/2271268289 wait complete. 2024-09-10T21:17:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5be00640 1 Processor -- start 2024-09-10T21:17:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5be00640 1 -- start start 2024-09-10T21:17:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd5c070090 0x7fcd5c0cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5c0aef50 0x7fcd5c09a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5be00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcd5c0caaf0 0x7fcd5c14add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c1684b0 con 0x7fcd5c0aef50 2024-09-10T21:17:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5be00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c168330 con 0x7fcd5c070090 2024-09-10T21:17:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5be00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5c168630 con 0x7fcd5c0caaf0 2024-09-10T21:17:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd53e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5c0aef50 0x7fcd5c09a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd53e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5c0aef50 0x7fcd5c09a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37808/0 (socket says 172.21.15.16:37808) 2024-09-10T21:17:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd53e00640 1 -- 172.21.15.16:0/1600205225 learned_addr learned my addr 172.21.15.16:0/1600205225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5b400640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcd5c0caaf0 0x7fcd5c14add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5aa00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd5c070090 0x7fcd5c0cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd53e00640 1 -- 172.21.15.16:0/1600205225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcd5c0caaf0 msgr2=0x7fcd5c14add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd53e00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcd5c0caaf0 0x7fcd5c14add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd53e00640 1 -- 172.21.15.16:0/1600205225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd5c070090 msgr2=0x7fcd5c0cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd53e00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd5c070090 0x7fcd5c0cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd53e00640 1 -- 172.21.15.16:0/1600205225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c153630 con 0x7fcd5c0aef50 2024-09-10T21:17:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5aa00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd5c070090 0x7fcd5c0cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd53e00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5c0aef50 0x7fcd5c09a9d0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fcd44001230 tx=0x7fcd4400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd52a00640 1 -- 172.21.15.16:0/1600205225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44016020 con 0x7fcd5c0aef50 2024-09-10T21:17:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd52a00640 1 -- 172.21.15.16:0/1600205225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd4400a930 con 0x7fcd5c0aef50 2024-09-10T21:17:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd52a00640 1 -- 172.21.15.16:0/1600205225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4400ad60 con 0x7fcd5c0aef50 2024-09-10T21:17:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5be00640 1 -- 172.21.15.16:0/1600205225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c152d90 con 0x7fcd5c0aef50 2024-09-10T21:17:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5be00640 1 -- 172.21.15.16:0/1600205225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c153350 con 0x7fcd5c0aef50 2024-09-10T21:17:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.577+0000 7fcd5be00640 1 -- 172.21.15.16:0/1600205225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c15b100 con 0x7fcd5c0aef50 2024-09-10T21:17:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.581+0000 7fcd52a00640 1 -- 172.21.15.16:0/1600205225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcd44012420 con 0x7fcd5c0aef50 2024-09-10T21:17:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.581+0000 7fcd52a00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcd380725e0 0x7fcd38074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.581+0000 7fcd52a00640 1 -- 172.21.15.16:0/1600205225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7fcd44093350 con 0x7fcd5c0aef50 2024-09-10T21:17:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.581+0000 7fcd5aa00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcd380725e0 0x7fcd38074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.581+0000 7fcd5aa00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcd380725e0 0x7fcd38074aa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c001750 tx=0x7fcd4c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.585+0000 7fcd52a00640 1 -- 172.21.15.16:0/1600205225 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcd44020020 con 0x7fcd5c0aef50 2024-09-10T21:17:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:32.825+0000 7fcd5be00640 1 -- 172.21.15.16:0/1600205225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) -- 0x7fcd5c0ca8d0 con 0x7fcd5c0aef50 2024-09-10T21:17:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.033+0000 7fcd52a00640 1 -- 172.21.15.16:0/1600205225 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v356) ==== 143+0+0 (secure 0 0 0) 0x7fcd4405ec90 con 0x7fcd5c0aef50 2024-09-10T21:17:33.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.033+0000 7fcd5be00640 1 -- 172.21.15.16:0/1600205225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcd380725e0 msgr2=0x7fcd38074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.033+0000 7fcd5be00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcd380725e0 0x7fcd38074aa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c001750 tx=0x7fcd4c009040 comp rx=0 tx=0).stop 2024-09-10T21:17:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.033+0000 7fcd5be00640 1 -- 172.21.15.16:0/1600205225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5c0aef50 msgr2=0x7fcd5c09a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.033+0000 7fcd5be00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5c0aef50 0x7fcd5c09a9d0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fcd44001230 tx=0x7fcd4400cb90 comp rx=0 tx=0).stop 2024-09-10T21:17:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.033+0000 7fcd5be00640 1 -- 172.21.15.16:0/1600205225 shutdown_connections 2024-09-10T21:17:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.033+0000 7fcd5be00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcd380725e0 0x7fcd38074aa0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.033+0000 7fcd5be00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcd5c0caaf0 0x7fcd5c14add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.033+0000 7fcd5be00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5c0aef50 0x7fcd5c09a9d0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.033+0000 7fcd5be00640 1 --2- 172.21.15.16:0/1600205225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd5c070090 0x7fcd5c0cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.033+0000 7fcd5be00640 1 -- 172.21.15.16:0/1600205225 >> 172.21.15.16:0/1600205225 conn(0x7fcd5c0cc810 msgr2=0x7fcd5c15c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.033+0000 7fcd5be00640 1 -- 172.21.15.16:0/1600205225 shutdown_connections 2024-09-10T21:17:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.033+0000 7fcd5be00640 1 -- 172.21.15.16:0/1600205225 wait complete. 2024-09-10T21:17:33.060 INFO:tasks.cephfs_test_runner:test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-09-10T21:17:33.060 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-10T21:17:33.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:17:33.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.201+0000 7f1bd8200640 1 -- 172.21.15.16:0/2699198146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bd00d22e0 msgr2=0x7f1bd0161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.201+0000 7f1bd8200640 1 --2- 172.21.15.16:0/2699198146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bd00d22e0 0x7f1bd0161360 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009900 tx=0x7f1bc401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.201+0000 7f1bd8200640 1 -- 172.21.15.16:0/2699198146 shutdown_connections 2024-09-10T21:17:33.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.201+0000 7f1bd8200640 1 --2- 172.21.15.16:0/2699198146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1bd00b2690 0x7f1bd01618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.201+0000 7f1bd8200640 1 --2- 172.21.15.16:0/2699198146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bd00d22e0 0x7f1bd0161360 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.201+0000 7f1bd8200640 1 -- 172.21.15.16:0/2699198146 >> 172.21.15.16:0/2699198146 conn(0x7f1bd00d4010 msgr2=0x7f1bd00d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:33.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.201+0000 7f1bd8200640 1 -- 172.21.15.16:0/2699198146 shutdown_connections 2024-09-10T21:17:33.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.201+0000 7f1bd8200640 1 -- 172.21.15.16:0/2699198146 wait complete. 2024-09-10T21:17:33.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd8200640 1 Processor -- start 2024-09-10T21:17:33.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd8200640 1 -- start start 2024-09-10T21:17:33.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd8200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1bd00b2690 0x7f1bd00cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:33.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd8200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bd00d22e0 0x7f1bd01574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:33.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd8200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd00d4b30 0x7f1bd014ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:33.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd8200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd016b730 con 0x7f1bd00d4b30 2024-09-10T21:17:33.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd8200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd016b5b0 con 0x7f1bd00d22e0 2024-09-10T21:17:33.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd8200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd016b8b0 con 0x7f1bd00b2690 2024-09-10T21:17:33.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd6200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd00d4b30 0x7f1bd014ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:33.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd6200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd00d4b30 0x7f1bd014ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37810/0 (socket says 172.21.15.16:37810) 2024-09-10T21:17:33.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd6200640 1 -- 172.21.15.16:0/2669181314 learned_addr learned my addr 172.21.15.16:0/2669181314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:33.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd5800640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1bd00b2690 0x7f1bd00cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:33.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd4e00640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bd00d22e0 0x7f1bd01574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:33.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd6200640 1 -- 172.21.15.16:0/2669181314 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1bd00b2690 msgr2=0x7f1bd00cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd6200640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1bd00b2690 0x7f1bd00cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd6200640 1 -- 172.21.15.16:0/2669181314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bd00d22e0 msgr2=0x7f1bd01574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd6200640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bd00d22e0 0x7f1bd01574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd6200640 1 -- 172.21.15.16:0/2669181314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd01fbe10 con 0x7f1bd00d4b30 2024-09-10T21:17:33.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd5800640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1bd00b2690 0x7f1bd00cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:33.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd6200640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd00d4b30 0x7f1bd014ab30 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009ca0 tx=0x7f1bc800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:33.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bcd600640 1 -- 172.21.15.16:0/2669181314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8010630 con 0x7f1bd00d4b30 2024-09-10T21:17:33.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bcd600640 1 -- 172.21.15.16:0/2669181314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bc80108e0 con 0x7f1bd00d4b30 2024-09-10T21:17:33.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bcd600640 1 -- 172.21.15.16:0/2669181314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8010d30 con 0x7f1bd00d4b30 2024-09-10T21:17:33.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd8200640 1 -- 172.21.15.16:0/2669181314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd00c6ef0 con 0x7f1bd00d4b30 2024-09-10T21:17:33.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.205+0000 7f1bd8200640 1 -- 172.21.15.16:0/2669181314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd01579e0 con 0x7f1bd00d4b30 2024-09-10T21:17:33.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.209+0000 7f1bd8200640 1 -- 172.21.15.16:0/2669181314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd00b37e0 con 0x7f1bd00d4b30 2024-09-10T21:17:33.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.209+0000 7f1bcd600640 1 -- 172.21.15.16:0/2669181314 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1bc8018420 con 0x7f1bd00d4b30 2024-09-10T21:17:33.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.209+0000 7f1bcd600640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1bb0072610 0x7f1bb0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:33.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.209+0000 7f1bcd600640 1 -- 172.21.15.16:0/2669181314 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f1bc8093ea0 con 0x7f1bd00d4b30 2024-09-10T21:17:33.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.209+0000 7f1bd5800640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1bb0072610 0x7f1bb0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:33.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.209+0000 7f1bd5800640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1bb0072610 0x7f1bb0074ad0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40af000 tx=0x7f1bc401b120 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:33.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.213+0000 7f1bcd600640 1 -- 172.21.15.16:0/2669181314 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1bc805f7e0 con 0x7f1bd00d4b30 2024-09-10T21:17:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.389+0000 7f1bd8200640 1 -- 172.21.15.16:0/2669181314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bd0158610 con 0x7f1bd00d4b30 2024-09-10T21:17:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.389+0000 7f1bcd600640 1 -- 172.21.15.16:0/2669181314 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+12994 (secure 0 0 0) 0x7f1bc805f0e0 con 0x7f1bd00d4b30 2024-09-10T21:17:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:33.395 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":108,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:17:32.028346+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6834/1108687782":"2024-09-11T21:17:29.018677+0000","172.21.15.52:6832/1108687782":"2024-09-11T21:17:29.018677+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:17:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.393+0000 7f1bd8200640 1 -- 172.21.15.16:0/2669181314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1bb0072610 msgr2=0x7f1bb0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.393+0000 7f1bd8200640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1bb0072610 0x7f1bb0074ad0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40af000 tx=0x7f1bc401b120 comp rx=0 tx=0).stop 2024-09-10T21:17:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.393+0000 7f1bd8200640 1 -- 172.21.15.16:0/2669181314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd00d4b30 msgr2=0x7f1bd014ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.393+0000 7f1bd8200640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd00d4b30 0x7f1bd014ab30 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009ca0 tx=0x7f1bc800c6a0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.393+0000 7f1bd8200640 1 -- 172.21.15.16:0/2669181314 shutdown_connections 2024-09-10T21:17:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.393+0000 7f1bd8200640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1bb0072610 0x7f1bb0074ad0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.393+0000 7f1bd8200640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd00d4b30 0x7f1bd014ab30 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.393+0000 7f1bd8200640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bd00d22e0 0x7f1bd01574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.393+0000 7f1bd8200640 1 --2- 172.21.15.16:0/2669181314 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1bd00b2690 0x7f1bd00cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.393+0000 7f1bd8200640 1 -- 172.21.15.16:0/2669181314 >> 172.21.15.16:0/2669181314 conn(0x7f1bd00d4010 msgr2=0x7f1bd015f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.393+0000 7f1bd8200640 1 -- 172.21.15.16:0/2669181314 shutdown_connections 2024-09-10T21:17:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.393+0000 7f1bd8200640 1 -- 172.21.15.16:0/2669181314 wait complete. 2024-09-10T21:17:33.417 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:10:17.493770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:17:33.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:17:33.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.565+0000 7f62c5c00640 1 -- 172.21.15.16:0/1141125558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a40051d0 msgr2=0x7f62a400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.565+0000 7f62c5c00640 1 --2- 172.21.15.16:0/1141125558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a40051d0 0x7f62a400b9d0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f62b8001e60 tx=0x7f62b801cbe0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.565+0000 7f62c5c00640 1 -- 172.21.15.16:0/1141125558 shutdown_connections 2024-09-10T21:17:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.565+0000 7f62c5c00640 1 --2- 172.21.15.16:0/1141125558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a40051d0 0x7f62a400b9d0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.565+0000 7f62c5c00640 1 -- 172.21.15.16:0/1141125558 >> 172.21.15.16:0/1141125558 conn(0x7f62c00b16a0 msgr2=0x7f62c00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.565+0000 7f62c5c00640 1 -- 172.21.15.16:0/1141125558 shutdown_connections 2024-09-10T21:17:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.565+0000 7f62c5c00640 1 -- 172.21.15.16:0/1141125558 wait complete. 2024-09-10T21:17:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.565+0000 7f62c5c00640 1 Processor -- start 2024-09-10T21:17:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.565+0000 7f62c5c00640 1 -- start start 2024-09-10T21:17:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62c5c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62a40051d0 0x7f62c015eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62c5c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62c015ba20 0x7f62c015b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62c5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62c01e62c0 0x7f62c0140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62c5c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62c006f9f0 con 0x7f62c01e62c0 2024-09-10T21:17:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62c5c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62c006f870 con 0x7f62c015ba20 2024-09-10T21:17:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62c5c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f62c00cf3e0 con 0x7f62a40051d0 2024-09-10T21:17:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62bf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62c01e62c0 0x7f62c0140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62bf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62c01e62c0 0x7f62c0140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37830/0 (socket says 172.21.15.16:37830) 2024-09-10T21:17:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62bf400640 1 -- 172.21.15.16:0/728217216 learned_addr learned my addr 172.21.15.16:0/728217216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62b7e00640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62c015ba20 0x7f62c015b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62bea00640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62a40051d0 0x7f62c015eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62bf400640 1 -- 172.21.15.16:0/728217216 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62a40051d0 msgr2=0x7f62c015eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62bf400640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62a40051d0 0x7f62c015eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62bf400640 1 -- 172.21.15.16:0/728217216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62c015ba20 msgr2=0x7f62c015b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62bf400640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62c015ba20 0x7f62c015b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62bf400640 1 -- 172.21.15.16:0/728217216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c01e5220 con 0x7f62c01e62c0 2024-09-10T21:17:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62bf400640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62c01e62c0 0x7f62c0140950 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f62b801ca10 tx=0x7f62b8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62b6a00640 1 -- 172.21.15.16:0/728217216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b80a59f0 con 0x7f62c01e62c0 2024-09-10T21:17:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62b6a00640 1 -- 172.21.15.16:0/728217216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62b8006200 con 0x7f62c01e62c0 2024-09-10T21:17:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62b6a00640 1 -- 172.21.15.16:0/728217216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8006650 con 0x7f62c01e62c0 2024-09-10T21:17:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62c5c00640 1 -- 172.21.15.16:0/728217216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c01e5500 con 0x7f62c01e62c0 2024-09-10T21:17:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62c5c00640 1 -- 172.21.15.16:0/728217216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c01e5ac0 con 0x7f62c01e62c0 2024-09-10T21:17:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62c5c00640 1 -- 172.21.15.16:0/728217216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c00cf8b0 con 0x7f62c01e62c0 2024-09-10T21:17:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.569+0000 7f62b6a00640 1 -- 172.21.15.16:0/728217216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f62b80ba410 con 0x7f62c01e62c0 2024-09-10T21:17:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.573+0000 7f62b6a00640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f629c072610 0x7f629c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.573+0000 7f62b6a00640 1 -- 172.21.15.16:0/728217216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f62b812ca90 con 0x7f62c01e62c0 2024-09-10T21:17:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.573+0000 7f62bea00640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f629c072610 0x7f629c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.573+0000 7f62bea00640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f629c072610 0x7f629c074ad0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f62c013bea0 tx=0x7f62b0001520 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:33.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.577+0000 7f62b6a00640 1 -- 172.21.15.16:0/728217216 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f62b80168f0 con 0x7f62c01e62c0 2024-09-10T21:17:33.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62c5c00640 1 -- 172.21.15.16:0/728217216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62c00736e0 con 0x7f62c01e62c0 2024-09-10T21:17:33.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62b6a00640 1 -- 172.21.15.16:0/728217216 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) ==== 93+0+10 (secure 0 0 0) 0x7f62b80f83d0 con 0x7f62c01e62c0 2024-09-10T21:17:33.756 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-10T21:17:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62c5c00640 1 -- 172.21.15.16:0/728217216 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f629c072610 msgr2=0x7f629c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62c5c00640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f629c072610 0x7f629c074ad0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f62c013bea0 tx=0x7f62b0001520 comp rx=0 tx=0).stop 2024-09-10T21:17:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62c5c00640 1 -- 172.21.15.16:0/728217216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62c01e62c0 msgr2=0x7f62c0140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62c5c00640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62c01e62c0 0x7f62c0140950 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f62b801ca10 tx=0x7f62b8003f60 comp rx=0 tx=0).stop 2024-09-10T21:17:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62c5c00640 1 -- 172.21.15.16:0/728217216 shutdown_connections 2024-09-10T21:17:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62c5c00640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f629c072610 0x7f629c074ad0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62c5c00640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62c01e62c0 0x7f62c0140950 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62c5c00640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62c015ba20 0x7f62c015b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62c5c00640 1 --2- 172.21.15.16:0/728217216 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62a40051d0 0x7f62c015eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62c5c00640 1 -- 172.21.15.16:0/728217216 >> 172.21.15.16:0/728217216 conn(0x7f62c00b16a0 msgr2=0x7f62c00726b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62c5c00640 1 -- 172.21.15.16:0/728217216 shutdown_connections 2024-09-10T21:17:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.753+0000 7f62c5c00640 1 -- 172.21.15.16:0/728217216 wait complete. 2024-09-10T21:17:33.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2024-09-10T21:17:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.921+0000 7f2a92800640 1 -- 172.21.15.16:0/771270490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a70003a10 msgr2=0x7f2a70003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.921+0000 7f2a92800640 1 --2- 172.21.15.16:0/771270490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a70003a10 0x7f2a70003df0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0017f0 tx=0x7f2a7c01c870 comp rx=0 tx=0).stop 2024-09-10T21:17:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a92800640 1 -- 172.21.15.16:0/771270490 shutdown_connections 2024-09-10T21:17:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a92800640 1 --2- 172.21.15.16:0/771270490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a70003a10 0x7f2a70003df0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a92800640 1 -- 172.21.15.16:0/771270490 >> 172.21.15.16:0/771270490 conn(0x7f2a8c0cfb20 msgr2=0x7f2a8c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a92800640 1 -- 172.21.15.16:0/771270490 shutdown_connections 2024-09-10T21:17:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a92800640 1 -- 172.21.15.16:0/771270490 wait complete. 2024-09-10T21:17:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a92800640 1 Processor -- start 2024-09-10T21:17:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a92800640 1 -- start start 2024-09-10T21:17:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a92800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a70003a10 0x7f2a8c14c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a92800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8c159ac0 0x7f2a8c159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a92800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c0c3db0 0x7f2a8c0c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a92800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c15eb30 con 0x7f2a8c0c3db0 2024-09-10T21:17:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a92800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c15e9b0 con 0x7f2a8c159ac0 2024-09-10T21:17:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a92800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2a8c15ecb0 con 0x7f2a70003a10 2024-09-10T21:17:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a8be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c0c3db0 0x7f2a8c0c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a8b400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a70003a10 0x7f2a8c14c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a8be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c0c3db0 0x7f2a8c0c4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37854/0 (socket says 172.21.15.16:37854) 2024-09-10T21:17:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a8be00640 1 -- 172.21.15.16:0/2782115015 learned_addr learned my addr 172.21.15.16:0/2782115015 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a83e00640 1 --2- 172.21.15.16:0/2782115015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8c159ac0 0x7f2a8c159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a8be00640 1 -- 172.21.15.16:0/2782115015 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a70003a10 msgr2=0x7f2a8c14c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a8be00640 1 --2- 172.21.15.16:0/2782115015 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a70003a10 0x7f2a8c14c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a8be00640 1 -- 172.21.15.16:0/2782115015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8c159ac0 msgr2=0x7f2a8c159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a8be00640 1 --2- 172.21.15.16:0/2782115015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8c159ac0 0x7f2a8c159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a8be00640 1 -- 172.21.15.16:0/2782115015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c0c4770 con 0x7f2a8c0c3db0 2024-09-10T21:17:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a8be00640 1 --2- 172.21.15.16:0/2782115015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c0c3db0 0x7f2a8c0c4230 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009fd0 tx=0x7f2a84010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.925+0000 7f2a83400640 1 -- 172.21.15.16:0/2782115015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84003e50 con 0x7f2a8c0c3db0 2024-09-10T21:17:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.929+0000 7f2a83400640 1 -- 172.21.15.16:0/2782115015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a84015070 con 0x7f2a8c0c3db0 2024-09-10T21:17:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.929+0000 7f2a92800640 1 -- 172.21.15.16:0/2782115015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c1e8830 con 0x7f2a8c0c3db0 2024-09-10T21:17:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.929+0000 7f2a83400640 1 -- 172.21.15.16:0/2782115015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8400e950 con 0x7f2a8c0c3db0 2024-09-10T21:17:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.929+0000 7f2a92800640 1 -- 172.21.15.16:0/2782115015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c1e8df0 con 0x7f2a8c0c3db0 2024-09-10T21:17:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.929+0000 7f2a92800640 1 -- 172.21.15.16:0/2782115015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a8c06a430 con 0x7f2a8c0c3db0 2024-09-10T21:17:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.929+0000 7f2a83400640 1 -- 172.21.15.16:0/2782115015 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2a84021030 con 0x7f2a8c0c3db0 2024-09-10T21:17:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.929+0000 7f2a83400640 1 --2- 172.21.15.16:0/2782115015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a68072780 0x7f2a68074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.929+0000 7f2a83400640 1 -- 172.21.15.16:0/2782115015 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f2a84093350 con 0x7f2a8c0c3db0 2024-09-10T21:17:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.929+0000 7f2a8b400640 1 --2- 172.21.15.16:0/2782115015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a68072780 0x7f2a68074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.929+0000 7f2a8b400640 1 --2- 172.21.15.16:0/2782115015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a68072780 0x7f2a68074c40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0017c0 tx=0x7f2a7c0b0040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:33.933+0000 7f2a83400640 1 -- 172.21.15.16:0/2782115015 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2a8405e640 con 0x7f2a8c0c3db0 2024-09-10T21:17:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:34.173+0000 7f2a92800640 1 -- 172.21.15.16:0/2782115015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) -- 0x7f2a8c15f210 con 0x7f2a8c0c3db0 2024-09-10T21:17:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.041+0000 7f2a83400640 1 -- 172.21.15.16:0/2782115015 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v358) ==== 154+0+0 (secure 0 0 0) 0x7f2a840673b0 con 0x7f2a8c0c3db0 2024-09-10T21:17:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.041+0000 7f2a92800640 1 -- 172.21.15.16:0/2782115015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a68072780 msgr2=0x7f2a68074c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.041+0000 7f2a92800640 1 --2- 172.21.15.16:0/2782115015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a68072780 0x7f2a68074c40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0017c0 tx=0x7f2a7c0b0040 comp rx=0 tx=0).stop 2024-09-10T21:17:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.041+0000 7f2a92800640 1 -- 172.21.15.16:0/2782115015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c0c3db0 msgr2=0x7f2a8c0c4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.041+0000 7f2a92800640 1 --2- 172.21.15.16:0/2782115015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c0c3db0 0x7f2a8c0c4230 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009fd0 tx=0x7f2a84010f40 comp rx=0 tx=0).stop 2024-09-10T21:17:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.041+0000 7f2a92800640 1 -- 172.21.15.16:0/2782115015 shutdown_connections 2024-09-10T21:17:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.041+0000 7f2a92800640 1 --2- 172.21.15.16:0/2782115015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2a68072780 0x7f2a68074c40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.041+0000 7f2a92800640 1 --2- 172.21.15.16:0/2782115015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c0c3db0 0x7f2a8c0c4230 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.041+0000 7f2a92800640 1 --2- 172.21.15.16:0/2782115015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a8c159ac0 0x7f2a8c159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.041+0000 7f2a92800640 1 --2- 172.21.15.16:0/2782115015 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2a70003a10 0x7f2a8c14c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.041+0000 7f2a92800640 1 -- 172.21.15.16:0/2782115015 >> 172.21.15.16:0/2782115015 conn(0x7f2a8c0cfb20 msgr2=0x7f2a8c06f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.045+0000 7f2a92800640 1 -- 172.21.15.16:0/2782115015 shutdown_connections 2024-09-10T21:17:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.045+0000 7f2a92800640 1 -- 172.21.15.16:0/2782115015 wait complete. 2024-09-10T21:17:35.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:17:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.217+0000 7efccec00640 1 -- 172.21.15.16:0/1512151368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc80ca170 msgr2=0x7efcc80ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:35.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.217+0000 7efccec00640 1 --2- 172.21.15.16:0/1512151368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc80ca170 0x7efcc80ca5b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7efcc806e1a0 tx=0x7efcc001ddf0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.217+0000 7efccec00640 1 -- 172.21.15.16:0/1512151368 shutdown_connections 2024-09-10T21:17:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.217+0000 7efccec00640 1 --2- 172.21.15.16:0/1512151368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc80ca170 0x7efcc80ca5b0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.217+0000 7efccec00640 1 --2- 172.21.15.16:0/1512151368 >> v2:172.21.15.52:3300/0 conn(0x7efcc815a270 0x7efcc80ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.217+0000 7efccec00640 1 -- 172.21.15.16:0/1512151368 >> 172.21.15.16:0/1512151368 conn(0x7efcc80cfb20 msgr2=0x7efcc80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.217+0000 7efccec00640 1 -- 172.21.15.16:0/1512151368 shutdown_connections 2024-09-10T21:17:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.217+0000 7efccec00640 1 -- 172.21.15.16:0/1512151368 wait complete. 2024-09-10T21:17:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.217+0000 7efccec00640 1 Processor -- start 2024-09-10T21:17:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.217+0000 7efccec00640 1 -- start start 2024-09-10T21:17:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efccec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc80ca170 0x7efcc80b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efccec00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efcc80d06b0 0x7efcc81f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efccec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcc815a270 0x7efcc81f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efccec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efcc8072b80 con 0x7efcc80ca170 2024-09-10T21:17:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efccec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efcc8072a00 con 0x7efcc815a270 2024-09-10T21:17:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efccec00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7efcc8072d00 con 0x7efcc80d06b0 2024-09-10T21:17:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efcc7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc80ca170 0x7efcc80b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efcc7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc80ca170 0x7efcc80b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37872/0 (socket says 172.21.15.16:37872) 2024-09-10T21:17:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efcc7e00640 1 -- 172.21.15.16:0/2772874753 learned_addr learned my addr 172.21.15.16:0/2772874753 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efcc7e00640 1 -- 172.21.15.16:0/2772874753 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efcc80d06b0 msgr2=0x7efcc81f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efcc7e00640 1 --2- 172.21.15.16:0/2772874753 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efcc80d06b0 0x7efcc81f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efcc7e00640 1 -- 172.21.15.16:0/2772874753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcc815a270 msgr2=0x7efcc81f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efcc7e00640 1 --2- 172.21.15.16:0/2772874753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcc815a270 0x7efcc81f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efcc7e00640 1 -- 172.21.15.16:0/2772874753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc80c8f50 con 0x7efcc80ca170 2024-09-10T21:17:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efcc7e00640 1 --2- 172.21.15.16:0/2772874753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc80ca170 0x7efcc80b5060 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7efcb800e7d0 tx=0x7efcb800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efcc4c00640 1 -- 172.21.15.16:0/2772874753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb800ccc0 con 0x7efcc80ca170 2024-09-10T21:17:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efcc4c00640 1 -- 172.21.15.16:0/2772874753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efcb8010430 con 0x7efcc80ca170 2024-09-10T21:17:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efcc4c00640 1 -- 172.21.15.16:0/2772874753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8010860 con 0x7efcc80ca170 2024-09-10T21:17:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efccec00640 1 -- 172.21.15.16:0/2772874753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc80c9230 con 0x7efcc80ca170 2024-09-10T21:17:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efccec00640 1 -- 172.21.15.16:0/2772874753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc80c97f0 con 0x7efcc80ca170 2024-09-10T21:17:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efccec00640 1 -- 172.21.15.16:0/2772874753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc80ca5b0 con 0x7efcc80ca170 2024-09-10T21:17:35.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.221+0000 7efcc4c00640 1 -- 172.21.15.16:0/2772874753 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efcb8010a00 con 0x7efcc80ca170 2024-09-10T21:17:35.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.225+0000 7efcc4c00640 1 --2- 172.21.15.16:0/2772874753 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efca40725e0 0x7efca4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:35.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.225+0000 7efcc4c00640 1 -- 172.21.15.16:0/2772874753 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7efcb8093970 con 0x7efcc80ca170 2024-09-10T21:17:35.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.225+0000 7efcbfe00640 1 --2- 172.21.15.16:0/2772874753 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efca40725e0 0x7efca4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:35.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.225+0000 7efcbfe00640 1 --2- 172.21.15.16:0/2772874753 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efca40725e0 0x7efca4074aa0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efcb00046c0 tx=0x7efcb0007910 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:35.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.229+0000 7efcc4c00640 1 -- 172.21.15.16:0/2772874753 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efcb805f340 con 0x7efcc80ca170 2024-09-10T21:17:35.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.401+0000 7efccec00640 1 -- 172.21.15.16:0/2772874753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7efcc8148d60 con 0x7efcc80ca170 2024-09-10T21:17:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.401+0000 7efcc4c00640 1 -- 172.21.15.16:0/2772874753 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v52) ==== 59+0+1998 (secure 0 0 0) 0x7efcb805ec40 con 0x7efcc80ca170 2024-09-10T21:17:35.406 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:17:35.406 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:17:35.406 INFO:teuthology.orchestra.run.smithi016.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:17:35.406 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:17:35.406 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4226.427518 2024-09-10T21:17:35.407 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5426.620519 2024-09-10T21:17:35.407 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4255.210203 2024-09-10T21:17:35.407 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4255.734871 2024-09-10T21:17:35.407 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4192.833008 2024-09-10T21:17:35.407 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4136.940759 2024-09-10T21:17:35.407 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4217.384701 2024-09-10T21:17:35.407 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5220.337972 2024-09-10T21:17:35.407 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-10T21:17:35.407 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:17:35.407 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-10T21:17:35.407 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-10T21:17:35.408 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:17:35.408 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:17:35.408 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-10T21:17:35.408 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:17:35.408 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:17:35.408 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:17:35.408 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-10T21:17:35.408 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-10T21:17:35.408 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-10T21:17:35.408 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:17:35.408 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:17:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.405+0000 7efccec00640 1 -- 172.21.15.16:0/2772874753 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efca40725e0 msgr2=0x7efca4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.405+0000 7efccec00640 1 --2- 172.21.15.16:0/2772874753 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efca40725e0 0x7efca4074aa0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efcb00046c0 tx=0x7efcb0007910 comp rx=0 tx=0).stop 2024-09-10T21:17:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.405+0000 7efccec00640 1 -- 172.21.15.16:0/2772874753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc80ca170 msgr2=0x7efcc80b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.405+0000 7efccec00640 1 --2- 172.21.15.16:0/2772874753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc80ca170 0x7efcc80b5060 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7efcb800e7d0 tx=0x7efcb800ec90 comp rx=0 tx=0).stop 2024-09-10T21:17:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.405+0000 7efccec00640 1 -- 172.21.15.16:0/2772874753 shutdown_connections 2024-09-10T21:17:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.405+0000 7efccec00640 1 --2- 172.21.15.16:0/2772874753 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efca40725e0 0x7efca4074aa0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.405+0000 7efccec00640 1 --2- 172.21.15.16:0/2772874753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcc815a270 0x7efcc81f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.405+0000 7efccec00640 1 --2- 172.21.15.16:0/2772874753 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efcc80d06b0 0x7efcc81f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.405+0000 7efccec00640 1 --2- 172.21.15.16:0/2772874753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc80ca170 0x7efcc80b5060 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.405+0000 7efccec00640 1 -- 172.21.15.16:0/2772874753 >> 172.21.15.16:0/2772874753 conn(0x7efcc80cfb20 msgr2=0x7efcc8156b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.405+0000 7efccec00640 1 -- 172.21.15.16:0/2772874753 shutdown_connections 2024-09-10T21:17:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.405+0000 7efccec00640 1 -- 172.21.15.16:0/2772874753 wait complete. 2024-09-10T21:17:35.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:17:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.573+0000 7f7fda200640 1 -- 172.21.15.16:0/1871169223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd415bdc0 msgr2=0x7f7fd40d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.573+0000 7f7fda200640 1 --2- 172.21.15.16:0/1871169223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd415bdc0 0x7f7fd40d2800 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4073000 tx=0x7f7fc401c6e0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.573+0000 7f7fda200640 1 -- 172.21.15.16:0/1871169223 shutdown_connections 2024-09-10T21:17:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.573+0000 7f7fda200640 1 --2- 172.21.15.16:0/1871169223 >> v2:172.21.15.52:3300/0 conn(0x7f7fd415f940 0x7f7fd415fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.573+0000 7f7fda200640 1 --2- 172.21.15.16:0/1871169223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd415bdc0 0x7f7fd40d2800 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.573+0000 7f7fda200640 1 -- 172.21.15.16:0/1871169223 >> 172.21.15.16:0/1871169223 conn(0x7f7fd40d1ce0 msgr2=0x7f7fd40d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.573+0000 7f7fda200640 1 -- 172.21.15.16:0/1871169223 shutdown_connections 2024-09-10T21:17:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.573+0000 7f7fda200640 1 -- 172.21.15.16:0/1871169223 wait complete. 2024-09-10T21:17:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fda200640 1 Processor -- start 2024-09-10T21:17:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fda200640 1 -- start start 2024-09-10T21:17:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fda200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd40cd300 0x7f7fd40c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fda200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd415bdc0 0x7f7fd41fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fda200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7fd415f940 0x7f7fd40b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fda200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4169070 con 0x7f7fd415bdc0 2024-09-10T21:17:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fda200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4168ef0 con 0x7f7fd40cd300 2024-09-10T21:17:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fda200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd41691f0 con 0x7f7fd415f940 2024-09-10T21:17:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fcbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd415bdc0 0x7f7fd41fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fcbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd415bdc0 0x7f7fd41fb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37898/0 (socket says 172.21.15.16:37898) 2024-09-10T21:17:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fcbe00640 1 -- 172.21.15.16:0/723933335 learned_addr learned my addr 172.21.15.16:0/723933335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fd3400640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd40cd300 0x7f7fd40c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fd3e00640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7fd415f940 0x7f7fd40b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fcbe00640 1 -- 172.21.15.16:0/723933335 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7fd415f940 msgr2=0x7f7fd40b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fcbe00640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7fd415f940 0x7f7fd40b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fcbe00640 1 -- 172.21.15.16:0/723933335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd40cd300 msgr2=0x7f7fd40c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fcbe00640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd40cd300 0x7f7fd40c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fcbe00640 1 -- 172.21.15.16:0/723933335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd40c5900 con 0x7f7fd415bdc0 2024-09-10T21:17:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fd3400640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd40cd300 0x7f7fd40c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fcbe00640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd415bdc0 0x7f7fd41fb9a0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00c450 tx=0x7f7fbc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fcb400640 1 -- 172.21.15.16:0/723933335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc017070 con 0x7f7fd415bdc0 2024-09-10T21:17:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fcb400640 1 -- 172.21.15.16:0/723933335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fbc003600 con 0x7f7fd415bdc0 2024-09-10T21:17:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fcb400640 1 -- 172.21.15.16:0/723933335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc003a50 con 0x7f7fd415bdc0 2024-09-10T21:17:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fda200640 1 -- 172.21.15.16:0/723933335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd40c5be0 con 0x7f7fd415bdc0 2024-09-10T21:17:35.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.577+0000 7f7fda200640 1 -- 172.21.15.16:0/723933335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd41fb480 con 0x7f7fd415bdc0 2024-09-10T21:17:35.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.581+0000 7f7fda200640 1 -- 172.21.15.16:0/723933335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd40d2800 con 0x7f7fd415bdc0 2024-09-10T21:17:35.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.581+0000 7f7fcb400640 1 -- 172.21.15.16:0/723933335 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7fbc003bf0 con 0x7f7fd415bdc0 2024-09-10T21:17:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.581+0000 7f7fcb400640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fb00725e0 0x7f7fb0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:35.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.581+0000 7f7fcb400640 1 -- 172.21.15.16:0/723933335 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f7fbc0926e0 con 0x7f7fd415bdc0 2024-09-10T21:17:35.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.581+0000 7f7fd3400640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fb00725e0 0x7f7fb0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:35.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.581+0000 7f7fd3400640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fb00725e0 0x7f7fb0074aa0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40af000 tx=0x7f7fc401af20 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:35.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.585+0000 7f7fcb400640 1 -- 172.21.15.16:0/723933335 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7fbc05e0f0 con 0x7f7fd415bdc0 2024-09-10T21:17:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.761+0000 7f7fda200640 1 -- 172.21.15.16:0/723933335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7fd40c5db0 con 0x7f7fd415bdc0 2024-09-10T21:17:35.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.765+0000 7f7fcb400640 1 -- 172.21.15.16:0/723933335 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v53)=0 v53) ==== 122+0+0 (secure 0 0 0) 0x7f7fbc05d9f0 con 0x7f7fd415bdc0 2024-09-10T21:17:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.769+0000 7f7fda200640 1 -- 172.21.15.16:0/723933335 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fb00725e0 msgr2=0x7f7fb0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.769+0000 7f7fda200640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fb00725e0 0x7f7fb0074aa0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40af000 tx=0x7f7fc401af20 comp rx=0 tx=0).stop 2024-09-10T21:17:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.769+0000 7f7fda200640 1 -- 172.21.15.16:0/723933335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd415bdc0 msgr2=0x7f7fd41fb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.769+0000 7f7fda200640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd415bdc0 0x7f7fd41fb9a0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00c450 tx=0x7f7fbc00c910 comp rx=0 tx=0).stop 2024-09-10T21:17:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.769+0000 7f7fda200640 1 -- 172.21.15.16:0/723933335 shutdown_connections 2024-09-10T21:17:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.769+0000 7f7fda200640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fb00725e0 0x7f7fb0074aa0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.769+0000 7f7fda200640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7fd415f940 0x7f7fd40b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.769+0000 7f7fda200640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd415bdc0 0x7f7fd41fb9a0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.769+0000 7f7fda200640 1 --2- 172.21.15.16:0/723933335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fd40cd300 0x7f7fd40c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.769+0000 7f7fda200640 1 -- 172.21.15.16:0/723933335 >> 172.21.15.16:0/723933335 conn(0x7f7fd40d1ce0 msgr2=0x7f7fd4156b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.769+0000 7f7fda200640 1 -- 172.21.15.16:0/723933335 shutdown_connections 2024-09-10T21:17:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:35.769+0000 7f7fda200640 1 -- 172.21.15.16:0/723933335 wait complete. 2024-09-10T21:17:35.792 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:17:35.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:17:35.894 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:17:35.982 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:17:35.983 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:17:35.983 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:35.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:17:35.991 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:17:35.992 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:36.001 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:17:36.001 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:17:36.001 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:36.043 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:17:36.044 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:17:36.044 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:36.054 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:17:36.054 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:17:36.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:36.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.229+0000 7f5c09e00640 1 -- 172.21.15.16:0/2008170661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be8003a60 msgr2=0x7f5be8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.229+0000 7f5c09e00640 1 --2- 172.21.15.16:0/2008170661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be8003a60 0x7f5be8003e80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009b60 tx=0x7f5bf401d2b0 comp rx=0 tx=0).stop 2024-09-10T21:17:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.229+0000 7f5c03400640 1 -- 172.21.15.16:0/2008170661 reap_dead start 2024-09-10T21:17:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.229+0000 7f5c09e00640 1 -- 172.21.15.16:0/2008170661 shutdown_connections 2024-09-10T21:17:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.229+0000 7f5c09e00640 1 -- 172.21.15.16:0/2008170661 >> 172.21.15.16:0/2008170661 conn(0x7f5c040d1d80 msgr2=0x7f5c040d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.229+0000 7f5c09e00640 1 -- 172.21.15.16:0/2008170661 shutdown_connections 2024-09-10T21:17:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.229+0000 7f5c09e00640 1 -- 172.21.15.16:0/2008170661 wait complete. 2024-09-10T21:17:36.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.229+0000 7f5c09e00640 1 Processor -- start 2024-09-10T21:17:36.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.229+0000 7f5c09e00640 1 -- start start 2024-09-10T21:17:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5c09e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5be8003a60 0x7f5c041f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5c09e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c040b6b70 0x7f5c0414be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5c09e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5c040b6fd0 0x7f5c0414c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5c09e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c040b2e30 con 0x7f5c040b6b70 2024-09-10T21:17:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5c09e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5c040b2cb0 con 0x7f5be8003a60 2024-09-10T21:17:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5c09e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f5c040b2fb0 con 0x7f5c040b6fd0 2024-09-10T21:17:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5bfbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c040b6b70 0x7f5c0414be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5bfbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c040b6b70 0x7f5c0414be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37904/0 (socket says 172.21.15.16:37904) 2024-09-10T21:17:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5bfbe00640 1 -- 172.21.15.16:0/3756041970 learned_addr learned my addr 172.21.15.16:0/3756041970 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5c02a00640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5be8003a60 0x7f5c041f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5c03400640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5c040b6fd0 0x7f5c0414c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5bfbe00640 1 -- 172.21.15.16:0/3756041970 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5c040b6fd0 msgr2=0x7f5c0414c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5bfbe00640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5c040b6fd0 0x7f5c0414c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5bfbe00640 1 -- 172.21.15.16:0/3756041970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5be8003a60 msgr2=0x7f5c041f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5bfbe00640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5be8003a60 0x7f5c041f8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5bfbe00640 1 -- 172.21.15.16:0/3756041970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c040b7510 con 0x7f5c040b6b70 2024-09-10T21:17:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5c02a00640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5be8003a60 0x7f5c041f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5c03400640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5c040b6fd0 0x7f5c0414c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5bfbe00640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c040b6b70 0x7f5c0414be20 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5bec011ae0 tx=0x7f5bec009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5bfb400640 1 -- 172.21.15.16:0/3756041970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec01e070 con 0x7f5c040b6b70 2024-09-10T21:17:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5bfb400640 1 -- 172.21.15.16:0/3756041970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bec00fe40 con 0x7f5c040b6b70 2024-09-10T21:17:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5bfb400640 1 -- 172.21.15.16:0/3756041970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0042b0 con 0x7f5c040b6b70 2024-09-10T21:17:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5c09e00640 1 -- 172.21.15.16:0/3756041970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c040b77f0 con 0x7f5c040b6b70 2024-09-10T21:17:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5c09e00640 1 -- 172.21.15.16:0/3756041970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c0414c8a0 con 0x7f5c040b6b70 2024-09-10T21:17:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5c09e00640 1 -- 172.21.15.16:0/3756041970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c040d0540 con 0x7f5c040b6b70 2024-09-10T21:17:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.233+0000 7f5bfb400640 1 -- 172.21.15.16:0/3756041970 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5bec003600 con 0x7f5c040b6b70 2024-09-10T21:17:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.237+0000 7f5bfb400640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5be0072610 0x7f5be0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.237+0000 7f5bfb400640 1 -- 172.21.15.16:0/3756041970 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f5bec093ce0 con 0x7f5c040b6b70 2024-09-10T21:17:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.237+0000 7f5c02a00640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5be0072610 0x7f5be0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.237+0000 7f5c02a00640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5be0072610 0x7f5be0074ad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40af000 tx=0x7f5bf401a050 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.241+0000 7f5bfb400640 1 -- 172.21.15.16:0/3756041970 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5bec05f120 con 0x7f5c040b6b70 2024-09-10T21:17:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.445+0000 7f5c09e00640 1 -- 172.21.15.16:0/3756041970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c040d2320 con 0x7f5c040b6b70 2024-09-10T21:17:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.445+0000 7f5bfb400640 1 -- 172.21.15.16:0/3756041970 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+3913 (secure 0 0 0) 0x7f5bec063b50 con 0x7f5c040b6b70 2024-09-10T21:17:36.453 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:36.453 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"btime":"2024-09-10T21:17:30:021031+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7101,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3765085271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3765085271},{"type":"v1","addr":"172.21.15.16:6836","nonce":3765085271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":7104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4198210360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":4198210360},{"type":"v1","addr":"172.21.15.16:6837","nonce":4198210360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/2954517211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":2954517211},{"type":"v1","addr":"172.21.15.52:6835","nonce":2954517211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":136124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/2168802523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2168802523},{"type":"v1","addr":"172.21.15.52:6834","nonce":2168802523}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[]} 2024-09-10T21:17:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 75 2024-09-10T21:17:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.449+0000 7f5c09e00640 1 -- 172.21.15.16:0/3756041970 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5be0072610 msgr2=0x7f5be0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.449+0000 7f5c09e00640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5be0072610 0x7f5be0074ad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40af000 tx=0x7f5bf401a050 comp rx=0 tx=0).stop 2024-09-10T21:17:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.449+0000 7f5c09e00640 1 -- 172.21.15.16:0/3756041970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c040b6b70 msgr2=0x7f5c0414be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.449+0000 7f5c09e00640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c040b6b70 0x7f5c0414be20 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5bec011ae0 tx=0x7f5bec009ea0 comp rx=0 tx=0).stop 2024-09-10T21:17:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.449+0000 7f5c09e00640 1 -- 172.21.15.16:0/3756041970 shutdown_connections 2024-09-10T21:17:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.449+0000 7f5c09e00640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f5be0072610 0x7f5be0074ad0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.449+0000 7f5c09e00640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f5c040b6fd0 0x7f5c0414c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.449+0000 7f5c09e00640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c040b6b70 0x7f5c0414be20 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.449+0000 7f5c09e00640 1 --2- 172.21.15.16:0/3756041970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5be8003a60 0x7f5c041f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.449+0000 7f5c09e00640 1 -- 172.21.15.16:0/3756041970 >> 172.21.15.16:0/3756041970 conn(0x7f5c040d1d80 msgr2=0x7f5c0406dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.449+0000 7f5c09e00640 1 -- 172.21.15.16:0/3756041970 shutdown_connections 2024-09-10T21:17:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.449+0000 7f5c09e00640 1 -- 172.21.15.16:0/3756041970 wait complete. 2024-09-10T21:17:36.472 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:17:36.473 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:17:36.473 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:17:36.473 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:17:36.473 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:17:36.473 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:17:36.473 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:17:36.474 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:17:36.474 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:17:36.474 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:17:36.474 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:17:36.474 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:17:36.474 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:17:36.474 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:17:36.475 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:17:36.475 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:17:36.486 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:17:36.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:17:36.487 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:17:36.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:17:36.490 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:17:36.490 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:17:36.491 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:17:36.492 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:17:36.534 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:17:36.534 DEBUG:teuthology.parallel:result is None 2024-09-10T21:17:36.535 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:17:36.535 DEBUG:teuthology.parallel:result is None 2024-09-10T21:17:36.538 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:17:36.538 DEBUG:teuthology.parallel:result is None 2024-09-10T21:17:36.540 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:17:36.540 DEBUG:teuthology.parallel:result is None 2024-09-10T21:17:36.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:17:36.621 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-10T21:17:36.621 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-10T21:17:36.635 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:17:36.635 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:17:36.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 -- 172.21.15.16:0/830593833 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3d0015be10 msgr2=0x7f3d000cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:36.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 --2- 172.21.15.16:0/830593833 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3d0015be10 0x7f3d000cbe90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009bf0 tx=0x7f3cf001cb50 comp rx=0 tx=0).stop 2024-09-10T21:17:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 -- 172.21.15.16:0/830593833 shutdown_connections 2024-09-10T21:17:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 --2- 172.21.15.16:0/830593833 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3d0015be10 0x7f3d000cbe90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 -- 172.21.15.16:0/830593833 >> 172.21.15.16:0/830593833 conn(0x7f3d000d0c30 msgr2=0x7f3d000d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 -- 172.21.15.16:0/830593833 shutdown_connections 2024-09-10T21:17:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 -- 172.21.15.16:0/830593833 wait complete. 2024-09-10T21:17:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 Processor -- start 2024-09-10T21:17:36.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 -- start start 2024-09-10T21:17:36.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d000c42c0 0x7f3d000c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:36.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d000cd070 0x7f3d000ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:36.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3d0015be10 0x7f3d001fb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0016a4c0 con 0x7f3d000c42c0 2024-09-10T21:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0016a340 con 0x7f3d000cd070 2024-09-10T21:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3d05e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0016a640 con 0x7f3d0015be10 2024-09-10T21:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3cfea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d000c42c0 0x7f3d000c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3cfea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d000c42c0 0x7f3d000c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37938/0 (socket says 172.21.15.16:37938) 2024-09-10T21:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3cfea00640 1 -- 172.21.15.16:0/517799982 learned_addr learned my addr 172.21.15.16:0/517799982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3cf6000640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d000cd070 0x7f3d000ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.689+0000 7f3cff400640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3d0015be10 0x7f3d001fb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cfea00640 1 -- 172.21.15.16:0/517799982 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3d0015be10 msgr2=0x7f3d001fb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cfea00640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3d0015be10 0x7f3d001fb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cfea00640 1 -- 172.21.15.16:0/517799982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d000cd070 msgr2=0x7f3d000ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cfea00640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d000cd070 0x7f3d000ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:36.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cfea00640 1 -- 172.21.15.16:0/517799982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d000b1a30 con 0x7f3d000c42c0 2024-09-10T21:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cff400640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3d0015be10 0x7f3d001fb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cf6000640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d000cd070 0x7f3d000ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cfea00640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d000c42c0 0x7f3d000c9ed0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f3cf001a780 tx=0x7f3cf0009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cf7e00640 1 -- 172.21.15.16:0/517799982 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf00bb070 con 0x7f3d000c42c0 2024-09-10T21:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cf7e00640 1 -- 172.21.15.16:0/517799982 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cf0004d40 con 0x7f3d000c42c0 2024-09-10T21:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cf7e00640 1 -- 172.21.15.16:0/517799982 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf00b2040 con 0x7f3d000c42c0 2024-09-10T21:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3d05e00640 1 -- 172.21.15.16:0/517799982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d00156db0 con 0x7f3d000c42c0 2024-09-10T21:17:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3d05e00640 1 -- 172.21.15.16:0/517799982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d000cf4e0 con 0x7f3d000c42c0 2024-09-10T21:17:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3d05e00640 1 -- 172.21.15.16:0/517799982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d000cbe90 con 0x7f3d000c42c0 2024-09-10T21:17:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cf7e00640 1 -- 172.21.15.16:0/517799982 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3cf00b5030 con 0x7f3d000c42c0 2024-09-10T21:17:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cf7e00640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3cdc0725e0 0x7f3cdc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cf7e00640 1 -- 172.21.15.16:0/517799982 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f3cf012cb60 con 0x7f3d000c42c0 2024-09-10T21:17:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.693+0000 7f3cf6000640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3cdc0725e0 0x7f3cdc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.697+0000 7f3cf6000640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3cdc0725e0 0x7f3cdc074aa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3d001fb3b0 tx=0x7f3ce8009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:36.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.697+0000 7f3cf7e00640 1 -- 172.21.15.16:0/517799982 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3cf00167d0 con 0x7f3d000c42c0 2024-09-10T21:17:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:36.881+0000 7f3d05e00640 1 -- 172.21.15.16:0/517799982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3d000a1310 con 0x7f3d000c42c0 2024-09-10T21:17:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.057+0000 7f3cf7e00640 1 -- 172.21.15.16:0/517799982 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v110) ==== 97+0+0 (secure 0 0 0) 0x7f3cf00f84a0 con 0x7f3d000c42c0 2024-09-10T21:17:37.063 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-10T21:17:37.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.061+0000 7f3d05e00640 1 -- 172.21.15.16:0/517799982 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3cdc0725e0 msgr2=0x7f3cdc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:37.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.061+0000 7f3d05e00640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3cdc0725e0 0x7f3cdc074aa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3d001fb3b0 tx=0x7f3ce8009040 comp rx=0 tx=0).stop 2024-09-10T21:17:37.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.061+0000 7f3d05e00640 1 -- 172.21.15.16:0/517799982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d000c42c0 msgr2=0x7f3d000c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:37.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.061+0000 7f3d05e00640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d000c42c0 0x7f3d000c9ed0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f3cf001a780 tx=0x7f3cf0009200 comp rx=0 tx=0).stop 2024-09-10T21:17:37.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.061+0000 7f3d05e00640 1 -- 172.21.15.16:0/517799982 shutdown_connections 2024-09-10T21:17:37.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.061+0000 7f3d05e00640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f3cdc0725e0 0x7f3cdc074aa0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.061+0000 7f3d05e00640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f3d0015be10 0x7f3d001fb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.061+0000 7f3d05e00640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d000cd070 0x7f3d000ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.061+0000 7f3d05e00640 1 --2- 172.21.15.16:0/517799982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d000c42c0 0x7f3d000c9ed0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.061+0000 7f3d05e00640 1 -- 172.21.15.16:0/517799982 >> 172.21.15.16:0/517799982 conn(0x7f3d000d0c30 msgr2=0x7f3d000737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:37.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.061+0000 7f3d05e00640 1 -- 172.21.15.16:0/517799982 shutdown_connections 2024-09-10T21:17:37.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.061+0000 7f3d05e00640 1 -- 172.21.15.16:0/517799982 wait complete. 2024-09-10T21:17:37.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:17:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 -- 172.21.15.16:0/1095733404 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb8040d0890 msgr2=0x7fb8040b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 --2- 172.21.15.16:0/1095733404 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb8040d0890 0x7fb8040b0020 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009c80 tx=0x7fb7fc01cda0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 -- 172.21.15.16:0/1095733404 shutdown_connections 2024-09-10T21:17:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 --2- 172.21.15.16:0/1095733404 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb8040d0890 0x7fb8040b0020 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 --2- 172.21.15.16:0/1095733404 >> v2:172.21.15.52:3300/0 conn(0x7fb8040b3200 0x7fb8040afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 -- 172.21.15.16:0/1095733404 >> 172.21.15.16:0/1095733404 conn(0x7fb8040d1d80 msgr2=0x7fb8040d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 -- 172.21.15.16:0/1095733404 shutdown_connections 2024-09-10T21:17:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 -- 172.21.15.16:0/1095733404 wait complete. 2024-09-10T21:17:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 Processor -- start 2024-09-10T21:17:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 -- start start 2024-09-10T21:17:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb8040d0890 0x7fb80414e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb80415f450 0x7fb80414f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb80414c060 0x7fb80414c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb804168d50 con 0x7fb80414c060 2024-09-10T21:17:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb804168bd0 con 0x7fb80415f450 2024-09-10T21:17:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb809200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb804168ed0 con 0x7fb8040d0890 2024-09-10T21:17:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb802800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb80414c060 0x7fb80414c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb802800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb80414c060 0x7fb80414c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37950/0 (socket says 172.21.15.16:37950) 2024-09-10T21:17:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb802800640 1 -- 172.21.15.16:0/3130219350 learned_addr learned my addr 172.21.15.16:0/3130219350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb7fbe00640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb80415f450 0x7fb80414f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb801e00640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb8040d0890 0x7fb80414e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb802800640 1 -- 172.21.15.16:0/3130219350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb8040d0890 msgr2=0x7fb80414e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb802800640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb8040d0890 0x7fb80414e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb802800640 1 -- 172.21.15.16:0/3130219350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb80415f450 msgr2=0x7fb80414f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb802800640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb80415f450 0x7fb80414f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.233+0000 7fb802800640 1 -- 172.21.15.16:0/3130219350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8040b11d0 con 0x7fb80414c060 2024-09-10T21:17:37.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.237+0000 7fb802800640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb80414c060 0x7fb80414c8e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009bd0 tx=0x7fb7fc005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:37.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.237+0000 7fb7fa000640 1 -- 172.21.15.16:0/3130219350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0b2040 con 0x7fb80414c060 2024-09-10T21:17:37.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.237+0000 7fb7fa000640 1 -- 172.21.15.16:0/3130219350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7fc004040 con 0x7fb80414c060 2024-09-10T21:17:37.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.237+0000 7fb7fa000640 1 -- 172.21.15.16:0/3130219350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0043f0 con 0x7fb80414c060 2024-09-10T21:17:37.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.237+0000 7fb809200640 1 -- 172.21.15.16:0/3130219350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80414ea70 con 0x7fb80414c060 2024-09-10T21:17:37.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.237+0000 7fb809200640 1 -- 172.21.15.16:0/3130219350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8040c8f80 con 0x7fb80414c060 2024-09-10T21:17:37.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.237+0000 7fb809200640 1 -- 172.21.15.16:0/3130219350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8040cfc80 con 0x7fb80414c060 2024-09-10T21:17:37.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.237+0000 7fb7fa000640 1 -- 172.21.15.16:0/3130219350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb7fc0b5070 con 0x7fb80414c060 2024-09-10T21:17:37.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.237+0000 7fb7fa000640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7e0072610 0x7fb7e0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:37.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.237+0000 7fb7fa000640 1 -- 172.21.15.16:0/3130219350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4857+0+0 (secure 0 0 0) 0x7fb7fc12cb70 con 0x7fb80414c060 2024-09-10T21:17:37.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.237+0000 7fb801e00640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7e0072610 0x7fb7e0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:37.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.241+0000 7fb801e00640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7e0072610 0x7fb7e0074ad0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb8040af7d0 tx=0x7fb7f4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:37.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.241+0000 7fb7fa000640 1 -- 172.21.15.16:0/3130219350 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb7fc016990 con 0x7fb80414c060 2024-09-10T21:17:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.477+0000 7fb809200640 1 -- 172.21.15.16:0/3130219350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb804151d30 con 0x7fb80414c060 2024-09-10T21:17:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.477+0000 7fb7fa000640 1 -- 172.21.15.16:0/3130219350 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v28) ==== 80+0+5304 (secure 0 0 0) 0x7fb7fc0f7f40 con 0x7fb80414c060 2024-09-10T21:17:37.482 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:37.483 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:17:37.483 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-10T21:17:37.483 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.483 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-10T21:17:37.483 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmhg+lJBAAWjuYW75BGo2dmyAUi7s9+A==", 2024-09-10T21:17:37.483 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.483 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:37.483 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:17:37.483 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:17:37.483 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.484 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.484 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.484 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-10T21:17:37.484 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmQr7+MBAAOzXhWIirBOoi/7wKnrFgmg==", 2024-09-10T21:17:37.484 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.484 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:37.484 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:17:37.484 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:17:37.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.485 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.485 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.485 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-10T21:17:37.485 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmxxoALBAAgkmw91/VP1Uv2WMZsJKAyA==", 2024-09-10T21:17:37.485 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.485 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:37.485 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:17:37.485 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:17:37.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.485 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.486 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.486 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-10T21:17:37.486 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmA4tzOBAAiS+8n+Gokz6yXapVf2ecYg==", 2024-09-10T21:17:37.486 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.486 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:37.486 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:17:37.486 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:17:37.486 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.486 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.486 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.487 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-10T21:17:37.487 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHteBmRoahLRAA32PsUkqtGkeI7tMFRwas5Q==", 2024-09-10T21:17:37.487 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.487 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:37.487 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:37.487 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.487 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.487 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.487 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-10T21:17:37.487 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCLteBmGmEMBhAAr1yQEf/NkiN4+ezAQ9VR2A==", 2024-09-10T21:17:37.488 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.488 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:37.488 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:37.488 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.488 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.488 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.488 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-10T21:17:37.488 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCOteBmmMuAHBAAt+G/KApsZymjTLRSmyRbdw==", 2024-09-10T21:17:37.488 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.488 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:37.488 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:37.489 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.489 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.489 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.489 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-10T21:17:37.489 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCRteBmoQfNMBAA0vnIqlYoI13KNAAfNmJg3w==", 2024-09-10T21:17:37.489 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.489 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:37.489 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:37.489 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.489 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.490 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.490 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-10T21:17:37.490 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCWteBmc/7lDxAADlSi4UpcFJVNdp6YTkpdSw==", 2024-09-10T21:17:37.490 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.490 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:37.490 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:37.490 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.490 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.490 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.490 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-10T21:17:37.490 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZteBmBZ1LJhAAPcbZa1LQu4kNbSsJ/8Z81A==", 2024-09-10T21:17:37.490 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCcteBm7/czOxAAqREnY84+LF1N9dRXOhzz6g==", 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.492 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.492 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-10T21:17:37.492 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgteBmG2J8FRAAB+cMFCnf9wJg1R3RD5SUJw==", 2024-09-10T21:17:37.492 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.492 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:17:37.492 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:17:37.492 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.492 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.492 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.492 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-10T21:17:37.493 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA==", 2024-09-10T21:17:37.493 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.493 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:37.493 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-10T21:17:37.493 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:17:37.493 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.493 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.493 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-10T21:17:37.494 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmuEm4ChAAe08QvhwIcN8ZHa3AoVcYww==", 2024-09-10T21:17:37.494 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.494 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:37.494 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:17:37.494 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:17:37.494 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:17:37.494 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.494 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.494 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-10T21:17:37.494 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBm50o1BxAAIFbVxVqU4rU8eKYpuZjTHA==", 2024-09-10T21:17:37.494 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.494 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:37.495 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:17:37.495 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:17:37.495 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:17:37.495 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.495 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.495 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-10T21:17:37.495 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmxtu0EBAAmN7+07lDiwjsIC/pNYUlSw==", 2024-09-10T21:17:37.495 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.495 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:17:37.495 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:17:37.495 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:17:37.496 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:17:37.496 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.496 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.496 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-10T21:17:37.496 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmfqgYCBAAm00tBe45phGieCcp1q46cw==", 2024-09-10T21:17:37.496 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.496 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:17:37.496 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:17:37.496 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:17:37.496 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.497 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.497 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.497 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-10T21:17:37.497 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmMVABIBAAoB72mPyXyUO6Y6k+fMqQQg==", 2024-09-10T21:17:37.497 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.497 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:17:37.497 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:17:37.497 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.497 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:17:37.497 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.497 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-10T21:17:37.498 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmeyYxHBAAvBkpTzjPQKAOlHzI1jQ46g==", 2024-09-10T21:17:37.498 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:17:37.498 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:17:37.498 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:17:37.498 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:17:37.498 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.498 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.498 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-10T21:17:37.498 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:17:37.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.481+0000 7fb809200640 1 -- 172.21.15.16:0/3130219350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7e0072610 msgr2=0x7fb7e0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:37.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.481+0000 7fb809200640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7e0072610 0x7fb7e0074ad0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb8040af7d0 tx=0x7fb7f4009040 comp rx=0 tx=0).stop 2024-09-10T21:17:37.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.481+0000 7fb809200640 1 -- 172.21.15.16:0/3130219350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb80414c060 msgr2=0x7fb80414c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:37.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.481+0000 7fb809200640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb80414c060 0x7fb80414c8e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009bd0 tx=0x7fb7fc005c20 comp rx=0 tx=0).stop 2024-09-10T21:17:37.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.481+0000 7fb809200640 1 -- 172.21.15.16:0/3130219350 shutdown_connections 2024-09-10T21:17:37.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.481+0000 7fb809200640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7e0072610 0x7fb7e0074ad0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.481+0000 7fb809200640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb80414c060 0x7fb80414c8e0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.481+0000 7fb809200640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb80415f450 0x7fb80414f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.481+0000 7fb809200640 1 --2- 172.21.15.16:0/3130219350 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb8040d0890 0x7fb80414e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.481+0000 7fb809200640 1 -- 172.21.15.16:0/3130219350 >> 172.21.15.16:0/3130219350 conn(0x7fb8040d1d80 msgr2=0x7fb804071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.481+0000 7fb809200640 1 -- 172.21.15.16:0/3130219350 shutdown_connections 2024-09-10T21:17:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.481+0000 7fb809200640 1 -- 172.21.15.16:0/3130219350 wait complete. 2024-09-10T21:17:37.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:17:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.645+0000 7fd0df800640 1 -- 172.21.15.16:0/323727111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d80d06b0 msgr2=0x7fd0d815a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.645+0000 7fd0df800640 1 --2- 172.21.15.16:0/323727111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d80d06b0 0x7fd0d815a270 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009840 tx=0x7fd0cc01c6b0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0df800640 1 -- 172.21.15.16:0/323727111 shutdown_connections 2024-09-10T21:17:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0df800640 1 --2- 172.21.15.16:0/323727111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d80d06b0 0x7fd0d815a270 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0df800640 1 -- 172.21.15.16:0/323727111 >> 172.21.15.16:0/323727111 conn(0x7fd0d80cfb20 msgr2=0x7fd0d80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0df800640 1 -- 172.21.15.16:0/323727111 shutdown_connections 2024-09-10T21:17:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0df800640 1 -- 172.21.15.16:0/323727111 wait complete. 2024-09-10T21:17:37.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0df800640 1 Processor -- start 2024-09-10T21:17:37.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0df800640 1 -- start start 2024-09-10T21:17:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0df800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0d80700a0 0x7fd0d814d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0df800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d80ce610 0x7fd0d8163390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0df800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d80d06b0 0x7fd0d814dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0df800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d8070660 con 0x7fd0d80d06b0 2024-09-10T21:17:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0df800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d80704e0 con 0x7fd0d80ce610 2024-09-10T21:17:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0df800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d80707e0 con 0x7fd0d80700a0 2024-09-10T21:17:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0dd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d80d06b0 0x7fd0d814dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0dd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d80d06b0 0x7fd0d814dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37960/0 (socket says 172.21.15.16:37960) 2024-09-10T21:17:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0dd800640 1 -- 172.21.15.16:0/927600789 learned_addr learned my addr 172.21.15.16:0/927600789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0dce00640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0d80700a0 0x7fd0d814d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0d7e00640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d80ce610 0x7fd0d8163390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:37.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0dd800640 1 -- 172.21.15.16:0/927600789 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0d80700a0 msgr2=0x7fd0d814d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0dd800640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0d80700a0 0x7fd0d814d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0dd800640 1 -- 172.21.15.16:0/927600789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d80ce610 msgr2=0x7fd0d8163390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0dd800640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d80ce610 0x7fd0d8163390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.649+0000 7fd0dd800640 1 -- 172.21.15.16:0/927600789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d80740c0 con 0x7fd0d80d06b0 2024-09-10T21:17:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.653+0000 7fd0dd800640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d80d06b0 0x7fd0d814dc90 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000cb90 tx=0x7fd0d000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.653+0000 7fd0d4c00640 1 -- 172.21.15.16:0/927600789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0016020 con 0x7fd0d80d06b0 2024-09-10T21:17:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.653+0000 7fd0d4c00640 1 -- 172.21.15.16:0/927600789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0d000ab40 con 0x7fd0d80d06b0 2024-09-10T21:17:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.653+0000 7fd0df800640 1 -- 172.21.15.16:0/927600789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d8074360 con 0x7fd0d80d06b0 2024-09-10T21:17:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.653+0000 7fd0d4c00640 1 -- 172.21.15.16:0/927600789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0012420 con 0x7fd0d80d06b0 2024-09-10T21:17:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.653+0000 7fd0df800640 1 -- 172.21.15.16:0/927600789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d8155100 con 0x7fd0d80d06b0 2024-09-10T21:17:37.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.653+0000 7fd0df800640 1 -- 172.21.15.16:0/927600789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d815a270 con 0x7fd0d80d06b0 2024-09-10T21:17:37.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.653+0000 7fd0d4c00640 1 -- 172.21.15.16:0/927600789 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd0d001b400 con 0x7fd0d80d06b0 2024-09-10T21:17:37.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.653+0000 7fd0d4c00640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd0b80725e0 0x7fd0b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:37.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.657+0000 7fd0d4c00640 1 -- 172.21.15.16:0/927600789 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4857+0+0 (secure 0 0 0) 0x7fd0d0093490 con 0x7fd0d80d06b0 2024-09-10T21:17:37.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.657+0000 7fd0dce00640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd0b80725e0 0x7fd0b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.657+0000 7fd0dce00640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd0b80725e0 0x7fd0b8074aa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0af000 tx=0x7fd0cc01aef0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:37.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.661+0000 7fd0d4c00640 1 -- 172.21.15.16:0/927600789 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd0d0020020 con 0x7fd0d80d06b0 2024-09-10T21:17:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.837+0000 7fd0df800640 1 -- 172.21.15.16:0/927600789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd0d8074530 con 0x7fd0d80d06b0 2024-09-10T21:17:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.837+0000 7fd0d4c00640 1 -- 172.21.15.16:0/927600789 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v112) ==== 84+0+65 (secure 0 0 0) 0x7fd0d005e7d0 con 0x7fd0d80d06b0 2024-09-10T21:17:37.842 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:37.842 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-10T21:17:37.842 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:17:37.843 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-10T21:17:37.843 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-10T21:17:37.843 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:17:37.843 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-10T21:17:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.841+0000 7fd0df800640 1 -- 172.21.15.16:0/927600789 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd0b80725e0 msgr2=0x7fd0b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.841+0000 7fd0df800640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd0b80725e0 0x7fd0b8074aa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0af000 tx=0x7fd0cc01aef0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.841+0000 7fd0df800640 1 -- 172.21.15.16:0/927600789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d80d06b0 msgr2=0x7fd0d814dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.841+0000 7fd0df800640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d80d06b0 0x7fd0d814dc90 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000cb90 tx=0x7fd0d000cc80 comp rx=0 tx=0).stop 2024-09-10T21:17:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.841+0000 7fd0df800640 1 -- 172.21.15.16:0/927600789 shutdown_connections 2024-09-10T21:17:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.841+0000 7fd0df800640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd0b80725e0 0x7fd0b8074aa0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.841+0000 7fd0df800640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0d80d06b0 0x7fd0d814dc90 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.841+0000 7fd0df800640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d80ce610 0x7fd0d8163390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.841+0000 7fd0df800640 1 --2- 172.21.15.16:0/927600789 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd0d80700a0 0x7fd0d814d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.841+0000 7fd0df800640 1 -- 172.21.15.16:0/927600789 >> 172.21.15.16:0/927600789 conn(0x7fd0d80cfb20 msgr2=0x7fd0d80cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.841+0000 7fd0df800640 1 -- 172.21.15.16:0/927600789 shutdown_connections 2024-09-10T21:17:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:37.841+0000 7fd0df800640 1 -- 172.21.15.16:0/927600789 wait complete. 2024-09-10T21:17:37.863 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:17:37.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:17:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.009+0000 7f907c000640 1 -- 172.21.15.16:0/2525660139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9074072710 msgr2=0x7f90740b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.009+0000 7f907c000640 1 --2- 172.21.15.16:0/2525660139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9074072710 0x7f90740b2e40 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f90740b28a0 tx=0x7f906001c8a0 comp rx=0 tx=0).stop 2024-09-10T21:17:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.009+0000 7f907c000640 1 -- 172.21.15.16:0/2525660139 shutdown_connections 2024-09-10T21:17:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.009+0000 7f907c000640 1 --2- 172.21.15.16:0/2525660139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9074072710 0x7f90740b2e40 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.009+0000 7f907c000640 1 --2- 172.21.15.16:0/2525660139 >> v2:172.21.15.52:3300/0 conn(0x7f907415bcc0 0x7f907415b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.009+0000 7f907c000640 1 -- 172.21.15.16:0/2525660139 >> 172.21.15.16:0/2525660139 conn(0x7f907406d1e0 msgr2=0x7f90740b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.009+0000 7f907c000640 1 -- 172.21.15.16:0/2525660139 shutdown_connections 2024-09-10T21:17:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.009+0000 7f907c000640 1 -- 172.21.15.16:0/2525660139 wait complete. 2024-09-10T21:17:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f907c000640 1 Processor -- start 2024-09-10T21:17:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f907c000640 1 -- start start 2024-09-10T21:17:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f907c000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9074072710 0x7f9074136190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f907c000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740b3380 0x7f9074152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f907c000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f907415bcc0 0x7f9074163c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f907c000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9074167b90 con 0x7f90740b3380 2024-09-10T21:17:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f907c000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9074072b80 con 0x7f907415bcc0 2024-09-10T21:17:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f907c000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9074167d10 con 0x7f9074072710 2024-09-10T21:17:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f9078c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740b3380 0x7f9074152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f9078c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740b3380 0x7f9074152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37992/0 (socket says 172.21.15.16:37992) 2024-09-10T21:17:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f9078c00640 1 -- 172.21.15.16:0/1755420654 learned_addr learned my addr 172.21.15.16:0/1755420654 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f9078c00640 1 -- 172.21.15.16:0/1755420654 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9074072710 msgr2=0x7f9074136190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f9078c00640 1 --2- 172.21.15.16:0/1755420654 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9074072710 0x7f9074136190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f9078c00640 1 -- 172.21.15.16:0/1755420654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f907415bcc0 msgr2=0x7f9074163c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f9078c00640 1 --2- 172.21.15.16:0/1755420654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f907415bcc0 0x7f9074163c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f9078c00640 1 -- 172.21.15.16:0/1755420654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9074164540 con 0x7f90740b3380 2024-09-10T21:17:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f9078c00640 1 --2- 172.21.15.16:0/1755420654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740b3380 0x7f9074152820 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f90600b0040 tx=0x7f90600a55b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f9071600640 1 -- 172.21.15.16:0/1755420654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90600b2040 con 0x7f90740b3380 2024-09-10T21:17:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f9071600640 1 -- 172.21.15.16:0/1755420654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9060004470 con 0x7f90740b3380 2024-09-10T21:17:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f9071600640 1 -- 172.21.15.16:0/1755420654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90600052d0 con 0x7f90740b3380 2024-09-10T21:17:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f907c000640 1 -- 172.21.15.16:0/1755420654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9074164820 con 0x7f90740b3380 2024-09-10T21:17:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f907c000640 1 -- 172.21.15.16:0/1755420654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90741542a0 con 0x7f90740b3380 2024-09-10T21:17:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.013+0000 7f907c000640 1 -- 172.21.15.16:0/1755420654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907406eaf0 con 0x7f90740b3380 2024-09-10T21:17:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.017+0000 7f9071600640 1 -- 172.21.15.16:0/1755420654 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9060005470 con 0x7f90740b3380 2024-09-10T21:17:38.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.017+0000 7f9071600640 1 --2- 172.21.15.16:0/1755420654 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9054072610 0x7f9054074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:38.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.017+0000 7f9071600640 1 -- 172.21.15.16:0/1755420654 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4857+0+0 (secure 0 0 0) 0x7f906012ca50 con 0x7f90740b3380 2024-09-10T21:17:38.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.017+0000 7f9079600640 1 --2- 172.21.15.16:0/1755420654 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9054072610 0x7f9054074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:38.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.017+0000 7f9079600640 1 --2- 172.21.15.16:0/1755420654 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9054072610 0x7f9054074ad0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f90680058d0 tx=0x7f9068005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:38.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.021+0000 7f9071600640 1 -- 172.21.15.16:0/1755420654 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9060016490 con 0x7f90740b3380 2024-09-10T21:17:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:38.197+0000 7f907c000640 1 -- 172.21.15.16:0/1755420654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f90740b2dc0 con 0x7f90740b3380 2024-09-10T21:17:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.073+0000 7f9071600640 1 -- 172.21.15.16:0/1755420654 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v113) ==== 138+0+0 (secure 0 0 0) 0x7f90600f83f0 con 0x7f90740b3380 2024-09-10T21:17:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-10T21:17:39.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.073+0000 7f907c000640 1 -- 172.21.15.16:0/1755420654 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9054072610 msgr2=0x7f9054074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:39.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.073+0000 7f907c000640 1 --2- 172.21.15.16:0/1755420654 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9054072610 0x7f9054074ad0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f90680058d0 tx=0x7f9068005860 comp rx=0 tx=0).stop 2024-09-10T21:17:39.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.073+0000 7f907c000640 1 -- 172.21.15.16:0/1755420654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740b3380 msgr2=0x7f9074152820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:39.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.073+0000 7f907c000640 1 --2- 172.21.15.16:0/1755420654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740b3380 0x7f9074152820 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f90600b0040 tx=0x7f90600a55b0 comp rx=0 tx=0).stop 2024-09-10T21:17:39.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.073+0000 7f907c000640 1 -- 172.21.15.16:0/1755420654 shutdown_connections 2024-09-10T21:17:39.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.073+0000 7f907c000640 1 --2- 172.21.15.16:0/1755420654 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9054072610 0x7f9054074ad0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:39.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.073+0000 7f907c000640 1 --2- 172.21.15.16:0/1755420654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f907415bcc0 0x7f9074163c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:39.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.073+0000 7f907c000640 1 --2- 172.21.15.16:0/1755420654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740b3380 0x7f9074152820 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:39.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.073+0000 7f907c000640 1 --2- 172.21.15.16:0/1755420654 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9074072710 0x7f9074136190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:39.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.073+0000 7f907c000640 1 -- 172.21.15.16:0/1755420654 >> 172.21.15.16:0/1755420654 conn(0x7f907406d1e0 msgr2=0x7f907414bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:39.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.073+0000 7f907c000640 1 -- 172.21.15.16:0/1755420654 shutdown_connections 2024-09-10T21:17:39.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.077+0000 7f907c000640 1 -- 172.21.15.16:0/1755420654 wait complete. 2024-09-10T21:17:39.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:17:39.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.265+0000 7fcbd7a00640 1 -- 172.21.15.16:0/430922382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd00cfe30 msgr2=0x7fcbd00ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:39.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.265+0000 7fcbd7a00640 1 --2- 172.21.15.16:0/430922382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd00cfe30 0x7fcbd00ce590 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009840 tx=0x7fcbc401f010 comp rx=0 tx=0).stop 2024-09-10T21:17:39.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.265+0000 7fcbd7a00640 1 -- 172.21.15.16:0/430922382 shutdown_connections 2024-09-10T21:17:39.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.265+0000 7fcbd7a00640 1 --2- 172.21.15.16:0/430922382 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcbd00d0210 0x7fcbd00cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:39.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.265+0000 7fcbd7a00640 1 --2- 172.21.15.16:0/430922382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd00cfe30 0x7fcbd00ce590 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:39.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.265+0000 7fcbd7a00640 1 -- 172.21.15.16:0/430922382 >> 172.21.15.16:0/430922382 conn(0x7fcbd00d1df0 msgr2=0x7fcbd00d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:39.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.265+0000 7fcbd7a00640 1 -- 172.21.15.16:0/430922382 shutdown_connections 2024-09-10T21:17:39.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.265+0000 7fcbd7a00640 1 -- 172.21.15.16:0/430922382 wait complete. 2024-09-10T21:17:39.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbd7a00640 1 Processor -- start 2024-09-10T21:17:39.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbd7a00640 1 -- start start 2024-09-10T21:17:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbd7a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcbd00ce040 0x7fcbd0147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbd7a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbd00cfe30 0x7fcbd00b6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbd7a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd00d0210 0x7fcbd00ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbd7a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd00b12e0 con 0x7fcbd00cfe30 2024-09-10T21:17:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbd7a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd00d2b20 con 0x7fcbd00d0210 2024-09-10T21:17:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbd7a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd00b1460 con 0x7fcbd00ce040 2024-09-10T21:17:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbcfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbd00cfe30 0x7fcbd00b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbcfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbd00cfe30 0x7fcbd00b6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37998/0 (socket says 172.21.15.16:37998) 2024-09-10T21:17:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbcfe00640 1 -- 172.21.15.16:0/1007796894 learned_addr learned my addr 172.21.15.16:0/1007796894 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbd5000640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcbd00ce040 0x7fcbd0147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbd5a00640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd00d0210 0x7fcbd00ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbcfe00640 1 -- 172.21.15.16:0/1007796894 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcbd00ce040 msgr2=0x7fcbd0147580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbcfe00640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcbd00ce040 0x7fcbd0147580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbcfe00640 1 -- 172.21.15.16:0/1007796894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd00d0210 msgr2=0x7fcbd00ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbcfe00640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd00d0210 0x7fcbd00ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbcfe00640 1 -- 172.21.15.16:0/1007796894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd01f9160 con 0x7fcbd00cfe30 2024-09-10T21:17:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbcfe00640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbd00cfe30 0x7fcbd00b6340 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00e970 tx=0x7fcbbc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbccc00640 1 -- 172.21.15.16:0/1007796894 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc00cd80 con 0x7fcbd00cfe30 2024-09-10T21:17:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbccc00640 1 -- 172.21.15.16:0/1007796894 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbbc004020 con 0x7fcbd00cfe30 2024-09-10T21:17:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbd7a00640 1 -- 172.21.15.16:0/1007796894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd00ca9d0 con 0x7fcbd00cfe30 2024-09-10T21:17:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbccc00640 1 -- 172.21.15.16:0/1007796894 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc004470 con 0x7fcbd00cfe30 2024-09-10T21:17:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbd7a00640 1 -- 172.21.15.16:0/1007796894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd01f88c0 con 0x7fcbd00cfe30 2024-09-10T21:17:39.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.269+0000 7fcbd7a00640 1 -- 172.21.15.16:0/1007796894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd00cf2b0 con 0x7fcbd00cfe30 2024-09-10T21:17:39.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.273+0000 7fcbccc00640 1 -- 172.21.15.16:0/1007796894 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcbbc003600 con 0x7fcbd00cfe30 2024-09-10T21:17:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.273+0000 7fcbccc00640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbb00725e0 0x7fcbb0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.273+0000 7fcbccc00640 1 -- 172.21.15.16:0/1007796894 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5227+0+0 (secure 0 0 0) 0x7fcbbc093b20 con 0x7fcbd00cfe30 2024-09-10T21:17:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.273+0000 7fcbd5000640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbb00725e0 0x7fcbb0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.277+0000 7fcbd5000640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbb00725e0 0x7fcbb0074aa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40b2000 tx=0x7fcbc401d850 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.277+0000 7fcbccc00640 1 -- 172.21.15.16:0/1007796894 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcbbc05ed80 con 0x7fcbd00cfe30 2024-09-10T21:17:39.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:39.453+0000 7fcbd7a00640 1 -- 172.21.15.16:0/1007796894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcbd00b4480 con 0x7fcbd00cfe30 2024-09-10T21:17:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.077+0000 7fcbccc00640 1 -- 172.21.15.16:0/1007796894 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v114) ==== 170+0+0 (secure 0 0 0) 0x7fcbbc0637b0 con 0x7fcbd00cfe30 2024-09-10T21:17:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-10T21:17:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.081+0000 7fcbd7a00640 1 -- 172.21.15.16:0/1007796894 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbb00725e0 msgr2=0x7fcbb0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.081+0000 7fcbd7a00640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbb00725e0 0x7fcbb0074aa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40b2000 tx=0x7fcbc401d850 comp rx=0 tx=0).stop 2024-09-10T21:17:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.081+0000 7fcbd7a00640 1 -- 172.21.15.16:0/1007796894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbd00cfe30 msgr2=0x7fcbd00b6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.081+0000 7fcbd7a00640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbd00cfe30 0x7fcbd00b6340 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00e970 tx=0x7fcbbc00ee30 comp rx=0 tx=0).stop 2024-09-10T21:17:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.081+0000 7fcbd7a00640 1 -- 172.21.15.16:0/1007796894 shutdown_connections 2024-09-10T21:17:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.081+0000 7fcbd7a00640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbb00725e0 0x7fcbb0074aa0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.081+0000 7fcbd7a00640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd00d0210 0x7fcbd00ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.081+0000 7fcbd7a00640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbd00cfe30 0x7fcbd00b6340 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.081+0000 7fcbd7a00640 1 --2- 172.21.15.16:0/1007796894 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcbd00ce040 0x7fcbd0147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.081+0000 7fcbd7a00640 1 -- 172.21.15.16:0/1007796894 >> 172.21.15.16:0/1007796894 conn(0x7fcbd00d1df0 msgr2=0x7fcbd00aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.085+0000 7fcbd7a00640 1 -- 172.21.15.16:0/1007796894 shutdown_connections 2024-09-10T21:17:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.085+0000 7fcbd7a00640 1 -- 172.21.15.16:0/1007796894 wait complete. 2024-09-10T21:17:40.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:17:40.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 -- 172.21.15.16:0/21686141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46580ce8c0 msgr2=0x7f46580ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:40.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 --2- 172.21.15.16:0/21686141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46580ce8c0 0x7f46580ceca0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f4648009900 tx=0x7f464801c8b0 comp rx=0 tx=0).stop 2024-09-10T21:17:40.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 -- 172.21.15.16:0/21686141 shutdown_connections 2024-09-10T21:17:40.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 --2- 172.21.15.16:0/21686141 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f46580b0980 0x7f46580ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:40.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 --2- 172.21.15.16:0/21686141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46580ce8c0 0x7f46580ceca0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:40.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 -- 172.21.15.16:0/21686141 >> 172.21.15.16:0/21686141 conn(0x7f46580d1c70 msgr2=0x7f46580d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:40.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 -- 172.21.15.16:0/21686141 shutdown_connections 2024-09-10T21:17:40.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 -- 172.21.15.16:0/21686141 wait complete. 2024-09-10T21:17:40.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 Processor -- start 2024-09-10T21:17:40.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 -- start start 2024-09-10T21:17:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f46580b0980 0x7f465806df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46580ce8c0 0x7f46580b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46580cf1e0 0x7f46580b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4658169300 con 0x7f46580ce8c0 2024-09-10T21:17:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4658169180 con 0x7f46580cf1e0 2024-09-10T21:17:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465d800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4658169480 con 0x7f46580b0980 2024-09-10T21:17:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f464fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46580ce8c0 0x7f46580b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f464fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46580ce8c0 0x7f46580b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38016/0 (socket says 172.21.15.16:38016) 2024-09-10T21:17:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f464fe00640 1 -- 172.21.15.16:0/2368735471 learned_addr learned my addr 172.21.15.16:0/2368735471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f4657e00640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f46580b0980 0x7f465806df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f465ce00640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46580cf1e0 0x7f46580b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f464fe00640 1 -- 172.21.15.16:0/2368735471 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f46580b0980 msgr2=0x7f465806df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f464fe00640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f46580b0980 0x7f465806df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f464fe00640 1 -- 172.21.15.16:0/2368735471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46580cf1e0 msgr2=0x7f46580b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.273+0000 7f464fe00640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46580cf1e0 0x7f46580b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.277+0000 7f464fe00640 1 -- 172.21.15.16:0/2368735471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4658154640 con 0x7f46580ce8c0 2024-09-10T21:17:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.277+0000 7f4657e00640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f46580b0980 0x7f465806df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:40.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.277+0000 7f464fe00640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46580ce8c0 0x7f46580b6c20 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f464000ea20 tx=0x7f464000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:40.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.277+0000 7f4654c00640 1 -- 172.21.15.16:0/2368735471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464000cd80 con 0x7f46580ce8c0 2024-09-10T21:17:40.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.277+0000 7f4654c00640 1 -- 172.21.15.16:0/2368735471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4640010430 con 0x7f46580ce8c0 2024-09-10T21:17:40.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.277+0000 7f4654c00640 1 -- 172.21.15.16:0/2368735471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640010860 con 0x7f46580ce8c0 2024-09-10T21:17:40.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.277+0000 7f465d800640 1 -- 172.21.15.16:0/2368735471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46580b78a0 con 0x7f46580ce8c0 2024-09-10T21:17:40.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.277+0000 7f465d800640 1 -- 172.21.15.16:0/2368735471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46581fb780 con 0x7f46580ce8c0 2024-09-10T21:17:40.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.277+0000 7f465d800640 1 -- 172.21.15.16:0/2368735471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46580ceca0 con 0x7f46580ce8c0 2024-09-10T21:17:40.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.277+0000 7f4654c00640 1 -- 172.21.15.16:0/2368735471 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4640010a00 con 0x7f46580ce8c0 2024-09-10T21:17:40.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.277+0000 7f4654c00640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46340725e0 0x7f4634074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:40.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.277+0000 7f4654c00640 1 -- 172.21.15.16:0/2368735471 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5609+0+0 (secure 0 0 0) 0x7f4640093db0 con 0x7f46580ce8c0 2024-09-10T21:17:40.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.281+0000 7f4657e00640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46340725e0 0x7f4634074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:40.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.281+0000 7f4657e00640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46340725e0 0x7f4634074aa0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f46480af000 tx=0x7f464801b0f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:40.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.285+0000 7f4654c00640 1 -- 172.21.15.16:0/2368735471 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4640098070 con 0x7f46580ce8c0 2024-09-10T21:17:40.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:40.505+0000 7f465d800640 1 -- 172.21.15.16:0/2368735471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4658157f60 con 0x7f46580ce8c0 2024-09-10T21:17:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.081+0000 7f4654c00640 1 -- 172.21.15.16:0/2368735471 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v78) ==== 174+0+0 (secure 0 0 0) 0x7f464005f4e0 con 0x7f46580ce8c0 2024-09-10T21:17:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 18 and data pool 19 2024-09-10T21:17:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.085+0000 7f465d800640 1 -- 172.21.15.16:0/2368735471 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46340725e0 msgr2=0x7f4634074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.085+0000 7f465d800640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46340725e0 0x7f4634074aa0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f46480af000 tx=0x7f464801b0f0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.085+0000 7f465d800640 1 -- 172.21.15.16:0/2368735471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46580ce8c0 msgr2=0x7f46580b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.085+0000 7f465d800640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46580ce8c0 0x7f46580b6c20 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f464000ea20 tx=0x7f464000eee0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.085+0000 7f465d800640 1 -- 172.21.15.16:0/2368735471 shutdown_connections 2024-09-10T21:17:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.085+0000 7f465d800640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46340725e0 0x7f4634074aa0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.085+0000 7f465d800640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46580cf1e0 0x7f46580b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.085+0000 7f465d800640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46580ce8c0 0x7f46580b6c20 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.085+0000 7f465d800640 1 --2- 172.21.15.16:0/2368735471 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f46580b0980 0x7f465806df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.085+0000 7f465d800640 1 -- 172.21.15.16:0/2368735471 >> 172.21.15.16:0/2368735471 conn(0x7f46580d1c70 msgr2=0x7f465806f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:41.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.085+0000 7f465d800640 1 -- 172.21.15.16:0/2368735471 shutdown_connections 2024-09-10T21:17:41.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.085+0000 7f465d800640 1 -- 172.21.15.16:0/2368735471 wait complete. 2024-09-10T21:17:41.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:17:41.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 -- 172.21.15.16:0/3178388947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815a270 msgr2=0x7f25c806e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:41.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 --2- 172.21.15.16:0/3178388947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815a270 0x7f25c806e640 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009840 tx=0x7f25b001c6b0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 -- 172.21.15.16:0/3178388947 shutdown_connections 2024-09-10T21:17:41.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 --2- 172.21.15.16:0/3178388947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815a270 0x7f25c806e640 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 -- 172.21.15.16:0/3178388947 >> 172.21.15.16:0/3178388947 conn(0x7f25c80cfb20 msgr2=0x7f25c80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:41.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 -- 172.21.15.16:0/3178388947 shutdown_connections 2024-09-10T21:17:41.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 -- 172.21.15.16:0/3178388947 wait complete. 2024-09-10T21:17:41.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 Processor -- start 2024-09-10T21:17:41.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 -- start start 2024-09-10T21:17:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f25c806eb80 0x7f25c8154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c80d06b0 0x7f25c8150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815a270 0x7f25c8152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8169310 con 0x7f25c80d06b0 2024-09-10T21:17:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8072f80 con 0x7f25c815a270 2024-09-10T21:17:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cee00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f25c8169490 con 0x7f25c806eb80 2024-09-10T21:17:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c80d06b0 0x7f25c8150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c80d06b0 0x7f25c8150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38032/0 (socket says 172.21.15.16:38032) 2024-09-10T21:17:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25bfe00640 1 -- 172.21.15.16:0/2822276290 learned_addr learned my addr 172.21.15.16:0/2822276290 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25c7e00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f25c806eb80 0x7f25c8154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.289+0000 7f25cce00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815a270 0x7f25c8152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:41.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25bfe00640 1 -- 172.21.15.16:0/2822276290 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f25c806eb80 msgr2=0x7f25c8154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:41.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25bfe00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f25c806eb80 0x7f25c8154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25bfe00640 1 -- 172.21.15.16:0/2822276290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815a270 msgr2=0x7f25c8152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:41.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25bfe00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815a270 0x7f25c8152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25bfe00640 1 -- 172.21.15.16:0/2822276290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c80ce3d0 con 0x7f25c80d06b0 2024-09-10T21:17:41.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25cce00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815a270 0x7f25c8152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:41.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25bfe00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c80d06b0 0x7f25c8150940 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f25b00b0040 tx=0x7f25b00a6820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25c4c00640 1 -- 172.21.15.16:0/2822276290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b00a6ad0 con 0x7f25c80d06b0 2024-09-10T21:17:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25c4c00640 1 -- 172.21.15.16:0/2822276290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25b00bc070 con 0x7f25c80d06b0 2024-09-10T21:17:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25c4c00640 1 -- 172.21.15.16:0/2822276290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0005030 con 0x7f25c80d06b0 2024-09-10T21:17:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25cee00640 1 -- 172.21.15.16:0/2822276290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c8158420 con 0x7f25c80d06b0 2024-09-10T21:17:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25cee00640 1 -- 172.21.15.16:0/2822276290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c81589e0 con 0x7f25c80d06b0 2024-09-10T21:17:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25cee00640 1 -- 172.21.15.16:0/2822276290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c806e640 con 0x7f25c80d06b0 2024-09-10T21:17:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.293+0000 7f25c4c00640 1 -- 172.21.15.16:0/2822276290 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f25c80ca980 con 0x7f25c80d06b0 2024-09-10T21:17:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.297+0000 7f25c4c00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f25a40725e0 0x7f25a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.297+0000 7f25c4c00640 1 -- 172.21.15.16:0/2822276290 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5709+0+0 (secure 0 0 0) 0x7f25b012d760 con 0x7f25c80d06b0 2024-09-10T21:17:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.297+0000 7f25c7e00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f25a40725e0 0x7f25a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.297+0000 7f25c7e00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f25a40725e0 0x7f25a4074aa0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f25b8007fd0 tx=0x7f25b8009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:41.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.297+0000 7f25c4c00640 1 -- 172.21.15.16:0/2822276290 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f25b00c5060 con 0x7f25c80d06b0 2024-09-10T21:17:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.473+0000 7f25cee00640 1 -- 172.21.15.16:0/2822276290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25c80b5aa0 con 0x7f25c80d06b0 2024-09-10T21:17:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.473+0000 7f25c4c00640 1 -- 172.21.15.16:0/2822276290 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16652 (secure 0 0 0) 0x7f25b0016330 con 0x7f25c80d06b0 2024-09-10T21:17:41.479 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:41.479 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":115,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:17:40.510647+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:17:38.209906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T21:17:39.468885+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:17:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.477+0000 7f25cee00640 1 -- 172.21.15.16:0/2822276290 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f25a40725e0 msgr2=0x7f25a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.477+0000 7f25cee00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f25a40725e0 0x7f25a4074aa0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f25b8007fd0 tx=0x7f25b8009040 comp rx=0 tx=0).stop 2024-09-10T21:17:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.477+0000 7f25cee00640 1 -- 172.21.15.16:0/2822276290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c80d06b0 msgr2=0x7f25c8150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.477+0000 7f25cee00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c80d06b0 0x7f25c8150940 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f25b00b0040 tx=0x7f25b00a6820 comp rx=0 tx=0).stop 2024-09-10T21:17:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.477+0000 7f25cee00640 1 -- 172.21.15.16:0/2822276290 shutdown_connections 2024-09-10T21:17:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.477+0000 7f25cee00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f25a40725e0 0x7f25a4074aa0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.477+0000 7f25cee00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815a270 0x7f25c8152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.477+0000 7f25cee00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c80d06b0 0x7f25c8150940 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.477+0000 7f25cee00640 1 --2- 172.21.15.16:0/2822276290 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f25c806eb80 0x7f25c8154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.477+0000 7f25cee00640 1 -- 172.21.15.16:0/2822276290 >> 172.21.15.16:0/2822276290 conn(0x7f25c80cfb20 msgr2=0x7f25c8073190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:41.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.477+0000 7f25cee00640 1 -- 172.21.15.16:0/2822276290 shutdown_connections 2024-09-10T21:17:41.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.477+0000 7f25cee00640 1 -- 172.21.15.16:0/2822276290 wait complete. 2024-09-10T21:17:41.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:17:41.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 -- 172.21.15.16:0/426385968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc80d0890 msgr2=0x7f6cc80b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:41.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 --2- 172.21.15.16:0/426385968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc80d0890 0x7f6cc80b0020 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009c80 tx=0x7f6cc401cd40 comp rx=0 tx=0).stop 2024-09-10T21:17:41.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 -- 172.21.15.16:0/426385968 shutdown_connections 2024-09-10T21:17:41.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 --2- 172.21.15.16:0/426385968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc80d0890 0x7f6cc80b0020 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 --2- 172.21.15.16:0/426385968 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6cc80b3200 0x7f6cc80afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 -- 172.21.15.16:0/426385968 >> 172.21.15.16:0/426385968 conn(0x7f6cc80d1d80 msgr2=0x7f6cc80d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:41.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 -- 172.21.15.16:0/426385968 shutdown_connections 2024-09-10T21:17:41.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 -- 172.21.15.16:0/426385968 wait complete. 2024-09-10T21:17:41.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 Processor -- start 2024-09-10T21:17:41.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 -- start start 2024-09-10T21:17:41.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc80d0890 0x7f6cc8156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:41.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6cc815f450 0x7f6cc80b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:41.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc814fe30 0x7f6cc815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:41.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc8168d50 con 0x7f6cc80d0890 2024-09-10T21:17:41.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc8168bd0 con 0x7f6cc814fe30 2024-09-10T21:17:41.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cd0800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc8168ed0 con 0x7f6cc815f450 2024-09-10T21:17:41.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6ccde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc80d0890 0x7f6cc8156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:41.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6ccde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc80d0890 0x7f6cc8156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38056/0 (socket says 172.21.15.16:38056) 2024-09-10T21:17:41.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6ccde00640 1 -- 172.21.15.16:0/994854363 learned_addr learned my addr 172.21.15.16:0/994854363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:41.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6ccd400640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6cc815f450 0x7f6cc80b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6cce800640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc814fe30 0x7f6cc815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6ccde00640 1 -- 172.21.15.16:0/994854363 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6cc815f450 msgr2=0x7f6cc80b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6ccde00640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6cc815f450 0x7f6cc80b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6ccde00640 1 -- 172.21.15.16:0/994854363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc814fe30 msgr2=0x7f6cc815ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6ccde00640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc814fe30 0x7f6cc815ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.645+0000 7f6ccde00640 1 -- 172.21.15.16:0/994854363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc815ec70 con 0x7f6cc80d0890 2024-09-10T21:17:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.649+0000 7f6ccde00640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc80d0890 0x7f6cc8156490 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00c3d0 tx=0x7f6cbc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.649+0000 7f6cc2000640 1 -- 172.21.15.16:0/994854363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc017070 con 0x7f6cc80d0890 2024-09-10T21:17:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.649+0000 7f6cc2000640 1 -- 172.21.15.16:0/994854363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cbc0053e0 con 0x7f6cc80d0890 2024-09-10T21:17:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.649+0000 7f6cc2000640 1 -- 172.21.15.16:0/994854363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc003880 con 0x7f6cc80d0890 2024-09-10T21:17:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.649+0000 7f6cd0800640 1 -- 172.21.15.16:0/994854363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc81f8ff0 con 0x7f6cc80d0890 2024-09-10T21:17:41.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.649+0000 7f6cd0800640 1 -- 172.21.15.16:0/994854363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc81f95b0 con 0x7f6cc80d0890 2024-09-10T21:17:41.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.649+0000 7f6cd0800640 1 -- 172.21.15.16:0/994854363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc80cfc80 con 0x7f6cc80d0890 2024-09-10T21:17:41.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.649+0000 7f6cc2000640 1 -- 172.21.15.16:0/994854363 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6cbc003a20 con 0x7f6cc80d0890 2024-09-10T21:17:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.649+0000 7f6cc2000640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6cac0725e0 0x7f6cac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.649+0000 7f6cc2000640 1 -- 172.21.15.16:0/994854363 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5709+0+0 (secure 0 0 0) 0x7f6cbc092890 con 0x7f6cc80d0890 2024-09-10T21:17:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.649+0000 7f6ccd400640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6cac0725e0 0x7f6cac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.653+0000 7f6ccd400640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6cac0725e0 0x7f6cac074aa0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8153ec0 tx=0x7f6cb80091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:41.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.653+0000 7f6cc2000640 1 -- 172.21.15.16:0/994854363 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6cbc05dfe0 con 0x7f6cc80d0890 2024-09-10T21:17:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.833+0000 7f6cd0800640 1 -- 172.21.15.16:0/994854363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6cc809fdc0 con 0x7f6cc80d0890 2024-09-10T21:17:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.833+0000 7f6cc2000640 1 -- 172.21.15.16:0/994854363 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16652 (secure 0 0 0) 0x7f6cbc05d8e0 con 0x7f6cc80d0890 2024-09-10T21:17:41.838 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:41.838 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":115,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:17:40.510647+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:17:38.209906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T21:17:39.468885+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:17:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.837+0000 7f6cd0800640 1 -- 172.21.15.16:0/994854363 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6cac0725e0 msgr2=0x7f6cac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.837+0000 7f6cd0800640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6cac0725e0 0x7f6cac074aa0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8153ec0 tx=0x7f6cb80091e0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.837+0000 7f6cd0800640 1 -- 172.21.15.16:0/994854363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc80d0890 msgr2=0x7f6cc8156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.837+0000 7f6cd0800640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc80d0890 0x7f6cc8156490 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00c3d0 tx=0x7f6cbc00c890 comp rx=0 tx=0).stop 2024-09-10T21:17:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.837+0000 7f6cd0800640 1 -- 172.21.15.16:0/994854363 shutdown_connections 2024-09-10T21:17:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.837+0000 7f6cd0800640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6cac0725e0 0x7f6cac074aa0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.837+0000 7f6cd0800640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc814fe30 0x7f6cc815ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.837+0000 7f6cd0800640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6cc815f450 0x7f6cc80b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.837+0000 7f6cd0800640 1 --2- 172.21.15.16:0/994854363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc80d0890 0x7f6cc8156490 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.837+0000 7f6cd0800640 1 -- 172.21.15.16:0/994854363 >> 172.21.15.16:0/994854363 conn(0x7f6cc80d1d80 msgr2=0x7f6cc8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.837+0000 7f6cd0800640 1 -- 172.21.15.16:0/994854363 shutdown_connections 2024-09-10T21:17:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:41.837+0000 7f6cd0800640 1 -- 172.21.15.16:0/994854363 wait complete. 2024-09-10T21:17:41.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:17:42.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.005+0000 7fc4fe600640 1 -- 172.21.15.16:0/3147370512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f80cba80 msgr2=0x7fc4f80ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:42.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.005+0000 7fc4fe600640 1 --2- 172.21.15.16:0/3147370512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f80cba80 0x7fc4f80ad850 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009bc0 tx=0x7fc4f001cac0 comp rx=0 tx=0).stop 2024-09-10T21:17:42.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.005+0000 7fc4fe600640 1 -- 172.21.15.16:0/3147370512 shutdown_connections 2024-09-10T21:17:42.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.005+0000 7fc4fe600640 1 --2- 172.21.15.16:0/3147370512 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f80cba80 0x7fc4f80ad850 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:42.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.005+0000 7fc4fe600640 1 --2- 172.21.15.16:0/3147370512 >> v2:172.21.15.52:3300/0 conn(0x7fc4f815e050 0x7fc4f80d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:42.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.005+0000 7fc4fe600640 1 -- 172.21.15.16:0/3147370512 >> 172.21.15.16:0/3147370512 conn(0x7fc4f80cd6a0 msgr2=0x7fc4f80cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:42.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.005+0000 7fc4fe600640 1 -- 172.21.15.16:0/3147370512 shutdown_connections 2024-09-10T21:17:42.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.005+0000 7fc4fe600640 1 -- 172.21.15.16:0/3147370512 wait complete. 2024-09-10T21:17:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.005+0000 7fc4fe600640 1 Processor -- start 2024-09-10T21:17:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.005+0000 7fc4fe600640 1 -- start start 2024-09-10T21:17:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4fe600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f80b0070 0x7fc4f8146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4fe600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cba80 0x7fc4f80703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4fe600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f815e050 0x7fc4f815ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4fe600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f80ae320 con 0x7fc4f80cba80 2024-09-10T21:17:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4fe600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f80ae1a0 con 0x7fc4f80b0070 2024-09-10T21:17:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4fe600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f816b390 con 0x7fc4f815e050 2024-09-10T21:17:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4eea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cba80 0x7fc4f80703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4eea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cba80 0x7fc4f80703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38064/0 (socket says 172.21.15.16:38064) 2024-09-10T21:17:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4eea00640 1 -- 172.21.15.16:0/3119921929 learned_addr learned my addr 172.21.15.16:0/3119921929 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4f7400640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f80b0070 0x7fc4f8146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4f7e00640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f815e050 0x7fc4f815ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4eea00640 1 -- 172.21.15.16:0/3119921929 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f815e050 msgr2=0x7fc4f815ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4eea00640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f815e050 0x7fc4f815ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4eea00640 1 -- 172.21.15.16:0/3119921929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f80b0070 msgr2=0x7fc4f8146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4eea00640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f80b0070 0x7fc4f8146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4eea00640 1 -- 172.21.15.16:0/3119921929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f80b4ac0 con 0x7fc4f80cba80 2024-09-10T21:17:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4eea00640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cba80 0x7fc4f80703e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c700 tx=0x7fc4e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4efe00640 1 -- 172.21.15.16:0/3119921929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0016020 con 0x7fc4f80cba80 2024-09-10T21:17:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4efe00640 1 -- 172.21.15.16:0/3119921929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4e000a930 con 0x7fc4f80cba80 2024-09-10T21:17:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4fe600640 1 -- 172.21.15.16:0/3119921929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f80b3d00 con 0x7fc4f80cba80 2024-09-10T21:17:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4efe00640 1 -- 172.21.15.16:0/3119921929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e000ad60 con 0x7fc4f80cba80 2024-09-10T21:17:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4fe600640 1 -- 172.21.15.16:0/3119921929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f80b42c0 con 0x7fc4f80cba80 2024-09-10T21:17:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.009+0000 7fc4fe600640 1 -- 172.21.15.16:0/3119921929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f80ad850 con 0x7fc4f80cba80 2024-09-10T21:17:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.013+0000 7fc4efe00640 1 -- 172.21.15.16:0/3119921929 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc4e0012420 con 0x7fc4f80cba80 2024-09-10T21:17:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.013+0000 7fc4efe00640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc4d40725e0 0x7fc4d4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.013+0000 7fc4efe00640 1 -- 172.21.15.16:0/3119921929 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5709+0+0 (secure 0 0 0) 0x7fc4e0093560 con 0x7fc4f80cba80 2024-09-10T21:17:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.013+0000 7fc4f7400640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc4d40725e0 0x7fc4d4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.013+0000 7fc4f7400640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc4d40725e0 0x7fc4d4074aa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009b40 tx=0x7fc4e80091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:42.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.017+0000 7fc4efe00640 1 -- 172.21.15.16:0/3119921929 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc4e0020020 con 0x7fc4f80cba80 2024-09-10T21:17:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:42.229+0000 7fc4fe600640 1 -- 172.21.15.16:0/3119921929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc4f80cc150 con 0x7fc4f80cba80 2024-09-10T21:17:43.072 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-09-10T21:17:43.069+0000 7f5f9a600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:17:43.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.113+0000 7fc4efe00640 1 -- 172.21.15.16:0/3119921929 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) ==== 118+0+0 (secure 0 0 0) 0x7fc4e0065020 con 0x7fc4f80cba80 2024-09-10T21:17:43.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.117+0000 7fc4fe600640 1 -- 172.21.15.16:0/3119921929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc4d40725e0 msgr2=0x7fc4d4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.117+0000 7fc4fe600640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc4d40725e0 0x7fc4d4074aa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009b40 tx=0x7fc4e80091e0 comp rx=0 tx=0).stop 2024-09-10T21:17:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.117+0000 7fc4fe600640 1 -- 172.21.15.16:0/3119921929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cba80 msgr2=0x7fc4f80703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.117+0000 7fc4fe600640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cba80 0x7fc4f80703e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c700 tx=0x7fc4e000cbc0 comp rx=0 tx=0).stop 2024-09-10T21:17:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.117+0000 7fc4fe600640 1 -- 172.21.15.16:0/3119921929 shutdown_connections 2024-09-10T21:17:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.117+0000 7fc4fe600640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc4d40725e0 0x7fc4d4074aa0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.117+0000 7fc4fe600640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc4f815e050 0x7fc4f815ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.117+0000 7fc4fe600640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4f80cba80 0x7fc4f80703e0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.117+0000 7fc4fe600640 1 --2- 172.21.15.16:0/3119921929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f80b0070 0x7fc4f8146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.117+0000 7fc4fe600640 1 -- 172.21.15.16:0/3119921929 >> 172.21.15.16:0/3119921929 conn(0x7fc4f80cd6a0 msgr2=0x7fc4f80b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.117+0000 7fc4fe600640 1 -- 172.21.15.16:0/3119921929 shutdown_connections 2024-09-10T21:17:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.117+0000 7fc4fe600640 1 -- 172.21.15.16:0/3119921929 wait complete. 2024-09-10T21:17:43.145 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:17:43.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:17:43.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 -- 172.21.15.16:0/2777227218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c400ce040 msgr2=0x7f9c400b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:43.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 --2- 172.21.15.16:0/2777227218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c400ce040 0x7f9c400b1710 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009bc0 tx=0x7f9c3801f3b0 comp rx=0 tx=0).stop 2024-09-10T21:17:43.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 -- 172.21.15.16:0/2777227218 shutdown_connections 2024-09-10T21:17:43.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 --2- 172.21.15.16:0/2777227218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c400ce040 0x7f9c400b1710 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:43.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 --2- 172.21.15.16:0/2777227218 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9c400d0210 0x7f9c400ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:43.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 -- 172.21.15.16:0/2777227218 >> 172.21.15.16:0/2777227218 conn(0x7f9c400d1df0 msgr2=0x7f9c400d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:43.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 -- 172.21.15.16:0/2777227218 shutdown_connections 2024-09-10T21:17:43.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 -- 172.21.15.16:0/2777227218 wait complete. 2024-09-10T21:17:43.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 Processor -- start 2024-09-10T21:17:43.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 -- start start 2024-09-10T21:17:43.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c400ce040 0x7f9c400b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:43.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c400cfe30 0x7f9c400b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:43.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9c400d0210 0x7f9c400aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c400b14f0 con 0x7f9c400ce040 2024-09-10T21:17:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9c400d2b20 con 0x7f9c400cfe30 2024-09-10T21:17:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c45400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9c400cf1e0 con 0x7f9c400d0210 2024-09-10T21:17:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c3e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c400ce040 0x7f9c400b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c3e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c400ce040 0x7f9c400b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43508/0 (socket says 172.21.15.16:43508) 2024-09-10T21:17:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c3e800640 1 -- 172.21.15.16:0/2812457460 learned_addr learned my addr 172.21.15.16:0/2812457460 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c3f200640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9c400d0210 0x7f9c400aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c3e800640 1 -- 172.21.15.16:0/2812457460 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9c400d0210 msgr2=0x7f9c400aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c3e800640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9c400d0210 0x7f9c400aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c3e800640 1 -- 172.21.15.16:0/2812457460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c400cfe30 msgr2=0x7f9c400b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c3de00640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c400cfe30 0x7f9c400b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c3e800640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c400cfe30 0x7f9c400b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c3e800640 1 -- 172.21.15.16:0/2812457460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4006e020 con 0x7f9c400ce040 2024-09-10T21:17:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c3f200640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9c400d0210 0x7f9c400aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.297+0000 7f9c3de00640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c400cfe30 0x7f9c400b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.301+0000 7f9c3e800640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c400ce040 0x7f9c400b5ab0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c490 tx=0x7f9c3000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.301+0000 7f9c36a00640 1 -- 172.21.15.16:0/2812457460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30011790 con 0x7f9c400ce040 2024-09-10T21:17:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.301+0000 7f9c36a00640 1 -- 172.21.15.16:0/2812457460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c30017070 con 0x7f9c400ce040 2024-09-10T21:17:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.301+0000 7f9c36a00640 1 -- 172.21.15.16:0/2812457460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30011ce0 con 0x7f9c400ce040 2024-09-10T21:17:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.301+0000 7f9c45400640 1 -- 172.21.15.16:0/2812457460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4006e300 con 0x7f9c400ce040 2024-09-10T21:17:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.301+0000 7f9c45400640 1 -- 172.21.15.16:0/2812457460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4006e8c0 con 0x7f9c400ce040 2024-09-10T21:17:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.301+0000 7f9c45400640 1 -- 172.21.15.16:0/2812457460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c4006ed00 con 0x7f9c400ce040 2024-09-10T21:17:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.301+0000 7f9c36a00640 1 -- 172.21.15.16:0/2812457460 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9c3001c460 con 0x7f9c400ce040 2024-09-10T21:17:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.301+0000 7f9c36a00640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9c200725e0 0x7f9c20074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.301+0000 7f9c36a00640 1 -- 172.21.15.16:0/2812457460 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f9c30092ff0 con 0x7f9c400ce040 2024-09-10T21:17:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.301+0000 7f9c3de00640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9c200725e0 0x7f9c20074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.305+0000 7f9c3de00640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9c200725e0 0x7f9c20074aa0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9c401e7160 tx=0x7f9c2c005d30 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.305+0000 7f9c36a00640 1 -- 172.21.15.16:0/2812457460 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9c3005e7c0 con 0x7f9c400ce040 2024-09-10T21:17:43.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:43.517+0000 7f9c45400640 1 -- 172.21.15.16:0/2812457460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9c400b1710 con 0x7f9c400ce040 2024-09-10T21:17:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.121+0000 7f9c36a00640 1 -- 172.21.15.16:0/2812457460 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) ==== 122+0+0 (secure 0 0 0) 0x7f9c3005e0c0 con 0x7f9c400ce040 2024-09-10T21:17:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.121+0000 7f9c45400640 1 -- 172.21.15.16:0/2812457460 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9c200725e0 msgr2=0x7f9c20074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.121+0000 7f9c45400640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9c200725e0 0x7f9c20074aa0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9c401e7160 tx=0x7f9c2c005d30 comp rx=0 tx=0).stop 2024-09-10T21:17:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.121+0000 7f9c45400640 1 -- 172.21.15.16:0/2812457460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c400ce040 msgr2=0x7f9c400b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.121+0000 7f9c45400640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c400ce040 0x7f9c400b5ab0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c490 tx=0x7f9c3000c950 comp rx=0 tx=0).stop 2024-09-10T21:17:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.121+0000 7f9c45400640 1 -- 172.21.15.16:0/2812457460 shutdown_connections 2024-09-10T21:17:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.121+0000 7f9c45400640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9c200725e0 0x7f9c20074aa0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.121+0000 7f9c45400640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9c400d0210 0x7f9c400aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.121+0000 7f9c45400640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c400cfe30 0x7f9c400b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.121+0000 7f9c45400640 1 --2- 172.21.15.16:0/2812457460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c400ce040 0x7f9c400b5ab0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.121+0000 7f9c45400640 1 -- 172.21.15.16:0/2812457460 >> 172.21.15.16:0/2812457460 conn(0x7f9c400d1df0 msgr2=0x7f9c400d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.121+0000 7f9c45400640 1 -- 172.21.15.16:0/2812457460 shutdown_connections 2024-09-10T21:17:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.121+0000 7f9c45400640 1 -- 172.21.15.16:0/2812457460 wait complete. 2024-09-10T21:17:44.150 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:17:44.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.297+0000 7fdea1e00640 1 -- 172.21.15.16:0/2933668857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde80004810 msgr2=0x7fde80004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.297+0000 7fdea1e00640 1 --2- 172.21.15.16:0/2933668857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde80004810 0x7fde80004c90 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fde84009b60 tx=0x7fde8401d210 comp rx=0 tx=0).stop 2024-09-10T21:17:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.297+0000 7fde9b400640 1 -- 172.21.15.16:0/2933668857 reap_dead start 2024-09-10T21:17:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.297+0000 7fdea1e00640 1 -- 172.21.15.16:0/2933668857 shutdown_connections 2024-09-10T21:17:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.297+0000 7fdea1e00640 1 -- 172.21.15.16:0/2933668857 >> 172.21.15.16:0/2933668857 conn(0x7fde9c0cfb20 msgr2=0x7fde9c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.297+0000 7fdea1e00640 1 -- 172.21.15.16:0/2933668857 shutdown_connections 2024-09-10T21:17:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.297+0000 7fdea1e00640 1 -- 172.21.15.16:0/2933668857 wait complete. 2024-09-10T21:17:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.297+0000 7fdea1e00640 1 Processor -- start 2024-09-10T21:17:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fdea1e00640 1 -- start start 2024-09-10T21:17:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fdea1e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde80004810 0x7fde9c1f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fdea1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c1f5a20 0x7fde9c159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fdea1e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fde9c0c59f0 0x7fde9c0c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fdea1e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0cbb20 con 0x7fde9c1f5a20 2024-09-10T21:17:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fdea1e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0cb9a0 con 0x7fde80004810 2024-09-10T21:17:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fdea1e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fde9c0cafd0 con 0x7fde9c0c59f0 2024-09-10T21:17:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde9a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c1f5a20 0x7fde9c159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde9a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c1f5a20 0x7fde9c159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43534/0 (socket says 172.21.15.16:43534) 2024-09-10T21:17:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde9a000640 1 -- 172.21.15.16:0/3924965324 learned_addr learned my addr 172.21.15.16:0/3924965324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde9aa00640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde80004810 0x7fde9c1f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde9b400640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fde9c0c59f0 0x7fde9c0c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde9a000640 1 -- 172.21.15.16:0/3924965324 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fde9c0c59f0 msgr2=0x7fde9c0c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde9a000640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fde9c0c59f0 0x7fde9c0c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde9a000640 1 -- 172.21.15.16:0/3924965324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde80004810 msgr2=0x7fde9c1f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde9a000640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde80004810 0x7fde9c1f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde9a000640 1 -- 172.21.15.16:0/3924965324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c0c4260 con 0x7fde9c1f5a20 2024-09-10T21:17:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde9b400640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fde9c0c59f0 0x7fde9c0c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde9a000640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c1f5a20 0x7fde9c159350 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fde840af040 tx=0x7fde84005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde93400640 1 -- 172.21.15.16:0/3924965324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde840b1040 con 0x7fde9c1f5a20 2024-09-10T21:17:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde93400640 1 -- 172.21.15.16:0/3924965324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde840ac8a0 con 0x7fde9c1f5a20 2024-09-10T21:17:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fde93400640 1 -- 172.21.15.16:0/3924965324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde840accd0 con 0x7fde9c1f5a20 2024-09-10T21:17:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fdea1e00640 1 -- 172.21.15.16:0/3924965324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c0c9aa0 con 0x7fde9c1f5a20 2024-09-10T21:17:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fdea1e00640 1 -- 172.21.15.16:0/3924965324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c15bc00 con 0x7fde9c1f5a20 2024-09-10T21:17:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.301+0000 7fdea1e00640 1 -- 172.21.15.16:0/3924965324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9c06e8e0 con 0x7fde9c1f5a20 2024-09-10T21:17:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.305+0000 7fde93400640 1 -- 172.21.15.16:0/3924965324 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fde840046d0 con 0x7fde9c1f5a20 2024-09-10T21:17:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.305+0000 7fde93400640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde78072610 0x7fde78074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.305+0000 7fde93400640 1 -- 172.21.15.16:0/3924965324 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fde8412d630 con 0x7fde9c1f5a20 2024-09-10T21:17:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.305+0000 7fde9aa00640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde78072610 0x7fde78074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.305+0000 7fde9aa00640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde78072610 0x7fde78074ad0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0c68d0 tx=0x7fde8c009500 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.309+0000 7fde93400640 1 -- 172.21.15.16:0/3924965324 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fde84016f80 con 0x7fde9c1f5a20 2024-09-10T21:17:44.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.517+0000 7fdea1e00640 1 -- 172.21.15.16:0/3924965324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde9c069380 con 0x7fde9c1f5a20 2024-09-10T21:17:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.517+0000 7fde93400640 1 -- 172.21.15.16:0/3924965324 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5229 (secure 0 0 0) 0x7fde840f8c70 con 0x7fde9c1f5a20 2024-09-10T21:17:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:44.521 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"btime":"2024-09-10T21:17:44:118059+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3961362430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3961362430},{"type":"v1","addr":"172.21.15.52:6834","nonce":3961362430}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6967,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/831111367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":831111367},{"type":"v1","addr":"172.21.15.52:6835","nonce":831111367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7467,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/552929965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":552929965},{"type":"v1","addr":"172.21.15.16:6836","nonce":552929965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:40.510614+0000","modified":"2024-09-10T21:17:43.521144+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":136181},"failed":[],"damaged":[],"stopped":[],"info":{"gid_136181":{"gid":136181,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/1681945537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1681945537},{"type":"v1","addr":"172.21.15.16:6837","nonce":1681945537}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":136181,"qdb_cluster":[136181]},"id":9}]} 2024-09-10T21:17:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 82 2024-09-10T21:17:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.517+0000 7fdea1e00640 1 -- 172.21.15.16:0/3924965324 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde78072610 msgr2=0x7fde78074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.517+0000 7fdea1e00640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde78072610 0x7fde78074ad0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0c68d0 tx=0x7fde8c009500 comp rx=0 tx=0).stop 2024-09-10T21:17:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.517+0000 7fdea1e00640 1 -- 172.21.15.16:0/3924965324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c1f5a20 msgr2=0x7fde9c159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.517+0000 7fdea1e00640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c1f5a20 0x7fde9c159350 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fde840af040 tx=0x7fde84005620 comp rx=0 tx=0).stop 2024-09-10T21:17:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.517+0000 7fdea1e00640 1 -- 172.21.15.16:0/3924965324 shutdown_connections 2024-09-10T21:17:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.517+0000 7fdea1e00640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde78072610 0x7fde78074ad0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.517+0000 7fdea1e00640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fde9c0c59f0 0x7fde9c0c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.517+0000 7fdea1e00640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c1f5a20 0x7fde9c159350 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.517+0000 7fdea1e00640 1 --2- 172.21.15.16:0/3924965324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde80004810 0x7fde9c1f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.517+0000 7fdea1e00640 1 -- 172.21.15.16:0/3924965324 >> 172.21.15.16:0/3924965324 conn(0x7fde9c0cfb20 msgr2=0x7fde9c0b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.521+0000 7fdea1e00640 1 -- 172.21.15.16:0/3924965324 shutdown_connections 2024-09-10T21:17:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.521+0000 7fdea1e00640 1 -- 172.21.15.16:0/3924965324 wait complete. 2024-09-10T21:17:44.543 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.685+0000 7fdea6200640 1 -- 172.21.15.16:0/1639025245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea00b13c0 msgr2=0x7fdea0071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.685+0000 7fdea6200640 1 --2- 172.21.15.16:0/1639025245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea00b13c0 0x7fdea0071520 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fdea0073290 tx=0x7fde8801c7a0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.685+0000 7fdea6200640 1 -- 172.21.15.16:0/1639025245 shutdown_connections 2024-09-10T21:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.685+0000 7fdea6200640 1 --2- 172.21.15.16:0/1639025245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea00b13c0 0x7fdea0071520 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.685+0000 7fdea6200640 1 --2- 172.21.15.16:0/1639025245 >> v2:172.21.15.52:3300/0 conn(0x7fdea015f400 0x7fdea0070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.685+0000 7fdea6200640 1 -- 172.21.15.16:0/1639025245 >> 172.21.15.16:0/1639025245 conn(0x7fdea009b130 msgr2=0x7fdea00d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.685+0000 7fdea6200640 1 -- 172.21.15.16:0/1639025245 shutdown_connections 2024-09-10T21:17:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.685+0000 7fdea6200640 1 -- 172.21.15.16:0/1639025245 wait complete. 2024-09-10T21:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fdea6200640 1 Processor -- start 2024-09-10T21:17:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fdea6200640 1 -- start start 2024-09-10T21:17:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fdea6200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdea00b13c0 0x7fdea00b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fdea6200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea00cfc80 0x7fdea0150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fdea6200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdea015f400 0x7fdea00cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fdea6200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01695f0 con 0x7fdea00cfc80 2024-09-10T21:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fdea6200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdea00d1270 con 0x7fdea015f400 2024-09-10T21:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fdea6200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fdea0169770 con 0x7fdea00b13c0 2024-09-10T21:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fde97e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea00cfc80 0x7fdea0150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fde97e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea00cfc80 0x7fdea0150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43560/0 (socket says 172.21.15.16:43560) 2024-09-10T21:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fde97e00640 1 -- 172.21.15.16:0/1242132360 learned_addr learned my addr 172.21.15.16:0/1242132360 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fde97e00640 1 -- 172.21.15.16:0/1242132360 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdea00b13c0 msgr2=0x7fdea00b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fde97e00640 1 --2- 172.21.15.16:0/1242132360 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdea00b13c0 0x7fdea00b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fde97e00640 1 -- 172.21.15.16:0/1242132360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdea015f400 msgr2=0x7fdea00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fde97e00640 1 --2- 172.21.15.16:0/1242132360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdea015f400 0x7fdea00cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fde97e00640 1 -- 172.21.15.16:0/1242132360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea0150b50 con 0x7fdea00cfc80 2024-09-10T21:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fde97e00640 1 --2- 172.21.15.16:0/1242132360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea00cfc80 0x7fdea0150610 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fde880b0040 tx=0x7fde880a55b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fde97400640 1 -- 172.21.15.16:0/1242132360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde880b2040 con 0x7fdea00cfc80 2024-09-10T21:17:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fde97400640 1 -- 172.21.15.16:0/1242132360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde88004380 con 0x7fdea00cfc80 2024-09-10T21:17:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fde97400640 1 -- 172.21.15.16:0/1242132360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde880052d0 con 0x7fdea00cfc80 2024-09-10T21:17:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fdea6200640 1 -- 172.21.15.16:0/1242132360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea0152250 con 0x7fdea00cfc80 2024-09-10T21:17:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fdea6200640 1 -- 172.21.15.16:0/1242132360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea00b5490 con 0x7fdea00cfc80 2024-09-10T21:17:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.689+0000 7fdea6200640 1 -- 172.21.15.16:0/1242132360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea00d00d0 con 0x7fdea00cfc80 2024-09-10T21:17:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.693+0000 7fde97400640 1 -- 172.21.15.16:0/1242132360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fde88005470 con 0x7fdea00cfc80 2024-09-10T21:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.693+0000 7fde97400640 1 --2- 172.21.15.16:0/1242132360 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde7c0725e0 0x7fde7c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.693+0000 7fde97400640 1 -- 172.21.15.16:0/1242132360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fde8812c970 con 0x7fdea00cfc80 2024-09-10T21:17:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.693+0000 7fde9f400640 1 --2- 172.21.15.16:0/1242132360 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde7c0725e0 0x7fde7c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.693+0000 7fde9f400640 1 --2- 172.21.15.16:0/1242132360 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde7c0725e0 0x7fde7c074aa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fde90005870 tx=0x7fde90005800 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.697+0000 7fde97400640 1 -- 172.21.15.16:0/1242132360 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fde880164b0 con 0x7fdea00cfc80 2024-09-10T21:17:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.869+0000 7fdea6200640 1 -- 172.21.15.16:0/1242132360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdea0152520 con 0x7fdea00cfc80 2024-09-10T21:17:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.873+0000 7fde97400640 1 -- 172.21.15.16:0/1242132360 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16639 (secure 0 0 0) 0x7fde880f7fc0 con 0x7fdea00cfc80 2024-09-10T21:17:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:44.876 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":116,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:17:42.104454+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:17:38.209906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T21:17:39.468885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:17:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.873+0000 7fdea6200640 1 -- 172.21.15.16:0/1242132360 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde7c0725e0 msgr2=0x7fde7c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.873+0000 7fdea6200640 1 --2- 172.21.15.16:0/1242132360 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde7c0725e0 0x7fde7c074aa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fde90005870 tx=0x7fde90005800 comp rx=0 tx=0).stop 2024-09-10T21:17:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.873+0000 7fdea6200640 1 -- 172.21.15.16:0/1242132360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea00cfc80 msgr2=0x7fdea0150610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.873+0000 7fdea6200640 1 --2- 172.21.15.16:0/1242132360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea00cfc80 0x7fdea0150610 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fde880b0040 tx=0x7fde880a55b0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.873+0000 7fdea6200640 1 -- 172.21.15.16:0/1242132360 shutdown_connections 2024-09-10T21:17:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.873+0000 7fdea6200640 1 --2- 172.21.15.16:0/1242132360 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fde7c0725e0 0x7fde7c074aa0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.873+0000 7fdea6200640 1 --2- 172.21.15.16:0/1242132360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdea015f400 0x7fdea00cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.873+0000 7fdea6200640 1 --2- 172.21.15.16:0/1242132360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea00cfc80 0x7fdea0150610 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.873+0000 7fdea6200640 1 --2- 172.21.15.16:0/1242132360 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdea00b13c0 0x7fdea00b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.873+0000 7fdea6200640 1 -- 172.21.15.16:0/1242132360 >> 172.21.15.16:0/1242132360 conn(0x7fdea009b130 msgr2=0x7fdea00b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.873+0000 7fdea6200640 1 -- 172.21.15.16:0/1242132360 shutdown_connections 2024-09-10T21:17:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:44.873+0000 7fdea6200640 1 -- 172.21.15.16:0/1242132360 wait complete. 2024-09-10T21:17:44.898 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:17:44.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:17:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.041+0000 7f6f46000640 1 -- 172.21.15.16:0/4267245189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f400d06b0 msgr2=0x7f6f400700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.041+0000 7f6f46000640 1 --2- 172.21.15.16:0/4267245189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f400d06b0 0x7f6f400700a0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009840 tx=0x7f6f2801d1f0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.041+0000 7f6f46000640 1 -- 172.21.15.16:0/4267245189 shutdown_connections 2024-09-10T21:17:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.041+0000 7f6f46000640 1 --2- 172.21.15.16:0/4267245189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f400d06b0 0x7f6f400700a0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.041+0000 7f6f46000640 1 -- 172.21.15.16:0/4267245189 >> 172.21.15.16:0/4267245189 conn(0x7f6f400cfb20 msgr2=0x7f6f400cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.041+0000 7f6f46000640 1 -- 172.21.15.16:0/4267245189 shutdown_connections 2024-09-10T21:17:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.041+0000 7f6f46000640 1 -- 172.21.15.16:0/4267245189 wait complete. 2024-09-10T21:17:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.041+0000 7f6f46000640 1 Processor -- start 2024-09-10T21:17:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.041+0000 7f6f46000640 1 -- start start 2024-09-10T21:17:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f46000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f400705e0 0x7f6f40152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f46000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f400d06b0 0x7f6f40151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f46000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f4015a270 0x7f6f40155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f46000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f401686a0 con 0x7f6f400705e0 2024-09-10T21:17:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f46000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6f40168520 con 0x7f6f400d06b0 2024-09-10T21:17:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f46000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6f40168820 con 0x7f6f4015a270 2024-09-10T21:17:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f400705e0 0x7f6f40152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f400705e0 0x7f6f40152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43578/0 (socket says 172.21.15.16:43578) 2024-09-10T21:17:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f3f400640 1 -- 172.21.15.16:0/4025904529 learned_addr learned my addr 172.21.15.16:0/4025904529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f3fe00640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f4015a270 0x7f6f40155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f37e00640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f400d06b0 0x7f6f40151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f3f400640 1 -- 172.21.15.16:0/4025904529 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f4015a270 msgr2=0x7f6f40155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f3f400640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f4015a270 0x7f6f40155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f3f400640 1 -- 172.21.15.16:0/4025904529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f400d06b0 msgr2=0x7f6f40151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f3f400640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f400d06b0 0x7f6f40151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f3f400640 1 -- 172.21.15.16:0/4025904529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f40163770 con 0x7f6f400705e0 2024-09-10T21:17:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f3f400640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f400705e0 0x7f6f40152220 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000b440 tx=0x7f6f3000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f37400640 1 -- 172.21.15.16:0/4025904529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30015020 con 0x7f6f400705e0 2024-09-10T21:17:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f37400640 1 -- 172.21.15.16:0/4025904529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f30011980 con 0x7f6f400705e0 2024-09-10T21:17:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f37400640 1 -- 172.21.15.16:0/4025904529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30011dd0 con 0x7f6f400705e0 2024-09-10T21:17:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f46000640 1 -- 172.21.15.16:0/4025904529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f40157a10 con 0x7f6f400705e0 2024-09-10T21:17:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f46000640 1 -- 172.21.15.16:0/4025904529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f40163390 con 0x7f6f400705e0 2024-09-10T21:17:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f46000640 1 -- 172.21.15.16:0/4025904529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f400700a0 con 0x7f6f400705e0 2024-09-10T21:17:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.045+0000 7f6f37400640 1 -- 172.21.15.16:0/4025904529 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6f3001a400 con 0x7f6f400705e0 2024-09-10T21:17:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.049+0000 7f6f37400640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f1c0725e0 0x7f6f1c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.049+0000 7f6f37400640 1 -- 172.21.15.16:0/4025904529 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f6f30093470 con 0x7f6f400705e0 2024-09-10T21:17:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.049+0000 7f6f37e00640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f1c0725e0 0x7f6f1c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.049+0000 7f6f37e00640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f1c0725e0 0x7f6f1c074aa0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6f280af000 tx=0x7f6f2801afd0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.053+0000 7f6f37400640 1 -- 172.21.15.16:0/4025904529 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6f3001f030 con 0x7f6f400705e0 2024-09-10T21:17:45.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.225+0000 7f6f46000640 1 -- 172.21.15.16:0/4025904529 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f40157850 con 0x7f6f1c0725e0 2024-09-10T21:17:45.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.229+0000 7f6f37400640 1 -- 172.21.15.16:0/4025904529 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240389 (secure 0 0 0) 0x7f6f280042a0 con 0x7f6f1c0725e0 2024-09-10T21:17:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:45.238 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:17:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.233+0000 7f6f46000640 1 -- 172.21.15.16:0/4025904529 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f1c0725e0 msgr2=0x7f6f1c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.233+0000 7f6f46000640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f1c0725e0 0x7f6f1c074aa0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6f280af000 tx=0x7f6f2801afd0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.233+0000 7f6f46000640 1 -- 172.21.15.16:0/4025904529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f400705e0 msgr2=0x7f6f40152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.233+0000 7f6f46000640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f400705e0 0x7f6f40152220 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000b440 tx=0x7f6f3000b900 comp rx=0 tx=0).stop 2024-09-10T21:17:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.233+0000 7f6f46000640 1 -- 172.21.15.16:0/4025904529 shutdown_connections 2024-09-10T21:17:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.233+0000 7f6f46000640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f1c0725e0 0x7f6f1c074aa0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.233+0000 7f6f46000640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6f4015a270 0x7f6f40155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.233+0000 7f6f46000640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f400d06b0 0x7f6f40151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.233+0000 7f6f46000640 1 --2- 172.21.15.16:0/4025904529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f400705e0 0x7f6f40152220 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.233+0000 7f6f46000640 1 -- 172.21.15.16:0/4025904529 >> 172.21.15.16:0/4025904529 conn(0x7f6f400cfb20 msgr2=0x7f6f400c5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.233+0000 7f6f46000640 1 -- 172.21.15.16:0/4025904529 shutdown_connections 2024-09-10T21:17:45.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.233+0000 7f6f46000640 1 -- 172.21.15.16:0/4025904529 wait complete. 2024-09-10T21:17:45.257 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-10T21:17:43.556677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":110,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":38,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":286644,"kb_used_data":12144,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754688076,"statfs":{"total":773094113280,"available":772800589824,"internally_reserved":0,"allocated":12435456,"data_stored":5984822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.480340"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806293+0000","last_change":"2024-09-10T21:17:41.113139+0000","last_active":"2024-09-10T21:17:42.806293+0000","last_peered":"2024-09-10T21:17:42.806293+0000","last_clean":"2024-09-10T21:17:42.806293+0000","last_became_active":"2024-09-10T21:17:41.112880+0000","last_became_peered":"2024-09-10T21:17:41.112880+0000","last_unstale":"2024-09-10T21:17:42.806293+0000","last_undegraded":"2024-09-10T21:17:42.806293+0000","last_fullsized":"2024-09-10T21:17:42.806293+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:01.443372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805905+0000","last_change":"2024-09-10T21:17:41.110077+0000","last_active":"2024-09-10T21:17:42.805905+0000","last_peered":"2024-09-10T21:17:42.805905+0000","last_clean":"2024-09-10T21:17:42.805905+0000","last_became_active":"2024-09-10T21:17:41.109774+0000","last_became_peered":"2024-09-10T21:17:41.109774+0000","last_unstale":"2024-09-10T21:17:42.805905+0000","last_undegraded":"2024-09-10T21:17:42.805905+0000","last_fullsized":"2024-09-10T21:17:42.805905+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:00.040600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807155+0000","last_change":"2024-09-10T21:17:41.112501+0000","last_active":"2024-09-10T21:17:42.807155+0000","last_peered":"2024-09-10T21:17:42.807155+0000","last_clean":"2024-09-10T21:17:42.807155+0000","last_became_active":"2024-09-10T21:17:41.112180+0000","last_became_peered":"2024-09-10T21:17:41.112180+0000","last_unstale":"2024-09-10T21:17:42.807155+0000","last_undegraded":"2024-09-10T21:17:42.807155+0000","last_fullsized":"2024-09-10T21:17:42.807155+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:14.476873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807747+0000","last_change":"2024-09-10T21:17:41.119900+0000","last_active":"2024-09-10T21:17:42.807747+0000","last_peered":"2024-09-10T21:17:42.807747+0000","last_clean":"2024-09-10T21:17:42.807747+0000","last_became_active":"2024-09-10T21:17:41.119550+0000","last_became_peered":"2024-09-10T21:17:41.119550+0000","last_unstale":"2024-09-10T21:17:42.807747+0000","last_undegraded":"2024-09-10T21:17:42.807747+0000","last_fullsized":"2024-09-10T21:17:42.807747+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:35.051017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805457+0000","last_change":"2024-09-10T21:17:41.131830+0000","last_active":"2024-09-10T21:17:42.805457+0000","last_peered":"2024-09-10T21:17:42.805457+0000","last_clean":"2024-09-10T21:17:42.805457+0000","last_became_active":"2024-09-10T21:17:41.131569+0000","last_became_peered":"2024-09-10T21:17:41.131569+0000","last_unstale":"2024-09-10T21:17:42.805457+0000","last_undegraded":"2024-09-10T21:17:42.805457+0000","last_fullsized":"2024-09-10T21:17:42.805457+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:18.442664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808094+0000","last_change":"2024-09-10T21:17:41.106857+0000","last_active":"2024-09-10T21:17:42.808094+0000","last_peered":"2024-09-10T21:17:42.808094+0000","last_clean":"2024-09-10T21:17:42.808094+0000","last_became_active":"2024-09-10T21:17:41.106561+0000","last_became_peered":"2024-09-10T21:17:41.106561+0000","last_unstale":"2024-09-10T21:17:42.808094+0000","last_undegraded":"2024-09-10T21:17:42.808094+0000","last_fullsized":"2024-09-10T21:17:42.808094+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:23.543864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808503+0000","last_change":"2024-09-10T21:17:41.111524+0000","last_active":"2024-09-10T21:17:42.808503+0000","last_peered":"2024-09-10T21:17:42.808503+0000","last_clean":"2024-09-10T21:17:42.808503+0000","last_became_active":"2024-09-10T21:17:41.111265+0000","last_became_peered":"2024-09-10T21:17:41.111265+0000","last_unstale":"2024-09-10T21:17:42.808503+0000","last_undegraded":"2024-09-10T21:17:42.808503+0000","last_fullsized":"2024-09-10T21:17:42.808503+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:40.878106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805743+0000","last_change":"2024-09-10T21:17:41.116782+0000","last_active":"2024-09-10T21:17:42.805743+0000","last_peered":"2024-09-10T21:17:42.805743+0000","last_clean":"2024-09-10T21:17:42.805743+0000","last_became_active":"2024-09-10T21:17:41.116353+0000","last_became_peered":"2024-09-10T21:17:41.116353+0000","last_unstale":"2024-09-10T21:17:42.805743+0000","last_undegraded":"2024-09-10T21:17:42.805743+0000","last_fullsized":"2024-09-10T21:17:42.805743+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:37.175886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806131+0000","last_change":"2024-09-10T21:17:40.108290+0000","last_active":"2024-09-10T21:17:42.806131+0000","last_peered":"2024-09-10T21:17:42.806131+0000","last_clean":"2024-09-10T21:17:42.806131+0000","last_became_active":"2024-09-10T21:17:40.107987+0000","last_became_peered":"2024-09-10T21:17:40.107987+0000","last_unstale":"2024-09-10T21:17:42.806131+0000","last_undegraded":"2024-09-10T21:17:42.806131+0000","last_fullsized":"2024-09-10T21:17:42.806131+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:37.425705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806057+0000","last_change":"2024-09-10T21:17:40.107030+0000","last_active":"2024-09-10T21:17:42.806057+0000","last_peered":"2024-09-10T21:17:42.806057+0000","last_clean":"2024-09-10T21:17:42.806057+0000","last_became_active":"2024-09-10T21:17:40.106742+0000","last_became_peered":"2024-09-10T21:17:40.106742+0000","last_unstale":"2024-09-10T21:17:42.806057+0000","last_undegraded":"2024-09-10T21:17:42.806057+0000","last_fullsized":"2024-09-10T21:17:42.806057+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:32.357990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807915+0000","last_change":"2024-09-10T21:17:41.118414+0000","last_active":"2024-09-10T21:17:42.807915+0000","last_peered":"2024-09-10T21:17:42.807915+0000","last_clean":"2024-09-10T21:17:42.807915+0000","last_became_active":"2024-09-10T21:17:41.118142+0000","last_became_peered":"2024-09-10T21:17:41.118142+0000","last_unstale":"2024-09-10T21:17:42.807915+0000","last_undegraded":"2024-09-10T21:17:42.807915+0000","last_fullsized":"2024-09-10T21:17:42.807915+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:37.934038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"115'6","reported_seq":21,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807781+0000","last_change":"2024-09-10T21:17:40.103617+0000","last_active":"2024-09-10T21:17:42.807781+0000","last_peered":"2024-09-10T21:17:42.807781+0000","last_clean":"2024-09-10T21:17:42.807781+0000","last_became_active":"2024-09-10T21:17:40.103347+0000","last_became_peered":"2024-09-10T21:17:40.103347+0000","last_unstale":"2024-09-10T21:17:42.807781+0000","last_undegraded":"2024-09-10T21:17:42.807781+0000","last_fullsized":"2024-09-10T21:17:42.807781+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:14.981740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808191+0000","last_change":"2024-09-10T21:17:41.110275+0000","last_active":"2024-09-10T21:17:42.808191+0000","last_peered":"2024-09-10T21:17:42.808191+0000","last_clean":"2024-09-10T21:17:42.808191+0000","last_became_active":"2024-09-10T21:17:41.109945+0000","last_became_peered":"2024-09-10T21:17:41.109945+0000","last_unstale":"2024-09-10T21:17:42.808191+0000","last_undegraded":"2024-09-10T21:17:42.808191+0000","last_fullsized":"2024-09-10T21:17:42.808191+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:10.883358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805890+0000","last_change":"2024-09-10T21:17:40.107336+0000","last_active":"2024-09-10T21:17:42.805890+0000","last_peered":"2024-09-10T21:17:42.805890+0000","last_clean":"2024-09-10T21:17:42.805890+0000","last_became_active":"2024-09-10T21:17:40.106303+0000","last_became_peered":"2024-09-10T21:17:40.106303+0000","last_unstale":"2024-09-10T21:17:42.805890+0000","last_undegraded":"2024-09-10T21:17:42.805890+0000","last_fullsized":"2024-09-10T21:17:42.805890+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:55.297764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808380+0000","last_change":"2024-09-10T21:17:41.130115+0000","last_active":"2024-09-10T21:17:42.808380+0000","last_peered":"2024-09-10T21:17:42.808380+0000","last_clean":"2024-09-10T21:17:42.808380+0000","last_became_active":"2024-09-10T21:17:41.129815+0000","last_became_peered":"2024-09-10T21:17:41.129815+0000","last_unstale":"2024-09-10T21:17:42.808380+0000","last_undegraded":"2024-09-10T21:17:42.808380+0000","last_fullsized":"2024-09-10T21:17:42.808380+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:30.750812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806219+0000","last_change":"2024-09-10T21:17:40.109507+0000","last_active":"2024-09-10T21:17:42.806219+0000","last_peered":"2024-09-10T21:17:42.806219+0000","last_clean":"2024-09-10T21:17:42.806219+0000","last_became_active":"2024-09-10T21:17:40.108093+0000","last_became_peered":"2024-09-10T21:17:40.108093+0000","last_unstale":"2024-09-10T21:17:42.806219+0000","last_undegraded":"2024-09-10T21:17:42.806219+0000","last_fullsized":"2024-09-10T21:17:42.806219+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:34.048006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805866+0000","last_change":"2024-09-10T21:17:40.104038+0000","last_active":"2024-09-10T21:17:42.805866+0000","last_peered":"2024-09-10T21:17:42.805866+0000","last_clean":"2024-09-10T21:17:42.805866+0000","last_became_active":"2024-09-10T21:17:40.103746+0000","last_became_peered":"2024-09-10T21:17:40.103746+0000","last_unstale":"2024-09-10T21:17:42.805866+0000","last_undegraded":"2024-09-10T21:17:42.805866+0000","last_fullsized":"2024-09-10T21:17:42.805866+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:30.390294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805536+0000","last_change":"2024-09-10T21:17:41.128674+0000","last_active":"2024-09-10T21:17:42.805536+0000","last_peered":"2024-09-10T21:17:42.805536+0000","last_clean":"2024-09-10T21:17:42.805536+0000","last_became_active":"2024-09-10T21:17:41.128331+0000","last_became_peered":"2024-09-10T21:17:41.128331+0000","last_unstale":"2024-09-10T21:17:42.805536+0000","last_undegraded":"2024-09-10T21:17:42.805536+0000","last_fullsized":"2024-09-10T21:17:42.805536+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:36.702717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:40.075516+0000","last_change":"2024-09-10T21:17:40.075516+0000","last_active":"2024-09-10T21:17:40.075516+0000","last_peered":"2024-09-10T21:17:40.075516+0000","last_clean":"2024-09-10T21:17:40.075516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:40.075516+0000","last_undegraded":"2024-09-10T21:17:40.075516+0000","last_fullsized":"2024-09-10T21:17:40.075516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:17:39.072506+0000","last_change":"2024-09-10T21:17:39.072506+0000","last_active":"2024-09-10T21:17:39.072506+0000","last_peered":"2024-09-10T21:17:39.072506+0000","last_clean":"2024-09-10T21:17:39.072506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:17:39.072506+0000","last_undegraded":"2024-09-10T21:17:39.072506+0000","last_fullsized":"2024-09-10T21:17:39.072506+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":278,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807515+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:17:42.807515+0000","last_peered":"2024-09-10T21:17:42.807515+0000","last_clean":"2024-09-10T21:17:42.807515+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:17:42.807515+0000","last_undegraded":"2024-09-10T21:17:42.807515+0000","last_fullsized":"2024-09-10T21:17:42.807515+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":2},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":12,"acting":12,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36712,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1392640,"data_stored":594798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509533,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37296,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":1986560,"data_stored":1185166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36716,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335124,"statfs":{"total":96636764160,"available":96599166976,"internally_reserved":0,"allocated":1392640,"data_stored":594798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32548,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1392640,"data_stored":594798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509532,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32560,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1392640,"data_stored":594798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32548,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1392640,"data_stored":594798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542236,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40912,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330928,"statfs":{"total":96636764160,"available":96594870272,"internally_reserved":0,"allocated":1433600,"data_stored":613699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542235,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37352,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334488,"statfs":{"total":96636764160,"available":96598515712,"internally_reserved":0,"allocated":2052096,"data_stored":1211967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:17:45.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:17:45.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.445+0000 7f66d0200640 1 -- 172.21.15.16:0/408707707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c80cd6f0 msgr2=0x7f66c80726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:45.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.445+0000 7f66d0200640 1 --2- 172.21.15.16:0/408707707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c80cd6f0 0x7f66c80726b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f66b400b3f0 tx=0x7f66b401d180 comp rx=0 tx=0).stop 2024-09-10T21:17:45.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.445+0000 7f66d0200640 1 -- 172.21.15.16:0/408707707 shutdown_connections 2024-09-10T21:17:45.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.445+0000 7f66d0200640 1 --2- 172.21.15.16:0/408707707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c80cd6f0 0x7f66c80726b0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.445+0000 7f66d0200640 1 -- 172.21.15.16:0/408707707 >> 172.21.15.16:0/408707707 conn(0x7f66c80cfb20 msgr2=0x7f66c80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:45.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.445+0000 7f66d0200640 1 -- 172.21.15.16:0/408707707 shutdown_connections 2024-09-10T21:17:45.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.445+0000 7f66d0200640 1 -- 172.21.15.16:0/408707707 wait complete. 2024-09-10T21:17:45.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66d0200640 1 Processor -- start 2024-09-10T21:17:45.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66d0200640 1 -- start start 2024-09-10T21:17:45.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66d0200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c80ca170 0x7f66c814bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:45.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66d0200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c80cd6f0 0x7f66c80ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:45.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66d0200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66c80cf370 0x7f66c815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:45.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66d0200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66c81698f0 con 0x7f66c80cd6f0 2024-09-10T21:17:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66d0200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8169770 con 0x7f66c80ca170 2024-09-10T21:17:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66d0200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f66c8169a70 con 0x7f66c80cf370 2024-09-10T21:17:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66cce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c80cd6f0 0x7f66c80ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66cce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c80cd6f0 0x7f66c80ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43608/0 (socket says 172.21.15.16:43608) 2024-09-10T21:17:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66cce00640 1 -- 172.21.15.16:0/2405434221 learned_addr learned my addr 172.21.15.16:0/2405434221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66cd800640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c80ca170 0x7f66c814bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66ce200640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66c80cf370 0x7f66c815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66cce00640 1 -- 172.21.15.16:0/2405434221 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66c80cf370 msgr2=0x7f66c815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66cce00640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66c80cf370 0x7f66c815bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66cce00640 1 -- 172.21.15.16:0/2405434221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c80ca170 msgr2=0x7f66c814bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66cce00640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c80ca170 0x7f66c814bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66cce00640 1 -- 172.21.15.16:0/2405434221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c806f720 con 0x7f66c80cd6f0 2024-09-10T21:17:45.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66cd800640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c80ca170 0x7f66c814bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:45.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66ce200640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66c80cf370 0x7f66c815bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:45.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66cce00640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c80cd6f0 0x7f66c80ad850 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f66b4019d90 tx=0x7f66b40a5e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:45.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66c5600640 1 -- 172.21.15.16:0/2405434221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40b1070 con 0x7f66c80cd6f0 2024-09-10T21:17:45.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66c5600640 1 -- 172.21.15.16:0/2405434221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66b40b2040 con 0x7f66c80cd6f0 2024-09-10T21:17:45.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66c5600640 1 -- 172.21.15.16:0/2405434221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40053d0 con 0x7f66c80cd6f0 2024-09-10T21:17:45.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66d0200640 1 -- 172.21.15.16:0/2405434221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c806fa00 con 0x7f66c80cd6f0 2024-09-10T21:17:45.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66d0200640 1 -- 172.21.15.16:0/2405434221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c815bab0 con 0x7f66c80cd6f0 2024-09-10T21:17:45.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.449+0000 7f66d0200640 1 -- 172.21.15.16:0/2405434221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c80ca550 con 0x7f66c80cd6f0 2024-09-10T21:17:45.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.453+0000 7f66c5600640 1 -- 172.21.15.16:0/2405434221 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f66b40092b0 con 0x7f66c80cd6f0 2024-09-10T21:17:45.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.453+0000 7f66c5600640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66a80725e0 0x7f66a8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:45.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.453+0000 7f66c5600640 1 -- 172.21.15.16:0/2405434221 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f66b412d560 con 0x7f66c80cd6f0 2024-09-10T21:17:45.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.453+0000 7f66cd800640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66a80725e0 0x7f66a8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:45.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.453+0000 7f66cd800640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66a80725e0 0x7f66a8074aa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f66bc001750 tx=0x7f66bc009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:45.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.457+0000 7f66c5600640 1 -- 172.21.15.16:0/2405434221 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f66b4016d70 con 0x7f66c80cd6f0 2024-09-10T21:17:45.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.629+0000 7f66d0200640 1 -- 172.21.15.16:0/2405434221 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66c8076c50 con 0x7f66a80725e0 2024-09-10T21:17:45.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.633+0000 7f66c5600640 1 -- 172.21.15.16:0/2405434221 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252471 (secure 0 0 0) 0x7f66c8076c50 con 0x7f66a80725e0 2024-09-10T21:17:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:45.641 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:17:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.637+0000 7f66d0200640 1 -- 172.21.15.16:0/2405434221 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66a80725e0 msgr2=0x7f66a8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.637+0000 7f66d0200640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66a80725e0 0x7f66a8074aa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f66bc001750 tx=0x7f66bc009040 comp rx=0 tx=0).stop 2024-09-10T21:17:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.637+0000 7f66d0200640 1 -- 172.21.15.16:0/2405434221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c80cd6f0 msgr2=0x7f66c80ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.637+0000 7f66d0200640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c80cd6f0 0x7f66c80ad850 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f66b4019d90 tx=0x7f66b40a5e80 comp rx=0 tx=0).stop 2024-09-10T21:17:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.637+0000 7f66d0200640 1 -- 172.21.15.16:0/2405434221 shutdown_connections 2024-09-10T21:17:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.637+0000 7f66d0200640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66a80725e0 0x7f66a8074aa0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.637+0000 7f66d0200640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66c80cf370 0x7f66c815bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.637+0000 7f66d0200640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c80cd6f0 0x7f66c80ad850 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.637+0000 7f66d0200640 1 --2- 172.21.15.16:0/2405434221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c80ca170 0x7f66c814bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.637+0000 7f66d0200640 1 -- 172.21.15.16:0/2405434221 >> 172.21.15.16:0/2405434221 conn(0x7f66c80cfb20 msgr2=0x7f66c815dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:45.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.637+0000 7f66d0200640 1 -- 172.21.15.16:0/2405434221 shutdown_connections 2024-09-10T21:17:45.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.637+0000 7f66d0200640 1 -- 172.21.15.16:0/2405434221 wait complete. 2024-09-10T21:17:45.660 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-10T21:17:45.557026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":286952,"kb_used_data":12476,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754687768,"statfs":{"total":773094113280,"available":772800274432,"internally_reserved":0,"allocated":12775424,"data_stored":6138976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001024"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807251+0000","last_change":"2024-09-10T21:17:41.111383+0000","last_active":"2024-09-10T21:17:42.807251+0000","last_peered":"2024-09-10T21:17:42.807251+0000","last_clean":"2024-09-10T21:17:42.807251+0000","last_became_active":"2024-09-10T21:17:41.111011+0000","last_became_peered":"2024-09-10T21:17:41.111011+0000","last_unstale":"2024-09-10T21:17:42.807251+0000","last_undegraded":"2024-09-10T21:17:42.807251+0000","last_fullsized":"2024-09-10T21:17:42.807251+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:53.582959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807459+0000","last_change":"2024-09-10T21:17:41.098954+0000","last_active":"2024-09-10T21:17:42.807459+0000","last_peered":"2024-09-10T21:17:42.807459+0000","last_clean":"2024-09-10T21:17:42.807459+0000","last_became_active":"2024-09-10T21:17:41.098648+0000","last_became_peered":"2024-09-10T21:17:41.098648+0000","last_unstale":"2024-09-10T21:17:42.807459+0000","last_undegraded":"2024-09-10T21:17:42.807459+0000","last_fullsized":"2024-09-10T21:17:42.807459+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:41.614876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808445+0000","last_change":"2024-09-10T21:17:41.109228+0000","last_active":"2024-09-10T21:17:42.808445+0000","last_peered":"2024-09-10T21:17:42.808445+0000","last_clean":"2024-09-10T21:17:42.808445+0000","last_became_active":"2024-09-10T21:17:41.108963+0000","last_became_peered":"2024-09-10T21:17:41.108963+0000","last_unstale":"2024-09-10T21:17:42.808445+0000","last_undegraded":"2024-09-10T21:17:42.808445+0000","last_fullsized":"2024-09-10T21:17:42.808445+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:54.815174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.112623+0000","last_change":"2024-09-10T21:17:41.103394+0000","last_active":"2024-09-10T21:17:42.112623+0000","last_peered":"2024-09-10T21:17:42.112623+0000","last_clean":"2024-09-10T21:17:42.112623+0000","last_became_active":"2024-09-10T21:17:41.103085+0000","last_became_peered":"2024-09-10T21:17:41.103085+0000","last_unstale":"2024-09-10T21:17:42.112623+0000","last_undegraded":"2024-09-10T21:17:42.112623+0000","last_fullsized":"2024-09-10T21:17:42.112623+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:05.539094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.112900+0000","last_change":"2024-09-10T21:17:41.132020+0000","last_active":"2024-09-10T21:17:42.112900+0000","last_peered":"2024-09-10T21:17:42.112900+0000","last_clean":"2024-09-10T21:17:42.112900+0000","last_became_active":"2024-09-10T21:17:41.131680+0000","last_became_peered":"2024-09-10T21:17:41.131680+0000","last_unstale":"2024-09-10T21:17:42.112900+0000","last_undegraded":"2024-09-10T21:17:42.112900+0000","last_fullsized":"2024-09-10T21:17:42.112900+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:52.659468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114116+0000","last_change":"2024-09-10T21:17:41.107312+0000","last_active":"2024-09-10T21:17:42.114116+0000","last_peered":"2024-09-10T21:17:42.114116+0000","last_clean":"2024-09-10T21:17:42.114116+0000","last_became_active":"2024-09-10T21:17:41.107035+0000","last_became_peered":"2024-09-10T21:17:41.107035+0000","last_unstale":"2024-09-10T21:17:42.114116+0000","last_undegraded":"2024-09-10T21:17:42.114116+0000","last_fullsized":"2024-09-10T21:17:42.114116+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:58.853676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.112966+0000","last_change":"2024-09-10T21:17:41.113839+0000","last_active":"2024-09-10T21:17:42.112966+0000","last_peered":"2024-09-10T21:17:42.112966+0000","last_clean":"2024-09-10T21:17:42.112966+0000","last_became_active":"2024-09-10T21:17:41.113530+0000","last_became_peered":"2024-09-10T21:17:41.113530+0000","last_unstale":"2024-09-10T21:17:42.112966+0000","last_undegraded":"2024-09-10T21:17:42.112966+0000","last_fullsized":"2024-09-10T21:17:42.112966+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:54.254925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.112737+0000","last_change":"2024-09-10T21:17:41.100167+0000","last_active":"2024-09-10T21:17:42.112737+0000","last_peered":"2024-09-10T21:17:42.112737+0000","last_clean":"2024-09-10T21:17:42.112737+0000","last_became_active":"2024-09-10T21:17:41.099890+0000","last_became_peered":"2024-09-10T21:17:41.099890+0000","last_unstale":"2024-09-10T21:17:42.112737+0000","last_undegraded":"2024-09-10T21:17:42.112737+0000","last_fullsized":"2024-09-10T21:17:42.112737+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:24.671984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808614+0000","last_change":"2024-09-10T21:17:41.108633+0000","last_active":"2024-09-10T21:17:42.808614+0000","last_peered":"2024-09-10T21:17:42.808614+0000","last_clean":"2024-09-10T21:17:42.808614+0000","last_became_active":"2024-09-10T21:17:41.108243+0000","last_became_peered":"2024-09-10T21:17:41.108243+0000","last_unstale":"2024-09-10T21:17:42.808614+0000","last_undegraded":"2024-09-10T21:17:42.808614+0000","last_fullsized":"2024-09-10T21:17:42.808614+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:27.070162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807625+0000","last_change":"2024-09-10T21:17:41.115018+0000","last_active":"2024-09-10T21:17:42.807625+0000","last_peered":"2024-09-10T21:17:42.807625+0000","last_clean":"2024-09-10T21:17:42.807625+0000","last_became_active":"2024-09-10T21:17:41.114110+0000","last_became_peered":"2024-09-10T21:17:41.114110+0000","last_unstale":"2024-09-10T21:17:42.807625+0000","last_undegraded":"2024-09-10T21:17:42.807625+0000","last_fullsized":"2024-09-10T21:17:42.807625+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:22.299269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.113177+0000","last_change":"2024-09-10T21:17:41.108235+0000","last_active":"2024-09-10T21:17:42.113177+0000","last_peered":"2024-09-10T21:17:42.113177+0000","last_clean":"2024-09-10T21:17:42.113177+0000","last_became_active":"2024-09-10T21:17:41.107926+0000","last_became_peered":"2024-09-10T21:17:41.107926+0000","last_unstale":"2024-09-10T21:17:42.113177+0000","last_undegraded":"2024-09-10T21:17:42.113177+0000","last_fullsized":"2024-09-10T21:17:42.113177+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:42.505627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114650+0000","last_change":"2024-09-10T21:17:41.128474+0000","last_active":"2024-09-10T21:17:42.114650+0000","last_peered":"2024-09-10T21:17:42.114650+0000","last_clean":"2024-09-10T21:17:42.114650+0000","last_became_active":"2024-09-10T21:17:41.127379+0000","last_became_peered":"2024-09-10T21:17:41.127379+0000","last_unstale":"2024-09-10T21:17:42.114650+0000","last_undegraded":"2024-09-10T21:17:42.114650+0000","last_fullsized":"2024-09-10T21:17:42.114650+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:19.369159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.112713+0000","last_change":"2024-09-10T21:17:41.106544+0000","last_active":"2024-09-10T21:17:42.112713+0000","last_peered":"2024-09-10T21:17:42.112713+0000","last_clean":"2024-09-10T21:17:42.112713+0000","last_became_active":"2024-09-10T21:17:41.106257+0000","last_became_peered":"2024-09-10T21:17:41.106257+0000","last_unstale":"2024-09-10T21:17:42.112713+0000","last_undegraded":"2024-09-10T21:17:42.112713+0000","last_fullsized":"2024-09-10T21:17:42.112713+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:20.214004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806268+0000","last_change":"2024-09-10T21:17:41.106055+0000","last_active":"2024-09-10T21:17:42.806268+0000","last_peered":"2024-09-10T21:17:42.806268+0000","last_clean":"2024-09-10T21:17:42.806268+0000","last_became_active":"2024-09-10T21:17:41.105812+0000","last_became_peered":"2024-09-10T21:17:41.105812+0000","last_unstale":"2024-09-10T21:17:42.806268+0000","last_undegraded":"2024-09-10T21:17:42.806268+0000","last_fullsized":"2024-09-10T21:17:42.806268+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:50.905513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806293+0000","last_change":"2024-09-10T21:17:41.113139+0000","last_active":"2024-09-10T21:17:42.806293+0000","last_peered":"2024-09-10T21:17:42.806293+0000","last_clean":"2024-09-10T21:17:42.806293+0000","last_became_active":"2024-09-10T21:17:41.112880+0000","last_became_peered":"2024-09-10T21:17:41.112880+0000","last_unstale":"2024-09-10T21:17:42.806293+0000","last_undegraded":"2024-09-10T21:17:42.806293+0000","last_fullsized":"2024-09-10T21:17:42.806293+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:01.443372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805676+0000","last_change":"2024-09-10T21:17:41.113569+0000","last_active":"2024-09-10T21:17:42.805676+0000","last_peered":"2024-09-10T21:17:42.805676+0000","last_clean":"2024-09-10T21:17:42.805676+0000","last_became_active":"2024-09-10T21:17:41.113295+0000","last_became_peered":"2024-09-10T21:17:41.113295+0000","last_unstale":"2024-09-10T21:17:42.805676+0000","last_undegraded":"2024-09-10T21:17:42.805676+0000","last_fullsized":"2024-09-10T21:17:42.805676+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:04.841353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808872+0000","last_change":"2024-09-10T21:17:41.108905+0000","last_active":"2024-09-10T21:17:42.808872+0000","last_peered":"2024-09-10T21:17:42.808872+0000","last_clean":"2024-09-10T21:17:42.808872+0000","last_became_active":"2024-09-10T21:17:41.108626+0000","last_became_peered":"2024-09-10T21:17:41.108626+0000","last_unstale":"2024-09-10T21:17:42.808872+0000","last_undegraded":"2024-09-10T21:17:42.808872+0000","last_fullsized":"2024-09-10T21:17:42.808872+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:33.330180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805905+0000","last_change":"2024-09-10T21:17:41.110077+0000","last_active":"2024-09-10T21:17:42.805905+0000","last_peered":"2024-09-10T21:17:42.805905+0000","last_clean":"2024-09-10T21:17:42.805905+0000","last_became_active":"2024-09-10T21:17:41.109774+0000","last_became_peered":"2024-09-10T21:17:41.109774+0000","last_unstale":"2024-09-10T21:17:42.805905+0000","last_undegraded":"2024-09-10T21:17:42.805905+0000","last_fullsized":"2024-09-10T21:17:42.805905+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:00.040600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807200+0000","last_change":"2024-09-10T21:17:41.101047+0000","last_active":"2024-09-10T21:17:42.807200+0000","last_peered":"2024-09-10T21:17:42.807200+0000","last_clean":"2024-09-10T21:17:42.807200+0000","last_became_active":"2024-09-10T21:17:41.100428+0000","last_became_peered":"2024-09-10T21:17:41.100428+0000","last_unstale":"2024-09-10T21:17:42.807200+0000","last_undegraded":"2024-09-10T21:17:42.807200+0000","last_fullsized":"2024-09-10T21:17:42.807200+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:24.381739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807155+0000","last_change":"2024-09-10T21:17:41.112501+0000","last_active":"2024-09-10T21:17:42.807155+0000","last_peered":"2024-09-10T21:17:42.807155+0000","last_clean":"2024-09-10T21:17:42.807155+0000","last_became_active":"2024-09-10T21:17:41.112180+0000","last_became_peered":"2024-09-10T21:17:41.112180+0000","last_unstale":"2024-09-10T21:17:42.807155+0000","last_undegraded":"2024-09-10T21:17:42.807155+0000","last_fullsized":"2024-09-10T21:17:42.807155+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:14.476873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114362+0000","last_change":"2024-09-10T21:17:41.127822+0000","last_active":"2024-09-10T21:17:42.114362+0000","last_peered":"2024-09-10T21:17:42.114362+0000","last_clean":"2024-09-10T21:17:42.114362+0000","last_became_active":"2024-09-10T21:17:41.126535+0000","last_became_peered":"2024-09-10T21:17:41.126535+0000","last_unstale":"2024-09-10T21:17:42.114362+0000","last_undegraded":"2024-09-10T21:17:42.114362+0000","last_fullsized":"2024-09-10T21:17:42.114362+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:34.715618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807463+0000","last_change":"2024-09-10T21:17:41.104012+0000","last_active":"2024-09-10T21:17:42.807463+0000","last_peered":"2024-09-10T21:17:42.807463+0000","last_clean":"2024-09-10T21:17:42.807463+0000","last_became_active":"2024-09-10T21:17:41.102621+0000","last_became_peered":"2024-09-10T21:17:41.102621+0000","last_unstale":"2024-09-10T21:17:42.807463+0000","last_undegraded":"2024-09-10T21:17:42.807463+0000","last_fullsized":"2024-09-10T21:17:42.807463+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:20.232350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807747+0000","last_change":"2024-09-10T21:17:41.119900+0000","last_active":"2024-09-10T21:17:42.807747+0000","last_peered":"2024-09-10T21:17:42.807747+0000","last_clean":"2024-09-10T21:17:42.807747+0000","last_became_active":"2024-09-10T21:17:41.119550+0000","last_became_peered":"2024-09-10T21:17:41.119550+0000","last_unstale":"2024-09-10T21:17:42.807747+0000","last_undegraded":"2024-09-10T21:17:42.807747+0000","last_fullsized":"2024-09-10T21:17:42.807747+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:35.051017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805457+0000","last_change":"2024-09-10T21:17:41.131830+0000","last_active":"2024-09-10T21:17:42.805457+0000","last_peered":"2024-09-10T21:17:42.805457+0000","last_clean":"2024-09-10T21:17:42.805457+0000","last_became_active":"2024-09-10T21:17:41.131569+0000","last_became_peered":"2024-09-10T21:17:41.131569+0000","last_unstale":"2024-09-10T21:17:42.805457+0000","last_undegraded":"2024-09-10T21:17:42.805457+0000","last_fullsized":"2024-09-10T21:17:42.805457+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:18.442664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.113111+0000","last_change":"2024-09-10T21:17:41.104403+0000","last_active":"2024-09-10T21:17:42.113111+0000","last_peered":"2024-09-10T21:17:42.113111+0000","last_clean":"2024-09-10T21:17:42.113111+0000","last_became_active":"2024-09-10T21:17:41.103649+0000","last_became_peered":"2024-09-10T21:17:41.103649+0000","last_unstale":"2024-09-10T21:17:42.113111+0000","last_undegraded":"2024-09-10T21:17:42.113111+0000","last_fullsized":"2024-09-10T21:17:42.113111+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:10.812197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807803+0000","last_change":"2024-09-10T21:17:41.100705+0000","last_active":"2024-09-10T21:17:42.807803+0000","last_peered":"2024-09-10T21:17:42.807803+0000","last_clean":"2024-09-10T21:17:42.807803+0000","last_became_active":"2024-09-10T21:17:41.100385+0000","last_became_peered":"2024-09-10T21:17:41.100385+0000","last_unstale":"2024-09-10T21:17:42.807803+0000","last_undegraded":"2024-09-10T21:17:42.807803+0000","last_fullsized":"2024-09-10T21:17:42.807803+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:46.827736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806999+0000","last_change":"2024-09-10T21:17:41.124089+0000","last_active":"2024-09-10T21:17:42.806999+0000","last_peered":"2024-09-10T21:17:42.806999+0000","last_clean":"2024-09-10T21:17:42.806999+0000","last_became_active":"2024-09-10T21:17:41.123767+0000","last_became_peered":"2024-09-10T21:17:41.123767+0000","last_unstale":"2024-09-10T21:17:42.806999+0000","last_undegraded":"2024-09-10T21:17:42.806999+0000","last_fullsized":"2024-09-10T21:17:42.806999+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:39.456050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808094+0000","last_change":"2024-09-10T21:17:41.106857+0000","last_active":"2024-09-10T21:17:42.808094+0000","last_peered":"2024-09-10T21:17:42.808094+0000","last_clean":"2024-09-10T21:17:42.808094+0000","last_became_active":"2024-09-10T21:17:41.106561+0000","last_became_peered":"2024-09-10T21:17:41.106561+0000","last_unstale":"2024-09-10T21:17:42.808094+0000","last_undegraded":"2024-09-10T21:17:42.808094+0000","last_fullsized":"2024-09-10T21:17:42.808094+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:23.543864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806796+0000","last_change":"2024-09-10T21:17:41.107605+0000","last_active":"2024-09-10T21:17:42.806796+0000","last_peered":"2024-09-10T21:17:42.806796+0000","last_clean":"2024-09-10T21:17:42.806796+0000","last_became_active":"2024-09-10T21:17:41.107300+0000","last_became_peered":"2024-09-10T21:17:41.107300+0000","last_unstale":"2024-09-10T21:17:42.806796+0000","last_undegraded":"2024-09-10T21:17:42.806796+0000","last_fullsized":"2024-09-10T21:17:42.806796+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:39.010140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807614+0000","last_change":"2024-09-10T21:17:41.103520+0000","last_active":"2024-09-10T21:17:42.807614+0000","last_peered":"2024-09-10T21:17:42.807614+0000","last_clean":"2024-09-10T21:17:42.807614+0000","last_became_active":"2024-09-10T21:17:41.103221+0000","last_became_peered":"2024-09-10T21:17:41.103221+0000","last_unstale":"2024-09-10T21:17:42.807614+0000","last_undegraded":"2024-09-10T21:17:42.807614+0000","last_fullsized":"2024-09-10T21:17:42.807614+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:54.880857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808503+0000","last_change":"2024-09-10T21:17:41.111524+0000","last_active":"2024-09-10T21:17:42.808503+0000","last_peered":"2024-09-10T21:17:42.808503+0000","last_clean":"2024-09-10T21:17:42.808503+0000","last_became_active":"2024-09-10T21:17:41.111265+0000","last_became_peered":"2024-09-10T21:17:41.111265+0000","last_unstale":"2024-09-10T21:17:42.808503+0000","last_undegraded":"2024-09-10T21:17:42.808503+0000","last_fullsized":"2024-09-10T21:17:42.808503+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:40.878106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808565+0000","last_change":"2024-09-10T21:17:41.111284+0000","last_active":"2024-09-10T21:17:42.808565+0000","last_peered":"2024-09-10T21:17:42.808565+0000","last_clean":"2024-09-10T21:17:42.808565+0000","last_became_active":"2024-09-10T21:17:41.109599+0000","last_became_peered":"2024-09-10T21:17:41.109599+0000","last_unstale":"2024-09-10T21:17:42.808565+0000","last_undegraded":"2024-09-10T21:17:42.808565+0000","last_fullsized":"2024-09-10T21:17:42.808565+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:40.021891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806806+0000","last_change":"2024-09-10T21:17:41.107410+0000","last_active":"2024-09-10T21:17:42.806806+0000","last_peered":"2024-09-10T21:17:42.806806+0000","last_clean":"2024-09-10T21:17:42.806806+0000","last_became_active":"2024-09-10T21:17:41.107083+0000","last_became_peered":"2024-09-10T21:17:41.107083+0000","last_unstale":"2024-09-10T21:17:42.806806+0000","last_undegraded":"2024-09-10T21:17:42.806806+0000","last_fullsized":"2024-09-10T21:17:42.806806+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:59.401864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"115'4","reported_seq":19,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.113700+0000","last_change":"2024-09-10T21:17:40.096270+0000","last_active":"2024-09-10T21:17:42.113700+0000","last_peered":"2024-09-10T21:17:42.113700+0000","last_clean":"2024-09-10T21:17:42.113700+0000","last_became_active":"2024-09-10T21:17:40.096009+0000","last_became_peered":"2024-09-10T21:17:40.096009+0000","last_unstale":"2024-09-10T21:17:42.113700+0000","last_undegraded":"2024-09-10T21:17:42.113700+0000","last_fullsized":"2024-09-10T21:17:42.113700+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:07.405706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808693+0000","last_change":"2024-09-10T21:17:41.103718+0000","last_active":"2024-09-10T21:17:42.808693+0000","last_peered":"2024-09-10T21:17:42.808693+0000","last_clean":"2024-09-10T21:17:42.808693+0000","last_became_active":"2024-09-10T21:17:41.103186+0000","last_became_peered":"2024-09-10T21:17:41.103186+0000","last_unstale":"2024-09-10T21:17:42.808693+0000","last_undegraded":"2024-09-10T21:17:42.808693+0000","last_fullsized":"2024-09-10T21:17:42.808693+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:13.640041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808788+0000","last_change":"2024-09-10T21:17:40.103449+0000","last_active":"2024-09-10T21:17:42.808788+0000","last_peered":"2024-09-10T21:17:42.808788+0000","last_clean":"2024-09-10T21:17:42.808788+0000","last_became_active":"2024-09-10T21:17:40.103132+0000","last_became_peered":"2024-09-10T21:17:40.103132+0000","last_unstale":"2024-09-10T21:17:42.808788+0000","last_undegraded":"2024-09-10T21:17:42.808788+0000","last_fullsized":"2024-09-10T21:17:42.808788+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:33.127608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808238+0000","last_change":"2024-09-10T21:17:41.111769+0000","last_active":"2024-09-10T21:17:42.808238+0000","last_peered":"2024-09-10T21:17:42.808238+0000","last_clean":"2024-09-10T21:17:42.808238+0000","last_became_active":"2024-09-10T21:17:41.111493+0000","last_became_peered":"2024-09-10T21:17:41.111493+0000","last_unstale":"2024-09-10T21:17:42.808238+0000","last_undegraded":"2024-09-10T21:17:42.808238+0000","last_fullsized":"2024-09-10T21:17:42.808238+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:25.844705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.113094+0000","last_change":"2024-09-10T21:17:40.102642+0000","last_active":"2024-09-10T21:17:42.113094+0000","last_peered":"2024-09-10T21:17:42.113094+0000","last_clean":"2024-09-10T21:17:42.113094+0000","last_became_active":"2024-09-10T21:17:40.102335+0000","last_became_peered":"2024-09-10T21:17:40.102335+0000","last_unstale":"2024-09-10T21:17:42.113094+0000","last_undegraded":"2024-09-10T21:17:42.113094+0000","last_fullsized":"2024-09-10T21:17:42.113094+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:38.007129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114780+0000","last_change":"2024-09-10T21:17:41.109393+0000","last_active":"2024-09-10T21:17:42.114780+0000","last_peered":"2024-09-10T21:17:42.114780+0000","last_clean":"2024-09-10T21:17:42.114780+0000","last_became_active":"2024-09-10T21:17:41.109123+0000","last_became_peered":"2024-09-10T21:17:41.109123+0000","last_unstale":"2024-09-10T21:17:42.114780+0000","last_undegraded":"2024-09-10T21:17:42.114780+0000","last_fullsized":"2024-09-10T21:17:42.114780+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:15.175350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807812+0000","last_change":"2024-09-10T21:17:40.109859+0000","last_active":"2024-09-10T21:17:42.807812+0000","last_peered":"2024-09-10T21:17:42.807812+0000","last_clean":"2024-09-10T21:17:42.807812+0000","last_became_active":"2024-09-10T21:17:40.108511+0000","last_became_peered":"2024-09-10T21:17:40.108511+0000","last_unstale":"2024-09-10T21:17:42.807812+0000","last_undegraded":"2024-09-10T21:17:42.807812+0000","last_fullsized":"2024-09-10T21:17:42.807812+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:11.145046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805743+0000","last_change":"2024-09-10T21:17:41.116782+0000","last_active":"2024-09-10T21:17:42.805743+0000","last_peered":"2024-09-10T21:17:42.805743+0000","last_clean":"2024-09-10T21:17:42.805743+0000","last_became_active":"2024-09-10T21:17:41.116353+0000","last_became_peered":"2024-09-10T21:17:41.116353+0000","last_unstale":"2024-09-10T21:17:42.805743+0000","last_undegraded":"2024-09-10T21:17:42.805743+0000","last_fullsized":"2024-09-10T21:17:42.805743+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:37.175886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806748+0000","last_change":"2024-09-10T21:17:40.103097+0000","last_active":"2024-09-10T21:17:42.806748+0000","last_peered":"2024-09-10T21:17:42.806748+0000","last_clean":"2024-09-10T21:17:42.806748+0000","last_became_active":"2024-09-10T21:17:40.102726+0000","last_became_peered":"2024-09-10T21:17:40.102726+0000","last_unstale":"2024-09-10T21:17:42.806748+0000","last_undegraded":"2024-09-10T21:17:42.806748+0000","last_fullsized":"2024-09-10T21:17:42.806748+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:32.778095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114444+0000","last_change":"2024-09-10T21:17:41.127997+0000","last_active":"2024-09-10T21:17:42.114444+0000","last_peered":"2024-09-10T21:17:42.114444+0000","last_clean":"2024-09-10T21:17:42.114444+0000","last_became_active":"2024-09-10T21:17:41.127149+0000","last_became_peered":"2024-09-10T21:17:41.127149+0000","last_unstale":"2024-09-10T21:17:42.114444+0000","last_undegraded":"2024-09-10T21:17:42.114444+0000","last_fullsized":"2024-09-10T21:17:42.114444+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:55.952079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806131+0000","last_change":"2024-09-10T21:17:40.108290+0000","last_active":"2024-09-10T21:17:42.806131+0000","last_peered":"2024-09-10T21:17:42.806131+0000","last_clean":"2024-09-10T21:17:42.806131+0000","last_became_active":"2024-09-10T21:17:40.107987+0000","last_became_peered":"2024-09-10T21:17:40.107987+0000","last_unstale":"2024-09-10T21:17:42.806131+0000","last_undegraded":"2024-09-10T21:17:42.806131+0000","last_fullsized":"2024-09-10T21:17:42.806131+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:37.425705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114041+0000","last_change":"2024-09-10T21:17:41.128485+0000","last_active":"2024-09-10T21:17:42.114041+0000","last_peered":"2024-09-10T21:17:42.114041+0000","last_clean":"2024-09-10T21:17:42.114041+0000","last_became_active":"2024-09-10T21:17:41.123666+0000","last_became_peered":"2024-09-10T21:17:41.123666+0000","last_unstale":"2024-09-10T21:17:42.114041+0000","last_undegraded":"2024-09-10T21:17:42.114041+0000","last_fullsized":"2024-09-10T21:17:42.114041+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:19.880919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808112+0000","last_change":"2024-09-10T21:17:40.106257+0000","last_active":"2024-09-10T21:17:42.808112+0000","last_peered":"2024-09-10T21:17:42.808112+0000","last_clean":"2024-09-10T21:17:42.808112+0000","last_became_active":"2024-09-10T21:17:40.105597+0000","last_became_peered":"2024-09-10T21:17:40.105597+0000","last_unstale":"2024-09-10T21:17:42.808112+0000","last_undegraded":"2024-09-10T21:17:42.808112+0000","last_fullsized":"2024-09-10T21:17:42.808112+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:13.031139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806589+0000","last_change":"2024-09-10T21:17:41.101932+0000","last_active":"2024-09-10T21:17:42.806589+0000","last_peered":"2024-09-10T21:17:42.806589+0000","last_clean":"2024-09-10T21:17:42.806589+0000","last_became_active":"2024-09-10T21:17:41.099541+0000","last_became_peered":"2024-09-10T21:17:41.099541+0000","last_unstale":"2024-09-10T21:17:42.806589+0000","last_undegraded":"2024-09-10T21:17:42.806589+0000","last_fullsized":"2024-09-10T21:17:42.806589+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:33.395004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806861+0000","last_change":"2024-09-10T21:17:40.105012+0000","last_active":"2024-09-10T21:17:42.806861+0000","last_peered":"2024-09-10T21:17:42.806861+0000","last_clean":"2024-09-10T21:17:42.806861+0000","last_became_active":"2024-09-10T21:17:40.102472+0000","last_became_peered":"2024-09-10T21:17:40.102472+0000","last_unstale":"2024-09-10T21:17:42.806861+0000","last_undegraded":"2024-09-10T21:17:42.806861+0000","last_fullsized":"2024-09-10T21:17:42.806861+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:23.148541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807958+0000","last_change":"2024-09-10T21:17:41.099941+0000","last_active":"2024-09-10T21:17:42.807958+0000","last_peered":"2024-09-10T21:17:42.807958+0000","last_clean":"2024-09-10T21:17:42.807958+0000","last_became_active":"2024-09-10T21:17:41.099616+0000","last_became_peered":"2024-09-10T21:17:41.099616+0000","last_unstale":"2024-09-10T21:17:42.807958+0000","last_undegraded":"2024-09-10T21:17:42.807958+0000","last_fullsized":"2024-09-10T21:17:42.807958+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:49.995325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114650+0000","last_change":"2024-09-10T21:17:40.101512+0000","last_active":"2024-09-10T21:17:42.114650+0000","last_peered":"2024-09-10T21:17:42.114650+0000","last_clean":"2024-09-10T21:17:42.114650+0000","last_became_active":"2024-09-10T21:17:40.101235+0000","last_became_peered":"2024-09-10T21:17:40.101235+0000","last_unstale":"2024-09-10T21:17:42.114650+0000","last_undegraded":"2024-09-10T21:17:42.114650+0000","last_fullsized":"2024-09-10T21:17:42.114650+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:42.947609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.113416+0000","last_change":"2024-09-10T21:17:41.113038+0000","last_active":"2024-09-10T21:17:42.113416+0000","last_peered":"2024-09-10T21:17:42.113416+0000","last_clean":"2024-09-10T21:17:42.113416+0000","last_became_active":"2024-09-10T21:17:41.112655+0000","last_became_peered":"2024-09-10T21:17:41.112655+0000","last_unstale":"2024-09-10T21:17:42.113416+0000","last_undegraded":"2024-09-10T21:17:42.113416+0000","last_fullsized":"2024-09-10T21:17:42.113416+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:06.889698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806057+0000","last_change":"2024-09-10T21:17:40.107030+0000","last_active":"2024-09-10T21:17:42.806057+0000","last_peered":"2024-09-10T21:17:42.806057+0000","last_clean":"2024-09-10T21:17:42.806057+0000","last_became_active":"2024-09-10T21:17:40.106742+0000","last_became_peered":"2024-09-10T21:17:40.106742+0000","last_unstale":"2024-09-10T21:17:42.806057+0000","last_undegraded":"2024-09-10T21:17:42.806057+0000","last_fullsized":"2024-09-10T21:17:42.806057+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:32.357990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114848+0000","last_change":"2024-09-10T21:17:41.128783+0000","last_active":"2024-09-10T21:17:42.114848+0000","last_peered":"2024-09-10T21:17:42.114848+0000","last_clean":"2024-09-10T21:17:42.114848+0000","last_became_active":"2024-09-10T21:17:41.127656+0000","last_became_peered":"2024-09-10T21:17:41.127656+0000","last_unstale":"2024-09-10T21:17:42.114848+0000","last_undegraded":"2024-09-10T21:17:42.114848+0000","last_fullsized":"2024-09-10T21:17:42.114848+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:54.282200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"115'3","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.113167+0000","last_change":"2024-09-10T21:17:40.092467+0000","last_active":"2024-09-10T21:17:42.113167+0000","last_peered":"2024-09-10T21:17:42.113167+0000","last_clean":"2024-09-10T21:17:42.113167+0000","last_became_active":"2024-09-10T21:17:40.092204+0000","last_became_peered":"2024-09-10T21:17:40.092204+0000","last_unstale":"2024-09-10T21:17:42.113167+0000","last_undegraded":"2024-09-10T21:17:42.113167+0000","last_fullsized":"2024-09-10T21:17:42.113167+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:56.257177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807915+0000","last_change":"2024-09-10T21:17:41.118414+0000","last_active":"2024-09-10T21:17:42.807915+0000","last_peered":"2024-09-10T21:17:42.807915+0000","last_clean":"2024-09-10T21:17:42.807915+0000","last_became_active":"2024-09-10T21:17:41.118142+0000","last_became_peered":"2024-09-10T21:17:41.118142+0000","last_unstale":"2024-09-10T21:17:42.807915+0000","last_undegraded":"2024-09-10T21:17:42.807915+0000","last_fullsized":"2024-09-10T21:17:42.807915+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:37.934038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"115'6","reported_seq":21,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807781+0000","last_change":"2024-09-10T21:17:40.103617+0000","last_active":"2024-09-10T21:17:42.807781+0000","last_peered":"2024-09-10T21:17:42.807781+0000","last_clean":"2024-09-10T21:17:42.807781+0000","last_became_active":"2024-09-10T21:17:40.103347+0000","last_became_peered":"2024-09-10T21:17:40.103347+0000","last_unstale":"2024-09-10T21:17:42.807781+0000","last_undegraded":"2024-09-10T21:17:42.807781+0000","last_fullsized":"2024-09-10T21:17:42.807781+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:14.981740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806572+0000","last_change":"2024-09-10T21:17:41.118286+0000","last_active":"2024-09-10T21:17:42.806572+0000","last_peered":"2024-09-10T21:17:42.806572+0000","last_clean":"2024-09-10T21:17:42.806572+0000","last_became_active":"2024-09-10T21:17:41.118009+0000","last_became_peered":"2024-09-10T21:17:41.118009+0000","last_unstale":"2024-09-10T21:17:42.806572+0000","last_undegraded":"2024-09-10T21:17:42.806572+0000","last_fullsized":"2024-09-10T21:17:42.806572+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:04.193364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806409+0000","last_change":"2024-09-10T21:17:40.110614+0000","last_active":"2024-09-10T21:17:42.806409+0000","last_peered":"2024-09-10T21:17:42.806409+0000","last_clean":"2024-09-10T21:17:42.806409+0000","last_became_active":"2024-09-10T21:17:40.110204+0000","last_became_peered":"2024-09-10T21:17:40.110204+0000","last_unstale":"2024-09-10T21:17:42.806409+0000","last_undegraded":"2024-09-10T21:17:42.806409+0000","last_fullsized":"2024-09-10T21:17:42.806409+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:30:43.381381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805739+0000","last_change":"2024-09-10T21:17:41.108725+0000","last_active":"2024-09-10T21:17:42.805739+0000","last_peered":"2024-09-10T21:17:42.805739+0000","last_clean":"2024-09-10T21:17:42.805739+0000","last_became_active":"2024-09-10T21:17:41.108445+0000","last_became_peered":"2024-09-10T21:17:41.108445+0000","last_unstale":"2024-09-10T21:17:42.805739+0000","last_undegraded":"2024-09-10T21:17:42.805739+0000","last_fullsized":"2024-09-10T21:17:42.805739+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:53.097106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.112784+0000","last_change":"2024-09-10T21:17:40.099850+0000","last_active":"2024-09-10T21:17:42.112784+0000","last_peered":"2024-09-10T21:17:42.112784+0000","last_clean":"2024-09-10T21:17:42.112784+0000","last_became_active":"2024-09-10T21:17:40.098315+0000","last_became_peered":"2024-09-10T21:17:40.098315+0000","last_unstale":"2024-09-10T21:17:42.112784+0000","last_undegraded":"2024-09-10T21:17:42.112784+0000","last_fullsized":"2024-09-10T21:17:42.112784+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:32.391994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114036+0000","last_change":"2024-09-10T21:17:41.110558+0000","last_active":"2024-09-10T21:17:42.114036+0000","last_peered":"2024-09-10T21:17:42.114036+0000","last_clean":"2024-09-10T21:17:42.114036+0000","last_became_active":"2024-09-10T21:17:41.110200+0000","last_became_peered":"2024-09-10T21:17:41.110200+0000","last_unstale":"2024-09-10T21:17:42.114036+0000","last_undegraded":"2024-09-10T21:17:42.114036+0000","last_fullsized":"2024-09-10T21:17:42.114036+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:06.889698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"115'8","reported_seq":23,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114003+0000","last_change":"2024-09-10T21:17:40.103879+0000","last_active":"2024-09-10T21:17:42.114003+0000","last_peered":"2024-09-10T21:17:42.114003+0000","last_clean":"2024-09-10T21:17:42.114003+0000","last_became_active":"2024-09-10T21:17:40.102946+0000","last_became_peered":"2024-09-10T21:17:40.102946+0000","last_unstale":"2024-09-10T21:17:42.114003+0000","last_undegraded":"2024-09-10T21:17:42.114003+0000","last_fullsized":"2024-09-10T21:17:42.114003+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:47.973599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.113285+0000","last_change":"2024-09-10T21:17:41.106772+0000","last_active":"2024-09-10T21:17:42.113285+0000","last_peered":"2024-09-10T21:17:42.113285+0000","last_clean":"2024-09-10T21:17:42.113285+0000","last_became_active":"2024-09-10T21:17:41.106457+0000","last_became_peered":"2024-09-10T21:17:41.106457+0000","last_unstale":"2024-09-10T21:17:42.113285+0000","last_undegraded":"2024-09-10T21:17:42.113285+0000","last_fullsized":"2024-09-10T21:17:42.113285+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:20.194725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114811+0000","last_change":"2024-09-10T21:17:40.100555+0000","last_active":"2024-09-10T21:17:42.114811+0000","last_peered":"2024-09-10T21:17:42.114811+0000","last_clean":"2024-09-10T21:17:42.114811+0000","last_became_active":"2024-09-10T21:17:40.100262+0000","last_became_peered":"2024-09-10T21:17:40.100262+0000","last_unstale":"2024-09-10T21:17:42.114811+0000","last_undegraded":"2024-09-10T21:17:42.114811+0000","last_fullsized":"2024-09-10T21:17:42.114811+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:18.680567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.113610+0000","last_change":"2024-09-10T21:17:41.101473+0000","last_active":"2024-09-10T21:17:42.113610+0000","last_peered":"2024-09-10T21:17:42.113610+0000","last_clean":"2024-09-10T21:17:42.113610+0000","last_became_active":"2024-09-10T21:17:41.101158+0000","last_became_peered":"2024-09-10T21:17:41.101158+0000","last_unstale":"2024-09-10T21:17:42.113610+0000","last_undegraded":"2024-09-10T21:17:42.113610+0000","last_fullsized":"2024-09-10T21:17:42.113610+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:23.496731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.112796+0000","last_change":"2024-09-10T21:17:40.097844+0000","last_active":"2024-09-10T21:17:42.112796+0000","last_peered":"2024-09-10T21:17:42.112796+0000","last_clean":"2024-09-10T21:17:42.112796+0000","last_became_active":"2024-09-10T21:17:40.097445+0000","last_became_peered":"2024-09-10T21:17:40.097445+0000","last_unstale":"2024-09-10T21:17:42.112796+0000","last_undegraded":"2024-09-10T21:17:42.112796+0000","last_fullsized":"2024-09-10T21:17:42.112796+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:23.378729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807849+0000","last_change":"2024-09-10T21:17:41.100102+0000","last_active":"2024-09-10T21:17:42.807849+0000","last_peered":"2024-09-10T21:17:42.807849+0000","last_clean":"2024-09-10T21:17:42.807849+0000","last_became_active":"2024-09-10T21:17:41.098898+0000","last_became_peered":"2024-09-10T21:17:41.098898+0000","last_unstale":"2024-09-10T21:17:42.807849+0000","last_undegraded":"2024-09-10T21:17:42.807849+0000","last_fullsized":"2024-09-10T21:17:42.807849+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:07.271443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808416+0000","last_change":"2024-09-10T21:17:40.107437+0000","last_active":"2024-09-10T21:17:42.808416+0000","last_peered":"2024-09-10T21:17:42.808416+0000","last_clean":"2024-09-10T21:17:42.808416+0000","last_became_active":"2024-09-10T21:17:40.107183+0000","last_became_peered":"2024-09-10T21:17:40.107183+0000","last_unstale":"2024-09-10T21:17:42.808416+0000","last_undegraded":"2024-09-10T21:17:42.808416+0000","last_fullsized":"2024-09-10T21:17:42.808416+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:35.892390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807224+0000","last_change":"2024-09-10T21:17:41.101116+0000","last_active":"2024-09-10T21:17:42.807224+0000","last_peered":"2024-09-10T21:17:42.807224+0000","last_clean":"2024-09-10T21:17:42.807224+0000","last_became_active":"2024-09-10T21:17:41.100847+0000","last_became_peered":"2024-09-10T21:17:41.100847+0000","last_unstale":"2024-09-10T21:17:42.807224+0000","last_undegraded":"2024-09-10T21:17:42.807224+0000","last_fullsized":"2024-09-10T21:17:42.807224+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:51.850309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.113042+0000","last_change":"2024-09-10T21:17:40.109909+0000","last_active":"2024-09-10T21:17:42.113042+0000","last_peered":"2024-09-10T21:17:42.113042+0000","last_clean":"2024-09-10T21:17:42.113042+0000","last_became_active":"2024-09-10T21:17:40.109662+0000","last_became_peered":"2024-09-10T21:17:40.109662+0000","last_unstale":"2024-09-10T21:17:42.113042+0000","last_undegraded":"2024-09-10T21:17:42.113042+0000","last_fullsized":"2024-09-10T21:17:42.113042+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:14.210573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807680+0000","last_change":"2024-09-10T21:17:41.092532+0000","last_active":"2024-09-10T21:17:42.807680+0000","last_peered":"2024-09-10T21:17:42.807680+0000","last_clean":"2024-09-10T21:17:42.807680+0000","last_became_active":"2024-09-10T21:17:41.092013+0000","last_became_peered":"2024-09-10T21:17:41.092013+0000","last_unstale":"2024-09-10T21:17:42.807680+0000","last_undegraded":"2024-09-10T21:17:42.807680+0000","last_fullsized":"2024-09-10T21:17:42.807680+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:08.028201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808377+0000","last_change":"2024-09-10T21:17:40.108145+0000","last_active":"2024-09-10T21:17:42.808377+0000","last_peered":"2024-09-10T21:17:42.808377+0000","last_clean":"2024-09-10T21:17:42.808377+0000","last_became_active":"2024-09-10T21:17:40.106654+0000","last_became_peered":"2024-09-10T21:17:40.106654+0000","last_unstale":"2024-09-10T21:17:42.808377+0000","last_undegraded":"2024-09-10T21:17:42.808377+0000","last_fullsized":"2024-09-10T21:17:42.808377+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:26.592528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808191+0000","last_change":"2024-09-10T21:17:41.110275+0000","last_active":"2024-09-10T21:17:42.808191+0000","last_peered":"2024-09-10T21:17:42.808191+0000","last_clean":"2024-09-10T21:17:42.808191+0000","last_became_active":"2024-09-10T21:17:41.109945+0000","last_became_peered":"2024-09-10T21:17:41.109945+0000","last_unstale":"2024-09-10T21:17:42.808191+0000","last_undegraded":"2024-09-10T21:17:42.808191+0000","last_fullsized":"2024-09-10T21:17:42.808191+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:10.883358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805890+0000","last_change":"2024-09-10T21:17:40.107336+0000","last_active":"2024-09-10T21:17:42.805890+0000","last_peered":"2024-09-10T21:17:42.805890+0000","last_clean":"2024-09-10T21:17:42.805890+0000","last_became_active":"2024-09-10T21:17:40.106303+0000","last_became_peered":"2024-09-10T21:17:40.106303+0000","last_unstale":"2024-09-10T21:17:42.805890+0000","last_undegraded":"2024-09-10T21:17:42.805890+0000","last_fullsized":"2024-09-10T21:17:42.805890+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:55.297764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807113+0000","last_change":"2024-09-10T21:17:41.110597+0000","last_active":"2024-09-10T21:17:42.807113+0000","last_peered":"2024-09-10T21:17:42.807113+0000","last_clean":"2024-09-10T21:17:42.807113+0000","last_became_active":"2024-09-10T21:17:41.109142+0000","last_became_peered":"2024-09-10T21:17:41.109142+0000","last_unstale":"2024-09-10T21:17:42.807113+0000","last_undegraded":"2024-09-10T21:17:42.807113+0000","last_fullsized":"2024-09-10T21:17:42.807113+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:50.078282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806128+0000","last_change":"2024-09-10T21:17:40.101272+0000","last_active":"2024-09-10T21:17:42.806128+0000","last_peered":"2024-09-10T21:17:42.806128+0000","last_clean":"2024-09-10T21:17:42.806128+0000","last_became_active":"2024-09-10T21:17:40.100936+0000","last_became_peered":"2024-09-10T21:17:40.100936+0000","last_unstale":"2024-09-10T21:17:42.806128+0000","last_undegraded":"2024-09-10T21:17:42.806128+0000","last_fullsized":"2024-09-10T21:17:42.806128+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:54.011412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114083+0000","last_change":"2024-09-10T21:17:41.123266+0000","last_active":"2024-09-10T21:17:42.114083+0000","last_peered":"2024-09-10T21:17:42.114083+0000","last_clean":"2024-09-10T21:17:42.114083+0000","last_became_active":"2024-09-10T21:17:41.110964+0000","last_became_peered":"2024-09-10T21:17:41.110964+0000","last_unstale":"2024-09-10T21:17:42.114083+0000","last_undegraded":"2024-09-10T21:17:42.114083+0000","last_fullsized":"2024-09-10T21:17:42.114083+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:11.844660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.112247+0000","last_change":"2024-09-10T21:17:40.100323+0000","last_active":"2024-09-10T21:17:42.112247+0000","last_peered":"2024-09-10T21:17:42.112247+0000","last_clean":"2024-09-10T21:17:42.112247+0000","last_became_active":"2024-09-10T21:17:40.099959+0000","last_became_peered":"2024-09-10T21:17:40.099959+0000","last_unstale":"2024-09-10T21:17:42.112247+0000","last_undegraded":"2024-09-10T21:17:42.112247+0000","last_fullsized":"2024-09-10T21:17:42.112247+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:08.147218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808380+0000","last_change":"2024-09-10T21:17:41.130115+0000","last_active":"2024-09-10T21:17:42.808380+0000","last_peered":"2024-09-10T21:17:42.808380+0000","last_clean":"2024-09-10T21:17:42.808380+0000","last_became_active":"2024-09-10T21:17:41.129815+0000","last_became_peered":"2024-09-10T21:17:41.129815+0000","last_unstale":"2024-09-10T21:17:42.808380+0000","last_undegraded":"2024-09-10T21:17:42.808380+0000","last_fullsized":"2024-09-10T21:17:42.808380+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:30.750812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806219+0000","last_change":"2024-09-10T21:17:40.109507+0000","last_active":"2024-09-10T21:17:42.806219+0000","last_peered":"2024-09-10T21:17:42.806219+0000","last_clean":"2024-09-10T21:17:42.806219+0000","last_became_active":"2024-09-10T21:17:40.108093+0000","last_became_peered":"2024-09-10T21:17:40.108093+0000","last_unstale":"2024-09-10T21:17:42.806219+0000","last_undegraded":"2024-09-10T21:17:42.806219+0000","last_fullsized":"2024-09-10T21:17:42.806219+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:34.048006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807510+0000","last_change":"2024-09-10T21:17:41.102495+0000","last_active":"2024-09-10T21:17:42.807510+0000","last_peered":"2024-09-10T21:17:42.807510+0000","last_clean":"2024-09-10T21:17:42.807510+0000","last_became_active":"2024-09-10T21:17:41.099292+0000","last_became_peered":"2024-09-10T21:17:41.099292+0000","last_unstale":"2024-09-10T21:17:42.807510+0000","last_undegraded":"2024-09-10T21:17:42.807510+0000","last_fullsized":"2024-09-10T21:17:42.807510+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:22.687793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"116'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:43.095178+0000","last_change":"2024-09-10T21:17:40.100584+0000","last_active":"2024-09-10T21:17:43.095178+0000","last_peered":"2024-09-10T21:17:43.095178+0000","last_clean":"2024-09-10T21:17:43.095178+0000","last_became_active":"2024-09-10T21:17:40.100103+0000","last_became_peered":"2024-09-10T21:17:40.100103+0000","last_unstale":"2024-09-10T21:17:43.095178+0000","last_undegraded":"2024-09-10T21:17:43.095178+0000","last_fullsized":"2024-09-10T21:17:43.095178+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:00.079074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806673+0000","last_change":"2024-09-10T21:17:41.115334+0000","last_active":"2024-09-10T21:17:42.806673+0000","last_peered":"2024-09-10T21:17:42.806673+0000","last_clean":"2024-09-10T21:17:42.806673+0000","last_became_active":"2024-09-10T21:17:41.114658+0000","last_became_peered":"2024-09-10T21:17:41.114658+0000","last_unstale":"2024-09-10T21:17:42.806673+0000","last_undegraded":"2024-09-10T21:17:42.806673+0000","last_fullsized":"2024-09-10T21:17:42.806673+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:40.108533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805866+0000","last_change":"2024-09-10T21:17:40.104038+0000","last_active":"2024-09-10T21:17:42.805866+0000","last_peered":"2024-09-10T21:17:42.805866+0000","last_clean":"2024-09-10T21:17:42.805866+0000","last_became_active":"2024-09-10T21:17:40.103746+0000","last_became_peered":"2024-09-10T21:17:40.103746+0000","last_unstale":"2024-09-10T21:17:42.805866+0000","last_undegraded":"2024-09-10T21:17:42.805866+0000","last_fullsized":"2024-09-10T21:17:42.805866+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:30.390294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.808271+0000","last_change":"2024-09-10T21:17:41.108650+0000","last_active":"2024-09-10T21:17:42.808271+0000","last_peered":"2024-09-10T21:17:42.808271+0000","last_clean":"2024-09-10T21:17:42.808271+0000","last_became_active":"2024-09-10T21:17:41.108316+0000","last_became_peered":"2024-09-10T21:17:41.108316+0000","last_unstale":"2024-09-10T21:17:42.808271+0000","last_undegraded":"2024-09-10T21:17:42.808271+0000","last_fullsized":"2024-09-10T21:17:42.808271+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:14.793315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.806810+0000","last_change":"2024-09-10T21:17:40.107823+0000","last_active":"2024-09-10T21:17:42.806810+0000","last_peered":"2024-09-10T21:17:42.806810+0000","last_clean":"2024-09-10T21:17:42.806810+0000","last_became_active":"2024-09-10T21:17:40.105937+0000","last_became_peered":"2024-09-10T21:17:40.105937+0000","last_unstale":"2024-09-10T21:17:42.806810+0000","last_undegraded":"2024-09-10T21:17:42.806810+0000","last_fullsized":"2024-09-10T21:17:42.806810+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:21.562017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114252+0000","last_change":"2024-09-10T21:17:41.127551+0000","last_active":"2024-09-10T21:17:42.114252+0000","last_peered":"2024-09-10T21:17:42.114252+0000","last_clean":"2024-09-10T21:17:42.114252+0000","last_became_active":"2024-09-10T21:17:41.126805+0000","last_became_peered":"2024-09-10T21:17:41.126805+0000","last_unstale":"2024-09-10T21:17:42.114252+0000","last_undegraded":"2024-09-10T21:17:42.114252+0000","last_fullsized":"2024-09-10T21:17:42.114252+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:40.263578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807022+0000","last_change":"2024-09-10T21:17:40.104484+0000","last_active":"2024-09-10T21:17:42.807022+0000","last_peered":"2024-09-10T21:17:42.807022+0000","last_clean":"2024-09-10T21:17:42.807022+0000","last_became_active":"2024-09-10T21:17:40.104217+0000","last_became_peered":"2024-09-10T21:17:40.104217+0000","last_unstale":"2024-09-10T21:17:42.807022+0000","last_undegraded":"2024-09-10T21:17:42.807022+0000","last_fullsized":"2024-09-10T21:17:42.807022+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:34.115641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114606+0000","last_change":"2024-09-10T21:17:41.113281+0000","last_active":"2024-09-10T21:17:42.114606+0000","last_peered":"2024-09-10T21:17:42.114606+0000","last_clean":"2024-09-10T21:17:42.114606+0000","last_became_active":"2024-09-10T21:17:41.110900+0000","last_became_peered":"2024-09-10T21:17:41.110900+0000","last_unstale":"2024-09-10T21:17:42.114606+0000","last_undegraded":"2024-09-10T21:17:42.114606+0000","last_fullsized":"2024-09-10T21:17:42.114606+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:30.011635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114219+0000","last_change":"2024-09-10T21:17:40.105517+0000","last_active":"2024-09-10T21:17:42.114219+0000","last_peered":"2024-09-10T21:17:42.114219+0000","last_clean":"2024-09-10T21:17:42.114219+0000","last_became_active":"2024-09-10T21:17:40.103750+0000","last_became_peered":"2024-09-10T21:17:40.103750+0000","last_unstale":"2024-09-10T21:17:42.114219+0000","last_undegraded":"2024-09-10T21:17:42.114219+0000","last_fullsized":"2024-09-10T21:17:42.114219+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:38.758700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805536+0000","last_change":"2024-09-10T21:17:41.128674+0000","last_active":"2024-09-10T21:17:42.805536+0000","last_peered":"2024-09-10T21:17:42.805536+0000","last_clean":"2024-09-10T21:17:42.805536+0000","last_became_active":"2024-09-10T21:17:41.128331+0000","last_became_peered":"2024-09-10T21:17:41.128331+0000","last_unstale":"2024-09-10T21:17:42.805536+0000","last_undegraded":"2024-09-10T21:17:42.805536+0000","last_fullsized":"2024-09-10T21:17:42.805536+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:36.702717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.112270+0000","last_change":"2024-09-10T21:17:40.100190+0000","last_active":"2024-09-10T21:17:42.112270+0000","last_peered":"2024-09-10T21:17:42.112270+0000","last_clean":"2024-09-10T21:17:42.112270+0000","last_became_active":"2024-09-10T21:17:40.098672+0000","last_became_peered":"2024-09-10T21:17:40.098672+0000","last_unstale":"2024-09-10T21:17:42.112270+0000","last_undegraded":"2024-09-10T21:17:42.112270+0000","last_fullsized":"2024-09-10T21:17:42.112270+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:07.032981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805453+0000","last_change":"2024-09-10T21:17:41.113326+0000","last_active":"2024-09-10T21:17:42.805453+0000","last_peered":"2024-09-10T21:17:42.805453+0000","last_clean":"2024-09-10T21:17:42.805453+0000","last_became_active":"2024-09-10T21:17:41.113033+0000","last_became_peered":"2024-09-10T21:17:41.113033+0000","last_unstale":"2024-09-10T21:17:42.805453+0000","last_undegraded":"2024-09-10T21:17:42.805453+0000","last_fullsized":"2024-09-10T21:17:42.805453+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:16.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.805901+0000","last_change":"2024-09-10T21:17:40.106840+0000","last_active":"2024-09-10T21:17:42.805901+0000","last_peered":"2024-09-10T21:17:42.805901+0000","last_clean":"2024-09-10T21:17:42.805901+0000","last_became_active":"2024-09-10T21:17:40.104591+0000","last_became_peered":"2024-09-10T21:17:40.104591+0000","last_unstale":"2024-09-10T21:17:42.805901+0000","last_undegraded":"2024-09-10T21:17:42.805901+0000","last_fullsized":"2024-09-10T21:17:42.805901+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:13.889301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.809001+0000","last_change":"2024-09-10T21:17:41.106390+0000","last_active":"2024-09-10T21:17:42.809001+0000","last_peered":"2024-09-10T21:17:42.809001+0000","last_clean":"2024-09-10T21:17:42.809001+0000","last_became_active":"2024-09-10T21:17:41.105932+0000","last_became_peered":"2024-09-10T21:17:41.105932+0000","last_unstale":"2024-09-10T21:17:42.809001+0000","last_undegraded":"2024-09-10T21:17:42.809001+0000","last_fullsized":"2024-09-10T21:17:42.809001+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:40.075516+0000","last_clean_scrub_stamp":"2024-09-10T21:17:40.075516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:19.287850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.114649+0000","last_change":"2024-09-10T21:17:40.102749+0000","last_active":"2024-09-10T21:17:42.114649+0000","last_peered":"2024-09-10T21:17:42.114649+0000","last_clean":"2024-09-10T21:17:42.114649+0000","last_became_active":"2024-09-10T21:17:40.102471+0000","last_became_peered":"2024-09-10T21:17:40.102471+0000","last_unstale":"2024-09-10T21:17:42.114649+0000","last_undegraded":"2024-09-10T21:17:42.114649+0000","last_fullsized":"2024-09-10T21:17:42.114649+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:17:39.072506+0000","last_clean_scrub_stamp":"2024-09-10T21:17:39.072506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:56.257177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":278,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T21:17:42.807515+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:17:42.807515+0000","last_peered":"2024-09-10T21:17:42.807515+0000","last_clean":"2024-09-10T21:17:42.807515+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:17:42.807515+0000","last_undegraded":"2024-09-10T21:17:42.807515+0000","last_fullsized":"2024-09-10T21:17:42.807515+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509532,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36756,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335084,"statfs":{"total":96636764160,"available":96599126016,"internally_reserved":0,"allocated":1441792,"data_stored":619965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509534,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2048000,"data_stored":1211075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509532,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36756,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335084,"statfs":{"total":96636764160,"available":96599126016,"internally_reserved":0,"allocated":1441792,"data_stored":619965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1445888,"data_stored":620035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509533,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":620637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509532,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":621633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542236,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40912,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330928,"statfs":{"total":96636764160,"available":96594870272,"internally_reserved":0,"allocated":1433600,"data_stored":613699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542235,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37352,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334488,"statfs":{"total":96636764160,"available":96598515712,"internally_reserved":0,"allocated":2052096,"data_stored":1211967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:17:45.664 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:17:45.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:17:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.857+0000 7f4ac2600640 1 -- 172.21.15.16:0/1268633986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4aa00041d0 msgr2=0x7f4aa0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.857+0000 7f4ac2600640 1 --2- 172.21.15.16:0/1268633986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4aa00041d0 0x7f4aa0004810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009b40 tx=0x7f4aa401d250 comp rx=0 tx=0).stop 2024-09-10T21:17:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.857+0000 7f4abbe00640 1 -- 172.21.15.16:0/1268633986 reap_dead start 2024-09-10T21:17:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.857+0000 7f4ac2600640 1 -- 172.21.15.16:0/1268633986 shutdown_connections 2024-09-10T21:17:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.857+0000 7f4ac2600640 1 -- 172.21.15.16:0/1268633986 >> 172.21.15.16:0/1268633986 conn(0x7f4abc0d1df0 msgr2=0x7f4abc0d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.857+0000 7f4ac2600640 1 -- 172.21.15.16:0/1268633986 shutdown_connections 2024-09-10T21:17:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.857+0000 7f4ac2600640 1 -- 172.21.15.16:0/1268633986 wait complete. 2024-09-10T21:17:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.857+0000 7f4ac2600640 1 Processor -- start 2024-09-10T21:17:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.857+0000 7f4ac2600640 1 -- start start 2024-09-10T21:17:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4ac2600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa00041d0 0x7f4abc0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4ac2600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4abc0c8ec0 0x7f4abc1f8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4ac2600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4abc0c9340 0x7f4abc1f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4ac2600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc0ceca0 con 0x7f4aa00041d0 2024-09-10T21:17:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4ac2600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc0b14f0 con 0x7f4abc0c9340 2024-09-10T21:17:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4ac2600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4abc0cee20 con 0x7f4abc0c8ec0 2024-09-10T21:17:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4abb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa00041d0 0x7f4abc0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4abb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa00041d0 0x7f4abc0cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43634/0 (socket says 172.21.15.16:43634) 2024-09-10T21:17:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4abb400640 1 -- 172.21.15.16:0/441237764 learned_addr learned my addr 172.21.15.16:0/441237764 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:45.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4ab3e00640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4abc0c8ec0 0x7f4abc1f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:45.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4abbe00640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4abc0c9340 0x7f4abc1f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:45.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4abb400640 1 -- 172.21.15.16:0/441237764 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4abc0c8ec0 msgr2=0x7f4abc1f8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:45.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4abb400640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4abc0c8ec0 0x7f4abc1f8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4abb400640 1 -- 172.21.15.16:0/441237764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4abc0c9340 msgr2=0x7f4abc1f8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:45.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4abb400640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4abc0c9340 0x7f4abc1f8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:45.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4abb400640 1 -- 172.21.15.16:0/441237764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4abc0b77a0 con 0x7f4aa00041d0 2024-09-10T21:17:45.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4abbe00640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4abc0c9340 0x7f4abc1f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:45.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4abb400640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa00041d0 0x7f4abc0cb6e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4aac003f50 tx=0x7f4aac0018d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4ab3400640 1 -- 172.21.15.16:0/441237764 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac011700 con 0x7f4aa00041d0 2024-09-10T21:17:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4ab3400640 1 -- 172.21.15.16:0/441237764 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4aac016020 con 0x7f4aa00041d0 2024-09-10T21:17:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4ab3400640 1 -- 172.21.15.16:0/441237764 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac011b20 con 0x7f4aa00041d0 2024-09-10T21:17:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4ac2600640 1 -- 172.21.15.16:0/441237764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4abc1f5240 con 0x7f4aa00041d0 2024-09-10T21:17:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4ac2600640 1 -- 172.21.15.16:0/441237764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4abc1f5800 con 0x7f4aa00041d0 2024-09-10T21:17:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.861+0000 7f4ac2600640 1 -- 172.21.15.16:0/441237764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4abc069bd0 con 0x7f4aa00041d0 2024-09-10T21:17:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.865+0000 7f4ab3400640 1 -- 172.21.15.16:0/441237764 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4aac011cc0 con 0x7f4aa00041d0 2024-09-10T21:17:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.865+0000 7f4ab3400640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a980725e0 0x7f4a98074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.865+0000 7f4ab3400640 1 -- 172.21.15.16:0/441237764 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f4aac093c60 con 0x7f4aa00041d0 2024-09-10T21:17:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.865+0000 7f4ab3e00640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a980725e0 0x7f4a98074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.865+0000 7f4ab3e00640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a980725e0 0x7f4a98074aa0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40ad000 tx=0x7f4aa4019ff0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:45.869+0000 7f4ab3400640 1 -- 172.21.15.16:0/441237764 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4aac020020 con 0x7f4aa00041d0 2024-09-10T21:17:46.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.105+0000 7f4ac2600640 1 -- 172.21.15.16:0/441237764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4abc0cf380 con 0x7f4aa00041d0 2024-09-10T21:17:46.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.105+0000 7f4ab3400640 1 -- 172.21.15.16:0/441237764 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v29) ==== 190+0+147 (secure 0 0 0) 0x7f4abc0b77a0 con 0x7f4aa00041d0 2024-09-10T21:17:46.111 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-10T21:17:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA== 2024-09-10T21:17:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-10T21:17:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-10T21:17:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:17:46.111 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-10T21:17:46.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.109+0000 7f4ac2600640 1 -- 172.21.15.16:0/441237764 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a980725e0 msgr2=0x7f4a98074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.109+0000 7f4ac2600640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a980725e0 0x7f4a98074aa0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40ad000 tx=0x7f4aa4019ff0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.109+0000 7f4ac2600640 1 -- 172.21.15.16:0/441237764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa00041d0 msgr2=0x7f4abc0cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.109+0000 7f4ac2600640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa00041d0 0x7f4abc0cb6e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4aac003f50 tx=0x7f4aac0018d0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.109+0000 7f4ac2600640 1 -- 172.21.15.16:0/441237764 shutdown_connections 2024-09-10T21:17:46.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.109+0000 7f4ac2600640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4a980725e0 0x7f4a98074aa0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.109+0000 7f4ac2600640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4abc0c9340 0x7f4abc1f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.109+0000 7f4ac2600640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4abc0c8ec0 0x7f4abc1f8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.109+0000 7f4ac2600640 1 --2- 172.21.15.16:0/441237764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa00041d0 0x7f4abc0cb6e0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.109+0000 7f4ac2600640 1 -- 172.21.15.16:0/441237764 >> 172.21.15.16:0/441237764 conn(0x7f4abc0d1df0 msgr2=0x7f4abc0af220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:46.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.109+0000 7f4ac2600640 1 -- 172.21.15.16:0/441237764 shutdown_connections 2024-09-10T21:17:46.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.109+0000 7f4ac2600640 1 -- 172.21.15.16:0/441237764 wait complete. 2024-09-10T21:17:46.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.277+0000 7f4595600640 1 -- 172.21.15.16:0/2993747336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459015f400 msgr2=0x7f4590070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.277+0000 7f4595600640 1 --2- 172.21.15.16:0/2993747336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459015f400 0x7f4590070fe0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f45900698f0 tx=0x7f458001c780 comp rx=0 tx=0).stop 2024-09-10T21:17:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.277+0000 7f4595600640 1 -- 172.21.15.16:0/2993747336 shutdown_connections 2024-09-10T21:17:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.277+0000 7f4595600640 1 --2- 172.21.15.16:0/2993747336 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45900cfc80 0x7f45900d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.277+0000 7f4595600640 1 --2- 172.21.15.16:0/2993747336 >> v2:172.21.15.52:3300/0 conn(0x7f45900b13c0 0x7f4590071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.277+0000 7f4595600640 1 --2- 172.21.15.16:0/2993747336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459015f400 0x7f4590070fe0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.277+0000 7f4595600640 1 -- 172.21.15.16:0/2993747336 >> 172.21.15.16:0/2993747336 conn(0x7f459009b130 msgr2=0x7f45900d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.277+0000 7f4595600640 1 -- 172.21.15.16:0/2993747336 shutdown_connections 2024-09-10T21:17:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.277+0000 7f4595600640 1 -- 172.21.15.16:0/2993747336 wait complete. 2024-09-10T21:17:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4595600640 1 Processor -- start 2024-09-10T21:17:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4595600640 1 -- start start 2024-09-10T21:17:46.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4595600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45900b13c0 0x7f4590163f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4595600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45900cfc80 0x7f45900b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4595600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459015f400 0x7f45900b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4595600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45900afd90 con 0x7f45900b13c0 2024-09-10T21:17:46.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4595600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45900d1260 con 0x7f459015f400 2024-09-10T21:17:46.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4595600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f45900aff10 con 0x7f45900cfc80 2024-09-10T21:17:46.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f458ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45900b13c0 0x7f4590163f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f458ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45900b13c0 0x7f4590163f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43652/0 (socket says 172.21.15.16:43652) 2024-09-10T21:17:46.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f458ea00640 1 -- 172.21.15.16:0/2409447516 learned_addr learned my addr 172.21.15.16:0/2409447516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:46.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f458f400640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459015f400 0x7f45900b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4587e00640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45900cfc80 0x7f45900b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f458ea00640 1 -- 172.21.15.16:0/2409447516 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45900cfc80 msgr2=0x7f45900b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f458ea00640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45900cfc80 0x7f45900b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f458ea00640 1 -- 172.21.15.16:0/2409447516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459015f400 msgr2=0x7f45900b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f458ea00640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459015f400 0x7f45900b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f458ea00640 1 -- 172.21.15.16:0/2409447516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4590164460 con 0x7f45900b13c0 2024-09-10T21:17:46.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f458ea00640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45900b13c0 0x7f4590163f20 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f4580000c10 tx=0x7f45800a6660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:46.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4586a00640 1 -- 172.21.15.16:0/2409447516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580004300 con 0x7f45900b13c0 2024-09-10T21:17:46.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4586a00640 1 -- 172.21.15.16:0/2409447516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45800bd070 con 0x7f45900b13c0 2024-09-10T21:17:46.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4586a00640 1 -- 172.21.15.16:0/2409447516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580005290 con 0x7f45900b13c0 2024-09-10T21:17:46.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4595600640 1 -- 172.21.15.16:0/2409447516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45900b6af0 con 0x7f45900b13c0 2024-09-10T21:17:46.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4595600640 1 -- 172.21.15.16:0/2409447516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4590158580 con 0x7f45900b13c0 2024-09-10T21:17:46.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.281+0000 7f4595600640 1 -- 172.21.15.16:0/2409447516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45900d0110 con 0x7f45900b13c0 2024-09-10T21:17:46.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.285+0000 7f4586a00640 1 -- 172.21.15.16:0/2409447516 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4580005430 con 0x7f45900b13c0 2024-09-10T21:17:46.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.285+0000 7f4586a00640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f456c0725e0 0x7f456c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.285+0000 7f4586a00640 1 -- 172.21.15.16:0/2409447516 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f458012e280 con 0x7f45900b13c0 2024-09-10T21:17:46.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.285+0000 7f4587e00640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f456c0725e0 0x7f456c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.285+0000 7f4587e00640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f456c0725e0 0x7f456c074aa0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f459006de10 tx=0x7f457800b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:46.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.289+0000 7f4586a00640 1 -- 172.21.15.16:0/2409447516 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f45900b6af0 con 0x7f45900b13c0 2024-09-10T21:17:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.501+0000 7f4595600640 1 -- 172.21.15.16:0/2409447516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45901f8310 con 0x7f45900b13c0 2024-09-10T21:17:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.501+0000 7f4586a00640 1 -- 172.21.15.16:0/2409447516 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5229 (secure 0 0 0) 0x7f4580016490 con 0x7f45900b13c0 2024-09-10T21:17:46.507 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:46.507 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"btime":"2024-09-10T21:17:44:118059+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3961362430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3961362430},{"type":"v1","addr":"172.21.15.52:6834","nonce":3961362430}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6967,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/831111367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":831111367},{"type":"v1","addr":"172.21.15.52:6835","nonce":831111367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7467,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/552929965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":552929965},{"type":"v1","addr":"172.21.15.16:6836","nonce":552929965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:40.510614+0000","modified":"2024-09-10T21:17:43.521144+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":136181},"failed":[],"damaged":[],"stopped":[],"info":{"gid_136181":{"gid":136181,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/1681945537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1681945537},{"type":"v1","addr":"172.21.15.16:6837","nonce":1681945537}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":136181,"qdb_cluster":[136181]},"id":9}]} 2024-09-10T21:17:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 82 2024-09-10T21:17:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.505+0000 7f4595600640 1 -- 172.21.15.16:0/2409447516 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f456c0725e0 msgr2=0x7f456c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.505+0000 7f4595600640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f456c0725e0 0x7f456c074aa0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f459006de10 tx=0x7f457800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.505+0000 7f4595600640 1 -- 172.21.15.16:0/2409447516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45900b13c0 msgr2=0x7f4590163f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.505+0000 7f4595600640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45900b13c0 0x7f4590163f20 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f4580000c10 tx=0x7f45800a6660 comp rx=0 tx=0).stop 2024-09-10T21:17:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.505+0000 7f4595600640 1 -- 172.21.15.16:0/2409447516 shutdown_connections 2024-09-10T21:17:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.505+0000 7f4595600640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f456c0725e0 0x7f456c074aa0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.505+0000 7f4595600640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459015f400 0x7f45900b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.505+0000 7f4595600640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45900cfc80 0x7f45900b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.505+0000 7f4595600640 1 --2- 172.21.15.16:0/2409447516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45900b13c0 0x7f4590163f20 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.505+0000 7f4595600640 1 -- 172.21.15.16:0/2409447516 >> 172.21.15.16:0/2409447516 conn(0x7f459009b130 msgr2=0x7f45900b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.505+0000 7f4595600640 1 -- 172.21.15.16:0/2409447516 shutdown_connections 2024-09-10T21:17:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:46.505+0000 7f4595600640 1 -- 172.21.15.16:0/2409447516 wait complete. 2024-09-10T21:17:46.931 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:17:40.510614+0000', 'modified': '2024-09-10T21:17:43.521144+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 136181}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_136181': {'gid': 136181, 'name': 'c', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6837/1681945537', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6835', 'nonce': 1681945537}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 1681945537}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 136181, 'qdb_cluster': [136181]} 2024-09-10T21:17:46.932 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:17:46.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:17:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.081+0000 7f8a29e00640 1 -- 172.21.15.16:0/1206370051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a08003cc0 msgr2=0x7f8a080040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.081+0000 7f8a29e00640 1 --2- 172.21.15.16:0/1206370051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a08003cc0 0x7f8a080040c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f8a14001800 tx=0x7f8a1401c8d0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.081+0000 7f8a23400640 1 -- 172.21.15.16:0/1206370051 reap_dead start 2024-09-10T21:17:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.081+0000 7f8a29e00640 1 -- 172.21.15.16:0/1206370051 shutdown_connections 2024-09-10T21:17:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.081+0000 7f8a29e00640 1 -- 172.21.15.16:0/1206370051 >> 172.21.15.16:0/1206370051 conn(0x7f8a240d0c30 msgr2=0x7f8a240d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.081+0000 7f8a29e00640 1 -- 172.21.15.16:0/1206370051 shutdown_connections 2024-09-10T21:17:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.081+0000 7f8a29e00640 1 -- 172.21.15.16:0/1206370051 wait complete. 2024-09-10T21:17:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.081+0000 7f8a29e00640 1 Processor -- start 2024-09-10T21:17:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.081+0000 7f8a29e00640 1 -- start start 2024-09-10T21:17:47.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a29e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a08003cc0 0x7f8a2415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a29e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a24158aa0 0x7f8a24157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a29e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a24154a10 0x7f8a241f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a29e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24168290 con 0x7f8a24158aa0 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a29e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24168110 con 0x7f8a08003cc0 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a29e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8a24168410 con 0x7f8a24154a10 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a24158aa0 0x7f8a24157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a24158aa0 0x7f8a24157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43680/0 (socket says 172.21.15.16:43680) 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a1be00640 1 -- 172.21.15.16:0/1681600844 learned_addr learned my addr 172.21.15.16:0/1681600844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a22a00640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a08003cc0 0x7f8a2415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a23400640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a24154a10 0x7f8a241f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a1be00640 1 -- 172.21.15.16:0/1681600844 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a24154a10 msgr2=0x7f8a241f95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a1be00640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a24154a10 0x7f8a241f95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a1be00640 1 -- 172.21.15.16:0/1681600844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a08003cc0 msgr2=0x7f8a2415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a1be00640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a08003cc0 0x7f8a2415a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a1be00640 1 -- 172.21.15.16:0/1681600844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a240b4d50 con 0x7f8a24158aa0 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a1be00640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a24158aa0 0x7f8a24157010 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0117a0 tx=0x7f8a0c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a1b400640 1 -- 172.21.15.16:0/1681600844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c004460 con 0x7f8a24158aa0 2024-09-10T21:17:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a1b400640 1 -- 172.21.15.16:0/1681600844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a0c016070 con 0x7f8a24158aa0 2024-09-10T21:17:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a1b400640 1 -- 172.21.15.16:0/1681600844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c00fba0 con 0x7f8a24158aa0 2024-09-10T21:17:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a29e00640 1 -- 172.21.15.16:0/1681600844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a241fa3b0 con 0x7f8a24158aa0 2024-09-10T21:17:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a29e00640 1 -- 172.21.15.16:0/1681600844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a240b6360 con 0x7f8a24158aa0 2024-09-10T21:17:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.085+0000 7f8a1b400640 1 -- 172.21.15.16:0/1681600844 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8a0c003600 con 0x7f8a24158aa0 2024-09-10T21:17:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a1b400640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8a00072610 0x7f8a00074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a1b400640 1 -- 172.21.15.16:0/1681600844 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f8a0c093ba0 con 0x7f8a24158aa0 2024-09-10T21:17:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a2d273000 1 Processor -- start 2024-09-10T21:17:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a22a00640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8a00072610 0x7f8a00074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a2d273000 1 -- start start 2024-09-10T21:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a22a00640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8a00072610 0x7f8a00074ad0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8a140017d0 tx=0x7f8a140196c0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a2d273000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771e9d0 0x55e82771edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a2d273000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771f2f0 0x55e8276d28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a2d273000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e8276d2e10 0x55e8276d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a2d273000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8275f33f0 con 0x55e82771f2f0 2024-09-10T21:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a2d273000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e827530b20 con 0x55e82771e9d0 2024-09-10T21:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a2d273000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55e82751ff50 con 0x55e8276d2e10 2024-09-10T21:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771f2f0 0x55e8276d28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771f2f0 0x55e8276d28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43684/0 (socket says 172.21.15.16:43684) 2024-09-10T21:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a1be00640 1 -- 172.21.15.16:0/1999763914 learned_addr learned my addr 172.21.15.16:0/1999763914 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a1be00640 1 -- 172.21.15.16:0/1999763914 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e8276d2e10 msgr2=0x55e8276d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a1be00640 1 --2- 172.21.15.16:0/1999763914 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e8276d2e10 0x55e8276d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a1be00640 1 -- 172.21.15.16:0/1999763914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771e9d0 msgr2=0x55e82771edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a1be00640 1 --2- 172.21.15.16:0/1999763914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771e9d0 0x55e82771edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a1be00640 1 -- 172.21.15.16:0/1999763914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8274e7590 con 0x55e82771f2f0 2024-09-10T21:17:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a1be00640 1 --2- 172.21.15.16:0/1999763914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771f2f0 0x55e8276d28d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c05e890 tx=0x7f8a0c005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fdb34f8c19ebf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a13e00640 1 -- 172.21.15.16:0/1999763914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0635c0 con 0x55e82771f2f0 2024-09-10T21:17:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a13e00640 1 -- 172.21.15.16:0/1999763914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a0c067fe0 con 0x55e82771f2f0 2024-09-10T21:17:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.089+0000 7f8a13e00640 1 -- 172.21.15.16:0/1999763914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c063100 con 0x55e82771f2f0 2024-09-10T21:17:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 -- 172.21.15.16:0/1999763914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771f2f0 msgr2=0x55e8276d28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1999763914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771f2f0 0x55e8276d28d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c05e890 tx=0x7f8a0c005890 comp rx=0 tx=0).stop 2024-09-10T21:17:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 -- 172.21.15.16:0/1999763914 shutdown_connections 2024-09-10T21:17:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1999763914 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e8276d2e10 0x55e8276d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1999763914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771f2f0 0x55e8276d28d0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1999763914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771e9d0 0x55e82771edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 -- 172.21.15.16:0/1999763914 >> 172.21.15.16:0/1999763914 conn(0x55e8275bc130 msgr2=0x55e827701e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:47.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 -- 172.21.15.16:0/1999763914 shutdown_connections 2024-09-10T21:17:47.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 -- 172.21.15.16:0/1999763914 wait complete. 2024-09-10T21:17:47.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 -- start start 2024-09-10T21:17:47.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 Processor -- start 2024-09-10T21:17:47.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8276d2e10 0x55e8276ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e82771e9d0 0x55e8276ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771f2f0 0x55e8277003b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8274ba5d0 con 0x55e8276d2e10 2024-09-10T21:17:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e827523d60 con 0x55e82771f2f0 2024-09-10T21:17:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55e8274c6500 con 0x55e82771e9d0 2024-09-10T21:17:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 10 client.0 _ll_register_callbacks cb 0x55e8274ca810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a22a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8276d2e10 0x55e8276ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a22a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8276d2e10 0x55e8276ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43694/0 (socket says 172.21.15.16:43694) 2024-09-10T21:17:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a22a00640 1 -- 172.21.15.16:0/564215823 learned_addr learned my addr 172.21.15.16:0/564215823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a23400640 1 --2- 172.21.15.16:0/564215823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771f2f0 0x55e8277003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a1be00640 1 --2- 172.21.15.16:0/564215823 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e82771e9d0 0x55e8276ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a22a00640 1 -- 172.21.15.16:0/564215823 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e82771e9d0 msgr2=0x55e8276ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a22a00640 1 --2- 172.21.15.16:0/564215823 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e82771e9d0 0x55e8276ffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a22a00640 1 -- 172.21.15.16:0/564215823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771f2f0 msgr2=0x55e8277003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a22a00640 1 --2- 172.21.15.16:0/564215823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771f2f0 0x55e8277003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a22a00640 1 -- 172.21.15.16:0/564215823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8274be170 con 0x55e8276d2e10 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a22a00640 1 --2- 172.21.15.16:0/564215823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8276d2e10 0x55e8276ff6a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005040 tx=0x7f8a08005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3961d5a1a6d4599 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a10c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a10c00640 1 -- 172.21.15.16:0/564215823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a140a5a50 con 0x55e8276d2e10 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a10c00640 1 -- 172.21.15.16:0/564215823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f8a140b6030 con 0x55e8276d2e10 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 -- 172.21.15.16:0/564215823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e8274b80f0 con 0x55e8276d2e10 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a10c00640 1 -- 172.21.15.16:0/564215823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8a140a5bf0 con 0x55e8276d2e10 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 10 client.7599 fetch_fsmap learned FSMap version 82 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.093+0000 7f8a2d273000 1 -- 172.21.15.16:0/564215823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55e8275a4ef0 con 0x55e8276d2e10 2024-09-10T21:17:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a10c00640 1 -- 172.21.15.16:0/564215823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55e8274b80f0 con 0x55e8276d2e10 2024-09-10T21:17:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a10c00640 1 -- 172.21.15.16:0/564215823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 82) ==== 3502+0+0 (secure 0 0 0) 0x7f8a14003600 con 0x55e8276d2e10 2024-09-10T21:17:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 10 client.7599 fetch_fsmap finished waiting for FSMap version 82 2024-09-10T21:17:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 10 client.7599 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:17:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 20 client.7599 populate_metadata read hostname 'smithi016' 2024-09-10T21:17:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 1 --2- 172.21.15.16:0/564215823 >> [v2:172.21.15.16:6835/1681945537,v1:172.21.15.16:6837/1681945537] conn(0x55e8277010f0 0x55e827730c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 4 client.7599 mds_command: new command op to 136181 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 1 -- 172.21.15.16:0/564215823 --> [v2:172.21.15.16:6835/1681945537,v1:172.21.15.16:6837/1681945537] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e827532a70 con 0x55e8277010f0 2024-09-10T21:17:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a1be00640 1 --2- 172.21.15.16:0/564215823 >> [v2:172.21.15.16:6835/1681945537,v1:172.21.15.16:6837/1681945537] conn(0x55e8277010f0 0x55e827730c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a1be00640 1 --2- 172.21.15.16:0/564215823 >> [v2:172.21.15.16:6835/1681945537,v1:172.21.15.16:6837/1681945537] conn(0x55e8277010f0 0x55e827730c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e777952199e4ffbf server_cookie=7e8934acc99fccc in_seq=0 out_seq=0 2024-09-10T21:17:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a10c00640 10 client.7599 ms_handle_connect on v2:172.21.15.16:6835/1681945537 2024-09-10T21:17:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a10c00640 1 -- 172.21.15.16:0/564215823 <== mds.0 v2:172.21.15.16:6835/1681945537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e827532a70 con 0x55e8277010f0 2024-09-10T21:17:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a10c00640 10 client.7599 handle_command_reply: tid=0 2024-09-10T21:17:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 1 client.7599 shutdown 2024-09-10T21:17:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 1 -- 172.21.15.16:0/564215823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8276d2e10 msgr2=0x55e8276ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 1 --2- 172.21.15.16:0/564215823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8276d2e10 0x55e8276ff6a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005040 tx=0x7f8a08005840 comp rx=0 tx=0).stop 2024-09-10T21:17:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 1 -- 172.21.15.16:0/564215823 shutdown_connections 2024-09-10T21:17:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 1 --2- 172.21.15.16:0/564215823 >> [v2:172.21.15.16:6835/1681945537,v1:172.21.15.16:6837/1681945537] conn(0x55e8277010f0 0x55e827730c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 1 --2- 172.21.15.16:0/564215823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771f2f0 0x55e8277003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 1 --2- 172.21.15.16:0/564215823 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e82771e9d0 0x55e8276ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 1 --2- 172.21.15.16:0/564215823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8276d2e10 0x55e8276ff6a0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a10c00640 0 client.7599 ms_handle_reset on v2:172.21.15.16:6835/1681945537 2024-09-10T21:17:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 1 -- 172.21.15.16:0/564215823 >> 172.21.15.16:0/564215823 conn(0x55e8275bc130 msgr2=0x55e8276d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 1 -- 172.21.15.16:0/564215823 shutdown_connections 2024-09-10T21:17:47.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 1 -- 172.21.15.16:0/564215823 wait complete. 2024-09-10T21:17:47.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.097+0000 7f8a2d273000 20 client.7599 trim_cache size 0 max 16384 2024-09-10T21:17:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 Processor -- start 2024-09-10T21:17:47.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 -- start start 2024-09-10T21:17:47.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e8276d2e10 0x55e8277350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 0x55e8277355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771f2f0 0x55e8276d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24202bd0 con 0x55e82771e9d0 2024-09-10T21:17:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24201cb0 con 0x55e82771f2f0 2024-09-10T21:17:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8a240b4440 con 0x55e8276d2e10 2024-09-10T21:17:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 0x55e8277355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 0x55e8277355e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43708/0 (socket says 172.21.15.16:43708) 2024-09-10T21:17:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a1be00640 1 -- 172.21.15.16:0/1986428687 learned_addr learned my addr 172.21.15.16:0/1986428687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a23400640 1 --2- 172.21.15.16:0/1986428687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771f2f0 0x55e8276d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a22a00640 1 --2- 172.21.15.16:0/1986428687 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e8276d2e10 0x55e8277350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a1be00640 1 -- 172.21.15.16:0/1986428687 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e8276d2e10 msgr2=0x55e8277350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a1be00640 1 --2- 172.21.15.16:0/1986428687 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e8276d2e10 0x55e8277350a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a1be00640 1 -- 172.21.15.16:0/1986428687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771f2f0 msgr2=0x55e8276d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a1be00640 1 --2- 172.21.15.16:0/1986428687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771f2f0 0x55e8276d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a1be00640 1 -- 172.21.15.16:0/1986428687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e82757bd90 con 0x55e82771e9d0 2024-09-10T21:17:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a23400640 1 --2- 172.21.15.16:0/1986428687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771f2f0 0x55e8276d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a22a00640 1 --2- 172.21.15.16:0/1986428687 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e8276d2e10 0x55e8277350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a1be00640 1 --2- 172.21.15.16:0/1986428687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 0x55e8277355e0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c05e890 tx=0x7f8a0c01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe9e1ea28371225 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a13400640 1 -- 172.21.15.16:0/1986428687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c063100 con 0x55e82771e9d0 2024-09-10T21:17:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a13400640 1 -- 172.21.15.16:0/1986428687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a0c0632a0 con 0x55e82771e9d0 2024-09-10T21:17:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a13400640 1 -- 172.21.15.16:0/1986428687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c05e030 con 0x55e82771e9d0 2024-09-10T21:17:47.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 -- 172.21.15.16:0/1986428687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 msgr2=0x55e8277355e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1986428687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 0x55e8277355e0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c05e890 tx=0x7f8a0c01c040 comp rx=0 tx=0).stop 2024-09-10T21:17:47.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 -- 172.21.15.16:0/1986428687 shutdown_connections 2024-09-10T21:17:47.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1986428687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e82771f2f0 0x55e8276d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1986428687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 0x55e8277355e0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1986428687 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e8276d2e10 0x55e8277350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 -- 172.21.15.16:0/1986428687 >> 172.21.15.16:0/1986428687 conn(0x55e8275bc130 msgr2=0x55e8276dab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 -- 172.21.15.16:0/1986428687 shutdown_connections 2024-09-10T21:17:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.117+0000 7f8a2d273000 1 -- 172.21.15.16:0/1986428687 wait complete. 2024-09-10T21:17:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 1 -- start start 2024-09-10T21:17:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 1 Processor -- start 2024-09-10T21:17:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e8276d2e10 0x55e8277992e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 0x55e827799820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e82771f2f0 0x55e827799d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8275a4a60 con 0x55e82771e9d0 2024-09-10T21:17:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e827528390 con 0x55e8276d2e10 2024-09-10T21:17:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x55e8275df730 con 0x55e82771f2f0 2024-09-10T21:17:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 10 client.0 _ll_register_callbacks cb 0x7f8a0800bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 0x55e827799820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 0x55e827799820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43716/0 (socket says 172.21.15.16:43716) 2024-09-10T21:17:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a1be00640 1 -- 172.21.15.16:0/1174418368 learned_addr learned my addr 172.21.15.16:0/1174418368 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a23400640 1 --2- 172.21.15.16:0/1174418368 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e82771f2f0 0x55e827799d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a22a00640 1 --2- 172.21.15.16:0/1174418368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e8276d2e10 0x55e8277992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a1be00640 1 -- 172.21.15.16:0/1174418368 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e82771f2f0 msgr2=0x55e827799d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a1be00640 1 --2- 172.21.15.16:0/1174418368 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e82771f2f0 0x55e827799d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a1be00640 1 -- 172.21.15.16:0/1174418368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e8276d2e10 msgr2=0x55e8277992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a1be00640 1 --2- 172.21.15.16:0/1174418368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e8276d2e10 0x55e8277992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a1be00640 1 -- 172.21.15.16:0/1174418368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8274e6a90 con 0x55e82771e9d0 2024-09-10T21:17:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a1be00640 1 --2- 172.21.15.16:0/1174418368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 0x55e827799820 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c05e890 tx=0x7f8a0c08b0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749ea098e20a7718 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a10c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:17:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a10c00640 1 -- 172.21.15.16:0/1174418368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c063100 con 0x55e82771e9d0 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a10c00640 1 -- 172.21.15.16:0/1174418368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f8a0c091ad0 con 0x55e82771e9d0 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 1 -- 172.21.15.16:0/1174418368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a0c0632a0 con 0x55e82771e9d0 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a10c00640 1 -- 172.21.15.16:0/1174418368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8a0c091d50 con 0x55e82771e9d0 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 10 client.7605 fetch_fsmap learned FSMap version 82 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 1 -- 172.21.15.16:0/1174418368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55e8274c5050 con 0x55e82771e9d0 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a10c00640 1 -- 172.21.15.16:0/1174418368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a0c0632a0 con 0x55e82771e9d0 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a10c00640 1 -- 172.21.15.16:0/1174418368 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 82) ==== 3502+0+0 (secure 0 0 0) 0x7f8a0c05ddf0 con 0x55e82771e9d0 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 10 client.7605 fetch_fsmap finished waiting for FSMap version 82 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 10 client.7605 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 20 client.7605 populate_metadata read hostname 'smithi016' 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1174418368 >> [v2:172.21.15.16:6835/1681945537,v1:172.21.15.16:6837/1681945537] conn(0x55e827754ae0 0x55e827754ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 4 client.7605 mds_command: new command op to 136181 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a2d273000 1 -- 172.21.15.16:0/1174418368 --> [v2:172.21.15.16:6835/1681945537,v1:172.21.15.16:6837/1681945537] -- command(tid 0: {"prefix": "status"}) -- 0x55e82752e520 con 0x55e827754ae0 2024-09-10T21:17:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.121+0000 7f8a22a00640 1 --2- 172.21.15.16:0/1174418368 >> [v2:172.21.15.16:6835/1681945537,v1:172.21.15.16:6837/1681945537] conn(0x55e827754ae0 0x55e827754ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a22a00640 1 --2- 172.21.15.16:0/1174418368 >> [v2:172.21.15.16:6835/1681945537,v1:172.21.15.16:6837/1681945537] conn(0x55e827754ae0 0x55e827754ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c136b9e7daf68a18 server_cookie=66f709b1ad6053ab in_seq=0 out_seq=0 2024-09-10T21:17:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a10c00640 10 client.7605 ms_handle_connect on v2:172.21.15.16:6835/1681945537 2024-09-10T21:17:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a10c00640 1 -- 172.21.15.16:0/1174418368 <== mds.0 v2:172.21.15.16:6835/1681945537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x55e82752e520 con 0x55e827754ae0 2024-09-10T21:17:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a10c00640 10 client.7605 handle_command_reply: tid=0 2024-09-10T21:17:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a2d273000 1 client.7605 shutdown 2024-09-10T21:17:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a2d273000 1 -- 172.21.15.16:0/1174418368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 msgr2=0x55e827799820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1174418368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 0x55e827799820 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c05e890 tx=0x7f8a0c08b0f0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a2d273000 1 -- 172.21.15.16:0/1174418368 shutdown_connections 2024-09-10T21:17:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1174418368 >> [v2:172.21.15.16:6835/1681945537,v1:172.21.15.16:6837/1681945537] conn(0x55e827754ae0 0x55e827754ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1174418368 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x55e82771f2f0 0x55e827799d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1174418368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e82771e9d0 0x55e827799820 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a10c00640 0 client.7605 ms_handle_reset on v2:172.21.15.16:6835/1681945537 2024-09-10T21:17:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a2d273000 1 --2- 172.21.15.16:0/1174418368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e8276d2e10 0x55e8277992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a2d273000 1 -- 172.21.15.16:0/1174418368 >> 172.21.15.16:0/1174418368 conn(0x55e8275bc130 msgr2=0x55e8276da2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a2d273000 1 -- 172.21.15.16:0/1174418368 shutdown_connections 2024-09-10T21:17:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a2d273000 1 -- 172.21.15.16:0/1174418368 wait complete. 2024-09-10T21:17:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a2d273000 20 client.7605 trim_cache size 0 max 16384 2024-09-10T21:17:47.133 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:17:47.133 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "eb19be60-6d8f-4816-b778-3d7d6698304d", 2024-09-10T21:17:47.133 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-10T21:17:47.133 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 136181, 2024-09-10T21:17:47.134 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-10T21:17:47.134 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-10T21:17:47.134 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-10T21:17:47.134 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.0417533670000001, 2024-09-10T21:17:47.134 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 82, 2024-09-10T21:17:47.134 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 116, 2024-09-10T21:17:47.134 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 116, 2024-09-10T21:17:47.134 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 10.501988738 2024-09-10T21:17:47.134 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:17:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a29e00640 1 -- 172.21.15.16:0/1681600844 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8a00072610 msgr2=0x7f8a00074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a29e00640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8a00072610 0x7f8a00074ad0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8a140017d0 tx=0x7f8a140196c0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a29e00640 1 -- 172.21.15.16:0/1681600844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a24158aa0 msgr2=0x7f8a24157010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a29e00640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a24158aa0 0x7f8a24157010 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0117a0 tx=0x7f8a0c011ba0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a29e00640 1 -- 172.21.15.16:0/1681600844 shutdown_connections 2024-09-10T21:17:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a29e00640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8a00072610 0x7f8a00074ad0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a29e00640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a24154a10 0x7f8a241f95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a29e00640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a24158aa0 0x7f8a24157010 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a29e00640 1 --2- 172.21.15.16:0/1681600844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a08003cc0 0x7f8a2415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a29e00640 1 -- 172.21.15.16:0/1681600844 >> 172.21.15.16:0/1681600844 conn(0x7f8a240d0c30 msgr2=0x7f8a24157840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a29e00640 1 -- 172.21.15.16:0/1681600844 shutdown_connections 2024-09-10T21:17:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.125+0000 7f8a29e00640 1 -- 172.21.15.16:0/1681600844 wait complete. 2024-09-10T21:17:47.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:17:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.293+0000 7f8b46800640 1 -- 172.21.15.16:0/2278307126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b240051d0 msgr2=0x7f8b2400dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.293+0000 7f8b46800640 1 --2- 172.21.15.16:0/2278307126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b240051d0 0x7f8b2400dbe0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8b400c3370 tx=0x7f8b3801d5f0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.293+0000 7f8b3fe00640 1 -- 172.21.15.16:0/2278307126 reap_dead start 2024-09-10T21:17:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.293+0000 7f8b46800640 1 -- 172.21.15.16:0/2278307126 shutdown_connections 2024-09-10T21:17:47.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.293+0000 7f8b46800640 1 -- 172.21.15.16:0/2278307126 >> 172.21.15.16:0/2278307126 conn(0x7f8b400cfb20 msgr2=0x7f8b400cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:47.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.293+0000 7f8b46800640 1 -- 172.21.15.16:0/2278307126 shutdown_connections 2024-09-10T21:17:47.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.293+0000 7f8b46800640 1 -- 172.21.15.16:0/2278307126 wait complete. 2024-09-10T21:17:47.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b46800640 1 Processor -- start 2024-09-10T21:17:47.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b46800640 1 -- start start 2024-09-10T21:17:47.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b46800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b240051d0 0x7f8b400c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b46800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8b401f7fe0 0x7f8b400b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b46800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b400c8fe0 0x7f8b400c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b46800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4006f050 con 0x7f8b240051d0 2024-09-10T21:17:47.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b46800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4006eed0 con 0x7f8b400c8fe0 2024-09-10T21:17:47.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b46800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8b4006f1d0 con 0x7f8b401f7fe0 2024-09-10T21:17:47.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b240051d0 0x7f8b400c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b240051d0 0x7f8b400c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43740/0 (socket says 172.21.15.16:43740) 2024-09-10T21:17:47.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b3f400640 1 -- 172.21.15.16:0/1812278874 learned_addr learned my addr 172.21.15.16:0/1812278874 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:47.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b37e00640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8b401f7fe0 0x7f8b400b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b3fe00640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b400c8fe0 0x7f8b400c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b3f400640 1 -- 172.21.15.16:0/1812278874 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8b401f7fe0 msgr2=0x7f8b400b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b3f400640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8b401f7fe0 0x7f8b400b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b3f400640 1 -- 172.21.15.16:0/1812278874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b400c8fe0 msgr2=0x7f8b400c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b3f400640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b400c8fe0 0x7f8b400c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b3f400640 1 -- 172.21.15.16:0/1812278874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b401f8470 con 0x7f8b240051d0 2024-09-10T21:17:47.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b3f400640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b240051d0 0x7f8b400c8340 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f8b30003e80 tx=0x7f8b300017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:47.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b37400640 1 -- 172.21.15.16:0/1812278874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30015070 con 0x7f8b240051d0 2024-09-10T21:17:47.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b37400640 1 -- 172.21.15.16:0/1812278874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b30011940 con 0x7f8b240051d0 2024-09-10T21:17:47.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b46800640 1 -- 172.21.15.16:0/1812278874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b401f8750 con 0x7f8b240051d0 2024-09-10T21:17:47.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b37400640 1 -- 172.21.15.16:0/1812278874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30011d90 con 0x7f8b240051d0 2024-09-10T21:17:47.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b46800640 1 -- 172.21.15.16:0/1812278874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b401f8d10 con 0x7f8b240051d0 2024-09-10T21:17:47.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.297+0000 7f8b46800640 1 -- 172.21.15.16:0/1812278874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b40072d40 con 0x7f8b240051d0 2024-09-10T21:17:47.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.301+0000 7f8b37400640 1 -- 172.21.15.16:0/1812278874 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8b300103c0 con 0x7f8b240051d0 2024-09-10T21:17:47.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.301+0000 7f8b37400640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8b1c0725e0 0x7f8b1c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.301+0000 7f8b37400640 1 -- 172.21.15.16:0/1812278874 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f8b300942c0 con 0x7f8b240051d0 2024-09-10T21:17:47.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.301+0000 7f8b37e00640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8b1c0725e0 0x7f8b1c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.301+0000 7f8b37e00640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8b1c0725e0 0x7f8b1c074aa0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8b28000c10 tx=0x7f8b280011f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.305+0000 7f8b37400640 1 -- 172.21.15.16:0/1812278874 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8b3005f3e0 con 0x7f8b240051d0 2024-09-10T21:17:47.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.517+0000 7f8b46800640 1 -- 172.21.15.16:0/1812278874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8b400ce8e0 con 0x7f8b240051d0 2024-09-10T21:17:47.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.517+0000 7f8b37400640 1 -- 172.21.15.16:0/1812278874 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7f8b30063e10 con 0x7f8b240051d0 2024-09-10T21:17:47.522 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.521+0000 7f8b46800640 1 -- 172.21.15.16:0/1812278874 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8b1c0725e0 msgr2=0x7f8b1c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.521+0000 7f8b46800640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8b1c0725e0 0x7f8b1c074aa0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8b28000c10 tx=0x7f8b280011f0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.521+0000 7f8b46800640 1 -- 172.21.15.16:0/1812278874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b240051d0 msgr2=0x7f8b400c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.521+0000 7f8b46800640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b240051d0 0x7f8b400c8340 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f8b30003e80 tx=0x7f8b300017b0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.521+0000 7f8b46800640 1 -- 172.21.15.16:0/1812278874 shutdown_connections 2024-09-10T21:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.521+0000 7f8b46800640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8b1c0725e0 0x7f8b1c074aa0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.521+0000 7f8b46800640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b400c8fe0 0x7f8b400c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.521+0000 7f8b46800640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8b401f7fe0 0x7f8b400b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.521+0000 7f8b46800640 1 --2- 172.21.15.16:0/1812278874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b240051d0 0x7f8b400c8340 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.521+0000 7f8b46800640 1 -- 172.21.15.16:0/1812278874 >> 172.21.15.16:0/1812278874 conn(0x7f8b400cfb20 msgr2=0x7f8b40159a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.521+0000 7f8b46800640 1 -- 172.21.15.16:0/1812278874 shutdown_connections 2024-09-10T21:17:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47.521+0000 7f8b46800640 1 -- 172.21.15.16:0/1812278874 wait complete. 2024-09-10T21:17:47.543 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:17:47.543 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:17:47.543 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-10T21:17:47.544 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:17:47.544 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:17:47.544 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:17:47.544 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:17:47.544 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:17:47.544 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:17:47.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:47.544 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-10T21:17:47.554 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:17:47.554 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-10T21:17:47.555 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:17:47.556 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41982sec preferred_lft 41982sec 2024-09-10T21:17:47.556 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-10T21:17:47.556 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:47.556 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:17:47.556 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:47.556 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-10T21:17:47.556 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:17:47.556 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:30:fb:74:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:47.556 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:17:47.556 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:47.556 INFO:teuthology.orchestra.run.smithi016.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:17:47.556 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:47.556 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:17:47.557 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:47.557 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:17:47.557 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:47.557 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:17:47.557 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:17:47.557 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:17:47.557 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:47.557 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:17:47.557 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:17:47.557 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:17:47.557 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:47.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:47.558 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:17:47.613 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-10T21:17:47.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:47.615 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-10T21:17:47.668 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-10T21:17:47.668 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:17:47.669 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:17:47.669 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:17:47.669 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:17:47.669 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:17:47.669 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:17:47.669 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:17:47.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:17:47.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:17:47.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:17:47.670 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:17:47.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:17:47.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:17:47.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:47.807 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:17:47.859 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:17:47.859 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:17:47.860 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:17:47.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:47.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-10T21:17:47.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:17:47.924 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:17:47.975 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:17:47.975 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:17:47.975 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:17:48.027 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:17:48.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:48.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:17:48.117 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:17:48.118 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:17:48.130 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:17:48.130 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:17:48.130 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:17:48.130 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:17:48.130 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:17:48.134 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:17:48.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:48.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:17:48.154 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:17:48.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:17:48.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:17:48.261 INFO:teuthology.orchestra.run.smithi016.stdout:client7078 2024-09-10T21:17:48.261 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:17:48.261 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:48.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:17:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 -- 172.21.15.16:0/3541728445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184c15bc70 msgr2=0x7f184c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 --2- 172.21.15.16:0/3541728445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184c15bc70 0x7f184c06ffe0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f184c15e510 tx=0x7f183401c600 comp rx=0 tx=0).stop 2024-09-10T21:17:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 -- 172.21.15.16:0/3541728445 shutdown_connections 2024-09-10T21:17:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 --2- 172.21.15.16:0/3541728445 >> v2:172.21.15.52:3300/0 conn(0x7f184c070520 0x7f184c15e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 --2- 172.21.15.16:0/3541728445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184c15bc70 0x7f184c06ffe0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 --2- 172.21.15.16:0/3541728445 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f184c154f90 0x7f184c072d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 -- 172.21.15.16:0/3541728445 >> 172.21.15.16:0/3541728445 conn(0x7f184c0cfb20 msgr2=0x7f184c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 -- 172.21.15.16:0/3541728445 shutdown_connections 2024-09-10T21:17:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 -- 172.21.15.16:0/3541728445 wait complete. 2024-09-10T21:17:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 Processor -- start 2024-09-10T21:17:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 -- start start 2024-09-10T21:17:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f184c070520 0x7f184c14d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f184c154f90 0x7f184c06f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184c15bc70 0x7f184c06fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f184c0cac00 con 0x7f184c15bc70 2024-09-10T21:17:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f184c0caa80 con 0x7f184c154f90 2024-09-10T21:17:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f1851200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f184c0cad80 con 0x7f184c070520 2024-09-10T21:17:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f184a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184c15bc70 0x7f184c06fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f184a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184c15bc70 0x7f184c06fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43760/0 (socket says 172.21.15.16:43760) 2024-09-10T21:17:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.453+0000 7f184a800640 1 -- 172.21.15.16:0/3269118149 learned_addr learned my addr 172.21.15.16:0/3269118149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f1849e00640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f184c070520 0x7f184c14d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f1841400640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f184c154f90 0x7f184c06f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f184a800640 1 -- 172.21.15.16:0/3269118149 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f184c070520 msgr2=0x7f184c14d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f184a800640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f184c070520 0x7f184c14d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f184a800640 1 -- 172.21.15.16:0/3269118149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f184c154f90 msgr2=0x7f184c06f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f184a800640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f184c154f90 0x7f184c06f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f184a800640 1 -- 172.21.15.16:0/3269118149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c0c58e0 con 0x7f184c15bc70 2024-09-10T21:17:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f1841400640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f184c154f90 0x7f184c06f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f184a800640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184c15bc70 0x7f184c06fbd0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f184400ea20 tx=0x7f184400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f1842a00640 1 -- 172.21.15.16:0/3269118149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184400cde0 con 0x7f184c15bc70 2024-09-10T21:17:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f1842a00640 1 -- 172.21.15.16:0/3269118149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1844004c40 con 0x7f184c15bc70 2024-09-10T21:17:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f1851200640 1 -- 172.21.15.16:0/3269118149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c0b35d0 con 0x7f184c15bc70 2024-09-10T21:17:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f1842a00640 1 -- 172.21.15.16:0/3269118149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844005280 con 0x7f184c15bc70 2024-09-10T21:17:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f1851200640 1 -- 172.21.15.16:0/3269118149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c0b3b90 con 0x7f184c15bc70 2024-09-10T21:17:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f1851200640 1 -- 172.21.15.16:0/3269118149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184c072d40 con 0x7f184c15bc70 2024-09-10T21:17:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f1842a00640 1 -- 172.21.15.16:0/3269118149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f184400f040 con 0x7f184c15bc70 2024-09-10T21:17:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.457+0000 7f1842a00640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1828072610 0x7f1828074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.461+0000 7f1842a00640 1 -- 172.21.15.16:0/3269118149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f1844093920 con 0x7f184c15bc70 2024-09-10T21:17:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.461+0000 7f1849e00640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1828072610 0x7f1828074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.461+0000 7f1849e00640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1828072610 0x7f1828074ad0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f183c00bb30 tx=0x7f183c00b510 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.465+0000 7f1842a00640 1 -- 172.21.15.16:0/3269118149 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1844016030 con 0x7f184c15bc70 2024-09-10T21:17:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.673+0000 7f1851200640 1 -- 172.21.15.16:0/3269118149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f184c14fe50 con 0x7f184c15bc70 2024-09-10T21:17:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.673+0000 7f1842a00640 1 -- 172.21.15.16:0/3269118149 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7f184405eff0 con 0x7f184c15bc70 2024-09-10T21:17:48.679 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:17:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.677+0000 7f1851200640 1 -- 172.21.15.16:0/3269118149 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1828072610 msgr2=0x7f1828074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.677+0000 7f1851200640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1828072610 0x7f1828074ad0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f183c00bb30 tx=0x7f183c00b510 comp rx=0 tx=0).stop 2024-09-10T21:17:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.677+0000 7f1851200640 1 -- 172.21.15.16:0/3269118149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184c15bc70 msgr2=0x7f184c06fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.677+0000 7f1851200640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184c15bc70 0x7f184c06fbd0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f184400ea20 tx=0x7f184400eee0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.677+0000 7f1851200640 1 -- 172.21.15.16:0/3269118149 shutdown_connections 2024-09-10T21:17:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.677+0000 7f1851200640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1828072610 0x7f1828074ad0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.677+0000 7f1851200640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184c15bc70 0x7f184c06fbd0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.677+0000 7f1851200640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f184c154f90 0x7f184c06f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.677+0000 7f1851200640 1 --2- 172.21.15.16:0/3269118149 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f184c070520 0x7f184c14d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.677+0000 7f1851200640 1 -- 172.21.15.16:0/3269118149 >> 172.21.15.16:0/3269118149 conn(0x7f184c0cfb20 msgr2=0x7f184c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.677+0000 7f1851200640 1 -- 172.21.15.16:0/3269118149 shutdown_connections 2024-09-10T21:17:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:48.677+0000 7f1851200640 1 -- 172.21.15.16:0/3269118149 wait complete. 2024-09-10T21:17:48.700 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:17:48.700 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:17:48.700 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-09-10T21:17:48.700 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T21:17:48.700 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:17:48.700 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:17:48.700 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:17:48.700 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:17:48.700 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:17:48.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:48.701 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-09-10T21:17:48.711 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:17:48.711 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:17:48.711 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:17:48.711 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:48.711 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:17:48.711 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:48.712 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:17:48.712 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:48.712 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-09-10T21:17:48.712 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:17:48.712 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:48.712 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-09-10T21:17:48.712 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:17:48.712 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:48.713 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-09-10T21:17:48.713 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:17:48.713 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 41982sec preferred_lft 41982sec 2024-09-10T21:17:48.713 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-09-10T21:17:48.713 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:48.713 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:17:48.713 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:48.713 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-09-10T21:17:48.713 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:17:48.714 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:7e:b4:71:a2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:48.714 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:17:48.714 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:48.714 INFO:teuthology.orchestra.run.smithi052.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:17:48.714 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:17:48.714 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:17:48.714 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:48.714 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:17:48.714 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:48.714 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:17:48.714 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:17:48.714 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:17:48.714 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:48.715 INFO:teuthology.orchestra.run.smithi052.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:17:48.715 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:17:48.715 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:17:48.715 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:17:48.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:48.716 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:17:48.780 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-09-10T21:17:48.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:48.782 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-09-10T21:17:48.834 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-09-10T21:17:48.834 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:17:48.834 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:17:48.834 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:17:48.834 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:17:48.834 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:17:48.834 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:17:48.834 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:17:48.835 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:17:48.835 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:17:48.835 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:17:48.835 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:17:48.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:17:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:17:48.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:17:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:17:48.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:48.986 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:17:48.995 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:17:48.995 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:17:48.996 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:17:48.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:48.996 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-09-10T21:17:49.060 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:17:49.060 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T21:17:49.116 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:17:49.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:17:49.117 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:17:49.167 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T21:17:49.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:49.167 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:17:49.257 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:17:49.257 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:17:49.270 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:17:49.270 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:17:49.270 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:17:49.270 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 1@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:17:49.270 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:17:49.275 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:17:49.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:17:49.275 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:17:49.295 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:17:49.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:17:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:17:49.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:17:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:17:49.393 INFO:teuthology.orchestra.run.smithi052.stdout:client136286 2024-09-10T21:17:49.393 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:17:49.393 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:49.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-09-10T21:17:49.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.549+0000 7f62a5000640 1 -- 172.21.15.16:0/353869544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6284007340 msgr2=0x7f6284009770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.549+0000 7f62a5000640 1 --2- 172.21.15.16:0/353869544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6284007340 0x7f6284009770 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f628800c290 tx=0x7f628801cf70 comp rx=0 tx=0).stop 2024-09-10T21:17:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.549+0000 7f629e800640 1 -- 172.21.15.16:0/353869544 reap_dead start 2024-09-10T21:17:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.549+0000 7f62a5000640 1 -- 172.21.15.16:0/353869544 shutdown_connections 2024-09-10T21:17:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.549+0000 7f62a5000640 1 -- 172.21.15.16:0/353869544 >> 172.21.15.16:0/353869544 conn(0x7f62a00b16a0 msgr2=0x7f62a00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.549+0000 7f62a5000640 1 -- 172.21.15.16:0/353869544 shutdown_connections 2024-09-10T21:17:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.549+0000 7f62a5000640 1 -- 172.21.15.16:0/353869544 wait complete. 2024-09-10T21:17:49.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f62a5000640 1 Processor -- start 2024-09-10T21:17:49.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f62a5000640 1 -- start start 2024-09-10T21:17:49.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f62a5000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6284007340 0x7f62a01f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f62a5000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62a00a9a80 0x7f62a0152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f62a5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a0162700 0x7f62a0136f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f62a5000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62a006fa70 con 0x7f62a0162700 2024-09-10T21:17:49.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f62a5000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62a006f8f0 con 0x7f6284007340 2024-09-10T21:17:49.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f62a5000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f62a006fbf0 con 0x7f62a00a9a80 2024-09-10T21:17:49.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f629e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a0162700 0x7f62a0136f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f629e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a0162700 0x7f62a0136f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43770/0 (socket says 172.21.15.16:43770) 2024-09-10T21:17:49.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f629e800640 1 -- 172.21.15.16:0/2690960794 learned_addr learned my addr 172.21.15.16:0/2690960794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:49.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f629de00640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6284007340 0x7f62a01f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f6297e00640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62a00a9a80 0x7f62a0152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f629e800640 1 -- 172.21.15.16:0/2690960794 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62a00a9a80 msgr2=0x7f62a0152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f629e800640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62a00a9a80 0x7f62a0152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f629e800640 1 -- 172.21.15.16:0/2690960794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6284007340 msgr2=0x7f62a01f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f629e800640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6284007340 0x7f62a01f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f629e800640 1 -- 172.21.15.16:0/2690960794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a013ba60 con 0x7f62a0162700 2024-09-10T21:17:49.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f629e800640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a0162700 0x7f62a0136f10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f62a006e850 tx=0x7f6298011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:49.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f6296000640 1 -- 172.21.15.16:0/2690960794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298004b30 con 0x7f62a0162700 2024-09-10T21:17:49.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f6296000640 1 -- 172.21.15.16:0/2690960794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6298004cd0 con 0x7f62a0162700 2024-09-10T21:17:49.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f6296000640 1 -- 172.21.15.16:0/2690960794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298003be0 con 0x7f62a0162700 2024-09-10T21:17:49.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f62a5000640 1 -- 172.21.15.16:0/2690960794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a013bd40 con 0x7f62a0162700 2024-09-10T21:17:49.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f62a5000640 1 -- 172.21.15.16:0/2690960794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a01f6150 con 0x7f62a0162700 2024-09-10T21:17:49.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.553+0000 7f62a5000640 1 -- 172.21.15.16:0/2690960794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a00b24d0 con 0x7f62a0162700 2024-09-10T21:17:49.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.557+0000 7f6296000640 1 -- 172.21.15.16:0/2690960794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6298018030 con 0x7f62a0162700 2024-09-10T21:17:49.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.557+0000 7f6296000640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f627c0725e0 0x7f627c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.557+0000 7f6296000640 1 -- 172.21.15.16:0/2690960794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f6298093780 con 0x7f62a0162700 2024-09-10T21:17:49.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.557+0000 7f629de00640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f627c0725e0 0x7f627c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.557+0000 7f629de00640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f627c0725e0 0x7f627c074aa0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6290009ce0 tx=0x7f62900091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:49.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.561+0000 7f6296000640 1 -- 172.21.15.16:0/2690960794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f629805eed0 con 0x7f62a0162700 2024-09-10T21:17:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.737+0000 7f62a5000640 1 -- 172.21.15.16:0/2690960794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f62a00af950 con 0x7f62a0162700 2024-09-10T21:17:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.741+0000 7f6296000640 1 -- 172.21.15.16:0/2690960794 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v54)=0 v54) ==== 121+0+0 (secure 0 0 0) 0x7f629805e7d0 con 0x7f62a0162700 2024-09-10T21:17:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.741+0000 7f62a5000640 1 -- 172.21.15.16:0/2690960794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f627c0725e0 msgr2=0x7f627c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.741+0000 7f62a5000640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f627c0725e0 0x7f627c074aa0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6290009ce0 tx=0x7f62900091e0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.741+0000 7f62a5000640 1 -- 172.21.15.16:0/2690960794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a0162700 msgr2=0x7f62a0136f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.741+0000 7f62a5000640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a0162700 0x7f62a0136f10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f62a006e850 tx=0x7f6298011ef0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.741+0000 7f62a5000640 1 -- 172.21.15.16:0/2690960794 shutdown_connections 2024-09-10T21:17:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.741+0000 7f62a5000640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f627c0725e0 0x7f627c074aa0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.741+0000 7f62a5000640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a0162700 0x7f62a0136f10 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.741+0000 7f62a5000640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f62a00a9a80 0x7f62a0152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.741+0000 7f62a5000640 1 --2- 172.21.15.16:0/2690960794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6284007340 0x7f62a01f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.741+0000 7f62a5000640 1 -- 172.21.15.16:0/2690960794 >> 172.21.15.16:0/2690960794 conn(0x7f62a00b16a0 msgr2=0x7f62a0138520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.741+0000 7f62a5000640 1 -- 172.21.15.16:0/2690960794 shutdown_connections 2024-09-10T21:17:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.741+0000 7f62a5000640 1 -- 172.21.15.16:0/2690960794 wait complete. 2024-09-10T21:17:49.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 100 2024-09-10T21:17:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 -- 172.21.15.16:0/366124853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0c34b0 msgr2=0x7f58cc15f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 --2- 172.21.15.16:0/366124853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0c34b0 0x7f58cc15f0a0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f58cc051fb0 tx=0x7f58c401f580 comp rx=0 tx=0).stop 2024-09-10T21:17:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 -- 172.21.15.16:0/366124853 shutdown_connections 2024-09-10T21:17:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 --2- 172.21.15.16:0/366124853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc0c34b0 0x7f58cc15f0a0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 --2- 172.21.15.16:0/366124853 >> v2:172.21.15.52:3300/0 conn(0x7f58cc073290 0x7f58cc0cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 -- 172.21.15.16:0/366124853 >> 172.21.15.16:0/366124853 conn(0x7f58cc0cfe20 msgr2=0x7f58cc0d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 -- 172.21.15.16:0/366124853 shutdown_connections 2024-09-10T21:17:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 -- 172.21.15.16:0/366124853 wait complete. 2024-09-10T21:17:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 Processor -- start 2024-09-10T21:17:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 -- start start 2024-09-10T21:17:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc073290 0x7f58cc0b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f58cc0c34b0 0x7f58cc09eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc0cf620 0x7f58cc0b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc0cbeb0 con 0x7f58cc073290 2024-09-10T21:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc073880 con 0x7f58cc0cf620 2024-09-10T21:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58cb200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc0cc030 con 0x7f58cc0c34b0 2024-09-10T21:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58c9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc073290 0x7f58cc0b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58c9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc073290 0x7f58cc0b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43788/0 (socket says 172.21.15.16:43788) 2024-09-10T21:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58c9e00640 1 -- 172.21.15.16:0/2467298797 learned_addr learned my addr 172.21.15.16:0/2467298797 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58c3e00640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f58cc0c34b0 0x7f58cc09eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58ca800640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc0cf620 0x7f58cc0b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58c9e00640 1 -- 172.21.15.16:0/2467298797 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f58cc0c34b0 msgr2=0x7f58cc09eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58c9e00640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f58cc0c34b0 0x7f58cc09eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58c9e00640 1 -- 172.21.15.16:0/2467298797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc0cf620 msgr2=0x7f58cc0b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58c9e00640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc0cf620 0x7f58cc0b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.913+0000 7f58c9e00640 1 -- 172.21.15.16:0/2467298797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc09f7f0 con 0x7f58cc073290 2024-09-10T21:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.917+0000 7f58c9e00640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc073290 0x7f58cc0b5710 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00d980 tx=0x7f58bc00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.917+0000 7f58c2000640 1 -- 172.21.15.16:0/2467298797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc017070 con 0x7f58cc073290 2024-09-10T21:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.917+0000 7f58c2000640 1 -- 172.21.15.16:0/2467298797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58bc009d20 con 0x7f58cc073290 2024-09-10T21:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.917+0000 7f58c2000640 1 -- 172.21.15.16:0/2467298797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc004280 con 0x7f58cc073290 2024-09-10T21:17:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.917+0000 7f58cb200640 1 -- 172.21.15.16:0/2467298797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc09fad0 con 0x7f58cc073290 2024-09-10T21:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.917+0000 7f58cb200640 1 -- 172.21.15.16:0/2467298797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc0a0090 con 0x7f58cc073290 2024-09-10T21:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.917+0000 7f58cb200640 1 -- 172.21.15.16:0/2467298797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58cc0cb260 con 0x7f58cc073290 2024-09-10T21:17:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.917+0000 7f58c2000640 1 -- 172.21.15.16:0/2467298797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f58bc003600 con 0x7f58cc073290 2024-09-10T21:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.917+0000 7f58c2000640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f58a8072610 0x7f58a8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.921+0000 7f58c2000640 1 -- 172.21.15.16:0/2467298797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f58bc093330 con 0x7f58cc073290 2024-09-10T21:17:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.921+0000 7f58c3e00640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f58a8072610 0x7f58a8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.921+0000 7f58c3e00640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f58a8072610 0x7f58a8074ad0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f58cc09dde0 tx=0x7f58b4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:49.925+0000 7f58c2000640 1 -- 172.21.15.16:0/2467298797 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f58bc05e320 con 0x7f58cc073290 2024-09-10T21:17:50.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.097+0000 7f58cb200640 1 -- 172.21.15.16:0/2467298797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7f58cc0707b0 con 0x7f58cc073290 2024-09-10T21:17:50.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.101+0000 7f58c2000640 1 -- 172.21.15.16:0/2467298797 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v55)=0 v55) ==== 119+0+0 (secure 0 0 0) 0x7f58bc062d50 con 0x7f58cc073290 2024-09-10T21:17:50.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.105+0000 7f58cb200640 1 -- 172.21.15.16:0/2467298797 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f58a8072610 msgr2=0x7f58a8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:50.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.105+0000 7f58cb200640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f58a8072610 0x7f58a8074ad0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f58cc09dde0 tx=0x7f58b4009040 comp rx=0 tx=0).stop 2024-09-10T21:17:50.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.105+0000 7f58cb200640 1 -- 172.21.15.16:0/2467298797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc073290 msgr2=0x7f58cc0b5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:50.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.105+0000 7f58cb200640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc073290 0x7f58cc0b5710 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00d980 tx=0x7f58bc00de40 comp rx=0 tx=0).stop 2024-09-10T21:17:50.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.105+0000 7f58cb200640 1 -- 172.21.15.16:0/2467298797 shutdown_connections 2024-09-10T21:17:50.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.105+0000 7f58cb200640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f58a8072610 0x7f58a8074ad0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.105+0000 7f58cb200640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc0cf620 0x7f58cc0b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.105+0000 7f58cb200640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f58cc0c34b0 0x7f58cc09eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.105+0000 7f58cb200640 1 --2- 172.21.15.16:0/2467298797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58cc073290 0x7f58cc0b5710 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.105+0000 7f58cb200640 1 -- 172.21.15.16:0/2467298797 >> 172.21.15.16:0/2467298797 conn(0x7f58cc0cfe20 msgr2=0x7f58cc0afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:50.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.105+0000 7f58cb200640 1 -- 172.21.15.16:0/2467298797 shutdown_connections 2024-09-10T21:17:50.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.105+0000 7f58cb200640 1 -- 172.21.15.16:0/2467298797 wait complete. 2024-09-10T21:17:50.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 200 2024-09-10T21:17:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.273+0000 7f6580200640 1 -- 172.21.15.16:0/4112257230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65780ce040 msgr2=0x7f65781671a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.273+0000 7f6580200640 1 --2- 172.21.15.16:0/4112257230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65780ce040 0x7f65781671a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f65780cda10 tx=0x7f657001f210 comp rx=0 tx=0).stop 2024-09-10T21:17:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.273+0000 7f6580200640 1 -- 172.21.15.16:0/4112257230 shutdown_connections 2024-09-10T21:17:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.273+0000 7f6580200640 1 --2- 172.21.15.16:0/4112257230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65780ce040 0x7f65781671a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.273+0000 7f6580200640 1 -- 172.21.15.16:0/4112257230 >> 172.21.15.16:0/4112257230 conn(0x7f65780d1df0 msgr2=0x7f65780d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.273+0000 7f6580200640 1 -- 172.21.15.16:0/4112257230 shutdown_connections 2024-09-10T21:17:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.273+0000 7f6580200640 1 -- 172.21.15.16:0/4112257230 wait complete. 2024-09-10T21:17:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f6580200640 1 Processor -- start 2024-09-10T21:17:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f6580200640 1 -- start start 2024-09-10T21:17:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f6580200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65780ce040 0x7f65780b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f6580200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65780cfe30 0x7f657815a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f6580200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f65780d0210 0x7f6578165120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f6580200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65780b12e0 con 0x7f65780cfe30 2024-09-10T21:17:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f6580200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f65780d2b20 con 0x7f65780ce040 2024-09-10T21:17:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f6580200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f65780b1460 con 0x7f65780d0210 2024-09-10T21:17:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f657ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65780cfe30 0x7f657815a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f657ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65780cfe30 0x7f657815a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43796/0 (socket says 172.21.15.16:43796) 2024-09-10T21:17:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f657ce00640 1 -- 172.21.15.16:0/4056435619 learned_addr learned my addr 172.21.15.16:0/4056435619 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f657e200640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f65780d0210 0x7f6578165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f657d800640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65780ce040 0x7f65780b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f657ce00640 1 -- 172.21.15.16:0/4056435619 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f65780d0210 msgr2=0x7f6578165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f657ce00640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f65780d0210 0x7f6578165120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f657ce00640 1 -- 172.21.15.16:0/4056435619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65780ce040 msgr2=0x7f65780b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f657ce00640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65780ce040 0x7f65780b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f657ce00640 1 -- 172.21.15.16:0/4056435619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6578167070 con 0x7f65780cfe30 2024-09-10T21:17:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f657e200640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f65780d0210 0x7f6578165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f657ce00640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65780cfe30 0x7f657815a070 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f656400c730 tx=0x7f656400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f6575600640 1 -- 172.21.15.16:0/4056435619 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564005480 con 0x7f65780cfe30 2024-09-10T21:17:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f6575600640 1 -- 172.21.15.16:0/4056435619 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f656400ce40 con 0x7f65780cfe30 2024-09-10T21:17:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f6575600640 1 -- 172.21.15.16:0/4056435619 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656400a880 con 0x7f65780cfe30 2024-09-10T21:17:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f6580200640 1 -- 172.21.15.16:0/4056435619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6578167350 con 0x7f65780cfe30 2024-09-10T21:17:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.277+0000 7f6580200640 1 -- 172.21.15.16:0/4056435619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65780b5cb0 con 0x7f65780cfe30 2024-09-10T21:17:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.281+0000 7f6580200640 1 -- 172.21.15.16:0/4056435619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65780ce590 con 0x7f65780cfe30 2024-09-10T21:17:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.281+0000 7f6575600640 1 -- 172.21.15.16:0/4056435619 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f656400aa20 con 0x7f65780cfe30 2024-09-10T21:17:50.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.281+0000 7f6575600640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65580725e0 0x7f6558074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:50.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.281+0000 7f6575600640 1 -- 172.21.15.16:0/4056435619 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f6564093130 con 0x7f65780cfe30 2024-09-10T21:17:50.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.281+0000 7f657d800640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65580725e0 0x7f6558074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.281+0000 7f657d800640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65580725e0 0x7f6558074aa0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f656c000f70 tx=0x7f656c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:50.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.285+0000 7f6575600640 1 -- 172.21.15.16:0/4056435619 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f656405e800 con 0x7f65780cfe30 2024-09-10T21:17:50.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.457+0000 7f6580200640 1 -- 172.21.15.16:0/4056435619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7f65780b5e80 con 0x7f65780cfe30 2024-09-10T21:17:50.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.461+0000 7f6575600640 1 -- 172.21.15.16:0/4056435619 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v56)=0 v56) ==== 128+0+0 (secure 0 0 0) 0x7f656405e100 con 0x7f65780cfe30 2024-09-10T21:17:50.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.465+0000 7f6580200640 1 -- 172.21.15.16:0/4056435619 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65580725e0 msgr2=0x7f6558074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:50.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.465+0000 7f6580200640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65580725e0 0x7f6558074aa0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f656c000f70 tx=0x7f656c009040 comp rx=0 tx=0).stop 2024-09-10T21:17:50.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.465+0000 7f6580200640 1 -- 172.21.15.16:0/4056435619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65780cfe30 msgr2=0x7f657815a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:50.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.465+0000 7f6580200640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65780cfe30 0x7f657815a070 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f656400c730 tx=0x7f656400cbf0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.465+0000 7f6580200640 1 -- 172.21.15.16:0/4056435619 shutdown_connections 2024-09-10T21:17:50.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.465+0000 7f6580200640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65580725e0 0x7f6558074aa0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.465+0000 7f6580200640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f65780d0210 0x7f6578165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.465+0000 7f6580200640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65780cfe30 0x7f657815a070 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.465+0000 7f6580200640 1 --2- 172.21.15.16:0/4056435619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65780ce040 0x7f65780b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.465+0000 7f6580200640 1 -- 172.21.15.16:0/4056435619 >> 172.21.15.16:0/4056435619 conn(0x7f65780d1df0 msgr2=0x7f65780ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:50.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.465+0000 7f6580200640 1 -- 172.21.15.16:0/4056435619 shutdown_connections 2024-09-10T21:17:50.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.465+0000 7f6580200640 1 -- 172.21.15.16:0/4056435619 wait complete. 2024-09-10T21:17:50.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-10T21:17:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.633+0000 7fcc1cc00640 1 -- 172.21.15.16:0/4020112429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbfc004810 msgr2=0x7fcbfc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.633+0000 7fcc1cc00640 1 --2- 172.21.15.16:0/4020112429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbfc004810 0x7fcbfc004c90 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fcc00001830 tx=0x7fcc0001c8a0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.633+0000 7fcc16800640 1 -- 172.21.15.16:0/4020112429 reap_dead start 2024-09-10T21:17:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc1cc00640 1 -- 172.21.15.16:0/4020112429 shutdown_connections 2024-09-10T21:17:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc1cc00640 1 -- 172.21.15.16:0/4020112429 >> 172.21.15.16:0/4020112429 conn(0x7fcc180d1ce0 msgr2=0x7fcc180d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc1cc00640 1 -- 172.21.15.16:0/4020112429 shutdown_connections 2024-09-10T21:17:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc1cc00640 1 -- 172.21.15.16:0/4020112429 wait complete. 2024-09-10T21:17:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc1cc00640 1 Processor -- start 2024-09-10T21:17:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc1cc00640 1 -- start start 2024-09-10T21:17:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc1cc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcbfc004810 0x7fcc181502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc1cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc18071cb0 0x7fcc1806f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc1cc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc181568a0 0x7fcc180c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc1cc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc18168170 con 0x7fcc18071cb0 2024-09-10T21:17:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc1cc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcc18167ff0 con 0x7fcc181568a0 2024-09-10T21:17:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc1cc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fcc181682f0 con 0x7fcbfc004810 2024-09-10T21:17:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc0d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc18071cb0 0x7fcc1806f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc0d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc18071cb0 0x7fcc1806f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43824/0 (socket says 172.21.15.16:43824) 2024-09-10T21:17:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc0d400640 1 -- 172.21.15.16:0/1805438484 learned_addr learned my addr 172.21.15.16:0/1805438484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc15e00640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcbfc004810 0x7fcc181502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc16800640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc181568a0 0x7fcc180c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc0d400640 1 -- 172.21.15.16:0/1805438484 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcbfc004810 msgr2=0x7fcc181502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc0d400640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcbfc004810 0x7fcc181502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc0d400640 1 -- 172.21.15.16:0/1805438484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc181568a0 msgr2=0x7fcc180c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc0d400640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc181568a0 0x7fcc180c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc0d400640 1 -- 172.21.15.16:0/1805438484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1815d550 con 0x7fcc18071cb0 2024-09-10T21:17:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc16800640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc181568a0 0x7fcc180c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc0d400640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc18071cb0 0x7fcc1806f3c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fcc0001cd80 tx=0x7fcc00003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.637+0000 7fcc0ea00640 1 -- 172.21.15.16:0/1805438484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00001480 con 0x7fcc18071cb0 2024-09-10T21:17:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.641+0000 7fcc0ea00640 1 -- 172.21.15.16:0/1805438484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc000064c0 con 0x7fcc18071cb0 2024-09-10T21:17:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.641+0000 7fcc0ea00640 1 -- 172.21.15.16:0/1805438484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc000068d0 con 0x7fcc18071cb0 2024-09-10T21:17:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.641+0000 7fcc1cc00640 1 -- 172.21.15.16:0/1805438484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1815ccb0 con 0x7fcc18071cb0 2024-09-10T21:17:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.641+0000 7fcc1cc00640 1 -- 172.21.15.16:0/1805438484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1815d270 con 0x7fcc18071cb0 2024-09-10T21:17:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.641+0000 7fcc1cc00640 1 -- 172.21.15.16:0/1805438484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1815f940 con 0x7fcc18071cb0 2024-09-10T21:17:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.641+0000 7fcc0ea00640 1 -- 172.21.15.16:0/1805438484 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcc000a5780 con 0x7fcc18071cb0 2024-09-10T21:17:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.641+0000 7fcc0ea00640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbf40725e0 0x7fcbf4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:50.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.641+0000 7fcc0ea00640 1 -- 172.21.15.16:0/1805438484 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fcc0012d3d0 con 0x7fcc18071cb0 2024-09-10T21:17:50.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.641+0000 7fcc15e00640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbf40725e0 0x7fcbf4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:50.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.641+0000 7fcc15e00640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbf40725e0 0x7fcbf4074aa0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fcc08001ec0 tx=0x7fcc08000bd0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:50.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.645+0000 7fcc0ea00640 1 -- 172.21.15.16:0/1805438484 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcc1815d550 con 0x7fcc18071cb0 2024-09-10T21:17:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.821+0000 7fcc1cc00640 1 -- 172.21.15.16:0/1805438484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7fcc1815b520 con 0x7fcc18071cb0 2024-09-10T21:17:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.821+0000 7fcc0ea00640 1 -- 172.21.15.16:0/1805438484 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v56) ==== 106+0+4 (secure 0 0 0) 0x7fcc00016650 con 0x7fcc18071cb0 2024-09-10T21:17:50.825 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-10T21:17:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.825+0000 7fcc1cc00640 1 -- 172.21.15.16:0/1805438484 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbf40725e0 msgr2=0x7fcbf4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.825+0000 7fcc1cc00640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbf40725e0 0x7fcbf4074aa0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fcc08001ec0 tx=0x7fcc08000bd0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.825+0000 7fcc1cc00640 1 -- 172.21.15.16:0/1805438484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc18071cb0 msgr2=0x7fcc1806f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.825+0000 7fcc1cc00640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc18071cb0 0x7fcc1806f3c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fcc0001cd80 tx=0x7fcc00003bb0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.825+0000 7fcc1cc00640 1 -- 172.21.15.16:0/1805438484 shutdown_connections 2024-09-10T21:17:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.825+0000 7fcc1cc00640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbf40725e0 0x7fcbf4074aa0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.825+0000 7fcc1cc00640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc181568a0 0x7fcc180c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.825+0000 7fcc1cc00640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc18071cb0 0x7fcc1806f3c0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.825+0000 7fcc1cc00640 1 --2- 172.21.15.16:0/1805438484 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcbfc004810 0x7fcc181502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.825+0000 7fcc1cc00640 1 -- 172.21.15.16:0/1805438484 >> 172.21.15.16:0/1805438484 conn(0x7fcc180d1ce0 msgr2=0x7fcc180cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.825+0000 7fcc1cc00640 1 -- 172.21.15.16:0/1805438484 shutdown_connections 2024-09-10T21:17:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.825+0000 7fcc1cc00640 1 -- 172.21.15.16:0/1805438484 wait complete. 2024-09-10T21:17:50.847 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-09-10T21:17:50.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.993+0000 7fca9ea00640 1 -- 172.21.15.16:0/4024439997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca980cd300 msgr2=0x7fca980d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:50.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.993+0000 7fca9ea00640 1 --2- 172.21.15.16:0/4024439997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca980cd300 0x7fca980d2800 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fca98073000 tx=0x7fca8001c7a0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.993+0000 7fca9ea00640 1 -- 172.21.15.16:0/4024439997 shutdown_connections 2024-09-10T21:17:50.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.993+0000 7fca9ea00640 1 --2- 172.21.15.16:0/4024439997 >> v2:172.21.15.52:3300/0 conn(0x7fca9815f940 0x7fca9815fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.993+0000 7fca9ea00640 1 --2- 172.21.15.16:0/4024439997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca980cd300 0x7fca980d2800 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:50.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.993+0000 7fca9ea00640 1 -- 172.21.15.16:0/4024439997 >> 172.21.15.16:0/4024439997 conn(0x7fca980d1ce0 msgr2=0x7fca980d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:50.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.993+0000 7fca9ea00640 1 -- 172.21.15.16:0/4024439997 shutdown_connections 2024-09-10T21:17:50.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.993+0000 7fca9ea00640 1 -- 172.21.15.16:0/4024439997 wait complete. 2024-09-10T21:17:50.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.993+0000 7fca9ea00640 1 Processor -- start 2024-09-10T21:17:50.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.993+0000 7fca9ea00640 1 -- start start 2024-09-10T21:17:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca9ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca980cd300 0x7fca98147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca9ea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca9815bdc0 0x7fca980c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca9ea00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fca9815f940 0x7fca9815b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca9ea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca98169010 con 0x7fca980cd300 2024-09-10T21:17:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca9ea00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fca98168e90 con 0x7fca9815bdc0 2024-09-10T21:17:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca9ea00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fca98169190 con 0x7fca9815f940 2024-09-10T21:17:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca97e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca980cd300 0x7fca98147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca97e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca980cd300 0x7fca98147130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43838/0 (socket says 172.21.15.16:43838) 2024-09-10T21:17:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca97e00640 1 -- 172.21.15.16:0/2740414652 learned_addr learned my addr 172.21.15.16:0/2740414652 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca8fe00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca9815bdc0 0x7fca980c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca9ca00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fca9815f940 0x7fca9815b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca97e00640 1 -- 172.21.15.16:0/2740414652 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fca9815f940 msgr2=0x7fca9815b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca97e00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fca9815f940 0x7fca9815b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca97e00640 1 -- 172.21.15.16:0/2740414652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca9815bdc0 msgr2=0x7fca980c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca97e00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca9815bdc0 0x7fca980c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca97e00640 1 -- 172.21.15.16:0/2740414652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca98155d40 con 0x7fca980cd300 2024-09-10T21:17:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca9ca00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fca9815f940 0x7fca9815b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca97e00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca980cd300 0x7fca98147130 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fca8800c500 tx=0x7fca8800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca94c00640 1 -- 172.21.15.16:0/2740414652 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88017070 con 0x7fca980cd300 2024-09-10T21:17:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca94c00640 1 -- 172.21.15.16:0/2740414652 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca880053c0 con 0x7fca980cd300 2024-09-10T21:17:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca94c00640 1 -- 172.21.15.16:0/2740414652 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88003860 con 0x7fca980cd300 2024-09-10T21:17:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca9ea00640 1 -- 172.21.15.16:0/2740414652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca98156020 con 0x7fca980cd300 2024-09-10T21:17:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca9ea00640 1 -- 172.21.15.16:0/2740414652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca980ca0b0 con 0x7fca980cd300 2024-09-10T21:17:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca9ea00640 1 -- 172.21.15.16:0/2740414652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca980d2800 con 0x7fca980cd300 2024-09-10T21:17:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:50.997+0000 7fca94c00640 1 -- 172.21.15.16:0/2740414652 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fca88003a00 con 0x7fca980cd300 2024-09-10T21:17:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.001+0000 7fca94c00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fca740725e0 0x7fca74074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.001+0000 7fca94c00640 1 -- 172.21.15.16:0/2740414652 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fca880928c0 con 0x7fca980cd300 2024-09-10T21:17:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.001+0000 7fca8fe00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fca740725e0 0x7fca74074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.001+0000 7fca8fe00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fca740725e0 0x7fca74074aa0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fca800af000 tx=0x7fca8001afe0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:51.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.005+0000 7fca94c00640 1 -- 172.21.15.16:0/2740414652 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fca8805d8c0 con 0x7fca980cd300 2024-09-10T21:17:51.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.177+0000 7fca9ea00640 1 -- 172.21.15.16:0/2740414652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7fca980704a0 con 0x7fca980cd300 2024-09-10T21:17:51.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.181+0000 7fca94c00640 1 -- 172.21.15.16:0/2740414652 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v57)=0 v57) ==== 124+0+0 (secure 0 0 0) 0x7fca880622f0 con 0x7fca980cd300 2024-09-10T21:17:51.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.185+0000 7fca9ea00640 1 -- 172.21.15.16:0/2740414652 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fca740725e0 msgr2=0x7fca74074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.185+0000 7fca9ea00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fca740725e0 0x7fca74074aa0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fca800af000 tx=0x7fca8001afe0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.185+0000 7fca9ea00640 1 -- 172.21.15.16:0/2740414652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca980cd300 msgr2=0x7fca98147130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.185+0000 7fca9ea00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca980cd300 0x7fca98147130 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fca8800c500 tx=0x7fca8800c9c0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.185+0000 7fca9ea00640 1 -- 172.21.15.16:0/2740414652 shutdown_connections 2024-09-10T21:17:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.185+0000 7fca9ea00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fca740725e0 0x7fca74074aa0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.185+0000 7fca9ea00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fca9815f940 0x7fca9815b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.185+0000 7fca9ea00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca9815bdc0 0x7fca980c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.185+0000 7fca9ea00640 1 --2- 172.21.15.16:0/2740414652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca980cd300 0x7fca98147130 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.185+0000 7fca9ea00640 1 -- 172.21.15.16:0/2740414652 >> 172.21.15.16:0/2740414652 conn(0x7fca980d1ce0 msgr2=0x7fca980b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.185+0000 7fca9ea00640 1 -- 172.21.15.16:0/2740414652 shutdown_connections 2024-09-10T21:17:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.185+0000 7fca9ea00640 1 -- 172.21.15.16:0/2740414652 wait complete. 2024-09-10T21:17:51.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-09-10T21:17:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.353+0000 7f0844800640 1 -- 172.21.15.16:0/792683978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c158600 msgr2=0x7f083c15f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.353+0000 7f0844800640 1 --2- 172.21.15.16:0/792683978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c158600 0x7f083c15f110 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f083c0d1cb0 tx=0x7f083001c550 comp rx=0 tx=0).stop 2024-09-10T21:17:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.353+0000 7f0844800640 1 -- 172.21.15.16:0/792683978 shutdown_connections 2024-09-10T21:17:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.353+0000 7f0844800640 1 --2- 172.21.15.16:0/792683978 >> v2:172.21.15.52:3300/0 conn(0x7f083c0cc9e0 0x7f083c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.353+0000 7f0844800640 1 --2- 172.21.15.16:0/792683978 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f083c0ce1c0 0x7f083c0cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.353+0000 7f0844800640 1 --2- 172.21.15.16:0/792683978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c158600 0x7f083c15f110 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.353+0000 7f0844800640 1 -- 172.21.15.16:0/792683978 >> 172.21.15.16:0/792683978 conn(0x7f083c0cd6a0 msgr2=0x7f083c0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.353+0000 7f0844800640 1 -- 172.21.15.16:0/792683978 shutdown_connections 2024-09-10T21:17:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.353+0000 7f0844800640 1 -- 172.21.15.16:0/792683978 wait complete. 2024-09-10T21:17:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0844800640 1 Processor -- start 2024-09-10T21:17:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0844800640 1 -- start start 2024-09-10T21:17:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0844800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0cc9e0 0x7f083c156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0844800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f083c0ce1c0 0x7f083c15c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0844800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f083c158600 0x7f083c14c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0844800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f083c070d70 con 0x7f083c0cc9e0 2024-09-10T21:17:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0844800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f083c070bf0 con 0x7f083c0ce1c0 2024-09-10T21:17:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0844800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f083c070ef0 con 0x7f083c158600 2024-09-10T21:17:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0841e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0cc9e0 0x7f083c156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0841e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0cc9e0 0x7f083c156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43852/0 (socket says 172.21.15.16:43852) 2024-09-10T21:17:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0841e00640 1 -- 172.21.15.16:0/1707427176 learned_addr learned my addr 172.21.15.16:0/1707427176 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0841400640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f083c0ce1c0 0x7f083c15c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0842800640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f083c158600 0x7f083c14c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0841e00640 1 -- 172.21.15.16:0/1707427176 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f083c158600 msgr2=0x7f083c14c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0841e00640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f083c158600 0x7f083c14c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0841e00640 1 -- 172.21.15.16:0/1707427176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f083c0ce1c0 msgr2=0x7f083c15c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0841e00640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f083c0ce1c0 0x7f083c15c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0841e00640 1 -- 172.21.15.16:0/1707427176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c156860 con 0x7f083c0cc9e0 2024-09-10T21:17:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0841e00640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0cc9e0 0x7f083c156280 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0830009850 tx=0x7f08300aa4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0836000640 1 -- 172.21.15.16:0/1707427176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300aab00 con 0x7f083c0cc9e0 2024-09-10T21:17:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0836000640 1 -- 172.21.15.16:0/1707427176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08300b8070 con 0x7f083c0cc9e0 2024-09-10T21:17:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0836000640 1 -- 172.21.15.16:0/1707427176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0830005020 con 0x7f083c0cc9e0 2024-09-10T21:17:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0844800640 1 -- 172.21.15.16:0/1707427176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c156b40 con 0x7f083c0cc9e0 2024-09-10T21:17:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0844800640 1 -- 172.21.15.16:0/1707427176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c0ce930 con 0x7f083c0cc9e0 2024-09-10T21:17:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.357+0000 7f0844800640 1 -- 172.21.15.16:0/1707427176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083c153a10 con 0x7f083c0cc9e0 2024-09-10T21:17:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.361+0000 7f0836000640 1 -- 172.21.15.16:0/1707427176 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f08300051e0 con 0x7f083c0cc9e0 2024-09-10T21:17:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.361+0000 7f0836000640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f08200725e0 0x7f0820074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.361+0000 7f0836000640 1 -- 172.21.15.16:0/1707427176 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f083012dae0 con 0x7f083c0cc9e0 2024-09-10T21:17:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.361+0000 7f0841400640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f08200725e0 0x7f0820074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.361+0000 7f0841400640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f08200725e0 0x7f0820074aa0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f082c0058d0 tx=0x7f082c005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.365+0000 7f0836000640 1 -- 172.21.15.16:0/1707427176 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0830016490 con 0x7f083c0cc9e0 2024-09-10T21:17:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.537+0000 7f0844800640 1 -- 172.21.15.16:0/1707427176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7f083c153730 con 0x7f083c0cc9e0 2024-09-10T21:17:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.541+0000 7f0836000640 1 -- 172.21.15.16:0/1707427176 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v57) ==== 106+0+8 (secure 0 0 0) 0x7f08300f9250 con 0x7f083c0cc9e0 2024-09-10T21:17:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:1048576 2024-09-10T21:17:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.541+0000 7f0844800640 1 -- 172.21.15.16:0/1707427176 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f08200725e0 msgr2=0x7f0820074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.541+0000 7f0844800640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f08200725e0 0x7f0820074aa0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f082c0058d0 tx=0x7f082c005860 comp rx=0 tx=0).stop 2024-09-10T21:17:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.541+0000 7f0844800640 1 -- 172.21.15.16:0/1707427176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0cc9e0 msgr2=0x7f083c156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.541+0000 7f0844800640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0cc9e0 0x7f083c156280 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0830009850 tx=0x7f08300aa4f0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.541+0000 7f0844800640 1 -- 172.21.15.16:0/1707427176 shutdown_connections 2024-09-10T21:17:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.541+0000 7f0844800640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f08200725e0 0x7f0820074aa0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.541+0000 7f0844800640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f083c158600 0x7f083c14c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.541+0000 7f0844800640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f083c0ce1c0 0x7f083c15c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.541+0000 7f0844800640 1 --2- 172.21.15.16:0/1707427176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c0cc9e0 0x7f083c156280 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.541+0000 7f0844800640 1 -- 172.21.15.16:0/1707427176 >> 172.21.15.16:0/1707427176 conn(0x7f083c0cd6a0 msgr2=0x7f083c1524d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.541+0000 7f0844800640 1 -- 172.21.15.16:0/1707427176 shutdown_connections 2024-09-10T21:17:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.541+0000 7f0844800640 1 -- 172.21.15.16:0/1707427176 wait complete. 2024-09-10T21:17:51.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-09-10T21:17:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 -- 172.21.15.16:0/3352020802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec004810 msgr2=0x7f72ec004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 --2- 172.21.15.16:0/3352020802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec004810 0x7f72ec004c90 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009b60 tx=0x7f72f001d270 comp rx=0 tx=0).stop 2024-09-10T21:17:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 -- 172.21.15.16:0/3352020802 shutdown_connections 2024-09-10T21:17:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 --2- 172.21.15.16:0/3352020802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec004810 0x7f72ec004c90 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 -- 172.21.15.16:0/3352020802 >> 172.21.15.16:0/3352020802 conn(0x7f73080d1df0 msgr2=0x7f73080d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 -- 172.21.15.16:0/3352020802 shutdown_connections 2024-09-10T21:17:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 -- 172.21.15.16:0/3352020802 wait complete. 2024-09-10T21:17:51.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 Processor -- start 2024-09-10T21:17:51.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 -- start start 2024-09-10T21:17:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec004810 0x7f73081f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73081f6f90 0x7f73080c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7308150fe0 0x7f73080b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73080b14f0 con 0x7f72ec004810 2024-09-10T21:17:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f73080d2b20 con 0x7f73081f6f90 2024-09-10T21:17:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f730da00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f73080ce590 con 0x7f7308150fe0 2024-09-10T21:17:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f7306a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec004810 0x7f73081f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f7306a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec004810 0x7f73081f3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43868/0 (socket says 172.21.15.16:43868) 2024-09-10T21:17:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f7306a00640 1 -- 172.21.15.16:0/3887861146 learned_addr learned my addr 172.21.15.16:0/3887861146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f7307400640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7308150fe0 0x7f73080b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f72ffe00640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73081f6f90 0x7f73080c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f7306a00640 1 -- 172.21.15.16:0/3887861146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7308150fe0 msgr2=0x7f73080b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f7306a00640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7308150fe0 0x7f73080b6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f7306a00640 1 -- 172.21.15.16:0/3887861146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73081f6f90 msgr2=0x7f73080c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f7306a00640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73081f6f90 0x7f73080c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.713+0000 7f7306a00640 1 -- 172.21.15.16:0/3887861146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7308151950 con 0x7f72ec004810 2024-09-10T21:17:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.717+0000 7f7306a00640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec004810 0x7f73081f3680 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f72f800e590 tx=0x7f72f8001830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.717+0000 7f72fea00640 1 -- 172.21.15.16:0/3887861146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f8015070 con 0x7f72ec004810 2024-09-10T21:17:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.717+0000 7f72fea00640 1 -- 172.21.15.16:0/3887861146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72f8010980 con 0x7f72ec004810 2024-09-10T21:17:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.717+0000 7f72fea00640 1 -- 172.21.15.16:0/3887861146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f8010dd0 con 0x7f72ec004810 2024-09-10T21:17:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.717+0000 7f730da00640 1 -- 172.21.15.16:0/3887861146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7308151c30 con 0x7f72ec004810 2024-09-10T21:17:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.717+0000 7f730da00640 1 -- 172.21.15.16:0/3887861146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73081f3160 con 0x7f72ec004810 2024-09-10T21:17:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.717+0000 7f730da00640 1 -- 172.21.15.16:0/3887861146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73080cf2b0 con 0x7f72ec004810 2024-09-10T21:17:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.717+0000 7f72fea00640 1 -- 172.21.15.16:0/3887861146 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f72f8011460 con 0x7f72ec004810 2024-09-10T21:17:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.717+0000 7f72fea00640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72e4072610 0x7f72e4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.717+0000 7f72fea00640 1 -- 172.21.15.16:0/3887861146 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f72f8093900 con 0x7f72ec004810 2024-09-10T21:17:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.717+0000 7f72ffe00640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72e4072610 0x7f72e4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.721+0000 7f72ffe00640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72e4072610 0x7f72e4074ad0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f72f00ae000 tx=0x7f72f001a010 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.721+0000 7f72fea00640 1 -- 172.21.15.16:0/3887861146 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f72f805e8f0 con 0x7f72ec004810 2024-09-10T21:17:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.897+0000 7f730da00640 1 -- 172.21.15.16:0/3887861146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7f73080b2440 con 0x7f72ec004810 2024-09-10T21:17:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.901+0000 7f72fea00640 1 -- 172.21.15.16:0/3887861146 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v57) ==== 112+0+10 (secure 0 0 0) 0x7f72f80930b0 con 0x7f72ec004810 2024-09-10T21:17:51.904 INFO:teuthology.orchestra.run.smithi016.stdout:60.000000 2024-09-10T21:17:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.901+0000 7f730da00640 1 -- 172.21.15.16:0/3887861146 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72e4072610 msgr2=0x7f72e4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.901+0000 7f730da00640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72e4072610 0x7f72e4074ad0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f72f00ae000 tx=0x7f72f001a010 comp rx=0 tx=0).stop 2024-09-10T21:17:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.901+0000 7f730da00640 1 -- 172.21.15.16:0/3887861146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec004810 msgr2=0x7f73081f3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.901+0000 7f730da00640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec004810 0x7f73081f3680 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f72f800e590 tx=0x7f72f8001830 comp rx=0 tx=0).stop 2024-09-10T21:17:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.901+0000 7f730da00640 1 -- 172.21.15.16:0/3887861146 shutdown_connections 2024-09-10T21:17:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.901+0000 7f730da00640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72e4072610 0x7f72e4074ad0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.901+0000 7f730da00640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7308150fe0 0x7f73080b6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.901+0000 7f730da00640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73081f6f90 0x7f73080c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.901+0000 7f730da00640 1 --2- 172.21.15.16:0/3887861146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec004810 0x7f73081f3680 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.901+0000 7f730da00640 1 -- 172.21.15.16:0/3887861146 >> 172.21.15.16:0/3887861146 conn(0x7f73080d1df0 msgr2=0x7f730815f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.901+0000 7f730da00640 1 -- 172.21.15.16:0/3887861146 shutdown_connections 2024-09-10T21:17:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:51.901+0000 7f730da00640 1 -- 172.21.15.16:0/3887861146 wait complete. 2024-09-10T21:17:51.926 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:17:51.926 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:51.936 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:17:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:17:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:17:52.040 INFO:teuthology.orchestra.run.smithi016.stdout:client7078 2024-09-10T21:17:52.041 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:17:52.041 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:17:52.050 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-10T21:17:52.050 DEBUG:teuthology.orchestra.run.smithi016:> import fcntl 2024-09-10T21:17:52.050 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2024-09-10T21:17:52.050 DEBUG:teuthology.orchestra.run.smithi016:> import time 2024-09-10T21:17:52.050 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-10T21:17:52.050 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:17:52.050 DEBUG:teuthology.orchestra.run.smithi016:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-10T21:17:52.050 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:17:52.050 DEBUG:teuthology.orchestra.run.smithi016:> n = 200 2024-09-10T21:17:52.050 DEBUG:teuthology.orchestra.run.smithi016:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> if not os.path.exists(abs_path): 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(abs_path) 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> handles = [] 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, n): 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> fname = "file_"+str(i) 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> path = os.path.join(abs_path, fname) 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> handles.append(open(path, '"'"'w'"'"')) 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> print("waiting with handles open", file=sys.stderr) 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> while True: 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> print("open_n_background: keeping files open", file=sys.stderr) 2024-09-10T21:17:52.051 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-10T21:17:52.052 DEBUG:teuthology.orchestra.run.smithi016:> if os.read(0, 4096) == b"": 2024-09-10T21:17:52.052 DEBUG:teuthology.orchestra.run.smithi016:> break 2024-09-10T21:17:52.052 DEBUG:teuthology.orchestra.run.smithi016:> except BlockingIOError: 2024-09-10T21:17:52.052 DEBUG:teuthology.orchestra.run.smithi016:> pass 2024-09-10T21:17:52.052 DEBUG:teuthology.orchestra.run.smithi016:> time.sleep(2) 2024-09-10T21:17:52.052 DEBUG:teuthology.orchestra.run.smithi016:> print("stdin closed, goodbye!", file=sys.stderr) 2024-09-10T21:17:52.052 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-10T21:17:52.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:17:52.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.257+0000 7f07c7c00640 1 -- 172.21.15.16:0/4242781163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c015ac50 msgr2=0x7f07c0073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:52.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.257+0000 7f07c7c00640 1 --2- 172.21.15.16:0/4242781163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c015ac50 0x7f07c0073750 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009840 tx=0x7f07b401c6b0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.257+0000 7f07c7c00640 1 -- 172.21.15.16:0/4242781163 shutdown_connections 2024-09-10T21:17:52.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.257+0000 7f07c7c00640 1 --2- 172.21.15.16:0/4242781163 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07c00d06e0 0x7f07c0073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.257+0000 7f07c7c00640 1 --2- 172.21.15.16:0/4242781163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c015ac50 0x7f07c0073750 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.257+0000 7f07c7c00640 1 -- 172.21.15.16:0/4242781163 >> 172.21.15.16:0/4242781163 conn(0x7f07c00d0c10 msgr2=0x7f07c00d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:52.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.257+0000 7f07c7c00640 1 -- 172.21.15.16:0/4242781163 shutdown_connections 2024-09-10T21:17:52.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.257+0000 7f07c7c00640 1 -- 172.21.15.16:0/4242781163 wait complete. 2024-09-10T21:17:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c7c00640 1 Processor -- start 2024-09-10T21:17:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c7c00640 1 -- start start 2024-09-10T21:17:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c7c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07c00cff40 0x7f07c0154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c7c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c00d06e0 0x7f07c00b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c7c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07c015ac50 0x7f07c0156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c7c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0167fa0 con 0x7f07c015ac50 2024-09-10T21:17:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c7c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0167e20 con 0x7f07c00d06e0 2024-09-10T21:17:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c7c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f07c0168120 con 0x7f07c00cff40 2024-09-10T21:17:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07c015ac50 0x7f07c0156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07c015ac50 0x7f07c0156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46302/0 (socket says 172.21.15.16:46302) 2024-09-10T21:17:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c5c00640 1 -- 172.21.15.16:0/1297872391 learned_addr learned my addr 172.21.15.16:0/1297872391 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c5200640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07c00cff40 0x7f07c0154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07bfe00640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c00d06e0 0x7f07c00b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c5c00640 1 -- 172.21.15.16:0/1297872391 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07c00cff40 msgr2=0x7f07c0154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c5c00640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07c00cff40 0x7f07c0154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c5c00640 1 -- 172.21.15.16:0/1297872391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c00d06e0 msgr2=0x7f07c00b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c5c00640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c00d06e0 0x7f07c00b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.261+0000 7f07c5c00640 1 -- 172.21.15.16:0/1297872391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c00b1f60 con 0x7f07c015ac50 2024-09-10T21:17:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.265+0000 7f07c5c00640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07c015ac50 0x7f07c0156c70 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f07b800cc20 tx=0x7f07b800cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.265+0000 7f07bcc00640 1 -- 172.21.15.16:0/1297872391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8016020 con 0x7f07c015ac50 2024-09-10T21:17:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.265+0000 7f07bcc00640 1 -- 172.21.15.16:0/1297872391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07b800ab60 con 0x7f07c015ac50 2024-09-10T21:17:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.265+0000 7f07c7c00640 1 -- 172.21.15.16:0/1297872391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c00b64e0 con 0x7f07c015ac50 2024-09-10T21:17:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.265+0000 7f07bcc00640 1 -- 172.21.15.16:0/1297872391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8012420 con 0x7f07c015ac50 2024-09-10T21:17:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.265+0000 7f07c7c00640 1 -- 172.21.15.16:0/1297872391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c0073e40 con 0x7f07c015ac50 2024-09-10T21:17:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.265+0000 7f07c7c00640 1 -- 172.21.15.16:0/1297872391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c00d18e0 con 0x7f07c015ac50 2024-09-10T21:17:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.265+0000 7f07bcc00640 1 -- 172.21.15.16:0/1297872391 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f07b80125c0 con 0x7f07c015ac50 2024-09-10T21:17:52.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.265+0000 7f07bcc00640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07a00725e0 0x7f07a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:52.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.265+0000 7f07bcc00640 1 -- 172.21.15.16:0/1297872391 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f07b80938a0 con 0x7f07c015ac50 2024-09-10T21:17:52.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.269+0000 7f07c5200640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07a00725e0 0x7f07a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:52.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.269+0000 7f07c5200640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07a00725e0 0x7f07a0074aa0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f07b40ae000 tx=0x7f07b401aef0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:52.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.273+0000 7f07bcc00640 1 -- 172.21.15.16:0/1297872391 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f07b8020020 con 0x7f07c015ac50 2024-09-10T21:17:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:waiting with handles open 2024-09-10T21:17:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:17:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.497+0000 7f07c7c00640 1 -- 172.21.15.16:0/1297872391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07c0070d00 con 0x7f07c015ac50 2024-09-10T21:17:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.497+0000 7f07bcc00640 1 -- 172.21.15.16:0/1297872391 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5229 (secure 0 0 0) 0x7f07b805e890 con 0x7f07c015ac50 2024-09-10T21:17:52.502 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:52.502 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"btime":"2024-09-10T21:17:44:118059+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3961362430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3961362430},{"type":"v1","addr":"172.21.15.52:6834","nonce":3961362430}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6967,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/831111367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":831111367},{"type":"v1","addr":"172.21.15.52:6835","nonce":831111367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7467,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/552929965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":552929965},{"type":"v1","addr":"172.21.15.16:6836","nonce":552929965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:40.510614+0000","modified":"2024-09-10T21:17:43.521144+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":136181},"failed":[],"damaged":[],"stopped":[],"info":{"gid_136181":{"gid":136181,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6837/1681945537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1681945537},{"type":"v1","addr":"172.21.15.16:6837","nonce":1681945537}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":136181,"qdb_cluster":[136181]},"id":9}]} 2024-09-10T21:17:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 82 2024-09-10T21:17:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.501+0000 7f07c7c00640 1 -- 172.21.15.16:0/1297872391 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07a00725e0 msgr2=0x7f07a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.501+0000 7f07c7c00640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07a00725e0 0x7f07a0074aa0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f07b40ae000 tx=0x7f07b401aef0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.501+0000 7f07c7c00640 1 -- 172.21.15.16:0/1297872391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07c015ac50 msgr2=0x7f07c0156c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.501+0000 7f07c7c00640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07c015ac50 0x7f07c0156c70 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f07b800cc20 tx=0x7f07b800cd10 comp rx=0 tx=0).stop 2024-09-10T21:17:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.501+0000 7f07c7c00640 1 -- 172.21.15.16:0/1297872391 shutdown_connections 2024-09-10T21:17:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.501+0000 7f07c7c00640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f07a00725e0 0x7f07a0074aa0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.501+0000 7f07c7c00640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07c015ac50 0x7f07c0156c70 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.501+0000 7f07c7c00640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c00d06e0 0x7f07c00b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.501+0000 7f07c7c00640 1 --2- 172.21.15.16:0/1297872391 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f07c00cff40 0x7f07c0154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.501+0000 7f07c7c00640 1 -- 172.21.15.16:0/1297872391 >> 172.21.15.16:0/1297872391 conn(0x7f07c00d0c10 msgr2=0x7f07c006d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.501+0000 7f07c7c00640 1 -- 172.21.15.16:0/1297872391 shutdown_connections 2024-09-10T21:17:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.501+0000 7f07c7c00640 1 -- 172.21.15.16:0/1297872391 wait complete. 2024-09-10T21:17:52.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-10T21:17:52.690 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":136286,"entity":{"name":{"type":"client","num":136286},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2804955592}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":3.4079438190000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9897830997782715,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.136286 192.168.0.1:0/2804955592","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi052","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":7078,"entity":{"name":{"type":"client","num":7078},"addr":{"type":"any","addr":"172.21.15.16:0","nonce":1241559650}},"state":"open","num_leases":1,"num_caps":202,"request_load_avg":225,"uptime":4.5479249570000002,"requests_in_flight":200,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":404.6176475925804,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7078 172.21.15.16:0/1241559650","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-10T21:17:52.691 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi052", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2804955592, "type": "any" }, "name": { "num": 136286, "type": "client" } }, "id": 136286, "inst": "client.136286 192.168.0.1:0/2804955592", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9897830997782715 }, "state": "open", "uptime": 3.407943819 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 1241559650, "type": "any" }, "name": { "num": 7078, "type": "client" } }, "id": 7078, "inst": "client.7078 172.21.15.16:0/1241559650", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 202, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 225, "requests_in_flight": 200, "session_cache_liveness": { "halflife": 300, "value": 404.6176475925804 }, "state": "open", "uptime": 4.547924957 } ] 2024-09-10T21:17:52.691 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-09-10T21:17:52.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:17:52.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.833+0000 7f0947400640 1 -- 172.21.15.16:0/1365695139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0940154f90 msgr2=0x7f0940072d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:52.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.833+0000 7f0947400640 1 --2- 172.21.15.16:0/1365695139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0940154f90 0x7f0940072d40 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f0934009840 tx=0x7f093401c7f0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.833+0000 7f0947400640 1 -- 172.21.15.16:0/1365695139 shutdown_connections 2024-09-10T21:17:52.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.833+0000 7f0947400640 1 --2- 172.21.15.16:0/1365695139 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f09400706f0 0x7f09400ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.833+0000 7f0947400640 1 --2- 172.21.15.16:0/1365695139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0940154f90 0x7f0940072d40 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.833+0000 7f0947400640 1 -- 172.21.15.16:0/1365695139 >> 172.21.15.16:0/1365695139 conn(0x7f09400cfb20 msgr2=0x7f09400cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:52.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.833+0000 7f0947400640 1 -- 172.21.15.16:0/1365695139 shutdown_connections 2024-09-10T21:17:52.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.833+0000 7f0947400640 1 -- 172.21.15.16:0/1365695139 wait complete. 2024-09-10T21:17:52.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.833+0000 7f0947400640 1 Processor -- start 2024-09-10T21:17:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.833+0000 7f0947400640 1 -- start start 2024-09-10T21:17:52.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0947400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09400706f0 0x7f09401452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:52.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0947400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0940154f90 0x7f09400c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0947400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f094015bc70 0x7f0940149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0947400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0940167980 con 0x7f09400706f0 2024-09-10T21:17:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0947400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09400cb6f0 con 0x7f0940154f90 2024-09-10T21:17:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0947400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0940167b00 con 0x7f094015bc70 2024-09-10T21:17:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0944a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09400706f0 0x7f09401452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0944a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09400706f0 0x7f09401452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46312/0 (socket says 172.21.15.16:46312) 2024-09-10T21:17:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0944a00640 1 -- 172.21.15.16:0/915438697 learned_addr learned my addr 172.21.15.16:0/915438697 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0945400640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f094015bc70 0x7f0940149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f093fe00640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0940154f90 0x7f09400c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0944a00640 1 -- 172.21.15.16:0/915438697 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f094015bc70 msgr2=0x7f0940149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0944a00640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f094015bc70 0x7f0940149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0944a00640 1 -- 172.21.15.16:0/915438697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0940154f90 msgr2=0x7f09400c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0944a00640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0940154f90 0x7f09400c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0944a00640 1 -- 172.21.15.16:0/915438697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09401457f0 con 0x7f09400706f0 2024-09-10T21:17:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0944a00640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09400706f0 0x7f09401452b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f093401d090 tx=0x7f09340a57e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f093cc00640 1 -- 172.21.15.16:0/915438697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09340b2040 con 0x7f09400706f0 2024-09-10T21:17:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f093cc00640 1 -- 172.21.15.16:0/915438697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0934004430 con 0x7f09400706f0 2024-09-10T21:17:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f093cc00640 1 -- 172.21.15.16:0/915438697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934003800 con 0x7f09400706f0 2024-09-10T21:17:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0947400640 1 -- 172.21.15.16:0/915438697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0940152270 con 0x7f09400706f0 2024-09-10T21:17:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0947400640 1 -- 172.21.15.16:0/915438697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09400b5560 con 0x7f09400706f0 2024-09-10T21:17:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f0947400640 1 -- 172.21.15.16:0/915438697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09400ca530 con 0x7f09400706f0 2024-09-10T21:17:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.837+0000 7f093cc00640 1 -- 172.21.15.16:0/915438697 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f09340b5070 con 0x7f09400706f0 2024-09-10T21:17:52.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.841+0000 7f093cc00640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09200725e0 0x7f0920074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:52.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.841+0000 7f093cc00640 1 -- 172.21.15.16:0/915438697 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f093412cc80 con 0x7f09400706f0 2024-09-10T21:17:52.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.841+0000 7f093fe00640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09200725e0 0x7f0920074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.841+0000 7f093fe00640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09200725e0 0x7f0920074aa0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f09400c9ee0 tx=0x7f092c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:52.845+0000 7f093cc00640 1 -- 172.21.15.16:0/915438697 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f09401457f0 con 0x7f09400706f0 2024-09-10T21:17:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.061+0000 7f0947400640 1 -- 172.21.15.16:0/915438697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f09400c3b90 con 0x7f09400706f0 2024-09-10T21:17:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.061+0000 7f093cc00640 1 -- 172.21.15.16:0/915438697 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0934016410 con 0x7f09400706f0 2024-09-10T21:17:53.067 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:53.067 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T21:17:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.065+0000 7f0947400640 1 -- 172.21.15.16:0/915438697 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09200725e0 msgr2=0x7f0920074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.065+0000 7f0947400640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09200725e0 0x7f0920074aa0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f09400c9ee0 tx=0x7f092c009040 comp rx=0 tx=0).stop 2024-09-10T21:17:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.065+0000 7f0947400640 1 -- 172.21.15.16:0/915438697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09400706f0 msgr2=0x7f09401452b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.065+0000 7f0947400640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09400706f0 0x7f09401452b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f093401d090 tx=0x7f09340a57e0 comp rx=0 tx=0).stop 2024-09-10T21:17:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.065+0000 7f0947400640 1 -- 172.21.15.16:0/915438697 shutdown_connections 2024-09-10T21:17:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.065+0000 7f0947400640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f09200725e0 0x7f0920074aa0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.065+0000 7f0947400640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f094015bc70 0x7f0940149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.065+0000 7f0947400640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0940154f90 0x7f09400c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.065+0000 7f0947400640 1 --2- 172.21.15.16:0/915438697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09400706f0 0x7f09401452b0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.065+0000 7f0947400640 1 -- 172.21.15.16:0/915438697 >> 172.21.15.16:0/915438697 conn(0x7f09400cfb20 msgr2=0x7f094006ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.065+0000 7f0947400640 1 -- 172.21.15.16:0/915438697 shutdown_connections 2024-09-10T21:17:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:53.065+0000 7f0947400640 1 -- 172.21.15.16:0/915438697 wait complete. 2024-09-10T21:17:53.089 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-10T21:17:53.089 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:17:54.110 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:17:54.114 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:17:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:17:56.386 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:17:58.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:17:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.245+0000 7fabbec00640 1 -- 172.21.15.16:0/15816146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab9c004d50 msgr2=0x7fab9c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.245+0000 7fabbec00640 1 --2- 172.21.15.16:0/15816146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab9c004d50 0x7fab9c0051d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fabb000f490 tx=0x7fabb001e410 comp rx=0 tx=0).stop 2024-09-10T21:17:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.245+0000 7fabbcc00640 1 -- 172.21.15.16:0/15816146 reap_dead start 2024-09-10T21:17:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.245+0000 7fabbec00640 1 -- 172.21.15.16:0/15816146 shutdown_connections 2024-09-10T21:17:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.245+0000 7fabbec00640 1 -- 172.21.15.16:0/15816146 >> 172.21.15.16:0/15816146 conn(0x7fabb80cfb20 msgr2=0x7fabb80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.245+0000 7fabbec00640 1 -- 172.21.15.16:0/15816146 shutdown_connections 2024-09-10T21:17:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.245+0000 7fabbec00640 1 -- 172.21.15.16:0/15816146 wait complete. 2024-09-10T21:17:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.245+0000 7fabbec00640 1 Processor -- start 2024-09-10T21:17:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.245+0000 7fabbec00640 1 -- start start 2024-09-10T21:17:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabbec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab9c004d50 0x7fabb80b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabbec00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fabb81e6d30 0x7fabb80b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabbec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabb8155d70 0x7fabb8157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabbec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabb80ca940 con 0x7fab9c004d50 2024-09-10T21:17:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabbec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fabb80ca7c0 con 0x7fabb8155d70 2024-09-10T21:17:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabbec00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fabb806e640 con 0x7fabb81e6d30 2024-09-10T21:17:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabb7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab9c004d50 0x7fabb80b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabb7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab9c004d50 0x7fabb80b53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46322/0 (socket says 172.21.15.16:46322) 2024-09-10T21:17:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabb7e00640 1 -- 172.21.15.16:0/1107541456 learned_addr learned my addr 172.21.15.16:0/1107541456 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabaf400640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fabb81e6d30 0x7fabb80b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabbcc00640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabb8155d70 0x7fabb8157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabb7e00640 1 -- 172.21.15.16:0/1107541456 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fabb81e6d30 msgr2=0x7fabb80b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabb7e00640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fabb81e6d30 0x7fabb80b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabb7e00640 1 -- 172.21.15.16:0/1107541456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabb8155d70 msgr2=0x7fabb8157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabb7e00640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabb8155d70 0x7fabb8157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabb7e00640 1 -- 172.21.15.16:0/1107541456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb81e7230 con 0x7fab9c004d50 2024-09-10T21:17:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabb7e00640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab9c004d50 0x7fabb80b53c0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7faba8006450 tx=0x7faba80039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabb4c00640 1 -- 172.21.15.16:0/1107541456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8015070 con 0x7fab9c004d50 2024-09-10T21:17:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabb4c00640 1 -- 172.21.15.16:0/1107541456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faba8011900 con 0x7fab9c004d50 2024-09-10T21:17:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabb4c00640 1 -- 172.21.15.16:0/1107541456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8011d50 con 0x7fab9c004d50 2024-09-10T21:17:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabbec00640 1 -- 172.21.15.16:0/1107541456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb81f8980 con 0x7fab9c004d50 2024-09-10T21:17:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabbec00640 1 -- 172.21.15.16:0/1107541456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb81e9880 con 0x7fab9c004d50 2024-09-10T21:17:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabbec00640 1 -- 172.21.15.16:0/1107541456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb81e9a80 con 0x7fab9c004d50 2024-09-10T21:17:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.249+0000 7fabb4c00640 1 -- 172.21.15.16:0/1107541456 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faba80103c0 con 0x7fab9c004d50 2024-09-10T21:17:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.253+0000 7fabb4c00640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fab94072610 0x7fab94074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.253+0000 7fabb4c00640 1 -- 172.21.15.16:0/1107541456 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7faba8094280 con 0x7fab9c004d50 2024-09-10T21:17:58.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.253+0000 7fabaf400640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fab94072610 0x7fab94074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.253+0000 7fabaf400640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fab94072610 0x7fab94074ad0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fabb81eadd0 tx=0x7faba0006b20 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:58.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.257+0000 7fabb4c00640 1 -- 172.21.15.16:0/1107541456 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fabb81e9a80 con 0x7fab9c004d50 2024-09-10T21:17:58.387 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:17:58.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabbec00640 1 -- 172.21.15.16:0/1107541456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fabb806eba0 con 0x7fab9c004d50 2024-09-10T21:17:58.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabb4c00640 1 -- 172.21.15.16:0/1107541456 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7faba805f8c0 con 0x7fab9c004d50 2024-09-10T21:17:58.469 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:58.469 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:17:58.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabbec00640 1 -- 172.21.15.16:0/1107541456 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fab94072610 msgr2=0x7fab94074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabbec00640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fab94072610 0x7fab94074ad0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fabb81eadd0 tx=0x7faba0006b20 comp rx=0 tx=0).stop 2024-09-10T21:17:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabbec00640 1 -- 172.21.15.16:0/1107541456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab9c004d50 msgr2=0x7fabb80b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabbec00640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab9c004d50 0x7fabb80b53c0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7faba8006450 tx=0x7faba80039d0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabbec00640 1 -- 172.21.15.16:0/1107541456 shutdown_connections 2024-09-10T21:17:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabbec00640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fab94072610 0x7fab94074ad0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabbec00640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabb8155d70 0x7fabb8157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabbec00640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fabb81e6d30 0x7fabb80b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabbec00640 1 --2- 172.21.15.16:0/1107541456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab9c004d50 0x7fabb80b53c0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabbec00640 1 -- 172.21.15.16:0/1107541456 >> 172.21.15.16:0/1107541456 conn(0x7fabb80cfb20 msgr2=0x7fabb8072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabbec00640 1 -- 172.21.15.16:0/1107541456 shutdown_connections 2024-09-10T21:17:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.465+0000 7fabbec00640 1 -- 172.21.15.16:0/1107541456 wait complete. 2024-09-10T21:17:58.490 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:17:58.490 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-09-10T21:17:58.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:17:58.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 -- 172.21.15.16:0/46966636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9d815bfc0 msgr2=0x7fb9d80d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 --2- 172.21.15.16:0/46966636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9d815bfc0 0x7fb9d80d0020 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009900 tx=0x7fb9c001c850 comp rx=0 tx=0).stop 2024-09-10T21:17:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 -- 172.21.15.16:0/46966636 shutdown_connections 2024-09-10T21:17:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 --2- 172.21.15.16:0/46966636 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb9d80cc5e0 0x7fb9d80d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 --2- 172.21.15.16:0/46966636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9d815bfc0 0x7fb9d80d0020 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 -- 172.21.15.16:0/46966636 >> 172.21.15.16:0/46966636 conn(0x7fb9d8074c50 msgr2=0x7fb9d809a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 -- 172.21.15.16:0/46966636 shutdown_connections 2024-09-10T21:17:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 -- 172.21.15.16:0/46966636 wait complete. 2024-09-10T21:17:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 Processor -- start 2024-09-10T21:17:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 -- start start 2024-09-10T21:17:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9d80cc5e0 0x7fb9d80ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9d80cfc40 0x7fb9d81facf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb9d815bfc0 0x7fb9d8165370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d8168b30 con 0x7fb9d80cc5e0 2024-09-10T21:17:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81689b0 con 0x7fb9d80cfc40 2024-09-10T21:17:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d7200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d8168cb0 con 0x7fb9d815bfc0 2024-09-10T21:17:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9d80cc5e0 0x7fb9d80ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9d80cc5e0 0x7fb9d80ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46346/0 (socket says 172.21.15.16:46346) 2024-09-10T21:17:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d5e00640 1 -- 172.21.15.16:0/797986724 learned_addr learned my addr 172.21.15.16:0/797986724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:17:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9cfe00640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9d80cfc40 0x7fb9d81facf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d6800640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb9d815bfc0 0x7fb9d8165370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d5e00640 1 -- 172.21.15.16:0/797986724 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb9d815bfc0 msgr2=0x7fb9d8165370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d5e00640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb9d815bfc0 0x7fb9d8165370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d5e00640 1 -- 172.21.15.16:0/797986724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9d80cfc40 msgr2=0x7fb9d81facf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d5e00640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9d80cfc40 0x7fb9d81facf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d5e00640 1 -- 172.21.15.16:0/797986724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d8072630 con 0x7fb9d80cc5e0 2024-09-10T21:17:58.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.633+0000 7fb9d6800640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb9d815bfc0 0x7fb9d8165370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:58.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.637+0000 7fb9d5e00640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9d80cc5e0 0x7fb9d80ae730 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800c400 tx=0x7fb9c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:58.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.637+0000 7fb9ce000640 1 -- 172.21.15.16:0/797986724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8017070 con 0x7fb9d80cc5e0 2024-09-10T21:17:58.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.637+0000 7fb9ce000640 1 -- 172.21.15.16:0/797986724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9c80053c0 con 0x7fb9d80cc5e0 2024-09-10T21:17:58.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.637+0000 7fb9d7200640 1 -- 172.21.15.16:0/797986724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d81fb420 con 0x7fb9d80cc5e0 2024-09-10T21:17:58.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.637+0000 7fb9ce000640 1 -- 172.21.15.16:0/797986724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8003860 con 0x7fb9d80cc5e0 2024-09-10T21:17:58.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.637+0000 7fb9d7200640 1 -- 172.21.15.16:0/797986724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d81fb9e0 con 0x7fb9d80cc5e0 2024-09-10T21:17:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.637+0000 7fb9d7200640 1 -- 172.21.15.16:0/797986724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9d80d1cb0 con 0x7fb9d80cc5e0 2024-09-10T21:17:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.637+0000 7fb9ce000640 1 -- 172.21.15.16:0/797986724 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb9c8003a00 con 0x7fb9d80cc5e0 2024-09-10T21:17:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.637+0000 7fb9ce000640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9b4072610 0x7fb9b4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.641+0000 7fb9ce000640 1 -- 172.21.15.16:0/797986724 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fb9c80928a0 con 0x7fb9d80cc5e0 2024-09-10T21:17:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.641+0000 7fb9cfe00640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9b4072610 0x7fb9b4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.641+0000 7fb9cfe00640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9b4072610 0x7fb9b4074ad0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00af000 tx=0x7fb9c00195f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.645+0000 7fb9ce000640 1 -- 172.21.15.16:0/797986724 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb9c805d890 con 0x7fb9d80cc5e0 2024-09-10T21:17:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.857+0000 7fb9d7200640 1 -- 172.21.15.16:0/797986724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb9d8071970 con 0x7fb9d80cc5e0 2024-09-10T21:17:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.857+0000 7fb9ce000640 1 -- 172.21.15.16:0/797986724 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7fb9c80622c0 con 0x7fb9d80cc5e0 2024-09-10T21:17:58.862 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:17:58.862 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:17:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.857+0000 7fb9d7200640 1 -- 172.21.15.16:0/797986724 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9b4072610 msgr2=0x7fb9b4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.857+0000 7fb9d7200640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9b4072610 0x7fb9b4074ad0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00af000 tx=0x7fb9c00195f0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.861+0000 7fb9d7200640 1 -- 172.21.15.16:0/797986724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9d80cc5e0 msgr2=0x7fb9d80ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.861+0000 7fb9d7200640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9d80cc5e0 0x7fb9d80ae730 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800c400 tx=0x7fb9c800c8c0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.861+0000 7fb9d7200640 1 -- 172.21.15.16:0/797986724 shutdown_connections 2024-09-10T21:17:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.861+0000 7fb9d7200640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb9b4072610 0x7fb9b4074ad0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.861+0000 7fb9d7200640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb9d815bfc0 0x7fb9d8165370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.861+0000 7fb9d7200640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9d80cfc40 0x7fb9d81facf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.861+0000 7fb9d7200640 1 --2- 172.21.15.16:0/797986724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9d80cc5e0 0x7fb9d80ae730 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.861+0000 7fb9d7200640 1 -- 172.21.15.16:0/797986724 >> 172.21.15.16:0/797986724 conn(0x7fb9d8074c50 msgr2=0x7fb9d8157f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.861+0000 7fb9d7200640 1 -- 172.21.15.16:0/797986724 shutdown_connections 2024-09-10T21:17:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58.861+0000 7fb9d7200640 1 -- 172.21.15.16:0/797986724 wait complete. 2024-09-10T21:17:58.883 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:17:58.883 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-09-10T21:17:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:17:58 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-10T21:18:00.390 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:18:00.390 INFO:teuthology.orchestra.run.smithi016.stderr:stdin closed, goodbye! 2024-09-10T21:18:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:18:00.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:00.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.605+0000 7fba7f200640 1 -- 172.21.15.16:0/2051476898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba64004810 msgr2=0x7fba64004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:00.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.605+0000 7fba7f200640 1 --2- 172.21.15.16:0/2051476898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba64004810 0x7fba64004c90 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fba80076730 tx=0x7fba6801d1f0 comp rx=0 tx=0).stop 2024-09-10T21:18:00.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.605+0000 7fba7e800640 1 -- 172.21.15.16:0/2051476898 reap_dead start 2024-09-10T21:18:00.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.605+0000 7fba7f200640 1 -- 172.21.15.16:0/2051476898 shutdown_connections 2024-09-10T21:18:00.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.605+0000 7fba7f200640 1 -- 172.21.15.16:0/2051476898 >> 172.21.15.16:0/2051476898 conn(0x7fba800cd5f0 msgr2=0x7fba800cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:00.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.605+0000 7fba7f200640 1 -- 172.21.15.16:0/2051476898 shutdown_connections 2024-09-10T21:18:00.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.605+0000 7fba7f200640 1 -- 172.21.15.16:0/2051476898 wait complete. 2024-09-10T21:18:00.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7f200640 1 Processor -- start 2024-09-10T21:18:00.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7f200640 1 -- start start 2024-09-10T21:18:00.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7f200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba64004810 0x7fba800ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:00.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba80155ac0 0x7fba801552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba801f9540 0x7fba801f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba80165bb0 con 0x7fba801f9540 2024-09-10T21:18:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fba80165a30 con 0x7fba80155ac0 2024-09-10T21:18:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7f200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fba80165d30 con 0x7fba64004810 2024-09-10T21:18:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba801f9540 0x7fba801f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba801f9540 0x7fba801f99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46352/0 (socket says 172.21.15.16:46352) 2024-09-10T21:18:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7e800640 1 -- 172.21.15.16:0/2480080520 learned_addr learned my addr 172.21.15.16:0/2480080520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7de00640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba64004810 0x7fba800ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba75400640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba80155ac0 0x7fba801552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7e800640 1 -- 172.21.15.16:0/2480080520 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba64004810 msgr2=0x7fba800ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7e800640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba64004810 0x7fba800ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7e800640 1 -- 172.21.15.16:0/2480080520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba80155ac0 msgr2=0x7fba801552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7e800640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba80155ac0 0x7fba801552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7e800640 1 -- 172.21.15.16:0/2480080520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba800c7e00 con 0x7fba801f9540 2024-09-10T21:18:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba75400640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba80155ac0 0x7fba801552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7e800640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba801f9540 0x7fba801f99c0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fba80072d00 tx=0x7fba7800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba76a00640 1 -- 172.21.15.16:0/2480080520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78015020 con 0x7fba801f9540 2024-09-10T21:18:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba76a00640 1 -- 172.21.15.16:0/2480080520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba7800cab0 con 0x7fba801f9540 2024-09-10T21:18:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7f200640 1 -- 172.21.15.16:0/2480080520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba800c80e0 con 0x7fba801f9540 2024-09-10T21:18:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba76a00640 1 -- 172.21.15.16:0/2480080520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78011460 con 0x7fba801f9540 2024-09-10T21:18:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.609+0000 7fba7f200640 1 -- 172.21.15.16:0/2480080520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba800c9180 con 0x7fba801f9540 2024-09-10T21:18:00.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.613+0000 7fba7f200640 1 -- 172.21.15.16:0/2480080520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba801f8e10 con 0x7fba801f9540 2024-09-10T21:18:00.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.613+0000 7fba76a00640 1 -- 172.21.15.16:0/2480080520 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fba78011600 con 0x7fba801f9540 2024-09-10T21:18:00.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.613+0000 7fba76a00640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba5c0725e0 0x7fba5c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:00.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.613+0000 7fba76a00640 1 -- 172.21.15.16:0/2480080520 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fba78093a10 con 0x7fba801f9540 2024-09-10T21:18:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.613+0000 7fba7de00640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba5c0725e0 0x7fba5c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.613+0000 7fba7de00640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba5c0725e0 0x7fba5c074aa0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fba70009ce0 tx=0x7fba700091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.617+0000 7fba76a00640 1 -- 172.21.15.16:0/2480080520 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fba7801f020 con 0x7fba801f9540 2024-09-10T21:18:00.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.825+0000 7fba7f200640 1 -- 172.21.15.16:0/2480080520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba8015be90 con 0x7fba801f9540 2024-09-10T21:18:00.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.825+0000 7fba76a00640 1 -- 172.21.15.16:0/2480080520 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5229 (secure 0 0 0) 0x7fba7805f0e0 con 0x7fba801f9540 2024-09-10T21:18:00.831 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:00.831 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"btime":"2024-09-10T21:17:57:759836+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3961362430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3961362430},{"type":"v1","addr":"172.21.15.52:6834","nonce":3961362430}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6967,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/831111367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":831111367},{"type":"v1","addr":"172.21.15.52:6835","nonce":831111367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7467,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/552929965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":552929965},{"type":"v1","addr":"172.21.15.16:6836","nonce":552929965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:40.510614+0000","modified":"2024-09-10T21:17:57.134218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":136181},"failed":[],"damaged":[],"stopped":[],"info":{"gid_136181":{"gid":136181,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":7,"addr":"172.21.15.16:6837/1681945537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1681945537},{"type":"v1","addr":"172.21.15.16:6837","nonce":1681945537}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":136181,"qdb_cluster":[136181]},"id":9}]} 2024-09-10T21:18:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 84 2024-09-10T21:18:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.829+0000 7fba7f200640 1 -- 172.21.15.16:0/2480080520 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba5c0725e0 msgr2=0x7fba5c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.829+0000 7fba7f200640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba5c0725e0 0x7fba5c074aa0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fba70009ce0 tx=0x7fba700091e0 comp rx=0 tx=0).stop 2024-09-10T21:18:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.829+0000 7fba7f200640 1 -- 172.21.15.16:0/2480080520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba801f9540 msgr2=0x7fba801f99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.829+0000 7fba7f200640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba801f9540 0x7fba801f99c0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fba80072d00 tx=0x7fba7800ec30 comp rx=0 tx=0).stop 2024-09-10T21:18:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.829+0000 7fba7f200640 1 -- 172.21.15.16:0/2480080520 shutdown_connections 2024-09-10T21:18:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.829+0000 7fba7f200640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba5c0725e0 0x7fba5c074aa0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.829+0000 7fba7f200640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba801f9540 0x7fba801f99c0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.829+0000 7fba7f200640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba80155ac0 0x7fba801552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.829+0000 7fba7f200640 1 --2- 172.21.15.16:0/2480080520 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba64004810 0x7fba800ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.829+0000 7fba7f200640 1 -- 172.21.15.16:0/2480080520 >> 172.21.15.16:0/2480080520 conn(0x7fba800cd5f0 msgr2=0x7fba8015b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:00.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.829+0000 7fba7f200640 1 -- 172.21.15.16:0/2480080520 shutdown_connections 2024-09-10T21:18:00.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:00.829+0000 7fba7f200640 1 -- 172.21.15.16:0/2480080520 wait complete. 2024-09-10T21:18:00.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-10T21:18:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":136286,"entity":{"name":{"type":"client","num":136286},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2804955592}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":11.735807796,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9328052594986138,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.136286 192.168.0.1:0/2804955592","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi052","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":7078,"entity":{"name":{"type":"client","num":7078},"addr":{"type":"any","addr":"172.21.15.16:0","nonce":1241559650}},"state":"open","num_leases":1,"num_caps":202,"request_load_avg":205,"uptime":12.875788933999999,"requests_in_flight":0,"num_completed_requests":200,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":760.65410895510695,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":174.31293016743408,"halflife":1.5},"recall_caps_throttle2o":{"value":38.663902541412632,"halflife":0.5},"session_cache_liveness":{"value":396.90664016178368,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7078 172.21.15.16:0/1241559650","completed_requests":[{"tid":29,"created_ino":"0x10000000001"},{"tid":30,"created_ino":"0x100000001f6"},{"tid":31,"created_ino":"0x100000001f7"},{"tid":32,"created_ino":"0x100000001f8"},{"tid":33,"created_ino":"0x100000001f9"},{"tid":34,"created_ino":"0x100000001fa"},{"tid":35,"created_ino":"0x100000001fb"},{"tid":36,"created_ino":"0x100000001fc"},{"tid":37,"created_ino":"0x100000001fd"},{"tid":38,"created_ino":"0x100000001fe"},{"tid":39,"created_ino":"0x100000001ff"},{"tid":40,"created_ino":"0x10000000200"},{"tid":41,"created_ino":"0x10000000201"},{"tid":42,"created_ino":"0x10000000202"},{"tid":43,"created_ino":"0x10000000203"},{"tid":44,"created_ino":"0x10000000204"},{"tid":45,"created_ino":"0x10000000205"},{"tid":46,"created_ino":"0x10000000206"},{"tid":47,"created_ino":"0x10000000207"},{"tid":48,"created_ino":"0x10000000208"},{"tid":49,"created_ino":"0x10000000209"},{"tid":50,"created_ino":"0x1000000020a"},{"tid":51,"created_ino":"0x1000000020b"},{"tid":52,"created_ino":"0x1000000020c"},{"tid":53,"created_ino":"0x1000000020d"},{"tid":54,"created_ino":"0x1000000020e"},{"tid":55,"created_ino":"0x1000000020f"},{"tid":56,"created_ino":"0x10000000210"},{"tid":57,"created_ino":"0x10000000211"},{"tid":58,"created_ino":"0x10000000212"},{"tid":59,"created_ino":"0x10000000213"},{"tid":60,"created_ino":"0x10000000214"},{"tid":61,"created_ino":"0x10000000215"},{"tid":62,"created_ino":"0x10000000216"},{"tid":63,"created_ino":"0x10000000217"},{"tid":64,"created_ino":"0x10000000218"},{"tid":65,"created_ino":"0x10000000219"},{"tid":66,"created_ino":"0x1000000021a"},{"tid":67,"created_ino":"0x1000000021b"},{"tid":68,"created_ino":"0x1000000021c"},{"tid":69,"created_ino":"0x1000000021d"},{"tid":70,"created_ino":"0x1000000021e"},{"tid":71,"created_ino":"0x1000000021f"},{"tid":72,"created_ino":"0x10000000220"},{"tid":73,"created_ino":"0x10000000221"},{"tid":74,"created_ino":"0x10000000222"},{"tid":75,"created_ino":"0x10000000223"},{"tid":76,"created_ino":"0x10000000224"},{"tid":77,"created_ino":"0x10000000225"},{"tid":78,"created_ino":"0x10000000226"},{"tid":79,"created_ino":"0x10000000227"},{"tid":80,"created_ino":"0x10000000228"},{"tid":81,"created_ino":"0x10000000229"},{"tid":82,"created_ino":"0x1000000022a"},{"tid":83,"created_ino":"0x1000000022b"},{"tid":84,"created_ino":"0x1000000022c"},{"tid":85,"created_ino":"0x1000000022d"},{"tid":86,"created_ino":"0x1000000022e"},{"tid":87,"created_ino":"0x1000000022f"},{"tid":88,"created_ino":"0x10000000230"},{"tid":89,"created_ino":"0x10000000231"},{"tid":90,"created_ino":"0x10000000232"},{"tid":91,"created_ino":"0x10000000233"},{"tid":92,"created_ino":"0x10000000234"},{"tid":93,"created_ino":"0x10000000235"},{"tid":94,"created_ino":"0x10000000236"},{"tid":95,"created_ino":"0x10000000237"},{"tid":96,"created_ino":"0x10000000238"},{"tid":97,"created_ino":"0x10000000239"},{"tid":98,"created_ino":"0x1000000023a"},{"tid":99,"created_ino":"0x1000000023b"},{"tid":100,"created_ino":"0x1000000023c"},{"tid":101,"created_ino":"0x1000000023d"},{"tid":102,"created_ino":"0x1000000023e"},{"tid":103,"created_ino":"0x1000000023f"},{"tid":104,"created_ino":"0x10000000240"},{"tid":105,"created_ino":"0x10000000241"},{"tid":106,"created_ino":"0x10000000242"},{"tid":107,"created_ino":"0x10000000243"},{"tid":108,"created_ino":"0x10000000244"},{"tid":109,"created_ino":"0x10000000245"},{"tid":110,"created_ino":"0x10000000246"},{"tid":111,"created_ino":"0x10000000247"},{"tid":112,"created_ino":"0x10000000248"},{"tid":113,"created_ino":"0x10000000249"},{"tid":114,"created_ino":"0x1000000024a"},{"tid":115,"created_ino":"0x1000000024b"},{"tid":116,"created_ino":"0x1000000024c"},{"tid":117,"created_ino":"0x1000000024d"},{"tid":118,"created_ino":"0x1000000024e"},{"tid":119,"created_ino":"0x1000000024f"},{"tid":120,"created_ino":"0x10000000250"},{"tid":121,"created_ino":"0x10000000251"},{"tid":122,"created_ino":"0x10000000252"},{"tid":123,"created_ino":"0x10000000253"},{"tid":124,"created_ino":"0x10000000254"},{"tid":125,"created_ino":"0x10000000255"},{"tid":126,"created_ino":"0x10000000256"},{"tid":127,"created_ino":"0x10000000257"},{"tid":128,"created_ino":"0x10000000258"},{"tid":129,"created_ino":"0x10000000259"},{"tid":130,"created_ino":"0x1000000025a"},{"tid":131,"created_ino":"0x1000000025b"},{"tid":132,"created_ino":"0x1000000025c"},{"tid":133,"created_ino":"0x1000000025d"},{"tid":134,"created_ino":"0x1000000025e"},{"tid":135,"created_ino":"0x1000000025f"},{"tid":136,"created_ino":"0x10000000260"},{"tid":137,"created_ino":"0x10000000261"},{"tid":138,"created_ino":"0x10000000262"},{"tid":139,"created_ino":"0x10000000263"},{"tid":140,"created_ino":"0x10000000264"},{"tid":141,"created_ino":"0x10000000265"},{"tid":142,"created_ino":"0x10000000266"},{"tid":143,"created_ino":"0x10000000267"},{"tid":144,"created_ino":"0x10000000268"},{"tid":145,"created_ino":"0x10000000269"},{"tid":146,"created_ino":"0x1000000026a"},{"tid":147,"created_ino":"0x1000000026b"},{"tid":148,"created_ino":"0x1000000026c"},{"tid":149,"created_ino":"0x1000000026d"},{"tid":150,"created_ino":"0x1000000026e"},{"tid":151,"created_ino":"0x1000000026f"},{"tid":152,"created_ino":"0x10000000270"},{"tid":153,"created_ino":"0x10000000271"},{"tid":154,"created_ino":"0x10000000272"},{"tid":155,"created_ino":"0x10000000273"},{"tid":156,"created_ino":"0x10000000274"},{"tid":157,"created_ino":"0x10000000275"},{"tid":158,"created_ino":"0x10000000276"},{"tid":159,"created_ino":"0x10000000277"},{"tid":160,"created_ino":"0x10000000278"},{"tid":161,"created_ino":"0x10000000279"},{"tid":162,"created_ino":"0x1000000027a"},{"tid":163,"created_ino":"0x1000000027b"},{"tid":164,"created_ino":"0x1000000027c"},{"tid":165,"created_ino":"0x1000000027d"},{"tid":166,"created_ino":"0x1000000027e"},{"tid":167,"created_ino":"0x1000000027f"},{"tid":168,"created_ino":"0x10000000280"},{"tid":169,"created_ino":"0x10000000281"},{"tid":170,"created_ino":"0x10000000282"},{"tid":171,"created_ino":"0x10000000283"},{"tid":172,"created_ino":"0x10000000284"},{"tid":173,"created_ino":"0x10000000285"},{"tid":174,"created_ino":"0x10000000286"},{"tid":175,"created_ino":"0x10000000287"},{"tid":176,"created_ino":"0x10000000288"},{"tid":177,"created_ino":"0x10000000289"},{"tid":178,"created_ino":"0x1000000028a"},{"tid":179,"created_ino":"0x1000000028b"},{"tid":180,"created_ino":"0x1000000028c"},{"tid":181,"created_ino":"0x1000000028d"},{"tid":182,"created_ino":"0x1000000028e"},{"tid":183,"created_ino":"0x1000000028f"},{"tid":184,"created_ino":"0x10000000290"},{"tid":185,"created_ino":"0x10000000291"},{"tid":186,"created_ino":"0x10000000292"},{"tid":187,"created_ino":"0x10000000293"},{"tid":188,"created_ino":"0x10000000294"},{"tid":189,"created_ino":"0x10000000295"},{"tid":190,"created_ino":"0x10000000296"},{"tid":191,"created_ino":"0x10000000297"},{"tid":192,"created_ino":"0x10000000298"},{"tid":193,"created_ino":"0x10000000299"},{"tid":194,"created_ino":"0x1000000029a"},{"tid":195,"created_ino":"0x1000000029b"},{"tid":196,"created_ino":"0x1000000029c"},{"tid":197,"created_ino":"0x1000000029d"},{"tid":198,"created_ino":"0x1000000029e"},{"tid":199,"created_ino":"0x1000000029f"},{"tid":200,"created_ino":"0x100000002a0"},{"tid":201,"created_ino":"0x100000002a1"},{"tid":202,"created_ino":"0x100000002a2"},{"tid":203,"created_ino":"0x100000002a3"},{"tid":204,"created_ino":"0x100000002a4"},{"tid":205,"created_ino":"0x100000002a5"},{"tid":206,"created_ino":"0x100000002a6"},{"tid":207,"created_ino":"0x100000002a7"},{"tid":208,"created_ino":"0x100000002a8"},{"tid":209,"created_ino":"0x100000002a9"},{"tid":210,"created_ino":"0x100000002aa"},{"tid":211,"created_ino":"0x100000002ab"},{"tid":212,"created_ino":"0x100000002ac"},{"tid":213,"created_ino":"0x100000002ad"},{"tid":214,"created_ino":"0x100000002ae"},{"tid":215,"created_ino":"0x100000002af"},{"tid":216,"created_ino":"0x100000002b0"},{"tid":217,"created_ino":"0x100000002b1"},{"tid":218,"created_ino":"0x100000002b2"},{"tid":219,"created_ino":"0x100000002b3"},{"tid":220,"created_ino":"0x100000002b4"},{"tid":221,"created_ino":"0x100000002b5"},{"tid":222,"created_ino":"0x100000002b6"},{"tid":223,"created_ino":"0x100000002b7"},{"tid":224,"created_ino":"0x100000002b8"},{"tid":225,"created_ino":"0x100000002b9"},{"tid":226,"created_ino":"0x100000002ba"},{"tid":227,"created_ino":"0x100000002bb"},{"tid":228,"created_ino":"0x100000002bc"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000002bd","length":802}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-10T21:18:01.020 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi052", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2804955592, "type": "any" }, "name": { "num": 136286, "type": "client" } }, "id": 136286, "inst": "client.136286 192.168.0.1:0/2804955592", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.932805259498614 }, "state": "open", "uptime": 11.735807796 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 29 }, { "created_ino": "0x100000001f6", "tid": 30 }, { "created_ino": "0x100000001f7", "tid": 31 }, { "created_ino": "0x100000001f8", "tid": 32 }, { "created_ino": "0x100000001f9", "tid": 33 }, { "created_ino": "0x100000001fa", "tid": 34 }, { "created_ino": "0x100000001fb", "tid": 35 }, { "created_ino": "0x100000001fc", "tid": 36 }, { "created_ino": "0x100000001fd", "tid": 37 }, { "created_ino": "0x100000001fe", "tid": 38 }, { "created_ino": "0x100000001ff", "tid": 39 }, { "created_ino": "0x10000000200", "tid": 40 }, { "created_ino": "0x10000000201", "tid": 41 }, { "created_ino": "0x10000000202", "tid": 42 }, { "created_ino": "0x10000000203", "tid": 43 }, { "created_ino": "0x10000000204", "tid": 44 }, { "created_ino": "0x10000000205", "tid": 45 }, { "created_ino": "0x10000000206", "tid": 46 }, { "created_ino": "0x10000000207", "tid": 47 }, { "created_ino": "0x10000000208", "tid": 48 }, { "created_ino": "0x10000000209", "tid": 49 }, { "created_ino": "0x1000000020a", "tid": 50 }, { "created_ino": "0x1000000020b", "tid": 51 }, { "created_ino": "0x1000000020c", "tid": 52 }, { "created_ino": "0x1000000020d", "tid": 53 }, { "created_ino": "0x1000000020e", "tid": 54 }, { "created_ino": "0x1000000020f", "tid": 55 }, { "created_ino": "0x10000000210", "tid": 56 }, { "created_ino": "0x10000000211", "tid": 57 }, { "created_ino": "0x10000000212", "tid": 58 }, { "created_ino": "0x10000000213", "tid": 59 }, { "created_ino": "0x10000000214", "tid": 60 }, { "created_ino": "0x10000000215", "tid": 61 }, { "created_ino": "0x10000000216", "tid": 62 }, { "created_ino": "0x10000000217", "tid": 63 }, { "created_ino": "0x10000000218", "tid": 64 }, { "created_ino": "0x10000000219", "tid": 65 }, { "created_ino": "0x1000000021a", "tid": 66 }, { "created_ino": "0x1000000021b", "tid": 67 }, { "created_ino": "0x1000000021c", "tid": 68 }, { "created_ino": "0x1000000021d", "tid": 69 }, { "created_ino": "0x1000000021e", "tid": 70 }, { "created_ino": "0x1000000021f", "tid": 71 }, { "created_ino": "0x10000000220", "tid": 72 }, { "created_ino": "0x10000000221", "tid": 73 }, { "created_ino": "0x10000000222", "tid": 74 }, { "created_ino": "0x10000000223", "tid": 75 }, { "created_ino": "0x10000000224", "tid": 76 }, { "created_ino": "0x10000000225", "tid": 77 }, { "created_ino": "0x10000000226", "tid": 78 }, { "created_ino": "0x10000000227", "tid": 79 }, { "created_ino": "0x10000000228", "tid": 80 }, { "created_ino": "0x10000000229", "tid": 81 }, { "created_ino": "0x1000000022a", "tid": 82 }, { "created_ino": "0x1000000022b", "tid": 83 }, { "created_ino": "0x1000000022c", "tid": 84 }, { "created_ino": "0x1000000022d", "tid": 85 }, { "created_ino": "0x1000000022e", "tid": 86 }, { "created_ino": "0x1000000022f", "tid": 87 }, { "created_ino": "0x10000000230", "tid": 88 }, { "created_ino": "0x10000000231", "tid": 89 }, { "created_ino": "0x10000000232", "tid": 90 }, { "created_ino": "0x10000000233", "tid": 91 }, { "created_ino": "0x10000000234", "tid": 92 }, { "created_ino": "0x10000000235", "tid": 93 }, { "created_ino": "0x10000000236", "tid": 94 }, { "created_ino": "0x10000000237", "tid": 95 }, { "created_ino": "0x10000000238", "tid": 96 }, { "created_ino": "0x10000000239", "tid": 97 }, { "created_ino": "0x1000000023a", "tid": 98 }, { "created_ino": "0x1000000023b", "tid": 99 }, { "created_ino": "0x1000000023c", "tid": 100 }, { "created_ino": "0x1000000023d", "tid": 101 }, { "created_ino": "0x1000000023e", "tid": 102 }, { "created_ino": "0x1000000023f", "tid": 103 }, { "created_ino": "0x10000000240", "tid": 104 }, { "created_ino": "0x10000000241", "tid": 105 }, { "created_ino": "0x10000000242", "tid": 106 }, { "created_ino": "0x10000000243", "tid": 107 }, { "created_ino": "0x10000000244", "tid": 108 }, { "created_ino": "0x10000000245", "tid": 109 }, { "created_ino": "0x10000000246", "tid": 110 }, { "created_ino": "0x10000000247", "tid": 111 }, { "created_ino": "0x10000000248", "tid": 112 }, { "created_ino": "0x10000000249", "tid": 113 }, { "created_ino": "0x1000000024a", "tid": 114 }, { "created_ino": "0x1000000024b", "tid": 115 }, { "created_ino": "0x1000000024c", "tid": 116 }, { "created_ino": "0x1000000024d", "tid": 117 }, { "created_ino": "0x1000000024e", "tid": 118 }, { "created_ino": "0x1000000024f", "tid": 119 }, { "created_ino": "0x10000000250", "tid": 120 }, { "created_ino": "0x10000000251", "tid": 121 }, { "created_ino": "0x10000000252", "tid": 122 }, { "created_ino": "0x10000000253", "tid": 123 }, { "created_ino": "0x10000000254", "tid": 124 }, { "created_ino": "0x10000000255", "tid": 125 }, { "created_ino": "0x10000000256", "tid": 126 }, { "created_ino": "0x10000000257", "tid": 127 }, { "created_ino": "0x10000000258", "tid": 128 }, { "created_ino": "0x10000000259", "tid": 129 }, { "created_ino": "0x1000000025a", "tid": 130 }, { "created_ino": "0x1000000025b", "tid": 131 }, { "created_ino": "0x1000000025c", "tid": 132 }, { "created_ino": "0x1000000025d", "tid": 133 }, { "created_ino": "0x1000000025e", "tid": 134 }, { "created_ino": "0x1000000025f", "tid": 135 }, { "created_ino": "0x10000000260", "tid": 136 }, { "created_ino": "0x10000000261", "tid": 137 }, { "created_ino": "0x10000000262", "tid": 138 }, { "created_ino": "0x10000000263", "tid": 139 }, { "created_ino": "0x10000000264", "tid": 140 }, { "created_ino": "0x10000000265", "tid": 141 }, { "created_ino": "0x10000000266", "tid": 142 }, { "created_ino": "0x10000000267", "tid": 143 }, { "created_ino": "0x10000000268", "tid": 144 }, { "created_ino": "0x10000000269", "tid": 145 }, { "created_ino": "0x1000000026a", "tid": 146 }, { "created_ino": "0x1000000026b", "tid": 147 }, { "created_ino": "0x1000000026c", "tid": 148 }, { "created_ino": "0x1000000026d", "tid": 149 }, { "created_ino": "0x1000000026e", "tid": 150 }, { "created_ino": "0x1000000026f", "tid": 151 }, { "created_ino": "0x10000000270", "tid": 152 }, { "created_ino": "0x10000000271", "tid": 153 }, { "created_ino": "0x10000000272", "tid": 154 }, { "created_ino": "0x10000000273", "tid": 155 }, { "created_ino": "0x10000000274", "tid": 156 }, { "created_ino": "0x10000000275", "tid": 157 }, { "created_ino": "0x10000000276", "tid": 158 }, { "created_ino": "0x10000000277", "tid": 159 }, { "created_ino": "0x10000000278", "tid": 160 }, { "created_ino": "0x10000000279", "tid": 161 }, { "created_ino": "0x1000000027a", "tid": 162 }, { "created_ino": "0x1000000027b", "tid": 163 }, { "created_ino": "0x1000000027c", "tid": 164 }, { "created_ino": "0x1000000027d", "tid": 165 }, { "created_ino": "0x1000000027e", "tid": 166 }, { "created_ino": "0x1000000027f", "tid": 167 }, { "created_ino": "0x10000000280", "tid": 168 }, { "created_ino": "0x10000000281", "tid": 169 }, { "created_ino": "0x10000000282", "tid": 170 }, { "created_ino": "0x10000000283", "tid": 171 }, { "created_ino": "0x10000000284", "tid": 172 }, { "created_ino": "0x10000000285", "tid": 173 }, { "created_ino": "0x10000000286", "tid": 174 }, { "created_ino": "0x10000000287", "tid": 175 }, { "created_ino": "0x10000000288", "tid": 176 }, { "created_ino": "0x10000000289", "tid": 177 }, { "created_ino": "0x1000000028a", "tid": 178 }, { "created_ino": "0x1000000028b", "tid": 179 }, { "created_ino": "0x1000000028c", "tid": 180 }, { "created_ino": "0x1000000028d", "tid": 181 }, { "created_ino": "0x1000000028e", "tid": 182 }, { "created_ino": "0x1000000028f", "tid": 183 }, { "created_ino": "0x10000000290", "tid": 184 }, { "created_ino": "0x10000000291", "tid": 185 }, { "created_ino": "0x10000000292", "tid": 186 }, { "created_ino": "0x10000000293", "tid": 187 }, { "created_ino": "0x10000000294", "tid": 188 }, { "created_ino": "0x10000000295", "tid": 189 }, { "created_ino": "0x10000000296", "tid": 190 }, { "created_ino": "0x10000000297", "tid": 191 }, { "created_ino": "0x10000000298", "tid": 192 }, { "created_ino": "0x10000000299", "tid": 193 }, { "created_ino": "0x1000000029a", "tid": 194 }, { "created_ino": "0x1000000029b", "tid": 195 }, { "created_ino": "0x1000000029c", "tid": 196 }, { "created_ino": "0x1000000029d", "tid": 197 }, { "created_ino": "0x1000000029e", "tid": 198 }, { "created_ino": "0x1000000029f", "tid": 199 }, { "created_ino": "0x100000002a0", "tid": 200 }, { "created_ino": "0x100000002a1", "tid": 201 }, { "created_ino": "0x100000002a2", "tid": 202 }, { "created_ino": "0x100000002a3", "tid": 203 }, { "created_ino": "0x100000002a4", "tid": 204 }, { "created_ino": "0x100000002a5", "tid": 205 }, { "created_ino": "0x100000002a6", "tid": 206 }, { "created_ino": "0x100000002a7", "tid": 207 }, { "created_ino": "0x100000002a8", "tid": 208 }, { "created_ino": "0x100000002a9", "tid": 209 }, { "created_ino": "0x100000002aa", "tid": 210 }, { "created_ino": "0x100000002ab", "tid": 211 }, { "created_ino": "0x100000002ac", "tid": 212 }, { "created_ino": "0x100000002ad", "tid": 213 }, { "created_ino": "0x100000002ae", "tid": 214 }, { "created_ino": "0x100000002af", "tid": 215 }, { "created_ino": "0x100000002b0", "tid": 216 }, { "created_ino": "0x100000002b1", "tid": 217 }, { "created_ino": "0x100000002b2", "tid": 218 }, { "created_ino": "0x100000002b3", "tid": 219 }, { "created_ino": "0x100000002b4", "tid": 220 }, { "created_ino": "0x100000002b5", "tid": 221 }, { "created_ino": "0x100000002b6", "tid": 222 }, { "created_ino": "0x100000002b7", "tid": 223 }, { "created_ino": "0x100000002b8", "tid": 224 }, { "created_ino": "0x100000002b9", "tid": 225 }, { "created_ino": "0x100000002ba", "tid": 226 }, { "created_ino": "0x100000002bb", "tid": 227 }, { "created_ino": "0x100000002bc", "tid": 228 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 1241559650, "type": "any" }, "name": { "num": 7078, "type": "client" } }, "id": 7078, "inst": "client.7078 172.21.15.16:0/1241559650", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 202, "num_completed_flushes": 0, "num_completed_requests": 200, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 802, "start": "0x100000002bd" } ], "recall_caps": { "halflife": 60, "value": 760.654108955107 }, "recall_caps_throttle": { "halflife": 1.5, "value": 174.31293016743408 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 38.66390254141263 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 205, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 396.9066401617837 }, "state": "open", "uptime": 12.875788934 } ] 2024-09-10T21:18:01.021 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:18:01.021 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:18:01.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:18:01.115 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:18:01.182 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:18:01.182 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:18:01.183 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 200\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi016') 2024-09-10T21:18:01.183 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:18:01.183 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:01.192 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:18:01.192 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:01.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:18:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 -- 172.21.15.16:0/4020724763 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f57e00cffe0 msgr2=0x7f57e00ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 --2- 172.21.15.16:0/4020724763 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f57e00cffe0 0x7f57e00ce2b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009900 tx=0x7f57d401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 -- 172.21.15.16:0/4020724763 shutdown_connections 2024-09-10T21:18:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 --2- 172.21.15.16:0/4020724763 >> v2:172.21.15.52:3300/0 conn(0x7f57e00b2860 0x7f57e0160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 --2- 172.21.15.16:0/4020724763 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f57e00cffe0 0x7f57e00ce2b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 -- 172.21.15.16:0/4020724763 >> 172.21.15.16:0/4020724763 conn(0x7f57e00d1d80 msgr2=0x7f57e00d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 -- 172.21.15.16:0/4020724763 shutdown_connections 2024-09-10T21:18:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 -- 172.21.15.16:0/4020724763 wait complete. 2024-09-10T21:18:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 Processor -- start 2024-09-10T21:18:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 -- start start 2024-09-10T21:18:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f57e00b2860 0x7f57e00b7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e00cffe0 0x7f57e00b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e00d03c0 0x7f57e015e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57e00cd8e0 con 0x7f57e00cffe0 2024-09-10T21:18:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f57e00cd760 con 0x7f57e00d03c0 2024-09-10T21:18:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e8200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f57e00cda60 con 0x7f57e00b2860 2024-09-10T21:18:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e00cffe0 0x7f57e00b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e5800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f57e00b2860 0x7f57e00b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e00cffe0 0x7f57e00b66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46364/0 (socket says 172.21.15.16:46364) 2024-09-10T21:18:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e4e00640 1 -- 172.21.15.16:0/4255759236 learned_addr learned my addr 172.21.15.16:0/4255759236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e6200640 1 --2- 172.21.15.16:0/4255759236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e00d03c0 0x7f57e015e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e4e00640 1 -- 172.21.15.16:0/4255759236 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f57e00b2860 msgr2=0x7f57e00b7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e4e00640 1 --2- 172.21.15.16:0/4255759236 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f57e00b2860 0x7f57e00b7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e4e00640 1 -- 172.21.15.16:0/4255759236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e00d03c0 msgr2=0x7f57e015e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e4e00640 1 --2- 172.21.15.16:0/4255759236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e00d03c0 0x7f57e015e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e4e00640 1 -- 172.21.15.16:0/4255759236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e00b6be0 con 0x7f57e00cffe0 2024-09-10T21:18:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.433+0000 7f57e4e00640 1 --2- 172.21.15.16:0/4255759236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e00cffe0 0x7f57e00b66a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00c3c0 tx=0x7f57cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.437+0000 7f57dd600640 1 -- 172.21.15.16:0/4255759236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0052d0 con 0x7f57e00cffe0 2024-09-10T21:18:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.437+0000 7f57dd600640 1 -- 172.21.15.16:0/4255759236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57cc007020 con 0x7f57e00cffe0 2024-09-10T21:18:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.437+0000 7f57e8200640 1 -- 172.21.15.16:0/4255759236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e0159600 con 0x7f57e00cffe0 2024-09-10T21:18:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.437+0000 7f57dd600640 1 -- 172.21.15.16:0/4255759236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc007470 con 0x7f57e00cffe0 2024-09-10T21:18:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.437+0000 7f57e8200640 1 -- 172.21.15.16:0/4255759236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e0159bc0 con 0x7f57e00cffe0 2024-09-10T21:18:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.437+0000 7f57e8200640 1 -- 172.21.15.16:0/4255759236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e00cf950 con 0x7f57e00cffe0 2024-09-10T21:18:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.437+0000 7f57dd600640 1 -- 172.21.15.16:0/4255759236 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f57cc0067e0 con 0x7f57e00cffe0 2024-09-10T21:18:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.437+0000 7f57dd600640 1 --2- 172.21.15.16:0/4255759236 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f57c00725e0 0x7f57c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.437+0000 7f57dd600640 1 -- 172.21.15.16:0/4255759236 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f57cc092700 con 0x7f57e00cffe0 2024-09-10T21:18:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.437+0000 7f57e5800640 1 --2- 172.21.15.16:0/4255759236 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f57c00725e0 0x7f57c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.441+0000 7f57e5800640 1 --2- 172.21.15.16:0/4255759236 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f57c00725e0 0x7f57c0074aa0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f57d40af000 tx=0x7f57d401b120 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.441+0000 7f57dd600640 1 -- 172.21.15.16:0/4255759236 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f57e00b6be0 con 0x7f57e00cffe0 2024-09-10T21:18:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.625+0000 7f57e8200640 1 -- 172.21.15.16:0/4255759236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f57e0160220 con 0x7f57e00cffe0 2024-09-10T21:18:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.629+0000 7f57dd600640 1 -- 172.21.15.16:0/4255759236 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v116) ==== 80+0+0 (secure 0 0 0) 0x7f57cc001090 con 0x7f57e00cffe0 2024-09-10T21:18:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:18:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.629+0000 7f57e8200640 1 -- 172.21.15.16:0/4255759236 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f57c00725e0 msgr2=0x7f57c0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.629+0000 7f57e8200640 1 --2- 172.21.15.16:0/4255759236 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f57c00725e0 0x7f57c0074aa0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f57d40af000 tx=0x7f57d401b120 comp rx=0 tx=0).stop 2024-09-10T21:18:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.629+0000 7f57e8200640 1 -- 172.21.15.16:0/4255759236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e00cffe0 msgr2=0x7f57e00b66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.629+0000 7f57e8200640 1 --2- 172.21.15.16:0/4255759236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e00cffe0 0x7f57e00b66a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00c3c0 tx=0x7f57cc00c880 comp rx=0 tx=0).stop 2024-09-10T21:18:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.629+0000 7f57e8200640 1 -- 172.21.15.16:0/4255759236 shutdown_connections 2024-09-10T21:18:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.629+0000 7f57e8200640 1 --2- 172.21.15.16:0/4255759236 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f57c00725e0 0x7f57c0074aa0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.629+0000 7f57e8200640 1 --2- 172.21.15.16:0/4255759236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e00d03c0 0x7f57e015e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.629+0000 7f57e8200640 1 --2- 172.21.15.16:0/4255759236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57e00cffe0 0x7f57e00b66a0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.629+0000 7f57e8200640 1 --2- 172.21.15.16:0/4255759236 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f57e00b2860 0x7f57e00b7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.629+0000 7f57e8200640 1 -- 172.21.15.16:0/4255759236 >> 172.21.15.16:0/4255759236 conn(0x7f57e00d1d80 msgr2=0x7f57e015b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.629+0000 7f57e8200640 1 -- 172.21.15.16:0/4255759236 shutdown_connections 2024-09-10T21:18:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.629+0000 7f57e8200640 1 -- 172.21.15.16:0/4255759236 wait complete. 2024-09-10T21:18:01.658 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:18:01.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:01.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:18:01.758 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:18:01.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:01.758 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:18:01.767 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:18:01.767 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:18:01.767 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:01.777 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:18:01.777 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:01.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:18:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.973+0000 7fdcf2200640 1 -- 172.21.15.16:0/1567140348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcec0b5050 msgr2=0x7fdcec15f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.973+0000 7fdcf2200640 1 --2- 172.21.15.16:0/1567140348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcec0b5050 0x7fdcec15f850 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009840 tx=0x7fdcdc01c820 comp rx=0 tx=0).stop 2024-09-10T21:18:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.973+0000 7fdcf2200640 1 -- 172.21.15.16:0/1567140348 shutdown_connections 2024-09-10T21:18:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.973+0000 7fdcf2200640 1 --2- 172.21.15.16:0/1567140348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcec0b5050 0x7fdcec15f850 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.973+0000 7fdcf2200640 1 -- 172.21.15.16:0/1567140348 >> 172.21.15.16:0/1567140348 conn(0x7fdcec0cfb20 msgr2=0x7fdcec0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.973+0000 7fdcf2200640 1 -- 172.21.15.16:0/1567140348 shutdown_connections 2024-09-10T21:18:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.973+0000 7fdcf2200640 1 -- 172.21.15.16:0/1567140348 wait complete. 2024-09-10T21:18:01.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcf2200640 1 Processor -- start 2024-09-10T21:18:01.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcf2200640 1 -- start start 2024-09-10T21:18:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcf2200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcec06e640 0x7fdcec155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcf2200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdcec0b5050 0x7fdcec15ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcf2200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0ca170 0x7fdcec0cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcf2200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec0ca910 con 0x7fdcec0ca170 2024-09-10T21:18:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcf2200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec0ca790 con 0x7fdcec06e640 2024-09-10T21:18:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcf2200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fdcec16a200 con 0x7fdcec0b5050 2024-09-10T21:18:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcebe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0ca170 0x7fdcec0cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcebe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0ca170 0x7fdcec0cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46366/0 (socket says 172.21.15.16:46366) 2024-09-10T21:18:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcebe00640 1 -- 172.21.15.16:0/2665664286 learned_addr learned my addr 172.21.15.16:0/2665664286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdceb400640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcec06e640 0x7fdcec155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdce2a00640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdcec0b5050 0x7fdcec15ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcebe00640 1 -- 172.21.15.16:0/2665664286 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdcec0b5050 msgr2=0x7fdcec15ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcebe00640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdcec0b5050 0x7fdcec15ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcebe00640 1 -- 172.21.15.16:0/2665664286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcec06e640 msgr2=0x7fdcec155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcebe00640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcec06e640 0x7fdcec155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcebe00640 1 -- 172.21.15.16:0/2665664286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec164fe0 con 0x7fdcec0ca170 2024-09-10T21:18:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcebe00640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0ca170 0x7fdcec0cc3d0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fdce400cb10 tx=0x7fdce400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdce3e00640 1 -- 172.21.15.16:0/2665664286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4016020 con 0x7fdcec0ca170 2024-09-10T21:18:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdce3e00640 1 -- 172.21.15.16:0/2665664286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdce400ce50 con 0x7fdcec0ca170 2024-09-10T21:18:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdce3e00640 1 -- 172.21.15.16:0/2665664286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce400ae00 con 0x7fdcec0ca170 2024-09-10T21:18:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcf2200640 1 -- 172.21.15.16:0/2665664286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec1652c0 con 0x7fdcec0ca170 2024-09-10T21:18:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcf2200640 1 -- 172.21.15.16:0/2665664286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec165880 con 0x7fdcec0ca170 2024-09-10T21:18:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.977+0000 7fdcf2200640 1 -- 172.21.15.16:0/2665664286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec15f850 con 0x7fdcec0ca170 2024-09-10T21:18:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.981+0000 7fdce3e00640 1 -- 172.21.15.16:0/2665664286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdce4012420 con 0x7fdcec0ca170 2024-09-10T21:18:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.981+0000 7fdce3e00640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdcc80725e0 0x7fdcc8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.981+0000 7fdce3e00640 1 -- 172.21.15.16:0/2665664286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fdce40937f0 con 0x7fdcec0ca170 2024-09-10T21:18:01.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.981+0000 7fdceb400640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdcc80725e0 0x7fdcc8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.981+0000 7fdceb400640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdcc80725e0 0x7fdcc8074aa0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0af000 tx=0x7fdcdc01b060 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:01.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:01.985+0000 7fdce3e00640 1 -- 172.21.15.16:0/2665664286 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdce4020020 con 0x7fdcec0ca170 2024-09-10T21:18:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.169+0000 7fdcf2200640 1 -- 172.21.15.16:0/2665664286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdcec15de10 con 0x7fdcec0ca170 2024-09-10T21:18:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.169+0000 7fdce3e00640 1 -- 172.21.15.16:0/2665664286 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v116) ==== 80+0+0 (secure 0 0 0) 0x7fdce405eec0 con 0x7fdcec0ca170 2024-09-10T21:18:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:18:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.173+0000 7fdcf2200640 1 -- 172.21.15.16:0/2665664286 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdcc80725e0 msgr2=0x7fdcc8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.173+0000 7fdcf2200640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdcc80725e0 0x7fdcc8074aa0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0af000 tx=0x7fdcdc01b060 comp rx=0 tx=0).stop 2024-09-10T21:18:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.173+0000 7fdcf2200640 1 -- 172.21.15.16:0/2665664286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0ca170 msgr2=0x7fdcec0cc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.173+0000 7fdcf2200640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0ca170 0x7fdcec0cc3d0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fdce400cb10 tx=0x7fdce400cc00 comp rx=0 tx=0).stop 2024-09-10T21:18:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.173+0000 7fdcf2200640 1 -- 172.21.15.16:0/2665664286 shutdown_connections 2024-09-10T21:18:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.173+0000 7fdcf2200640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdcc80725e0 0x7fdcc8074aa0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.173+0000 7fdcf2200640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0ca170 0x7fdcec0cc3d0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.173+0000 7fdcf2200640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdcec0b5050 0x7fdcec15ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.173+0000 7fdcf2200640 1 --2- 172.21.15.16:0/2665664286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcec06e640 0x7fdcec155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.173+0000 7fdcf2200640 1 -- 172.21.15.16:0/2665664286 >> 172.21.15.16:0/2665664286 conn(0x7fdcec0cfb20 msgr2=0x7fdcec15e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.173+0000 7fdcf2200640 1 -- 172.21.15.16:0/2665664286 shutdown_connections 2024-09-10T21:18:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.173+0000 7fdcf2200640 1 -- 172.21.15.16:0/2665664286 wait complete. 2024-09-10T21:18:02.195 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T21:18:02.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:02.196 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T21:18:02.224 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:18:02.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:02.224 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T21:18:02.277 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:18:02.277 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:18:02.277 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:02.286 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:18:02.286 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:18:02.286 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:18:02.286 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:02.332 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:18:02.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.473+0000 7f7170200640 1 -- 172.21.15.16:0/22496641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815f400 msgr2=0x7f7168070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.473+0000 7f7170200640 1 --2- 172.21.15.16:0/22496641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815f400 0x7f7168070fe0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f715c0098f0 tx=0x7f715c01c7c0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.473+0000 7f7170200640 1 -- 172.21.15.16:0/22496641 shutdown_connections 2024-09-10T21:18:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.473+0000 7f7170200640 1 --2- 172.21.15.16:0/22496641 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f71680b13c0 0x7f7168071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.473+0000 7f7170200640 1 --2- 172.21.15.16:0/22496641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815f400 0x7f7168070fe0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.473+0000 7f7170200640 1 -- 172.21.15.16:0/22496641 >> 172.21.15.16:0/22496641 conn(0x7f716809b130 msgr2=0x7f71680d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.473+0000 7f7170200640 1 -- 172.21.15.16:0/22496641 shutdown_connections 2024-09-10T21:18:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.473+0000 7f7170200640 1 -- 172.21.15.16:0/22496641 wait complete. 2024-09-10T21:18:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.473+0000 7f7170200640 1 Processor -- start 2024-09-10T21:18:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f7170200640 1 -- start start 2024-09-10T21:18:02.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f7170200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71680b13c0 0x7f71681f9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:02.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f7170200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680cfc80 0x7f716815adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:02.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f7170200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f716815f400 0x7f71681519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:02.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f7170200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7168169770 con 0x7f71680cfc80 2024-09-10T21:18:02.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f7170200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71681695f0 con 0x7f71680b13c0 2024-09-10T21:18:02.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f7170200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f71681698f0 con 0x7f716815f400 2024-09-10T21:18:02.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f716ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680cfc80 0x7f716815adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:02.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f716ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680cfc80 0x7f716815adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51992/0 (socket says 172.21.15.16:51992) 2024-09-10T21:18:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f716ce00640 1 -- 172.21.15.16:0/2490867041 learned_addr learned my addr 172.21.15.16:0/2490867041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f716d800640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71680b13c0 0x7f71681f9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f716e200640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f716815f400 0x7f71681519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f716ce00640 1 -- 172.21.15.16:0/2490867041 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f716815f400 msgr2=0x7f71681519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f716ce00640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f716815f400 0x7f71681519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f716ce00640 1 -- 172.21.15.16:0/2490867041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71680b13c0 msgr2=0x7f71681f9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f716ce00640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71680b13c0 0x7f71681f9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f716ce00640 1 -- 172.21.15.16:0/2490867041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71681fa100 con 0x7f71680cfc80 2024-09-10T21:18:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f716e200640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f716815f400 0x7f71681519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f716ce00640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680cfc80 0x7f716815adb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f715400d700 tx=0x7f715400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f7165600640 1 -- 172.21.15.16:0/2490867041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154017070 con 0x7f71680cfc80 2024-09-10T21:18:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f7165600640 1 -- 172.21.15.16:0/2490867041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7154009d20 con 0x7f71680cfc80 2024-09-10T21:18:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f7170200640 1 -- 172.21.15.16:0/2490867041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71680b58f0 con 0x7f71680cfc80 2024-09-10T21:18:02.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f7165600640 1 -- 172.21.15.16:0/2490867041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154005280 con 0x7f71680cfc80 2024-09-10T21:18:02.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f7170200640 1 -- 172.21.15.16:0/2490867041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71681511d0 con 0x7f71680cfc80 2024-09-10T21:18:02.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.477+0000 7f7170200640 1 -- 172.21.15.16:0/2490867041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71680d00d0 con 0x7f71680cfc80 2024-09-10T21:18:02.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.481+0000 7f7165600640 1 -- 172.21.15.16:0/2490867041 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7154005420 con 0x7f71680cfc80 2024-09-10T21:18:02.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.481+0000 7f7165600640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f71480725e0 0x7f7148074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:02.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.481+0000 7f7165600640 1 -- 172.21.15.16:0/2490867041 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f71540933a0 con 0x7f71680cfc80 2024-09-10T21:18:02.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.481+0000 7f716d800640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f71480725e0 0x7f7148074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:02.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.481+0000 7f716d800640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f71480725e0 0x7f7148074aa0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f715c0af000 tx=0x7f715c01b000 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:02.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.485+0000 7f7165600640 1 -- 172.21.15.16:0/2490867041 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f71681fa100 con 0x7f71680cfc80 2024-09-10T21:18:02.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.693+0000 7f7170200640 1 -- 172.21.15.16:0/2490867041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f716806cfb0 con 0x7f71680cfc80 2024-09-10T21:18:02.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.693+0000 7f7165600640 1 -- 172.21.15.16:0/2490867041 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5229 (secure 0 0 0) 0x7f7154067100 con 0x7f71680cfc80 2024-09-10T21:18:02.698 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:02.698 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"btime":"2024-09-10T21:17:57:759836+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3961362430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3961362430},{"type":"v1","addr":"172.21.15.52:6834","nonce":3961362430}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6967,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/831111367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":831111367},{"type":"v1","addr":"172.21.15.52:6835","nonce":831111367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7467,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/552929965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":552929965},{"type":"v1","addr":"172.21.15.16:6836","nonce":552929965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:40.510614+0000","modified":"2024-09-10T21:17:57.134218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":136181},"failed":[],"damaged":[],"stopped":[],"info":{"gid_136181":{"gid":136181,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":7,"addr":"172.21.15.16:6837/1681945537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1681945537},{"type":"v1","addr":"172.21.15.16:6837","nonce":1681945537}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":136181,"qdb_cluster":[136181]},"id":9}]} 2024-09-10T21:18:02.698 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 84 2024-09-10T21:18:02.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.697+0000 7f7170200640 1 -- 172.21.15.16:0/2490867041 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f71480725e0 msgr2=0x7f7148074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:02.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.697+0000 7f7170200640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f71480725e0 0x7f7148074aa0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f715c0af000 tx=0x7f715c01b000 comp rx=0 tx=0).stop 2024-09-10T21:18:02.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.697+0000 7f7170200640 1 -- 172.21.15.16:0/2490867041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680cfc80 msgr2=0x7f716815adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:02.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.697+0000 7f7170200640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680cfc80 0x7f716815adb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f715400d700 tx=0x7f715400dbc0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.697+0000 7f7170200640 1 -- 172.21.15.16:0/2490867041 shutdown_connections 2024-09-10T21:18:02.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.697+0000 7f7170200640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f71480725e0 0x7f7148074aa0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.697+0000 7f7170200640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f716815f400 0x7f71681519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.697+0000 7f7170200640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71680cfc80 0x7f716815adb0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.697+0000 7f7170200640 1 --2- 172.21.15.16:0/2490867041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71680b13c0 0x7f71681f9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.697+0000 7f7170200640 1 -- 172.21.15.16:0/2490867041 >> 172.21.15.16:0/2490867041 conn(0x7f716809b130 msgr2=0x7f71680cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.697+0000 7f7170200640 1 -- 172.21.15.16:0/2490867041 shutdown_connections 2024-09-10T21:18:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.697+0000 7f7170200640 1 -- 172.21.15.16:0/2490867041 wait complete. 2024-09-10T21:18:02.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 -- 172.21.15.16:0/3836154109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380ca170 msgr2=0x7f03380ca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 --2- 172.21.15.16:0/3836154109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380ca170 0x7f03380ca550 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f032c00b3f0 tx=0x7f032c01d180 comp rx=0 tx=0).stop 2024-09-10T21:18:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 -- 172.21.15.16:0/3836154109 shutdown_connections 2024-09-10T21:18:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 --2- 172.21.15.16:0/3836154109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380ca170 0x7f03380ca550 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 -- 172.21.15.16:0/3836154109 >> 172.21.15.16:0/3836154109 conn(0x7f03380cfb20 msgr2=0x7f03380cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 -- 172.21.15.16:0/3836154109 shutdown_connections 2024-09-10T21:18:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 -- 172.21.15.16:0/3836154109 wait complete. 2024-09-10T21:18:02.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 Processor -- start 2024-09-10T21:18:02.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 -- start start 2024-09-10T21:18:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f03380ca170 0x7f033814ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03380cd6f0 0x7f0338158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380cf370 0x7f03380b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0338157ef0 con 0x7f03380cf370 2024-09-10T21:18:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0338157d70 con 0x7f03380cd6f0 2024-09-10T21:18:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f0340000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0338169770 con 0x7f03380ca170 2024-09-10T21:18:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f033e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380cf370 0x7f03380b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f033e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380cf370 0x7f03380b09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52008/0 (socket says 172.21.15.16:52008) 2024-09-10T21:18:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f033e000640 1 -- 172.21.15.16:0/1308570271 learned_addr learned my addr 172.21.15.16:0/1308570271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f033d600640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f03380ca170 0x7f033814ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.865+0000 7f033cc00640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03380cd6f0 0x7f0338158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f033e000640 1 -- 172.21.15.16:0/1308570271 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f03380ca170 msgr2=0x7f033814ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f033e000640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f03380ca170 0x7f033814ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f033e000640 1 -- 172.21.15.16:0/1308570271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03380cd6f0 msgr2=0x7f0338158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f033e000640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03380cd6f0 0x7f0338158280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f033e000640 1 -- 172.21.15.16:0/1308570271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033806f690 con 0x7f03380cf370 2024-09-10T21:18:02.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f033cc00640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03380cd6f0 0x7f0338158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:02.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f033d600640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f03380ca170 0x7f033814ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f033e000640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380cf370 0x7f03380b09d0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f033000c780 tx=0x7f033000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f0335600640 1 -- 172.21.15.16:0/1308570271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330004b70 con 0x7f03380cf370 2024-09-10T21:18:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f0335600640 1 -- 172.21.15.16:0/1308570271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0330004e00 con 0x7f03380cf370 2024-09-10T21:18:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f0335600640 1 -- 172.21.15.16:0/1308570271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033000aa90 con 0x7f03380cf370 2024-09-10T21:18:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f0340000640 1 -- 172.21.15.16:0/1308570271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033806f970 con 0x7f03380cf370 2024-09-10T21:18:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f0340000640 1 -- 172.21.15.16:0/1308570271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f033815c800 con 0x7f03380cf370 2024-09-10T21:18:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f0340000640 1 -- 172.21.15.16:0/1308570271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03380ca550 con 0x7f03380cf370 2024-09-10T21:18:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f0335600640 1 -- 172.21.15.16:0/1308570271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f033000ac30 con 0x7f03380cf370 2024-09-10T21:18:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f0335600640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f03180725e0 0x7f0318074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f0335600640 1 -- 172.21.15.16:0/1308570271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f0330093560 con 0x7f03380cf370 2024-09-10T21:18:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.869+0000 7f033d600640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f03180725e0 0x7f0318074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.873+0000 7f033d600640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f03180725e0 0x7f0318074aa0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f032c0ae000 tx=0x7f032c01b9c0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:02.873+0000 7f0335600640 1 -- 172.21.15.16:0/1308570271 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0330020070 con 0x7f03380cf370 2024-09-10T21:18:03.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.081+0000 7f0340000640 1 -- 172.21.15.16:0/1308570271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03380b5aa0 con 0x7f03380cf370 2024-09-10T21:18:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.085+0000 7f0335600640 1 -- 172.21.15.16:0/1308570271 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5229 (secure 0 0 0) 0x7f033005ec20 con 0x7f03380cf370 2024-09-10T21:18:03.088 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:03.088 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"btime":"2024-09-10T21:17:57:759836+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3961362430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3961362430},{"type":"v1","addr":"172.21.15.52:6834","nonce":3961362430}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6967,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/831111367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":831111367},{"type":"v1","addr":"172.21.15.52:6835","nonce":831111367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7467,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/552929965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":552929965},{"type":"v1","addr":"172.21.15.16:6836","nonce":552929965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:40.510614+0000","modified":"2024-09-10T21:17:57.134218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":136181},"failed":[],"damaged":[],"stopped":[],"info":{"gid_136181":{"gid":136181,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":7,"addr":"172.21.15.16:6837/1681945537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1681945537},{"type":"v1","addr":"172.21.15.16:6837","nonce":1681945537}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":136181,"qdb_cluster":[136181]},"id":9}]} 2024-09-10T21:18:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 84 2024-09-10T21:18:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.085+0000 7f0340000640 1 -- 172.21.15.16:0/1308570271 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f03180725e0 msgr2=0x7f0318074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.085+0000 7f0340000640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f03180725e0 0x7f0318074aa0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f032c0ae000 tx=0x7f032c01b9c0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.085+0000 7f0340000640 1 -- 172.21.15.16:0/1308570271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380cf370 msgr2=0x7f03380b09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.085+0000 7f0340000640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380cf370 0x7f03380b09d0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f033000c780 tx=0x7f033000cc40 comp rx=0 tx=0).stop 2024-09-10T21:18:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.085+0000 7f0340000640 1 -- 172.21.15.16:0/1308570271 shutdown_connections 2024-09-10T21:18:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.085+0000 7f0340000640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f03180725e0 0x7f0318074aa0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.085+0000 7f0340000640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380cf370 0x7f03380b09d0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.085+0000 7f0340000640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03380cd6f0 0x7f0338158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.085+0000 7f0340000640 1 --2- 172.21.15.16:0/1308570271 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f03380ca170 0x7f033814ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.085+0000 7f0340000640 1 -- 172.21.15.16:0/1308570271 >> 172.21.15.16:0/1308570271 conn(0x7f03380cfb20 msgr2=0x7f03381568b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.085+0000 7f0340000640 1 -- 172.21.15.16:0/1308570271 shutdown_connections 2024-09-10T21:18:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.085+0000 7f0340000640 1 -- 172.21.15.16:0/1308570271 wait complete. 2024-09-10T21:18:03.110 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:18:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.253+0000 7fddac600640 1 -- 172.21.15.16:0/4018588980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c004810 msgr2=0x7fdd8c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.253+0000 7fddac600640 1 --2- 172.21.15.16:0/4018588980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c004810 0x7fdd8c004c90 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdd90006fc0 tx=0x7fdd9001cc00 comp rx=0 tx=0).stop 2024-09-10T21:18:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.253+0000 7fddaa600640 1 -- 172.21.15.16:0/4018588980 reap_dead start 2024-09-10T21:18:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.253+0000 7fddac600640 1 -- 172.21.15.16:0/4018588980 shutdown_connections 2024-09-10T21:18:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.253+0000 7fddac600640 1 -- 172.21.15.16:0/4018588980 >> 172.21.15.16:0/4018588980 conn(0x7fdda40d1d80 msgr2=0x7fdda40d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.253+0000 7fddac600640 1 -- 172.21.15.16:0/4018588980 shutdown_connections 2024-09-10T21:18:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.253+0000 7fddac600640 1 -- 172.21.15.16:0/4018588980 wait complete. 2024-09-10T21:18:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.253+0000 7fddac600640 1 Processor -- start 2024-09-10T21:18:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.253+0000 7fddac600640 1 -- start start 2024-09-10T21:18:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fddac600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c004810 0x7fdda40cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fddac600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdda40cd0c0 0x7fdda41f8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fddac600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdda41f87b0 0x7fdda40b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fddac600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdda40b2e30 con 0x7fdd8c004810 2024-09-10T21:18:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fddac600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdda40b2cb0 con 0x7fdda40cd0c0 2024-09-10T21:18:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fddac600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fdda40b2fb0 con 0x7fdda41f87b0 2024-09-10T21:18:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c004810 0x7fdda40cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c004810 0x7fdda40cf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52018/0 (socket says 172.21.15.16:52018) 2024-09-10T21:18:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda9c00640 1 -- 172.21.15.16:0/3533278794 learned_addr learned my addr 172.21.15.16:0/3533278794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fddaa600640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdda41f87b0 0x7fdda40b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda9200640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdda40cd0c0 0x7fdda41f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda9c00640 1 -- 172.21.15.16:0/3533278794 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdda41f87b0 msgr2=0x7fdda40b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda9c00640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdda41f87b0 0x7fdda40b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda9c00640 1 -- 172.21.15.16:0/3533278794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdda40cd0c0 msgr2=0x7fdda41f8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda9c00640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdda40cd0c0 0x7fdda41f8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda9c00640 1 -- 172.21.15.16:0/3533278794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda40cd960 con 0x7fdd8c004810 2024-09-10T21:18:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda9c00640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c004810 0x7fdda40cf220 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fdd98005ef0 tx=0x7fdd98011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda1600640 1 -- 172.21.15.16:0/3533278794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98011e90 con 0x7fdd8c004810 2024-09-10T21:18:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda1600640 1 -- 172.21.15.16:0/3533278794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd98018020 con 0x7fdd8c004810 2024-09-10T21:18:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda1600640 1 -- 172.21.15.16:0/3533278794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98003600 con 0x7fdd8c004810 2024-09-10T21:18:03.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fddac600640 1 -- 172.21.15.16:0/3533278794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda40cdc40 con 0x7fdd8c004810 2024-09-10T21:18:03.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fddac600640 1 -- 172.21.15.16:0/3533278794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda40cd540 con 0x7fdd8c004810 2024-09-10T21:18:03.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fddac600640 1 -- 172.21.15.16:0/3533278794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda40cf760 con 0x7fdd8c004810 2024-09-10T21:18:03.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.257+0000 7fdda1600640 1 -- 172.21.15.16:0/3533278794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdd98010040 con 0x7fdd8c004810 2024-09-10T21:18:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.261+0000 7fdda1600640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd840727b0 0x7fdd84074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.261+0000 7fdda1600640 1 -- 172.21.15.16:0/3533278794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fdd98096690 con 0x7fdd8c004810 2024-09-10T21:18:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.261+0000 7fdda9200640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd840727b0 0x7fdd84074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.261+0000 7fdda9200640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd840727b0 0x7fdd84074c70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdd900ac000 tx=0x7fdd90019a40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.265+0000 7fdda1600640 1 -- 172.21.15.16:0/3533278794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdd98022020 con 0x7fdd8c004810 2024-09-10T21:18:03.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fddac600640 1 -- 172.21.15.16:0/3533278794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdda40afae0 con 0x7fdd8c004810 2024-09-10T21:18:03.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fdda1600640 1 -- 172.21.15.16:0/3533278794 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16639 (secure 0 0 0) 0x7fdd980617d0 con 0x7fdd8c004810 2024-09-10T21:18:03.445 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:03.445 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":116,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:17:42.104454+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:17:38.209906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T21:17:39.468885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:18:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fddac600640 1 -- 172.21.15.16:0/3533278794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd840727b0 msgr2=0x7fdd84074c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fddac600640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd840727b0 0x7fdd84074c70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdd900ac000 tx=0x7fdd90019a40 comp rx=0 tx=0).stop 2024-09-10T21:18:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fddac600640 1 -- 172.21.15.16:0/3533278794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c004810 msgr2=0x7fdda40cf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fddac600640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c004810 0x7fdda40cf220 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fdd98005ef0 tx=0x7fdd98011740 comp rx=0 tx=0).stop 2024-09-10T21:18:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fddac600640 1 -- 172.21.15.16:0/3533278794 shutdown_connections 2024-09-10T21:18:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fddac600640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdd840727b0 0x7fdd84074c70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fddac600640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdda41f87b0 0x7fdda40b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fddac600640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdda40cd0c0 0x7fdda41f8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fddac600640 1 --2- 172.21.15.16:0/3533278794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c004810 0x7fdda40cf220 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fddac600640 1 -- 172.21.15.16:0/3533278794 >> 172.21.15.16:0/3533278794 conn(0x7fdda40d1d80 msgr2=0x7fdda415e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fddac600640 1 -- 172.21.15.16:0/3533278794 shutdown_connections 2024-09-10T21:18:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.441+0000 7fddac600640 1 -- 172.21.15.16:0/3533278794 wait complete. 2024-09-10T21:18:03.468 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:18:03.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.609+0000 7fe08ba00640 1 -- 172.21.15.16:0/3411193788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0840ca170 msgr2=0x7fe0840ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.609+0000 7fe08ba00640 1 --2- 172.21.15.16:0/3411193788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0840ca170 0x7fe0840ca5b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe08406e1a0 tx=0x7fe07c01ccc0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.609+0000 7fe08ba00640 1 -- 172.21.15.16:0/3411193788 shutdown_connections 2024-09-10T21:18:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.609+0000 7fe08ba00640 1 --2- 172.21.15.16:0/3411193788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0840ca170 0x7fe0840ca5b0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.609+0000 7fe08ba00640 1 --2- 172.21.15.16:0/3411193788 >> v2:172.21.15.52:3300/0 conn(0x7fe08415a270 0x7fe0840ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.609+0000 7fe08ba00640 1 -- 172.21.15.16:0/3411193788 >> 172.21.15.16:0/3411193788 conn(0x7fe0840cfb20 msgr2=0x7fe0840cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.609+0000 7fe08ba00640 1 -- 172.21.15.16:0/3411193788 shutdown_connections 2024-09-10T21:18:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.609+0000 7fe08ba00640 1 -- 172.21.15.16:0/3411193788 wait complete. 2024-09-10T21:18:03.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe08ba00640 1 Processor -- start 2024-09-10T21:18:03.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe08ba00640 1 -- start start 2024-09-10T21:18:03.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe08ba00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0840ca170 0x7fe0840b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:03.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe08ba00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe0840d06b0 0x7fe0840c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:03.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe08ba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08415a270 0x7fe084156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe08ba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe084072b80 con 0x7fe08415a270 2024-09-10T21:18:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe08ba00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe084072a00 con 0x7fe0840ca170 2024-09-10T21:18:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe08ba00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe084072d00 con 0x7fe0840d06b0 2024-09-10T21:18:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe089a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08415a270 0x7fe084156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe089a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08415a270 0x7fe084156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52024/0 (socket says 172.21.15.16:52024) 2024-09-10T21:18:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe089a00640 1 -- 172.21.15.16:0/1543620442 learned_addr learned my addr 172.21.15.16:0/1543620442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe083e00640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe0840d06b0 0x7fe0840c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe089000640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0840ca170 0x7fe0840b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe089a00640 1 -- 172.21.15.16:0/1543620442 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe0840d06b0 msgr2=0x7fe0840c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe089a00640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe0840d06b0 0x7fe0840c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe089a00640 1 -- 172.21.15.16:0/1543620442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0840ca170 msgr2=0x7fe0840b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe089a00640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0840ca170 0x7fe0840b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe089a00640 1 -- 172.21.15.16:0/1543620442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0841520f0 con 0x7fe08415a270 2024-09-10T21:18:03.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe089a00640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08415a270 0x7fe084156c60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe07c01b370 tx=0x7fe07c009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe080c00640 1 -- 172.21.15.16:0/1543620442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0bb070 con 0x7fe08415a270 2024-09-10T21:18:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe080c00640 1 -- 172.21.15.16:0/1543620442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe07c004d40 con 0x7fe08415a270 2024-09-10T21:18:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe080c00640 1 -- 172.21.15.16:0/1543620442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0b2040 con 0x7fe08415a270 2024-09-10T21:18:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe08ba00640 1 -- 172.21.15.16:0/1543620442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0840c9ae0 con 0x7fe08415a270 2024-09-10T21:18:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.613+0000 7fe08ba00640 1 -- 172.21.15.16:0/1543620442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0840c7000 con 0x7fe08415a270 2024-09-10T21:18:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.617+0000 7fe08ba00640 1 -- 172.21.15.16:0/1543620442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0840ca5b0 con 0x7fe08415a270 2024-09-10T21:18:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.617+0000 7fe080c00640 1 -- 172.21.15.16:0/1543620442 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe07c0b5030 con 0x7fe08415a270 2024-09-10T21:18:03.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.617+0000 7fe080c00640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0640725e0 0x7fe064074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:03.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.617+0000 7fe080c00640 1 -- 172.21.15.16:0/1543620442 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fe07c12d120 con 0x7fe08415a270 2024-09-10T21:18:03.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.617+0000 7fe089000640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0640725e0 0x7fe064074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.617+0000 7fe089000640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0640725e0 0x7fe064074aa0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe084151df0 tx=0x7fe07800b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:03.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.621+0000 7fe080c00640 1 -- 172.21.15.16:0/1543620442 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe07c0c4020 con 0x7fe08415a270 2024-09-10T21:18:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.833+0000 7fe08ba00640 1 -- 172.21.15.16:0/1543620442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0840b5aa0 con 0x7fe08415a270 2024-09-10T21:18:03.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.833+0000 7fe080c00640 1 -- 172.21.15.16:0/1543620442 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5229 (secure 0 0 0) 0x7fe07c016970 con 0x7fe08415a270 2024-09-10T21:18:03.838 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:03.838 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"btime":"2024-09-10T21:17:57:759836+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3961362430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3961362430},{"type":"v1","addr":"172.21.15.52:6834","nonce":3961362430}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6967,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/831111367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":831111367},{"type":"v1","addr":"172.21.15.52:6835","nonce":831111367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7467,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/552929965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":552929965},{"type":"v1","addr":"172.21.15.16:6836","nonce":552929965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:40.510614+0000","modified":"2024-09-10T21:17:57.134218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":136181},"failed":[],"damaged":[],"stopped":[],"info":{"gid_136181":{"gid":136181,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":7,"addr":"172.21.15.16:6837/1681945537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1681945537},{"type":"v1","addr":"172.21.15.16:6837","nonce":1681945537}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":136181,"qdb_cluster":[136181]},"id":9}]} 2024-09-10T21:18:03.838 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 84 2024-09-10T21:18:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.837+0000 7fe08ba00640 1 -- 172.21.15.16:0/1543620442 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0640725e0 msgr2=0x7fe064074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.837+0000 7fe08ba00640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0640725e0 0x7fe064074aa0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe084151df0 tx=0x7fe07800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.837+0000 7fe08ba00640 1 -- 172.21.15.16:0/1543620442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08415a270 msgr2=0x7fe084156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.837+0000 7fe08ba00640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08415a270 0x7fe084156c60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe07c01b370 tx=0x7fe07c009370 comp rx=0 tx=0).stop 2024-09-10T21:18:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.837+0000 7fe08ba00640 1 -- 172.21.15.16:0/1543620442 shutdown_connections 2024-09-10T21:18:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.837+0000 7fe08ba00640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0640725e0 0x7fe064074aa0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.837+0000 7fe08ba00640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08415a270 0x7fe084156c60 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.837+0000 7fe08ba00640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe0840d06b0 0x7fe0840c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.837+0000 7fe08ba00640 1 --2- 172.21.15.16:0/1543620442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0840ca170 0x7fe0840b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.837+0000 7fe08ba00640 1 -- 172.21.15.16:0/1543620442 >> 172.21.15.16:0/1543620442 conn(0x7fe0840cfb20 msgr2=0x7fe084073160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.837+0000 7fe08ba00640 1 -- 172.21.15.16:0/1543620442 shutdown_connections 2024-09-10T21:18:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:03.837+0000 7fe08ba00640 1 -- 172.21.15.16:0/1543620442 wait complete. 2024-09-10T21:18:03.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.001+0000 7f6427600640 1 -- 172.21.15.16:0/3264377003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200d0ad0 msgr2=0x7f642006e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.001+0000 7f6427600640 1 --2- 172.21.15.16:0/3264377003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200d0ad0 0x7f642006e020 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f6418009bc0 tx=0x7f641801cb30 comp rx=0 tx=0).stop 2024-09-10T21:18:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.001+0000 7f6427600640 1 -- 172.21.15.16:0/3264377003 shutdown_connections 2024-09-10T21:18:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.001+0000 7f6427600640 1 --2- 172.21.15.16:0/3264377003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200d0ad0 0x7f642006e020 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.001+0000 7f6427600640 1 --2- 172.21.15.16:0/3264377003 >> v2:172.21.15.52:3300/0 conn(0x7f64200b0b40 0x7f64200d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.001+0000 7f6427600640 1 -- 172.21.15.16:0/3264377003 >> 172.21.15.16:0/3264377003 conn(0x7f642009b130 msgr2=0x7f64200d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.001+0000 7f6427600640 1 -- 172.21.15.16:0/3264377003 shutdown_connections 2024-09-10T21:18:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.001+0000 7f6427600640 1 -- 172.21.15.16:0/3264377003 wait complete. 2024-09-10T21:18:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f6427600640 1 Processor -- start 2024-09-10T21:18:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f6427600640 1 -- start start 2024-09-10T21:18:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f6427600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64200b0b40 0x7f6420153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f6427600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200d0ad0 0x7f64200b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f6427600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6420161620 0x7f64200b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f6427600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f642016b100 con 0x7f64200d0ad0 2024-09-10T21:18:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f6427600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f642016af80 con 0x7f64200b0b40 2024-09-10T21:18:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f6427600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f642016b280 con 0x7f6420161620 2024-09-10T21:18:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f641fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200d0ad0 0x7f64200b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f641fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200d0ad0 0x7f64200b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52034/0 (socket says 172.21.15.16:52034) 2024-09-10T21:18:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f641fe00640 1 -- 172.21.15.16:0/3450278891 learned_addr learned my addr 172.21.15.16:0/3450278891 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f6424c00640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64200b0b40 0x7f6420153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f6425600640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6420161620 0x7f64200b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f641fe00640 1 -- 172.21.15.16:0/3450278891 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6420161620 msgr2=0x7f64200b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f641fe00640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6420161620 0x7f64200b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f641fe00640 1 -- 172.21.15.16:0/3450278891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64200b0b40 msgr2=0x7f6420153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f641fe00640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64200b0b40 0x7f6420153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f641fe00640 1 -- 172.21.15.16:0/3450278891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64200d1c10 con 0x7f64200d0ad0 2024-09-10T21:18:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f641fe00640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200d0ad0 0x7f64200b3ab0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f640c00d980 tx=0x7f640c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f641cc00640 1 -- 172.21.15.16:0/3450278891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c017070 con 0x7f64200d0ad0 2024-09-10T21:18:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f641cc00640 1 -- 172.21.15.16:0/3450278891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f640c009d20 con 0x7f64200d0ad0 2024-09-10T21:18:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f6427600640 1 -- 172.21.15.16:0/3450278891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64200d1ef0 con 0x7f64200d0ad0 2024-09-10T21:18:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f641cc00640 1 -- 172.21.15.16:0/3450278891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c004280 con 0x7f64200d0ad0 2024-09-10T21:18:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f6427600640 1 -- 172.21.15.16:0/3450278891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64200d24b0 con 0x7f64200d0ad0 2024-09-10T21:18:04.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.005+0000 7f6427600640 1 -- 172.21.15.16:0/3450278891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64200d0590 con 0x7f64200d0ad0 2024-09-10T21:18:04.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.009+0000 7f641cc00640 1 -- 172.21.15.16:0/3450278891 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f640c003600 con 0x7f64200d0ad0 2024-09-10T21:18:04.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.009+0000 7f641cc00640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f64000725e0 0x7f6400074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.009+0000 7f641cc00640 1 -- 172.21.15.16:0/3450278891 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f640c0936a0 con 0x7f64200d0ad0 2024-09-10T21:18:04.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.009+0000 7f6424c00640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f64000725e0 0x7f6400074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.009+0000 7f6424c00640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f64000725e0 0x7f6400074aa0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6414001770 tx=0x7f6414009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.013+0000 7f641cc00640 1 -- 172.21.15.16:0/3450278891 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f640c05e720 con 0x7f64200d0ad0 2024-09-10T21:18:04.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f6427600640 1 -- 172.21.15.16:0/3450278891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64200c53b0 con 0x7f64200d0ad0 2024-09-10T21:18:04.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f641cc00640 1 -- 172.21.15.16:0/3450278891 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5229 (secure 0 0 0) 0x7f640c063150 con 0x7f64200d0ad0 2024-09-10T21:18:04.229 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:04.229 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"btime":"2024-09-10T21:17:57:759836+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3961362430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3961362430},{"type":"v1","addr":"172.21.15.52:6834","nonce":3961362430}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6967,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/831111367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":831111367},{"type":"v1","addr":"172.21.15.52:6835","nonce":831111367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7467,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/552929965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":552929965},{"type":"v1","addr":"172.21.15.16:6836","nonce":552929965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:17:40.510614+0000","modified":"2024-09-10T21:17:57.134218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":136181},"failed":[],"damaged":[],"stopped":[],"info":{"gid_136181":{"gid":136181,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":7,"addr":"172.21.15.16:6837/1681945537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1681945537},{"type":"v1","addr":"172.21.15.16:6837","nonce":1681945537}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":136181,"qdb_cluster":[136181]},"id":9}]} 2024-09-10T21:18:04.230 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 84 2024-09-10T21:18:04.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f6427600640 1 -- 172.21.15.16:0/3450278891 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f64000725e0 msgr2=0x7f6400074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f6427600640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f64000725e0 0x7f6400074aa0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6414001770 tx=0x7f6414009040 comp rx=0 tx=0).stop 2024-09-10T21:18:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f6427600640 1 -- 172.21.15.16:0/3450278891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200d0ad0 msgr2=0x7f64200b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f6427600640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200d0ad0 0x7f64200b3ab0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f640c00d980 tx=0x7f640c00de40 comp rx=0 tx=0).stop 2024-09-10T21:18:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f6427600640 1 -- 172.21.15.16:0/3450278891 shutdown_connections 2024-09-10T21:18:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f6427600640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f64000725e0 0x7f6400074aa0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f6427600640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6420161620 0x7f64200b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f6427600640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200d0ad0 0x7f64200b3ab0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f6427600640 1 --2- 172.21.15.16:0/3450278891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64200b0b40 0x7f6420153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f6427600640 1 -- 172.21.15.16:0/3450278891 >> 172.21.15.16:0/3450278891 conn(0x7f642009b130 msgr2=0x7f64200ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f6427600640 1 -- 172.21.15.16:0/3450278891 shutdown_connections 2024-09-10T21:18:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.225+0000 7f6427600640 1 -- 172.21.15.16:0/3450278891 wait complete. 2024-09-10T21:18:04.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:18:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 -- 172.21.15.16:0/630121606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c161900 msgr2=0x7f676c0d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 --2- 172.21.15.16:0/630121606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c161900 0x7f676c0d0ee0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f6754009900 tx=0x7f675401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 -- 172.21.15.16:0/630121606 shutdown_connections 2024-09-10T21:18:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 --2- 172.21.15.16:0/630121606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c161900 0x7f676c0d0ee0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 -- 172.21.15.16:0/630121606 >> 172.21.15.16:0/630121606 conn(0x7f676c0d1d80 msgr2=0x7f676c0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 -- 172.21.15.16:0/630121606 shutdown_connections 2024-09-10T21:18:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 -- 172.21.15.16:0/630121606 wait complete. 2024-09-10T21:18:04.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 Processor -- start 2024-09-10T21:18:04.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 -- start start 2024-09-10T21:18:04.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f676c0b2290 0x7f676c1603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f676c161520 0x7f676c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c161900 0x7f676c15b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f676c16be80 con 0x7f676c161900 2024-09-10T21:18:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f676c06e930 con 0x7f676c0b2290 2024-09-10T21:18:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f6770e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f676c16c000 con 0x7f676c161520 2024-09-10T21:18:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.393+0000 7f676a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c161900 0x7f676c15b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f676a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c161900 0x7f676c15b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52052/0 (socket says 172.21.15.16:52052) 2024-09-10T21:18:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f676a800640 1 -- 172.21.15.16:0/3414606052 learned_addr learned my addr 172.21.15.16:0/3414606052 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f6769e00640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f676c0b2290 0x7f676c1603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f6763e00640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f676c161520 0x7f676c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f676a800640 1 -- 172.21.15.16:0/3414606052 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f676c161520 msgr2=0x7f676c0af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f676a800640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f676c161520 0x7f676c0af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f676a800640 1 -- 172.21.15.16:0/3414606052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f676c0b2290 msgr2=0x7f676c1603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f676a800640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f676c0b2290 0x7f676c1603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f676a800640 1 -- 172.21.15.16:0/3414606052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c0b5dd0 con 0x7f676c161900 2024-09-10T21:18:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f676a800640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c161900 0x7f676c15b420 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f676400c7c0 tx=0x7f676400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f6762000640 1 -- 172.21.15.16:0/3414606052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764003b80 con 0x7f676c161900 2024-09-10T21:18:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f6762000640 1 -- 172.21.15.16:0/3414606052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6764003d20 con 0x7f676c161900 2024-09-10T21:18:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f6762000640 1 -- 172.21.15.16:0/3414606052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676400aa50 con 0x7f676c161900 2024-09-10T21:18:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f6770e00640 1 -- 172.21.15.16:0/3414606052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676c0b6090 con 0x7f676c161900 2024-09-10T21:18:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f6770e00640 1 -- 172.21.15.16:0/3414606052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f676c0b6650 con 0x7f676c161900 2024-09-10T21:18:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f6770e00640 1 -- 172.21.15.16:0/3414606052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f676c160cd0 con 0x7f676c161900 2024-09-10T21:18:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.397+0000 7f6762000640 1 -- 172.21.15.16:0/3414606052 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f676400abf0 con 0x7f676c161900 2024-09-10T21:18:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.401+0000 7f6762000640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67480725e0 0x7f6748074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.401+0000 7f6762000640 1 -- 172.21.15.16:0/3414606052 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f6764092f10 con 0x7f676c161900 2024-09-10T21:18:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.401+0000 7f6769e00640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67480725e0 0x7f6748074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.401+0000 7f6769e00640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67480725e0 0x7f6748074aa0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f675c007fd0 tx=0x7f675c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:04.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.405+0000 7f6762000640 1 -- 172.21.15.16:0/3414606052 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f676c160cd0 con 0x7f676c161900 2024-09-10T21:18:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.581+0000 7f6770e00640 1 -- 172.21.15.16:0/3414606052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f676c0b26c0 con 0x7f676c161900 2024-09-10T21:18:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.581+0000 7f6762000640 1 -- 172.21.15.16:0/3414606052 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16639 (secure 0 0 0) 0x7f676405df90 con 0x7f676c161900 2024-09-10T21:18:04.585 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:04.585 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":116,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:17:42.104454+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:17:38.209906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T21:17:39.468885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:18:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.585+0000 7f6770e00640 1 -- 172.21.15.16:0/3414606052 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67480725e0 msgr2=0x7f6748074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.585+0000 7f6770e00640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67480725e0 0x7f6748074aa0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f675c007fd0 tx=0x7f675c009040 comp rx=0 tx=0).stop 2024-09-10T21:18:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.585+0000 7f6770e00640 1 -- 172.21.15.16:0/3414606052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c161900 msgr2=0x7f676c15b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.585+0000 7f6770e00640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c161900 0x7f676c15b420 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f676400c7c0 tx=0x7f676400cc80 comp rx=0 tx=0).stop 2024-09-10T21:18:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.585+0000 7f6770e00640 1 -- 172.21.15.16:0/3414606052 shutdown_connections 2024-09-10T21:18:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.585+0000 7f6770e00640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f67480725e0 0x7f6748074aa0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.585+0000 7f6770e00640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c161900 0x7f676c15b420 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.585+0000 7f6770e00640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f676c161520 0x7f676c0af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.585+0000 7f6770e00640 1 --2- 172.21.15.16:0/3414606052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f676c0b2290 0x7f676c1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.585+0000 7f6770e00640 1 -- 172.21.15.16:0/3414606052 >> 172.21.15.16:0/3414606052 conn(0x7f676c0d1d80 msgr2=0x7f676c0d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.585+0000 7f6770e00640 1 -- 172.21.15.16:0/3414606052 shutdown_connections 2024-09-10T21:18:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.585+0000 7f6770e00640 1 -- 172.21.15.16:0/3414606052 wait complete. 2024-09-10T21:18:04.608 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:18:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 -- 172.21.15.16:0/557822750 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76e80706f0 msgr2=0x7f76e8165780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 --2- 172.21.15.16:0/557822750 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76e80706f0 0x7f76e8165780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009c20 tx=0x7f76e001cae0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 -- 172.21.15.16:0/557822750 shutdown_connections 2024-09-10T21:18:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 --2- 172.21.15.16:0/557822750 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76e80706f0 0x7f76e8165780 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 -- 172.21.15.16:0/557822750 >> 172.21.15.16:0/557822750 conn(0x7f76e80cfb20 msgr2=0x7f76e80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 -- 172.21.15.16:0/557822750 shutdown_connections 2024-09-10T21:18:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 -- 172.21.15.16:0/557822750 wait complete. 2024-09-10T21:18:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 Processor -- start 2024-09-10T21:18:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 -- start start 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76e80706f0 0x7f76e80c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76e8155000 0x7f76e81579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76e815bbb0 0x7f76e813c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8168cf0 con 0x7f76e8155000 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8168b70 con 0x7f76e815bbb0 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76ee000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f76e8168e70 con 0x7f76e80706f0 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76dea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76e8155000 0x7f76e81579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76dea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76e8155000 0x7f76e81579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52064/0 (socket says 172.21.15.16:52064) 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76dea00640 1 -- 172.21.15.16:0/3344029406 learned_addr learned my addr 172.21.15.16:0/3344029406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76e7400640 1 --2- 172.21.15.16:0/3344029406 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76e80706f0 0x7f76e80c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76e7e00640 1 --2- 172.21.15.16:0/3344029406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76e815bbb0 0x7f76e813c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76dea00640 1 -- 172.21.15.16:0/3344029406 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76e80706f0 msgr2=0x7f76e80c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76dea00640 1 --2- 172.21.15.16:0/3344029406 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76e80706f0 0x7f76e80c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76dea00640 1 -- 172.21.15.16:0/3344029406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76e815bbb0 msgr2=0x7f76e813c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76dea00640 1 --2- 172.21.15.16:0/3344029406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76e815bbb0 0x7f76e813c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76dea00640 1 -- 172.21.15.16:0/3344029406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e80c9ad0 con 0x7f76e8155000 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76e7400640 1 --2- 172.21.15.16:0/3344029406 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76e80706f0 0x7f76e80c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.753+0000 7f76e7e00640 1 --2- 172.21.15.16:0/3344029406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76e815bbb0 0x7f76e813c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.757+0000 7f76dea00640 1 --2- 172.21.15.16:0/3344029406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76e8155000 0x7f76e81579d0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f76d000c400 tx=0x7f76d000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.757+0000 7f76dfe00640 1 -- 172.21.15.16:0/3344029406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0017070 con 0x7f76e8155000 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.757+0000 7f76dfe00640 1 -- 172.21.15.16:0/3344029406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76d0003600 con 0x7f76e8155000 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.757+0000 7f76dfe00640 1 -- 172.21.15.16:0/3344029406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0003a50 con 0x7f76e8155000 2024-09-10T21:18:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.757+0000 7f76ee000640 1 -- 172.21.15.16:0/3344029406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e80c9d70 con 0x7f76e8155000 2024-09-10T21:18:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.757+0000 7f76ee000640 1 -- 172.21.15.16:0/3344029406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e80a9800 con 0x7f76e8155000 2024-09-10T21:18:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.757+0000 7f76ee000640 1 -- 172.21.15.16:0/3344029406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e8072d40 con 0x7f76e8155000 2024-09-10T21:18:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.757+0000 7f76dfe00640 1 -- 172.21.15.16:0/3344029406 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f76d0003bf0 con 0x7f76e8155000 2024-09-10T21:18:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.757+0000 7f76dfe00640 1 --2- 172.21.15.16:0/3344029406 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76c40725e0 0x7f76c4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.757+0000 7f76dfe00640 1 -- 172.21.15.16:0/3344029406 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f76d00928d0 con 0x7f76e8155000 2024-09-10T21:18:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.757+0000 7f76e7400640 1 --2- 172.21.15.16:0/3344029406 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76c40725e0 0x7f76c4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.761+0000 7f76e7400640 1 --2- 172.21.15.16:0/3344029406 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76c40725e0 0x7f76c4074aa0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f76d8001a20 tx=0x7f76d8009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:04.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.761+0000 7f76dfe00640 1 -- 172.21.15.16:0/3344029406 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f76d005df10 con 0x7f76e8155000 2024-09-10T21:18:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.977+0000 7f76ee000640 1 -- 172.21.15.16:0/3344029406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f76e8069970 con 0x7f76e8155000 2024-09-10T21:18:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:04.977+0000 7f76dfe00640 1 -- 172.21.15.16:0/3344029406 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v84) ==== 316+0+0 (secure 0 0 0) 0x7f76d005d810 con 0x7f76e8155000 2024-09-10T21:18:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-09-10T21:18:04.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:18:04.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-09-10T21:18:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.137+0000 7fecd3c00640 1 -- 172.21.15.16:0/1001025701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0004810 msgr2=0x7fecb0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.137+0000 7fecd3c00640 1 --2- 172.21.15.16:0/1001025701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0004810 0x7fecb0004c90 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fecb40016f0 tx=0x7fecb401c750 comp rx=0 tx=0).stop 2024-09-10T21:18:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.137+0000 7fecd3c00640 1 -- 172.21.15.16:0/1001025701 shutdown_connections 2024-09-10T21:18:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.137+0000 7fecd3c00640 1 --2- 172.21.15.16:0/1001025701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0004810 0x7fecb0004c90 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.137+0000 7fecd3c00640 1 -- 172.21.15.16:0/1001025701 >> 172.21.15.16:0/1001025701 conn(0x7feccc0d1df0 msgr2=0x7feccc0d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.137+0000 7fecd3c00640 1 -- 172.21.15.16:0/1001025701 shutdown_connections 2024-09-10T21:18:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.137+0000 7fecd3c00640 1 -- 172.21.15.16:0/1001025701 wait complete. 2024-09-10T21:18:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecd3c00640 1 Processor -- start 2024-09-10T21:18:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecd3c00640 1 -- start start 2024-09-10T21:18:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecd3c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecb0004810 0x7feccc1f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecd3c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feccc1e9ee0 0x7feccc0b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecd3c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feccc1f5630 0x7feccc070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecd3c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feccc0d2a90 con 0x7feccc1e9ee0 2024-09-10T21:18:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecd3c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feccc0d2910 con 0x7fecb0004810 2024-09-10T21:18:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecd3c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7feccc0b12e0 con 0x7feccc1f5630 2024-09-10T21:18:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecc3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feccc1e9ee0 0x7feccc0b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecc3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feccc1e9ee0 0x7feccc0b73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52082/0 (socket says 172.21.15.16:52082) 2024-09-10T21:18:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecc3e00640 1 -- 172.21.15.16:0/3808692768 learned_addr learned my addr 172.21.15.16:0/3808692768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecd1200640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecb0004810 0x7feccc1f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecd1c00640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feccc1f5630 0x7feccc070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecc3e00640 1 -- 172.21.15.16:0/3808692768 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feccc1f5630 msgr2=0x7feccc070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecc3e00640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feccc1f5630 0x7feccc070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecc3e00640 1 -- 172.21.15.16:0/3808692768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecb0004810 msgr2=0x7feccc1f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecc3e00640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecb0004810 0x7feccc1f8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecc3e00640 1 -- 172.21.15.16:0/3808692768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc1f4e50 con 0x7feccc1e9ee0 2024-09-10T21:18:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecc3e00640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feccc1e9ee0 0x7feccc0b73c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fecb401ae50 tx=0x7fecb4006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecc9600640 1 -- 172.21.15.16:0/3808692768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4003680 con 0x7feccc1e9ee0 2024-09-10T21:18:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecc9600640 1 -- 172.21.15.16:0/3808692768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecb4003910 con 0x7feccc1e9ee0 2024-09-10T21:18:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecc9600640 1 -- 172.21.15.16:0/3808692768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4003d20 con 0x7feccc1e9ee0 2024-09-10T21:18:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecd3c00640 1 -- 172.21.15.16:0/3808692768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc1f8980 con 0x7feccc1e9ee0 2024-09-10T21:18:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.141+0000 7fecd3c00640 1 -- 172.21.15.16:0/3808692768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc1f8f30 con 0x7feccc1e9ee0 2024-09-10T21:18:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.145+0000 7fecd3c00640 1 -- 172.21.15.16:0/3808692768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc0ceec0 con 0x7feccc1e9ee0 2024-09-10T21:18:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.145+0000 7fecc9600640 1 -- 172.21.15.16:0/3808692768 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fecb40b5020 con 0x7feccc1e9ee0 2024-09-10T21:18:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.145+0000 7fecc9600640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feca80725e0 0x7feca8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.145+0000 7fecc9600640 1 -- 172.21.15.16:0/3808692768 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fecb412c920 con 0x7feccc1e9ee0 2024-09-10T21:18:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.145+0000 7fecd1200640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feca80725e0 0x7feca8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.145+0000 7fecd1200640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feca80725e0 0x7feca8074aa0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fecbc001ec0 tx=0x7fecbc001d00 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:05.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.149+0000 7fecc9600640 1 -- 172.21.15.16:0/3808692768 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fecb40bf070 con 0x7feccc1e9ee0 2024-09-10T21:18:05.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.365+0000 7fecd3c00640 1 -- 172.21.15.16:0/3808692768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7feccc0b0430 con 0x7feccc1e9ee0 2024-09-10T21:18:05.368 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-10T21:18:05.365+0000 7ff429400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:18:05.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.365+0000 7fecc9600640 1 -- 172.21.15.16:0/3808692768 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v85) ==== 191+0+0 (secure 0 0 0) 0x7fecb40164b0 con 0x7feccc1e9ee0 2024-09-10T21:18:05.370 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:18:05.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.369+0000 7fecd3c00640 1 -- 172.21.15.16:0/3808692768 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feca80725e0 msgr2=0x7feca8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:05.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.369+0000 7fecd3c00640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feca80725e0 0x7feca8074aa0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fecbc001ec0 tx=0x7fecbc001d00 comp rx=0 tx=0).stop 2024-09-10T21:18:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.369+0000 7fecd3c00640 1 -- 172.21.15.16:0/3808692768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feccc1e9ee0 msgr2=0x7feccc0b73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.369+0000 7fecd3c00640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feccc1e9ee0 0x7feccc0b73c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fecb401ae50 tx=0x7fecb4006950 comp rx=0 tx=0).stop 2024-09-10T21:18:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.369+0000 7fecd3c00640 1 -- 172.21.15.16:0/3808692768 shutdown_connections 2024-09-10T21:18:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.369+0000 7fecd3c00640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feca80725e0 0x7feca8074aa0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.369+0000 7fecd3c00640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feccc1f5630 0x7feccc070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.369+0000 7fecd3c00640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feccc1e9ee0 0x7feccc0b73c0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.369+0000 7fecd3c00640 1 --2- 172.21.15.16:0/3808692768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecb0004810 0x7feccc1f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.369+0000 7fecd3c00640 1 -- 172.21.15.16:0/3808692768 >> 172.21.15.16:0/3808692768 conn(0x7feccc0d1df0 msgr2=0x7feccc158430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.369+0000 7fecd3c00640 1 -- 172.21.15.16:0/3808692768 shutdown_connections 2024-09-10T21:18:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.369+0000 7fecd3c00640 1 -- 172.21.15.16:0/3808692768 wait complete. 2024-09-10T21:18:05.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:18:05.451 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-10T21:18:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.553+0000 7ff727600640 1 -- 172.21.15.16:0/2525381010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72013d020 msgr2=0x7ff72013f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.553+0000 7ff727600640 1 --2- 172.21.15.16:0/2525381010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72013d020 0x7ff72013f590 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff718009800 tx=0x7ff71801ca70 comp rx=0 tx=0).stop 2024-09-10T21:18:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.553+0000 7ff727600640 1 -- 172.21.15.16:0/2525381010 shutdown_connections 2024-09-10T21:18:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.553+0000 7ff727600640 1 --2- 172.21.15.16:0/2525381010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72013d020 0x7ff72013f590 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.553+0000 7ff727600640 1 --2- 172.21.15.16:0/2525381010 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff720072910 0x7ff72013d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.553+0000 7ff727600640 1 -- 172.21.15.16:0/2525381010 >> 172.21.15.16:0/2525381010 conn(0x7ff7200b16a0 msgr2=0x7ff7200b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.553+0000 7ff727600640 1 -- 172.21.15.16:0/2525381010 shutdown_connections 2024-09-10T21:18:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.553+0000 7ff727600640 1 -- 172.21.15.16:0/2525381010 wait complete. 2024-09-10T21:18:05.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.553+0000 7ff727600640 1 Processor -- start 2024-09-10T21:18:05.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.553+0000 7ff727600640 1 -- start start 2024-09-10T21:18:05.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff727600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff720072910 0x7ff7201fbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:05.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff727600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff720072cf0 0x7ff7200ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff727600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72013d020 0x7ff7201380b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff727600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201673d0 con 0x7ff72013d020 2024-09-10T21:18:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff727600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff720167250 con 0x7ff720072910 2024-09-10T21:18:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff727600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff720167550 con 0x7ff720072cf0 2024-09-10T21:18:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff725600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72013d020 0x7ff7201380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff725600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72013d020 0x7ff7201380b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52118/0 (socket says 172.21.15.16:52118) 2024-09-10T21:18:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff725600640 1 -- 172.21.15.16:0/3514535827 learned_addr learned my addr 172.21.15.16:0/3514535827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff724c00640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff720072910 0x7ff7201fbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff71fe00640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff720072cf0 0x7ff7200ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff725600640 1 -- 172.21.15.16:0/3514535827 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff720072cf0 msgr2=0x7ff7200ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff725600640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff720072cf0 0x7ff7200ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff725600640 1 -- 172.21.15.16:0/3514535827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff720072910 msgr2=0x7ff7201fbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff725600640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff720072910 0x7ff7201fbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff725600640 1 -- 172.21.15.16:0/3514535827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7200aebc0 con 0x7ff72013d020 2024-09-10T21:18:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff725600640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72013d020 0x7ff7201380b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff718009750 tx=0x7ff718003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff71cc00640 1 -- 172.21.15.16:0/3514535827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7180bc070 con 0x7ff72013d020 2024-09-10T21:18:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff71cc00640 1 -- 172.21.15.16:0/3514535827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff718003e20 con 0x7ff72013d020 2024-09-10T21:18:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff71cc00640 1 -- 172.21.15.16:0/3514535827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7180b2040 con 0x7ff72013d020 2024-09-10T21:18:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff727600640 1 -- 172.21.15.16:0/3514535827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7200aeea0 con 0x7ff72013d020 2024-09-10T21:18:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff727600640 1 -- 172.21.15.16:0/3514535827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff720151ca0 con 0x7ff72013d020 2024-09-10T21:18:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.557+0000 7ff727600640 1 -- 172.21.15.16:0/3514535827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff72013d570 con 0x7ff72013d020 2024-09-10T21:18:05.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.561+0000 7ff71cc00640 1 -- 172.21.15.16:0/3514535827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff7180b5070 con 0x7ff72013d020 2024-09-10T21:18:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.561+0000 7ff71cc00640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff7000725e0 0x7ff700074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.561+0000 7ff71cc00640 1 -- 172.21.15.16:0/3514535827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5795+0+0 (secure 0 0 0) 0x7ff71812c990 con 0x7ff72013d020 2024-09-10T21:18:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.561+0000 7ff724c00640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff7000725e0 0x7ff700074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.561+0000 7ff724c00640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff7000725e0 0x7ff700074aa0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff7201542d0 tx=0x7ff71400b490 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:05.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.565+0000 7ff71cc00640 1 -- 172.21.15.16:0/3514535827 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff718016720 con 0x7ff72013d020 2024-09-10T21:18:05.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:05.777+0000 7ff727600640 1 -- 172.21.15.16:0/3514535827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff72013c0a0 con 0x7ff72013d020 2024-09-10T21:18:06.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.373+0000 7ff71cc00640 1 -- 172.21.15.16:0/3514535827 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v86) ==== 104+0+0 (secure 0 0 0) 0x7ff7180f79b0 con 0x7ff72013d020 2024-09-10T21:18:06.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.373+0000 7ff727600640 1 -- 172.21.15.16:0/3514535827 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff7000725e0 msgr2=0x7ff700074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:06.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.373+0000 7ff727600640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff7000725e0 0x7ff700074aa0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff7201542d0 tx=0x7ff71400b490 comp rx=0 tx=0).stop 2024-09-10T21:18:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.373+0000 7ff727600640 1 -- 172.21.15.16:0/3514535827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72013d020 msgr2=0x7ff7201380b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.373+0000 7ff727600640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72013d020 0x7ff7201380b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff718009750 tx=0x7ff718003c30 comp rx=0 tx=0).stop 2024-09-10T21:18:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.373+0000 7ff727600640 1 -- 172.21.15.16:0/3514535827 shutdown_connections 2024-09-10T21:18:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.373+0000 7ff727600640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff7000725e0 0x7ff700074aa0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.373+0000 7ff727600640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72013d020 0x7ff7201380b0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.373+0000 7ff727600640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff720072cf0 0x7ff7200ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.373+0000 7ff727600640 1 --2- 172.21.15.16:0/3514535827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff720072910 0x7ff7201fbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.373+0000 7ff727600640 1 -- 172.21.15.16:0/3514535827 >> 172.21.15.16:0/3514535827 conn(0x7ff7200b16a0 msgr2=0x7ff7200b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.377+0000 7ff727600640 1 -- 172.21.15.16:0/3514535827 shutdown_connections 2024-09-10T21:18:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.377+0000 7ff727600640 1 -- 172.21.15.16:0/3514535827 wait complete. 2024-09-10T21:18:06.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:18:06.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.557+0000 7f3811400640 1 -- 172.21.15.16:0/1215080172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f00050e0 msgr2=0x7f37f0005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.557+0000 7f3811400640 1 --2- 172.21.15.16:0/1215080172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f00050e0 0x7f37f0005920 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f37f4006f90 tx=0x7f37f401cee0 comp rx=0 tx=0).stop 2024-09-10T21:18:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.557+0000 7f380b200640 1 -- 172.21.15.16:0/1215080172 reap_dead start 2024-09-10T21:18:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.557+0000 7f3811400640 1 -- 172.21.15.16:0/1215080172 shutdown_connections 2024-09-10T21:18:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.557+0000 7f3811400640 1 -- 172.21.15.16:0/1215080172 >> 172.21.15.16:0/1215080172 conn(0x7f380c073450 msgr2=0x7f380c0d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.557+0000 7f3811400640 1 -- 172.21.15.16:0/1215080172 shutdown_connections 2024-09-10T21:18:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.557+0000 7f3811400640 1 -- 172.21.15.16:0/1215080172 wait complete. 2024-09-10T21:18:06.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3811400640 1 Processor -- start 2024-09-10T21:18:06.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3811400640 1 -- start start 2024-09-10T21:18:06.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3811400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f00050e0 0x7f380c0c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:06.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3811400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f380c1f5540 0x7f380c0c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:06.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3811400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f380c1e8510 0x7f380c1e89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:06.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3811400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f380c0b0b40 con 0x7f37f00050e0 2024-09-10T21:18:06.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3811400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f380c0b09c0 con 0x7f380c1f5540 2024-09-10T21:18:06.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3811400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f380c0b0cc0 con 0x7f380c1e8510 2024-09-10T21:18:06.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f380a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f00050e0 0x7f380c0c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:06.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f380a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f00050e0 0x7f380c0c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52146/0 (socket says 172.21.15.16:52146) 2024-09-10T21:18:06.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f380a800640 1 -- 172.21.15.16:0/597820538 learned_addr learned my addr 172.21.15.16:0/597820538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:06.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f380b200640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f380c1e8510 0x7f380c1e89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:06.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3801e00640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f380c1f5540 0x7f380c0c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:06.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f380a800640 1 -- 172.21.15.16:0/597820538 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f380c1e8510 msgr2=0x7f380c1e89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:06.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f380a800640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f380c1e8510 0x7f380c1e89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:06.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f380a800640 1 -- 172.21.15.16:0/597820538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f380c1f5540 msgr2=0x7f380c0c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:06.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f380a800640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f380c1f5540 0x7f380c0c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:06.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f380a800640 1 -- 172.21.15.16:0/597820538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c0c3f20 con 0x7f37f00050e0 2024-09-10T21:18:06.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f380b200640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f380c1e8510 0x7f380c1e89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:06.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3801e00640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f380c1f5540 0x7f380c0c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:06.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f380a800640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f00050e0 0x7f380c0c6980 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f37fc011a50 tx=0x7f37fc011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:06.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3803400640 1 -- 172.21.15.16:0/597820538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc01f070 con 0x7f37f00050e0 2024-09-10T21:18:06.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3803400640 1 -- 172.21.15.16:0/597820538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37fc00fe20 con 0x7f37f00050e0 2024-09-10T21:18:06.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3803400640 1 -- 172.21.15.16:0/597820538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc012690 con 0x7f37f00050e0 2024-09-10T21:18:06.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3811400640 1 -- 172.21.15.16:0/597820538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c0c4200 con 0x7f37f00050e0 2024-09-10T21:18:06.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3811400640 1 -- 172.21.15.16:0/597820538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c0c47c0 con 0x7f37f00050e0 2024-09-10T21:18:06.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.561+0000 7f3811400640 1 -- 172.21.15.16:0/597820538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f380c0c70c0 con 0x7f37f00050e0 2024-09-10T21:18:06.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.565+0000 7f3803400640 1 -- 172.21.15.16:0/597820538 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f37fc012830 con 0x7f37f00050e0 2024-09-10T21:18:06.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.565+0000 7f3803400640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f37e8072610 0x7f37e8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:06.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.565+0000 7f3803400640 1 -- 172.21.15.16:0/597820538 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5795+0+0 (secure 0 0 0) 0x7f37fc094110 con 0x7f37f00050e0 2024-09-10T21:18:06.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.565+0000 7f3801e00640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f37e8072610 0x7f37e8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:06.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.565+0000 7f3801e00640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f37e8072610 0x7f37e8074ad0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f37f40aa000 tx=0x7f37f4019d20 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.569+0000 7f3803400640 1 -- 172.21.15.16:0/597820538 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f37fc05f0a0 con 0x7f37f00050e0 2024-09-10T21:18:06.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:06.745+0000 7f3811400640 1 -- 172.21.15.16:0/597820538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f380c15cd90 con 0x7f37f00050e0 2024-09-10T21:18:07.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.373+0000 7f3803400640 1 -- 172.21.15.16:0/597820538 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v118) ==== 181+0+0 (secure 0 0 0) 0x7f37fc063ad0 con 0x7f37f00050e0 2024-09-10T21:18:07.378 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:18:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.377+0000 7f3811400640 1 -- 172.21.15.16:0/597820538 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f37e8072610 msgr2=0x7f37e8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.377+0000 7f3811400640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f37e8072610 0x7f37e8074ad0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f37f40aa000 tx=0x7f37f4019d20 comp rx=0 tx=0).stop 2024-09-10T21:18:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.377+0000 7f3811400640 1 -- 172.21.15.16:0/597820538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f00050e0 msgr2=0x7f380c0c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.377+0000 7f3811400640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f00050e0 0x7f380c0c6980 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f37fc011a50 tx=0x7f37fc011f10 comp rx=0 tx=0).stop 2024-09-10T21:18:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.377+0000 7f3811400640 1 -- 172.21.15.16:0/597820538 shutdown_connections 2024-09-10T21:18:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.377+0000 7f3811400640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f37e8072610 0x7f37e8074ad0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:07.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.377+0000 7f3811400640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f380c1e8510 0x7f380c1e89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:07.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.377+0000 7f3811400640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f380c1f5540 0x7f380c0c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:07.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.377+0000 7f3811400640 1 --2- 172.21.15.16:0/597820538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f00050e0 0x7f380c0c6980 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:07.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.377+0000 7f3811400640 1 -- 172.21.15.16:0/597820538 >> 172.21.15.16:0/597820538 conn(0x7f380c073450 msgr2=0x7f380c0cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:07.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.377+0000 7f3811400640 1 -- 172.21.15.16:0/597820538 shutdown_connections 2024-09-10T21:18:07.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.377+0000 7f3811400640 1 -- 172.21.15.16:0/597820538 wait complete. 2024-09-10T21:18:07.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:18:07.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.557+0000 7f727cc00640 1 -- 172.21.15.16:0/2225017610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727815a480 msgr2=0x7f72780b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:07.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.557+0000 7f727cc00640 1 --2- 172.21.15.16:0/2225017610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727815a480 0x7f72780b2b10 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f7278074bc0 tx=0x7f726401db30 comp rx=0 tx=0).stop 2024-09-10T21:18:07.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.557+0000 7f727cc00640 1 -- 172.21.15.16:0/2225017610 shutdown_connections 2024-09-10T21:18:07.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.557+0000 7f727cc00640 1 --2- 172.21.15.16:0/2225017610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727815a480 0x7f72780b2b10 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.557+0000 7f727cc00640 1 --2- 172.21.15.16:0/2225017610 >> v2:172.21.15.52:3300/0 conn(0x7f727815a0a0 0x7f72780cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.557+0000 7f727cc00640 1 -- 172.21.15.16:0/2225017610 >> 172.21.15.16:0/2225017610 conn(0x7f727806d1e0 msgr2=0x7f72780b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.557+0000 7f727cc00640 1 -- 172.21.15.16:0/2225017610 shutdown_connections 2024-09-10T21:18:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.557+0000 7f727cc00640 1 -- 172.21.15.16:0/2225017610 wait complete. 2024-09-10T21:18:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f727cc00640 1 Processor -- start 2024-09-10T21:18:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f727cc00640 1 -- start start 2024-09-10T21:18:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f727cc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727806e490 0x7f7278155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f727cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727815a0a0 0x7f72781f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f727cc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f727815a480 0x7f7278164290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f727cc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72780ce460 con 0x7f727815a0a0 2024-09-10T21:18:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f727cc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f72780ce2e0 con 0x7f727806e490 2024-09-10T21:18:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f727cc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f72780ce5e0 con 0x7f727815a480 2024-09-10T21:18:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f7275400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727815a0a0 0x7f72781f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f7275400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727815a0a0 0x7f72781f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52162/0 (socket says 172.21.15.16:52162) 2024-09-10T21:18:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f7275400640 1 -- 172.21.15.16:0/886240167 learned_addr learned my addr 172.21.15.16:0/886240167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f7276800640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f727815a480 0x7f7278164290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:07.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f7275e00640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727806e490 0x7f7278155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f7275400640 1 -- 172.21.15.16:0/886240167 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f727815a480 msgr2=0x7f7278164290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f7275400640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f727815a480 0x7f7278164290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f7275400640 1 -- 172.21.15.16:0/886240167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727806e490 msgr2=0x7f7278155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f7275400640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727806e490 0x7f7278155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f7275400640 1 -- 172.21.15.16:0/886240167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7278161150 con 0x7f727815a0a0 2024-09-10T21:18:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f7275e00640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727806e490 0x7f7278155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f7275400640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727815a0a0 0x7f72781f4ec0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f72640b1040 tx=0x7f7264004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f726e000640 1 -- 172.21.15.16:0/886240167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72640b3070 con 0x7f727815a0a0 2024-09-10T21:18:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f726e000640 1 -- 172.21.15.16:0/886240167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7264005530 con 0x7f727815a0a0 2024-09-10T21:18:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f726e000640 1 -- 172.21.15.16:0/886240167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264004930 con 0x7f727815a0a0 2024-09-10T21:18:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f727cc00640 1 -- 172.21.15.16:0/886240167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7278163340 con 0x7f727815a0a0 2024-09-10T21:18:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f727cc00640 1 -- 172.21.15.16:0/886240167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7278162d80 con 0x7f727815a0a0 2024-09-10T21:18:07.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.561+0000 7f727cc00640 1 -- 172.21.15.16:0/886240167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72780cf5f0 con 0x7f727815a0a0 2024-09-10T21:18:07.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.565+0000 7f726e000640 1 -- 172.21.15.16:0/886240167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f726400b040 con 0x7f727815a0a0 2024-09-10T21:18:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.565+0000 7f726e000640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72580725e0 0x7f7258074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.565+0000 7f726e000640 1 -- 172.21.15.16:0/886240167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5357+0+0 (secure 0 0 0) 0x7f726412d280 con 0x7f727815a0a0 2024-09-10T21:18:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.565+0000 7f7275e00640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72580725e0 0x7f7258074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:07.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.565+0000 7f7275e00640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72580725e0 0x7f7258074aa0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7268005d10 tx=0x7f7268005c80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:07.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.569+0000 7f726e000640 1 -- 172.21.15.16:0/886240167 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f72640177e0 con 0x7f727815a0a0 2024-09-10T21:18:07.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:07.741+0000 7f727cc00640 1 -- 172.21.15.16:0/886240167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7278161320 con 0x7f727815a0a0 2024-09-10T21:18:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.381+0000 7f726e000640 1 -- 172.21.15.16:0/886240167 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v119) ==== 169+0+0 (secure 0 0 0) 0x7f72640f8ab0 con 0x7f727815a0a0 2024-09-10T21:18:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-10T21:18:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.385+0000 7f727cc00640 1 -- 172.21.15.16:0/886240167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72580725e0 msgr2=0x7f7258074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.385+0000 7f727cc00640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72580725e0 0x7f7258074aa0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7268005d10 tx=0x7f7268005c80 comp rx=0 tx=0).stop 2024-09-10T21:18:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.385+0000 7f727cc00640 1 -- 172.21.15.16:0/886240167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727815a0a0 msgr2=0x7f72781f4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.385+0000 7f727cc00640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727815a0a0 0x7f72781f4ec0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f72640b1040 tx=0x7f7264004600 comp rx=0 tx=0).stop 2024-09-10T21:18:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.389+0000 7f727cc00640 1 -- 172.21.15.16:0/886240167 shutdown_connections 2024-09-10T21:18:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.389+0000 7f727cc00640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f72580725e0 0x7f7258074aa0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.389+0000 7f727cc00640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f727815a480 0x7f7278164290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.389+0000 7f727cc00640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727815a0a0 0x7f72781f4ec0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.389+0000 7f727cc00640 1 --2- 172.21.15.16:0/886240167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727806e490 0x7f7278155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.389+0000 7f727cc00640 1 -- 172.21.15.16:0/886240167 >> 172.21.15.16:0/886240167 conn(0x7f727806d1e0 msgr2=0x7f72781552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.389+0000 7f727cc00640 1 -- 172.21.15.16:0/886240167 shutdown_connections 2024-09-10T21:18:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.389+0000 7f727cc00640 1 -- 172.21.15.16:0/886240167 wait complete. 2024-09-10T21:18:08.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:18:08.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.569+0000 7f8c37e00640 1 -- 172.21.15.16:0/988994500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c380ce350 msgr2=0x7f8c38070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.569+0000 7f8c37e00640 1 --2- 172.21.15.16:0/988994500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c380ce350 0x7f8c38070440 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009bc0 tx=0x7f8c3001cb70 comp rx=0 tx=0).stop 2024-09-10T21:18:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.569+0000 7f8c37e00640 1 -- 172.21.15.16:0/988994500 shutdown_connections 2024-09-10T21:18:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.569+0000 7f8c37e00640 1 --2- 172.21.15.16:0/988994500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c380ce350 0x7f8c38070440 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.569+0000 7f8c37e00640 1 -- 172.21.15.16:0/988994500 >> 172.21.15.16:0/988994500 conn(0x7f8c380cc8e0 msgr2=0x7f8c380cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.569+0000 7f8c37e00640 1 -- 172.21.15.16:0/988994500 shutdown_connections 2024-09-10T21:18:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.569+0000 7f8c37e00640 1 -- 172.21.15.16:0/988994500 wait complete. 2024-09-10T21:18:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c37e00640 1 Processor -- start 2024-09-10T21:18:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c37e00640 1 -- start start 2024-09-10T21:18:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c37e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c380cd550 0x7f8c38149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c37e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c380ce350 0x7f8c38072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c37e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c38154780 0x7f8c38072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c37e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c38168fd0 con 0x7f8c380cd550 2024-09-10T21:18:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c37e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8c380ada20 con 0x7f8c38154780 2024-09-10T21:18:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c37e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8c38169150 con 0x7f8c380ce350 2024-09-10T21:18:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c36a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c380cd550 0x7f8c38149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c36a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c380cd550 0x7f8c38149a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52178/0 (socket says 172.21.15.16:52178) 2024-09-10T21:18:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c36a00640 1 -- 172.21.15.16:0/1191423556 learned_addr learned my addr 172.21.15.16:0/1191423556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c37400640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c38154780 0x7f8c38072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c2e000640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c380ce350 0x7f8c38072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c36a00640 1 -- 172.21.15.16:0/1191423556 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c380ce350 msgr2=0x7f8c38072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c36a00640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c380ce350 0x7f8c38072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c36a00640 1 -- 172.21.15.16:0/1191423556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c38154780 msgr2=0x7f8c38072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c36a00640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c38154780 0x7f8c38072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c36a00640 1 -- 172.21.15.16:0/1191423556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c38149f40 con 0x7f8c380cd550 2024-09-10T21:18:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c37400640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c38154780 0x7f8c38072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c36a00640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c380cd550 0x7f8c38149a00 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800c430 tx=0x7f8c2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c2f400640 1 -- 172.21.15.16:0/1191423556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28011790 con 0x7f8c380cd550 2024-09-10T21:18:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c2f400640 1 -- 172.21.15.16:0/1191423556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c28017070 con 0x7f8c380cd550 2024-09-10T21:18:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c2f400640 1 -- 172.21.15.16:0/1191423556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28011ce0 con 0x7f8c380cd550 2024-09-10T21:18:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c37e00640 1 -- 172.21.15.16:0/1191423556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c380cb810 con 0x7f8c380cd550 2024-09-10T21:18:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c37e00640 1 -- 172.21.15.16:0/1191423556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c3814d8e0 con 0x7f8c380cd550 2024-09-10T21:18:08.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.573+0000 7f8c37e00640 1 -- 172.21.15.16:0/1191423556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c3814d6d0 con 0x7f8c380cd550 2024-09-10T21:18:08.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.577+0000 7f8c2f400640 1 -- 172.21.15.16:0/1191423556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8c2801c460 con 0x7f8c380cd550 2024-09-10T21:18:08.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.577+0000 7f8c2f400640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8c140725e0 0x7f8c14074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:08.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.577+0000 7f8c2f400640 1 -- 172.21.15.16:0/1191423556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4943+0+0 (secure 0 0 0) 0x7f8c28092d90 con 0x7f8c380cd550 2024-09-10T21:18:08.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.577+0000 7f8c2e000640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8c140725e0 0x7f8c14074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:08.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.577+0000 7f8c2e000640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8c140725e0 0x7f8c14074aa0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8c38152fb0 tx=0x7f8c20009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:08.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.581+0000 7f8c2f400640 1 -- 172.21.15.16:0/1191423556 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8c2805e790 con 0x7f8c380cd550 2024-09-10T21:18:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.757+0000 7f8c37e00640 1 -- 172.21.15.16:0/1191423556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f8c38070440 con 0x7f8c380cd550 2024-09-10T21:18:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.765+0000 7f8c2f400640 1 -- 172.21.15.16:0/1191423556 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v58) ==== 71+0+0 (secure 0 0 0) 0x7f8c2805e090 con 0x7f8c380cd550 2024-09-10T21:18:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.765+0000 7f8c37e00640 1 -- 172.21.15.16:0/1191423556 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8c140725e0 msgr2=0x7f8c14074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.765+0000 7f8c37e00640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8c140725e0 0x7f8c14074aa0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8c38152fb0 tx=0x7f8c20009040 comp rx=0 tx=0).stop 2024-09-10T21:18:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.765+0000 7f8c37e00640 1 -- 172.21.15.16:0/1191423556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c380cd550 msgr2=0x7f8c38149a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.765+0000 7f8c37e00640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c380cd550 0x7f8c38149a00 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800c430 tx=0x7f8c2800c8f0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.765+0000 7f8c37e00640 1 -- 172.21.15.16:0/1191423556 shutdown_connections 2024-09-10T21:18:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.765+0000 7f8c37e00640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8c140725e0 0x7f8c14074aa0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.765+0000 7f8c37e00640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c38154780 0x7f8c38072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.765+0000 7f8c37e00640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c380ce350 0x7f8c38072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.765+0000 7f8c37e00640 1 --2- 172.21.15.16:0/1191423556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c380cd550 0x7f8c38149a00 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.765+0000 7f8c37e00640 1 -- 172.21.15.16:0/1191423556 >> 172.21.15.16:0/1191423556 conn(0x7f8c380cc8e0 msgr2=0x7f8c380c6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.765+0000 7f8c37e00640 1 -- 172.21.15.16:0/1191423556 shutdown_connections 2024-09-10T21:18:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.765+0000 7f8c37e00640 1 -- 172.21.15.16:0/1191423556 wait complete. 2024-09-10T21:18:08.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2024-09-10T21:18:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.941+0000 7f36dc800640 1 -- 172.21.15.16:0/415597944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc003a60 msgr2=0x7f36bc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.941+0000 7f36dc800640 1 --2- 172.21.15.16:0/415597944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc003a60 0x7f36bc003e80 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f36c800abc0 tx=0x7f36c801cd40 comp rx=0 tx=0).stop 2024-09-10T21:18:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.941+0000 7f36dc800640 1 -- 172.21.15.16:0/415597944 shutdown_connections 2024-09-10T21:18:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.941+0000 7f36dc800640 1 --2- 172.21.15.16:0/415597944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc003a60 0x7f36bc003e80 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.941+0000 7f36dc800640 1 -- 172.21.15.16:0/415597944 >> 172.21.15.16:0/415597944 conn(0x7f36d40cfb20 msgr2=0x7f36d40cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.941+0000 7f36dc800640 1 -- 172.21.15.16:0/415597944 shutdown_connections 2024-09-10T21:18:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.941+0000 7f36dc800640 1 -- 172.21.15.16:0/415597944 wait complete. 2024-09-10T21:18:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.941+0000 7f36dc800640 1 Processor -- start 2024-09-10T21:18:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.941+0000 7f36dc800640 1 -- start start 2024-09-10T21:18:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36dc800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc003a60 0x7f36d40c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36dc800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36d41f4320 0x7f36d414ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36dc800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f36d40c6bf0 0x7f36d40c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36dc800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36d40700a0 con 0x7f36bc003a60 2024-09-10T21:18:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36dc800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f36d406ecb0 con 0x7f36d41f4320 2024-09-10T21:18:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36dc800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f36d4070220 con 0x7f36d40c6bf0 2024-09-10T21:18:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36d9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc003a60 0x7f36d40c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36d9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc003a60 0x7f36d40c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52204/0 (socket says 172.21.15.16:52204) 2024-09-10T21:18:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36d9e00640 1 -- 172.21.15.16:0/2234061783 learned_addr learned my addr 172.21.15.16:0/2234061783 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36d9e00640 1 -- 172.21.15.16:0/2234061783 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f36d40c6bf0 msgr2=0x7f36d40c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36d9e00640 1 --2- 172.21.15.16:0/2234061783 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f36d40c6bf0 0x7f36d40c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36d9e00640 1 -- 172.21.15.16:0/2234061783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36d41f4320 msgr2=0x7f36d414ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36d9e00640 1 --2- 172.21.15.16:0/2234061783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36d41f4320 0x7f36d414ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36d9e00640 1 -- 172.21.15.16:0/2234061783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d40c74f0 con 0x7f36bc003a60 2024-09-10T21:18:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36d9e00640 1 --2- 172.21.15.16:0/2234061783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc003a60 0x7f36d40c8590 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f36c800ab10 tx=0x7f36c8005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36ce000640 1 -- 172.21.15.16:0/2234061783 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c80b2040 con 0x7f36bc003a60 2024-09-10T21:18:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36ce000640 1 -- 172.21.15.16:0/2234061783 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36c80046d0 con 0x7f36bc003a60 2024-09-10T21:18:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36dc800640 1 -- 172.21.15.16:0/2234061783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d40c77d0 con 0x7f36bc003a60 2024-09-10T21:18:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36ce000640 1 -- 172.21.15.16:0/2234061783 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c8004b00 con 0x7f36bc003a60 2024-09-10T21:18:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36dc800640 1 -- 172.21.15.16:0/2234061783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d40c7d90 con 0x7f36bc003a60 2024-09-10T21:18:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36dc800640 1 -- 172.21.15.16:0/2234061783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d4070780 con 0x7f36bc003a60 2024-09-10T21:18:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.945+0000 7f36ce000640 1 -- 172.21.15.16:0/2234061783 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f36c80b5070 con 0x7f36bc003a60 2024-09-10T21:18:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.949+0000 7f36ce000640 1 --2- 172.21.15.16:0/2234061783 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f36b80725e0 0x7f36b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.949+0000 7f36ce000640 1 -- 172.21.15.16:0/2234061783 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4943+0+0 (secure 0 0 0) 0x7f36c812d1a0 con 0x7f36bc003a60 2024-09-10T21:18:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.949+0000 7f36d9400640 1 --2- 172.21.15.16:0/2234061783 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f36b80725e0 0x7f36b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.949+0000 7f36d9400640 1 --2- 172.21.15.16:0/2234061783 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f36b80725e0 0x7f36b8074aa0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f36c4006860 tx=0x7f36c40067f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:08.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:08.953+0000 7f36ce000640 1 -- 172.21.15.16:0/2234061783 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f36c8016a80 con 0x7f36bc003a60 2024-09-10T21:18:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.185+0000 7f36dc800640 1 -- 172.21.15.16:0/2234061783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) -- 0x7f36d40c33f0 con 0x7f36bc003a60 2024-09-10T21:18:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.381+0000 7f36ce000640 1 -- 172.21.15.16:0/2234061783 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v390) ==== 151+0+0 (secure 0 0 0) 0x7f36c80f8b70 con 0x7f36bc003a60 2024-09-10T21:18:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.385+0000 7f36dc800640 1 -- 172.21.15.16:0/2234061783 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f36b80725e0 msgr2=0x7f36b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.385+0000 7f36dc800640 1 --2- 172.21.15.16:0/2234061783 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f36b80725e0 0x7f36b8074aa0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f36c4006860 tx=0x7f36c40067f0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.385+0000 7f36dc800640 1 -- 172.21.15.16:0/2234061783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc003a60 msgr2=0x7f36d40c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.385+0000 7f36dc800640 1 --2- 172.21.15.16:0/2234061783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc003a60 0x7f36d40c8590 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f36c800ab10 tx=0x7f36c8005640 comp rx=0 tx=0).stop 2024-09-10T21:18:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.385+0000 7f36dc800640 1 -- 172.21.15.16:0/2234061783 shutdown_connections 2024-09-10T21:18:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.385+0000 7f36dc800640 1 --2- 172.21.15.16:0/2234061783 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f36b80725e0 0x7f36b8074aa0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.385+0000 7f36dc800640 1 --2- 172.21.15.16:0/2234061783 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f36d40c6bf0 0x7f36d40c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.385+0000 7f36dc800640 1 --2- 172.21.15.16:0/2234061783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36d41f4320 0x7f36d414ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.385+0000 7f36dc800640 1 --2- 172.21.15.16:0/2234061783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc003a60 0x7f36d40c8590 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.385+0000 7f36dc800640 1 -- 172.21.15.16:0/2234061783 >> 172.21.15.16:0/2234061783 conn(0x7f36d40cfb20 msgr2=0x7f36d4151110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.385+0000 7f36dc800640 1 -- 172.21.15.16:0/2234061783 shutdown_connections 2024-09-10T21:18:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.385+0000 7f36dc800640 1 -- 172.21.15.16:0/2234061783 wait complete. 2024-09-10T21:18:09.409 INFO:tasks.cephfs_test_runner:test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-09-10T21:18:09.409 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-10T21:18:09.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:18:09.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.553+0000 7f660f200640 1 -- 172.21.15.16:0/2882769613 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66100d2150 msgr2=0x7f66100b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.553+0000 7f660f200640 1 --2- 172.21.15.16:0/2882769613 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66100d2150 0x7f66100b1fb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6608009bc0 tx=0x7f660801cb10 comp rx=0 tx=0).stop 2024-09-10T21:18:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.553+0000 7f660f200640 1 -- 172.21.15.16:0/2882769613 shutdown_connections 2024-09-10T21:18:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.553+0000 7f660f200640 1 --2- 172.21.15.16:0/2882769613 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66100d2150 0x7f66100b1fb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.553+0000 7f660f200640 1 --2- 172.21.15.16:0/2882769613 >> v2:172.21.15.52:3300/0 conn(0x7f661015ddb0 0x7f66100cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.553+0000 7f660f200640 1 -- 172.21.15.16:0/2882769613 >> 172.21.15.16:0/2882769613 conn(0x7f6610074c50 msgr2=0x7f661009a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.553+0000 7f660f200640 1 -- 172.21.15.16:0/2882769613 shutdown_connections 2024-09-10T21:18:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.553+0000 7f660f200640 1 -- 172.21.15.16:0/2882769613 wait complete. 2024-09-10T21:18:09.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660f200640 1 Processor -- start 2024-09-10T21:18:09.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660f200640 1 -- start start 2024-09-10T21:18:09.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66100d0210 0x7f6610154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:09.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660f200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66100d2150 0x7f661015cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:09.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661015ddb0 0x7f66100af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:09.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f661016b1c0 con 0x7f66100d0210 2024-09-10T21:18:09.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f661016b040 con 0x7f661015ddb0 2024-09-10T21:18:09.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660f200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f661016b340 con 0x7f66100d2150 2024-09-10T21:18:09.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66100d0210 0x7f6610154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:09.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66100d0210 0x7f6610154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52212/0 (socket says 172.21.15.16:52212) 2024-09-10T21:18:09.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660de00640 1 -- 172.21.15.16:0/1956006319 learned_addr learned my addr 172.21.15.16:0/1956006319 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:09.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660e800640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661015ddb0 0x7f66100af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:09.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660d400640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66100d2150 0x7f661015cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:09.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660de00640 1 -- 172.21.15.16:0/1956006319 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66100d2150 msgr2=0x7f661015cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660de00640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66100d2150 0x7f661015cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660de00640 1 -- 172.21.15.16:0/1956006319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661015ddb0 msgr2=0x7f66100af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660de00640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661015ddb0 0x7f66100af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660de00640 1 -- 172.21.15.16:0/1956006319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66101545a0 con 0x7f66100d0210 2024-09-10T21:18:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660d400640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66100d2150 0x7f661015cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660de00640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66100d0210 0x7f6610154060 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f660000e7a0 tx=0x7f660000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f6606000640 1 -- 172.21.15.16:0/1956006319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660000ccc0 con 0x7f66100d0210 2024-09-10T21:18:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f6606000640 1 -- 172.21.15.16:0/1956006319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6600010430 con 0x7f66100d0210 2024-09-10T21:18:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f6606000640 1 -- 172.21.15.16:0/1956006319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600010860 con 0x7f66100d0210 2024-09-10T21:18:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660f200640 1 -- 172.21.15.16:0/1956006319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66101fb520 con 0x7f66100d0210 2024-09-10T21:18:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660f200640 1 -- 172.21.15.16:0/1956006319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66101fbae0 con 0x7f66100d0210 2024-09-10T21:18:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.557+0000 7f660f200640 1 -- 172.21.15.16:0/1956006319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66100b1fb0 con 0x7f66100d0210 2024-09-10T21:18:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.561+0000 7f6606000640 1 -- 172.21.15.16:0/1956006319 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6600010a00 con 0x7f66100d0210 2024-09-10T21:18:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.561+0000 7f6606000640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65f00725e0 0x7f65f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.561+0000 7f6606000640 1 -- 172.21.15.16:0/1956006319 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4943+0+0 (secure 0 0 0) 0x7f6600093970 con 0x7f66100d0210 2024-09-10T21:18:09.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.561+0000 7f660d400640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65f00725e0 0x7f65f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:09.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.561+0000 7f660d400640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65f00725e0 0x7f65f0074aa0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f65fc007a60 tx=0x7f65fc009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:09.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.565+0000 7f6606000640 1 -- 172.21.15.16:0/1956006319 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f660005ec70 con 0x7f66100d0210 2024-09-10T21:18:09.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.741+0000 7f660f200640 1 -- 172.21.15.16:0/1956006319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66100b3bf0 con 0x7f66100d0210 2024-09-10T21:18:09.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.745+0000 7f6606000640 1 -- 172.21.15.16:0/1956006319 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+12994 (secure 0 0 0) 0x7f66000636a0 con 0x7f66100d0210 2024-09-10T21:18:09.748 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:09.748 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":119,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:18:08.378071+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6837/1681945537":"2024-09-11T21:18:05.367462+0000","172.21.15.16:6835/1681945537":"2024-09-11T21:18:05.367462+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:18:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.745+0000 7f660f200640 1 -- 172.21.15.16:0/1956006319 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65f00725e0 msgr2=0x7f65f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.745+0000 7f660f200640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65f00725e0 0x7f65f0074aa0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f65fc007a60 tx=0x7f65fc009040 comp rx=0 tx=0).stop 2024-09-10T21:18:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.745+0000 7f660f200640 1 -- 172.21.15.16:0/1956006319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66100d0210 msgr2=0x7f6610154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.745+0000 7f660f200640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66100d0210 0x7f6610154060 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f660000e7a0 tx=0x7f660000ec60 comp rx=0 tx=0).stop 2024-09-10T21:18:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.745+0000 7f660f200640 1 -- 172.21.15.16:0/1956006319 shutdown_connections 2024-09-10T21:18:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.745+0000 7f660f200640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65f00725e0 0x7f65f0074aa0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.745+0000 7f660f200640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661015ddb0 0x7f66100af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.745+0000 7f660f200640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66100d2150 0x7f661015cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.745+0000 7f660f200640 1 --2- 172.21.15.16:0/1956006319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66100d0210 0x7f6610154060 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.745+0000 7f660f200640 1 -- 172.21.15.16:0/1956006319 >> 172.21.15.16:0/1956006319 conn(0x7f6610074c50 msgr2=0x7f66100cd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.745+0000 7f660f200640 1 -- 172.21.15.16:0/1956006319 shutdown_connections 2024-09-10T21:18:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.745+0000 7f660f200640 1 -- 172.21.15.16:0/1956006319 wait complete. 2024-09-10T21:18:09.769 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:10:17.493770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:18:09.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:18:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 -- 172.21.15.16:0/1852917174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55d40062e0 msgr2=0x7f55d400cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 --2- 172.21.15.16:0/1852917174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55d40062e0 0x7f55d400cae0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009e30 tx=0x7f55e401d1b0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f1400640 1 -- 172.21.15.16:0/1852917174 reap_dead start 2024-09-10T21:18:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 -- 172.21.15.16:0/1852917174 shutdown_connections 2024-09-10T21:18:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 -- 172.21.15.16:0/1852917174 >> 172.21.15.16:0/1852917174 conn(0x7f55ec0b16a0 msgr2=0x7f55ec0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 -- 172.21.15.16:0/1852917174 shutdown_connections 2024-09-10T21:18:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 -- 172.21.15.16:0/1852917174 wait complete. 2024-09-10T21:18:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 Processor -- start 2024-09-10T21:18:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 -- start start 2024-09-10T21:18:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55d40062e0 0x7f55ec133de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ec136720 0x7f55ec1502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55ec148ed0 0x7f55ec15ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec072830 con 0x7f55d40062e0 2024-09-10T21:18:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec0726b0 con 0x7f55ec148ed0 2024-09-10T21:18:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f3400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f55ec0729b0 con 0x7f55ec136720 2024-09-10T21:18:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f0a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55d40062e0 0x7f55ec133de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f0a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55d40062e0 0x7f55ec133de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52226/0 (socket says 172.21.15.16:52226) 2024-09-10T21:18:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f0a00640 1 -- 172.21.15.16:0/3757602042 learned_addr learned my addr 172.21.15.16:0/3757602042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f1400640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55ec148ed0 0x7f55ec15ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55ebe00640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ec136720 0x7f55ec1502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f0a00640 1 -- 172.21.15.16:0/3757602042 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ec136720 msgr2=0x7f55ec1502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f0a00640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ec136720 0x7f55ec1502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f0a00640 1 -- 172.21.15.16:0/3757602042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55ec148ed0 msgr2=0x7f55ec15ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f0a00640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55ec148ed0 0x7f55ec15ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55f0a00640 1 -- 172.21.15.16:0/3757602042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ec1f3970 con 0x7f55d40062e0 2024-09-10T21:18:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.917+0000 7f55ebe00640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ec136720 0x7f55ec1502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.921+0000 7f55f0a00640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55d40062e0 0x7f55ec133de0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f55e000f7b0 tx=0x7f55e000fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.921+0000 7f55e8c00640 1 -- 172.21.15.16:0/3757602042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e000bcb0 con 0x7f55d40062e0 2024-09-10T21:18:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.921+0000 7f55e8c00640 1 -- 172.21.15.16:0/3757602042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55e000fe60 con 0x7f55d40062e0 2024-09-10T21:18:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.921+0000 7f55e8c00640 1 -- 172.21.15.16:0/3757602042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e00116a0 con 0x7f55d40062e0 2024-09-10T21:18:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.921+0000 7f55f3400640 1 -- 172.21.15.16:0/3757602042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ec136c90 con 0x7f55d40062e0 2024-09-10T21:18:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.921+0000 7f55f3400640 1 -- 172.21.15.16:0/3757602042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ec137250 con 0x7f55d40062e0 2024-09-10T21:18:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.921+0000 7f55f3400640 1 -- 172.21.15.16:0/3757602042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55ec06ea50 con 0x7f55d40062e0 2024-09-10T21:18:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.921+0000 7f55e8c00640 1 -- 172.21.15.16:0/3757602042 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f55e0011840 con 0x7f55d40062e0 2024-09-10T21:18:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.921+0000 7f55e8c00640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55cc0725e0 0x7f55cc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.921+0000 7f55e8c00640 1 -- 172.21.15.16:0/3757602042 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4943+0+0 (secure 0 0 0) 0x7f55e0093af0 con 0x7f55d40062e0 2024-09-10T21:18:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.921+0000 7f55ebe00640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55cc0725e0 0x7f55cc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.925+0000 7f55ebe00640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55cc0725e0 0x7f55cc074aa0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f55d8001870 tx=0x7f55d8001d60 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:09.925+0000 7f55e8c00640 1 -- 172.21.15.16:0/3757602042 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f55e005ede0 con 0x7f55d40062e0 2024-09-10T21:18:10.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.101+0000 7f55f3400640 1 -- 172.21.15.16:0/3757602042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f55ec0b2c50 con 0x7f55d40062e0 2024-09-10T21:18:10.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.101+0000 7f55e8c00640 1 -- 172.21.15.16:0/3757602042 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7f55e0063810 con 0x7f55d40062e0 2024-09-10T21:18:10.106 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-10T21:18:10.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.105+0000 7f55f3400640 1 -- 172.21.15.16:0/3757602042 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55cc0725e0 msgr2=0x7f55cc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:10.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.105+0000 7f55f3400640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55cc0725e0 0x7f55cc074aa0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f55d8001870 tx=0x7f55d8001d60 comp rx=0 tx=0).stop 2024-09-10T21:18:10.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.105+0000 7f55f3400640 1 -- 172.21.15.16:0/3757602042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55d40062e0 msgr2=0x7f55ec133de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:10.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.105+0000 7f55f3400640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55d40062e0 0x7f55ec133de0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f55e000f7b0 tx=0x7f55e000fc70 comp rx=0 tx=0).stop 2024-09-10T21:18:10.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.105+0000 7f55f3400640 1 -- 172.21.15.16:0/3757602042 shutdown_connections 2024-09-10T21:18:10.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.105+0000 7f55f3400640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f55cc0725e0 0x7f55cc074aa0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:10.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.105+0000 7f55f3400640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55ec148ed0 0x7f55ec15ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:10.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.105+0000 7f55f3400640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f55ec136720 0x7f55ec1502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:10.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.105+0000 7f55f3400640 1 --2- 172.21.15.16:0/3757602042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55d40062e0 0x7f55ec133de0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:10.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.105+0000 7f55f3400640 1 -- 172.21.15.16:0/3757602042 >> 172.21.15.16:0/3757602042 conn(0x7f55ec0b16a0 msgr2=0x7f55ec0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:10.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.105+0000 7f55f3400640 1 -- 172.21.15.16:0/3757602042 shutdown_connections 2024-09-10T21:18:10.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.105+0000 7f55f3400640 1 -- 172.21.15.16:0/3757602042 wait complete. 2024-09-10T21:18:10.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2024-09-10T21:18:10.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 -- 172.21.15.16:0/575410399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c7c005220 msgr2=0x7f7c7c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 --2- 172.21.15.16:0/575410399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c7c005220 0x7f7c7c00b9d0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009eb0 tx=0x7f7c8c01d5d0 comp rx=0 tx=0).stop 2024-09-10T21:18:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 -- 172.21.15.16:0/575410399 shutdown_connections 2024-09-10T21:18:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 --2- 172.21.15.16:0/575410399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c7c005220 0x7f7c7c00b9d0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 -- 172.21.15.16:0/575410399 >> 172.21.15.16:0/575410399 conn(0x7f7c940d1d80 msgr2=0x7f7c940d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 -- 172.21.15.16:0/575410399 shutdown_connections 2024-09-10T21:18:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 -- 172.21.15.16:0/575410399 wait complete. 2024-09-10T21:18:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 Processor -- start 2024-09-10T21:18:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 -- start start 2024-09-10T21:18:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c7c005220 0x7f7c9406e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7c9414e240 0x7f7c9415b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c9414e6c0 0x7f7c9415b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c940cfc80 con 0x7f7c7c005220 2024-09-10T21:18:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7c940b2ec0 con 0x7f7c9414e6c0 2024-09-10T21:18:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9c000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7c940cfe00 con 0x7f7c9414e240 2024-09-10T21:18:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c99600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c7c005220 0x7f7c9406e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c99600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c7c005220 0x7f7c9406e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52252/0 (socket says 172.21.15.16:52252) 2024-09-10T21:18:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c99600640 1 -- 172.21.15.16:0/3428860939 learned_addr learned my addr 172.21.15.16:0/3428860939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c9a000640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c9414e6c0 0x7f7c9415b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.273+0000 7f7c98c00640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7c9414e240 0x7f7c9415b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c99600640 1 -- 172.21.15.16:0/3428860939 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7c9414e240 msgr2=0x7f7c9415b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c99600640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7c9414e240 0x7f7c9415b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c99600640 1 -- 172.21.15.16:0/3428860939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c9414e6c0 msgr2=0x7f7c9415b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c99600640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c9414e6c0 0x7f7c9415b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c99600640 1 -- 172.21.15.16:0/3428860939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c940c6f10 con 0x7f7c7c005220 2024-09-10T21:18:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c9a000640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c9414e6c0 0x7f7c9415b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c98c00640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7c9414e240 0x7f7c9415b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c99600640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c7c005220 0x7f7c9406e700 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f7c88006680 tx=0x7f7c880039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c91600640 1 -- 172.21.15.16:0/3428860939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c88015070 con 0x7f7c7c005220 2024-09-10T21:18:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c91600640 1 -- 172.21.15.16:0/3428860939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c88011a00 con 0x7f7c7c005220 2024-09-10T21:18:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c91600640 1 -- 172.21.15.16:0/3428860939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c88011e50 con 0x7f7c7c005220 2024-09-10T21:18:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c9c000640 1 -- 172.21.15.16:0/3428860939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c940c71f0 con 0x7f7c7c005220 2024-09-10T21:18:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c9c000640 1 -- 172.21.15.16:0/3428860939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c940c77b0 con 0x7f7c7c005220 2024-09-10T21:18:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c9c000640 1 -- 172.21.15.16:0/3428860939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c940d02f0 con 0x7f7c7c005220 2024-09-10T21:18:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c91600640 1 -- 172.21.15.16:0/3428860939 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7c88010420 con 0x7f7c7c005220 2024-09-10T21:18:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c91600640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7c740727b0 0x7f7c74074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c91600640 1 -- 172.21.15.16:0/3428860939 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4943+0+0 (secure 0 0 0) 0x7f7c88094040 con 0x7f7c7c005220 2024-09-10T21:18:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.277+0000 7f7c98c00640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7c740727b0 0x7f7c74074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.281+0000 7f7c98c00640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7c740727b0 0x7f7c74074c70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7c80001100 tx=0x7f7c800060b0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:10.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.281+0000 7f7c91600640 1 -- 172.21.15.16:0/3428860939 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7c8805f980 con 0x7f7c7c005220 2024-09-10T21:18:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:10.521+0000 7f7c9c000640 1 -- 172.21.15.16:0/3428860939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) -- 0x7f7c940b4370 con 0x7f7c7c005220 2024-09-10T21:18:11.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.389+0000 7f7c91600640 1 -- 172.21.15.16:0/3428860939 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v392) ==== 151+0+0 (secure 0 0 0) 0x7f7c8805f280 con 0x7f7c7c005220 2024-09-10T21:18:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.393+0000 7f7c9c000640 1 -- 172.21.15.16:0/3428860939 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7c740727b0 msgr2=0x7f7c74074c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.393+0000 7f7c9c000640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7c740727b0 0x7f7c74074c70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7c80001100 tx=0x7f7c800060b0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.393+0000 7f7c9c000640 1 -- 172.21.15.16:0/3428860939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c7c005220 msgr2=0x7f7c9406e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.393+0000 7f7c9c000640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c7c005220 0x7f7c9406e700 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f7c88006680 tx=0x7f7c880039f0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.393+0000 7f7c9c000640 1 -- 172.21.15.16:0/3428860939 shutdown_connections 2024-09-10T21:18:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.393+0000 7f7c9c000640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7c740727b0 0x7f7c74074c70 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.393+0000 7f7c9c000640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c9414e6c0 0x7f7c9415b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.393+0000 7f7c9c000640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7c9414e240 0x7f7c9415b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.393+0000 7f7c9c000640 1 --2- 172.21.15.16:0/3428860939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c7c005220 0x7f7c9406e700 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.393+0000 7f7c9c000640 1 -- 172.21.15.16:0/3428860939 >> 172.21.15.16:0/3428860939 conn(0x7f7c940d1d80 msgr2=0x7f7c940b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.393+0000 7f7c9c000640 1 -- 172.21.15.16:0/3428860939 shutdown_connections 2024-09-10T21:18:11.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.393+0000 7f7c9c000640 1 -- 172.21.15.16:0/3428860939 wait complete. 2024-09-10T21:18:11.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:18:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.569+0000 7f6668400640 1 -- 172.21.15.16:0/2188602035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66600b13c0 msgr2=0x7f66600afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.569+0000 7f6668400640 1 --2- 172.21.15.16:0/2188602035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66600b13c0 0x7f66600afac0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6658009bc0 tx=0x7f665801ca80 comp rx=0 tx=0).stop 2024-09-10T21:18:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.569+0000 7f6668400640 1 -- 172.21.15.16:0/2188602035 shutdown_connections 2024-09-10T21:18:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.569+0000 7f6668400640 1 --2- 172.21.15.16:0/2188602035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66600b13c0 0x7f66600afac0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.569+0000 7f6668400640 1 --2- 172.21.15.16:0/2188602035 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66600716f0 0x7f66600af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.569+0000 7f6668400640 1 -- 172.21.15.16:0/2188602035 >> 172.21.15.16:0/2188602035 conn(0x7f666009b130 msgr2=0x7f66600d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.569+0000 7f6668400640 1 -- 172.21.15.16:0/2188602035 shutdown_connections 2024-09-10T21:18:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.569+0000 7f6668400640 1 -- 172.21.15.16:0/2188602035 wait complete. 2024-09-10T21:18:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.569+0000 7f6668400640 1 Processor -- start 2024-09-10T21:18:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.569+0000 7f6668400640 1 -- start start 2024-09-10T21:18:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6668400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66600716f0 0x7f66601fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6668400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66600b13c0 0x7f666015ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6668400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666015f400 0x7f66601574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6668400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6660168fc0 con 0x7f666015f400 2024-09-10T21:18:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6668400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6660168e40 con 0x7f66600b13c0 2024-09-10T21:18:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6668400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6660169140 con 0x7f66600716f0 2024-09-10T21:18:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6666400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666015f400 0x7f66601574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6666400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666015f400 0x7f66601574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52264/0 (socket says 172.21.15.16:52264) 2024-09-10T21:18:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6666400640 1 -- 172.21.15.16:0/3320148073 learned_addr learned my addr 172.21.15.16:0/3320148073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6665a00640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66600716f0 0x7f66601fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6665000640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66600b13c0 0x7f666015ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6666400640 1 -- 172.21.15.16:0/3320148073 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66600716f0 msgr2=0x7f66601fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6666400640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66600716f0 0x7f66601fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6666400640 1 -- 172.21.15.16:0/3320148073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66600b13c0 msgr2=0x7f666015ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6666400640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66600b13c0 0x7f666015ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6666400640 1 -- 172.21.15.16:0/3320148073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6660157fa0 con 0x7f666015f400 2024-09-10T21:18:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6665000640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66600b13c0 0x7f666015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6666400640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666015f400 0x7f66601574e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f665801a6b0 tx=0x7f66580a59e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f665d600640 1 -- 172.21.15.16:0/3320148073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66580bb070 con 0x7f666015f400 2024-09-10T21:18:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f665d600640 1 -- 172.21.15.16:0/3320148073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6658004e00 con 0x7f666015f400 2024-09-10T21:18:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6668400640 1 -- 172.21.15.16:0/3320148073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6660158280 con 0x7f666015f400 2024-09-10T21:18:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f665d600640 1 -- 172.21.15.16:0/3320148073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66580b2040 con 0x7f666015f400 2024-09-10T21:18:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6668400640 1 -- 172.21.15.16:0/3320148073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66601fb1f0 con 0x7f666015f400 2024-09-10T21:18:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f6668400640 1 -- 172.21.15.16:0/3320148073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6660070fe0 con 0x7f666015f400 2024-09-10T21:18:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.573+0000 7f665d600640 1 -- 172.21.15.16:0/3320148073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f66580b5030 con 0x7f666015f400 2024-09-10T21:18:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.577+0000 7f665d600640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66400725e0 0x7f6640074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.577+0000 7f665d600640 1 -- 172.21.15.16:0/3320148073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4943+0+0 (secure 0 0 0) 0x7f665812ce40 con 0x7f666015f400 2024-09-10T21:18:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.577+0000 7f6665a00640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66400725e0 0x7f6640074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.577+0000 7f6665a00640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66400725e0 0x7f6640074aa0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6654007fd0 tx=0x7f6654009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.581+0000 7f665d600640 1 -- 172.21.15.16:0/3320148073 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6658016790 con 0x7f666015f400 2024-09-10T21:18:11.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f6668400640 1 -- 172.21.15.16:0/3320148073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f66600d0b80 con 0x7f666015f400 2024-09-10T21:18:11.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f665d600640 1 -- 172.21.15.16:0/3320148073 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v58) ==== 59+0+1998 (secure 0 0 0) 0x7f66580f8130 con 0x7f666015f400 2024-09-10T21:18:11.761 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:18:11.761 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:18:11.761 INFO:teuthology.orchestra.run.smithi016.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:18:11.761 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:18:11.761 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4226.427518 2024-09-10T21:18:11.761 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5426.620519 2024-09-10T21:18:11.761 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4255.210203 2024-09-10T21:18:11.762 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4255.734871 2024-09-10T21:18:11.762 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4192.833008 2024-09-10T21:18:11.762 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4136.940759 2024-09-10T21:18:11.762 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4217.384701 2024-09-10T21:18:11.762 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5220.337972 2024-09-10T21:18:11.762 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-10T21:18:11.762 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:18:11.762 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-10T21:18:11.762 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-10T21:18:11.762 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:18:11.762 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:18:11.762 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-10T21:18:11.763 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:18:11.763 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:18:11.763 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:18:11.763 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-10T21:18:11.763 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-10T21:18:11.763 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-10T21:18:11.763 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:18:11.763 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:18:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f6668400640 1 -- 172.21.15.16:0/3320148073 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66400725e0 msgr2=0x7f6640074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f6668400640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66400725e0 0x7f6640074aa0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6654007fd0 tx=0x7f6654009040 comp rx=0 tx=0).stop 2024-09-10T21:18:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f6668400640 1 -- 172.21.15.16:0/3320148073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666015f400 msgr2=0x7f66601574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f6668400640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666015f400 0x7f66601574e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f665801a6b0 tx=0x7f66580a59e0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f6668400640 1 -- 172.21.15.16:0/3320148073 shutdown_connections 2024-09-10T21:18:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f6668400640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f66400725e0 0x7f6640074aa0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f6668400640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f666015f400 0x7f66601574e0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f6668400640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66600b13c0 0x7f666015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f6668400640 1 --2- 172.21.15.16:0/3320148073 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f66600716f0 0x7f66601fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f6668400640 1 -- 172.21.15.16:0/3320148073 >> 172.21.15.16:0/3320148073 conn(0x7f666009b130 msgr2=0x7f66600cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f6668400640 1 -- 172.21.15.16:0/3320148073 shutdown_connections 2024-09-10T21:18:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.757+0000 7f6668400640 1 -- 172.21.15.16:0/3320148073 wait complete. 2024-09-10T21:18:11.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:18:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 -- 172.21.15.16:0/3304323934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8364151210 msgr2=0x7f83640726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 --2- 172.21.15.16:0/3304323934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8364151210 0x7f83640726b0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f8364074b60 tx=0x7f835401db30 comp rx=0 tx=0).stop 2024-09-10T21:18:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 -- 172.21.15.16:0/3304323934 shutdown_connections 2024-09-10T21:18:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 --2- 172.21.15.16:0/3304323934 >> v2:172.21.15.52:3300/0 conn(0x7f8364072bf0 0x7f836415dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 --2- 172.21.15.16:0/3304323934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8364151210 0x7f83640726b0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 -- 172.21.15.16:0/3304323934 >> 172.21.15.16:0/3304323934 conn(0x7f83640b16a0 msgr2=0x7f83640b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 -- 172.21.15.16:0/3304323934 shutdown_connections 2024-09-10T21:18:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 -- 172.21.15.16:0/3304323934 wait complete. 2024-09-10T21:18:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 Processor -- start 2024-09-10T21:18:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 -- start start 2024-09-10T21:18:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8364072bf0 0x7f836415b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8364151210 0x7f83640ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f836415e2c0 0x7f836415c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f836415b230 con 0x7f8364072bf0 2024-09-10T21:18:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f836415b0b0 con 0x7f8364151210 2024-09-10T21:18:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8369400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f836415b3b0 con 0x7f836415e2c0 2024-09-10T21:18:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8362800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8364072bf0 0x7f836415b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8362800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8364072bf0 0x7f836415b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52276/0 (socket says 172.21.15.16:52276) 2024-09-10T21:18:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.929+0000 7f8362800640 1 -- 172.21.15.16:0/3740888888 learned_addr learned my addr 172.21.15.16:0/3740888888 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f8363200640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f836415e2c0 0x7f836415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f8362800640 1 -- 172.21.15.16:0/3740888888 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f836415e2c0 msgr2=0x7f836415c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f8362800640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f836415e2c0 0x7f836415c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f835be00640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8364151210 0x7f83640ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f8362800640 1 -- 172.21.15.16:0/3740888888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8364151210 msgr2=0x7f83640ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f8362800640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8364151210 0x7f83640ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f8362800640 1 -- 172.21.15.16:0/3740888888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836415be00 con 0x7f8364072bf0 2024-09-10T21:18:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f8363200640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f836415e2c0 0x7f836415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f835be00640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8364151210 0x7f83640ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f8362800640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8364072bf0 0x7f836415b8c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f835401f140 tx=0x7f8354005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f835aa00640 1 -- 172.21.15.16:0/3740888888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540b3040 con 0x7f8364072bf0 2024-09-10T21:18:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f835aa00640 1 -- 172.21.15.16:0/3740888888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8354005320 con 0x7f8364072bf0 2024-09-10T21:18:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f835aa00640 1 -- 172.21.15.16:0/3740888888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540046d0 con 0x7f8364072bf0 2024-09-10T21:18:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f8369400640 1 -- 172.21.15.16:0/3740888888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83641fb030 con 0x7f8364072bf0 2024-09-10T21:18:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f8369400640 1 -- 172.21.15.16:0/3740888888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83641fb5f0 con 0x7f8364072bf0 2024-09-10T21:18:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f8369400640 1 -- 172.21.15.16:0/3740888888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83640b2c50 con 0x7f8364072bf0 2024-09-10T21:18:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.933+0000 7f835aa00640 1 -- 172.21.15.16:0/3740888888 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f835400b040 con 0x7f8364072bf0 2024-09-10T21:18:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.937+0000 7f835aa00640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f83400725e0 0x7f8340074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.937+0000 7f835aa00640 1 -- 172.21.15.16:0/3740888888 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4943+0+0 (secure 0 0 0) 0x7f835412d6e0 con 0x7f8364072bf0 2024-09-10T21:18:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.937+0000 7f835be00640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f83400725e0 0x7f8340074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.937+0000 7f835be00640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f83400725e0 0x7f8340074aa0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f83641fadf0 tx=0x7f834c005f90 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:11.941+0000 7f835aa00640 1 -- 172.21.15.16:0/3740888888 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8354017720 con 0x7f8364072bf0 2024-09-10T21:18:12.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.112+0000 7f8369400640 1 -- 172.21.15.16:0/3740888888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f83640ce230 con 0x7f8364072bf0 2024-09-10T21:18:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.116+0000 7f835aa00640 1 -- 172.21.15.16:0/3740888888 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v59)=0 v59) ==== 122+0+0 (secure 0 0 0) 0x7f83540f90b0 con 0x7f8364072bf0 2024-09-10T21:18:12.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.120+0000 7f8369400640 1 -- 172.21.15.16:0/3740888888 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f83400725e0 msgr2=0x7f8340074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:12.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.120+0000 7f8369400640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f83400725e0 0x7f8340074aa0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f83641fadf0 tx=0x7f834c005f90 comp rx=0 tx=0).stop 2024-09-10T21:18:12.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.120+0000 7f8369400640 1 -- 172.21.15.16:0/3740888888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8364072bf0 msgr2=0x7f836415b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:12.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.120+0000 7f8369400640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8364072bf0 0x7f836415b8c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f835401f140 tx=0x7f8354005ca0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.120+0000 7f8369400640 1 -- 172.21.15.16:0/3740888888 shutdown_connections 2024-09-10T21:18:12.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.120+0000 7f8369400640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f83400725e0 0x7f8340074aa0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.120+0000 7f8369400640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f836415e2c0 0x7f836415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.120+0000 7f8369400640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8364151210 0x7f83640ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.120+0000 7f8369400640 1 --2- 172.21.15.16:0/3740888888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8364072bf0 0x7f836415b8c0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.120+0000 7f8369400640 1 -- 172.21.15.16:0/3740888888 >> 172.21.15.16:0/3740888888 conn(0x7f83640b16a0 msgr2=0x7f8364159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:12.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.120+0000 7f8369400640 1 -- 172.21.15.16:0/3740888888 shutdown_connections 2024-09-10T21:18:12.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.120+0000 7f8369400640 1 -- 172.21.15.16:0/3740888888 wait complete. 2024-09-10T21:18:12.145 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:18:12.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:18:12.210 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:18:12.306 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:18:12.306 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:18:12.307 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:12.315 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:18:12.316 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:18:12.316 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:12.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:18:12.325 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:18:12.325 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:12.367 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:18:12.367 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:18:12.367 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:12.376 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:18:12.376 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:18:12.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.552+0000 7fe8ed200640 1 -- 172.21.15.16:0/1958183820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc004d50 msgr2=0x7fe8cc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.552+0000 7fe8ed200640 1 --2- 172.21.15.16:0/1958183820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc004d50 0x7fe8cc0051d0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009ee0 tx=0x7fe8e001d5f0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.552+0000 7fe8e7e00640 1 -- 172.21.15.16:0/1958183820 reap_dead start 2024-09-10T21:18:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.552+0000 7fe8ed200640 1 -- 172.21.15.16:0/1958183820 shutdown_connections 2024-09-10T21:18:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.552+0000 7fe8ed200640 1 -- 172.21.15.16:0/1958183820 >> 172.21.15.16:0/1958183820 conn(0x7fe8e80d1d00 msgr2=0x7fe8e80d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.552+0000 7fe8ed200640 1 -- 172.21.15.16:0/1958183820 shutdown_connections 2024-09-10T21:18:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.552+0000 7fe8ed200640 1 -- 172.21.15.16:0/1958183820 wait complete. 2024-09-10T21:18:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8ed200640 1 Processor -- start 2024-09-10T21:18:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8ed200640 1 -- start start 2024-09-10T21:18:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8ed200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8cc004d50 0x7fe8e81553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8ed200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b5ee0 0x7fe8e80cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8ed200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe8e80cdb20 0x7fe8e80cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8ed200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81607e0 con 0x7fe8e80b5ee0 2024-09-10T21:18:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8ed200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8160660 con 0x7fe8cc004d50 2024-09-10T21:18:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8ed200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e8160960 con 0x7fe8e80cdb20 2024-09-10T21:18:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8dea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b5ee0 0x7fe8e80cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8dea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b5ee0 0x7fe8e80cb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52608/0 (socket says 172.21.15.16:52608) 2024-09-10T21:18:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8dea00640 1 -- 172.21.15.16:0/3590909904 learned_addr learned my addr 172.21.15.16:0/3590909904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8e7400640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8cc004d50 0x7fe8e81553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8e7e00640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe8e80cdb20 0x7fe8e80cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8dea00640 1 -- 172.21.15.16:0/3590909904 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe8e80cdb20 msgr2=0x7fe8e80cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8dea00640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe8e80cdb20 0x7fe8e80cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8dea00640 1 -- 172.21.15.16:0/3590909904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8cc004d50 msgr2=0x7fe8e81553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8dea00640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8cc004d50 0x7fe8e81553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8dea00640 1 -- 172.21.15.16:0/3590909904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e80b6360 con 0x7fe8e80b5ee0 2024-09-10T21:18:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8dea00640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b5ee0 0x7fe8e80cb8f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000e310 tx=0x7fe8d000e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8dfe00640 1 -- 172.21.15.16:0/3590909904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0012850 con 0x7fe8e80b5ee0 2024-09-10T21:18:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8dfe00640 1 -- 172.21.15.16:0/3590909904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8d0012ae0 con 0x7fe8e80b5ee0 2024-09-10T21:18:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8dfe00640 1 -- 172.21.15.16:0/3590909904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d001a400 con 0x7fe8e80b5ee0 2024-09-10T21:18:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8ed200640 1 -- 172.21.15.16:0/3590909904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e81f48b0 con 0x7fe8e80b5ee0 2024-09-10T21:18:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8ed200640 1 -- 172.21.15.16:0/3590909904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e81f4e70 con 0x7fe8e80b5ee0 2024-09-10T21:18:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.556+0000 7fe8ed200640 1 -- 172.21.15.16:0/3590909904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e80d0120 con 0x7fe8e80b5ee0 2024-09-10T21:18:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.560+0000 7fe8dfe00640 1 -- 172.21.15.16:0/3590909904 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe8d0022030 con 0x7fe8e80b5ee0 2024-09-10T21:18:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.560+0000 7fe8dfe00640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe8c4072610 0x7fe8c4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.560+0000 7fe8dfe00640 1 -- 172.21.15.16:0/3590909904 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4943+0+0 (secure 0 0 0) 0x7fe8d0096100 con 0x7fe8e80b5ee0 2024-09-10T21:18:12.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.560+0000 7fe8e7400640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe8c4072610 0x7fe8c4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:12.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.560+0000 7fe8e7400640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe8c4072610 0x7fe8c4074ad0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009bb0 tx=0x7fe8d80091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:12.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.564+0000 7fe8dfe00640 1 -- 172.21.15.16:0/3590909904 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe8e80b6360 con 0x7fe8e80b5ee0 2024-09-10T21:18:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8ed200640 1 -- 172.21.15.16:0/3590909904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8e80b4b50 con 0x7fe8e80b5ee0 2024-09-10T21:18:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8dfe00640 1 -- 172.21.15.16:0/3590909904 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+3905 (secure 0 0 0) 0x7fe8d0016030 con 0x7fe8e80b5ee0 2024-09-10T21:18:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:12.777 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"btime":"2024-09-10T21:18:06:370235+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3961362430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3961362430},{"type":"v1","addr":"172.21.15.52:6834","nonce":3961362430}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6967,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/831111367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":831111367},{"type":"v1","addr":"172.21.15.52:6835","nonce":831111367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7467,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/552929965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":552929965},{"type":"v1","addr":"172.21.15.16:6836","nonce":552929965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7770,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3401373770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":3401373770},{"type":"v1","addr":"172.21.15.16:6837","nonce":3401373770}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[]} 2024-09-10T21:18:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 86 2024-09-10T21:18:12.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8ed200640 1 -- 172.21.15.16:0/3590909904 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe8c4072610 msgr2=0x7fe8c4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:12.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8ed200640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe8c4072610 0x7fe8c4074ad0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009bb0 tx=0x7fe8d80091e0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8ed200640 1 -- 172.21.15.16:0/3590909904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b5ee0 msgr2=0x7fe8e80cb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:12.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8ed200640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b5ee0 0x7fe8e80cb8f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000e310 tx=0x7fe8d000e7d0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8ed200640 1 -- 172.21.15.16:0/3590909904 shutdown_connections 2024-09-10T21:18:12.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8ed200640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe8c4072610 0x7fe8c4074ad0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8ed200640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe8e80cdb20 0x7fe8e80cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8ed200640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b5ee0 0x7fe8e80cb8f0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8ed200640 1 --2- 172.21.15.16:0/3590909904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8cc004d50 0x7fe8e81553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:12.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8ed200640 1 -- 172.21.15.16:0/3590909904 >> 172.21.15.16:0/3590909904 conn(0x7fe8e80d1d00 msgr2=0x7fe8e806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:12.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8ed200640 1 -- 172.21.15.16:0/3590909904 shutdown_connections 2024-09-10T21:18:12.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:12.772+0000 7fe8ed200640 1 -- 172.21.15.16:0/3590909904 wait complete. 2024-09-10T21:18:12.799 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:18:12.799 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:18:12.799 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:18:12.799 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:18:12.799 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:18:12.800 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:18:12.800 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:18:12.800 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:18:12.800 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:18:12.800 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:18:12.800 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:18:12.800 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:18:12.801 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:18:12.801 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:18:12.801 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:18:12.801 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:18:12.811 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:18:12.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:18:12.812 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:18:12.812 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:18:12.814 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:18:12.815 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:18:12.816 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:18:12.816 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:18:12.858 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:18:12.858 DEBUG:teuthology.parallel:result is None 2024-09-10T21:18:12.859 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:18:12.859 DEBUG:teuthology.parallel:result is None 2024-09-10T21:18:12.861 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:18:12.862 DEBUG:teuthology.parallel:result is None 2024-09-10T21:18:12.864 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:18:12.864 DEBUG:teuthology.parallel:result is None 2024-09-10T21:18:12.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:18:12.944 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-10T21:18:12.945 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-10T21:18:12.957 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:18:12.958 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:18:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.012+0000 7f2b50a00640 1 -- 172.21.15.16:0/2013692836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b4c15f400 msgr2=0x7f2b4c070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.012+0000 7f2b50a00640 1 --2- 172.21.15.16:0/2013692836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b4c15f400 0x7f2b4c070fe0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c005fd0 tx=0x7f2b3c01c700 comp rx=0 tx=0).stop 2024-09-10T21:18:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.012+0000 7f2b50a00640 1 -- 172.21.15.16:0/2013692836 shutdown_connections 2024-09-10T21:18:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.012+0000 7f2b50a00640 1 --2- 172.21.15.16:0/2013692836 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b4c0b13c0 0x7f2b4c071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.012+0000 7f2b50a00640 1 --2- 172.21.15.16:0/2013692836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b4c15f400 0x7f2b4c070fe0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.012+0000 7f2b50a00640 1 -- 172.21.15.16:0/2013692836 >> 172.21.15.16:0/2013692836 conn(0x7f2b4c09b130 msgr2=0x7f2b4c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.012+0000 7f2b50a00640 1 -- 172.21.15.16:0/2013692836 shutdown_connections 2024-09-10T21:18:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.012+0000 7f2b50a00640 1 -- 172.21.15.16:0/2013692836 wait complete. 2024-09-10T21:18:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b50a00640 1 Processor -- start 2024-09-10T21:18:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b50a00640 1 -- start start 2024-09-10T21:18:13.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b50a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b4c0b13c0 0x7f2b4c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:13.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b50a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4c0cfc80 0x7f2b4c1f8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:13.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b50a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b4c15f400 0x7f2b4c15de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:13.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b50a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c169770 con 0x7f2b4c0cfc80 2024-09-10T21:18:13.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b50a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c1695f0 con 0x7f2b4c0b13c0 2024-09-10T21:18:13.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b50a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4c1698f0 con 0x7f2b4c15f400 2024-09-10T21:18:13.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b41400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4c0cfc80 0x7f2b4c1f8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:13.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b41400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4c0cfc80 0x7f2b4c1f8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52652/0 (socket says 172.21.15.16:52652) 2024-09-10T21:18:13.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b41400640 1 -- 172.21.15.16:0/1891491237 learned_addr learned my addr 172.21.15.16:0/1891491237 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:13.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b49e00640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b4c0b13c0 0x7f2b4c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:13.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b4a800640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b4c15f400 0x7f2b4c15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:13.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b41400640 1 -- 172.21.15.16:0/1891491237 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b4c15f400 msgr2=0x7f2b4c15de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:13.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b41400640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b4c15f400 0x7f2b4c15de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b41400640 1 -- 172.21.15.16:0/1891491237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b4c0b13c0 msgr2=0x7f2b4c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:13.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b41400640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b4c0b13c0 0x7f2b4c15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b41400640 1 -- 172.21.15.16:0/1891491237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c15a120 con 0x7f2b4c0cfc80 2024-09-10T21:18:13.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b4a800640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b4c15f400 0x7f2b4c15de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:13.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b41400640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4c0cfc80 0x7f2b4c1f8ce0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400e9c0 tx=0x7f2b3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:13.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b42a00640 1 -- 172.21.15.16:0/1891491237 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3400cd80 con 0x7f2b4c0cfc80 2024-09-10T21:18:13.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b42a00640 1 -- 172.21.15.16:0/1891491237 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b34005020 con 0x7f2b4c0cfc80 2024-09-10T21:18:13.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b42a00640 1 -- 172.21.15.16:0/1891491237 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34005450 con 0x7f2b4c0cfc80 2024-09-10T21:18:13.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b50a00640 1 -- 172.21.15.16:0/1891491237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4c15a400 con 0x7f2b4c0cfc80 2024-09-10T21:18:13.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.016+0000 7f2b50a00640 1 -- 172.21.15.16:0/1891491237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4c15a9c0 con 0x7f2b4c0cfc80 2024-09-10T21:18:13.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.020+0000 7f2b50a00640 1 -- 172.21.15.16:0/1891491237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b4c0d00d0 con 0x7f2b4c0cfc80 2024-09-10T21:18:13.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.020+0000 7f2b42a00640 1 -- 172.21.15.16:0/1891491237 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b340047b0 con 0x7f2b4c0cfc80 2024-09-10T21:18:13.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.020+0000 7f2b42a00640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b280725e0 0x7f2b28074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:13.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.020+0000 7f2b42a00640 1 -- 172.21.15.16:0/1891491237 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4943+0+0 (secure 0 0 0) 0x7f2b34093b80 con 0x7f2b4c0cfc80 2024-09-10T21:18:13.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.020+0000 7f2b49e00640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b280725e0 0x7f2b28074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:13.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.024+0000 7f2b49e00640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b280725e0 0x7f2b28074aa0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0ae000 tx=0x7f2b3c01af40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:13.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.024+0000 7f2b42a00640 1 -- 172.21.15.16:0/1891491237 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b3405f4c0 con 0x7f2b4c0cfc80 2024-09-10T21:18:13.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.212+0000 7f2b50a00640 1 -- 172.21.15.16:0/1891491237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2b4c0cd2d0 con 0x7f2b4c0cfc80 2024-09-10T21:18:13.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.404+0000 7f2b42a00640 1 -- 172.21.15.16:0/1891491237 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v121) ==== 97+0+0 (secure 0 0 0) 0x7f2b3405edc0 con 0x7f2b4c0cfc80 2024-09-10T21:18:13.410 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-10T21:18:13.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.408+0000 7f2b50a00640 1 -- 172.21.15.16:0/1891491237 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b280725e0 msgr2=0x7f2b28074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:13.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.408+0000 7f2b50a00640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b280725e0 0x7f2b28074aa0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0ae000 tx=0x7f2b3c01af40 comp rx=0 tx=0).stop 2024-09-10T21:18:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.408+0000 7f2b50a00640 1 -- 172.21.15.16:0/1891491237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4c0cfc80 msgr2=0x7f2b4c1f8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.408+0000 7f2b50a00640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4c0cfc80 0x7f2b4c1f8ce0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400e9c0 tx=0x7f2b3400ee80 comp rx=0 tx=0).stop 2024-09-10T21:18:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.408+0000 7f2b50a00640 1 -- 172.21.15.16:0/1891491237 shutdown_connections 2024-09-10T21:18:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.408+0000 7f2b50a00640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2b280725e0 0x7f2b28074aa0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.408+0000 7f2b50a00640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2b4c15f400 0x7f2b4c15de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.408+0000 7f2b50a00640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4c0cfc80 0x7f2b4c1f8ce0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.408+0000 7f2b50a00640 1 --2- 172.21.15.16:0/1891491237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b4c0b13c0 0x7f2b4c15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.408+0000 7f2b50a00640 1 -- 172.21.15.16:0/1891491237 >> 172.21.15.16:0/1891491237 conn(0x7f2b4c09b130 msgr2=0x7f2b4c15c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.408+0000 7f2b50a00640 1 -- 172.21.15.16:0/1891491237 shutdown_connections 2024-09-10T21:18:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.408+0000 7f2b50a00640 1 -- 172.21.15.16:0/1891491237 wait complete. 2024-09-10T21:18:13.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:18:13.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.576+0000 7f93d1a00640 1 -- 172.21.15.16:0/1891550049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93cc0cfb60 msgr2=0x7f93cc0cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:13.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.576+0000 7f93d1a00640 1 --2- 172.21.15.16:0/1891550049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93cc0cfb60 0x7f93cc0cffc0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009900 tx=0x7f93b401c7c0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.576+0000 7f93d1a00640 1 -- 172.21.15.16:0/1891550049 shutdown_connections 2024-09-10T21:18:13.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.576+0000 7f93d1a00640 1 --2- 172.21.15.16:0/1891550049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93cc0cfb60 0x7f93cc0cffc0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.576+0000 7f93d1a00640 1 --2- 172.21.15.16:0/1891550049 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f93cc15f350 0x7f93cc0cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.576+0000 7f93d1a00640 1 -- 172.21.15.16:0/1891550049 >> 172.21.15.16:0/1891550049 conn(0x7f93cc0d1d80 msgr2=0x7f93cc0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:13.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.576+0000 7f93d1a00640 1 -- 172.21.15.16:0/1891550049 shutdown_connections 2024-09-10T21:18:13.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.576+0000 7f93d1a00640 1 -- 172.21.15.16:0/1891550049 wait complete. 2024-09-10T21:18:13.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93d1a00640 1 Processor -- start 2024-09-10T21:18:13.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93d1a00640 1 -- start start 2024-09-10T21:18:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93d1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93cc0cfb60 0x7f93cc0c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93d1a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f93cc0d0500 0x7f93cc157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93d1a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93cc15f350 0x7f93cc1fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93d1a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc169020 con 0x7f93cc0cfb60 2024-09-10T21:18:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93d1a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc168ea0 con 0x7f93cc15f350 2024-09-10T21:18:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93d1a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f93cc1691a0 con 0x7f93cc0d0500 2024-09-10T21:18:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93caa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93cc0cfb60 0x7f93cc0c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93caa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93cc0cfb60 0x7f93cc0c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52668/0 (socket says 172.21.15.16:52668) 2024-09-10T21:18:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93caa00640 1 -- 172.21.15.16:0/1479710629 learned_addr learned my addr 172.21.15.16:0/1479710629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93c3e00640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f93cc0d0500 0x7f93cc157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93cb400640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93cc15f350 0x7f93cc1fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93caa00640 1 -- 172.21.15.16:0/1479710629 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f93cc0d0500 msgr2=0x7f93cc157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93caa00640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f93cc0d0500 0x7f93cc157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93caa00640 1 -- 172.21.15.16:0/1479710629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93cc15f350 msgr2=0x7f93cc1fb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93caa00640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93cc15f350 0x7f93cc1fb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93caa00640 1 -- 172.21.15.16:0/1479710629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93cc0c8ab0 con 0x7f93cc0cfb60 2024-09-10T21:18:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93cb400640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93cc15f350 0x7f93cc1fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93caa00640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93cc0cfb60 0x7f93cc0c84d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00d720 tx=0x7f93bc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93c2a00640 1 -- 172.21.15.16:0/1479710629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc017070 con 0x7f93cc0cfb60 2024-09-10T21:18:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93c2a00640 1 -- 172.21.15.16:0/1479710629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93bc009d20 con 0x7f93cc0cfb60 2024-09-10T21:18:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93c2a00640 1 -- 172.21.15.16:0/1479710629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc005280 con 0x7f93cc0cfb60 2024-09-10T21:18:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93d1a00640 1 -- 172.21.15.16:0/1479710629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93cc06ea10 con 0x7f93cc0cfb60 2024-09-10T21:18:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.580+0000 7f93d1a00640 1 -- 172.21.15.16:0/1479710629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93cc158300 con 0x7f93cc0cfb60 2024-09-10T21:18:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.584+0000 7f93d1a00640 1 -- 172.21.15.16:0/1479710629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93cc0cffc0 con 0x7f93cc0cfb60 2024-09-10T21:18:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.584+0000 7f93c2a00640 1 -- 172.21.15.16:0/1479710629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f93bc005420 con 0x7f93cc0cfb60 2024-09-10T21:18:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.584+0000 7f93c2a00640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f93a80725e0 0x7f93a8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.584+0000 7f93c2a00640 1 -- 172.21.15.16:0/1479710629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4857+0+0 (secure 0 0 0) 0x7f93bc0930c0 con 0x7f93cc0cfb60 2024-09-10T21:18:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.584+0000 7f93c3e00640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f93a80725e0 0x7f93a8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:13.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.584+0000 7f93c3e00640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f93a80725e0 0x7f93a8074aa0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f93b40af000 tx=0x7f93b401b060 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.588+0000 7f93c2a00640 1 -- 172.21.15.16:0/1479710629 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f93bc05ea50 con 0x7f93cc0cfb60 2024-09-10T21:18:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93d1a00640 1 -- 172.21.15.16:0/1479710629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f93cc0b7710 con 0x7f93cc0cfb60 2024-09-10T21:18:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93c2a00640 1 -- 172.21.15.16:0/1479710629 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v30) ==== 80+0+5304 (secure 0 0 0) 0x7f93bc067170 con 0x7f93cc0cfb60 2024-09-10T21:18:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:13.828 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:18:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-10T21:18:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-10T21:18:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmhg+lJBAAWjuYW75BGo2dmyAUi7s9+A==", 2024-09-10T21:18:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:18:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:18:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-10T21:18:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmQr7+MBAAOzXhWIirBOoi/7wKnrFgmg==", 2024-09-10T21:18:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:18:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:18:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-10T21:18:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmxxoALBAAgkmw91/VP1Uv2WMZsJKAyA==", 2024-09-10T21:18:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:18:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:18:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-10T21:18:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmA4tzOBAAiS+8n+Gokz6yXapVf2ecYg==", 2024-09-10T21:18:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:18:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:18:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-10T21:18:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHteBmRoahLRAA32PsUkqtGkeI7tMFRwas5Q==", 2024-09-10T21:18:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-10T21:18:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCLteBmGmEMBhAAr1yQEf/NkiN4+ezAQ9VR2A==", 2024-09-10T21:18:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-10T21:18:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCOteBmmMuAHBAAt+G/KApsZymjTLRSmyRbdw==", 2024-09-10T21:18:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-10T21:18:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCRteBmoQfNMBAA0vnIqlYoI13KNAAfNmJg3w==", 2024-09-10T21:18:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-10T21:18:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCWteBmc/7lDxAADlSi4UpcFJVNdp6YTkpdSw==", 2024-09-10T21:18:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZteBmBZ1LJhAAPcbZa1LQu4kNbSsJ/8Z81A==", 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCcteBm7/czOxAAqREnY84+LF1N9dRXOhzz6g==", 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgteBmG2J8FRAAB+cMFCnf9wJg1R3RD5SUJw==", 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.836 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:13.837 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:13.837 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.837 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.837 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.837 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-10T21:18:13.837 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA==", 2024-09-10T21:18:13.837 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.837 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:13.837 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-10T21:18:13.837 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:18:13.837 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmuEm4ChAAe08QvhwIcN8ZHa3AoVcYww==", 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBm50o1BxAAIFbVxVqU4rU8eKYpuZjTHA==", 2024-09-10T21:18:13.838 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.839 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:13.839 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:18:13.839 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:18:13.839 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:18:13.839 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.839 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.839 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-10T21:18:13.839 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmxtu0EBAAmN7+07lDiwjsIC/pNYUlSw==", 2024-09-10T21:18:13.839 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.839 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:13.839 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:18:13.839 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmfqgYCBAAm00tBe45phGieCcp1q46cw==", 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.840 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-10T21:18:13.841 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmMVABIBAAoB72mPyXyUO6Y6k+fMqQQg==", 2024-09-10T21:18:13.841 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.841 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:18:13.841 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:18:13.841 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.841 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:13.841 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:13.841 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-10T21:18:13.841 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmeyYxHBAAvBkpTzjPQKAOlHzI1jQ46g==", 2024-09-10T21:18:13.841 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:13.841 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:18:13.841 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:18:13.842 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:13.842 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.842 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:13.842 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-10T21:18:13.842 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:18:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93d1a00640 1 -- 172.21.15.16:0/1479710629 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f93a80725e0 msgr2=0x7f93a8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93d1a00640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f93a80725e0 0x7f93a8074aa0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f93b40af000 tx=0x7f93b401b060 comp rx=0 tx=0).stop 2024-09-10T21:18:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93d1a00640 1 -- 172.21.15.16:0/1479710629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93cc0cfb60 msgr2=0x7f93cc0c84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93d1a00640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93cc0cfb60 0x7f93cc0c84d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00d720 tx=0x7f93bc00dbe0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93d1a00640 1 -- 172.21.15.16:0/1479710629 shutdown_connections 2024-09-10T21:18:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93d1a00640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f93a80725e0 0x7f93a8074aa0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93d1a00640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93cc15f350 0x7f93cc1fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93d1a00640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f93cc0d0500 0x7f93cc157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93d1a00640 1 --2- 172.21.15.16:0/1479710629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93cc0cfb60 0x7f93cc0c84d0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93d1a00640 1 -- 172.21.15.16:0/1479710629 >> 172.21.15.16:0/1479710629 conn(0x7f93cc0d1d80 msgr2=0x7f93cc0b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93d1a00640 1 -- 172.21.15.16:0/1479710629 shutdown_connections 2024-09-10T21:18:13.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.824+0000 7f93d1a00640 1 -- 172.21.15.16:0/1479710629 wait complete. 2024-09-10T21:18:13.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:18:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.992+0000 7fd6fa600640 1 -- 172.21.15.16:0/177017988 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6f415bc70 msgr2=0x7fd6f406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.992+0000 7fd6fa600640 1 --2- 172.21.15.16:0/177017988 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6f415bc70 0x7fd6f406ffe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009900 tx=0x7fd6dc01c880 comp rx=0 tx=0).stop 2024-09-10T21:18:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.992+0000 7fd6fa600640 1 -- 172.21.15.16:0/177017988 shutdown_connections 2024-09-10T21:18:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.992+0000 7fd6fa600640 1 --2- 172.21.15.16:0/177017988 >> v2:172.21.15.52:3300/0 conn(0x7fd6f4070520 0x7fd6f40ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.992+0000 7fd6fa600640 1 --2- 172.21.15.16:0/177017988 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6f415bc70 0x7fd6f406ffe0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.992+0000 7fd6fa600640 1 -- 172.21.15.16:0/177017988 >> 172.21.15.16:0/177017988 conn(0x7fd6f40cfb20 msgr2=0x7fd6f40cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.992+0000 7fd6fa600640 1 -- 172.21.15.16:0/177017988 shutdown_connections 2024-09-10T21:18:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.992+0000 7fd6fa600640 1 -- 172.21.15.16:0/177017988 wait complete. 2024-09-10T21:18:13.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.992+0000 7fd6fa600640 1 Processor -- start 2024-09-10T21:18:13.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.992+0000 7fd6fa600640 1 -- start start 2024-09-10T21:18:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6fa600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6f4070520 0x7fd6f41443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6fa600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6f4154f90 0x7fd6f414b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6fa600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f415bc70 0x7fd6f414bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6fa600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f4167b00 con 0x7fd6f415bc70 2024-09-10T21:18:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6fa600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f4167980 con 0x7fd6f4070520 2024-09-10T21:18:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6fa600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f4167c80 con 0x7fd6f4154f90 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f415bc70 0x7fd6f414bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f415bc70 0x7fd6f414bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52674/0 (socket says 172.21.15.16:52674) 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6f3e00640 1 -- 172.21.15.16:0/2220900397 learned_addr learned my addr 172.21.15.16:0/2220900397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6f3400640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6f4070520 0x7fd6f41443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6ebe00640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6f4154f90 0x7fd6f414b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6f3e00640 1 -- 172.21.15.16:0/2220900397 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6f4154f90 msgr2=0x7fd6f414b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6f3e00640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6f4154f90 0x7fd6f414b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6f3e00640 1 -- 172.21.15.16:0/2220900397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6f4070520 msgr2=0x7fd6f41443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6f3e00640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6f4070520 0x7fd6f41443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6f3e00640 1 -- 172.21.15.16:0/2220900397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f4144ee0 con 0x7fd6f415bc70 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6f3e00640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f415bc70 0x7fd6f414bc70 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00eb20 tx=0x7fd6ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6eb400640 1 -- 172.21.15.16:0/2220900397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec004b30 con 0x7fd6f415bc70 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6eb400640 1 -- 172.21.15.16:0/2220900397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6ec004cf0 con 0x7fd6f415bc70 2024-09-10T21:18:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6eb400640 1 -- 172.21.15.16:0/2220900397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec005280 con 0x7fd6f415bc70 2024-09-10T21:18:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6fa600640 1 -- 172.21.15.16:0/2220900397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f4144c00 con 0x7fd6f415bc70 2024-09-10T21:18:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6fa600640 1 -- 172.21.15.16:0/2220900397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f40c43e0 con 0x7fd6f415bc70 2024-09-10T21:18:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6fa600640 1 -- 172.21.15.16:0/2220900397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f40ca530 con 0x7fd6f415bc70 2024-09-10T21:18:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:13.996+0000 7fd6eb400640 1 -- 172.21.15.16:0/2220900397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd6ec005420 con 0x7fd6f415bc70 2024-09-10T21:18:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.000+0000 7fd6eb400640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd6d0072610 0x7fd6d0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.000+0000 7fd6eb400640 1 -- 172.21.15.16:0/2220900397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4857+0+0 (secure 0 0 0) 0x7fd6ec093bf0 con 0x7fd6f415bc70 2024-09-10T21:18:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.000+0000 7fd6f3400640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd6d0072610 0x7fd6d0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.000+0000 7fd6f3400640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd6d0072610 0x7fd6d0074ad0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4000f70 tx=0x7fd6e4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.004+0000 7fd6eb400640 1 -- 172.21.15.16:0/2220900397 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd6ec05ee20 con 0x7fd6f415bc70 2024-09-10T21:18:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6fa600640 1 -- 172.21.15.16:0/2220900397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd6f406f9f0 con 0x7fd6f415bc70 2024-09-10T21:18:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6eb400640 1 -- 172.21.15.16:0/2220900397 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v123) ==== 84+0+65 (secure 0 0 0) 0x7fd6ec063850 con 0x7fd6f415bc70 2024-09-10T21:18:14.185 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:14.185 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-10T21:18:14.185 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:14.185 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-10T21:18:14.185 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-10T21:18:14.185 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:14.185 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-10T21:18:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6fa600640 1 -- 172.21.15.16:0/2220900397 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd6d0072610 msgr2=0x7fd6d0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6fa600640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd6d0072610 0x7fd6d0074ad0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4000f70 tx=0x7fd6e4009040 comp rx=0 tx=0).stop 2024-09-10T21:18:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6fa600640 1 -- 172.21.15.16:0/2220900397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f415bc70 msgr2=0x7fd6f414bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6fa600640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f415bc70 0x7fd6f414bc70 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00eb20 tx=0x7fd6ec00c2d0 comp rx=0 tx=0).stop 2024-09-10T21:18:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6fa600640 1 -- 172.21.15.16:0/2220900397 shutdown_connections 2024-09-10T21:18:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6fa600640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd6d0072610 0x7fd6d0074ad0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6fa600640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f415bc70 0x7fd6f414bc70 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6fa600640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6f4154f90 0x7fd6f414b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6fa600640 1 --2- 172.21.15.16:0/2220900397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6f4070520 0x7fd6f41443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6fa600640 1 -- 172.21.15.16:0/2220900397 >> 172.21.15.16:0/2220900397 conn(0x7fd6f40cfb20 msgr2=0x7fd6f414e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6fa600640 1 -- 172.21.15.16:0/2220900397 shutdown_connections 2024-09-10T21:18:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.180+0000 7fd6fa600640 1 -- 172.21.15.16:0/2220900397 wait complete. 2024-09-10T21:18:14.207 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:18:14.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:18:14.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.352+0000 7fcc07200640 1 -- 172.21.15.16:0/3064138336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc080ce9f0 msgr2=0x7fcc080cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:14.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.352+0000 7fcc07200640 1 --2- 172.21.15.16:0/3064138336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc080ce9f0 0x7fcc080cfe50 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009bc0 tx=0x7fcc0001cba0 comp rx=0 tx=0).stop 2024-09-10T21:18:14.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.352+0000 7fcc07200640 1 -- 172.21.15.16:0/3064138336 shutdown_connections 2024-09-10T21:18:14.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.352+0000 7fcc07200640 1 --2- 172.21.15.16:0/3064138336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc080ce9f0 0x7fcc080cfe50 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:14.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.352+0000 7fcc07200640 1 --2- 172.21.15.16:0/3064138336 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcc080ce610 0x7fcc080d2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:14.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.352+0000 7fcc07200640 1 -- 172.21.15.16:0/3064138336 >> 172.21.15.16:0/3064138336 conn(0x7fcc080d1b70 msgr2=0x7fcc080d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:14.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.352+0000 7fcc07200640 1 -- 172.21.15.16:0/3064138336 shutdown_connections 2024-09-10T21:18:14.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.352+0000 7fcc07200640 1 -- 172.21.15.16:0/3064138336 wait complete. 2024-09-10T21:18:14.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.352+0000 7fcc07200640 1 Processor -- start 2024-09-10T21:18:14.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.352+0000 7fcc07200640 1 -- start start 2024-09-10T21:18:14.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc07200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc080b2a10 0x7fcc080cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:14.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc07200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc080ce610 0x7fcc0815b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:14.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc07200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcc080ce9f0 0x7fcc080c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc07200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc080b2460 con 0x7fcc080b2a10 2024-09-10T21:18:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc07200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcc080b22e0 con 0x7fcc080ce610 2024-09-10T21:18:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc07200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fcc080b25e0 con 0x7fcc080ce9f0 2024-09-10T21:18:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc05e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc080b2a10 0x7fcc080cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc05e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc080b2a10 0x7fcc080cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52688/0 (socket says 172.21.15.16:52688) 2024-09-10T21:18:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc05e00640 1 -- 172.21.15.16:0/2768864814 learned_addr learned my addr 172.21.15.16:0/2768864814 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc06800640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcc080ce9f0 0x7fcc080c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc05e00640 1 -- 172.21.15.16:0/2768864814 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcc080ce9f0 msgr2=0x7fcc080c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc05e00640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcc080ce9f0 0x7fcc080c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcbffe00640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc080ce610 0x7fcc0815b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc05e00640 1 -- 172.21.15.16:0/2768864814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc080ce610 msgr2=0x7fcc0815b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:14.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc05e00640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc080ce610 0x7fcc0815b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:14.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc05e00640 1 -- 172.21.15.16:0/2768864814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc080cbee0 con 0x7fcc080b2a10 2024-09-10T21:18:14.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc06800640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcc080ce9f0 0x7fcc080c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:14.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcbffe00640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc080ce610 0x7fcc0815b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:14.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc05e00640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc080b2a10 0x7fcc080cb9a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800c9a0 tx=0x7fcbf800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:14.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcbfe000640 1 -- 172.21.15.16:0/2768864814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8007c10 con 0x7fcc080b2a10 2024-09-10T21:18:14.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcbfe000640 1 -- 172.21.15.16:0/2768864814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbf800f450 con 0x7fcc080b2a10 2024-09-10T21:18:14.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcbfe000640 1 -- 172.21.15.16:0/2768864814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf800f8a0 con 0x7fcc080b2a10 2024-09-10T21:18:14.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc07200640 1 -- 172.21.15.16:0/2768864814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc080ca620 con 0x7fcc080b2a10 2024-09-10T21:18:14.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc07200640 1 -- 172.21.15.16:0/2768864814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc080caec0 con 0x7fcc080b2a10 2024-09-10T21:18:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.356+0000 7fcc07200640 1 -- 172.21.15.16:0/2768864814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc080b4430 con 0x7fcc080b2a10 2024-09-10T21:18:14.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.360+0000 7fcbfe000640 1 -- 172.21.15.16:0/2768864814 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcbf8020030 con 0x7fcc080b2a10 2024-09-10T21:18:14.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.360+0000 7fcbfe000640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbe40725e0 0x7fcbe4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:14.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.360+0000 7fcbfe000640 1 -- 172.21.15.16:0/2768864814 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4857+0+0 (secure 0 0 0) 0x7fcbf8093460 con 0x7fcc080b2a10 2024-09-10T21:18:14.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.360+0000 7fcbffe00640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbe40725e0 0x7fcbe4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:14.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.360+0000 7fcbffe00640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbe40725e0 0x7fcbe4074aa0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0001230 tx=0x7fcbf0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:14.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.364+0000 7fcbfe000640 1 -- 172.21.15.16:0/2768864814 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcbf8014030 con 0x7fcc080b2a10 2024-09-10T21:18:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:14.544+0000 7fcc07200640 1 -- 172.21.15.16:0/2768864814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcc080d2610 con 0x7fcc080b2a10 2024-09-10T21:18:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.416+0000 7fcbfe000640 1 -- 172.21.15.16:0/2768864814 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v124) ==== 138+0+0 (secure 0 0 0) 0x7fcbf805edf0 con 0x7fcc080b2a10 2024-09-10T21:18:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-10T21:18:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.420+0000 7fcc07200640 1 -- 172.21.15.16:0/2768864814 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbe40725e0 msgr2=0x7fcbe4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.420+0000 7fcc07200640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbe40725e0 0x7fcbe4074aa0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0001230 tx=0x7fcbf0009040 comp rx=0 tx=0).stop 2024-09-10T21:18:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.420+0000 7fcc07200640 1 -- 172.21.15.16:0/2768864814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc080b2a10 msgr2=0x7fcc080cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.420+0000 7fcc07200640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc080b2a10 0x7fcc080cb9a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800c9a0 tx=0x7fcbf800ce60 comp rx=0 tx=0).stop 2024-09-10T21:18:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.420+0000 7fcc07200640 1 -- 172.21.15.16:0/2768864814 shutdown_connections 2024-09-10T21:18:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.420+0000 7fcc07200640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fcbe40725e0 0x7fcbe4074aa0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:15.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.420+0000 7fcc07200640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fcc080ce9f0 0x7fcc080c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.420+0000 7fcc07200640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc080ce610 0x7fcc0815b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.420+0000 7fcc07200640 1 --2- 172.21.15.16:0/2768864814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc080b2a10 0x7fcc080cb9a0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.420+0000 7fcc07200640 1 -- 172.21.15.16:0/2768864814 >> 172.21.15.16:0/2768864814 conn(0x7fcc080d1b70 msgr2=0x7fcc080afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.420+0000 7fcc07200640 1 -- 172.21.15.16:0/2768864814 shutdown_connections 2024-09-10T21:18:15.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.424+0000 7fcc07200640 1 -- 172.21.15.16:0/2768864814 wait complete. 2024-09-10T21:18:15.455 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:18:15.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.608+0000 7ff8d1800640 1 -- 172.21.15.16:0/535948787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc0ce040 msgr2=0x7ff8cc1671a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:15.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.608+0000 7ff8d1800640 1 --2- 172.21.15.16:0/535948787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc0ce040 0x7ff8cc1671a0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0cda10 tx=0x7ff8c401f390 comp rx=0 tx=0).stop 2024-09-10T21:18:15.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.608+0000 7ff8d1800640 1 -- 172.21.15.16:0/535948787 shutdown_connections 2024-09-10T21:18:15.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.608+0000 7ff8d1800640 1 --2- 172.21.15.16:0/535948787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc0ce040 0x7ff8cc1671a0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:15.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.608+0000 7ff8d1800640 1 -- 172.21.15.16:0/535948787 >> 172.21.15.16:0/535948787 conn(0x7ff8cc0d1df0 msgr2=0x7ff8cc0d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:15.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.608+0000 7ff8d1800640 1 -- 172.21.15.16:0/535948787 shutdown_connections 2024-09-10T21:18:15.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8d1800640 1 -- 172.21.15.16:0/535948787 wait complete. 2024-09-10T21:18:15.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8d1800640 1 Processor -- start 2024-09-10T21:18:15.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8d1800640 1 -- start start 2024-09-10T21:18:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8d1800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8cc0ce040 0x7ff8cc14ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8d1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc0cfe30 0x7ff8cc156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8d1800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff8cc0d0210 0x7ff8cc155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8d1800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc0d2a90 con 0x7ff8cc0cfe30 2024-09-10T21:18:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8d1800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc0d2910 con 0x7ff8cc0ce040 2024-09-10T21:18:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8d1800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff8cc0b12e0 con 0x7ff8cc0d0210 2024-09-10T21:18:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc0cfe30 0x7ff8cc156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc0cfe30 0x7ff8cc156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52720/0 (socket says 172.21.15.16:52720) 2024-09-10T21:18:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8c3e00640 1 -- 172.21.15.16:0/2797522071 learned_addr learned my addr 172.21.15.16:0/2797522071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8cb400640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff8cc0d0210 0x7ff8cc155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8caa00640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8cc0ce040 0x7ff8cc14ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8c3e00640 1 -- 172.21.15.16:0/2797522071 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff8cc0d0210 msgr2=0x7ff8cc155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8c3e00640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff8cc0d0210 0x7ff8cc155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8c3e00640 1 -- 172.21.15.16:0/2797522071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8cc0ce040 msgr2=0x7ff8cc14ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8c3e00640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8cc0ce040 0x7ff8cc14ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8c3e00640 1 -- 172.21.15.16:0/2797522071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc0b5bd0 con 0x7ff8cc0cfe30 2024-09-10T21:18:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8caa00640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8cc0ce040 0x7ff8cc14ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.612+0000 7ff8c3e00640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc0cfe30 0x7ff8cc156e60 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400c400 tx=0x7ff8b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.616+0000 7ff8c2a00640 1 -- 172.21.15.16:0/2797522071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4011790 con 0x7ff8cc0cfe30 2024-09-10T21:18:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.616+0000 7ff8c2a00640 1 -- 172.21.15.16:0/2797522071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8b4017070 con 0x7ff8cc0cfe30 2024-09-10T21:18:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.616+0000 7ff8c2a00640 1 -- 172.21.15.16:0/2797522071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4011ce0 con 0x7ff8cc0cfe30 2024-09-10T21:18:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.616+0000 7ff8d1800640 1 -- 172.21.15.16:0/2797522071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc0b5eb0 con 0x7ff8cc0cfe30 2024-09-10T21:18:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.616+0000 7ff8d1800640 1 -- 172.21.15.16:0/2797522071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc0b6470 con 0x7ff8cc0cfe30 2024-09-10T21:18:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.616+0000 7ff8d1800640 1 -- 172.21.15.16:0/2797522071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8cc0ce590 con 0x7ff8cc0cfe30 2024-09-10T21:18:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.616+0000 7ff8c2a00640 1 -- 172.21.15.16:0/2797522071 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff8b401c460 con 0x7ff8cc0cfe30 2024-09-10T21:18:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.620+0000 7ff8c2a00640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff8a80725e0 0x7ff8a8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.620+0000 7ff8c2a00640 1 -- 172.21.15.16:0/2797522071 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5227+0+0 (secure 0 0 0) 0x7ff8b40937c0 con 0x7ff8cc0cfe30 2024-09-10T21:18:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.620+0000 7ff8caa00640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff8a80725e0 0x7ff8a8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.620+0000 7ff8caa00640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff8a80725e0 0x7ff8a8074aa0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc000c10 tx=0x7ff8bc009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.624+0000 7ff8c2a00640 1 -- 172.21.15.16:0/2797522071 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff8b405f0f0 con 0x7ff8cc0cfe30 2024-09-10T21:18:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:15.800+0000 7ff8d1800640 1 -- 172.21.15.16:0/2797522071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff8cc0cc290 con 0x7ff8cc0cfe30 2024-09-10T21:18:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.424+0000 7ff8c2a00640 1 -- 172.21.15.16:0/2797522071 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v125) ==== 170+0+0 (secure 0 0 0) 0x7ff8b405e9f0 con 0x7ff8cc0cfe30 2024-09-10T21:18:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-10T21:18:16.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.428+0000 7ff8d1800640 1 -- 172.21.15.16:0/2797522071 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff8a80725e0 msgr2=0x7ff8a8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:16.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.428+0000 7ff8d1800640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff8a80725e0 0x7ff8a8074aa0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc000c10 tx=0x7ff8bc009040 comp rx=0 tx=0).stop 2024-09-10T21:18:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.428+0000 7ff8d1800640 1 -- 172.21.15.16:0/2797522071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc0cfe30 msgr2=0x7ff8cc156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.428+0000 7ff8d1800640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc0cfe30 0x7ff8cc156e60 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400c400 tx=0x7ff8b400c8c0 comp rx=0 tx=0).stop 2024-09-10T21:18:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.428+0000 7ff8d1800640 1 -- 172.21.15.16:0/2797522071 shutdown_connections 2024-09-10T21:18:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.428+0000 7ff8d1800640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff8a80725e0 0x7ff8a8074aa0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.428+0000 7ff8d1800640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff8cc0d0210 0x7ff8cc155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.428+0000 7ff8d1800640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc0cfe30 0x7ff8cc156e60 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.428+0000 7ff8d1800640 1 --2- 172.21.15.16:0/2797522071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8cc0ce040 0x7ff8cc14ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.428+0000 7ff8d1800640 1 -- 172.21.15.16:0/2797522071 >> 172.21.15.16:0/2797522071 conn(0x7ff8cc0d1df0 msgr2=0x7ff8cc0af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.432+0000 7ff8d1800640 1 -- 172.21.15.16:0/2797522071 shutdown_connections 2024-09-10T21:18:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.432+0000 7ff8d1800640 1 -- 172.21.15.16:0/2797522071 wait complete. 2024-09-10T21:18:16.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:18:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 -- 172.21.15.16:0/4026834646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8151210 msgr2=0x7f25c80726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 --2- 172.21.15.16:0/4026834646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8151210 0x7f25c80726b0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009900 tx=0x7f25b801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:18:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 -- 172.21.15.16:0/4026834646 shutdown_connections 2024-09-10T21:18:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 --2- 172.21.15.16:0/4026834646 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f25c8072bf0 0x7f25c815dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 --2- 172.21.15.16:0/4026834646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8151210 0x7f25c80726b0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 -- 172.21.15.16:0/4026834646 >> 172.21.15.16:0/4026834646 conn(0x7f25c80b16a0 msgr2=0x7f25c80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 -- 172.21.15.16:0/4026834646 shutdown_connections 2024-09-10T21:18:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 -- 172.21.15.16:0/4026834646 wait complete. 2024-09-10T21:18:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 Processor -- start 2024-09-10T21:18:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 -- start start 2024-09-10T21:18:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8072bf0 0x7f25c813c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f25c8151210 0x7f25c81fb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815e2c0 0x7f25c80af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f25c815b320 con 0x7f25c8072bf0 2024-09-10T21:18:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f25c815b1a0 con 0x7f25c815e2c0 2024-09-10T21:18:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cf000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f25c815b4a0 con 0x7f25c8151210 2024-09-10T21:18:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25c7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8072bf0 0x7f25c813c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25c7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8072bf0 0x7f25c813c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52740/0 (socket says 172.21.15.16:52740) 2024-09-10T21:18:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25c7e00640 1 -- 172.21.15.16:0/9858647 learned_addr learned my addr 172.21.15.16:0/9858647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25bf400640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f25c8151210 0x7f25c81fb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25cd000640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815e2c0 0x7f25c80af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25c7e00640 1 -- 172.21.15.16:0/9858647 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f25c8151210 msgr2=0x7f25c81fb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25c7e00640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f25c8151210 0x7f25c81fb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25c7e00640 1 -- 172.21.15.16:0/9858647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815e2c0 msgr2=0x7f25c80af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25c7e00640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815e2c0 0x7f25c80af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.624+0000 7f25c7e00640 1 -- 172.21.15.16:0/9858647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c81fc000 con 0x7f25c8072bf0 2024-09-10T21:18:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.628+0000 7f25c7e00640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8072bf0 0x7f25c813c730 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009850 tx=0x7f25b80a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.628+0000 7f25c4c00640 1 -- 172.21.15.16:0/9858647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b80b2040 con 0x7f25c8072bf0 2024-09-10T21:18:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.628+0000 7f25c4c00640 1 -- 172.21.15.16:0/9858647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25b8004380 con 0x7f25c8072bf0 2024-09-10T21:18:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.628+0000 7f25cf000640 1 -- 172.21.15.16:0/9858647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c81fc2e0 con 0x7f25c8072bf0 2024-09-10T21:18:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.628+0000 7f25c4c00640 1 -- 172.21.15.16:0/9858647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b80052d0 con 0x7f25c8072bf0 2024-09-10T21:18:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.628+0000 7f25cf000640 1 -- 172.21.15.16:0/9858647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c813d4f0 con 0x7f25c8072bf0 2024-09-10T21:18:16.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.628+0000 7f25cf000640 1 -- 172.21.15.16:0/9858647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c815dd80 con 0x7f25c8072bf0 2024-09-10T21:18:16.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.628+0000 7f25c4c00640 1 -- 172.21.15.16:0/9858647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f25b8005470 con 0x7f25c8072bf0 2024-09-10T21:18:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.628+0000 7f25c4c00640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f25a4072610 0x7f25a4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.628+0000 7f25c4c00640 1 -- 172.21.15.16:0/9858647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5609+0+0 (secure 0 0 0) 0x7f25b812cb60 con 0x7f25c8072bf0 2024-09-10T21:18:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.632+0000 7f25bf400640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f25a4072610 0x7f25a4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.632+0000 7f25bf400640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f25a4072610 0x7f25a4074ad0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f25c8150870 tx=0x7f25b000b490 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:16.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.632+0000 7f25c4c00640 1 -- 172.21.15.16:0/9858647 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f25b80164d0 con 0x7f25c8072bf0 2024-09-10T21:18:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:16.848+0000 7f25cf000640 1 -- 172.21.15.16:0/9858647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f25c80b2c50 con 0x7f25c8072bf0 2024-09-10T21:18:17.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.428+0000 7f25c4c00640 1 -- 172.21.15.16:0/9858647 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v89) ==== 174+0+0 (secure 0 0 0) 0x7f25b80f7c40 con 0x7f25c8072bf0 2024-09-10T21:18:17.433 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 20 and data pool 21 2024-09-10T21:18:17.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.432+0000 7f25cf000640 1 -- 172.21.15.16:0/9858647 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f25a4072610 msgr2=0x7f25a4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:17.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.432+0000 7f25cf000640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f25a4072610 0x7f25a4074ad0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f25c8150870 tx=0x7f25b000b490 comp rx=0 tx=0).stop 2024-09-10T21:18:17.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.432+0000 7f25cf000640 1 -- 172.21.15.16:0/9858647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8072bf0 msgr2=0x7f25c813c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:17.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.432+0000 7f25cf000640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8072bf0 0x7f25c813c730 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009850 tx=0x7f25b80a55a0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.432+0000 7f25cf000640 1 -- 172.21.15.16:0/9858647 shutdown_connections 2024-09-10T21:18:17.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.432+0000 7f25cf000640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f25a4072610 0x7f25a4074ad0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.432+0000 7f25cf000640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25c815e2c0 0x7f25c80af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.432+0000 7f25cf000640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f25c8151210 0x7f25c81fb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.432+0000 7f25cf000640 1 --2- 172.21.15.16:0/9858647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8072bf0 0x7f25c813c730 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.432+0000 7f25cf000640 1 -- 172.21.15.16:0/9858647 >> 172.21.15.16:0/9858647 conn(0x7f25c80b16a0 msgr2=0x7f25c80d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:17.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.432+0000 7f25cf000640 1 -- 172.21.15.16:0/9858647 shutdown_connections 2024-09-10T21:18:17.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.432+0000 7f25cf000640 1 -- 172.21.15.16:0/9858647 wait complete. 2024-09-10T21:18:17.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:18:17.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 -- 172.21.15.16:0/2504434115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ffc1599d0 msgr2=0x7f7ffc15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:17.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 --2- 172.21.15.16:0/2504434115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ffc1599d0 0x7f7ffc15f830 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009840 tx=0x7f7fec01c790 comp rx=0 tx=0).stop 2024-09-10T21:18:17.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 -- 172.21.15.16:0/2504434115 shutdown_connections 2024-09-10T21:18:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 --2- 172.21.15.16:0/2504434115 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ffc06e640 0x7f7ffc0ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 --2- 172.21.15.16:0/2504434115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ffc1599d0 0x7f7ffc15f830 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 -- 172.21.15.16:0/2504434115 >> 172.21.15.16:0/2504434115 conn(0x7f7ffc0cfb20 msgr2=0x7f7ffc0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 -- 172.21.15.16:0/2504434115 shutdown_connections 2024-09-10T21:18:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 -- 172.21.15.16:0/2504434115 wait complete. 2024-09-10T21:18:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 Processor -- start 2024-09-10T21:18:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 -- start start 2024-09-10T21:18:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ffc06e640 0x7f7ffc074360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ffc0ca6b0 0x7f7ffc0c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ffc1599d0 0x7f7ffc156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc168d50 con 0x7f7ffc0ca6b0 2024-09-10T21:18:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc168bd0 con 0x7f7ffc1599d0 2024-09-10T21:18:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.624+0000 7f8002a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc168ed0 con 0x7f7ffc06e640 2024-09-10T21:18:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ff3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ffc0ca6b0 0x7f7ffc0c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ff3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ffc0ca6b0 0x7f7ffc0c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52744/0 (socket says 172.21.15.16:52744) 2024-09-10T21:18:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ff3e00640 1 -- 172.21.15.16:0/3832507904 learned_addr learned my addr 172.21.15.16:0/3832507904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f8000a00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ffc1599d0 0x7f7ffc156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ffbe00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ffc06e640 0x7f7ffc074360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ff3e00640 1 -- 172.21.15.16:0/3832507904 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ffc06e640 msgr2=0x7f7ffc074360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ff3e00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ffc06e640 0x7f7ffc074360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ff3e00640 1 -- 172.21.15.16:0/3832507904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ffc1599d0 msgr2=0x7f7ffc156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ff3e00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ffc1599d0 0x7f7ffc156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ff3e00640 1 -- 172.21.15.16:0/3832507904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc156870 con 0x7f7ffc0ca6b0 2024-09-10T21:18:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f8000a00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ffc1599d0 0x7f7ffc156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ff3e00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ffc0ca6b0 0x7f7ffc0c46a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400e9c0 tx=0x7f7fe400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ff8c00640 1 -- 172.21.15.16:0/3832507904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400cd80 con 0x7f7ffc0ca6b0 2024-09-10T21:18:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ff8c00640 1 -- 172.21.15.16:0/3832507904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fe4004020 con 0x7f7ffc0ca6b0 2024-09-10T21:18:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ff8c00640 1 -- 172.21.15.16:0/3832507904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4004450 con 0x7f7ffc0ca6b0 2024-09-10T21:18:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f8002a00640 1 -- 172.21.15.16:0/3832507904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc0c77a0 con 0x7f7ffc0ca6b0 2024-09-10T21:18:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f8002a00640 1 -- 172.21.15.16:0/3832507904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc156c30 con 0x7f7ffc0ca6b0 2024-09-10T21:18:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f8002a00640 1 -- 172.21.15.16:0/3832507904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ffc0c7cc0 con 0x7f7ffc0ca6b0 2024-09-10T21:18:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.628+0000 7f7ff8c00640 1 -- 172.21.15.16:0/3832507904 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7fe4003600 con 0x7f7ffc0ca6b0 2024-09-10T21:18:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.632+0000 7f7ff8c00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fd80725e0 0x7f7fd8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:17.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.632+0000 7f7ff8c00640 1 -- 172.21.15.16:0/3832507904 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5709+0+0 (secure 0 0 0) 0x7f7fe4093d10 con 0x7f7ffc0ca6b0 2024-09-10T21:18:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.632+0000 7f7ffbe00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fd80725e0 0x7f7fd8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.632+0000 7f7ffbe00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fd80725e0 0x7f7fd8074aa0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0af000 tx=0x7f7fec01afd0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.636+0000 7f7ff8c00640 1 -- 172.21.15.16:0/3832507904 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7fe4099020 con 0x7f7ffc0ca6b0 2024-09-10T21:18:17.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.812+0000 7f8002a00640 1 -- 172.21.15.16:0/3832507904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ffc0ca170 con 0x7f7ffc0ca6b0 2024-09-10T21:18:17.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.812+0000 7f7ff8c00640 1 -- 172.21.15.16:0/3832507904 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16704 (secure 0 0 0) 0x7f7fe405f350 con 0x7f7ffc0ca6b0 2024-09-10T21:18:17.818 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:17.818 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":126,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:18:16.852290+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:18:14.559149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T21:18:15.813518+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:18:17.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.816+0000 7f8002a00640 1 -- 172.21.15.16:0/3832507904 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fd80725e0 msgr2=0x7f7fd8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:17.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.816+0000 7f8002a00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fd80725e0 0x7f7fd8074aa0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0af000 tx=0x7f7fec01afd0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.816+0000 7f8002a00640 1 -- 172.21.15.16:0/3832507904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ffc0ca6b0 msgr2=0x7f7ffc0c46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:17.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.816+0000 7f8002a00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ffc0ca6b0 0x7f7ffc0c46a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400e9c0 tx=0x7f7fe400ee80 comp rx=0 tx=0).stop 2024-09-10T21:18:17.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.816+0000 7f8002a00640 1 -- 172.21.15.16:0/3832507904 shutdown_connections 2024-09-10T21:18:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.816+0000 7f8002a00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7fd80725e0 0x7f7fd8074aa0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.816+0000 7f8002a00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ffc1599d0 0x7f7ffc156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.816+0000 7f8002a00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ffc0ca6b0 0x7f7ffc0c46a0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.816+0000 7f8002a00640 1 --2- 172.21.15.16:0/3832507904 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7ffc06e640 0x7f7ffc074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.816+0000 7f8002a00640 1 -- 172.21.15.16:0/3832507904 >> 172.21.15.16:0/3832507904 conn(0x7f7ffc0cfb20 msgr2=0x7f7ffc0cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.816+0000 7f8002a00640 1 -- 172.21.15.16:0/3832507904 shutdown_connections 2024-09-10T21:18:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.816+0000 7f8002a00640 1 -- 172.21.15.16:0/3832507904 wait complete. 2024-09-10T21:18:17.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:18:17.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.980+0000 7f82ab000640 1 -- 172.21.15.16:0/444547637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f82a40b13c0 msgr2=0x7f82a4071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:17.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.980+0000 7f82ab000640 1 --2- 172.21.15.16:0/444547637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f82a40b13c0 0x7f82a4071520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f828c009840 tx=0x7f828c01c700 comp rx=0 tx=0).stop 2024-09-10T21:18:17.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.980+0000 7f82ab000640 1 -- 172.21.15.16:0/444547637 shutdown_connections 2024-09-10T21:18:17.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.980+0000 7f82ab000640 1 --2- 172.21.15.16:0/444547637 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f82a40b13c0 0x7f82a4071520 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.980+0000 7f82ab000640 1 --2- 172.21.15.16:0/444547637 >> v2:172.21.15.52:3300/0 conn(0x7f82a415f400 0x7f82a4070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.980+0000 7f82ab000640 1 -- 172.21.15.16:0/444547637 >> 172.21.15.16:0/444547637 conn(0x7f82a409b130 msgr2=0x7f82a40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:17.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.980+0000 7f82ab000640 1 -- 172.21.15.16:0/444547637 shutdown_connections 2024-09-10T21:18:17.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.980+0000 7f82ab000640 1 -- 172.21.15.16:0/444547637 wait complete. 2024-09-10T21:18:17.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82ab000640 1 Processor -- start 2024-09-10T21:18:17.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82ab000640 1 -- start start 2024-09-10T21:18:17.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82ab000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82a40b13c0 0x7f82a40b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:17.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82ab000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a40cfc80 0x7f82a415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:17.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82ab000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f82a415f400 0x7f82a40b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:17.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82ab000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4169770 con 0x7f82a40cfc80 2024-09-10T21:18:17.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82ab000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41695f0 con 0x7f82a40b13c0 2024-09-10T21:18:17.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82ab000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f82a41698f0 con 0x7f82a415f400 2024-09-10T21:18:17.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f829be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a40cfc80 0x7f82a415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:17.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f829be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a40cfc80 0x7f82a415e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52756/0 (socket says 172.21.15.16:52756) 2024-09-10T21:18:17.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f829be00640 1 -- 172.21.15.16:0/2046108988 learned_addr learned my addr 172.21.15.16:0/2046108988 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:17.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82a3e00640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82a40b13c0 0x7f82a40b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:17.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82a9000640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f82a415f400 0x7f82a40b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:17.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f829be00640 1 -- 172.21.15.16:0/2046108988 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f82a415f400 msgr2=0x7f82a40b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:17.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f829be00640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f82a415f400 0x7f82a40b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f829be00640 1 -- 172.21.15.16:0/2046108988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82a40b13c0 msgr2=0x7f82a40b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:17.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f829be00640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82a40b13c0 0x7f82a40b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:17.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f829be00640 1 -- 172.21.15.16:0/2046108988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a415d250 con 0x7f82a40cfc80 2024-09-10T21:18:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f829be00640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a40cfc80 0x7f82a415e2f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f828c01cfa0 tx=0x7f828c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82a0c00640 1 -- 172.21.15.16:0/2046108988 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c0be020 con 0x7f82a40cfc80 2024-09-10T21:18:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82a0c00640 1 -- 172.21.15.16:0/2046108988 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f828c004130 con 0x7f82a40cfc80 2024-09-10T21:18:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82a0c00640 1 -- 172.21.15.16:0/2046108988 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c003600 con 0x7f82a40cfc80 2024-09-10T21:18:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82ab000640 1 -- 172.21.15.16:0/2046108988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a415d530 con 0x7f82a40cfc80 2024-09-10T21:18:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.984+0000 7f82ab000640 1 -- 172.21.15.16:0/2046108988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a415daf0 con 0x7f82a40cfc80 2024-09-10T21:18:17.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.988+0000 7f82ab000640 1 -- 172.21.15.16:0/2046108988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a40d00d0 con 0x7f82a40cfc80 2024-09-10T21:18:17.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.988+0000 7f82a0c00640 1 -- 172.21.15.16:0/2046108988 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f828c0037a0 con 0x7f82a40cfc80 2024-09-10T21:18:17.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.988+0000 7f82a0c00640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8280072610 0x7f8280074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:17.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.988+0000 7f82a0c00640 1 -- 172.21.15.16:0/2046108988 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5709+0+0 (secure 0 0 0) 0x7f828c12e4e0 con 0x7f82a40cfc80 2024-09-10T21:18:17.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.988+0000 7f82a3e00640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8280072610 0x7f8280074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:17.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.988+0000 7f82a3e00640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8280072610 0x7f8280074ad0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8294005fd0 tx=0x7f8294005f40 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:17.992+0000 7f82a0c00640 1 -- 172.21.15.16:0/2046108988 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f828c0c8030 con 0x7f82a40cfc80 2024-09-10T21:18:18.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.168+0000 7f82ab000640 1 -- 172.21.15.16:0/2046108988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82a406f870 con 0x7f82a40cfc80 2024-09-10T21:18:18.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.168+0000 7f82a0c00640 1 -- 172.21.15.16:0/2046108988 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16704 (secure 0 0 0) 0x7f828c016410 con 0x7f82a40cfc80 2024-09-10T21:18:18.174 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:18.175 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":126,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:18:16.852290+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:18:14.559149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T21:18:15.813518+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:18:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.172+0000 7f82ab000640 1 -- 172.21.15.16:0/2046108988 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8280072610 msgr2=0x7f8280074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.172+0000 7f82ab000640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8280072610 0x7f8280074ad0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8294005fd0 tx=0x7f8294005f40 comp rx=0 tx=0).stop 2024-09-10T21:18:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.172+0000 7f82ab000640 1 -- 172.21.15.16:0/2046108988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a40cfc80 msgr2=0x7f82a415e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.172+0000 7f82ab000640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a40cfc80 0x7f82a415e2f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f828c01cfa0 tx=0x7f828c005db0 comp rx=0 tx=0).stop 2024-09-10T21:18:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.172+0000 7f82ab000640 1 -- 172.21.15.16:0/2046108988 shutdown_connections 2024-09-10T21:18:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.172+0000 7f82ab000640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8280072610 0x7f8280074ad0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.172+0000 7f82ab000640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f82a415f400 0x7f82a40b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.172+0000 7f82ab000640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a40cfc80 0x7f82a415e2f0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.172+0000 7f82ab000640 1 --2- 172.21.15.16:0/2046108988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82a40b13c0 0x7f82a40b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.172+0000 7f82ab000640 1 -- 172.21.15.16:0/2046108988 >> 172.21.15.16:0/2046108988 conn(0x7f82a409b130 msgr2=0x7f82a40cd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.172+0000 7f82ab000640 1 -- 172.21.15.16:0/2046108988 shutdown_connections 2024-09-10T21:18:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.172+0000 7f82ab000640 1 -- 172.21.15.16:0/2046108988 wait complete. 2024-09-10T21:18:18.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:18:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.340+0000 7f7556e00640 1 -- 172.21.15.16:0/2665675310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75500d0210 msgr2=0x7f75500cead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.340+0000 7f7556e00640 1 --2- 172.21.15.16:0/2665675310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75500d0210 0x7f75500cead0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f75500cda10 tx=0x7f753c01efd0 comp rx=0 tx=0).stop 2024-09-10T21:18:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.340+0000 7f7556e00640 1 -- 172.21.15.16:0/2665675310 shutdown_connections 2024-09-10T21:18:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.340+0000 7f7556e00640 1 --2- 172.21.15.16:0/2665675310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75500d0210 0x7f75500cead0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.340+0000 7f7556e00640 1 --2- 172.21.15.16:0/2665675310 >> v2:172.21.15.52:3300/0 conn(0x7f75500cfe30 0x7f75500ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.340+0000 7f7556e00640 1 -- 172.21.15.16:0/2665675310 >> 172.21.15.16:0/2665675310 conn(0x7f75500d1df0 msgr2=0x7f75500d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.340+0000 7f7556e00640 1 -- 172.21.15.16:0/2665675310 shutdown_connections 2024-09-10T21:18:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.340+0000 7f7556e00640 1 -- 172.21.15.16:0/2665675310 wait complete. 2024-09-10T21:18:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f7556e00640 1 Processor -- start 2024-09-10T21:18:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f7556e00640 1 -- start start 2024-09-10T21:18:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f7556e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75500ce040 0x7f755015f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f7556e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75500cfe30 0x7f75500c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f7556e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75500d0210 0x7f75500ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f7556e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75500b12e0 con 0x7f75500cfe30 2024-09-10T21:18:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f7556e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f75500d2b20 con 0x7f75500d0210 2024-09-10T21:18:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f7556e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f75500b1460 con 0x7f75500ce040 2024-09-10T21:18:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f754f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75500cfe30 0x7f75500c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f754f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75500cfe30 0x7f75500c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52782/0 (socket says 172.21.15.16:52782) 2024-09-10T21:18:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f754f400640 1 -- 172.21.15.16:0/934268145 learned_addr learned my addr 172.21.15.16:0/934268145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f754f400640 1 -- 172.21.15.16:0/934268145 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75500ce040 msgr2=0x7f755015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f754f400640 1 --2- 172.21.15.16:0/934268145 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75500ce040 0x7f755015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f754f400640 1 -- 172.21.15.16:0/934268145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75500d0210 msgr2=0x7f75500ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f754f400640 1 --2- 172.21.15.16:0/934268145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75500d0210 0x7f75500ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f754f400640 1 -- 172.21.15.16:0/934268145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75501f49d0 con 0x7f75500cfe30 2024-09-10T21:18:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f754f400640 1 --2- 172.21.15.16:0/934268145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75500cfe30 0x7f75500c9f40 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f753c009790 tx=0x7f753c004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f7547e00640 1 -- 172.21.15.16:0/934268145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c0be070 con 0x7f75500cfe30 2024-09-10T21:18:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f7547e00640 1 -- 172.21.15.16:0/934268145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f753c005320 con 0x7f75500cfe30 2024-09-10T21:18:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f7547e00640 1 -- 172.21.15.16:0/934268145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c0b5040 con 0x7f75500cfe30 2024-09-10T21:18:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f7556e00640 1 -- 172.21.15.16:0/934268145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75501f4c90 con 0x7f75500cfe30 2024-09-10T21:18:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.344+0000 7f7556e00640 1 -- 172.21.15.16:0/934268145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75501f5250 con 0x7f75500cfe30 2024-09-10T21:18:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.348+0000 7f7556e00640 1 -- 172.21.15.16:0/934268145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75500cf2b0 con 0x7f75500cfe30 2024-09-10T21:18:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.348+0000 7f7547e00640 1 -- 172.21.15.16:0/934268145 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f753c0b8030 con 0x7f75500cfe30 2024-09-10T21:18:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.348+0000 7f7547e00640 1 --2- 172.21.15.16:0/934268145 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f75300725e0 0x7f7530074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.348+0000 7f7547e00640 1 -- 172.21.15.16:0/934268145 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5709+0+0 (secure 0 0 0) 0x7f753c12fe70 con 0x7f75500cfe30 2024-09-10T21:18:18.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.348+0000 7f754fe00640 1 --2- 172.21.15.16:0/934268145 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f75300725e0 0x7f7530074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:18.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.348+0000 7f754fe00640 1 --2- 172.21.15.16:0/934268145 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f75300725e0 0x7f7530074aa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f75400058d0 tx=0x7f7540005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:18.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.352+0000 7f7547e00640 1 -- 172.21.15.16:0/934268145 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f753c018b30 con 0x7f75500cfe30 2024-09-10T21:18:18.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:18.568+0000 7f7556e00640 1 -- 172.21.15.16:0/934268145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f75501e7520 con 0x7f75500cfe30 2024-09-10T21:18:19.415 INFO:tasks.ceph.mds.b.smithi052.stderr:2024-09-10T21:18:19.408+0000 7f3e8d600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:18:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.448+0000 7f7547e00640 1 -- 172.21.15.16:0/934268145 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v92) ==== 118+0+0 (secure 0 0 0) 0x7f753c0fb4b0 con 0x7f75500cfe30 2024-09-10T21:18:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.452+0000 7f7556e00640 1 -- 172.21.15.16:0/934268145 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f75300725e0 msgr2=0x7f7530074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.452+0000 7f7556e00640 1 --2- 172.21.15.16:0/934268145 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f75300725e0 0x7f7530074aa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f75400058d0 tx=0x7f7540005860 comp rx=0 tx=0).stop 2024-09-10T21:18:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.452+0000 7f7556e00640 1 -- 172.21.15.16:0/934268145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75500cfe30 msgr2=0x7f75500c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.452+0000 7f7556e00640 1 --2- 172.21.15.16:0/934268145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75500cfe30 0x7f75500c9f40 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f753c009790 tx=0x7f753c004620 comp rx=0 tx=0).stop 2024-09-10T21:18:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.452+0000 7f7556e00640 1 -- 172.21.15.16:0/934268145 shutdown_connections 2024-09-10T21:18:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.452+0000 7f7556e00640 1 --2- 172.21.15.16:0/934268145 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f75300725e0 0x7f7530074aa0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.452+0000 7f7556e00640 1 --2- 172.21.15.16:0/934268145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75500d0210 0x7f75500ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.452+0000 7f7556e00640 1 --2- 172.21.15.16:0/934268145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75500cfe30 0x7f75500c9f40 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.452+0000 7f7556e00640 1 --2- 172.21.15.16:0/934268145 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75500ce040 0x7f755015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.452+0000 7f7556e00640 1 -- 172.21.15.16:0/934268145 >> 172.21.15.16:0/934268145 conn(0x7f75500d1df0 msgr2=0x7f75500721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.452+0000 7f7556e00640 1 -- 172.21.15.16:0/934268145 shutdown_connections 2024-09-10T21:18:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.452+0000 7f7556e00640 1 -- 172.21.15.16:0/934268145 wait complete. 2024-09-10T21:18:19.482 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:18:19.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:18:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 -- 172.21.15.16:0/2623274921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40cd300 msgr2=0x7f1de40d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 --2- 172.21.15.16:0/2623274921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40cd300 0x7f1de40d2800 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009900 tx=0x7f1dd001c7c0 comp rx=0 tx=0).stop 2024-09-10T21:18:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 -- 172.21.15.16:0/2623274921 shutdown_connections 2024-09-10T21:18:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 --2- 172.21.15.16:0/2623274921 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415f940 0x7f1de415fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 --2- 172.21.15.16:0/2623274921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40cd300 0x7f1de40d2800 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 -- 172.21.15.16:0/2623274921 >> 172.21.15.16:0/2623274921 conn(0x7f1de40d1ce0 msgr2=0x7f1de40d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 -- 172.21.15.16:0/2623274921 shutdown_connections 2024-09-10T21:18:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 -- 172.21.15.16:0/2623274921 wait complete. 2024-09-10T21:18:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 Processor -- start 2024-09-10T21:18:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 -- start start 2024-09-10T21:18:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40cd300 0x7f1de40c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415bdc0 0x7f1de4152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1de415f940 0x7f1de4153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1de41690c0 con 0x7f1de40cd300 2024-09-10T21:18:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1de4168f40 con 0x7f1de415f940 2024-09-10T21:18:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1deba00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f1de4169240 con 0x7f1de415bdc0 2024-09-10T21:18:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1de9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40cd300 0x7f1de40c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1de9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40cd300 0x7f1de40c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52800/0 (socket says 172.21.15.16:52800) 2024-09-10T21:18:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1de9000640 1 -- 172.21.15.16:0/713613360 learned_addr learned my addr 172.21.15.16:0/713613360 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1de3e00640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415bdc0 0x7f1de4152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1de9a00640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1de415f940 0x7f1de4153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1de9000640 1 -- 172.21.15.16:0/713613360 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415bdc0 msgr2=0x7f1de4152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1de9000640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415bdc0 0x7f1de4152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1de9000640 1 -- 172.21.15.16:0/713613360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1de415f940 msgr2=0x7f1de4153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1de9000640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1de415f940 0x7f1de4153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1de9000640 1 -- 172.21.15.16:0/713613360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de4152800 con 0x7f1de40cd300 2024-09-10T21:18:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.632+0000 7f1de9a00640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1de415f940 0x7f1de4153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.636+0000 7f1de9000640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40cd300 0x7f1de40c7030 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800c500 tx=0x7f1dd800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.636+0000 7f1de0c00640 1 -- 172.21.15.16:0/713613360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8017070 con 0x7f1de40cd300 2024-09-10T21:18:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.636+0000 7f1deba00640 1 -- 172.21.15.16:0/713613360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de415f0e0 con 0x7f1de40cd300 2024-09-10T21:18:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.636+0000 7f1de0c00640 1 -- 172.21.15.16:0/713613360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dd80053c0 con 0x7f1de40cd300 2024-09-10T21:18:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.636+0000 7f1de0c00640 1 -- 172.21.15.16:0/713613360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8012890 con 0x7f1de40cd300 2024-09-10T21:18:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.636+0000 7f1deba00640 1 -- 172.21.15.16:0/713613360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de40c7900 con 0x7f1de40cd300 2024-09-10T21:18:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.636+0000 7f1deba00640 1 -- 172.21.15.16:0/713613360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de41529d0 con 0x7f1de40cd300 2024-09-10T21:18:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.636+0000 7f1de0c00640 1 -- 172.21.15.16:0/713613360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1dd8003600 con 0x7f1de40cd300 2024-09-10T21:18:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.636+0000 7f1de0c00640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1dc4072610 0x7f1dc4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.636+0000 7f1de0c00640 1 -- 172.21.15.16:0/713613360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f1dd8092820 con 0x7f1de40cd300 2024-09-10T21:18:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.636+0000 7f1de3e00640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1dc4072610 0x7f1dc4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.640+0000 7f1de3e00640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1dc4072610 0x7f1dc4074ad0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00af000 tx=0x7f1dd001b000 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.640+0000 7f1de0c00640 1 -- 172.21.15.16:0/713613360 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1dd805d820 con 0x7f1de40cd300 2024-09-10T21:18:19.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:19.852+0000 7f1deba00640 1 -- 172.21.15.16:0/713613360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1de40704a0 con 0x7f1de40cd300 2024-09-10T21:18:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.456+0000 7f1de0c00640 1 -- 172.21.15.16:0/713613360 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v93) ==== 122+0+0 (secure 0 0 0) 0x7f1dd8062250 con 0x7f1de40cd300 2024-09-10T21:18:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.460+0000 7f1deba00640 1 -- 172.21.15.16:0/713613360 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1dc4072610 msgr2=0x7f1dc4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.460+0000 7f1deba00640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1dc4072610 0x7f1dc4074ad0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00af000 tx=0x7f1dd001b000 comp rx=0 tx=0).stop 2024-09-10T21:18:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.460+0000 7f1deba00640 1 -- 172.21.15.16:0/713613360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40cd300 msgr2=0x7f1de40c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.460+0000 7f1deba00640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40cd300 0x7f1de40c7030 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800c500 tx=0x7f1dd800c9c0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.460+0000 7f1deba00640 1 -- 172.21.15.16:0/713613360 shutdown_connections 2024-09-10T21:18:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.460+0000 7f1deba00640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f1dc4072610 0x7f1dc4074ad0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.460+0000 7f1deba00640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1de415f940 0x7f1de4153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.460+0000 7f1deba00640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1de415bdc0 0x7f1de4152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.460+0000 7f1deba00640 1 --2- 172.21.15.16:0/713613360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1de40cd300 0x7f1de40c7030 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.460+0000 7f1deba00640 1 -- 172.21.15.16:0/713613360 >> 172.21.15.16:0/713613360 conn(0x7f1de40d1ce0 msgr2=0x7f1de40b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.460+0000 7f1deba00640 1 -- 172.21.15.16:0/713613360 shutdown_connections 2024-09-10T21:18:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.460+0000 7f1deba00640 1 -- 172.21.15.16:0/713613360 wait complete. 2024-09-10T21:18:20.487 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:18:20.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.632+0000 7fe3f8200640 1 -- 172.21.15.16:0/4197314177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f015f400 msgr2=0x7fe3f0070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.632+0000 7fe3f8200640 1 --2- 172.21.15.16:0/4197314177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f015f400 0x7fe3f0070fe0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00698f0 tx=0x7fe3e401c780 comp rx=0 tx=0).stop 2024-09-10T21:18:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.632+0000 7fe3f8200640 1 -- 172.21.15.16:0/4197314177 shutdown_connections 2024-09-10T21:18:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.632+0000 7fe3f8200640 1 --2- 172.21.15.16:0/4197314177 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe3f00cfc80 0x7fe3f00d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.632+0000 7fe3f8200640 1 --2- 172.21.15.16:0/4197314177 >> v2:172.21.15.52:3300/0 conn(0x7fe3f00b13c0 0x7fe3f0071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.632+0000 7fe3f8200640 1 --2- 172.21.15.16:0/4197314177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f015f400 0x7fe3f0070fe0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.632+0000 7fe3f8200640 1 -- 172.21.15.16:0/4197314177 >> 172.21.15.16:0/4197314177 conn(0x7fe3f009b130 msgr2=0x7fe3f00d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:20.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.632+0000 7fe3f8200640 1 -- 172.21.15.16:0/4197314177 shutdown_connections 2024-09-10T21:18:20.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.632+0000 7fe3f8200640 1 -- 172.21.15.16:0/4197314177 wait complete. 2024-09-10T21:18:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f8200640 1 Processor -- start 2024-09-10T21:18:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f8200640 1 -- start start 2024-09-10T21:18:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f8200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f00b13c0 0x7fe3f01f8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f8200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe3f00cfc80 0x7fe3f00cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f8200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3f015f400 0x7fe3f0164500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f8200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f00afd90 con 0x7fe3f00b13c0 2024-09-10T21:18:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f8200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f00d1260 con 0x7fe3f015f400 2024-09-10T21:18:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f8200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f00aff10 con 0x7fe3f00cfc80 2024-09-10T21:18:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f00b13c0 0x7fe3f01f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f00b13c0 0x7fe3f01f8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52830/0 (socket says 172.21.15.16:52830) 2024-09-10T21:18:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f5800640 1 -- 172.21.15.16:0/4084196081 learned_addr learned my addr 172.21.15.16:0/4084196081 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f6200640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3f015f400 0x7fe3f0164500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f4e00640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe3f00cfc80 0x7fe3f00cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f5800640 1 -- 172.21.15.16:0/4084196081 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe3f00cfc80 msgr2=0x7fe3f00cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f5800640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe3f00cfc80 0x7fe3f00cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f5800640 1 -- 172.21.15.16:0/4084196081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3f015f400 msgr2=0x7fe3f0164500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f5800640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3f015f400 0x7fe3f0164500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f5800640 1 -- 172.21.15.16:0/4084196081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f0157e90 con 0x7fe3f00b13c0 2024-09-10T21:18:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f5800640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f00b13c0 0x7fe3f01f8890 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe3e401ae30 tx=0x7fe3e40a6660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3ed600640 1 -- 172.21.15.16:0/4084196081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4004300 con 0x7fe3f00b13c0 2024-09-10T21:18:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3ed600640 1 -- 172.21.15.16:0/4084196081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3e40044a0 con 0x7fe3f00b13c0 2024-09-10T21:18:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3ed600640 1 -- 172.21.15.16:0/4084196081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40052d0 con 0x7fe3f00b13c0 2024-09-10T21:18:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f8200640 1 -- 172.21.15.16:0/4084196081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f0158170 con 0x7fe3f00b13c0 2024-09-10T21:18:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.636+0000 7fe3f8200640 1 -- 172.21.15.16:0/4084196081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f0156300 con 0x7fe3f00b13c0 2024-09-10T21:18:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.640+0000 7fe3f8200640 1 -- 172.21.15.16:0/4084196081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f00d0110 con 0x7fe3f00b13c0 2024-09-10T21:18:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.640+0000 7fe3ed600640 1 -- 172.21.15.16:0/4084196081 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe3e4005470 con 0x7fe3f00b13c0 2024-09-10T21:18:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.640+0000 7fe3ed600640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe3d00725e0 0x7fe3d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.640+0000 7fe3ed600640 1 -- 172.21.15.16:0/4084196081 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7fe3e412e0c0 con 0x7fe3f00b13c0 2024-09-10T21:18:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.640+0000 7fe3f4e00640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe3d00725e0 0x7fe3d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.640+0000 7fe3f4e00640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe3d00725e0 0x7fe3d0074aa0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc005e10 tx=0x7fe3dc005c60 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:20.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.644+0000 7fe3ed600640 1 -- 172.21.15.16:0/4084196081 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe3e4016490 con 0x7fe3f00b13c0 2024-09-10T21:18:20.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.852+0000 7fe3f8200640 1 -- 172.21.15.16:0/4084196081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3f00ce610 con 0x7fe3f00b13c0 2024-09-10T21:18:20.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.852+0000 7fe3ed600640 1 -- 172.21.15.16:0/4084196081 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5227 (secure 0 0 0) 0x7fe3e40f9790 con 0x7fe3f00b13c0 2024-09-10T21:18:20.858 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:20.858 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"btime":"2024-09-10T21:18:20:455300+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3971491960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3971491960},{"type":"v1","addr":"172.21.15.52:6834","nonce":3971491960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7854,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4224090659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4224090659},{"type":"v1","addr":"172.21.15.16:6836","nonce":4224090659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1749071285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1749071285},{"type":"v1","addr":"172.21.15.16:6837","nonce":1749071285}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:16.852256+0000","modified":"2024-09-10T21:18:19.856561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/3672156966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3672156966},{"type":"v1","addr":"172.21.15.52:6835","nonce":3672156966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7869,"qdb_cluster":[7869]},"id":10}]} 2024-09-10T21:18:20.858 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 93 2024-09-10T21:18:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.856+0000 7fe3f8200640 1 -- 172.21.15.16:0/4084196081 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe3d00725e0 msgr2=0x7fe3d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.856+0000 7fe3f8200640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe3d00725e0 0x7fe3d0074aa0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc005e10 tx=0x7fe3dc005c60 comp rx=0 tx=0).stop 2024-09-10T21:18:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.856+0000 7fe3f8200640 1 -- 172.21.15.16:0/4084196081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f00b13c0 msgr2=0x7fe3f01f8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.856+0000 7fe3f8200640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f00b13c0 0x7fe3f01f8890 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe3e401ae30 tx=0x7fe3e40a6660 comp rx=0 tx=0).stop 2024-09-10T21:18:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.856+0000 7fe3f8200640 1 -- 172.21.15.16:0/4084196081 shutdown_connections 2024-09-10T21:18:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.856+0000 7fe3f8200640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe3d00725e0 0x7fe3d0074aa0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.856+0000 7fe3f8200640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3f015f400 0x7fe3f0164500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.856+0000 7fe3f8200640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe3f00cfc80 0x7fe3f00cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.856+0000 7fe3f8200640 1 --2- 172.21.15.16:0/4084196081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f00b13c0 0x7fe3f01f8890 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.856+0000 7fe3f8200640 1 -- 172.21.15.16:0/4084196081 >> 172.21.15.16:0/4084196081 conn(0x7fe3f009b130 msgr2=0x7fe3f00cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.856+0000 7fe3f8200640 1 -- 172.21.15.16:0/4084196081 shutdown_connections 2024-09-10T21:18:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:20.856+0000 7fe3f8200640 1 -- 172.21.15.16:0/4084196081 wait complete. 2024-09-10T21:18:20.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:18:21.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.020+0000 7f222ca00640 1 -- 172.21.15.16:0/2973814546 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22280ca170 msgr2=0x7f22280ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.020+0000 7f222ca00640 1 --2- 172.21.15.16:0/2973814546 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22280ca170 0x7f22280ca5b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f222000ad00 tx=0x7f222001dda0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.020+0000 7f222ca00640 1 -- 172.21.15.16:0/2973814546 shutdown_connections 2024-09-10T21:18:21.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.020+0000 7f222ca00640 1 --2- 172.21.15.16:0/2973814546 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22280ca170 0x7f22280ca5b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.020+0000 7f222ca00640 1 -- 172.21.15.16:0/2973814546 >> 172.21.15.16:0/2973814546 conn(0x7f22280cfb20 msgr2=0x7f22280cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:21.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.020+0000 7f222ca00640 1 -- 172.21.15.16:0/2973814546 shutdown_connections 2024-09-10T21:18:21.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.020+0000 7f222ca00640 1 -- 172.21.15.16:0/2973814546 wait complete. 2024-09-10T21:18:21.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f222ca00640 1 Processor -- start 2024-09-10T21:18:21.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f222ca00640 1 -- start start 2024-09-10T21:18:21.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f222ca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22280ca170 0x7f22281515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:21.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f222ca00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22280d06b0 0x7f22280c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:21.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f222ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815a270 0x7f22280737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:21.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f222ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2228072f40 con 0x7f222815a270 2024-09-10T21:18:21.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f222ca00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2228072dc0 con 0x7f22280ca170 2024-09-10T21:18:21.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f222ca00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2228168200 con 0x7f22280d06b0 2024-09-10T21:18:21.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f2226800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815a270 0x7f22280737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:21.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f2226800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815a270 0x7f22280737b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52834/0 (socket says 172.21.15.16:52834) 2024-09-10T21:18:21.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f2226800640 1 -- 172.21.15.16:0/2434739411 learned_addr learned my addr 172.21.15.16:0/2434739411 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:21.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f2225400640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22280d06b0 0x7f22280c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:21.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f2225e00640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22280ca170 0x7f22281515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:21.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f2226800640 1 -- 172.21.15.16:0/2434739411 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22280d06b0 msgr2=0x7f22280c2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f2226800640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22280d06b0 0x7f22280c2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f2226800640 1 -- 172.21.15.16:0/2434739411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22280ca170 msgr2=0x7f22281515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f2226800640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22280ca170 0x7f22281515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f2226800640 1 -- 172.21.15.16:0/2434739411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22281638e0 con 0x7f222815a270 2024-09-10T21:18:21.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f2226800640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815a270 0x7f22280737b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f22200b1040 tx=0x7f2220003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:21.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f221e000640 1 -- 172.21.15.16:0/2434739411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22200b3040 con 0x7f222815a270 2024-09-10T21:18:21.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f222ca00640 1 -- 172.21.15.16:0/2434739411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2228163bc0 con 0x7f222815a270 2024-09-10T21:18:21.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f221e000640 1 -- 172.21.15.16:0/2434739411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2220004e20 con 0x7f222815a270 2024-09-10T21:18:21.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f221e000640 1 -- 172.21.15.16:0/2434739411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22200042a0 con 0x7f222815a270 2024-09-10T21:18:21.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f222ca00640 1 -- 172.21.15.16:0/2434739411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22281532f0 con 0x7f222815a270 2024-09-10T21:18:21.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.024+0000 7f222ca00640 1 -- 172.21.15.16:0/2434739411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22280ca5b0 con 0x7f222815a270 2024-09-10T21:18:21.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.028+0000 7f221e000640 1 -- 172.21.15.16:0/2434739411 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f222000b040 con 0x7f222815a270 2024-09-10T21:18:21.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.028+0000 7f221e000640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f22080725e0 0x7f2208074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:21.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.028+0000 7f221e000640 1 -- 172.21.15.16:0/2434739411 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f222012d880 con 0x7f222815a270 2024-09-10T21:18:21.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.028+0000 7f2225e00640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f22080725e0 0x7f2208074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:21.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.028+0000 7f2225e00640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f22080725e0 0x7f2208074aa0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2218001230 tx=0x7f2218009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:21.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.032+0000 7f221e000640 1 -- 172.21.15.16:0/2434739411 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2220017a20 con 0x7f222815a270 2024-09-10T21:18:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.204+0000 7f222ca00640 1 -- 172.21.15.16:0/2434739411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22280b5aa0 con 0x7f222815a270 2024-09-10T21:18:21.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.208+0000 7f221e000640 1 -- 172.21.15.16:0/2434739411 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16691 (secure 0 0 0) 0x7f22200f8ed0 con 0x7f222815a270 2024-09-10T21:18:21.212 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:21.213 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":127,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:18:18.441972+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:18:14.559149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T21:18:15.813518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:18:21.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.208+0000 7f222ca00640 1 -- 172.21.15.16:0/2434739411 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f22080725e0 msgr2=0x7f2208074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.208+0000 7f222ca00640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f22080725e0 0x7f2208074aa0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2218001230 tx=0x7f2218009040 comp rx=0 tx=0).stop 2024-09-10T21:18:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.208+0000 7f222ca00640 1 -- 172.21.15.16:0/2434739411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815a270 msgr2=0x7f22280737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.208+0000 7f222ca00640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815a270 0x7f22280737b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f22200b1040 tx=0x7f2220003f40 comp rx=0 tx=0).stop 2024-09-10T21:18:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.208+0000 7f222ca00640 1 -- 172.21.15.16:0/2434739411 shutdown_connections 2024-09-10T21:18:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.208+0000 7f222ca00640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f22080725e0 0x7f2208074aa0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.208+0000 7f222ca00640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815a270 0x7f22280737b0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.208+0000 7f222ca00640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22280d06b0 0x7f22280c2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.208+0000 7f222ca00640 1 --2- 172.21.15.16:0/2434739411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22280ca170 0x7f22281515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.208+0000 7f222ca00640 1 -- 172.21.15.16:0/2434739411 >> 172.21.15.16:0/2434739411 conn(0x7f22280cfb20 msgr2=0x7f22280c6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.208+0000 7f222ca00640 1 -- 172.21.15.16:0/2434739411 shutdown_connections 2024-09-10T21:18:21.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.208+0000 7f222ca00640 1 -- 172.21.15.16:0/2434739411 wait complete. 2024-09-10T21:18:21.234 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:18:21.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:18:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.376+0000 7f997e000640 1 -- 172.21.15.16:0/1075939502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99780b1b40 msgr2=0x7f99780b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.376+0000 7f997e000640 1 --2- 172.21.15.16:0/1075939502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99780b1b40 0x7f99780b1fb0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f9978074370 tx=0x7f997001caf0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.376+0000 7f997e000640 1 -- 172.21.15.16:0/1075939502 shutdown_connections 2024-09-10T21:18:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.376+0000 7f997e000640 1 --2- 172.21.15.16:0/1075939502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99780b1b40 0x7f99780b1fb0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.376+0000 7f997e000640 1 --2- 172.21.15.16:0/1075939502 >> v2:172.21.15.52:3300/0 conn(0x7f99780d2820 0x7f99780b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.376+0000 7f997e000640 1 -- 172.21.15.16:0/1075939502 >> 172.21.15.16:0/1075939502 conn(0x7f99780d1d00 msgr2=0x7f99780d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.376+0000 7f997e000640 1 -- 172.21.15.16:0/1075939502 shutdown_connections 2024-09-10T21:18:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.376+0000 7f997e000640 1 -- 172.21.15.16:0/1075939502 wait complete. 2024-09-10T21:18:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.376+0000 7f997e000640 1 Processor -- start 2024-09-10T21:18:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.376+0000 7f997e000640 1 -- start start 2024-09-10T21:18:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99780b1b40 0x7f9978157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997e000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f99780cfd40 0x7f99780cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997e000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99780d2820 0x7f99781fa570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997e000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f99780b05d0 con 0x7f99780b1b40 2024-09-10T21:18:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997e000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f99780b0450 con 0x7f99780d2820 2024-09-10T21:18:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997e000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f99780b0750 con 0x7f99780cfd40 2024-09-10T21:18:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99780b1b40 0x7f9978157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99780b1b40 0x7f9978157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52850/0 (socket says 172.21.15.16:52850) 2024-09-10T21:18:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997cc00640 1 -- 172.21.15.16:0/3874613761 learned_addr learned my addr 172.21.15.16:0/3874613761 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f9977e00640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f99780cfd40 0x7f99780cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997d600640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99780d2820 0x7f99781fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997cc00640 1 -- 172.21.15.16:0/3874613761 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f99780cfd40 msgr2=0x7f99780cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997cc00640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f99780cfd40 0x7f99780cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997cc00640 1 -- 172.21.15.16:0/3874613761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99780d2820 msgr2=0x7f99781fa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997cc00640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99780d2820 0x7f99781fa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997cc00640 1 -- 172.21.15.16:0/3874613761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99780b5de0 con 0x7f99780b1b40 2024-09-10T21:18:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f9977e00640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f99780cfd40 0x7f99780cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997d600640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99780d2820 0x7f99781fa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997cc00640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99780b1b40 0x7f9978157ba0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f996c00c450 tx=0x7f996c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f9974c00640 1 -- 172.21.15.16:0/3874613761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c016020 con 0x7f99780b1b40 2024-09-10T21:18:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f9974c00640 1 -- 172.21.15.16:0/3874613761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f996c012920 con 0x7f99780b1b40 2024-09-10T21:18:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f9974c00640 1 -- 172.21.15.16:0/3874613761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c012d70 con 0x7f99780b1b40 2024-09-10T21:18:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997e000640 1 -- 172.21.15.16:0/3874613761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997815f770 con 0x7f99780b1b40 2024-09-10T21:18:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997e000640 1 -- 172.21.15.16:0/3874613761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997815fd30 con 0x7f99780b1b40 2024-09-10T21:18:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.380+0000 7f997e000640 1 -- 172.21.15.16:0/3874613761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99780b1fb0 con 0x7f99780b1b40 2024-09-10T21:18:21.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.384+0000 7f9974c00640 1 -- 172.21.15.16:0/3874613761 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f996c01b400 con 0x7f99780b1b40 2024-09-10T21:18:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.384+0000 7f9974c00640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f99580725e0 0x7f9958074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.384+0000 7f9974c00640 1 -- 172.21.15.16:0/3874613761 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f996c093270 con 0x7f99780b1b40 2024-09-10T21:18:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.384+0000 7f9977e00640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f99580725e0 0x7f9958074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.384+0000 7f9977e00640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f99580725e0 0x7f9958074aa0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9964005fd0 tx=0x7f9964005dc0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.388+0000 7f9974c00640 1 -- 172.21.15.16:0/3874613761 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f996c020030 con 0x7f99780b1b40 2024-09-10T21:18:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.560+0000 7f997e000640 1 -- 172.21.15.16:0/3874613761 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99780cbed0 con 0x7f99580725e0 2024-09-10T21:18:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.564+0000 7f9974c00640 1 -- 172.21.15.16:0/3874613761 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252466 (secure 0 0 0) 0x7f99780cbed0 con 0x7f99580725e0 2024-09-10T21:18:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:18:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.568+0000 7f997e000640 1 -- 172.21.15.16:0/3874613761 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f99580725e0 msgr2=0x7f9958074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.568+0000 7f997e000640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f99580725e0 0x7f9958074aa0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9964005fd0 tx=0x7f9964005dc0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.568+0000 7f997e000640 1 -- 172.21.15.16:0/3874613761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99780b1b40 msgr2=0x7f9978157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.568+0000 7f997e000640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99780b1b40 0x7f9978157ba0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f996c00c450 tx=0x7f996c00c910 comp rx=0 tx=0).stop 2024-09-10T21:18:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.568+0000 7f997e000640 1 -- 172.21.15.16:0/3874613761 shutdown_connections 2024-09-10T21:18:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.568+0000 7f997e000640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f99580725e0 0x7f9958074aa0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.568+0000 7f997e000640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99780d2820 0x7f99781fa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.568+0000 7f997e000640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f99780cfd40 0x7f99780cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.568+0000 7f997e000640 1 --2- 172.21.15.16:0/3874613761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99780b1b40 0x7f9978157ba0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.568+0000 7f997e000640 1 -- 172.21.15.16:0/3874613761 >> 172.21.15.16:0/3874613761 conn(0x7f99780d1d00 msgr2=0x7f99780d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.568+0000 7f997e000640 1 -- 172.21.15.16:0/3874613761 shutdown_connections 2024-09-10T21:18:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.568+0000 7f997e000640 1 -- 172.21.15.16:0/3874613761 wait complete. 2024-09-10T21:18:21.591 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-10T21:18:19.563899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":288024,"kb_used_data":13512,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754686696,"statfs":{"total":773094113280,"available":772799176704,"internally_reserved":0,"allocated":13836288,"data_stored":6600001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.140659"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.462598+0000","last_change":"2024-09-10T21:18:17.455433+0000","last_active":"2024-09-10T21:18:17.462598+0000","last_peered":"2024-09-10T21:18:17.462598+0000","last_clean":"2024-09-10T21:18:17.462598+0000","last_became_active":"2024-09-10T21:18:17.455130+0000","last_became_peered":"2024-09-10T21:18:17.455130+0000","last_unstale":"2024-09-10T21:18:17.462598+0000","last_undegraded":"2024-09-10T21:18:17.462598+0000","last_fullsized":"2024-09-10T21:18:17.462598+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:22.073443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.470887+0000","last_change":"2024-09-10T21:18:17.464548+0000","last_active":"2024-09-10T21:18:17.470887+0000","last_peered":"2024-09-10T21:18:17.470887+0000","last_clean":"2024-09-10T21:18:17.470887+0000","last_became_active":"2024-09-10T21:18:17.464243+0000","last_became_peered":"2024-09-10T21:18:17.464243+0000","last_unstale":"2024-09-10T21:18:17.470887+0000","last_undegraded":"2024-09-10T21:18:17.470887+0000","last_fullsized":"2024-09-10T21:18:17.470887+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:51:33.650219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.466420+0000","last_change":"2024-09-10T21:18:17.455534+0000","last_active":"2024-09-10T21:18:17.466420+0000","last_peered":"2024-09-10T21:18:17.466420+0000","last_clean":"2024-09-10T21:18:17.466420+0000","last_became_active":"2024-09-10T21:18:17.455239+0000","last_became_peered":"2024-09-10T21:18:17.455239+0000","last_unstale":"2024-09-10T21:18:17.466420+0000","last_undegraded":"2024-09-10T21:18:17.466420+0000","last_fullsized":"2024-09-10T21:18:17.466420+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:37.078920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450076+0000","last_change":"2024-09-10T21:18:17.471715+0000","last_active":"2024-09-10T21:18:18.450076+0000","last_peered":"2024-09-10T21:18:18.450076+0000","last_clean":"2024-09-10T21:18:18.450076+0000","last_became_active":"2024-09-10T21:18:17.470547+0000","last_became_peered":"2024-09-10T21:18:17.470547+0000","last_unstale":"2024-09-10T21:18:18.450076+0000","last_undegraded":"2024-09-10T21:18:18.450076+0000","last_fullsized":"2024-09-10T21:18:18.450076+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:32.967483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.475664+0000","last_change":"2024-09-10T21:18:17.461762+0000","last_active":"2024-09-10T21:18:17.475664+0000","last_peered":"2024-09-10T21:18:17.475664+0000","last_clean":"2024-09-10T21:18:17.475664+0000","last_became_active":"2024-09-10T21:18:17.457569+0000","last_became_peered":"2024-09-10T21:18:17.457569+0000","last_unstale":"2024-09-10T21:18:17.475664+0000","last_undegraded":"2024-09-10T21:18:17.475664+0000","last_fullsized":"2024-09-10T21:18:17.475664+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:35.617615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449720+0000","last_change":"2024-09-10T21:18:17.463255+0000","last_active":"2024-09-10T21:18:18.449720+0000","last_peered":"2024-09-10T21:18:18.449720+0000","last_clean":"2024-09-10T21:18:18.449720+0000","last_became_active":"2024-09-10T21:18:17.462928+0000","last_became_peered":"2024-09-10T21:18:17.462928+0000","last_unstale":"2024-09-10T21:18:18.449720+0000","last_undegraded":"2024-09-10T21:18:18.449720+0000","last_fullsized":"2024-09-10T21:18:18.449720+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:16.738072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.451894+0000","last_change":"2024-09-10T21:18:17.444816+0000","last_active":"2024-09-10T21:18:17.451894+0000","last_peered":"2024-09-10T21:18:17.451894+0000","last_clean":"2024-09-10T21:18:17.451894+0000","last_became_active":"2024-09-10T21:18:17.444529+0000","last_became_peered":"2024-09-10T21:18:17.444529+0000","last_unstale":"2024-09-10T21:18:17.451894+0000","last_undegraded":"2024-09-10T21:18:17.451894+0000","last_fullsized":"2024-09-10T21:18:17.451894+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:19.731748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.460592+0000","last_change":"2024-09-10T21:18:17.453908+0000","last_active":"2024-09-10T21:18:17.460592+0000","last_peered":"2024-09-10T21:18:17.460592+0000","last_clean":"2024-09-10T21:18:17.460592+0000","last_became_active":"2024-09-10T21:18:17.453571+0000","last_became_peered":"2024-09-10T21:18:17.453571+0000","last_unstale":"2024-09-10T21:18:17.460592+0000","last_undegraded":"2024-09-10T21:18:17.460592+0000","last_fullsized":"2024-09-10T21:18:17.460592+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:23.490272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.451433+0000","last_change":"2024-09-10T21:18:17.462106+0000","last_active":"2024-09-10T21:18:18.451433+0000","last_peered":"2024-09-10T21:18:18.451433+0000","last_clean":"2024-09-10T21:18:18.451433+0000","last_became_active":"2024-09-10T21:18:17.461759+0000","last_became_peered":"2024-09-10T21:18:17.461759+0000","last_unstale":"2024-09-10T21:18:18.451433+0000","last_undegraded":"2024-09-10T21:18:18.451433+0000","last_fullsized":"2024-09-10T21:18:18.451433+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:45.628404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.471495+0000","last_change":"2024-09-10T21:18:17.466009+0000","last_active":"2024-09-10T21:18:17.471495+0000","last_peered":"2024-09-10T21:18:17.471495+0000","last_clean":"2024-09-10T21:18:17.471495+0000","last_became_active":"2024-09-10T21:18:17.460658+0000","last_became_peered":"2024-09-10T21:18:17.460658+0000","last_unstale":"2024-09-10T21:18:17.471495+0000","last_undegraded":"2024-09-10T21:18:17.471495+0000","last_fullsized":"2024-09-10T21:18:17.471495+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:48.966839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450953+0000","last_change":"2024-09-10T21:18:17.463023+0000","last_active":"2024-09-10T21:18:18.450953+0000","last_peered":"2024-09-10T21:18:18.450953+0000","last_clean":"2024-09-10T21:18:18.450953+0000","last_became_active":"2024-09-10T21:18:17.462719+0000","last_became_peered":"2024-09-10T21:18:17.462719+0000","last_unstale":"2024-09-10T21:18:18.450953+0000","last_undegraded":"2024-09-10T21:18:18.450953+0000","last_fullsized":"2024-09-10T21:18:18.450953+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:16.392165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.465930+0000","last_change":"2024-09-10T21:18:17.456151+0000","last_active":"2024-09-10T21:18:17.465930+0000","last_peered":"2024-09-10T21:18:17.465930+0000","last_clean":"2024-09-10T21:18:17.465930+0000","last_became_active":"2024-09-10T21:18:17.455835+0000","last_became_peered":"2024-09-10T21:18:17.455835+0000","last_unstale":"2024-09-10T21:18:17.465930+0000","last_undegraded":"2024-09-10T21:18:17.465930+0000","last_fullsized":"2024-09-10T21:18:17.465930+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:31.566129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.474916+0000","last_change":"2024-09-10T21:18:17.454689+0000","last_active":"2024-09-10T21:18:17.474916+0000","last_peered":"2024-09-10T21:18:17.474916+0000","last_clean":"2024-09-10T21:18:17.474916+0000","last_became_active":"2024-09-10T21:18:17.454390+0000","last_became_peered":"2024-09-10T21:18:17.454390+0000","last_unstale":"2024-09-10T21:18:17.474916+0000","last_undegraded":"2024-09-10T21:18:17.474916+0000","last_fullsized":"2024-09-10T21:18:17.474916+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:20.233731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.467468+0000","last_change":"2024-09-10T21:18:17.460348+0000","last_active":"2024-09-10T21:18:17.467468+0000","last_peered":"2024-09-10T21:18:17.467468+0000","last_clean":"2024-09-10T21:18:17.467468+0000","last_became_active":"2024-09-10T21:18:17.460103+0000","last_became_peered":"2024-09-10T21:18:17.460103+0000","last_unstale":"2024-09-10T21:18:17.467468+0000","last_undegraded":"2024-09-10T21:18:17.467468+0000","last_fullsized":"2024-09-10T21:18:17.467468+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:22.030981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.475801+0000","last_change":"2024-09-10T21:18:17.459466+0000","last_active":"2024-09-10T21:18:17.475801+0000","last_peered":"2024-09-10T21:18:17.475801+0000","last_clean":"2024-09-10T21:18:17.475801+0000","last_became_active":"2024-09-10T21:18:17.459171+0000","last_became_peered":"2024-09-10T21:18:17.459171+0000","last_unstale":"2024-09-10T21:18:17.475801+0000","last_undegraded":"2024-09-10T21:18:17.475801+0000","last_fullsized":"2024-09-10T21:18:17.475801+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:05.053362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.464669+0000","last_change":"2024-09-10T21:18:17.459861+0000","last_active":"2024-09-10T21:18:17.464669+0000","last_peered":"2024-09-10T21:18:17.464669+0000","last_clean":"2024-09-10T21:18:17.464669+0000","last_became_active":"2024-09-10T21:18:17.459596+0000","last_became_peered":"2024-09-10T21:18:17.459596+0000","last_unstale":"2024-09-10T21:18:17.464669+0000","last_undegraded":"2024-09-10T21:18:17.464669+0000","last_fullsized":"2024-09-10T21:18:17.464669+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:23.490272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.466902+0000","last_change":"2024-09-10T21:18:17.460115+0000","last_active":"2024-09-10T21:18:17.466902+0000","last_peered":"2024-09-10T21:18:17.466902+0000","last_clean":"2024-09-10T21:18:17.466902+0000","last_became_active":"2024-09-10T21:18:17.458546+0000","last_became_peered":"2024-09-10T21:18:17.458546+0000","last_unstale":"2024-09-10T21:18:17.466902+0000","last_undegraded":"2024-09-10T21:18:17.466902+0000","last_fullsized":"2024-09-10T21:18:17.466902+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:21.518628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.475477+0000","last_change":"2024-09-10T21:18:17.458335+0000","last_active":"2024-09-10T21:18:17.475477+0000","last_peered":"2024-09-10T21:18:17.475477+0000","last_clean":"2024-09-10T21:18:17.475477+0000","last_became_active":"2024-09-10T21:18:17.458022+0000","last_became_peered":"2024-09-10T21:18:17.458022+0000","last_unstale":"2024-09-10T21:18:17.475477+0000","last_undegraded":"2024-09-10T21:18:17.475477+0000","last_fullsized":"2024-09-10T21:18:17.475477+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:06.117473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.460570+0000","last_change":"2024-09-10T21:18:17.454330+0000","last_active":"2024-09-10T21:18:17.460570+0000","last_peered":"2024-09-10T21:18:17.460570+0000","last_clean":"2024-09-10T21:18:17.460570+0000","last_became_active":"2024-09-10T21:18:17.452857+0000","last_became_peered":"2024-09-10T21:18:17.452857+0000","last_unstale":"2024-09-10T21:18:17.460570+0000","last_undegraded":"2024-09-10T21:18:17.460570+0000","last_fullsized":"2024-09-10T21:18:17.460570+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:49.357785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.463476+0000","last_change":"2024-09-10T21:18:17.460034+0000","last_active":"2024-09-10T21:18:17.463476+0000","last_peered":"2024-09-10T21:18:17.463476+0000","last_clean":"2024-09-10T21:18:17.463476+0000","last_became_active":"2024-09-10T21:18:17.459465+0000","last_became_peered":"2024-09-10T21:18:17.459465+0000","last_unstale":"2024-09-10T21:18:17.463476+0000","last_undegraded":"2024-09-10T21:18:17.463476+0000","last_fullsized":"2024-09-10T21:18:17.463476+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:36.547391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.467113+0000","last_change":"2024-09-10T21:18:17.464677+0000","last_active":"2024-09-10T21:18:17.467113+0000","last_peered":"2024-09-10T21:18:17.467113+0000","last_clean":"2024-09-10T21:18:17.467113+0000","last_became_active":"2024-09-10T21:18:17.464365+0000","last_became_peered":"2024-09-10T21:18:17.464365+0000","last_unstale":"2024-09-10T21:18:17.467113+0000","last_undegraded":"2024-09-10T21:18:17.467113+0000","last_fullsized":"2024-09-10T21:18:17.467113+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:39.589294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.471934+0000","last_change":"2024-09-10T21:18:17.469210+0000","last_active":"2024-09-10T21:18:17.471934+0000","last_peered":"2024-09-10T21:18:17.471934+0000","last_clean":"2024-09-10T21:18:17.471934+0000","last_became_active":"2024-09-10T21:18:17.467834+0000","last_became_peered":"2024-09-10T21:18:17.467834+0000","last_unstale":"2024-09-10T21:18:17.471934+0000","last_undegraded":"2024-09-10T21:18:17.471934+0000","last_fullsized":"2024-09-10T21:18:17.471934+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:05.602519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449902+0000","last_change":"2024-09-10T21:18:17.459615+0000","last_active":"2024-09-10T21:18:18.449902+0000","last_peered":"2024-09-10T21:18:18.449902+0000","last_clean":"2024-09-10T21:18:18.449902+0000","last_became_active":"2024-09-10T21:18:17.459272+0000","last_became_peered":"2024-09-10T21:18:17.459272+0000","last_unstale":"2024-09-10T21:18:18.449902+0000","last_undegraded":"2024-09-10T21:18:18.449902+0000","last_fullsized":"2024-09-10T21:18:18.449902+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:31.547340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.476259+0000","last_change":"2024-09-10T21:18:17.470840+0000","last_active":"2024-09-10T21:18:17.476259+0000","last_peered":"2024-09-10T21:18:17.476259+0000","last_clean":"2024-09-10T21:18:17.476259+0000","last_became_active":"2024-09-10T21:18:17.470558+0000","last_became_peered":"2024-09-10T21:18:17.470558+0000","last_unstale":"2024-09-10T21:18:17.476259+0000","last_undegraded":"2024-09-10T21:18:17.476259+0000","last_fullsized":"2024-09-10T21:18:17.476259+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:53.192529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.470466+0000","last_change":"2024-09-10T21:18:17.461564+0000","last_active":"2024-09-10T21:18:17.470466+0000","last_peered":"2024-09-10T21:18:17.470466+0000","last_clean":"2024-09-10T21:18:17.470466+0000","last_became_active":"2024-09-10T21:18:17.461263+0000","last_became_peered":"2024-09-10T21:18:17.461263+0000","last_unstale":"2024-09-10T21:18:17.470466+0000","last_undegraded":"2024-09-10T21:18:17.470466+0000","last_fullsized":"2024-09-10T21:18:17.470466+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:51.557073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.462822+0000","last_change":"2024-09-10T21:18:17.459403+0000","last_active":"2024-09-10T21:18:17.462822+0000","last_peered":"2024-09-10T21:18:17.462822+0000","last_clean":"2024-09-10T21:18:17.462822+0000","last_became_active":"2024-09-10T21:18:17.459059+0000","last_became_peered":"2024-09-10T21:18:17.459059+0000","last_unstale":"2024-09-10T21:18:17.462822+0000","last_undegraded":"2024-09-10T21:18:17.462822+0000","last_fullsized":"2024-09-10T21:18:17.462822+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:37.451914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.470981+0000","last_change":"2024-09-10T21:18:17.465604+0000","last_active":"2024-09-10T21:18:17.470981+0000","last_peered":"2024-09-10T21:18:17.470981+0000","last_clean":"2024-09-10T21:18:17.470981+0000","last_became_active":"2024-09-10T21:18:17.465279+0000","last_became_peered":"2024-09-10T21:18:17.465279+0000","last_unstale":"2024-09-10T21:18:17.470981+0000","last_undegraded":"2024-09-10T21:18:17.470981+0000","last_fullsized":"2024-09-10T21:18:17.470981+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:53.537805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.470929+0000","last_change":"2024-09-10T21:18:17.466628+0000","last_active":"2024-09-10T21:18:17.470929+0000","last_peered":"2024-09-10T21:18:17.470929+0000","last_clean":"2024-09-10T21:18:17.470929+0000","last_became_active":"2024-09-10T21:18:17.466267+0000","last_became_peered":"2024-09-10T21:18:17.466267+0000","last_unstale":"2024-09-10T21:18:17.470929+0000","last_undegraded":"2024-09-10T21:18:17.470929+0000","last_fullsized":"2024-09-10T21:18:17.470929+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:33.088284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.462303+0000","last_change":"2024-09-10T21:18:17.458670+0000","last_active":"2024-09-10T21:18:17.462303+0000","last_peered":"2024-09-10T21:18:17.462303+0000","last_clean":"2024-09-10T21:18:17.462303+0000","last_became_active":"2024-09-10T21:18:17.458404+0000","last_became_peered":"2024-09-10T21:18:17.458404+0000","last_unstale":"2024-09-10T21:18:17.462303+0000","last_undegraded":"2024-09-10T21:18:17.462303+0000","last_fullsized":"2024-09-10T21:18:17.462303+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:42.105578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.467495+0000","last_change":"2024-09-10T21:18:17.461628+0000","last_active":"2024-09-10T21:18:17.467495+0000","last_peered":"2024-09-10T21:18:17.467495+0000","last_clean":"2024-09-10T21:18:17.467495+0000","last_became_active":"2024-09-10T21:18:17.461406+0000","last_became_peered":"2024-09-10T21:18:17.461406+0000","last_unstale":"2024-09-10T21:18:17.467495+0000","last_undegraded":"2024-09-10T21:18:17.467495+0000","last_fullsized":"2024-09-10T21:18:17.467495+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:33.906846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.467756+0000","last_change":"2024-09-10T21:18:17.456278+0000","last_active":"2024-09-10T21:18:17.467756+0000","last_peered":"2024-09-10T21:18:17.467756+0000","last_clean":"2024-09-10T21:18:17.467756+0000","last_became_active":"2024-09-10T21:18:17.455986+0000","last_became_peered":"2024-09-10T21:18:17.455986+0000","last_unstale":"2024-09-10T21:18:17.467756+0000","last_undegraded":"2024-09-10T21:18:17.467756+0000","last_fullsized":"2024-09-10T21:18:17.467756+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:53.429871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.472062+0000","last_change":"2024-09-10T21:18:17.470807+0000","last_active":"2024-09-10T21:18:17.472062+0000","last_peered":"2024-09-10T21:18:17.472062+0000","last_clean":"2024-09-10T21:18:17.472062+0000","last_became_active":"2024-09-10T21:18:17.469846+0000","last_became_peered":"2024-09-10T21:18:17.469846+0000","last_unstale":"2024-09-10T21:18:17.472062+0000","last_undegraded":"2024-09-10T21:18:17.472062+0000","last_fullsized":"2024-09-10T21:18:17.472062+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:20.052187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.471694+0000","last_change":"2024-09-10T21:18:17.454333+0000","last_active":"2024-09-10T21:18:17.471694+0000","last_peered":"2024-09-10T21:18:17.471694+0000","last_clean":"2024-09-10T21:18:17.471694+0000","last_became_active":"2024-09-10T21:18:17.454038+0000","last_became_peered":"2024-09-10T21:18:17.454038+0000","last_unstale":"2024-09-10T21:18:17.471694+0000","last_undegraded":"2024-09-10T21:18:17.471694+0000","last_fullsized":"2024-09-10T21:18:17.471694+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:33.175493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"126'4","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.467051+0000","last_change":"2024-09-10T21:18:16.453681+0000","last_active":"2024-09-10T21:18:17.467051+0000","last_peered":"2024-09-10T21:18:17.467051+0000","last_clean":"2024-09-10T21:18:17.467051+0000","last_became_active":"2024-09-10T21:18:16.453327+0000","last_became_peered":"2024-09-10T21:18:16.453327+0000","last_unstale":"2024-09-10T21:18:17.467051+0000","last_undegraded":"2024-09-10T21:18:17.467051+0000","last_fullsized":"2024-09-10T21:18:17.467051+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:23.960174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.467591+0000","last_change":"2024-09-10T21:18:17.462020+0000","last_active":"2024-09-10T21:18:17.467591+0000","last_peered":"2024-09-10T21:18:17.467591+0000","last_clean":"2024-09-10T21:18:17.467591+0000","last_became_active":"2024-09-10T21:18:17.461715+0000","last_became_peered":"2024-09-10T21:18:17.461715+0000","last_unstale":"2024-09-10T21:18:17.467591+0000","last_undegraded":"2024-09-10T21:18:17.467591+0000","last_fullsized":"2024-09-10T21:18:17.467591+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:49.357785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.451320+0000","last_change":"2024-09-10T21:18:16.460877+0000","last_active":"2024-09-10T21:18:18.451320+0000","last_peered":"2024-09-10T21:18:18.451320+0000","last_clean":"2024-09-10T21:18:18.451320+0000","last_became_active":"2024-09-10T21:18:16.460616+0000","last_became_peered":"2024-09-10T21:18:16.460616+0000","last_unstale":"2024-09-10T21:18:18.451320+0000","last_undegraded":"2024-09-10T21:18:18.451320+0000","last_fullsized":"2024-09-10T21:18:18.451320+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:53.332618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450582+0000","last_change":"2024-09-10T21:18:17.467486+0000","last_active":"2024-09-10T21:18:18.450582+0000","last_peered":"2024-09-10T21:18:18.450582+0000","last_clean":"2024-09-10T21:18:18.450582+0000","last_became_active":"2024-09-10T21:18:17.467192+0000","last_became_peered":"2024-09-10T21:18:17.467192+0000","last_unstale":"2024-09-10T21:18:18.450582+0000","last_undegraded":"2024-09-10T21:18:18.450582+0000","last_fullsized":"2024-09-10T21:18:18.450582+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:22.161838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"126'2","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.474584+0000","last_change":"2024-09-10T21:18:16.450007+0000","last_active":"2024-09-10T21:18:17.474584+0000","last_peered":"2024-09-10T21:18:17.474584+0000","last_clean":"2024-09-10T21:18:17.474584+0000","last_became_active":"2024-09-10T21:18:16.448556+0000","last_became_peered":"2024-09-10T21:18:16.448556+0000","last_unstale":"2024-09-10T21:18:17.474584+0000","last_undegraded":"2024-09-10T21:18:17.474584+0000","last_fullsized":"2024-09-10T21:18:17.474584+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:58.542805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450078+0000","last_change":"2024-09-10T21:18:17.468571+0000","last_active":"2024-09-10T21:18:18.450078+0000","last_peered":"2024-09-10T21:18:18.450078+0000","last_clean":"2024-09-10T21:18:18.450078+0000","last_became_active":"2024-09-10T21:18:17.468322+0000","last_became_peered":"2024-09-10T21:18:17.468322+0000","last_unstale":"2024-09-10T21:18:18.450078+0000","last_undegraded":"2024-09-10T21:18:18.450078+0000","last_fullsized":"2024-09-10T21:18:18.450078+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:16.803317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"126'2","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.466770+0000","last_change":"2024-09-10T21:18:16.454414+0000","last_active":"2024-09-10T21:18:17.466770+0000","last_peered":"2024-09-10T21:18:17.466770+0000","last_clean":"2024-09-10T21:18:17.466770+0000","last_became_active":"2024-09-10T21:18:16.454132+0000","last_became_peered":"2024-09-10T21:18:16.454132+0000","last_unstale":"2024-09-10T21:18:17.466770+0000","last_undegraded":"2024-09-10T21:18:17.466770+0000","last_fullsized":"2024-09-10T21:18:17.466770+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:26.959815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.468082+0000","last_change":"2024-09-10T21:18:17.459136+0000","last_active":"2024-09-10T21:18:17.468082+0000","last_peered":"2024-09-10T21:18:17.468082+0000","last_clean":"2024-09-10T21:18:17.468082+0000","last_became_active":"2024-09-10T21:18:17.458857+0000","last_became_peered":"2024-09-10T21:18:17.458857+0000","last_unstale":"2024-09-10T21:18:17.468082+0000","last_undegraded":"2024-09-10T21:18:17.468082+0000","last_fullsized":"2024-09-10T21:18:17.468082+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:40.891058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"126'1","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.474535+0000","last_change":"2024-09-10T21:18:16.455034+0000","last_active":"2024-09-10T21:18:17.474535+0000","last_peered":"2024-09-10T21:18:17.474535+0000","last_clean":"2024-09-10T21:18:17.474535+0000","last_became_active":"2024-09-10T21:18:16.454213+0000","last_became_peered":"2024-09-10T21:18:16.454213+0000","last_unstale":"2024-09-10T21:18:17.474535+0000","last_undegraded":"2024-09-10T21:18:17.474535+0000","last_fullsized":"2024-09-10T21:18:17.474535+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:44.998071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450218+0000","last_change":"2024-09-10T21:18:17.473440+0000","last_active":"2024-09-10T21:18:18.450218+0000","last_peered":"2024-09-10T21:18:18.450218+0000","last_clean":"2024-09-10T21:18:18.450218+0000","last_became_active":"2024-09-10T21:18:17.470781+0000","last_became_peered":"2024-09-10T21:18:17.470781+0000","last_unstale":"2024-09-10T21:18:18.450218+0000","last_undegraded":"2024-09-10T21:18:18.450218+0000","last_fullsized":"2024-09-10T21:18:18.450218+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:59.187772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"126'1","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.475956+0000","last_change":"2024-09-10T21:18:16.461209+0000","last_active":"2024-09-10T21:18:17.475956+0000","last_peered":"2024-09-10T21:18:17.475956+0000","last_clean":"2024-09-10T21:18:17.475956+0000","last_became_active":"2024-09-10T21:18:16.459922+0000","last_became_peered":"2024-09-10T21:18:16.459922+0000","last_unstale":"2024-09-10T21:18:17.475956+0000","last_undegraded":"2024-09-10T21:18:17.475956+0000","last_fullsized":"2024-09-10T21:18:17.475956+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:36.787152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.472110+0000","last_change":"2024-09-10T21:18:17.470809+0000","last_active":"2024-09-10T21:18:17.472110+0000","last_peered":"2024-09-10T21:18:17.472110+0000","last_clean":"2024-09-10T21:18:17.472110+0000","last_became_active":"2024-09-10T21:18:17.470215+0000","last_became_peered":"2024-09-10T21:18:17.470215+0000","last_unstale":"2024-09-10T21:18:17.472110+0000","last_undegraded":"2024-09-10T21:18:17.472110+0000","last_fullsized":"2024-09-10T21:18:17.472110+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:00.803035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449419+0000","last_change":"2024-09-10T21:18:16.452184+0000","last_active":"2024-09-10T21:18:18.449419+0000","last_peered":"2024-09-10T21:18:18.449419+0000","last_clean":"2024-09-10T21:18:18.449419+0000","last_became_active":"2024-09-10T21:18:16.451895+0000","last_became_peered":"2024-09-10T21:18:16.451895+0000","last_unstale":"2024-09-10T21:18:18.449419+0000","last_undegraded":"2024-09-10T21:18:18.449419+0000","last_fullsized":"2024-09-10T21:18:18.449419+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:35.807764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449670+0000","last_change":"2024-09-10T21:18:17.446741+0000","last_active":"2024-09-10T21:18:18.449670+0000","last_peered":"2024-09-10T21:18:18.449670+0000","last_clean":"2024-09-10T21:18:18.449670+0000","last_became_active":"2024-09-10T21:18:17.446462+0000","last_became_peered":"2024-09-10T21:18:17.446462+0000","last_unstale":"2024-09-10T21:18:18.449670+0000","last_undegraded":"2024-09-10T21:18:18.449670+0000","last_fullsized":"2024-09-10T21:18:18.449670+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:50.237205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.446575+0000","last_change":"2024-09-10T21:18:16.456352+0000","last_active":"2024-09-10T21:18:17.446575+0000","last_peered":"2024-09-10T21:18:17.446575+0000","last_clean":"2024-09-10T21:18:17.446575+0000","last_became_active":"2024-09-10T21:18:16.454832+0000","last_became_peered":"2024-09-10T21:18:16.454832+0000","last_unstale":"2024-09-10T21:18:17.446575+0000","last_undegraded":"2024-09-10T21:18:17.446575+0000","last_fullsized":"2024-09-10T21:18:17.446575+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:51.063254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450289+0000","last_change":"2024-09-10T21:18:17.456882+0000","last_active":"2024-09-10T21:18:18.450289+0000","last_peered":"2024-09-10T21:18:18.450289+0000","last_clean":"2024-09-10T21:18:18.450289+0000","last_became_active":"2024-09-10T21:18:17.456343+0000","last_became_peered":"2024-09-10T21:18:17.456343+0000","last_unstale":"2024-09-10T21:18:18.450289+0000","last_undegraded":"2024-09-10T21:18:18.450289+0000","last_fullsized":"2024-09-10T21:18:18.450289+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:56.596133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.451361+0000","last_change":"2024-09-10T21:18:16.456043+0000","last_active":"2024-09-10T21:18:18.451361+0000","last_peered":"2024-09-10T21:18:18.451361+0000","last_clean":"2024-09-10T21:18:18.451361+0000","last_became_active":"2024-09-10T21:18:16.455629+0000","last_became_peered":"2024-09-10T21:18:16.455629+0000","last_unstale":"2024-09-10T21:18:18.451361+0000","last_undegraded":"2024-09-10T21:18:18.451361+0000","last_fullsized":"2024-09-10T21:18:18.451361+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:47.641586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.468943+0000","last_change":"2024-09-10T21:18:17.460429+0000","last_active":"2024-09-10T21:18:17.468943+0000","last_peered":"2024-09-10T21:18:17.468943+0000","last_clean":"2024-09-10T21:18:17.468943+0000","last_became_active":"2024-09-10T21:18:17.460126+0000","last_became_peered":"2024-09-10T21:18:17.460126+0000","last_unstale":"2024-09-10T21:18:17.468943+0000","last_undegraded":"2024-09-10T21:18:17.468943+0000","last_fullsized":"2024-09-10T21:18:17.468943+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:31.992739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.445589+0000","last_change":"2024-09-10T21:18:16.456383+0000","last_active":"2024-09-10T21:18:17.445589+0000","last_peered":"2024-09-10T21:18:17.445589+0000","last_clean":"2024-09-10T21:18:17.445589+0000","last_became_active":"2024-09-10T21:18:16.456028+0000","last_became_peered":"2024-09-10T21:18:16.456028+0000","last_unstale":"2024-09-10T21:18:17.445589+0000","last_undegraded":"2024-09-10T21:18:17.445589+0000","last_fullsized":"2024-09-10T21:18:17.445589+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:05.929715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.464665+0000","last_change":"2024-09-10T21:18:17.458171+0000","last_active":"2024-09-10T21:18:17.464665+0000","last_peered":"2024-09-10T21:18:17.464665+0000","last_clean":"2024-09-10T21:18:17.464665+0000","last_became_active":"2024-09-10T21:18:17.457891+0000","last_became_peered":"2024-09-10T21:18:17.457891+0000","last_unstale":"2024-09-10T21:18:17.464665+0000","last_undegraded":"2024-09-10T21:18:17.464665+0000","last_fullsized":"2024-09-10T21:18:17.464665+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:19.493906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"126'6","reported_seq":19,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.476539+0000","last_change":"2024-09-10T21:18:16.445813+0000","last_active":"2024-09-10T21:18:17.476539+0000","last_peered":"2024-09-10T21:18:17.476539+0000","last_clean":"2024-09-10T21:18:17.476539+0000","last_became_active":"2024-09-10T21:18:16.445298+0000","last_became_peered":"2024-09-10T21:18:16.445298+0000","last_unstale":"2024-09-10T21:18:17.476539+0000","last_undegraded":"2024-09-10T21:18:17.476539+0000","last_fullsized":"2024-09-10T21:18:17.476539+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:49.180925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449704+0000","last_change":"2024-09-10T21:18:17.461088+0000","last_active":"2024-09-10T21:18:18.449704+0000","last_peered":"2024-09-10T21:18:18.449704+0000","last_clean":"2024-09-10T21:18:18.449704+0000","last_became_active":"2024-09-10T21:18:17.460878+0000","last_became_peered":"2024-09-10T21:18:17.460878+0000","last_unstale":"2024-09-10T21:18:18.449704+0000","last_undegraded":"2024-09-10T21:18:18.449704+0000","last_fullsized":"2024-09-10T21:18:18.449704+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:05.686097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"126'2","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.475302+0000","last_change":"2024-09-10T21:18:16.451482+0000","last_active":"2024-09-10T21:18:17.475302+0000","last_peered":"2024-09-10T21:18:17.475302+0000","last_clean":"2024-09-10T21:18:17.475302+0000","last_became_active":"2024-09-10T21:18:16.449493+0000","last_became_peered":"2024-09-10T21:18:16.449493+0000","last_unstale":"2024-09-10T21:18:17.475302+0000","last_undegraded":"2024-09-10T21:18:17.475302+0000","last_fullsized":"2024-09-10T21:18:17.475302+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:23.477903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.462218+0000","last_change":"2024-09-10T21:18:17.458633+0000","last_active":"2024-09-10T21:18:17.462218+0000","last_peered":"2024-09-10T21:18:17.462218+0000","last_clean":"2024-09-10T21:18:17.462218+0000","last_became_active":"2024-09-10T21:18:17.457955+0000","last_became_peered":"2024-09-10T21:18:17.457955+0000","last_unstale":"2024-09-10T21:18:17.462218+0000","last_undegraded":"2024-09-10T21:18:17.462218+0000","last_fullsized":"2024-09-10T21:18:17.462218+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:06.016891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.452293+0000","last_change":"2024-09-10T21:18:16.455787+0000","last_active":"2024-09-10T21:18:17.452293+0000","last_peered":"2024-09-10T21:18:17.452293+0000","last_clean":"2024-09-10T21:18:17.452293+0000","last_became_active":"2024-09-10T21:18:16.455434+0000","last_became_peered":"2024-09-10T21:18:16.455434+0000","last_unstale":"2024-09-10T21:18:17.452293+0000","last_undegraded":"2024-09-10T21:18:17.452293+0000","last_fullsized":"2024-09-10T21:18:17.452293+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:17.447633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449604+0000","last_change":"2024-09-10T21:18:17.463487+0000","last_active":"2024-09-10T21:18:18.449604+0000","last_peered":"2024-09-10T21:18:18.449604+0000","last_clean":"2024-09-10T21:18:18.449604+0000","last_became_active":"2024-09-10T21:18:17.455930+0000","last_became_peered":"2024-09-10T21:18:17.455930+0000","last_unstale":"2024-09-10T21:18:18.449604+0000","last_undegraded":"2024-09-10T21:18:18.449604+0000","last_fullsized":"2024-09-10T21:18:18.449604+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:09.546582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.448981+0000","last_change":"2024-09-10T21:18:16.452258+0000","last_active":"2024-09-10T21:18:17.448981+0000","last_peered":"2024-09-10T21:18:17.448981+0000","last_clean":"2024-09-10T21:18:17.448981+0000","last_became_active":"2024-09-10T21:18:16.451978+0000","last_became_peered":"2024-09-10T21:18:16.451978+0000","last_unstale":"2024-09-10T21:18:17.448981+0000","last_undegraded":"2024-09-10T21:18:17.448981+0000","last_fullsized":"2024-09-10T21:18:17.448981+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:34.941666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450475+0000","last_change":"2024-09-10T21:18:17.466944+0000","last_active":"2024-09-10T21:18:18.450475+0000","last_peered":"2024-09-10T21:18:18.450475+0000","last_clean":"2024-09-10T21:18:18.450475+0000","last_became_active":"2024-09-10T21:18:17.458829+0000","last_became_peered":"2024-09-10T21:18:17.458829+0000","last_unstale":"2024-09-10T21:18:18.450475+0000","last_undegraded":"2024-09-10T21:18:18.450475+0000","last_fullsized":"2024-09-10T21:18:18.450475+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:39.734114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"126'8","reported_seq":21,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.458114+0000","last_change":"2024-09-10T21:18:16.450107+0000","last_active":"2024-09-10T21:18:17.458114+0000","last_peered":"2024-09-10T21:18:17.458114+0000","last_clean":"2024-09-10T21:18:17.458114+0000","last_became_active":"2024-09-10T21:18:16.449784+0000","last_became_peered":"2024-09-10T21:18:16.449784+0000","last_unstale":"2024-09-10T21:18:17.458114+0000","last_undegraded":"2024-09-10T21:18:17.458114+0000","last_fullsized":"2024-09-10T21:18:17.458114+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:18.728448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.476062+0000","last_change":"2024-09-10T21:18:17.469640+0000","last_active":"2024-09-10T21:18:17.476062+0000","last_peered":"2024-09-10T21:18:17.476062+0000","last_clean":"2024-09-10T21:18:17.476062+0000","last_became_active":"2024-09-10T21:18:17.469378+0000","last_became_peered":"2024-09-10T21:18:17.469378+0000","last_unstale":"2024-09-10T21:18:17.476062+0000","last_undegraded":"2024-09-10T21:18:17.476062+0000","last_fullsized":"2024-09-10T21:18:17.476062+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:40.261097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450465+0000","last_change":"2024-09-10T21:18:16.443607+0000","last_active":"2024-09-10T21:18:18.450465+0000","last_peered":"2024-09-10T21:18:18.450465+0000","last_clean":"2024-09-10T21:18:18.450465+0000","last_became_active":"2024-09-10T21:18:16.443263+0000","last_became_peered":"2024-09-10T21:18:16.443263+0000","last_unstale":"2024-09-10T21:18:18.450465+0000","last_undegraded":"2024-09-10T21:18:18.450465+0000","last_fullsized":"2024-09-10T21:18:18.450465+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:08.281114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.476400+0000","last_change":"2024-09-10T21:18:17.475328+0000","last_active":"2024-09-10T21:18:17.476400+0000","last_peered":"2024-09-10T21:18:17.476400+0000","last_clean":"2024-09-10T21:18:17.476400+0000","last_became_active":"2024-09-10T21:18:17.475010+0000","last_became_peered":"2024-09-10T21:18:17.475010+0000","last_unstale":"2024-09-10T21:18:17.476400+0000","last_undegraded":"2024-09-10T21:18:17.476400+0000","last_fullsized":"2024-09-10T21:18:17.476400+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:54.055985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"126'2","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.465921+0000","last_change":"2024-09-10T21:18:16.451256+0000","last_active":"2024-09-10T21:18:17.465921+0000","last_peered":"2024-09-10T21:18:17.465921+0000","last_clean":"2024-09-10T21:18:17.465921+0000","last_became_active":"2024-09-10T21:18:16.450969+0000","last_became_peered":"2024-09-10T21:18:16.450969+0000","last_unstale":"2024-09-10T21:18:17.465921+0000","last_undegraded":"2024-09-10T21:18:17.465921+0000","last_fullsized":"2024-09-10T21:18:17.465921+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:18.490606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.476168+0000","last_change":"2024-09-10T21:18:17.469966+0000","last_active":"2024-09-10T21:18:17.476168+0000","last_peered":"2024-09-10T21:18:17.476168+0000","last_clean":"2024-09-10T21:18:17.476168+0000","last_became_active":"2024-09-10T21:18:17.469152+0000","last_became_peered":"2024-09-10T21:18:17.469152+0000","last_unstale":"2024-09-10T21:18:17.476168+0000","last_undegraded":"2024-09-10T21:18:17.476168+0000","last_fullsized":"2024-09-10T21:18:17.476168+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:57.173465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"126'1","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.474200+0000","last_change":"2024-09-10T21:18:16.453893+0000","last_active":"2024-09-10T21:18:17.474200+0000","last_peered":"2024-09-10T21:18:17.474200+0000","last_clean":"2024-09-10T21:18:17.474200+0000","last_became_active":"2024-09-10T21:18:16.453623+0000","last_became_peered":"2024-09-10T21:18:16.453623+0000","last_unstale":"2024-09-10T21:18:17.474200+0000","last_undegraded":"2024-09-10T21:18:17.474200+0000","last_fullsized":"2024-09-10T21:18:17.474200+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:36.504695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.451007+0000","last_change":"2024-09-10T21:18:17.467206+0000","last_active":"2024-09-10T21:18:18.451007+0000","last_peered":"2024-09-10T21:18:18.451007+0000","last_clean":"2024-09-10T21:18:18.451007+0000","last_became_active":"2024-09-10T21:18:17.466909+0000","last_became_peered":"2024-09-10T21:18:17.466909+0000","last_unstale":"2024-09-10T21:18:18.451007+0000","last_undegraded":"2024-09-10T21:18:18.451007+0000","last_fullsized":"2024-09-10T21:18:18.451007+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:47.366859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.452137+0000","last_change":"2024-09-10T21:18:16.452924+0000","last_active":"2024-09-10T21:18:17.452137+0000","last_peered":"2024-09-10T21:18:17.452137+0000","last_clean":"2024-09-10T21:18:17.452137+0000","last_became_active":"2024-09-10T21:18:16.451756+0000","last_became_peered":"2024-09-10T21:18:16.451756+0000","last_unstale":"2024-09-10T21:18:17.452137+0000","last_undegraded":"2024-09-10T21:18:17.452137+0000","last_fullsized":"2024-09-10T21:18:17.452137+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:25.758216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.469764+0000","last_change":"2024-09-10T21:18:17.465068+0000","last_active":"2024-09-10T21:18:17.469764+0000","last_peered":"2024-09-10T21:18:17.469764+0000","last_clean":"2024-09-10T21:18:17.469764+0000","last_became_active":"2024-09-10T21:18:17.464618+0000","last_became_peered":"2024-09-10T21:18:17.464618+0000","last_unstale":"2024-09-10T21:18:17.469764+0000","last_undegraded":"2024-09-10T21:18:17.469764+0000","last_fullsized":"2024-09-10T21:18:17.469764+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:01.943203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450065+0000","last_change":"2024-09-10T21:18:16.460009+0000","last_active":"2024-09-10T21:18:18.450065+0000","last_peered":"2024-09-10T21:18:18.450065+0000","last_clean":"2024-09-10T21:18:18.450065+0000","last_became_active":"2024-09-10T21:18:16.459722+0000","last_became_peered":"2024-09-10T21:18:16.459722+0000","last_unstale":"2024-09-10T21:18:18.450065+0000","last_undegraded":"2024-09-10T21:18:18.450065+0000","last_fullsized":"2024-09-10T21:18:18.450065+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:34.122921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.472080+0000","last_change":"2024-09-10T21:18:17.470830+0000","last_active":"2024-09-10T21:18:17.472080+0000","last_peered":"2024-09-10T21:18:17.472080+0000","last_clean":"2024-09-10T21:18:17.472080+0000","last_became_active":"2024-09-10T21:18:17.469755+0000","last_became_peered":"2024-09-10T21:18:17.469755+0000","last_unstale":"2024-09-10T21:18:17.472080+0000","last_undegraded":"2024-09-10T21:18:17.472080+0000","last_fullsized":"2024-09-10T21:18:17.472080+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:59.187712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449647+0000","last_change":"2024-09-10T21:18:16.442459+0000","last_active":"2024-09-10T21:18:18.449647+0000","last_peered":"2024-09-10T21:18:18.449647+0000","last_clean":"2024-09-10T21:18:18.449647+0000","last_became_active":"2024-09-10T21:18:16.442119+0000","last_became_peered":"2024-09-10T21:18:16.442119+0000","last_unstale":"2024-09-10T21:18:18.449647+0000","last_undegraded":"2024-09-10T21:18:18.449647+0000","last_fullsized":"2024-09-10T21:18:18.449647+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:01.779397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.471610+0000","last_change":"2024-09-10T21:18:17.466122+0000","last_active":"2024-09-10T21:18:17.471610+0000","last_peered":"2024-09-10T21:18:17.471610+0000","last_clean":"2024-09-10T21:18:17.471610+0000","last_became_active":"2024-09-10T21:18:17.461777+0000","last_became_peered":"2024-09-10T21:18:17.461777+0000","last_unstale":"2024-09-10T21:18:17.471610+0000","last_undegraded":"2024-09-10T21:18:17.471610+0000","last_fullsized":"2024-09-10T21:18:17.471610+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:49.964624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450228+0000","last_change":"2024-09-10T21:18:16.443440+0000","last_active":"2024-09-10T21:18:18.450228+0000","last_peered":"2024-09-10T21:18:18.450228+0000","last_clean":"2024-09-10T21:18:18.450228+0000","last_became_active":"2024-09-10T21:18:16.443120+0000","last_became_peered":"2024-09-10T21:18:16.443120+0000","last_unstale":"2024-09-10T21:18:18.450228+0000","last_undegraded":"2024-09-10T21:18:18.450228+0000","last_fullsized":"2024-09-10T21:18:18.450228+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:18.457315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.471683+0000","last_change":"2024-09-10T21:18:17.470314+0000","last_active":"2024-09-10T21:18:17.471683+0000","last_peered":"2024-09-10T21:18:17.471683+0000","last_clean":"2024-09-10T21:18:17.471683+0000","last_became_active":"2024-09-10T21:18:17.469668+0000","last_became_peered":"2024-09-10T21:18:17.469668+0000","last_unstale":"2024-09-10T21:18:17.471683+0000","last_undegraded":"2024-09-10T21:18:17.471683+0000","last_fullsized":"2024-09-10T21:18:17.471683+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:40.103869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449875+0000","last_change":"2024-09-10T21:18:16.452036+0000","last_active":"2024-09-10T21:18:18.449875+0000","last_peered":"2024-09-10T21:18:18.449875+0000","last_clean":"2024-09-10T21:18:18.449875+0000","last_became_active":"2024-09-10T21:18:16.447891+0000","last_became_peered":"2024-09-10T21:18:16.447891+0000","last_unstale":"2024-09-10T21:18:18.449875+0000","last_undegraded":"2024-09-10T21:18:18.449875+0000","last_fullsized":"2024-09-10T21:18:18.449875+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:24.797913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450490+0000","last_change":"2024-09-10T21:18:17.456606+0000","last_active":"2024-09-10T21:18:18.450490+0000","last_peered":"2024-09-10T21:18:18.450490+0000","last_clean":"2024-09-10T21:18:18.450490+0000","last_became_active":"2024-09-10T21:18:17.455955+0000","last_became_peered":"2024-09-10T21:18:17.455955+0000","last_unstale":"2024-09-10T21:18:18.450490+0000","last_undegraded":"2024-09-10T21:18:18.450490+0000","last_fullsized":"2024-09-10T21:18:18.450490+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:18.632289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450619+0000","last_change":"2024-09-10T21:18:16.460481+0000","last_active":"2024-09-10T21:18:18.450619+0000","last_peered":"2024-09-10T21:18:18.450619+0000","last_clean":"2024-09-10T21:18:18.450619+0000","last_became_active":"2024-09-10T21:18:16.460171+0000","last_became_peered":"2024-09-10T21:18:16.460171+0000","last_unstale":"2024-09-10T21:18:18.450619+0000","last_undegraded":"2024-09-10T21:18:18.450619+0000","last_fullsized":"2024-09-10T21:18:18.450619+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:21:30.031820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450360+0000","last_change":"2024-09-10T21:18:17.467980+0000","last_active":"2024-09-10T21:18:18.450360+0000","last_peered":"2024-09-10T21:18:18.450360+0000","last_clean":"2024-09-10T21:18:18.450360+0000","last_became_active":"2024-09-10T21:18:17.467708+0000","last_became_peered":"2024-09-10T21:18:17.467708+0000","last_unstale":"2024-09-10T21:18:18.450360+0000","last_undegraded":"2024-09-10T21:18:18.450360+0000","last_fullsized":"2024-09-10T21:18:18.450360+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:33.671251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.451060+0000","last_change":"2024-09-10T21:18:16.454248+0000","last_active":"2024-09-10T21:18:17.451060+0000","last_peered":"2024-09-10T21:18:17.451060+0000","last_clean":"2024-09-10T21:18:17.451060+0000","last_became_active":"2024-09-10T21:18:16.453174+0000","last_became_peered":"2024-09-10T21:18:16.453174+0000","last_unstale":"2024-09-10T21:18:17.451060+0000","last_undegraded":"2024-09-10T21:18:17.451060+0000","last_fullsized":"2024-09-10T21:18:17.451060+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:55.505246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.463014+0000","last_change":"2024-09-10T21:18:17.459711+0000","last_active":"2024-09-10T21:18:17.463014+0000","last_peered":"2024-09-10T21:18:17.463014+0000","last_clean":"2024-09-10T21:18:17.463014+0000","last_became_active":"2024-09-10T21:18:17.459065+0000","last_became_peered":"2024-09-10T21:18:17.459065+0000","last_unstale":"2024-09-10T21:18:17.463014+0000","last_undegraded":"2024-09-10T21:18:17.463014+0000","last_fullsized":"2024-09-10T21:18:17.463014+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:44.043625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.440198+0000","last_change":"2024-09-10T21:18:16.446308+0000","last_active":"2024-09-10T21:18:17.440198+0000","last_peered":"2024-09-10T21:18:17.440198+0000","last_clean":"2024-09-10T21:18:17.440198+0000","last_became_active":"2024-09-10T21:18:16.445489+0000","last_became_peered":"2024-09-10T21:18:16.445489+0000","last_unstale":"2024-09-10T21:18:17.440198+0000","last_undegraded":"2024-09-10T21:18:17.440198+0000","last_fullsized":"2024-09-10T21:18:17.440198+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:10.399454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.460595+0000","last_change":"2024-09-10T21:18:17.454539+0000","last_active":"2024-09-10T21:18:17.460595+0000","last_peered":"2024-09-10T21:18:17.460595+0000","last_clean":"2024-09-10T21:18:17.460595+0000","last_became_active":"2024-09-10T21:18:17.454226+0000","last_became_peered":"2024-09-10T21:18:17.454226+0000","last_unstale":"2024-09-10T21:18:17.460595+0000","last_undegraded":"2024-09-10T21:18:17.460595+0000","last_fullsized":"2024-09-10T21:18:17.460595+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:28:26.311973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.439844+0000","last_change":"2024-09-10T21:18:16.447641+0000","last_active":"2024-09-10T21:18:17.439844+0000","last_peered":"2024-09-10T21:18:17.439844+0000","last_clean":"2024-09-10T21:18:17.439844+0000","last_became_active":"2024-09-10T21:18:16.447352+0000","last_became_peered":"2024-09-10T21:18:16.447352+0000","last_unstale":"2024-09-10T21:18:17.439844+0000","last_undegraded":"2024-09-10T21:18:17.439844+0000","last_fullsized":"2024-09-10T21:18:17.439844+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:11.041118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.468104+0000","last_change":"2024-09-10T21:18:17.462166+0000","last_active":"2024-09-10T21:18:17.468104+0000","last_peered":"2024-09-10T21:18:17.468104+0000","last_clean":"2024-09-10T21:18:17.468104+0000","last_became_active":"2024-09-10T21:18:17.460893+0000","last_became_peered":"2024-09-10T21:18:17.460893+0000","last_unstale":"2024-09-10T21:18:17.468104+0000","last_undegraded":"2024-09-10T21:18:17.468104+0000","last_fullsized":"2024-09-10T21:18:17.468104+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:28:26.311973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.448411+0000","last_change":"2024-09-10T21:18:16.455549+0000","last_active":"2024-09-10T21:18:17.448411+0000","last_peered":"2024-09-10T21:18:17.448411+0000","last_clean":"2024-09-10T21:18:17.448411+0000","last_became_active":"2024-09-10T21:18:16.455163+0000","last_became_peered":"2024-09-10T21:18:16.455163+0000","last_unstale":"2024-09-10T21:18:17.448411+0000","last_undegraded":"2024-09-10T21:18:17.448411+0000","last_fullsized":"2024-09-10T21:18:17.448411+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:30.556397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450030+0000","last_change":"2024-09-10T21:18:17.445068+0000","last_active":"2024-09-10T21:18:18.450030+0000","last_peered":"2024-09-10T21:18:18.450030+0000","last_clean":"2024-09-10T21:18:18.450030+0000","last_became_active":"2024-09-10T21:18:17.444775+0000","last_became_peered":"2024-09-10T21:18:17.444775+0000","last_unstale":"2024-09-10T21:18:18.450030+0000","last_undegraded":"2024-09-10T21:18:18.450030+0000","last_fullsized":"2024-09-10T21:18:18.450030+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:01.659784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.448215+0000","last_change":"2024-09-10T21:18:16.459506+0000","last_active":"2024-09-10T21:18:17.448215+0000","last_peered":"2024-09-10T21:18:17.448215+0000","last_clean":"2024-09-10T21:18:17.448215+0000","last_became_active":"2024-09-10T21:18:16.459180+0000","last_became_peered":"2024-09-10T21:18:16.459180+0000","last_unstale":"2024-09-10T21:18:17.448215+0000","last_undegraded":"2024-09-10T21:18:17.448215+0000","last_fullsized":"2024-09-10T21:18:17.448215+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:24.903044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450260+0000","last_change":"2024-09-10T21:18:17.462548+0000","last_active":"2024-09-10T21:18:18.450260+0000","last_peered":"2024-09-10T21:18:18.450260+0000","last_clean":"2024-09-10T21:18:18.450260+0000","last_became_active":"2024-09-10T21:18:17.457883+0000","last_became_peered":"2024-09-10T21:18:17.457883+0000","last_unstale":"2024-09-10T21:18:18.450260+0000","last_undegraded":"2024-09-10T21:18:18.450260+0000","last_fullsized":"2024-09-10T21:18:18.450260+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:09.889770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.443049+0000","last_change":"2024-09-10T21:18:16.449310+0000","last_active":"2024-09-10T21:18:17.443049+0000","last_peered":"2024-09-10T21:18:17.443049+0000","last_clean":"2024-09-10T21:18:17.443049+0000","last_became_active":"2024-09-10T21:18:16.449019+0000","last_became_peered":"2024-09-10T21:18:16.449019+0000","last_unstale":"2024-09-10T21:18:17.443049+0000","last_undegraded":"2024-09-10T21:18:17.443049+0000","last_fullsized":"2024-09-10T21:18:17.443049+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:27.574906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.469097+0000","last_change":"2024-09-10T21:18:17.460556+0000","last_active":"2024-09-10T21:18:17.469097+0000","last_peered":"2024-09-10T21:18:17.469097+0000","last_clean":"2024-09-10T21:18:17.469097+0000","last_became_active":"2024-09-10T21:18:17.460249+0000","last_became_peered":"2024-09-10T21:18:17.460249+0000","last_unstale":"2024-09-10T21:18:17.469097+0000","last_undegraded":"2024-09-10T21:18:17.469097+0000","last_fullsized":"2024-09-10T21:18:17.469097+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:10.625775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449235+0000","last_change":"2024-09-10T21:18:16.460962+0000","last_active":"2024-09-10T21:18:18.449235+0000","last_peered":"2024-09-10T21:18:18.449235+0000","last_clean":"2024-09-10T21:18:18.449235+0000","last_became_active":"2024-09-10T21:18:16.460678+0000","last_became_peered":"2024-09-10T21:18:16.460678+0000","last_unstale":"2024-09-10T21:18:18.449235+0000","last_undegraded":"2024-09-10T21:18:18.449235+0000","last_fullsized":"2024-09-10T21:18:18.449235+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:02.771696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.471218+0000","last_change":"2024-09-10T21:18:17.468458+0000","last_active":"2024-09-10T21:18:17.471218+0000","last_peered":"2024-09-10T21:18:17.471218+0000","last_clean":"2024-09-10T21:18:17.471218+0000","last_became_active":"2024-09-10T21:18:17.468182+0000","last_became_peered":"2024-09-10T21:18:17.468182+0000","last_unstale":"2024-09-10T21:18:17.471218+0000","last_undegraded":"2024-09-10T21:18:17.471218+0000","last_fullsized":"2024-09-10T21:18:17.471218+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:22.073443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"126'3","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.476061+0000","last_change":"2024-09-10T21:18:16.450778+0000","last_active":"2024-09-10T21:18:17.476061+0000","last_peered":"2024-09-10T21:18:17.476061+0000","last_clean":"2024-09-10T21:18:17.476061+0000","last_became_active":"2024-09-10T21:18:16.450507+0000","last_became_peered":"2024-09-10T21:18:16.450507+0000","last_unstale":"2024-09-10T21:18:17.476061+0000","last_undegraded":"2024-09-10T21:18:17.476061+0000","last_fullsized":"2024-09-10T21:18:17.476061+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:27.028438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":296,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.445103+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:18:17.445103+0000","last_peered":"2024-09-10T21:18:17.445103+0000","last_clean":"2024-09-10T21:18:17.445103+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:18:17.445103+0000","last_undegraded":"2024-09-10T21:18:17.445103+0000","last_fullsized":"2024-09-10T21:18:17.445103+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36888,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334952,"statfs":{"total":96636764160,"available":96598990848,"internally_reserved":0,"allocated":1572864,"data_stored":676865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509541,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37492,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334348,"statfs":{"total":96636764160,"available":96598372352,"internally_reserved":0,"allocated":2191360,"data_stored":1269039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509539,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1593344,"data_stored":679081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509538,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1581056,"data_stored":677481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":683187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509539,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1576960,"data_stored":676955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542243,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41032,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330808,"statfs":{"total":96636764160,"available":96594747392,"internally_reserved":0,"allocated":1560576,"data_stored":670090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542242,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37476,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334364,"statfs":{"total":96636764160,"available":96598388736,"internally_reserved":0,"allocated":2170880,"data_stored":1267303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:18:21.595 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:18:21.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.776+0000 7f06f6800640 1 -- 172.21.15.16:0/1539675769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06f00ce2f0 msgr2=0x7f06f00ce710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.776+0000 7f06f6800640 1 --2- 172.21.15.16:0/1539675769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06f00ce2f0 0x7f06f00ce710 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009900 tx=0x7f06dc01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.776+0000 7f06f6800640 1 -- 172.21.15.16:0/1539675769 shutdown_connections 2024-09-10T21:18:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.776+0000 7f06f6800640 1 --2- 172.21.15.16:0/1539675769 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f06f006dcf0 0x7f06f006e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.776+0000 7f06f6800640 1 --2- 172.21.15.16:0/1539675769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06f00ce2f0 0x7f06f00ce710 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.776+0000 7f06f6800640 1 -- 172.21.15.16:0/1539675769 >> 172.21.15.16:0/1539675769 conn(0x7f06f00d1d00 msgr2=0x7f06f00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.776+0000 7f06f6800640 1 -- 172.21.15.16:0/1539675769 shutdown_connections 2024-09-10T21:18:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.776+0000 7f06f6800640 1 -- 172.21.15.16:0/1539675769 wait complete. 2024-09-10T21:18:21.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.776+0000 7f06f6800640 1 Processor -- start 2024-09-10T21:18:21.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.776+0000 7f06f6800640 1 -- start start 2024-09-10T21:18:21.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f6800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f06f006dcf0 0x7f06f01f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:21.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f6800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06f00ce2f0 0x7f06f00c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:21.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06f0161410 0x7f06f00af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:21.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f6800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06f00ccb70 con 0x7f06f0161410 2024-09-10T21:18:21.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f6800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f06f00cc9f0 con 0x7f06f00ce2f0 2024-09-10T21:18:21.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f6800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f06f006e6c0 con 0x7f06f006dcf0 2024-09-10T21:18:21.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06f0161410 0x7f06f00af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:21.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06f0161410 0x7f06f00af120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52872/0 (socket says 172.21.15.16:52872) 2024-09-10T21:18:21.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f5e00640 1 -- 172.21.15.16:0/2002863599 learned_addr learned my addr 172.21.15.16:0/2002863599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:21.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f4a00640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06f00ce2f0 0x7f06f00c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:21.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f5400640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f06f006dcf0 0x7f06f01f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:21.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f5e00640 1 -- 172.21.15.16:0/2002863599 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f06f006dcf0 msgr2=0x7f06f01f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f5e00640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f06f006dcf0 0x7f06f01f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f5e00640 1 -- 172.21.15.16:0/2002863599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06f00ce2f0 msgr2=0x7f06f00c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f5e00640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06f00ce2f0 0x7f06f00c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f5e00640 1 -- 172.21.15.16:0/2002863599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f00c6ac0 con 0x7f06f0161410 2024-09-10T21:18:21.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f5e00640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06f0161410 0x7f06f00af120 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f06e800eb20 tx=0x7f06e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:21.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06ed600640 1 -- 172.21.15.16:0/2002863599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8004b30 con 0x7f06f0161410 2024-09-10T21:18:21.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06ed600640 1 -- 172.21.15.16:0/2002863599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06e8004cf0 con 0x7f06f0161410 2024-09-10T21:18:21.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06ed600640 1 -- 172.21.15.16:0/2002863599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8005280 con 0x7f06f0161410 2024-09-10T21:18:21.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f6800640 1 -- 172.21.15.16:0/2002863599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f00c6da0 con 0x7f06f0161410 2024-09-10T21:18:21.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f6800640 1 -- 172.21.15.16:0/2002863599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f015c660 con 0x7f06f0161410 2024-09-10T21:18:21.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.780+0000 7f06f6800640 1 -- 172.21.15.16:0/2002863599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f006f340 con 0x7f06f0161410 2024-09-10T21:18:21.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.784+0000 7f06ed600640 1 -- 172.21.15.16:0/2002863599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f06e8005420 con 0x7f06f0161410 2024-09-10T21:18:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.784+0000 7f06ed600640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f06d00725e0 0x7f06d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:21.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.784+0000 7f06ed600640 1 -- 172.21.15.16:0/2002863599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f06e8093ea0 con 0x7f06f0161410 2024-09-10T21:18:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.784+0000 7f06f5400640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f06d00725e0 0x7f06d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.784+0000 7f06f5400640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f06d00725e0 0x7f06d0074aa0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f06f00c5730 tx=0x7f06e4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.788+0000 7f06ed600640 1 -- 172.21.15.16:0/2002863599 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f06e805ef20 con 0x7f06f0161410 2024-09-10T21:18:21.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.956+0000 7f06f6800640 1 -- 172.21.15.16:0/2002863599 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06f00b1da0 con 0x7f06d00725e0 2024-09-10T21:18:21.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.960+0000 7f06ed600640 1 -- 172.21.15.16:0/2002863599 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252466 (secure 0 0 0) 0x7f06f00b1da0 con 0x7f06d00725e0 2024-09-10T21:18:21.969 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:18:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.964+0000 7f06f6800640 1 -- 172.21.15.16:0/2002863599 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f06d00725e0 msgr2=0x7f06d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.964+0000 7f06f6800640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f06d00725e0 0x7f06d0074aa0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f06f00c5730 tx=0x7f06e4009040 comp rx=0 tx=0).stop 2024-09-10T21:18:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.964+0000 7f06f6800640 1 -- 172.21.15.16:0/2002863599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06f0161410 msgr2=0x7f06f00af120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.964+0000 7f06f6800640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06f0161410 0x7f06f00af120 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f06e800eb20 tx=0x7f06e800c2d0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.964+0000 7f06f6800640 1 -- 172.21.15.16:0/2002863599 shutdown_connections 2024-09-10T21:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.964+0000 7f06f6800640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f06d00725e0 0x7f06d0074aa0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.964+0000 7f06f6800640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06f0161410 0x7f06f00af120 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.964+0000 7f06f6800640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06f00ce2f0 0x7f06f00c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.964+0000 7f06f6800640 1 --2- 172.21.15.16:0/2002863599 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f06f006dcf0 0x7f06f01f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.964+0000 7f06f6800640 1 -- 172.21.15.16:0/2002863599 >> 172.21.15.16:0/2002863599 conn(0x7f06f00d1d00 msgr2=0x7f06f00b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.964+0000 7f06f6800640 1 -- 172.21.15.16:0/2002863599 shutdown_connections 2024-09-10T21:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:21.964+0000 7f06f6800640 1 -- 172.21.15.16:0/2002863599 wait complete. 2024-09-10T21:18:21.989 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-10T21:18:21.567516+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":288024,"kb_used_data":13512,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754686696,"statfs":{"total":773094113280,"available":772799176704,"internally_reserved":0,"allocated":13836288,"data_stored":6600001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.004214"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.462598+0000","last_change":"2024-09-10T21:18:17.455433+0000","last_active":"2024-09-10T21:18:17.462598+0000","last_peered":"2024-09-10T21:18:17.462598+0000","last_clean":"2024-09-10T21:18:17.462598+0000","last_became_active":"2024-09-10T21:18:17.455130+0000","last_became_peered":"2024-09-10T21:18:17.455130+0000","last_unstale":"2024-09-10T21:18:17.462598+0000","last_undegraded":"2024-09-10T21:18:17.462598+0000","last_fullsized":"2024-09-10T21:18:17.462598+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:22.073443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.470887+0000","last_change":"2024-09-10T21:18:17.464548+0000","last_active":"2024-09-10T21:18:17.470887+0000","last_peered":"2024-09-10T21:18:17.470887+0000","last_clean":"2024-09-10T21:18:17.470887+0000","last_became_active":"2024-09-10T21:18:17.464243+0000","last_became_peered":"2024-09-10T21:18:17.464243+0000","last_unstale":"2024-09-10T21:18:17.470887+0000","last_undegraded":"2024-09-10T21:18:17.470887+0000","last_fullsized":"2024-09-10T21:18:17.470887+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:51:33.650219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.466420+0000","last_change":"2024-09-10T21:18:17.455534+0000","last_active":"2024-09-10T21:18:17.466420+0000","last_peered":"2024-09-10T21:18:17.466420+0000","last_clean":"2024-09-10T21:18:17.466420+0000","last_became_active":"2024-09-10T21:18:17.455239+0000","last_became_peered":"2024-09-10T21:18:17.455239+0000","last_unstale":"2024-09-10T21:18:17.466420+0000","last_undegraded":"2024-09-10T21:18:17.466420+0000","last_fullsized":"2024-09-10T21:18:17.466420+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:37.078920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450076+0000","last_change":"2024-09-10T21:18:17.471715+0000","last_active":"2024-09-10T21:18:18.450076+0000","last_peered":"2024-09-10T21:18:18.450076+0000","last_clean":"2024-09-10T21:18:18.450076+0000","last_became_active":"2024-09-10T21:18:17.470547+0000","last_became_peered":"2024-09-10T21:18:17.470547+0000","last_unstale":"2024-09-10T21:18:18.450076+0000","last_undegraded":"2024-09-10T21:18:18.450076+0000","last_fullsized":"2024-09-10T21:18:18.450076+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:32.967483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.475664+0000","last_change":"2024-09-10T21:18:17.461762+0000","last_active":"2024-09-10T21:18:17.475664+0000","last_peered":"2024-09-10T21:18:17.475664+0000","last_clean":"2024-09-10T21:18:17.475664+0000","last_became_active":"2024-09-10T21:18:17.457569+0000","last_became_peered":"2024-09-10T21:18:17.457569+0000","last_unstale":"2024-09-10T21:18:17.475664+0000","last_undegraded":"2024-09-10T21:18:17.475664+0000","last_fullsized":"2024-09-10T21:18:17.475664+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:35.617615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449720+0000","last_change":"2024-09-10T21:18:17.463255+0000","last_active":"2024-09-10T21:18:18.449720+0000","last_peered":"2024-09-10T21:18:18.449720+0000","last_clean":"2024-09-10T21:18:18.449720+0000","last_became_active":"2024-09-10T21:18:17.462928+0000","last_became_peered":"2024-09-10T21:18:17.462928+0000","last_unstale":"2024-09-10T21:18:18.449720+0000","last_undegraded":"2024-09-10T21:18:18.449720+0000","last_fullsized":"2024-09-10T21:18:18.449720+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:16.738072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.451894+0000","last_change":"2024-09-10T21:18:17.444816+0000","last_active":"2024-09-10T21:18:17.451894+0000","last_peered":"2024-09-10T21:18:17.451894+0000","last_clean":"2024-09-10T21:18:17.451894+0000","last_became_active":"2024-09-10T21:18:17.444529+0000","last_became_peered":"2024-09-10T21:18:17.444529+0000","last_unstale":"2024-09-10T21:18:17.451894+0000","last_undegraded":"2024-09-10T21:18:17.451894+0000","last_fullsized":"2024-09-10T21:18:17.451894+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:19.731748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.460592+0000","last_change":"2024-09-10T21:18:17.453908+0000","last_active":"2024-09-10T21:18:17.460592+0000","last_peered":"2024-09-10T21:18:17.460592+0000","last_clean":"2024-09-10T21:18:17.460592+0000","last_became_active":"2024-09-10T21:18:17.453571+0000","last_became_peered":"2024-09-10T21:18:17.453571+0000","last_unstale":"2024-09-10T21:18:17.460592+0000","last_undegraded":"2024-09-10T21:18:17.460592+0000","last_fullsized":"2024-09-10T21:18:17.460592+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:23.490272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.451433+0000","last_change":"2024-09-10T21:18:17.462106+0000","last_active":"2024-09-10T21:18:18.451433+0000","last_peered":"2024-09-10T21:18:18.451433+0000","last_clean":"2024-09-10T21:18:18.451433+0000","last_became_active":"2024-09-10T21:18:17.461759+0000","last_became_peered":"2024-09-10T21:18:17.461759+0000","last_unstale":"2024-09-10T21:18:18.451433+0000","last_undegraded":"2024-09-10T21:18:18.451433+0000","last_fullsized":"2024-09-10T21:18:18.451433+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:45.628404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.471495+0000","last_change":"2024-09-10T21:18:17.466009+0000","last_active":"2024-09-10T21:18:17.471495+0000","last_peered":"2024-09-10T21:18:17.471495+0000","last_clean":"2024-09-10T21:18:17.471495+0000","last_became_active":"2024-09-10T21:18:17.460658+0000","last_became_peered":"2024-09-10T21:18:17.460658+0000","last_unstale":"2024-09-10T21:18:17.471495+0000","last_undegraded":"2024-09-10T21:18:17.471495+0000","last_fullsized":"2024-09-10T21:18:17.471495+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:48.966839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450953+0000","last_change":"2024-09-10T21:18:17.463023+0000","last_active":"2024-09-10T21:18:18.450953+0000","last_peered":"2024-09-10T21:18:18.450953+0000","last_clean":"2024-09-10T21:18:18.450953+0000","last_became_active":"2024-09-10T21:18:17.462719+0000","last_became_peered":"2024-09-10T21:18:17.462719+0000","last_unstale":"2024-09-10T21:18:18.450953+0000","last_undegraded":"2024-09-10T21:18:18.450953+0000","last_fullsized":"2024-09-10T21:18:18.450953+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:16.392165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.465930+0000","last_change":"2024-09-10T21:18:17.456151+0000","last_active":"2024-09-10T21:18:17.465930+0000","last_peered":"2024-09-10T21:18:17.465930+0000","last_clean":"2024-09-10T21:18:17.465930+0000","last_became_active":"2024-09-10T21:18:17.455835+0000","last_became_peered":"2024-09-10T21:18:17.455835+0000","last_unstale":"2024-09-10T21:18:17.465930+0000","last_undegraded":"2024-09-10T21:18:17.465930+0000","last_fullsized":"2024-09-10T21:18:17.465930+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:31.566129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.474916+0000","last_change":"2024-09-10T21:18:17.454689+0000","last_active":"2024-09-10T21:18:17.474916+0000","last_peered":"2024-09-10T21:18:17.474916+0000","last_clean":"2024-09-10T21:18:17.474916+0000","last_became_active":"2024-09-10T21:18:17.454390+0000","last_became_peered":"2024-09-10T21:18:17.454390+0000","last_unstale":"2024-09-10T21:18:17.474916+0000","last_undegraded":"2024-09-10T21:18:17.474916+0000","last_fullsized":"2024-09-10T21:18:17.474916+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:20.233731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.467468+0000","last_change":"2024-09-10T21:18:17.460348+0000","last_active":"2024-09-10T21:18:17.467468+0000","last_peered":"2024-09-10T21:18:17.467468+0000","last_clean":"2024-09-10T21:18:17.467468+0000","last_became_active":"2024-09-10T21:18:17.460103+0000","last_became_peered":"2024-09-10T21:18:17.460103+0000","last_unstale":"2024-09-10T21:18:17.467468+0000","last_undegraded":"2024-09-10T21:18:17.467468+0000","last_fullsized":"2024-09-10T21:18:17.467468+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:22.030981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.475801+0000","last_change":"2024-09-10T21:18:17.459466+0000","last_active":"2024-09-10T21:18:17.475801+0000","last_peered":"2024-09-10T21:18:17.475801+0000","last_clean":"2024-09-10T21:18:17.475801+0000","last_became_active":"2024-09-10T21:18:17.459171+0000","last_became_peered":"2024-09-10T21:18:17.459171+0000","last_unstale":"2024-09-10T21:18:17.475801+0000","last_undegraded":"2024-09-10T21:18:17.475801+0000","last_fullsized":"2024-09-10T21:18:17.475801+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:05.053362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.464669+0000","last_change":"2024-09-10T21:18:17.459861+0000","last_active":"2024-09-10T21:18:17.464669+0000","last_peered":"2024-09-10T21:18:17.464669+0000","last_clean":"2024-09-10T21:18:17.464669+0000","last_became_active":"2024-09-10T21:18:17.459596+0000","last_became_peered":"2024-09-10T21:18:17.459596+0000","last_unstale":"2024-09-10T21:18:17.464669+0000","last_undegraded":"2024-09-10T21:18:17.464669+0000","last_fullsized":"2024-09-10T21:18:17.464669+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:23.490272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.466902+0000","last_change":"2024-09-10T21:18:17.460115+0000","last_active":"2024-09-10T21:18:17.466902+0000","last_peered":"2024-09-10T21:18:17.466902+0000","last_clean":"2024-09-10T21:18:17.466902+0000","last_became_active":"2024-09-10T21:18:17.458546+0000","last_became_peered":"2024-09-10T21:18:17.458546+0000","last_unstale":"2024-09-10T21:18:17.466902+0000","last_undegraded":"2024-09-10T21:18:17.466902+0000","last_fullsized":"2024-09-10T21:18:17.466902+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:21.518628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.475477+0000","last_change":"2024-09-10T21:18:17.458335+0000","last_active":"2024-09-10T21:18:17.475477+0000","last_peered":"2024-09-10T21:18:17.475477+0000","last_clean":"2024-09-10T21:18:17.475477+0000","last_became_active":"2024-09-10T21:18:17.458022+0000","last_became_peered":"2024-09-10T21:18:17.458022+0000","last_unstale":"2024-09-10T21:18:17.475477+0000","last_undegraded":"2024-09-10T21:18:17.475477+0000","last_fullsized":"2024-09-10T21:18:17.475477+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:06.117473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.460570+0000","last_change":"2024-09-10T21:18:17.454330+0000","last_active":"2024-09-10T21:18:17.460570+0000","last_peered":"2024-09-10T21:18:17.460570+0000","last_clean":"2024-09-10T21:18:17.460570+0000","last_became_active":"2024-09-10T21:18:17.452857+0000","last_became_peered":"2024-09-10T21:18:17.452857+0000","last_unstale":"2024-09-10T21:18:17.460570+0000","last_undegraded":"2024-09-10T21:18:17.460570+0000","last_fullsized":"2024-09-10T21:18:17.460570+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:49.357785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.463476+0000","last_change":"2024-09-10T21:18:17.460034+0000","last_active":"2024-09-10T21:18:17.463476+0000","last_peered":"2024-09-10T21:18:17.463476+0000","last_clean":"2024-09-10T21:18:17.463476+0000","last_became_active":"2024-09-10T21:18:17.459465+0000","last_became_peered":"2024-09-10T21:18:17.459465+0000","last_unstale":"2024-09-10T21:18:17.463476+0000","last_undegraded":"2024-09-10T21:18:17.463476+0000","last_fullsized":"2024-09-10T21:18:17.463476+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:36.547391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.467113+0000","last_change":"2024-09-10T21:18:17.464677+0000","last_active":"2024-09-10T21:18:17.467113+0000","last_peered":"2024-09-10T21:18:17.467113+0000","last_clean":"2024-09-10T21:18:17.467113+0000","last_became_active":"2024-09-10T21:18:17.464365+0000","last_became_peered":"2024-09-10T21:18:17.464365+0000","last_unstale":"2024-09-10T21:18:17.467113+0000","last_undegraded":"2024-09-10T21:18:17.467113+0000","last_fullsized":"2024-09-10T21:18:17.467113+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:39.589294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.471934+0000","last_change":"2024-09-10T21:18:17.469210+0000","last_active":"2024-09-10T21:18:17.471934+0000","last_peered":"2024-09-10T21:18:17.471934+0000","last_clean":"2024-09-10T21:18:17.471934+0000","last_became_active":"2024-09-10T21:18:17.467834+0000","last_became_peered":"2024-09-10T21:18:17.467834+0000","last_unstale":"2024-09-10T21:18:17.471934+0000","last_undegraded":"2024-09-10T21:18:17.471934+0000","last_fullsized":"2024-09-10T21:18:17.471934+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:05.602519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449902+0000","last_change":"2024-09-10T21:18:17.459615+0000","last_active":"2024-09-10T21:18:18.449902+0000","last_peered":"2024-09-10T21:18:18.449902+0000","last_clean":"2024-09-10T21:18:18.449902+0000","last_became_active":"2024-09-10T21:18:17.459272+0000","last_became_peered":"2024-09-10T21:18:17.459272+0000","last_unstale":"2024-09-10T21:18:18.449902+0000","last_undegraded":"2024-09-10T21:18:18.449902+0000","last_fullsized":"2024-09-10T21:18:18.449902+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:31.547340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.476259+0000","last_change":"2024-09-10T21:18:17.470840+0000","last_active":"2024-09-10T21:18:17.476259+0000","last_peered":"2024-09-10T21:18:17.476259+0000","last_clean":"2024-09-10T21:18:17.476259+0000","last_became_active":"2024-09-10T21:18:17.470558+0000","last_became_peered":"2024-09-10T21:18:17.470558+0000","last_unstale":"2024-09-10T21:18:17.476259+0000","last_undegraded":"2024-09-10T21:18:17.476259+0000","last_fullsized":"2024-09-10T21:18:17.476259+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:53.192529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.470466+0000","last_change":"2024-09-10T21:18:17.461564+0000","last_active":"2024-09-10T21:18:17.470466+0000","last_peered":"2024-09-10T21:18:17.470466+0000","last_clean":"2024-09-10T21:18:17.470466+0000","last_became_active":"2024-09-10T21:18:17.461263+0000","last_became_peered":"2024-09-10T21:18:17.461263+0000","last_unstale":"2024-09-10T21:18:17.470466+0000","last_undegraded":"2024-09-10T21:18:17.470466+0000","last_fullsized":"2024-09-10T21:18:17.470466+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:51.557073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.462822+0000","last_change":"2024-09-10T21:18:17.459403+0000","last_active":"2024-09-10T21:18:17.462822+0000","last_peered":"2024-09-10T21:18:17.462822+0000","last_clean":"2024-09-10T21:18:17.462822+0000","last_became_active":"2024-09-10T21:18:17.459059+0000","last_became_peered":"2024-09-10T21:18:17.459059+0000","last_unstale":"2024-09-10T21:18:17.462822+0000","last_undegraded":"2024-09-10T21:18:17.462822+0000","last_fullsized":"2024-09-10T21:18:17.462822+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:37.451914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.470981+0000","last_change":"2024-09-10T21:18:17.465604+0000","last_active":"2024-09-10T21:18:17.470981+0000","last_peered":"2024-09-10T21:18:17.470981+0000","last_clean":"2024-09-10T21:18:17.470981+0000","last_became_active":"2024-09-10T21:18:17.465279+0000","last_became_peered":"2024-09-10T21:18:17.465279+0000","last_unstale":"2024-09-10T21:18:17.470981+0000","last_undegraded":"2024-09-10T21:18:17.470981+0000","last_fullsized":"2024-09-10T21:18:17.470981+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:53.537805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.470929+0000","last_change":"2024-09-10T21:18:17.466628+0000","last_active":"2024-09-10T21:18:17.470929+0000","last_peered":"2024-09-10T21:18:17.470929+0000","last_clean":"2024-09-10T21:18:17.470929+0000","last_became_active":"2024-09-10T21:18:17.466267+0000","last_became_peered":"2024-09-10T21:18:17.466267+0000","last_unstale":"2024-09-10T21:18:17.470929+0000","last_undegraded":"2024-09-10T21:18:17.470929+0000","last_fullsized":"2024-09-10T21:18:17.470929+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:33.088284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.462303+0000","last_change":"2024-09-10T21:18:17.458670+0000","last_active":"2024-09-10T21:18:17.462303+0000","last_peered":"2024-09-10T21:18:17.462303+0000","last_clean":"2024-09-10T21:18:17.462303+0000","last_became_active":"2024-09-10T21:18:17.458404+0000","last_became_peered":"2024-09-10T21:18:17.458404+0000","last_unstale":"2024-09-10T21:18:17.462303+0000","last_undegraded":"2024-09-10T21:18:17.462303+0000","last_fullsized":"2024-09-10T21:18:17.462303+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:42.105578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.467495+0000","last_change":"2024-09-10T21:18:17.461628+0000","last_active":"2024-09-10T21:18:17.467495+0000","last_peered":"2024-09-10T21:18:17.467495+0000","last_clean":"2024-09-10T21:18:17.467495+0000","last_became_active":"2024-09-10T21:18:17.461406+0000","last_became_peered":"2024-09-10T21:18:17.461406+0000","last_unstale":"2024-09-10T21:18:17.467495+0000","last_undegraded":"2024-09-10T21:18:17.467495+0000","last_fullsized":"2024-09-10T21:18:17.467495+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:33.906846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.467756+0000","last_change":"2024-09-10T21:18:17.456278+0000","last_active":"2024-09-10T21:18:17.467756+0000","last_peered":"2024-09-10T21:18:17.467756+0000","last_clean":"2024-09-10T21:18:17.467756+0000","last_became_active":"2024-09-10T21:18:17.455986+0000","last_became_peered":"2024-09-10T21:18:17.455986+0000","last_unstale":"2024-09-10T21:18:17.467756+0000","last_undegraded":"2024-09-10T21:18:17.467756+0000","last_fullsized":"2024-09-10T21:18:17.467756+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:53.429871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.472062+0000","last_change":"2024-09-10T21:18:17.470807+0000","last_active":"2024-09-10T21:18:17.472062+0000","last_peered":"2024-09-10T21:18:17.472062+0000","last_clean":"2024-09-10T21:18:17.472062+0000","last_became_active":"2024-09-10T21:18:17.469846+0000","last_became_peered":"2024-09-10T21:18:17.469846+0000","last_unstale":"2024-09-10T21:18:17.472062+0000","last_undegraded":"2024-09-10T21:18:17.472062+0000","last_fullsized":"2024-09-10T21:18:17.472062+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:20.052187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.471694+0000","last_change":"2024-09-10T21:18:17.454333+0000","last_active":"2024-09-10T21:18:17.471694+0000","last_peered":"2024-09-10T21:18:17.471694+0000","last_clean":"2024-09-10T21:18:17.471694+0000","last_became_active":"2024-09-10T21:18:17.454038+0000","last_became_peered":"2024-09-10T21:18:17.454038+0000","last_unstale":"2024-09-10T21:18:17.471694+0000","last_undegraded":"2024-09-10T21:18:17.471694+0000","last_fullsized":"2024-09-10T21:18:17.471694+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:33.175493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"126'4","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.467051+0000","last_change":"2024-09-10T21:18:16.453681+0000","last_active":"2024-09-10T21:18:17.467051+0000","last_peered":"2024-09-10T21:18:17.467051+0000","last_clean":"2024-09-10T21:18:17.467051+0000","last_became_active":"2024-09-10T21:18:16.453327+0000","last_became_peered":"2024-09-10T21:18:16.453327+0000","last_unstale":"2024-09-10T21:18:17.467051+0000","last_undegraded":"2024-09-10T21:18:17.467051+0000","last_fullsized":"2024-09-10T21:18:17.467051+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:23.960174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.467591+0000","last_change":"2024-09-10T21:18:17.462020+0000","last_active":"2024-09-10T21:18:17.467591+0000","last_peered":"2024-09-10T21:18:17.467591+0000","last_clean":"2024-09-10T21:18:17.467591+0000","last_became_active":"2024-09-10T21:18:17.461715+0000","last_became_peered":"2024-09-10T21:18:17.461715+0000","last_unstale":"2024-09-10T21:18:17.467591+0000","last_undegraded":"2024-09-10T21:18:17.467591+0000","last_fullsized":"2024-09-10T21:18:17.467591+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:49.357785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.451320+0000","last_change":"2024-09-10T21:18:16.460877+0000","last_active":"2024-09-10T21:18:18.451320+0000","last_peered":"2024-09-10T21:18:18.451320+0000","last_clean":"2024-09-10T21:18:18.451320+0000","last_became_active":"2024-09-10T21:18:16.460616+0000","last_became_peered":"2024-09-10T21:18:16.460616+0000","last_unstale":"2024-09-10T21:18:18.451320+0000","last_undegraded":"2024-09-10T21:18:18.451320+0000","last_fullsized":"2024-09-10T21:18:18.451320+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:53.332618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450582+0000","last_change":"2024-09-10T21:18:17.467486+0000","last_active":"2024-09-10T21:18:18.450582+0000","last_peered":"2024-09-10T21:18:18.450582+0000","last_clean":"2024-09-10T21:18:18.450582+0000","last_became_active":"2024-09-10T21:18:17.467192+0000","last_became_peered":"2024-09-10T21:18:17.467192+0000","last_unstale":"2024-09-10T21:18:18.450582+0000","last_undegraded":"2024-09-10T21:18:18.450582+0000","last_fullsized":"2024-09-10T21:18:18.450582+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:22.161838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"126'2","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.474584+0000","last_change":"2024-09-10T21:18:16.450007+0000","last_active":"2024-09-10T21:18:17.474584+0000","last_peered":"2024-09-10T21:18:17.474584+0000","last_clean":"2024-09-10T21:18:17.474584+0000","last_became_active":"2024-09-10T21:18:16.448556+0000","last_became_peered":"2024-09-10T21:18:16.448556+0000","last_unstale":"2024-09-10T21:18:17.474584+0000","last_undegraded":"2024-09-10T21:18:17.474584+0000","last_fullsized":"2024-09-10T21:18:17.474584+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:58.542805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450078+0000","last_change":"2024-09-10T21:18:17.468571+0000","last_active":"2024-09-10T21:18:18.450078+0000","last_peered":"2024-09-10T21:18:18.450078+0000","last_clean":"2024-09-10T21:18:18.450078+0000","last_became_active":"2024-09-10T21:18:17.468322+0000","last_became_peered":"2024-09-10T21:18:17.468322+0000","last_unstale":"2024-09-10T21:18:18.450078+0000","last_undegraded":"2024-09-10T21:18:18.450078+0000","last_fullsized":"2024-09-10T21:18:18.450078+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:16.803317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"126'2","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.466770+0000","last_change":"2024-09-10T21:18:16.454414+0000","last_active":"2024-09-10T21:18:17.466770+0000","last_peered":"2024-09-10T21:18:17.466770+0000","last_clean":"2024-09-10T21:18:17.466770+0000","last_became_active":"2024-09-10T21:18:16.454132+0000","last_became_peered":"2024-09-10T21:18:16.454132+0000","last_unstale":"2024-09-10T21:18:17.466770+0000","last_undegraded":"2024-09-10T21:18:17.466770+0000","last_fullsized":"2024-09-10T21:18:17.466770+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:26.959815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.468082+0000","last_change":"2024-09-10T21:18:17.459136+0000","last_active":"2024-09-10T21:18:17.468082+0000","last_peered":"2024-09-10T21:18:17.468082+0000","last_clean":"2024-09-10T21:18:17.468082+0000","last_became_active":"2024-09-10T21:18:17.458857+0000","last_became_peered":"2024-09-10T21:18:17.458857+0000","last_unstale":"2024-09-10T21:18:17.468082+0000","last_undegraded":"2024-09-10T21:18:17.468082+0000","last_fullsized":"2024-09-10T21:18:17.468082+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:40.891058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"126'1","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.474535+0000","last_change":"2024-09-10T21:18:16.455034+0000","last_active":"2024-09-10T21:18:17.474535+0000","last_peered":"2024-09-10T21:18:17.474535+0000","last_clean":"2024-09-10T21:18:17.474535+0000","last_became_active":"2024-09-10T21:18:16.454213+0000","last_became_peered":"2024-09-10T21:18:16.454213+0000","last_unstale":"2024-09-10T21:18:17.474535+0000","last_undegraded":"2024-09-10T21:18:17.474535+0000","last_fullsized":"2024-09-10T21:18:17.474535+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:44.998071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450218+0000","last_change":"2024-09-10T21:18:17.473440+0000","last_active":"2024-09-10T21:18:18.450218+0000","last_peered":"2024-09-10T21:18:18.450218+0000","last_clean":"2024-09-10T21:18:18.450218+0000","last_became_active":"2024-09-10T21:18:17.470781+0000","last_became_peered":"2024-09-10T21:18:17.470781+0000","last_unstale":"2024-09-10T21:18:18.450218+0000","last_undegraded":"2024-09-10T21:18:18.450218+0000","last_fullsized":"2024-09-10T21:18:18.450218+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:59.187772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"126'1","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.475956+0000","last_change":"2024-09-10T21:18:16.461209+0000","last_active":"2024-09-10T21:18:17.475956+0000","last_peered":"2024-09-10T21:18:17.475956+0000","last_clean":"2024-09-10T21:18:17.475956+0000","last_became_active":"2024-09-10T21:18:16.459922+0000","last_became_peered":"2024-09-10T21:18:16.459922+0000","last_unstale":"2024-09-10T21:18:17.475956+0000","last_undegraded":"2024-09-10T21:18:17.475956+0000","last_fullsized":"2024-09-10T21:18:17.475956+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:36.787152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.472110+0000","last_change":"2024-09-10T21:18:17.470809+0000","last_active":"2024-09-10T21:18:17.472110+0000","last_peered":"2024-09-10T21:18:17.472110+0000","last_clean":"2024-09-10T21:18:17.472110+0000","last_became_active":"2024-09-10T21:18:17.470215+0000","last_became_peered":"2024-09-10T21:18:17.470215+0000","last_unstale":"2024-09-10T21:18:17.472110+0000","last_undegraded":"2024-09-10T21:18:17.472110+0000","last_fullsized":"2024-09-10T21:18:17.472110+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:00.803035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449419+0000","last_change":"2024-09-10T21:18:16.452184+0000","last_active":"2024-09-10T21:18:18.449419+0000","last_peered":"2024-09-10T21:18:18.449419+0000","last_clean":"2024-09-10T21:18:18.449419+0000","last_became_active":"2024-09-10T21:18:16.451895+0000","last_became_peered":"2024-09-10T21:18:16.451895+0000","last_unstale":"2024-09-10T21:18:18.449419+0000","last_undegraded":"2024-09-10T21:18:18.449419+0000","last_fullsized":"2024-09-10T21:18:18.449419+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:35.807764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449670+0000","last_change":"2024-09-10T21:18:17.446741+0000","last_active":"2024-09-10T21:18:18.449670+0000","last_peered":"2024-09-10T21:18:18.449670+0000","last_clean":"2024-09-10T21:18:18.449670+0000","last_became_active":"2024-09-10T21:18:17.446462+0000","last_became_peered":"2024-09-10T21:18:17.446462+0000","last_unstale":"2024-09-10T21:18:18.449670+0000","last_undegraded":"2024-09-10T21:18:18.449670+0000","last_fullsized":"2024-09-10T21:18:18.449670+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:50.237205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.446575+0000","last_change":"2024-09-10T21:18:16.456352+0000","last_active":"2024-09-10T21:18:17.446575+0000","last_peered":"2024-09-10T21:18:17.446575+0000","last_clean":"2024-09-10T21:18:17.446575+0000","last_became_active":"2024-09-10T21:18:16.454832+0000","last_became_peered":"2024-09-10T21:18:16.454832+0000","last_unstale":"2024-09-10T21:18:17.446575+0000","last_undegraded":"2024-09-10T21:18:17.446575+0000","last_fullsized":"2024-09-10T21:18:17.446575+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:51.063254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450289+0000","last_change":"2024-09-10T21:18:17.456882+0000","last_active":"2024-09-10T21:18:18.450289+0000","last_peered":"2024-09-10T21:18:18.450289+0000","last_clean":"2024-09-10T21:18:18.450289+0000","last_became_active":"2024-09-10T21:18:17.456343+0000","last_became_peered":"2024-09-10T21:18:17.456343+0000","last_unstale":"2024-09-10T21:18:18.450289+0000","last_undegraded":"2024-09-10T21:18:18.450289+0000","last_fullsized":"2024-09-10T21:18:18.450289+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:56.596133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.451361+0000","last_change":"2024-09-10T21:18:16.456043+0000","last_active":"2024-09-10T21:18:18.451361+0000","last_peered":"2024-09-10T21:18:18.451361+0000","last_clean":"2024-09-10T21:18:18.451361+0000","last_became_active":"2024-09-10T21:18:16.455629+0000","last_became_peered":"2024-09-10T21:18:16.455629+0000","last_unstale":"2024-09-10T21:18:18.451361+0000","last_undegraded":"2024-09-10T21:18:18.451361+0000","last_fullsized":"2024-09-10T21:18:18.451361+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:47.641586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.468943+0000","last_change":"2024-09-10T21:18:17.460429+0000","last_active":"2024-09-10T21:18:17.468943+0000","last_peered":"2024-09-10T21:18:17.468943+0000","last_clean":"2024-09-10T21:18:17.468943+0000","last_became_active":"2024-09-10T21:18:17.460126+0000","last_became_peered":"2024-09-10T21:18:17.460126+0000","last_unstale":"2024-09-10T21:18:17.468943+0000","last_undegraded":"2024-09-10T21:18:17.468943+0000","last_fullsized":"2024-09-10T21:18:17.468943+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:31.992739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.445589+0000","last_change":"2024-09-10T21:18:16.456383+0000","last_active":"2024-09-10T21:18:17.445589+0000","last_peered":"2024-09-10T21:18:17.445589+0000","last_clean":"2024-09-10T21:18:17.445589+0000","last_became_active":"2024-09-10T21:18:16.456028+0000","last_became_peered":"2024-09-10T21:18:16.456028+0000","last_unstale":"2024-09-10T21:18:17.445589+0000","last_undegraded":"2024-09-10T21:18:17.445589+0000","last_fullsized":"2024-09-10T21:18:17.445589+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:05.929715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.464665+0000","last_change":"2024-09-10T21:18:17.458171+0000","last_active":"2024-09-10T21:18:17.464665+0000","last_peered":"2024-09-10T21:18:17.464665+0000","last_clean":"2024-09-10T21:18:17.464665+0000","last_became_active":"2024-09-10T21:18:17.457891+0000","last_became_peered":"2024-09-10T21:18:17.457891+0000","last_unstale":"2024-09-10T21:18:17.464665+0000","last_undegraded":"2024-09-10T21:18:17.464665+0000","last_fullsized":"2024-09-10T21:18:17.464665+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:19.493906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"126'6","reported_seq":19,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.476539+0000","last_change":"2024-09-10T21:18:16.445813+0000","last_active":"2024-09-10T21:18:17.476539+0000","last_peered":"2024-09-10T21:18:17.476539+0000","last_clean":"2024-09-10T21:18:17.476539+0000","last_became_active":"2024-09-10T21:18:16.445298+0000","last_became_peered":"2024-09-10T21:18:16.445298+0000","last_unstale":"2024-09-10T21:18:17.476539+0000","last_undegraded":"2024-09-10T21:18:17.476539+0000","last_fullsized":"2024-09-10T21:18:17.476539+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:49.180925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449704+0000","last_change":"2024-09-10T21:18:17.461088+0000","last_active":"2024-09-10T21:18:18.449704+0000","last_peered":"2024-09-10T21:18:18.449704+0000","last_clean":"2024-09-10T21:18:18.449704+0000","last_became_active":"2024-09-10T21:18:17.460878+0000","last_became_peered":"2024-09-10T21:18:17.460878+0000","last_unstale":"2024-09-10T21:18:18.449704+0000","last_undegraded":"2024-09-10T21:18:18.449704+0000","last_fullsized":"2024-09-10T21:18:18.449704+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:05.686097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"126'2","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.475302+0000","last_change":"2024-09-10T21:18:16.451482+0000","last_active":"2024-09-10T21:18:17.475302+0000","last_peered":"2024-09-10T21:18:17.475302+0000","last_clean":"2024-09-10T21:18:17.475302+0000","last_became_active":"2024-09-10T21:18:16.449493+0000","last_became_peered":"2024-09-10T21:18:16.449493+0000","last_unstale":"2024-09-10T21:18:17.475302+0000","last_undegraded":"2024-09-10T21:18:17.475302+0000","last_fullsized":"2024-09-10T21:18:17.475302+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:23.477903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.462218+0000","last_change":"2024-09-10T21:18:17.458633+0000","last_active":"2024-09-10T21:18:17.462218+0000","last_peered":"2024-09-10T21:18:17.462218+0000","last_clean":"2024-09-10T21:18:17.462218+0000","last_became_active":"2024-09-10T21:18:17.457955+0000","last_became_peered":"2024-09-10T21:18:17.457955+0000","last_unstale":"2024-09-10T21:18:17.462218+0000","last_undegraded":"2024-09-10T21:18:17.462218+0000","last_fullsized":"2024-09-10T21:18:17.462218+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:06.016891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.452293+0000","last_change":"2024-09-10T21:18:16.455787+0000","last_active":"2024-09-10T21:18:17.452293+0000","last_peered":"2024-09-10T21:18:17.452293+0000","last_clean":"2024-09-10T21:18:17.452293+0000","last_became_active":"2024-09-10T21:18:16.455434+0000","last_became_peered":"2024-09-10T21:18:16.455434+0000","last_unstale":"2024-09-10T21:18:17.452293+0000","last_undegraded":"2024-09-10T21:18:17.452293+0000","last_fullsized":"2024-09-10T21:18:17.452293+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:17.447633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449604+0000","last_change":"2024-09-10T21:18:17.463487+0000","last_active":"2024-09-10T21:18:18.449604+0000","last_peered":"2024-09-10T21:18:18.449604+0000","last_clean":"2024-09-10T21:18:18.449604+0000","last_became_active":"2024-09-10T21:18:17.455930+0000","last_became_peered":"2024-09-10T21:18:17.455930+0000","last_unstale":"2024-09-10T21:18:18.449604+0000","last_undegraded":"2024-09-10T21:18:18.449604+0000","last_fullsized":"2024-09-10T21:18:18.449604+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:09.546582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.448981+0000","last_change":"2024-09-10T21:18:16.452258+0000","last_active":"2024-09-10T21:18:17.448981+0000","last_peered":"2024-09-10T21:18:17.448981+0000","last_clean":"2024-09-10T21:18:17.448981+0000","last_became_active":"2024-09-10T21:18:16.451978+0000","last_became_peered":"2024-09-10T21:18:16.451978+0000","last_unstale":"2024-09-10T21:18:17.448981+0000","last_undegraded":"2024-09-10T21:18:17.448981+0000","last_fullsized":"2024-09-10T21:18:17.448981+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:34.941666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450475+0000","last_change":"2024-09-10T21:18:17.466944+0000","last_active":"2024-09-10T21:18:18.450475+0000","last_peered":"2024-09-10T21:18:18.450475+0000","last_clean":"2024-09-10T21:18:18.450475+0000","last_became_active":"2024-09-10T21:18:17.458829+0000","last_became_peered":"2024-09-10T21:18:17.458829+0000","last_unstale":"2024-09-10T21:18:18.450475+0000","last_undegraded":"2024-09-10T21:18:18.450475+0000","last_fullsized":"2024-09-10T21:18:18.450475+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:39.734114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"126'8","reported_seq":21,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.458114+0000","last_change":"2024-09-10T21:18:16.450107+0000","last_active":"2024-09-10T21:18:17.458114+0000","last_peered":"2024-09-10T21:18:17.458114+0000","last_clean":"2024-09-10T21:18:17.458114+0000","last_became_active":"2024-09-10T21:18:16.449784+0000","last_became_peered":"2024-09-10T21:18:16.449784+0000","last_unstale":"2024-09-10T21:18:17.458114+0000","last_undegraded":"2024-09-10T21:18:17.458114+0000","last_fullsized":"2024-09-10T21:18:17.458114+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:18.728448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.476062+0000","last_change":"2024-09-10T21:18:17.469640+0000","last_active":"2024-09-10T21:18:17.476062+0000","last_peered":"2024-09-10T21:18:17.476062+0000","last_clean":"2024-09-10T21:18:17.476062+0000","last_became_active":"2024-09-10T21:18:17.469378+0000","last_became_peered":"2024-09-10T21:18:17.469378+0000","last_unstale":"2024-09-10T21:18:17.476062+0000","last_undegraded":"2024-09-10T21:18:17.476062+0000","last_fullsized":"2024-09-10T21:18:17.476062+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:40.261097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450465+0000","last_change":"2024-09-10T21:18:16.443607+0000","last_active":"2024-09-10T21:18:18.450465+0000","last_peered":"2024-09-10T21:18:18.450465+0000","last_clean":"2024-09-10T21:18:18.450465+0000","last_became_active":"2024-09-10T21:18:16.443263+0000","last_became_peered":"2024-09-10T21:18:16.443263+0000","last_unstale":"2024-09-10T21:18:18.450465+0000","last_undegraded":"2024-09-10T21:18:18.450465+0000","last_fullsized":"2024-09-10T21:18:18.450465+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:08.281114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.476400+0000","last_change":"2024-09-10T21:18:17.475328+0000","last_active":"2024-09-10T21:18:17.476400+0000","last_peered":"2024-09-10T21:18:17.476400+0000","last_clean":"2024-09-10T21:18:17.476400+0000","last_became_active":"2024-09-10T21:18:17.475010+0000","last_became_peered":"2024-09-10T21:18:17.475010+0000","last_unstale":"2024-09-10T21:18:17.476400+0000","last_undegraded":"2024-09-10T21:18:17.476400+0000","last_fullsized":"2024-09-10T21:18:17.476400+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:54.055985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"126'2","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.465921+0000","last_change":"2024-09-10T21:18:16.451256+0000","last_active":"2024-09-10T21:18:17.465921+0000","last_peered":"2024-09-10T21:18:17.465921+0000","last_clean":"2024-09-10T21:18:17.465921+0000","last_became_active":"2024-09-10T21:18:16.450969+0000","last_became_peered":"2024-09-10T21:18:16.450969+0000","last_unstale":"2024-09-10T21:18:17.465921+0000","last_undegraded":"2024-09-10T21:18:17.465921+0000","last_fullsized":"2024-09-10T21:18:17.465921+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:18.490606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.476168+0000","last_change":"2024-09-10T21:18:17.469966+0000","last_active":"2024-09-10T21:18:17.476168+0000","last_peered":"2024-09-10T21:18:17.476168+0000","last_clean":"2024-09-10T21:18:17.476168+0000","last_became_active":"2024-09-10T21:18:17.469152+0000","last_became_peered":"2024-09-10T21:18:17.469152+0000","last_unstale":"2024-09-10T21:18:17.476168+0000","last_undegraded":"2024-09-10T21:18:17.476168+0000","last_fullsized":"2024-09-10T21:18:17.476168+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:57.173465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"126'1","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.474200+0000","last_change":"2024-09-10T21:18:16.453893+0000","last_active":"2024-09-10T21:18:17.474200+0000","last_peered":"2024-09-10T21:18:17.474200+0000","last_clean":"2024-09-10T21:18:17.474200+0000","last_became_active":"2024-09-10T21:18:16.453623+0000","last_became_peered":"2024-09-10T21:18:16.453623+0000","last_unstale":"2024-09-10T21:18:17.474200+0000","last_undegraded":"2024-09-10T21:18:17.474200+0000","last_fullsized":"2024-09-10T21:18:17.474200+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:36.504695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.451007+0000","last_change":"2024-09-10T21:18:17.467206+0000","last_active":"2024-09-10T21:18:18.451007+0000","last_peered":"2024-09-10T21:18:18.451007+0000","last_clean":"2024-09-10T21:18:18.451007+0000","last_became_active":"2024-09-10T21:18:17.466909+0000","last_became_peered":"2024-09-10T21:18:17.466909+0000","last_unstale":"2024-09-10T21:18:18.451007+0000","last_undegraded":"2024-09-10T21:18:18.451007+0000","last_fullsized":"2024-09-10T21:18:18.451007+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:47.366859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.452137+0000","last_change":"2024-09-10T21:18:16.452924+0000","last_active":"2024-09-10T21:18:17.452137+0000","last_peered":"2024-09-10T21:18:17.452137+0000","last_clean":"2024-09-10T21:18:17.452137+0000","last_became_active":"2024-09-10T21:18:16.451756+0000","last_became_peered":"2024-09-10T21:18:16.451756+0000","last_unstale":"2024-09-10T21:18:17.452137+0000","last_undegraded":"2024-09-10T21:18:17.452137+0000","last_fullsized":"2024-09-10T21:18:17.452137+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:25.758216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.469764+0000","last_change":"2024-09-10T21:18:17.465068+0000","last_active":"2024-09-10T21:18:17.469764+0000","last_peered":"2024-09-10T21:18:17.469764+0000","last_clean":"2024-09-10T21:18:17.469764+0000","last_became_active":"2024-09-10T21:18:17.464618+0000","last_became_peered":"2024-09-10T21:18:17.464618+0000","last_unstale":"2024-09-10T21:18:17.469764+0000","last_undegraded":"2024-09-10T21:18:17.469764+0000","last_fullsized":"2024-09-10T21:18:17.469764+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:01.943203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450065+0000","last_change":"2024-09-10T21:18:16.460009+0000","last_active":"2024-09-10T21:18:18.450065+0000","last_peered":"2024-09-10T21:18:18.450065+0000","last_clean":"2024-09-10T21:18:18.450065+0000","last_became_active":"2024-09-10T21:18:16.459722+0000","last_became_peered":"2024-09-10T21:18:16.459722+0000","last_unstale":"2024-09-10T21:18:18.450065+0000","last_undegraded":"2024-09-10T21:18:18.450065+0000","last_fullsized":"2024-09-10T21:18:18.450065+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:34.122921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.472080+0000","last_change":"2024-09-10T21:18:17.470830+0000","last_active":"2024-09-10T21:18:17.472080+0000","last_peered":"2024-09-10T21:18:17.472080+0000","last_clean":"2024-09-10T21:18:17.472080+0000","last_became_active":"2024-09-10T21:18:17.469755+0000","last_became_peered":"2024-09-10T21:18:17.469755+0000","last_unstale":"2024-09-10T21:18:17.472080+0000","last_undegraded":"2024-09-10T21:18:17.472080+0000","last_fullsized":"2024-09-10T21:18:17.472080+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:59.187712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449647+0000","last_change":"2024-09-10T21:18:16.442459+0000","last_active":"2024-09-10T21:18:18.449647+0000","last_peered":"2024-09-10T21:18:18.449647+0000","last_clean":"2024-09-10T21:18:18.449647+0000","last_became_active":"2024-09-10T21:18:16.442119+0000","last_became_peered":"2024-09-10T21:18:16.442119+0000","last_unstale":"2024-09-10T21:18:18.449647+0000","last_undegraded":"2024-09-10T21:18:18.449647+0000","last_fullsized":"2024-09-10T21:18:18.449647+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:01.779397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.471610+0000","last_change":"2024-09-10T21:18:17.466122+0000","last_active":"2024-09-10T21:18:17.471610+0000","last_peered":"2024-09-10T21:18:17.471610+0000","last_clean":"2024-09-10T21:18:17.471610+0000","last_became_active":"2024-09-10T21:18:17.461777+0000","last_became_peered":"2024-09-10T21:18:17.461777+0000","last_unstale":"2024-09-10T21:18:17.471610+0000","last_undegraded":"2024-09-10T21:18:17.471610+0000","last_fullsized":"2024-09-10T21:18:17.471610+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:49.964624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450228+0000","last_change":"2024-09-10T21:18:16.443440+0000","last_active":"2024-09-10T21:18:18.450228+0000","last_peered":"2024-09-10T21:18:18.450228+0000","last_clean":"2024-09-10T21:18:18.450228+0000","last_became_active":"2024-09-10T21:18:16.443120+0000","last_became_peered":"2024-09-10T21:18:16.443120+0000","last_unstale":"2024-09-10T21:18:18.450228+0000","last_undegraded":"2024-09-10T21:18:18.450228+0000","last_fullsized":"2024-09-10T21:18:18.450228+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:18.457315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.471683+0000","last_change":"2024-09-10T21:18:17.470314+0000","last_active":"2024-09-10T21:18:17.471683+0000","last_peered":"2024-09-10T21:18:17.471683+0000","last_clean":"2024-09-10T21:18:17.471683+0000","last_became_active":"2024-09-10T21:18:17.469668+0000","last_became_peered":"2024-09-10T21:18:17.469668+0000","last_unstale":"2024-09-10T21:18:17.471683+0000","last_undegraded":"2024-09-10T21:18:17.471683+0000","last_fullsized":"2024-09-10T21:18:17.471683+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:40.103869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449875+0000","last_change":"2024-09-10T21:18:16.452036+0000","last_active":"2024-09-10T21:18:18.449875+0000","last_peered":"2024-09-10T21:18:18.449875+0000","last_clean":"2024-09-10T21:18:18.449875+0000","last_became_active":"2024-09-10T21:18:16.447891+0000","last_became_peered":"2024-09-10T21:18:16.447891+0000","last_unstale":"2024-09-10T21:18:18.449875+0000","last_undegraded":"2024-09-10T21:18:18.449875+0000","last_fullsized":"2024-09-10T21:18:18.449875+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:24.797913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450490+0000","last_change":"2024-09-10T21:18:17.456606+0000","last_active":"2024-09-10T21:18:18.450490+0000","last_peered":"2024-09-10T21:18:18.450490+0000","last_clean":"2024-09-10T21:18:18.450490+0000","last_became_active":"2024-09-10T21:18:17.455955+0000","last_became_peered":"2024-09-10T21:18:17.455955+0000","last_unstale":"2024-09-10T21:18:18.450490+0000","last_undegraded":"2024-09-10T21:18:18.450490+0000","last_fullsized":"2024-09-10T21:18:18.450490+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:18.632289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450619+0000","last_change":"2024-09-10T21:18:16.460481+0000","last_active":"2024-09-10T21:18:18.450619+0000","last_peered":"2024-09-10T21:18:18.450619+0000","last_clean":"2024-09-10T21:18:18.450619+0000","last_became_active":"2024-09-10T21:18:16.460171+0000","last_became_peered":"2024-09-10T21:18:16.460171+0000","last_unstale":"2024-09-10T21:18:18.450619+0000","last_undegraded":"2024-09-10T21:18:18.450619+0000","last_fullsized":"2024-09-10T21:18:18.450619+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:21:30.031820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450360+0000","last_change":"2024-09-10T21:18:17.467980+0000","last_active":"2024-09-10T21:18:18.450360+0000","last_peered":"2024-09-10T21:18:18.450360+0000","last_clean":"2024-09-10T21:18:18.450360+0000","last_became_active":"2024-09-10T21:18:17.467708+0000","last_became_peered":"2024-09-10T21:18:17.467708+0000","last_unstale":"2024-09-10T21:18:18.450360+0000","last_undegraded":"2024-09-10T21:18:18.450360+0000","last_fullsized":"2024-09-10T21:18:18.450360+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:33.671251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.451060+0000","last_change":"2024-09-10T21:18:16.454248+0000","last_active":"2024-09-10T21:18:17.451060+0000","last_peered":"2024-09-10T21:18:17.451060+0000","last_clean":"2024-09-10T21:18:17.451060+0000","last_became_active":"2024-09-10T21:18:16.453174+0000","last_became_peered":"2024-09-10T21:18:16.453174+0000","last_unstale":"2024-09-10T21:18:17.451060+0000","last_undegraded":"2024-09-10T21:18:17.451060+0000","last_fullsized":"2024-09-10T21:18:17.451060+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:55.505246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.463014+0000","last_change":"2024-09-10T21:18:17.459711+0000","last_active":"2024-09-10T21:18:17.463014+0000","last_peered":"2024-09-10T21:18:17.463014+0000","last_clean":"2024-09-10T21:18:17.463014+0000","last_became_active":"2024-09-10T21:18:17.459065+0000","last_became_peered":"2024-09-10T21:18:17.459065+0000","last_unstale":"2024-09-10T21:18:17.463014+0000","last_undegraded":"2024-09-10T21:18:17.463014+0000","last_fullsized":"2024-09-10T21:18:17.463014+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:44.043625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.440198+0000","last_change":"2024-09-10T21:18:16.446308+0000","last_active":"2024-09-10T21:18:17.440198+0000","last_peered":"2024-09-10T21:18:17.440198+0000","last_clean":"2024-09-10T21:18:17.440198+0000","last_became_active":"2024-09-10T21:18:16.445489+0000","last_became_peered":"2024-09-10T21:18:16.445489+0000","last_unstale":"2024-09-10T21:18:17.440198+0000","last_undegraded":"2024-09-10T21:18:17.440198+0000","last_fullsized":"2024-09-10T21:18:17.440198+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:10.399454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.460595+0000","last_change":"2024-09-10T21:18:17.454539+0000","last_active":"2024-09-10T21:18:17.460595+0000","last_peered":"2024-09-10T21:18:17.460595+0000","last_clean":"2024-09-10T21:18:17.460595+0000","last_became_active":"2024-09-10T21:18:17.454226+0000","last_became_peered":"2024-09-10T21:18:17.454226+0000","last_unstale":"2024-09-10T21:18:17.460595+0000","last_undegraded":"2024-09-10T21:18:17.460595+0000","last_fullsized":"2024-09-10T21:18:17.460595+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:28:26.311973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.439844+0000","last_change":"2024-09-10T21:18:16.447641+0000","last_active":"2024-09-10T21:18:17.439844+0000","last_peered":"2024-09-10T21:18:17.439844+0000","last_clean":"2024-09-10T21:18:17.439844+0000","last_became_active":"2024-09-10T21:18:16.447352+0000","last_became_peered":"2024-09-10T21:18:16.447352+0000","last_unstale":"2024-09-10T21:18:17.439844+0000","last_undegraded":"2024-09-10T21:18:17.439844+0000","last_fullsized":"2024-09-10T21:18:17.439844+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:11.041118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.468104+0000","last_change":"2024-09-10T21:18:17.462166+0000","last_active":"2024-09-10T21:18:17.468104+0000","last_peered":"2024-09-10T21:18:17.468104+0000","last_clean":"2024-09-10T21:18:17.468104+0000","last_became_active":"2024-09-10T21:18:17.460893+0000","last_became_peered":"2024-09-10T21:18:17.460893+0000","last_unstale":"2024-09-10T21:18:17.468104+0000","last_undegraded":"2024-09-10T21:18:17.468104+0000","last_fullsized":"2024-09-10T21:18:17.468104+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:28:26.311973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.448411+0000","last_change":"2024-09-10T21:18:16.455549+0000","last_active":"2024-09-10T21:18:17.448411+0000","last_peered":"2024-09-10T21:18:17.448411+0000","last_clean":"2024-09-10T21:18:17.448411+0000","last_became_active":"2024-09-10T21:18:16.455163+0000","last_became_peered":"2024-09-10T21:18:16.455163+0000","last_unstale":"2024-09-10T21:18:17.448411+0000","last_undegraded":"2024-09-10T21:18:17.448411+0000","last_fullsized":"2024-09-10T21:18:17.448411+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:30.556397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450030+0000","last_change":"2024-09-10T21:18:17.445068+0000","last_active":"2024-09-10T21:18:18.450030+0000","last_peered":"2024-09-10T21:18:18.450030+0000","last_clean":"2024-09-10T21:18:18.450030+0000","last_became_active":"2024-09-10T21:18:17.444775+0000","last_became_peered":"2024-09-10T21:18:17.444775+0000","last_unstale":"2024-09-10T21:18:18.450030+0000","last_undegraded":"2024-09-10T21:18:18.450030+0000","last_fullsized":"2024-09-10T21:18:18.450030+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:01.659784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.448215+0000","last_change":"2024-09-10T21:18:16.459506+0000","last_active":"2024-09-10T21:18:17.448215+0000","last_peered":"2024-09-10T21:18:17.448215+0000","last_clean":"2024-09-10T21:18:17.448215+0000","last_became_active":"2024-09-10T21:18:16.459180+0000","last_became_peered":"2024-09-10T21:18:16.459180+0000","last_unstale":"2024-09-10T21:18:17.448215+0000","last_undegraded":"2024-09-10T21:18:17.448215+0000","last_fullsized":"2024-09-10T21:18:17.448215+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:24.903044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.450260+0000","last_change":"2024-09-10T21:18:17.462548+0000","last_active":"2024-09-10T21:18:18.450260+0000","last_peered":"2024-09-10T21:18:18.450260+0000","last_clean":"2024-09-10T21:18:18.450260+0000","last_became_active":"2024-09-10T21:18:17.457883+0000","last_became_peered":"2024-09-10T21:18:17.457883+0000","last_unstale":"2024-09-10T21:18:18.450260+0000","last_undegraded":"2024-09-10T21:18:18.450260+0000","last_fullsized":"2024-09-10T21:18:18.450260+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:09.889770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.443049+0000","last_change":"2024-09-10T21:18:16.449310+0000","last_active":"2024-09-10T21:18:17.443049+0000","last_peered":"2024-09-10T21:18:17.443049+0000","last_clean":"2024-09-10T21:18:17.443049+0000","last_became_active":"2024-09-10T21:18:16.449019+0000","last_became_peered":"2024-09-10T21:18:16.449019+0000","last_unstale":"2024-09-10T21:18:17.443049+0000","last_undegraded":"2024-09-10T21:18:17.443049+0000","last_fullsized":"2024-09-10T21:18:17.443049+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:27.574906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.469097+0000","last_change":"2024-09-10T21:18:17.460556+0000","last_active":"2024-09-10T21:18:17.469097+0000","last_peered":"2024-09-10T21:18:17.469097+0000","last_clean":"2024-09-10T21:18:17.469097+0000","last_became_active":"2024-09-10T21:18:17.460249+0000","last_became_peered":"2024-09-10T21:18:17.460249+0000","last_unstale":"2024-09-10T21:18:17.469097+0000","last_undegraded":"2024-09-10T21:18:17.469097+0000","last_fullsized":"2024-09-10T21:18:17.469097+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:10.625775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T21:18:18.449235+0000","last_change":"2024-09-10T21:18:16.460962+0000","last_active":"2024-09-10T21:18:18.449235+0000","last_peered":"2024-09-10T21:18:18.449235+0000","last_clean":"2024-09-10T21:18:18.449235+0000","last_became_active":"2024-09-10T21:18:16.460678+0000","last_became_peered":"2024-09-10T21:18:16.460678+0000","last_unstale":"2024-09-10T21:18:18.449235+0000","last_undegraded":"2024-09-10T21:18:18.449235+0000","last_fullsized":"2024-09-10T21:18:18.449235+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:02.771696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.471218+0000","last_change":"2024-09-10T21:18:17.468458+0000","last_active":"2024-09-10T21:18:17.471218+0000","last_peered":"2024-09-10T21:18:17.471218+0000","last_clean":"2024-09-10T21:18:17.471218+0000","last_became_active":"2024-09-10T21:18:17.468182+0000","last_became_peered":"2024-09-10T21:18:17.468182+0000","last_unstale":"2024-09-10T21:18:17.471218+0000","last_undegraded":"2024-09-10T21:18:17.471218+0000","last_fullsized":"2024-09-10T21:18:17.471218+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:16.422597+0000","last_clean_scrub_stamp":"2024-09-10T21:18:16.422597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:22.073443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"126'3","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.476061+0000","last_change":"2024-09-10T21:18:16.450778+0000","last_active":"2024-09-10T21:18:17.476061+0000","last_peered":"2024-09-10T21:18:17.476061+0000","last_clean":"2024-09-10T21:18:17.476061+0000","last_became_active":"2024-09-10T21:18:16.450507+0000","last_became_peered":"2024-09-10T21:18:16.450507+0000","last_unstale":"2024-09-10T21:18:17.476061+0000","last_undegraded":"2024-09-10T21:18:17.476061+0000","last_fullsized":"2024-09-10T21:18:17.476061+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:15.419297+0000","last_clean_scrub_stamp":"2024-09-10T21:18:15.419297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:27.028438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":296,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-10T21:18:17.445103+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:18:17.445103+0000","last_peered":"2024-09-10T21:18:17.445103+0000","last_clean":"2024-09-10T21:18:17.445103+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:18:17.445103+0000","last_undegraded":"2024-09-10T21:18:17.445103+0000","last_fullsized":"2024-09-10T21:18:17.445103+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36888,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334952,"statfs":{"total":96636764160,"available":96598990848,"internally_reserved":0,"allocated":1572864,"data_stored":676865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509541,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37492,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334348,"statfs":{"total":96636764160,"available":96598372352,"internally_reserved":0,"allocated":2191360,"data_stored":1269039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509539,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1593344,"data_stored":679081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509538,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1581056,"data_stored":677481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":683187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509539,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1576960,"data_stored":676955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542243,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41032,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330808,"statfs":{"total":96636764160,"available":96594747392,"internally_reserved":0,"allocated":1560576,"data_stored":670090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542242,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37476,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334364,"statfs":{"total":96636764160,"available":96598388736,"internally_reserved":0,"allocated":2170880,"data_stored":1267303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:18:21.993 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:18:21.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:18:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 -- 172.21.15.16:0/3833450545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32006eb70 msgr2=0x7fe320165030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 --2- 172.21.15.16:0/3833450545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32006eb70 0x7fe320165030 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe31800acd0 tx=0x7fe31801dec0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 -- 172.21.15.16:0/3833450545 shutdown_connections 2024-09-10T21:18:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 --2- 172.21.15.16:0/3833450545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32006eb70 0x7fe320165030 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 -- 172.21.15.16:0/3833450545 >> 172.21.15.16:0/3833450545 conn(0x7fe3200cc4b0 msgr2=0x7fe3200cc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 -- 172.21.15.16:0/3833450545 shutdown_connections 2024-09-10T21:18:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 -- 172.21.15.16:0/3833450545 wait complete. 2024-09-10T21:18:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 Processor -- start 2024-09-10T21:18:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 -- start start 2024-09-10T21:18:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32006eb70 0x7fe32014f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe32015a6b0 0x7fe3200c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe32015b190 0x7fe32015b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe320167c90 con 0x7fe32006eb70 2024-09-10T21:18:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe320167b10 con 0x7fe32015a6b0 2024-09-10T21:18:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.184+0000 7fe324a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe320167e10 con 0x7fe32015b190 2024-09-10T21:18:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe31f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32006eb70 0x7fe32014f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe31f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32006eb70 0x7fe32014f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58494/0 (socket says 172.21.15.16:58494) 2024-09-10T21:18:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe31f400640 1 -- 172.21.15.16:0/324333245 learned_addr learned my addr 172.21.15.16:0/324333245 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe316a00640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe32015a6b0 0x7fe3200c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe31fe00640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe32015b190 0x7fe32015b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe31f400640 1 -- 172.21.15.16:0/324333245 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe32015b190 msgr2=0x7fe32015b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe31f400640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe32015b190 0x7fe32015b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe31f400640 1 -- 172.21.15.16:0/324333245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe32015a6b0 msgr2=0x7fe3200c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe31f400640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe32015a6b0 0x7fe3200c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe31f400640 1 -- 172.21.15.16:0/324333245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3201f5940 con 0x7fe32006eb70 2024-09-10T21:18:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe31f400640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32006eb70 0x7fe32014f5e0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe310001ec0 tx=0x7fe31000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe317e00640 1 -- 172.21.15.16:0/324333245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310017070 con 0x7fe32006eb70 2024-09-10T21:18:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe317e00640 1 -- 172.21.15.16:0/324333245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe310003600 con 0x7fe32006eb70 2024-09-10T21:18:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe324a00640 1 -- 172.21.15.16:0/324333245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3201f5c20 con 0x7fe32006eb70 2024-09-10T21:18:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe317e00640 1 -- 172.21.15.16:0/324333245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310003a50 con 0x7fe32006eb70 2024-09-10T21:18:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe324a00640 1 -- 172.21.15.16:0/324333245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3201f61e0 con 0x7fe32006eb70 2024-09-10T21:18:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe324a00640 1 -- 172.21.15.16:0/324333245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe32006e460 con 0x7fe32006eb70 2024-09-10T21:18:22.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.188+0000 7fe317e00640 1 -- 172.21.15.16:0/324333245 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe310003bf0 con 0x7fe32006eb70 2024-09-10T21:18:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.192+0000 7fe317e00640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe2fc0725e0 0x7fe2fc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.192+0000 7fe317e00640 1 -- 172.21.15.16:0/324333245 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7fe3100927a0 con 0x7fe32006eb70 2024-09-10T21:18:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.192+0000 7fe316a00640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe2fc0725e0 0x7fe2fc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:22.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.192+0000 7fe316a00640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe2fc0725e0 0x7fe2fc074aa0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe320164560 tx=0x7fe308001420 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:22.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.196+0000 7fe317e00640 1 -- 172.21.15.16:0/324333245 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe31005d820 con 0x7fe32006eb70 2024-09-10T21:18:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.436+0000 7fe324a00640 1 -- 172.21.15.16:0/324333245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe3200b2390 con 0x7fe32006eb70 2024-09-10T21:18:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.440+0000 7fe317e00640 1 -- 172.21.15.16:0/324333245 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v31) ==== 190+0+147 (secure 0 0 0) 0x7fe310062250 con 0x7fe32006eb70 2024-09-10T21:18:22.444 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-10T21:18:22.444 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA== 2024-09-10T21:18:22.444 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-10T21:18:22.444 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-10T21:18:22.444 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:18:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-10T21:18:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.440+0000 7fe324a00640 1 -- 172.21.15.16:0/324333245 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe2fc0725e0 msgr2=0x7fe2fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.440+0000 7fe324a00640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe2fc0725e0 0x7fe2fc074aa0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe320164560 tx=0x7fe308001420 comp rx=0 tx=0).stop 2024-09-10T21:18:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.440+0000 7fe324a00640 1 -- 172.21.15.16:0/324333245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32006eb70 msgr2=0x7fe32014f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.440+0000 7fe324a00640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32006eb70 0x7fe32014f5e0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe310001ec0 tx=0x7fe31000c890 comp rx=0 tx=0).stop 2024-09-10T21:18:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.440+0000 7fe324a00640 1 -- 172.21.15.16:0/324333245 shutdown_connections 2024-09-10T21:18:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.440+0000 7fe324a00640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe2fc0725e0 0x7fe2fc074aa0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.440+0000 7fe324a00640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe32015b190 0x7fe32015b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.440+0000 7fe324a00640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe32015a6b0 0x7fe3200c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.440+0000 7fe324a00640 1 --2- 172.21.15.16:0/324333245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe32006eb70 0x7fe32014f5e0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.440+0000 7fe324a00640 1 -- 172.21.15.16:0/324333245 >> 172.21.15.16:0/324333245 conn(0x7fe3200cc4b0 msgr2=0x7fe3200cb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.440+0000 7fe324a00640 1 -- 172.21.15.16:0/324333245 shutdown_connections 2024-09-10T21:18:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.440+0000 7fe324a00640 1 -- 172.21.15.16:0/324333245 wait complete. 2024-09-10T21:18:22.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:22.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 -- 172.21.15.16:0/3890703682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35000d0890 msgr2=0x7f35000b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:22.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 --2- 172.21.15.16:0/3890703682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35000d0890 0x7f35000b0020 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009c80 tx=0x7f34f801cd40 comp rx=0 tx=0).stop 2024-09-10T21:18:22.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 -- 172.21.15.16:0/3890703682 shutdown_connections 2024-09-10T21:18:22.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 --2- 172.21.15.16:0/3890703682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35000d0890 0x7f35000b0020 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 --2- 172.21.15.16:0/3890703682 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f35000b3200 0x7f35000afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 -- 172.21.15.16:0/3890703682 >> 172.21.15.16:0/3890703682 conn(0x7f35000d1d80 msgr2=0x7f35000d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:22.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 -- 172.21.15.16:0/3890703682 shutdown_connections 2024-09-10T21:18:22.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 -- 172.21.15.16:0/3890703682 wait complete. 2024-09-10T21:18:22.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 Processor -- start 2024-09-10T21:18:22.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 -- start start 2024-09-10T21:18:22.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f35000d0890 0x7f350006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:22.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350015f450 0x7f3500141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3500142980 0x7f3500142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3500168d50 con 0x7f3500142980 2024-09-10T21:18:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3500168bd0 con 0x7f350015f450 2024-09-10T21:18:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.608+0000 7f3505600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3500168ed0 con 0x7f35000d0890 2024-09-10T21:18:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34ff400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3500142980 0x7f3500142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34ff400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3500142980 0x7f3500142330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58512/0 (socket says 172.21.15.16:58512) 2024-09-10T21:18:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34ff400640 1 -- 172.21.15.16:0/4013501495 learned_addr learned my addr 172.21.15.16:0/4013501495 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34fea00640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f35000d0890 0x7f350006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34f6000640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350015f450 0x7f3500141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34ff400640 1 -- 172.21.15.16:0/4013501495 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f35000d0890 msgr2=0x7f350006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34ff400640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f35000d0890 0x7f350006ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34ff400640 1 -- 172.21.15.16:0/4013501495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350015f450 msgr2=0x7f3500141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:22.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34ff400640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350015f450 0x7f3500141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34ff400640 1 -- 172.21.15.16:0/4013501495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350006e7a0 con 0x7f3500142980 2024-09-10T21:18:22.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34f6000640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350015f450 0x7f3500141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:22.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34ff400640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3500142980 0x7f3500142330 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009bd0 tx=0x7f34f8009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:22.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34f7400640 1 -- 172.21.15.16:0/4013501495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f80bb070 con 0x7f3500142980 2024-09-10T21:18:22.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34f7400640 1 -- 172.21.15.16:0/4013501495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34f8004e00 con 0x7f3500142980 2024-09-10T21:18:22.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34f7400640 1 -- 172.21.15.16:0/4013501495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f80b2040 con 0x7f3500142980 2024-09-10T21:18:22.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f3505600640 1 -- 172.21.15.16:0/4013501495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35000cf670 con 0x7f3500142980 2024-09-10T21:18:22.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f3505600640 1 -- 172.21.15.16:0/4013501495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35000b4f30 con 0x7f3500142980 2024-09-10T21:18:22.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f3505600640 1 -- 172.21.15.16:0/4013501495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35000cfc80 con 0x7f3500142980 2024-09-10T21:18:22.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.612+0000 7f34f7400640 1 -- 172.21.15.16:0/4013501495 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f34f80b5030 con 0x7f3500142980 2024-09-10T21:18:22.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.616+0000 7f34f7400640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f34dc072610 0x7f34dc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:22.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.616+0000 7f34f7400640 1 -- 172.21.15.16:0/4013501495 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f34f812cde0 con 0x7f3500142980 2024-09-10T21:18:22.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.616+0000 7f34fea00640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f34dc072610 0x7f34dc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.616+0000 7f34fea00640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f34dc072610 0x7f34dc074ad0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f34f0007fd0 tx=0x7f34f0009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.620+0000 7f34f7400640 1 -- 172.21.15.16:0/4013501495 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f34f8016990 con 0x7f3500142980 2024-09-10T21:18:22.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.828+0000 7f3505600640 1 -- 172.21.15.16:0/4013501495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35000b1e40 con 0x7f3500142980 2024-09-10T21:18:22.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.828+0000 7f34f7400640 1 -- 172.21.15.16:0/4013501495 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5227 (secure 0 0 0) 0x7f34f80f7dd0 con 0x7f3500142980 2024-09-10T21:18:22.834 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:22.834 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"btime":"2024-09-10T21:18:20:455300+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3971491960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3971491960},{"type":"v1","addr":"172.21.15.52:6834","nonce":3971491960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7854,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4224090659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4224090659},{"type":"v1","addr":"172.21.15.16:6836","nonce":4224090659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1749071285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1749071285},{"type":"v1","addr":"172.21.15.16:6837","nonce":1749071285}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:16.852256+0000","modified":"2024-09-10T21:18:19.856561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/3672156966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3672156966},{"type":"v1","addr":"172.21.15.52:6835","nonce":3672156966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7869,"qdb_cluster":[7869]},"id":10}]} 2024-09-10T21:18:22.834 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 93 2024-09-10T21:18:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.832+0000 7f3505600640 1 -- 172.21.15.16:0/4013501495 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f34dc072610 msgr2=0x7f34dc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.832+0000 7f3505600640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f34dc072610 0x7f34dc074ad0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f34f0007fd0 tx=0x7f34f0009040 comp rx=0 tx=0).stop 2024-09-10T21:18:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.832+0000 7f3505600640 1 -- 172.21.15.16:0/4013501495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3500142980 msgr2=0x7f3500142330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.832+0000 7f3505600640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3500142980 0x7f3500142330 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009bd0 tx=0x7f34f8009390 comp rx=0 tx=0).stop 2024-09-10T21:18:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.832+0000 7f3505600640 1 -- 172.21.15.16:0/4013501495 shutdown_connections 2024-09-10T21:18:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.832+0000 7f3505600640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f34dc072610 0x7f34dc074ad0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.832+0000 7f3505600640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3500142980 0x7f3500142330 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.832+0000 7f3505600640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350015f450 0x7f3500141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.832+0000 7f3505600640 1 --2- 172.21.15.16:0/4013501495 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f35000d0890 0x7f350006ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.832+0000 7f3505600640 1 -- 172.21.15.16:0/4013501495 >> 172.21.15.16:0/4013501495 conn(0x7f35000d1d80 msgr2=0x7f3500152340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.832+0000 7f3505600640 1 -- 172.21.15.16:0/4013501495 shutdown_connections 2024-09-10T21:18:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:22.832+0000 7f3505600640 1 -- 172.21.15.16:0/4013501495 wait complete. 2024-09-10T21:18:23.258 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 93, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:18:16.852256+0000', 'modified': '2024-09-10T21:18:19.856561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7869}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7869': {'gid': 7869, 'name': 'b', 'rank': 0, 'incarnation': 89, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.52:6835/3672156966', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6833', 'nonce': 3672156966}, {'type': 'v1', 'addr': '172.21.15.52:6835', 'nonce': 3672156966}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7869, 'qdb_cluster': [7869]} 2024-09-10T21:18:23.258 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:18:23.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T21:18:23.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.408+0000 7f0d83600640 1 -- 172.21.15.16:0/986453689 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0d7c0ce450 msgr2=0x7f0d7c0ce870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.408+0000 7f0d83600640 1 --2- 172.21.15.16:0/986453689 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0d7c0ce450 0x7f0d7c0ce870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009900 tx=0x7f0d6801c7c0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.408+0000 7f0d83600640 1 -- 172.21.15.16:0/986453689 shutdown_connections 2024-09-10T21:18:23.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.408+0000 7f0d83600640 1 --2- 172.21.15.16:0/986453689 >> v2:172.21.15.52:3300/0 conn(0x7f0d7c0ca170 0x7f0d7c0ca5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.408+0000 7f0d83600640 1 --2- 172.21.15.16:0/986453689 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0d7c0ce450 0x7f0d7c0ce870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.408+0000 7f0d83600640 1 -- 172.21.15.16:0/986453689 >> 172.21.15.16:0/986453689 conn(0x7f0d7c0cfb20 msgr2=0x7f0d7c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:23.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.408+0000 7f0d83600640 1 -- 172.21.15.16:0/986453689 shutdown_connections 2024-09-10T21:18:23.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.408+0000 7f0d83600640 1 -- 172.21.15.16:0/986453689 wait complete. 2024-09-10T21:18:23.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.408+0000 7f0d83600640 1 Processor -- start 2024-09-10T21:18:23.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.408+0000 7f0d83600640 1 -- start start 2024-09-10T21:18:23.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d83600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0ca170 0x7f0d7c1594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d83600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c0ce450 0x7f0d7c157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d83600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0d7c15a270 0x7f0d7c150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d83600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c072af0 con 0x7f0d7c0ca170 2024-09-10T21:18:23.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d83600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c072970 con 0x7f0d7c0ce450 2024-09-10T21:18:23.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d83600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0d7c072c70 con 0x7f0d7c15a270 2024-09-10T21:18:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d80c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0ca170 0x7f0d7c1594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d80c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0ca170 0x7f0d7c1594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58520/0 (socket says 172.21.15.16:58520) 2024-09-10T21:18:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d80c00640 1 -- 172.21.15.16:0/2997674085 learned_addr learned my addr 172.21.15.16:0/2997674085 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d80c00640 1 -- 172.21.15.16:0/2997674085 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0d7c15a270 msgr2=0x7f0d7c150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d80c00640 1 --2- 172.21.15.16:0/2997674085 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0d7c15a270 0x7f0d7c150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d80c00640 1 -- 172.21.15.16:0/2997674085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c0ce450 msgr2=0x7f0d7c157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d80c00640 1 --2- 172.21.15.16:0/2997674085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c0ce450 0x7f0d7c157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d80c00640 1 -- 172.21.15.16:0/2997674085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c0b1290 con 0x7f0d7c0ca170 2024-09-10T21:18:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d80c00640 1 --2- 172.21.15.16:0/2997674085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0ca170 0x7f0d7c1594f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000c3c0 tx=0x7f0d7000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d78c00640 1 -- 172.21.15.16:0/2997674085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700052d0 con 0x7f0d7c0ca170 2024-09-10T21:18:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d78c00640 1 -- 172.21.15.16:0/2997674085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d70007020 con 0x7f0d7c0ca170 2024-09-10T21:18:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d78c00640 1 -- 172.21.15.16:0/2997674085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70007470 con 0x7f0d7c0ca170 2024-09-10T21:18:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d83600640 1 -- 172.21.15.16:0/2997674085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7c158320 con 0x7f0d7c0ca170 2024-09-10T21:18:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.412+0000 7f0d83600640 1 -- 172.21.15.16:0/2997674085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7c1588e0 con 0x7f0d7c0ca170 2024-09-10T21:18:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d78c00640 1 -- 172.21.15.16:0/2997674085 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0d700067e0 con 0x7f0d7c0ca170 2024-09-10T21:18:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d78c00640 1 --2- 172.21.15.16:0/2997674085 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0d5c0725e0 0x7f0d5c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d86a77000 1 Processor -- start 2024-09-10T21:18:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d78c00640 1 -- 172.21.15.16:0/2997674085 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f0d700926a0 con 0x7f0d7c0ca170 2024-09-10T21:18:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d7be00640 1 --2- 172.21.15.16:0/2997674085 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0d5c0725e0 0x7f0d5c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d86a77000 1 -- start start 2024-09-10T21:18:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d7be00640 1 --2- 172.21.15.16:0/2997674085 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0d5c0725e0 0x7f0d5c074aa0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0d680af000 tx=0x7f0d68019560 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d86a77000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b07bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d86a77000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 0x556d1b02f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d86a77000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0301f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d86a77000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556d1af503f0 con 0x556d1b07b9d0 2024-09-10T21:18:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d86a77000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556d1ae8db20 con 0x556d1b02fe10 2024-09-10T21:18:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d86a77000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x556d1ae7cf50 con 0x556d1b07c2f0 2024-09-10T21:18:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d80c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b07bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d80c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b07bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58524/0 (socket says 172.21.15.16:58524) 2024-09-10T21:18:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d80c00640 1 -- 172.21.15.16:0/2087317354 learned_addr learned my addr 172.21.15.16:0/2087317354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d81600640 1 --2- 172.21.15.16:0/2087317354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0301f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d7be00640 1 --2- 172.21.15.16:0/2087317354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 0x556d1b02f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d80c00640 1 -- 172.21.15.16:0/2087317354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 msgr2=0x556d1b02f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d80c00640 1 --2- 172.21.15.16:0/2087317354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 0x556d1b02f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d80c00640 1 -- 172.21.15.16:0/2087317354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 msgr2=0x556d1b0301f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d80c00640 1 --2- 172.21.15.16:0/2087317354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0301f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d80c00640 1 -- 172.21.15.16:0/2087317354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d1ae44590 con 0x556d1b07b9d0 2024-09-10T21:18:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d81600640 1 --2- 172.21.15.16:0/2087317354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0301f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d80c00640 1 --2- 172.21.15.16:0/2087317354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b07bdb0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f0d7005d370 tx=0x7f0d70000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576caae1e0dc6b9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.416+0000 7f0d6d600640 1 -- 172.21.15.16:0/2087317354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70062150 con 0x556d1b07b9d0 2024-09-10T21:18:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d6d600640 1 -- 172.21.15.16:0/2087317354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d70066b70 con 0x556d1b07b9d0 2024-09-10T21:18:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d6d600640 1 -- 172.21.15.16:0/2087317354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70064380 con 0x556d1b07b9d0 2024-09-10T21:18:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 -- 172.21.15.16:0/2087317354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 msgr2=0x556d1b07bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 --2- 172.21.15.16:0/2087317354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b07bdb0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f0d7005d370 tx=0x7f0d70000bd0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 -- 172.21.15.16:0/2087317354 shutdown_connections 2024-09-10T21:18:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 --2- 172.21.15.16:0/2087317354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0301f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 --2- 172.21.15.16:0/2087317354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 0x556d1b02f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 --2- 172.21.15.16:0/2087317354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b07bdb0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 -- 172.21.15.16:0/2087317354 >> 172.21.15.16:0/2087317354 conn(0x556d1af19130 msgr2=0x556d1b05ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 -- 172.21.15.16:0/2087317354 shutdown_connections 2024-09-10T21:18:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 -- 172.21.15.16:0/2087317354 wait complete. 2024-09-10T21:18:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 -- start start 2024-09-10T21:18:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 Processor -- start 2024-09-10T21:18:23.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b02fe10 0x556d1b05c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b05ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b07c2f0 0x556d1b05d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556d1ae175d0 con 0x556d1b07b9d0 2024-09-10T21:18:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556d1ae80d60 con 0x556d1b07c2f0 2024-09-10T21:18:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x556d1ae23500 con 0x556d1b02fe10 2024-09-10T21:18:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d86a77000 10 client.0 _ll_register_callbacks cb 0x556d1ae27810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b05ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b05ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58540/0 (socket says 172.21.15.16:58540) 2024-09-10T21:18:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d7be00640 1 -- 172.21.15.16:0/3179984518 learned_addr learned my addr 172.21.15.16:0/3179984518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d81600640 1 --2- 172.21.15.16:0/3179984518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b07c2f0 0x556d1b05d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d80c00640 1 --2- 172.21.15.16:0/3179984518 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b02fe10 0x556d1b05c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d7be00640 1 -- 172.21.15.16:0/3179984518 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b02fe10 msgr2=0x556d1b05c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d7be00640 1 --2- 172.21.15.16:0/3179984518 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b02fe10 0x556d1b05c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d7be00640 1 -- 172.21.15.16:0/3179984518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b07c2f0 msgr2=0x556d1b05d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d7be00640 1 --2- 172.21.15.16:0/3179984518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b07c2f0 0x556d1b05d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d7be00640 1 -- 172.21.15.16:0/3179984518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d1ae1b170 con 0x556d1b07b9d0 2024-09-10T21:18:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d7be00640 1 --2- 172.21.15.16:0/3179984518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b05ce70 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f0d6801a3f0 tx=0x7f0d68009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b853dab9b12d3208 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d62000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:18:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.420+0000 7f0d62000640 1 -- 172.21.15.16:0/3179984518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680046d0 con 0x556d1b07b9d0 2024-09-10T21:18:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 -- 172.21.15.16:0/3179984518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d1ae150f0 con 0x556d1b07b9d0 2024-09-10T21:18:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d62000640 1 -- 172.21.15.16:0/3179984518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f0d68004870 con 0x556d1b07b9d0 2024-09-10T21:18:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d62000640 1 -- 172.21.15.16:0/3179984518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f0d680d1050 con 0x556d1b07b9d0 2024-09-10T21:18:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 10 client.7989 fetch_fsmap learned FSMap version 93 2024-09-10T21:18:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 -- 172.21.15.16:0/3179984518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) -- 0x556d1af01ef0 con 0x556d1b07b9d0 2024-09-10T21:18:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d62000640 1 -- 172.21.15.16:0/3179984518 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x556d1ae150f0 con 0x556d1b07b9d0 2024-09-10T21:18:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d62000640 1 -- 172.21.15.16:0/3179984518 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 93) ==== 3502+0+0 (secure 0 0 0) 0x7f0d680d4020 con 0x556d1b07b9d0 2024-09-10T21:18:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 10 client.7989 fetch_fsmap finished waiting for FSMap version 93 2024-09-10T21:18:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 10 client.7989 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:18:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 20 client.7989 populate_metadata read hostname 'smithi016' 2024-09-10T21:18:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 --2- 172.21.15.16:0/3179984518 >> [v2:172.21.15.52:6833/3672156966,v1:172.21.15.52:6835/3672156966] conn(0x556d1b05e0f0 0x556d1b08dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 4 client.7989 mds_command: new command op to 7869 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:18:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 -- 172.21.15.16:0/3179984518 --> [v2:172.21.15.52:6833/3672156966,v1:172.21.15.52:6835/3672156966] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d1ae8fa70 con 0x556d1b05e0f0 2024-09-10T21:18:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d80c00640 1 --2- 172.21.15.16:0/3179984518 >> [v2:172.21.15.52:6833/3672156966,v1:172.21.15.52:6835/3672156966] conn(0x556d1b05e0f0 0x556d1b08dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d80c00640 1 --2- 172.21.15.16:0/3179984518 >> [v2:172.21.15.52:6833/3672156966,v1:172.21.15.52:6835/3672156966] conn(0x556d1b05e0f0 0x556d1b08dc90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f8fed78263423e server_cookie=5532436a59085a46 in_seq=0 out_seq=0 2024-09-10T21:18:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d62000640 10 client.7989 ms_handle_connect on v2:172.21.15.52:6833/3672156966 2024-09-10T21:18:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d62000640 1 -- 172.21.15.16:0/3179984518 <== mds.0 v2:172.21.15.52:6833/3672156966 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556d1ae8fa70 con 0x556d1b05e0f0 2024-09-10T21:18:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d62000640 10 client.7989 handle_command_reply: tid=0 2024-09-10T21:18:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 client.7989 shutdown 2024-09-10T21:18:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 -- 172.21.15.16:0/3179984518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 msgr2=0x556d1b05ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 --2- 172.21.15.16:0/3179984518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b05ce70 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f0d6801a3f0 tx=0x7f0d68009260 comp rx=0 tx=0).stop 2024-09-10T21:18:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 -- 172.21.15.16:0/3179984518 shutdown_connections 2024-09-10T21:18:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 --2- 172.21.15.16:0/3179984518 >> [v2:172.21.15.52:6833/3672156966,v1:172.21.15.52:6835/3672156966] conn(0x556d1b05e0f0 0x556d1b08dc90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 --2- 172.21.15.16:0/3179984518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b07c2f0 0x556d1b05d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 --2- 172.21.15.16:0/3179984518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b05ce70 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 --2- 172.21.15.16:0/3179984518 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b02fe10 0x556d1b05c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d62000640 0 client.7989 ms_handle_reset on v2:172.21.15.52:6833/3672156966 2024-09-10T21:18:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 -- 172.21.15.16:0/3179984518 >> 172.21.15.16:0/3179984518 conn(0x556d1af19130 msgr2=0x556d1b02efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 -- 172.21.15.16:0/3179984518 shutdown_connections 2024-09-10T21:18:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 1 -- 172.21.15.16:0/3179984518 wait complete. 2024-09-10T21:18:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.424+0000 7f0d86a77000 20 client.7989 trim_cache size 0 max 16384 2024-09-10T21:18:23.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d86a77000 1 Processor -- start 2024-09-10T21:18:23.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d86a77000 1 -- start start 2024-09-10T21:18:23.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d86a77000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0edfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d86a77000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b0ee520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d86a77000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 0x556d1b0eea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d86a77000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c203ea0 con 0x556d1b07b9d0 2024-09-10T21:18:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d86a77000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c156c70 con 0x556d1b02fe10 2024-09-10T21:18:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d86a77000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0d7c1fbae0 con 0x556d1b07c2f0 2024-09-10T21:18:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b0ee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b0ee520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58542/0 (socket says 172.21.15.16:58542) 2024-09-10T21:18:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d7be00640 1 -- 172.21.15.16:0/68271676 learned_addr learned my addr 172.21.15.16:0/68271676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d80c00640 1 --2- 172.21.15.16:0/68271676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0edfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d81600640 1 --2- 172.21.15.16:0/68271676 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 0x556d1b0eea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d7be00640 1 -- 172.21.15.16:0/68271676 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 msgr2=0x556d1b0eea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:23.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d7be00640 1 --2- 172.21.15.16:0/68271676 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 0x556d1b0eea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d7be00640 1 -- 172.21.15.16:0/68271676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 msgr2=0x556d1b0edfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:23.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d7be00640 1 --2- 172.21.15.16:0/68271676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0edfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.444+0000 7f0d7be00640 1 -- 172.21.15.16:0/68271676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d1aed8d90 con 0x556d1b07b9d0 2024-09-10T21:18:23.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d7be00640 1 --2- 172.21.15.16:0/68271676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b0ee520 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f0d68000c10 tx=0x7f0d68004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a98e7452e8c29ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:23.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d6cc00640 1 -- 172.21.15.16:0/68271676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680046d0 con 0x556d1b07b9d0 2024-09-10T21:18:23.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d6cc00640 1 -- 172.21.15.16:0/68271676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d680c9070 con 0x556d1b07b9d0 2024-09-10T21:18:23.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d6cc00640 1 -- 172.21.15.16:0/68271676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68003860 con 0x556d1b07b9d0 2024-09-10T21:18:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 -- 172.21.15.16:0/68271676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 msgr2=0x556d1b0ee520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 --2- 172.21.15.16:0/68271676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b0ee520 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f0d68000c10 tx=0x7f0d68004fa0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 -- 172.21.15.16:0/68271676 shutdown_connections 2024-09-10T21:18:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 --2- 172.21.15.16:0/68271676 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 0x556d1b0eea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 --2- 172.21.15.16:0/68271676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b0ee520 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 --2- 172.21.15.16:0/68271676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0edfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 -- 172.21.15.16:0/68271676 >> 172.21.15.16:0/68271676 conn(0x556d1af19130 msgr2=0x556d1b037a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 -- 172.21.15.16:0/68271676 shutdown_connections 2024-09-10T21:18:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 -- 172.21.15.16:0/68271676 wait complete. 2024-09-10T21:18:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 -- start start 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 Processor -- start 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0f2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b0f2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 0x556d1b0f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c156620 con 0x556d1b07b9d0 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556d1ae175d0 con 0x556d1b02fe10 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x556d1b092ce0 con 0x556d1b07c2f0 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d86a77000 10 client.0 _ll_register_callbacks cb 0x7f0d740093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b0f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b0f2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58546/0 (socket says 172.21.15.16:58546) 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d7be00640 1 -- 172.21.15.16:0/2019336907 learned_addr learned my addr 172.21.15.16:0/2019336907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d80c00640 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d81600640 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 0x556d1b0f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d7be00640 1 -- 172.21.15.16:0/2019336907 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 msgr2=0x556d1b0f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d7be00640 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 0x556d1b0f2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d7be00640 1 -- 172.21.15.16:0/2019336907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 msgr2=0x556d1b0f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d7be00640 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0f2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d7be00640 1 -- 172.21.15.16:0/2019336907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d1ae43a90 con 0x556d1b07b9d0 2024-09-10T21:18:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d80c00640 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d7be00640 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b0f2770 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009900 tx=0x7f0d68004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae24df73353b177 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d62000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:18:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.448+0000 7f0d62000640 1 -- 172.21.15.16:0/2019336907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68003800 con 0x556d1b07b9d0 2024-09-10T21:18:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d62000640 1 -- 172.21.15.16:0/2019336907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f0d680039a0 con 0x556d1b07b9d0 2024-09-10T21:18:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 -- 172.21.15.16:0/2019336907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d680c9070 con 0x556d1b07b9d0 2024-09-10T21:18:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d62000640 1 -- 172.21.15.16:0/2019336907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f0d68003ba0 con 0x556d1b07b9d0 2024-09-10T21:18:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 10 client.7995 fetch_fsmap learned FSMap version 93 2024-09-10T21:18:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 -- 172.21.15.16:0/2019336907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) -- 0x556d1ae22050 con 0x556d1b07b9d0 2024-09-10T21:18:23.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d62000640 1 -- 172.21.15.16:0/2019336907 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d680c9070 con 0x556d1b07b9d0 2024-09-10T21:18:23.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d62000640 1 -- 172.21.15.16:0/2019336907 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 93) ==== 3502+0+0 (secure 0 0 0) 0x7f0d680d4070 con 0x556d1b07b9d0 2024-09-10T21:18:23.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 10 client.7995 fetch_fsmap finished waiting for FSMap version 93 2024-09-10T21:18:23.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 10 client.7995 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:18:23.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 20 client.7995 populate_metadata read hostname 'smithi016' 2024-09-10T21:18:23.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.52:6833/3672156966,v1:172.21.15.52:6835/3672156966] conn(0x556d1b090330 0x556d1b090710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 4 client.7995 mds_command: new command op to 7869 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:18:23.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 -- 172.21.15.16:0/2019336907 --> [v2:172.21.15.52:6833/3672156966,v1:172.21.15.52:6835/3672156966] -- command(tid 0: {"prefix": "status"}) -- 0x556d1ae8b520 con 0x556d1b090330 2024-09-10T21:18:23.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d80c00640 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.52:6833/3672156966,v1:172.21.15.52:6835/3672156966] conn(0x556d1b090330 0x556d1b090710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d80c00640 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.52:6833/3672156966,v1:172.21.15.52:6835/3672156966] conn(0x556d1b090330 0x556d1b090710 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae81e070e76a046 server_cookie=1fd78071fb05d944 in_seq=0 out_seq=0 2024-09-10T21:18:23.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d62000640 10 client.7995 ms_handle_connect on v2:172.21.15.52:6833/3672156966 2024-09-10T21:18:23.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d62000640 1 -- 172.21.15.16:0/2019336907 <== mds.0 v2:172.21.15.52:6833/3672156966 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556d1ae8b520 con 0x556d1b090330 2024-09-10T21:18:23.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d62000640 10 client.7995 handle_command_reply: tid=0 2024-09-10T21:18:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 client.7995 shutdown 2024-09-10T21:18:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 -- 172.21.15.16:0/2019336907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 msgr2=0x556d1b0f2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b0f2770 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009900 tx=0x7f0d68004c50 comp rx=0 tx=0).stop 2024-09-10T21:18:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 -- 172.21.15.16:0/2019336907 shutdown_connections 2024-09-10T21:18:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.52:6833/3672156966,v1:172.21.15.52:6835/3672156966] conn(0x556d1b090330 0x556d1b090710 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x556d1b07c2f0 0x556d1b0f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d1b07b9d0 0x556d1b0f2770 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 --2- 172.21.15.16:0/2019336907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d1b02fe10 0x556d1b0f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d62000640 0 client.7995 ms_handle_reset on v2:172.21.15.52:6833/3672156966 2024-09-10T21:18:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 -- 172.21.15.16:0/2019336907 >> 172.21.15.16:0/2019336907 conn(0x556d1af19130 msgr2=0x556d1b07a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 -- 172.21.15.16:0/2019336907 shutdown_connections 2024-09-10T21:18:23.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 1 -- 172.21.15.16:0/2019336907 wait complete. 2024-09-10T21:18:23.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d86a77000 20 client.7995 trim_cache size 0 max 16384 2024-09-10T21:18:23.463 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:18:23.463 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "eb19be60-6d8f-4816-b778-3d7d6698304d", 2024-09-10T21:18:23.463 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-10T21:18:23.463 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7869, 2024-09-10T21:18:23.463 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-10T21:18:23.463 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-10T21:18:23.464 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-10T21:18:23.464 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.0232571889999997, 2024-09-10T21:18:23.464 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 93, 2024-09-10T21:18:23.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 127, 2024-09-10T21:18:23.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 127, 2024-09-10T21:18:23.464 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 10.496401275 2024-09-10T21:18:23.464 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:18:23.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d83600640 1 -- 172.21.15.16:0/2997674085 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0d5c0725e0 msgr2=0x7f0d5c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d83600640 1 --2- 172.21.15.16:0/2997674085 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0d5c0725e0 0x7f0d5c074aa0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0d680af000 tx=0x7f0d68019560 comp rx=0 tx=0).stop 2024-09-10T21:18:23.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d83600640 1 -- 172.21.15.16:0/2997674085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0ca170 msgr2=0x7f0d7c1594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d83600640 1 --2- 172.21.15.16:0/2997674085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0ca170 0x7f0d7c1594f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000c3c0 tx=0x7f0d7000c880 comp rx=0 tx=0).stop 2024-09-10T21:18:23.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d83600640 1 -- 172.21.15.16:0/2997674085 shutdown_connections 2024-09-10T21:18:23.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d83600640 1 --2- 172.21.15.16:0/2997674085 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0d5c0725e0 0x7f0d5c074aa0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d83600640 1 --2- 172.21.15.16:0/2997674085 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0d7c15a270 0x7f0d7c150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d83600640 1 --2- 172.21.15.16:0/2997674085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d7c0ce450 0x7f0d7c157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d83600640 1 --2- 172.21.15.16:0/2997674085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0ca170 0x7f0d7c1594f0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d83600640 1 -- 172.21.15.16:0/2997674085 >> 172.21.15.16:0/2997674085 conn(0x7f0d7c0cfb20 msgr2=0x7f0d7c0737b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:23.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d83600640 1 -- 172.21.15.16:0/2997674085 shutdown_connections 2024-09-10T21:18:23.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.452+0000 7f0d83600640 1 -- 172.21.15.16:0/2997674085 wait complete. 2024-09-10T21:18:23.478 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:18:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.620+0000 7f7d9c000640 1 -- 172.21.15.16:0/3084195969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d940b13c0 msgr2=0x7f7d94071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.620+0000 7f7d9c000640 1 --2- 172.21.15.16:0/3084195969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d940b13c0 0x7f7d94071520 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f7d94073290 tx=0x7f7d8001c7a0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.620+0000 7f7d9c000640 1 -- 172.21.15.16:0/3084195969 shutdown_connections 2024-09-10T21:18:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.620+0000 7f7d9c000640 1 --2- 172.21.15.16:0/3084195969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d940b13c0 0x7f7d94071520 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.620+0000 7f7d9c000640 1 --2- 172.21.15.16:0/3084195969 >> v2:172.21.15.52:3300/0 conn(0x7f7d9415f400 0x7f7d94070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.620+0000 7f7d9c000640 1 -- 172.21.15.16:0/3084195969 >> 172.21.15.16:0/3084195969 conn(0x7f7d9409b130 msgr2=0x7f7d940d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.620+0000 7f7d9c000640 1 -- 172.21.15.16:0/3084195969 shutdown_connections 2024-09-10T21:18:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.620+0000 7f7d9c000640 1 -- 172.21.15.16:0/3084195969 wait complete. 2024-09-10T21:18:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.620+0000 7f7d9c000640 1 Processor -- start 2024-09-10T21:18:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.620+0000 7f7d9c000640 1 -- start start 2024-09-10T21:18:23.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d9c000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d940b13c0 0x7f7d9415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d9c000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d940cfc80 0x7f7d941648a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d9c000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d9415f400 0x7f7d940cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d9c000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d941695f0 con 0x7f7d940cfc80 2024-09-10T21:18:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d9c000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7d940d1270 con 0x7f7d940b13c0 2024-09-10T21:18:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d9c000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7d94169770 con 0x7f7d9415f400 2024-09-10T21:18:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d98c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d940cfc80 0x7f7d941648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d98c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d940cfc80 0x7f7d941648a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58564/0 (socket says 172.21.15.16:58564) 2024-09-10T21:18:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d98c00640 1 -- 172.21.15.16:0/3344081235 learned_addr learned my addr 172.21.15.16:0/3344081235 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d99600640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d940b13c0 0x7f7d9415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d9a000640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d9415f400 0x7f7d940cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d98c00640 1 -- 172.21.15.16:0/3344081235 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d9415f400 msgr2=0x7f7d940cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d98c00640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d9415f400 0x7f7d940cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d98c00640 1 -- 172.21.15.16:0/3344081235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d940b13c0 msgr2=0x7f7d9415a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d98c00640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d940b13c0 0x7f7d9415a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d98c00640 1 -- 172.21.15.16:0/3344081235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d941660b0 con 0x7f7d940cfc80 2024-09-10T21:18:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d98c00640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d940cfc80 0x7f7d941648a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f7d800b0040 tx=0x7f7d800a55f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d91600640 1 -- 172.21.15.16:0/3344081235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d800b2040 con 0x7f7d940cfc80 2024-09-10T21:18:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d91600640 1 -- 172.21.15.16:0/3344081235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d80004490 con 0x7f7d940cfc80 2024-09-10T21:18:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d91600640 1 -- 172.21.15.16:0/3344081235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80005280 con 0x7f7d940cfc80 2024-09-10T21:18:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d9c000640 1 -- 172.21.15.16:0/3344081235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d94166390 con 0x7f7d940cfc80 2024-09-10T21:18:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d9c000640 1 -- 172.21.15.16:0/3344081235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9415ae20 con 0x7f7d940cfc80 2024-09-10T21:18:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d9c000640 1 -- 172.21.15.16:0/3344081235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d940d00d0 con 0x7f7d940cfc80 2024-09-10T21:18:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.624+0000 7f7d91600640 1 -- 172.21.15.16:0/3344081235 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7d80005420 con 0x7f7d940cfc80 2024-09-10T21:18:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.628+0000 7f7d91600640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7d740725e0 0x7f7d74074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.628+0000 7f7d91600640 1 -- 172.21.15.16:0/3344081235 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f7d8012ccd0 con 0x7f7d940cfc80 2024-09-10T21:18:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.628+0000 7f7d99600640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7d740725e0 0x7f7d74074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.628+0000 7f7d99600640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7d740725e0 0x7f7d74074aa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7d88005fd0 tx=0x7f7d8800b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.632+0000 7f7d91600640 1 -- 172.21.15.16:0/3344081235 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7d800164b0 con 0x7f7d940cfc80 2024-09-10T21:18:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d9c000640 1 -- 172.21.15.16:0/3344081235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7d94166560 con 0x7f7d940cfc80 2024-09-10T21:18:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d91600640 1 -- 172.21.15.16:0/3344081235 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v93) ==== 53+0+73 (secure 0 0 0) 0x7f7d800f8320 con 0x7f7d940cfc80 2024-09-10T21:18:23.848 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:18:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d9c000640 1 -- 172.21.15.16:0/3344081235 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7d740725e0 msgr2=0x7f7d74074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d9c000640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7d740725e0 0x7f7d74074aa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7d88005fd0 tx=0x7f7d8800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d9c000640 1 -- 172.21.15.16:0/3344081235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d940cfc80 msgr2=0x7f7d941648a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d9c000640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d940cfc80 0x7f7d941648a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f7d800b0040 tx=0x7f7d800a55f0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d9c000640 1 -- 172.21.15.16:0/3344081235 shutdown_connections 2024-09-10T21:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d9c000640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f7d740725e0 0x7f7d74074aa0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d9c000640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7d9415f400 0x7f7d940cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d9c000640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d940cfc80 0x7f7d941648a0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d9c000640 1 --2- 172.21.15.16:0/3344081235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d940b13c0 0x7f7d9415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d9c000640 1 -- 172.21.15.16:0/3344081235 >> 172.21.15.16:0/3344081235 conn(0x7f7d9409b130 msgr2=0x7f7d9406dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d9c000640 1 -- 172.21.15.16:0/3344081235 shutdown_connections 2024-09-10T21:18:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:23.844+0000 7f7d9c000640 1 -- 172.21.15.16:0/3344081235 wait complete. 2024-09-10T21:18:23.869 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:18:23.870 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:18:23.870 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-10T21:18:23.870 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:18:23.870 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:18:23.870 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:18:23.870 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:18:23.870 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:18:23.870 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:18:23.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:23.871 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-10T21:18:23.880 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:18:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:18:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:18:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:18:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:23.881 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:18:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:18:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-10T21:18:23.881 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:18:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:18:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-10T21:18:23.881 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:18:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:18:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-10T21:18:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:18:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41946sec preferred_lft 41946sec 2024-09-10T21:18:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-10T21:18:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:23.882 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:18:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:18:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-10T21:18:23.882 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:18:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:30:fb:74:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:18:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:18:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:23.883 INFO:teuthology.orchestra.run.smithi016.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:18:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:18:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:18:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:18:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:23.883 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:18:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:18:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:18:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:23.883 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:18:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:18:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:18:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:23.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:23.884 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:18:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-10T21:18:23.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:23.943 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-10T21:18:23.993 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-10T21:18:23.993 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:18:23.993 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:18:23.993 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:18:23.993 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:18:23.993 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:18:23.993 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:18:23.993 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:18:23.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:18:23.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:18:23.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:18:23.994 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:18:24.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:18:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:18:24.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:24.131 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:18:24.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:18:24.179 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:18:24.184 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:18:24.184 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:18:24.185 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:18:24.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:24.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-10T21:18:24.290 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:18:24.291 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:18:24.343 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:18:24.344 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:18:24.344 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:18:24.394 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:18:24.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:24.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:18:24.490 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:18:24.507 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:18:24.511 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:18:24.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:24.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:18:24.531 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:18:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:18:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:18:24.646 INFO:teuthology.orchestra.run.smithi016.stdout:client7414 2024-09-10T21:18:24.646 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:18:24.646 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:24.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:18:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.844+0000 7f161fe00640 1 -- 172.21.15.16:0/3738662492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1604004810 msgr2=0x7f1604004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.844+0000 7f161fe00640 1 --2- 172.21.15.16:0/3738662492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1604004810 0x7f1604004c90 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f1608006fa0 tx=0x7f160801cea0 comp rx=0 tx=0).stop 2024-09-10T21:18:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.844+0000 7f161fe00640 1 -- 172.21.15.16:0/3738662492 shutdown_connections 2024-09-10T21:18:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.844+0000 7f161fe00640 1 --2- 172.21.15.16:0/3738662492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1604004810 0x7f1604004c90 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:24.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.844+0000 7f161fe00640 1 -- 172.21.15.16:0/3738662492 >> 172.21.15.16:0/3738662492 conn(0x7f16200d1d00 msgr2=0x7f16200d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:24.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.844+0000 7f161fe00640 1 -- 172.21.15.16:0/3738662492 shutdown_connections 2024-09-10T21:18:24.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.844+0000 7f161fe00640 1 -- 172.21.15.16:0/3738662492 wait complete. 2024-09-10T21:18:24.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.844+0000 7f161fe00640 1 Processor -- start 2024-09-10T21:18:24.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.844+0000 7f161fe00640 1 -- start start 2024-09-10T21:18:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161fe00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1604004810 0x7f16201f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16200ce610 0x7f16201fae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16201fb380 0x7f162015bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16200af450 con 0x7f16201fb380 2024-09-10T21:18:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161fe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f16201609e0 con 0x7f16200ce610 2024-09-10T21:18:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161fe00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f16200af5d0 con 0x7f1604004810 2024-09-10T21:18:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16201fb380 0x7f162015bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:24.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16201fb380 0x7f162015bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58574/0 (socket says 172.21.15.16:58574) 2024-09-10T21:18:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161ea00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1604004810 0x7f16201f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161f400640 1 -- 172.21.15.16:0/1329070332 learned_addr learned my addr 172.21.15.16:0/1329070332 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f1617e00640 1 --2- 172.21.15.16:0/1329070332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16200ce610 0x7f16201fae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161f400640 1 -- 172.21.15.16:0/1329070332 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1604004810 msgr2=0x7f16201f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161f400640 1 --2- 172.21.15.16:0/1329070332 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1604004810 0x7f16201f8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161f400640 1 -- 172.21.15.16:0/1329070332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16200ce610 msgr2=0x7f16201fae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161f400640 1 --2- 172.21.15.16:0/1329070332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16200ce610 0x7f16201fae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161f400640 1 -- 172.21.15.16:0/1329070332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16201f87a0 con 0x7f16201fb380 2024-09-10T21:18:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161f400640 1 --2- 172.21.15.16:0/1329070332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16201fb380 0x7f162015bf20 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f16200d2e00 tx=0x7f1618010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f1616a00640 1 -- 172.21.15.16:0/1329070332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161800ee70 con 0x7f16201fb380 2024-09-10T21:18:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f1616a00640 1 -- 172.21.15.16:0/1329070332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1618015070 con 0x7f16201fb380 2024-09-10T21:18:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f1616a00640 1 -- 172.21.15.16:0/1329070332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618003d50 con 0x7f16201fb380 2024-09-10T21:18:24.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161fe00640 1 -- 172.21.15.16:0/1329070332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16201f77b0 con 0x7f16201fb380 2024-09-10T21:18:24.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161fe00640 1 -- 172.21.15.16:0/1329070332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16201fbb50 con 0x7f16201fb380 2024-09-10T21:18:24.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f161fe00640 1 -- 172.21.15.16:0/1329070332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16201fbd20 con 0x7f16201fb380 2024-09-10T21:18:24.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.848+0000 7f1616a00640 1 -- 172.21.15.16:0/1329070332 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1618004030 con 0x7f16201fb380 2024-09-10T21:18:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.852+0000 7f1616a00640 1 --2- 172.21.15.16:0/1329070332 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15fc0725e0 0x7f15fc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.852+0000 7f1616a00640 1 -- 172.21.15.16:0/1329070332 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f1618093c90 con 0x7f16201fb380 2024-09-10T21:18:24.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.852+0000 7f161ea00640 1 --2- 172.21.15.16:0/1329070332 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15fc0725e0 0x7f15fc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.852+0000 7f161ea00640 1 --2- 172.21.15.16:0/1329070332 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15fc0725e0 0x7f15fc074aa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1610000c10 tx=0x7f1610006b20 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:24.856+0000 7f1616a00640 1 -- 172.21.15.16:0/1329070332 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f161805ec80 con 0x7f16201fb380 2024-09-10T21:18:25.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.064+0000 7f161fe00640 1 -- 172.21.15.16:0/1329070332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f162015f430 con 0x7f16201fb380 2024-09-10T21:18:25.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.064+0000 7f1616a00640 1 -- 172.21.15.16:0/1329070332 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v93) ==== 53+0+73 (secure 0 0 0) 0x7f16180636b0 con 0x7f16201fb380 2024-09-10T21:18:25.069 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:18:25.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.064+0000 7f161fe00640 1 -- 172.21.15.16:0/1329070332 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15fc0725e0 msgr2=0x7f15fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.064+0000 7f161fe00640 1 --2- 172.21.15.16:0/1329070332 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15fc0725e0 0x7f15fc074aa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1610000c10 tx=0x7f1610006b20 comp rx=0 tx=0).stop 2024-09-10T21:18:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.064+0000 7f161fe00640 1 -- 172.21.15.16:0/1329070332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16201fb380 msgr2=0x7f162015bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.064+0000 7f161fe00640 1 --2- 172.21.15.16:0/1329070332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16201fb380 0x7f162015bf20 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f16200d2e00 tx=0x7f1618010e80 comp rx=0 tx=0).stop 2024-09-10T21:18:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.064+0000 7f161fe00640 1 -- 172.21.15.16:0/1329070332 shutdown_connections 2024-09-10T21:18:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.064+0000 7f161fe00640 1 --2- 172.21.15.16:0/1329070332 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15fc0725e0 0x7f15fc074aa0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.064+0000 7f161fe00640 1 --2- 172.21.15.16:0/1329070332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16201fb380 0x7f162015bf20 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.064+0000 7f161fe00640 1 --2- 172.21.15.16:0/1329070332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16200ce610 0x7f16201fae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.064+0000 7f161fe00640 1 --2- 172.21.15.16:0/1329070332 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f1604004810 0x7f16201f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.064+0000 7f161fe00640 1 -- 172.21.15.16:0/1329070332 >> 172.21.15.16:0/1329070332 conn(0x7f16200d1d00 msgr2=0x7f1620070720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.068+0000 7f161fe00640 1 -- 172.21.15.16:0/1329070332 shutdown_connections 2024-09-10T21:18:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.068+0000 7f161fe00640 1 -- 172.21.15.16:0/1329070332 wait complete. 2024-09-10T21:18:25.092 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:18:25.092 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:18:25.092 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-09-10T21:18:25.092 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T21:18:25.092 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:18:25.092 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:18:25.092 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:18:25.092 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:18:25.092 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:18:25.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:25.093 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-09-10T21:18:25.103 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:18:25.103 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:18:25.103 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:18:25.103 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:25.103 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:18:25.103 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:25.103 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:18:25.103 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:18:25.103 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-09-10T21:18:25.103 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:18:25.103 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:18:25.104 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-09-10T21:18:25.104 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:18:25.104 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-09-10T21:18:25.104 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-09-10T21:18:25.104 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:18:25.104 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 41945sec preferred_lft 41945sec 2024-09-10T21:18:25.104 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-09-10T21:18:25.104 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:25.104 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:18:25.104 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-09-10T21:18:25.104 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-09-10T21:18:25.104 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:18:25.105 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:7e:b4:71:a2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:18:25.105 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:18:25.105 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:18:25.105 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:18:25.105 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:18:25.105 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:25.105 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:18:25.105 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:18:25.105 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:18:25.105 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:18:25.106 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:25.106 INFO:teuthology.orchestra.run.smithi052.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:18:25.106 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:18:25.106 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:18:25.106 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:18:25.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:25.107 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:18:25.171 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-09-10T21:18:25.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:25.174 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-09-10T21:18:25.226 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-09-10T21:18:25.226 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:18:25.226 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:18:25.226 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:18:25.226 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:18:25.226 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:18:25.227 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:18:25.227 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:18:25.227 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:18:25.227 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:18:25.227 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:18:25.227 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:18:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:18:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:18:25.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:18:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:18:25.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:25.373 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:18:25.383 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:18:25.383 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:18:25.383 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:18:25.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:25.383 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-09-10T21:18:25.452 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:18:25.452 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T21:18:25.507 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:18:25.508 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:18:25.508 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:18:25.559 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T21:18:25.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:25.560 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:18:25.653 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:18:25.653 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:18:25.666 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:18:25.666 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:18:25.666 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:18:25.667 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 1@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:18:25.667 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:18:25.670 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:18:25.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:25.670 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:18:25.689 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:18:25.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:18:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:18:25.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:18:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:18:25.793 INFO:teuthology.orchestra.run.smithi052.stdout:client7426 2024-09-10T21:18:25.793 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:18:25.794 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:25.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-09-10T21:18:25.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.948+0000 7f9a3a800640 1 -- 172.21.15.16:0/2375554307 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a3415dd80 msgr2=0x7f9a3415e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.948+0000 7f9a3a800640 1 --2- 172.21.15.16:0/2375554307 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a3415dd80 0x7f9a3415e160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00a950 tx=0x7f9a1c01da90 comp rx=0 tx=0).stop 2024-09-10T21:18:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.948+0000 7f9a3a800640 1 -- 172.21.15.16:0/2375554307 shutdown_connections 2024-09-10T21:18:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.948+0000 7f9a3a800640 1 --2- 172.21.15.16:0/2375554307 >> v2:172.21.15.52:3300/0 conn(0x7f9a3406f800 0x7f9a3406fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.948+0000 7f9a3a800640 1 --2- 172.21.15.16:0/2375554307 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a3415dd80 0x7f9a3415e160 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.948+0000 7f9a3a800640 1 -- 172.21.15.16:0/2375554307 >> 172.21.15.16:0/2375554307 conn(0x7f9a340b16a0 msgr2=0x7f9a340b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.948+0000 7f9a3a800640 1 -- 172.21.15.16:0/2375554307 shutdown_connections 2024-09-10T21:18:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.948+0000 7f9a3a800640 1 -- 172.21.15.16:0/2375554307 wait complete. 2024-09-10T21:18:25.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a3a800640 1 Processor -- start 2024-09-10T21:18:25.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a3a800640 1 -- start start 2024-09-10T21:18:25.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a3a800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a3406f800 0x7f9a341f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:25.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a3a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a34151210 0x7f9a3414a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:25.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a3a800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a3415dd80 0x7f9a340b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:25.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a3a800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a34070540 con 0x7f9a34151210 2024-09-10T21:18:25.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a3a800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9a340703c0 con 0x7f9a3415dd80 2024-09-10T21:18:25.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a3a800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9a340706c0 con 0x7f9a3406f800 2024-09-10T21:18:25.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a2aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a34151210 0x7f9a3414a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:25.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a2aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a34151210 0x7f9a3414a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58576/0 (socket says 172.21.15.16:58576) 2024-09-10T21:18:25.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a33400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a3406f800 0x7f9a341f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:25.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a2aa00640 1 -- 172.21.15.16:0/1047309091 learned_addr learned my addr 172.21.15.16:0/1047309091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:25.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a33e00640 1 --2- 172.21.15.16:0/1047309091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a3415dd80 0x7f9a340b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:25.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a2aa00640 1 -- 172.21.15.16:0/1047309091 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a3406f800 msgr2=0x7f9a341f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:25.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a2aa00640 1 --2- 172.21.15.16:0/1047309091 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a3406f800 0x7f9a341f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:25.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a2aa00640 1 -- 172.21.15.16:0/1047309091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a3415dd80 msgr2=0x7f9a340b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:25.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a2aa00640 1 --2- 172.21.15.16:0/1047309091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a3415dd80 0x7f9a340b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:25.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a2aa00640 1 -- 172.21.15.16:0/1047309091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a34164c30 con 0x7f9a34151210 2024-09-10T21:18:25.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a2aa00640 1 --2- 172.21.15.16:0/1047309091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a34151210 0x7f9a3414a660 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c01c140 tx=0x7f9a1c004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:25.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a2be00640 1 -- 172.21.15.16:0/1047309091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0b3040 con 0x7f9a34151210 2024-09-10T21:18:25.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a2be00640 1 -- 172.21.15.16:0/1047309091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a1c0a6bc0 con 0x7f9a34151210 2024-09-10T21:18:25.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a2be00640 1 -- 172.21.15.16:0/1047309091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c003600 con 0x7f9a34151210 2024-09-10T21:18:25.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a3a800640 1 -- 172.21.15.16:0/1047309091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a340ae420 con 0x7f9a34151210 2024-09-10T21:18:25.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.952+0000 7f9a3a800640 1 -- 172.21.15.16:0/1047309091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a340ccc10 con 0x7f9a34151210 2024-09-10T21:18:25.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.956+0000 7f9a3a800640 1 -- 172.21.15.16:0/1047309091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a3406fc40 con 0x7f9a34151210 2024-09-10T21:18:25.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.956+0000 7f9a2be00640 1 -- 172.21.15.16:0/1047309091 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9a1c00b040 con 0x7f9a34151210 2024-09-10T21:18:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.956+0000 7f9a2be00640 1 --2- 172.21.15.16:0/1047309091 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9a100725e0 0x7f9a10074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.956+0000 7f9a2be00640 1 -- 172.21.15.16:0/1047309091 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f9a1c12caf0 con 0x7f9a34151210 2024-09-10T21:18:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.956+0000 7f9a33400640 1 --2- 172.21.15.16:0/1047309091 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9a100725e0 0x7f9a10074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:25.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.956+0000 7f9a33400640 1 --2- 172.21.15.16:0/1047309091 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9a100725e0 0x7f9a10074aa0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9a24001ec0 tx=0x7f9a24001a80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:25.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:25.960+0000 7f9a2be00640 1 -- 172.21.15.16:0/1047309091 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9a1c017680 con 0x7f9a34151210 2024-09-10T21:18:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.132+0000 7f9a3a800640 1 -- 172.21.15.16:0/1047309091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f9a340b2c50 con 0x7f9a34151210 2024-09-10T21:18:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.136+0000 7f9a2be00640 1 -- 172.21.15.16:0/1047309091 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v60)=0 v60) ==== 121+0+0 (secure 0 0 0) 0x7f9a1c0f8130 con 0x7f9a34151210 2024-09-10T21:18:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.140+0000 7f9a3a800640 1 -- 172.21.15.16:0/1047309091 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9a100725e0 msgr2=0x7f9a10074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.140+0000 7f9a3a800640 1 --2- 172.21.15.16:0/1047309091 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9a100725e0 0x7f9a10074aa0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9a24001ec0 tx=0x7f9a24001a80 comp rx=0 tx=0).stop 2024-09-10T21:18:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.140+0000 7f9a3a800640 1 -- 172.21.15.16:0/1047309091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a34151210 msgr2=0x7f9a3414a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.140+0000 7f9a3a800640 1 --2- 172.21.15.16:0/1047309091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a34151210 0x7f9a3414a660 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c01c140 tx=0x7f9a1c004620 comp rx=0 tx=0).stop 2024-09-10T21:18:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.140+0000 7f9a3a800640 1 -- 172.21.15.16:0/1047309091 shutdown_connections 2024-09-10T21:18:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.140+0000 7f9a3a800640 1 --2- 172.21.15.16:0/1047309091 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9a100725e0 0x7f9a10074aa0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.140+0000 7f9a3a800640 1 --2- 172.21.15.16:0/1047309091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a3415dd80 0x7f9a340b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.140+0000 7f9a3a800640 1 --2- 172.21.15.16:0/1047309091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a34151210 0x7f9a3414a660 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.140+0000 7f9a3a800640 1 --2- 172.21.15.16:0/1047309091 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9a3406f800 0x7f9a341f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.140+0000 7f9a3a800640 1 -- 172.21.15.16:0/1047309091 >> 172.21.15.16:0/1047309091 conn(0x7f9a340b16a0 msgr2=0x7f9a340d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.140+0000 7f9a3a800640 1 -- 172.21.15.16:0/1047309091 shutdown_connections 2024-09-10T21:18:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.140+0000 7f9a3a800640 1 -- 172.21.15.16:0/1047309091 wait complete. 2024-09-10T21:18:26.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2024-09-10T21:18:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.308+0000 7f8fdf400640 1 -- 172.21.15.16:0/1201239408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fbc003a60 msgr2=0x7f8fbc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.308+0000 7f8fdf400640 1 --2- 172.21.15.16:0/1201239408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fbc003a60 0x7f8fbc003e80 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009b60 tx=0x7f8fc801d2b0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.308+0000 7f8fdd400640 1 -- 172.21.15.16:0/1201239408 reap_dead start 2024-09-10T21:18:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.308+0000 7f8fdf400640 1 -- 172.21.15.16:0/1201239408 shutdown_connections 2024-09-10T21:18:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.308+0000 7f8fdf400640 1 -- 172.21.15.16:0/1201239408 >> 172.21.15.16:0/1201239408 conn(0x7f8fd80d1df0 msgr2=0x7f8fd80d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.308+0000 7f8fdf400640 1 -- 172.21.15.16:0/1201239408 shutdown_connections 2024-09-10T21:18:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.308+0000 7f8fdf400640 1 -- 172.21.15.16:0/1201239408 wait complete. 2024-09-10T21:18:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fdf400640 1 Processor -- start 2024-09-10T21:18:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fdf400640 1 -- start start 2024-09-10T21:18:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fdf400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fbc003a60 0x7f8fd80cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fdf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd80c9630 0x7f8fd81f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fdf400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8fd80c9ab0 0x7f8fd81f8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fdf400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd80d2ad0 con 0x7f8fd80c9630 2024-09-10T21:18:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fdf400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd80b14f0 con 0x7f8fbc003a60 2024-09-10T21:18:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fdf400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8fd80cf1e0 con 0x7f8fd80c9ab0 2024-09-10T21:18:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fcfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd80c9630 0x7f8fd81f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fcfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd80c9630 0x7f8fd81f84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58586/0 (socket says 172.21.15.16:58586) 2024-09-10T21:18:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fcfe00640 1 -- 172.21.15.16:0/819434757 learned_addr learned my addr 172.21.15.16:0/819434757 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fcfe00640 1 -- 172.21.15.16:0/819434757 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8fd80c9ab0 msgr2=0x7f8fd81f8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fcfe00640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8fd80c9ab0 0x7f8fd81f8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fcfe00640 1 -- 172.21.15.16:0/819434757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fbc003a60 msgr2=0x7f8fd80cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fdca00640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fbc003a60 0x7f8fd80cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fcfe00640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fbc003a60 0x7f8fd80cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fcfe00640 1 -- 172.21.15.16:0/819434757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd81f9030 con 0x7f8fd80c9630 2024-09-10T21:18:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fdca00640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fbc003a60 0x7f8fd80cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fcfe00640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd80c9630 0x7f8fd81f84c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0001720 tx=0x7f8fc000e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fd5600640 1 -- 172.21.15.16:0/819434757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0012850 con 0x7f8fd80c9630 2024-09-10T21:18:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fd5600640 1 -- 172.21.15.16:0/819434757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fc0012ae0 con 0x7f8fd80c9630 2024-09-10T21:18:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fd5600640 1 -- 172.21.15.16:0/819434757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc001a400 con 0x7f8fd80c9630 2024-09-10T21:18:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fdf400640 1 -- 172.21.15.16:0/819434757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd81f9310 con 0x7f8fd80c9630 2024-09-10T21:18:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.312+0000 7f8fdf400640 1 -- 172.21.15.16:0/819434757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd80cb810 con 0x7f8fd80c9630 2024-09-10T21:18:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.316+0000 7f8fdf400640 1 -- 172.21.15.16:0/819434757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd80ce590 con 0x7f8fd80c9630 2024-09-10T21:18:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.316+0000 7f8fd5600640 1 -- 172.21.15.16:0/819434757 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8fc001a5a0 con 0x7f8fd80c9630 2024-09-10T21:18:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.316+0000 7f8fd5600640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8fb40725e0 0x7f8fb4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.316+0000 7f8fd5600640 1 -- 172.21.15.16:0/819434757 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f8fc0096c30 con 0x7f8fd80c9630 2024-09-10T21:18:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.316+0000 7f8fdca00640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8fb40725e0 0x7f8fb4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.316+0000 7f8fdca00640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8fb40725e0 0x7f8fb4074aa0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009b60 tx=0x7f8fc801a050 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:26.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.320+0000 7f8fd5600640 1 -- 172.21.15.16:0/819434757 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8fc0029080 con 0x7f8fd80c9630 2024-09-10T21:18:26.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.492+0000 7f8fdf400640 1 -- 172.21.15.16:0/819434757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7f8fd80b2900 con 0x7f8fd80c9630 2024-09-10T21:18:26.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.496+0000 7f8fd5600640 1 -- 172.21.15.16:0/819434757 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v61)=0 v61) ==== 119+0+0 (secure 0 0 0) 0x7f8fc0016030 con 0x7f8fd80c9630 2024-09-10T21:18:26.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.500+0000 7f8fdf400640 1 -- 172.21.15.16:0/819434757 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8fb40725e0 msgr2=0x7f8fb4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:26.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.500+0000 7f8fdf400640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8fb40725e0 0x7f8fb4074aa0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009b60 tx=0x7f8fc801a050 comp rx=0 tx=0).stop 2024-09-10T21:18:26.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.500+0000 7f8fdf400640 1 -- 172.21.15.16:0/819434757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd80c9630 msgr2=0x7f8fd81f84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:26.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.500+0000 7f8fdf400640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd80c9630 0x7f8fd81f84c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0001720 tx=0x7f8fc000e6b0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.500+0000 7f8fdf400640 1 -- 172.21.15.16:0/819434757 shutdown_connections 2024-09-10T21:18:26.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.500+0000 7f8fdf400640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8fb40725e0 0x7f8fb4074aa0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.500+0000 7f8fdf400640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8fd80c9ab0 0x7f8fd81f8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.500+0000 7f8fdf400640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd80c9630 0x7f8fd81f84c0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.500+0000 7f8fdf400640 1 --2- 172.21.15.16:0/819434757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fbc003a60 0x7f8fd80cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.500+0000 7f8fdf400640 1 -- 172.21.15.16:0/819434757 >> 172.21.15.16:0/819434757 conn(0x7f8fd80d1df0 msgr2=0x7f8fd80b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:26.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.500+0000 7f8fdf400640 1 -- 172.21.15.16:0/819434757 shutdown_connections 2024-09-10T21:18:26.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.500+0000 7f8fdf400640 1 -- 172.21.15.16:0/819434757 wait complete. 2024-09-10T21:18:26.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2024-09-10T21:18:26.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.668+0000 7f2cd2600640 1 -- 172.21.15.16:0/2973844562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4004d50 msgr2=0x7f2cb40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:26.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.668+0000 7f2cd2600640 1 --2- 172.21.15.16:0/2973844562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cb4004d50 0x7f2cb40051d0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4006fd0 tx=0x7f2cc401cc20 comp rx=0 tx=0).stop 2024-09-10T21:18:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.668+0000 7f2cd1c00640 1 -- 172.21.15.16:0/2973844562 reap_dead start 2024-09-10T21:18:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.668+0000 7f2cd2600640 1 -- 172.21.15.16:0/2973844562 shutdown_connections 2024-09-10T21:18:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.668+0000 7f2cd2600640 1 -- 172.21.15.16:0/2973844562 >> 172.21.15.16:0/2973844562 conn(0x7f2ccc0d1c70 msgr2=0x7f2ccc0d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.668+0000 7f2cd2600640 1 -- 172.21.15.16:0/2973844562 shutdown_connections 2024-09-10T21:18:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.668+0000 7f2cd2600640 1 -- 172.21.15.16:0/2973844562 wait complete. 2024-09-10T21:18:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.668+0000 7f2cd2600640 1 Processor -- start 2024-09-10T21:18:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.668+0000 7f2cd2600640 1 -- start start 2024-09-10T21:18:26.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cd2600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2cb4004d50 0x7f2ccc157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:26.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cd2600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ccc0b4aa0 0x7f2ccc1f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:26.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cd2600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ccc0b54b0 0x7f2ccc1f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:26.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cd2600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc0cee20 con 0x7f2ccc0b4aa0 2024-09-10T21:18:26.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cd2600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc0ceca0 con 0x7f2ccc0b54b0 2024-09-10T21:18:26.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cd2600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2ccc0cefa0 con 0x7f2cb4004d50 2024-09-10T21:18:26.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2ccbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ccc0b4aa0 0x7f2ccc1f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:26.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2ccbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ccc0b4aa0 0x7f2ccc1f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58614/0 (socket says 172.21.15.16:58614) 2024-09-10T21:18:26.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2ccbe00640 1 -- 172.21.15.16:0/130076804 learned_addr learned my addr 172.21.15.16:0/130076804 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:26.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cd1200640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2cb4004d50 0x7f2ccc157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:26.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2ccbe00640 1 -- 172.21.15.16:0/130076804 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2cb4004d50 msgr2=0x7f2ccc157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2ccbe00640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2cb4004d50 0x7f2ccc157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2ccbe00640 1 -- 172.21.15.16:0/130076804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ccc0b54b0 msgr2=0x7f2ccc1f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cd1c00640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ccc0b54b0 0x7f2ccc1f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2ccbe00640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ccc0b54b0 0x7f2ccc1f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2ccbe00640 1 -- 172.21.15.16:0/130076804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc0b3050 con 0x7f2ccc0b4aa0 2024-09-10T21:18:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2ccbe00640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ccc0b4aa0 0x7f2ccc1f4db0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8010fb0 tx=0x7f2cb800e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cc8c00640 1 -- 172.21.15.16:0/130076804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8012850 con 0x7f2ccc0b4aa0 2024-09-10T21:18:26.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cc8c00640 1 -- 172.21.15.16:0/130076804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cb8012ae0 con 0x7f2ccc0b4aa0 2024-09-10T21:18:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cc8c00640 1 -- 172.21.15.16:0/130076804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb801a400 con 0x7f2ccc0b4aa0 2024-09-10T21:18:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cd2600640 1 -- 172.21.15.16:0/130076804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc0b7b50 con 0x7f2ccc0b4aa0 2024-09-10T21:18:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cd2600640 1 -- 172.21.15.16:0/130076804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc0b7590 con 0x7f2ccc0b4aa0 2024-09-10T21:18:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cd2600640 1 -- 172.21.15.16:0/130076804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ccc0ae940 con 0x7f2ccc0b4aa0 2024-09-10T21:18:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.672+0000 7f2cc8c00640 1 -- 172.21.15.16:0/130076804 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2cb80233f0 con 0x7f2ccc0b4aa0 2024-09-10T21:18:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.676+0000 7f2cc8c00640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2cac0725e0 0x7f2cac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.676+0000 7f2cc8c00640 1 -- 172.21.15.16:0/130076804 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f2cb80969c0 con 0x7f2ccc0b4aa0 2024-09-10T21:18:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.676+0000 7f2cd1200640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2cac0725e0 0x7f2cac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:26.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.676+0000 7f2cd1200640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2cac0725e0 0x7f2cac074aa0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009ba0 tx=0x7f2cc00091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:26.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.680+0000 7f2cc8c00640 1 -- 172.21.15.16:0/130076804 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2cb8016030 con 0x7f2ccc0b4aa0 2024-09-10T21:18:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.852+0000 7f2cd2600640 1 -- 172.21.15.16:0/130076804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7f2ccc15b920 con 0x7f2ccc0b4aa0 2024-09-10T21:18:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.856+0000 7f2cc8c00640 1 -- 172.21.15.16:0/130076804 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v62)=0 v62) ==== 128+0+0 (secure 0 0 0) 0x7f2cb80620c0 con 0x7f2ccc0b4aa0 2024-09-10T21:18:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.860+0000 7f2cd2600640 1 -- 172.21.15.16:0/130076804 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2cac0725e0 msgr2=0x7f2cac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.860+0000 7f2cd2600640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2cac0725e0 0x7f2cac074aa0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009ba0 tx=0x7f2cc00091e0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.860+0000 7f2cd2600640 1 -- 172.21.15.16:0/130076804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ccc0b4aa0 msgr2=0x7f2ccc1f4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.860+0000 7f2cd2600640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ccc0b4aa0 0x7f2ccc1f4db0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8010fb0 tx=0x7f2cb800e770 comp rx=0 tx=0).stop 2024-09-10T21:18:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.860+0000 7f2cd2600640 1 -- 172.21.15.16:0/130076804 shutdown_connections 2024-09-10T21:18:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.860+0000 7f2cd2600640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2cac0725e0 0x7f2cac074aa0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.860+0000 7f2cd2600640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ccc0b54b0 0x7f2ccc1f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.860+0000 7f2cd2600640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ccc0b4aa0 0x7f2ccc1f4db0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.860+0000 7f2cd2600640 1 --2- 172.21.15.16:0/130076804 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2cb4004d50 0x7f2ccc157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.860+0000 7f2cd2600640 1 -- 172.21.15.16:0/130076804 >> 172.21.15.16:0/130076804 conn(0x7f2ccc0d1c70 msgr2=0x7f2ccc160660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.860+0000 7f2cd2600640 1 -- 172.21.15.16:0/130076804 shutdown_connections 2024-09-10T21:18:26.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:26.860+0000 7f2cd2600640 1 -- 172.21.15.16:0/130076804 wait complete. 2024-09-10T21:18:26.884 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-10T21:18:27.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.028+0000 7fe64b600640 1 -- 172.21.15.16:0/3448765763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c004810 msgr2=0x7fe62c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.028+0000 7fe64b600640 1 --2- 172.21.15.16:0/3448765763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c004810 0x7fe62c004c90 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe644161c10 tx=0x7fe63001cbe0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.028+0000 7fe649600640 1 -- 172.21.15.16:0/3448765763 reap_dead start 2024-09-10T21:18:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.028+0000 7fe64b600640 1 -- 172.21.15.16:0/3448765763 shutdown_connections 2024-09-10T21:18:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.028+0000 7fe64b600640 1 -- 172.21.15.16:0/3448765763 >> 172.21.15.16:0/3448765763 conn(0x7fe6440d3ed0 msgr2=0x7fe6440d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.028+0000 7fe64b600640 1 -- 172.21.15.16:0/3448765763 shutdown_connections 2024-09-10T21:18:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.028+0000 7fe64b600640 1 -- 172.21.15.16:0/3448765763 wait complete. 2024-09-10T21:18:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe64b600640 1 Processor -- start 2024-09-10T21:18:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe64b600640 1 -- start start 2024-09-10T21:18:27.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe64b600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe62c004810 0x7fe6440b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:27.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe64b600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6441f7f60 0x7fe6440c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:27.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe64b600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6441f42b0 0x7fe64406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:27.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe64b600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6440d2850 con 0x7fe6441f42b0 2024-09-10T21:18:27.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe64b600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe6440d26d0 con 0x7fe6441f7f60 2024-09-10T21:18:27.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe64b600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe6440b4800 con 0x7fe62c004810 2024-09-10T21:18:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe649600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6441f42b0 0x7fe64406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe649600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6441f42b0 0x7fe64406da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58618/0 (socket says 172.21.15.16:58618) 2024-09-10T21:18:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe649600640 1 -- 172.21.15.16:0/558342825 learned_addr learned my addr 172.21.15.16:0/558342825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe648c00640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe62c004810 0x7fe6440b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe643e00640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6441f7f60 0x7fe6440c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe649600640 1 -- 172.21.15.16:0/558342825 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe62c004810 msgr2=0x7fe6440b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe649600640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe62c004810 0x7fe6440b7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe649600640 1 -- 172.21.15.16:0/558342825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6441f7f60 msgr2=0x7fe6440c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe649600640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6441f7f60 0x7fe6440c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe649600640 1 -- 172.21.15.16:0/558342825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6440d47c0 con 0x7fe6441f42b0 2024-09-10T21:18:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe649600640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6441f42b0 0x7fe64406da60 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe6440d3a60 tx=0x7fe63c006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe640c00640 1 -- 172.21.15.16:0/558342825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c015070 con 0x7fe6441f42b0 2024-09-10T21:18:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe640c00640 1 -- 172.21.15.16:0/558342825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe63c011900 con 0x7fe6441f42b0 2024-09-10T21:18:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe640c00640 1 -- 172.21.15.16:0/558342825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c011d50 con 0x7fe6441f42b0 2024-09-10T21:18:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe64b600640 1 -- 172.21.15.16:0/558342825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6440ca520 con 0x7fe6441f42b0 2024-09-10T21:18:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.032+0000 7fe64b600640 1 -- 172.21.15.16:0/558342825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6441f87b0 con 0x7fe6441f42b0 2024-09-10T21:18:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.036+0000 7fe64b600640 1 -- 172.21.15.16:0/558342825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6440b40f0 con 0x7fe6441f42b0 2024-09-10T21:18:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.036+0000 7fe640c00640 1 -- 172.21.15.16:0/558342825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe63c0103c0 con 0x7fe6441f42b0 2024-09-10T21:18:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.036+0000 7fe640c00640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6240725e0 0x7fe624074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.036+0000 7fe640c00640 1 -- 172.21.15.16:0/558342825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7fe63c093d70 con 0x7fe6441f42b0 2024-09-10T21:18:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.036+0000 7fe648c00640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6240725e0 0x7fe624074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.036+0000 7fe648c00640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6240725e0 0x7fe624074aa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe64408d7d0 tx=0x7fe638006730 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:27.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.040+0000 7fe640c00640 1 -- 172.21.15.16:0/558342825 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe63c05edf0 con 0x7fe6441f42b0 2024-09-10T21:18:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.216+0000 7fe64b600640 1 -- 172.21.15.16:0/558342825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7fe6440d1450 con 0x7fe6441f42b0 2024-09-10T21:18:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.216+0000 7fe640c00640 1 -- 172.21.15.16:0/558342825 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v62) ==== 106+0+4 (secure 0 0 0) 0x7fe63c063820 con 0x7fe6441f42b0 2024-09-10T21:18:27.221 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-10T21:18:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.216+0000 7fe64b600640 1 -- 172.21.15.16:0/558342825 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6240725e0 msgr2=0x7fe624074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.216+0000 7fe64b600640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6240725e0 0x7fe624074aa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe64408d7d0 tx=0x7fe638006730 comp rx=0 tx=0).stop 2024-09-10T21:18:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.216+0000 7fe64b600640 1 -- 172.21.15.16:0/558342825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6441f42b0 msgr2=0x7fe64406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.216+0000 7fe64b600640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6441f42b0 0x7fe64406da60 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe6440d3a60 tx=0x7fe63c006430 comp rx=0 tx=0).stop 2024-09-10T21:18:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.220+0000 7fe64b600640 1 -- 172.21.15.16:0/558342825 shutdown_connections 2024-09-10T21:18:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.220+0000 7fe64b600640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6240725e0 0x7fe624074aa0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.220+0000 7fe64b600640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6441f42b0 0x7fe64406da60 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.220+0000 7fe64b600640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6441f7f60 0x7fe6440c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.220+0000 7fe64b600640 1 --2- 172.21.15.16:0/558342825 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe62c004810 0x7fe6440b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.220+0000 7fe64b600640 1 -- 172.21.15.16:0/558342825 >> 172.21.15.16:0/558342825 conn(0x7fe6440d3ed0 msgr2=0x7fe6440b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.220+0000 7fe64b600640 1 -- 172.21.15.16:0/558342825 shutdown_connections 2024-09-10T21:18:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.220+0000 7fe64b600640 1 -- 172.21.15.16:0/558342825 wait complete. 2024-09-10T21:18:27.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-09-10T21:18:27.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.388+0000 7f2d9b800640 1 -- 172.21.15.16:0/3150687783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d9406f7e0 msgr2=0x7f2d9406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.388+0000 7f2d9b800640 1 --2- 172.21.15.16:0/3150687783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d9406f7e0 0x7f2d9406fbc0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009840 tx=0x7f2d8801c6c0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.388+0000 7f2d9b800640 1 -- 172.21.15.16:0/3150687783 shutdown_connections 2024-09-10T21:18:27.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.388+0000 7f2d9b800640 1 --2- 172.21.15.16:0/3150687783 >> v2:172.21.15.52:3300/0 conn(0x7f2d940cc710 0x7f2d9406d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.388+0000 7f2d9b800640 1 --2- 172.21.15.16:0/3150687783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d9406f7e0 0x7f2d9406fbc0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.388+0000 7f2d9b800640 1 -- 172.21.15.16:0/3150687783 >> 172.21.15.16:0/3150687783 conn(0x7f2d940cd6a0 msgr2=0x7f2d940cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.388+0000 7f2d9b800640 1 -- 172.21.15.16:0/3150687783 shutdown_connections 2024-09-10T21:18:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.388+0000 7f2d9b800640 1 -- 172.21.15.16:0/3150687783 wait complete. 2024-09-10T21:18:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.388+0000 7f2d9b800640 1 Processor -- start 2024-09-10T21:18:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.388+0000 7f2d9b800640 1 -- start start 2024-09-10T21:18:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d9b800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d9406f7e0 0x7f2d940cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d9b800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d940747d0 0x7f2d9414f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d9b800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d940cc710 0x7f2d94164cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d9b800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9406d9b0 con 0x7f2d940cc710 2024-09-10T21:18:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d9b800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9406d830 con 0x7f2d9406f7e0 2024-09-10T21:18:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d9b800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2d94169120 con 0x7f2d940747d0 2024-09-10T21:18:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d99800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d940cc710 0x7f2d94164cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d99800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d940cc710 0x7f2d94164cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58634/0 (socket says 172.21.15.16:58634) 2024-09-10T21:18:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d99800640 1 -- 172.21.15.16:0/3799993204 learned_addr learned my addr 172.21.15.16:0/3799993204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d98e00640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d9406f7e0 0x7f2d940cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d93e00640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d940747d0 0x7f2d9414f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d99800640 1 -- 172.21.15.16:0/3799993204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d940747d0 msgr2=0x7f2d9414f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d99800640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d940747d0 0x7f2d9414f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d99800640 1 -- 172.21.15.16:0/3799993204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d9406f7e0 msgr2=0x7f2d940cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d99800640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d9406f7e0 0x7f2d940cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d99800640 1 -- 172.21.15.16:0/3799993204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d940d0800 con 0x7f2d940cc710 2024-09-10T21:18:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d98e00640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d9406f7e0 0x7f2d940cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d99800640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d940cc710 0x7f2d94164cc0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00da40 tx=0x7f2d8c00df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d90c00640 1 -- 172.21.15.16:0/3799993204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c017070 con 0x7f2d940cc710 2024-09-10T21:18:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d90c00640 1 -- 172.21.15.16:0/3799993204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d8c004b30 con 0x7f2d940cc710 2024-09-10T21:18:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d90c00640 1 -- 172.21.15.16:0/3799993204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c005020 con 0x7f2d940cc710 2024-09-10T21:18:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d9b800640 1 -- 172.21.15.16:0/3799993204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d940d0240 con 0x7f2d940cc710 2024-09-10T21:18:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d9b800640 1 -- 172.21.15.16:0/3799993204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d941585a0 con 0x7f2d940cc710 2024-09-10T21:18:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.392+0000 7f2d9b800640 1 -- 172.21.15.16:0/3799993204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d9406fbc0 con 0x7f2d940cc710 2024-09-10T21:18:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.396+0000 7f2d90c00640 1 -- 172.21.15.16:0/3799993204 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2d8c0051c0 con 0x7f2d940cc710 2024-09-10T21:18:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.396+0000 7f2d90c00640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d74072610 0x7f2d74074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.396+0000 7f2d90c00640 1 -- 172.21.15.16:0/3799993204 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f2d8c093740 con 0x7f2d940cc710 2024-09-10T21:18:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.396+0000 7f2d98e00640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d74072610 0x7f2d74074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.396+0000 7f2d98e00640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d74072610 0x7f2d74074ad0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2d880af000 tx=0x7f2d8801af00 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.400+0000 7f2d90c00640 1 -- 172.21.15.16:0/3799993204 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2d8c020070 con 0x7f2d940cc710 2024-09-10T21:18:27.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.576+0000 7f2d9b800640 1 -- 172.21.15.16:0/3799993204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f2d940c41a0 con 0x7f2d940cc710 2024-09-10T21:18:27.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.580+0000 7f2d90c00640 1 -- 172.21.15.16:0/3799993204 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v63)=0 v63) ==== 124+0+0 (secure 0 0 0) 0x7f2d8c05ed80 con 0x7f2d940cc710 2024-09-10T21:18:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.580+0000 7f2d9b800640 1 -- 172.21.15.16:0/3799993204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d74072610 msgr2=0x7f2d74074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.580+0000 7f2d9b800640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d74072610 0x7f2d74074ad0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2d880af000 tx=0x7f2d8801af00 comp rx=0 tx=0).stop 2024-09-10T21:18:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.580+0000 7f2d9b800640 1 -- 172.21.15.16:0/3799993204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d940cc710 msgr2=0x7f2d94164cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.580+0000 7f2d9b800640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d940cc710 0x7f2d94164cc0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00da40 tx=0x7f2d8c00df00 comp rx=0 tx=0).stop 2024-09-10T21:18:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.580+0000 7f2d9b800640 1 -- 172.21.15.16:0/3799993204 shutdown_connections 2024-09-10T21:18:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.580+0000 7f2d9b800640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d74072610 0x7f2d74074ad0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.580+0000 7f2d9b800640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d940cc710 0x7f2d94164cc0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.580+0000 7f2d9b800640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d940747d0 0x7f2d9414f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.580+0000 7f2d9b800640 1 --2- 172.21.15.16:0/3799993204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d9406f7e0 0x7f2d940cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.580+0000 7f2d9b800640 1 -- 172.21.15.16:0/3799993204 >> 172.21.15.16:0/3799993204 conn(0x7f2d940cd6a0 msgr2=0x7f2d940736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.580+0000 7f2d9b800640 1 -- 172.21.15.16:0/3799993204 shutdown_connections 2024-09-10T21:18:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.580+0000 7f2d9b800640 1 -- 172.21.15.16:0/3799993204 wait complete. 2024-09-10T21:18:27.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-09-10T21:18:27.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.748+0000 7fed4f200640 1 -- 172.21.15.16:0/943319387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed48161430 msgr2=0x7fed480d1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.748+0000 7fed4f200640 1 --2- 172.21.15.16:0/943319387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed48161430 0x7fed480d1aa0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fed38009900 tx=0x7fed3801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.748+0000 7fed4f200640 1 -- 172.21.15.16:0/943319387 shutdown_connections 2024-09-10T21:18:27.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.748+0000 7fed4f200640 1 --2- 172.21.15.16:0/943319387 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed480cf480 0x7fed480b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.748+0000 7fed4f200640 1 --2- 172.21.15.16:0/943319387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed48161430 0x7fed480d1aa0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.748+0000 7fed4f200640 1 -- 172.21.15.16:0/943319387 >> 172.21.15.16:0/943319387 conn(0x7fed480d3ed0 msgr2=0x7fed480d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:27.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.748+0000 7fed4f200640 1 -- 172.21.15.16:0/943319387 shutdown_connections 2024-09-10T21:18:27.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.748+0000 7fed4f200640 1 -- 172.21.15.16:0/943319387 wait complete. 2024-09-10T21:18:27.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4f200640 1 Processor -- start 2024-09-10T21:18:27.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4f200640 1 -- start start 2024-09-10T21:18:27.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4f200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed480cf480 0x7fed481576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:27.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed48161430 0x7fed48157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:27.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed48161810 0x7fed480b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:27.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed4816ae80 con 0x7fed48161810 2024-09-10T21:18:27.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fed4816ad00 con 0x7fed48161430 2024-09-10T21:18:27.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4f200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fed4816b000 con 0x7fed480cf480 2024-09-10T21:18:27.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed48161810 0x7fed480b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:27.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed48161810 0x7fed480b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58660/0 (socket says 172.21.15.16:58660) 2024-09-10T21:18:27.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4d200640 1 -- 172.21.15.16:0/3942233647 learned_addr learned my addr 172.21.15.16:0/3942233647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:27.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed3fe00640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed48161430 0x7fed48157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:27.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed47e00640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed480cf480 0x7fed481576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:27.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4d200640 1 -- 172.21.15.16:0/3942233647 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed480cf480 msgr2=0x7fed481576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4d200640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed480cf480 0x7fed481576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4d200640 1 -- 172.21.15.16:0/3942233647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed48161430 msgr2=0x7fed48157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4d200640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed48161430 0x7fed48157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4d200640 1 -- 172.21.15.16:0/3942233647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed48154850 con 0x7fed48161810 2024-09-10T21:18:27.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4d200640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed48161810 0x7fed480b2690 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fed4000ea20 tx=0x7fed4000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:27.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed44c00640 1 -- 172.21.15.16:0/3942233647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4000cde0 con 0x7fed48161810 2024-09-10T21:18:27.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed44c00640 1 -- 172.21.15.16:0/3942233647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed40004c40 con 0x7fed48161810 2024-09-10T21:18:27.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4f200640 1 -- 172.21.15.16:0/3942233647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed481537b0 con 0x7fed48161810 2024-09-10T21:18:27.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed44c00640 1 -- 172.21.15.16:0/3942233647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40005280 con 0x7fed48161810 2024-09-10T21:18:27.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.752+0000 7fed4f200640 1 -- 172.21.15.16:0/3942233647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed48153d70 con 0x7fed48161810 2024-09-10T21:18:27.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.756+0000 7fed4f200640 1 -- 172.21.15.16:0/3942233647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed480b40d0 con 0x7fed48161810 2024-09-10T21:18:27.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.756+0000 7fed44c00640 1 -- 172.21.15.16:0/3942233647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fed4000f040 con 0x7fed48161810 2024-09-10T21:18:27.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.756+0000 7fed44c00640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed240725e0 0x7fed24074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:27.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.756+0000 7fed44c00640 1 -- 172.21.15.16:0/3942233647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7fed40093810 con 0x7fed48161810 2024-09-10T21:18:27.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.756+0000 7fed47e00640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed240725e0 0x7fed24074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:27.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.756+0000 7fed47e00640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed240725e0 0x7fed24074aa0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fed380af000 tx=0x7fed3801b120 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.760+0000 7fed44c00640 1 -- 172.21.15.16:0/3942233647 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fed40016030 con 0x7fed48161810 2024-09-10T21:18:27.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed4f200640 1 -- 172.21.15.16:0/3942233647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7fed480b2170 con 0x7fed48161810 2024-09-10T21:18:27.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed44c00640 1 -- 172.21.15.16:0/3942233647 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v63) ==== 106+0+8 (secure 0 0 0) 0x7fed4005e890 con 0x7fed48161810 2024-09-10T21:18:27.941 INFO:teuthology.orchestra.run.smithi016.stdout:1048576 2024-09-10T21:18:27.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed4f200640 1 -- 172.21.15.16:0/3942233647 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed240725e0 msgr2=0x7fed24074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed4f200640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed240725e0 0x7fed24074aa0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fed380af000 tx=0x7fed3801b120 comp rx=0 tx=0).stop 2024-09-10T21:18:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed4f200640 1 -- 172.21.15.16:0/3942233647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed48161810 msgr2=0x7fed480b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed4f200640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed48161810 0x7fed480b2690 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fed4000ea20 tx=0x7fed4000eee0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed4f200640 1 -- 172.21.15.16:0/3942233647 shutdown_connections 2024-09-10T21:18:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed4f200640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fed240725e0 0x7fed24074aa0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed4f200640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed48161810 0x7fed480b2690 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed4f200640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed48161430 0x7fed48157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed4f200640 1 --2- 172.21.15.16:0/3942233647 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fed480cf480 0x7fed481576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed4f200640 1 -- 172.21.15.16:0/3942233647 >> 172.21.15.16:0/3942233647 conn(0x7fed480d3ed0 msgr2=0x7fed4806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed4f200640 1 -- 172.21.15.16:0/3942233647 shutdown_connections 2024-09-10T21:18:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:27.936+0000 7fed4f200640 1 -- 172.21.15.16:0/3942233647 wait complete. 2024-09-10T21:18:27.962 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-09-10T21:18:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 -- 172.21.15.16:0/670584057 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40b00ce1c0 msgr2=0x7f40b00cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 --2- 172.21.15.16:0/670584057 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40b00ce1c0 0x7f40b00cc4a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f409c009840 tx=0x7f409c01c790 comp rx=0 tx=0).stop 2024-09-10T21:18:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 -- 172.21.15.16:0/670584057 shutdown_connections 2024-09-10T21:18:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 --2- 172.21.15.16:0/670584057 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40b00ce1c0 0x7f40b00cc4a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 --2- 172.21.15.16:0/670584057 >> v2:172.21.15.52:3300/0 conn(0x7f40b0158600 0x7f40b015f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 -- 172.21.15.16:0/670584057 >> 172.21.15.16:0/670584057 conn(0x7f40b00cd6a0 msgr2=0x7f40b00cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 -- 172.21.15.16:0/670584057 shutdown_connections 2024-09-10T21:18:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 -- 172.21.15.16:0/670584057 wait complete. 2024-09-10T21:18:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 Processor -- start 2024-09-10T21:18:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 -- start start 2024-09-10T21:18:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b00cc9e0 0x7f40b00cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40b00ce1c0 0x7f40b015c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b0158600 0x7f40b014dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40b0167e60 con 0x7f40b0158600 2024-09-10T21:18:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40b0167ce0 con 0x7f40b00cc9e0 2024-09-10T21:18:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b8200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f40b0167fe0 con 0x7f40b00ce1c0 2024-09-10T21:18:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b6200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b0158600 0x7f40b014dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b4e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40b00ce1c0 0x7f40b015c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b5800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b00cc9e0 0x7f40b00cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b6200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b0158600 0x7f40b014dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58676/0 (socket says 172.21.15.16:58676) 2024-09-10T21:18:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.108+0000 7f40b6200640 1 -- 172.21.15.16:0/2440999542 learned_addr learned my addr 172.21.15.16:0/2440999542 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:28.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40b6200640 1 -- 172.21.15.16:0/2440999542 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40b00ce1c0 msgr2=0x7f40b015c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40b6200640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40b00ce1c0 0x7f40b015c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40b6200640 1 -- 172.21.15.16:0/2440999542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b00cc9e0 msgr2=0x7f40b00cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40b6200640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b00cc9e0 0x7f40b00cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40b6200640 1 -- 172.21.15.16:0/2440999542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b00d0250 con 0x7f40b0158600 2024-09-10T21:18:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40b4e00640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40b00ce1c0 0x7f40b015c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40b5800640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b00cc9e0 0x7f40b00cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40b6200640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b0158600 0x7f40b014dc30 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f40a800c7f0 tx=0x7f40a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40ad600640 1 -- 172.21.15.16:0/2440999542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8003b80 con 0x7f40b0158600 2024-09-10T21:18:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40ad600640 1 -- 172.21.15.16:0/2440999542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40a8003d20 con 0x7f40b0158600 2024-09-10T21:18:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40b8200640 1 -- 172.21.15.16:0/2440999542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b00d0530 con 0x7f40b0158600 2024-09-10T21:18:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40ad600640 1 -- 172.21.15.16:0/2440999542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a800aab0 con 0x7f40b0158600 2024-09-10T21:18:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40b8200640 1 -- 172.21.15.16:0/2440999542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b00d0810 con 0x7f40b0158600 2024-09-10T21:18:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40b8200640 1 -- 172.21.15.16:0/2440999542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b0163fb0 con 0x7f40b0158600 2024-09-10T21:18:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.112+0000 7f40ad600640 1 -- 172.21.15.16:0/2440999542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f40a800ac50 con 0x7f40b0158600 2024-09-10T21:18:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.116+0000 7f40ad600640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4090072610 0x7f4090074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.116+0000 7f40ad600640 1 -- 172.21.15.16:0/2440999542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f40a8093110 con 0x7f40b0158600 2024-09-10T21:18:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.116+0000 7f40b5800640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4090072610 0x7f4090074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.116+0000 7f40b5800640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4090072610 0x7f4090074ad0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009f30 tx=0x7f40a4009260 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.120+0000 7f40ad600640 1 -- 172.21.15.16:0/2440999542 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f40a805dff0 con 0x7f40b0158600 2024-09-10T21:18:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.292+0000 7f40b8200640 1 -- 172.21.15.16:0/2440999542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7f40b0154c70 con 0x7f40b0158600 2024-09-10T21:18:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.292+0000 7f40ad600640 1 -- 172.21.15.16:0/2440999542 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v63) ==== 112+0+10 (secure 0 0 0) 0x7f40a8062a20 con 0x7f40b0158600 2024-09-10T21:18:28.299 INFO:teuthology.orchestra.run.smithi016.stdout:60.000000 2024-09-10T21:18:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.296+0000 7f40b8200640 1 -- 172.21.15.16:0/2440999542 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4090072610 msgr2=0x7f4090074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.296+0000 7f40b8200640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4090072610 0x7f4090074ad0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009f30 tx=0x7f40a4009260 comp rx=0 tx=0).stop 2024-09-10T21:18:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.296+0000 7f40b8200640 1 -- 172.21.15.16:0/2440999542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b0158600 msgr2=0x7f40b014dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.296+0000 7f40b8200640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b0158600 0x7f40b014dc30 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f40a800c7f0 tx=0x7f40a800ccb0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.296+0000 7f40b8200640 1 -- 172.21.15.16:0/2440999542 shutdown_connections 2024-09-10T21:18:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.296+0000 7f40b8200640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4090072610 0x7f4090074ad0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.296+0000 7f40b8200640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b0158600 0x7f40b014dc30 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.296+0000 7f40b8200640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40b00ce1c0 0x7f40b015c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.296+0000 7f40b8200640 1 --2- 172.21.15.16:0/2440999542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b00cc9e0 0x7f40b00cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.296+0000 7f40b8200640 1 -- 172.21.15.16:0/2440999542 >> 172.21.15.16:0/2440999542 conn(0x7f40b00cd6a0 msgr2=0x7f40b00b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.296+0000 7f40b8200640 1 -- 172.21.15.16:0/2440999542 shutdown_connections 2024-09-10T21:18:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.296+0000 7f40b8200640 1 -- 172.21.15.16:0/2440999542 wait complete. 2024-09-10T21:18:28.322 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:18:28.323 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:28.331 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:18:28.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:18:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:18:28.437 INFO:teuthology.orchestra.run.smithi016.stdout:client7414 2024-09-10T21:18:28.437 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:18:28.437 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:28.446 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-10T21:18:28.446 DEBUG:teuthology.orchestra.run.smithi016:> import fcntl 2024-09-10T21:18:28.446 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> import time 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> n = 400 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> abs_path = "/home/ubuntu/cephtest/mnt.0/." 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> if not os.path.exists(abs_path): 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(abs_path) 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> handles = [] 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, n): 2024-09-10T21:18:28.447 DEBUG:teuthology.orchestra.run.smithi016:> fname = "file_"+str(i) 2024-09-10T21:18:28.448 DEBUG:teuthology.orchestra.run.smithi016:> path = os.path.join(abs_path, fname) 2024-09-10T21:18:28.448 DEBUG:teuthology.orchestra.run.smithi016:> handles.append(open(path, '"'"'w'"'"')) 2024-09-10T21:18:28.448 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-10T21:18:28.448 DEBUG:teuthology.orchestra.run.smithi016:> print("waiting with handles open", file=sys.stderr) 2024-09-10T21:18:28.448 DEBUG:teuthology.orchestra.run.smithi016:> while True: 2024-09-10T21:18:28.448 DEBUG:teuthology.orchestra.run.smithi016:> print("open_n_background: keeping files open", file=sys.stderr) 2024-09-10T21:18:28.448 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-10T21:18:28.448 DEBUG:teuthology.orchestra.run.smithi016:> if os.read(0, 4096) == b"": 2024-09-10T21:18:28.448 DEBUG:teuthology.orchestra.run.smithi016:> break 2024-09-10T21:18:28.449 DEBUG:teuthology.orchestra.run.smithi016:> except BlockingIOError: 2024-09-10T21:18:28.449 DEBUG:teuthology.orchestra.run.smithi016:> pass 2024-09-10T21:18:28.449 DEBUG:teuthology.orchestra.run.smithi016:> time.sleep(2) 2024-09-10T21:18:28.449 DEBUG:teuthology.orchestra.run.smithi016:> print("stdin closed, goodbye!", file=sys.stderr) 2024-09-10T21:18:28.449 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-10T21:18:28.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:28.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.636+0000 7f8b6f000640 1 -- 172.21.15.16:0/2317058683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b680ca170 msgr2=0x7f8b680ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:28.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.636+0000 7f8b6f000640 1 --2- 172.21.15.16:0/2317058683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b680ca170 0x7f8b680ca5b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f8b6806e1a0 tx=0x7f8b6001ccc0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.636+0000 7f8b6f000640 1 -- 172.21.15.16:0/2317058683 shutdown_connections 2024-09-10T21:18:28.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.636+0000 7f8b6f000640 1 --2- 172.21.15.16:0/2317058683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b680ca170 0x7f8b680ca5b0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.636+0000 7f8b6f000640 1 --2- 172.21.15.16:0/2317058683 >> v2:172.21.15.52:3300/0 conn(0x7f8b6815a270 0x7f8b680ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.636+0000 7f8b6f000640 1 -- 172.21.15.16:0/2317058683 >> 172.21.15.16:0/2317058683 conn(0x7f8b680cfb20 msgr2=0x7f8b680cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:28.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.636+0000 7f8b6f000640 1 -- 172.21.15.16:0/2317058683 shutdown_connections 2024-09-10T21:18:28.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.636+0000 7f8b6f000640 1 -- 172.21.15.16:0/2317058683 wait complete. 2024-09-10T21:18:28.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.636+0000 7f8b6f000640 1 Processor -- start 2024-09-10T21:18:28.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.636+0000 7f8b6f000640 1 -- start start 2024-09-10T21:18:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b6f000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b680ca170 0x7f8b6815b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b6f000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b680d06b0 0x7f8b68156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b6f000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8b6815a270 0x7f8b680b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b6f000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68072b80 con 0x7f8b680d06b0 2024-09-10T21:18:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b6f000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68072a00 con 0x7f8b680ca170 2024-09-10T21:18:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b6f000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8b68072d00 con 0x7f8b6815a270 2024-09-10T21:18:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b5fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b680d06b0 0x7f8b68156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b67e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b680ca170 0x7f8b6815b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b5fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b680d06b0 0x7f8b68156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58686/0 (socket says 172.21.15.16:58686) 2024-09-10T21:18:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b5fe00640 1 -- 172.21.15.16:0/1014214828 learned_addr learned my addr 172.21.15.16:0/1014214828 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b6d000640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8b6815a270 0x7f8b680b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b67e00640 1 -- 172.21.15.16:0/1014214828 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8b6815a270 msgr2=0x7f8b680b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b67e00640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8b6815a270 0x7f8b680b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b67e00640 1 -- 172.21.15.16:0/1014214828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b680d06b0 msgr2=0x7f8b68156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b67e00640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b680d06b0 0x7f8b68156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b67e00640 1 -- 172.21.15.16:0/1014214828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b681f74d0 con 0x7f8b680ca170 2024-09-10T21:18:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b5fe00640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b680d06b0 0x7f8b68156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:28.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b67e00640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b680ca170 0x7f8b6815b270 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800e7f0 tx=0x7f8b5800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:28.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b64c00640 1 -- 172.21.15.16:0/1014214828 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5800cdc0 con 0x7f8b680ca170 2024-09-10T21:18:28.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b64c00640 1 -- 172.21.15.16:0/1014214828 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b58017440 con 0x7f8b680ca170 2024-09-10T21:18:28.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b64c00640 1 -- 172.21.15.16:0/1014214828 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58017870 con 0x7f8b680ca170 2024-09-10T21:18:28.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b6f000640 1 -- 172.21.15.16:0/1014214828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b680c5140 con 0x7f8b680ca170 2024-09-10T21:18:28.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b6f000640 1 -- 172.21.15.16:0/1014214828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b680c4e60 con 0x7f8b680ca170 2024-09-10T21:18:28.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.640+0000 7f8b6f000640 1 -- 172.21.15.16:0/1014214828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b680ca5b0 con 0x7f8b680ca170 2024-09-10T21:18:28.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.644+0000 7f8b64c00640 1 -- 172.21.15.16:0/1014214828 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8b58015070 con 0x7f8b680ca170 2024-09-10T21:18:28.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.644+0000 7f8b64c00640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8b440725e0 0x7f8b44074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:28.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.644+0000 7f8b64c00640 1 -- 172.21.15.16:0/1014214828 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f8b580938f0 con 0x7f8b680ca170 2024-09-10T21:18:28.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.644+0000 7f8b5fe00640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8b440725e0 0x7f8b44074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:28.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.644+0000 7f8b5fe00640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8b440725e0 0x7f8b44074aa0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8b680b4100 tx=0x7f8b50009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.648+0000 7f8b64c00640 1 -- 172.21.15.16:0/1014214828 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8b5805e8e0 con 0x7f8b680ca170 2024-09-10T21:18:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.880+0000 7f8b6f000640 1 -- 172.21.15.16:0/1014214828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b680b5aa0 con 0x7f8b680ca170 2024-09-10T21:18:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.880+0000 7f8b64c00640 1 -- 172.21.15.16:0/1014214828 <== mon.1 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5227 (secure 0 0 0) 0x7f8b58063310 con 0x7f8b680ca170 2024-09-10T21:18:28.886 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:28.886 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"btime":"2024-09-10T21:18:20:455300+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3971491960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3971491960},{"type":"v1","addr":"172.21.15.52:6834","nonce":3971491960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7854,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4224090659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4224090659},{"type":"v1","addr":"172.21.15.16:6836","nonce":4224090659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1749071285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1749071285},{"type":"v1","addr":"172.21.15.16:6837","nonce":1749071285}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:16.852256+0000","modified":"2024-09-10T21:18:19.856561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6835/3672156966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3672156966},{"type":"v1","addr":"172.21.15.52:6835","nonce":3672156966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7869,"qdb_cluster":[7869]},"id":10}]} 2024-09-10T21:18:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 93 2024-09-10T21:18:28.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.884+0000 7f8b6f000640 1 -- 172.21.15.16:0/1014214828 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8b440725e0 msgr2=0x7f8b44074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:28.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.884+0000 7f8b6f000640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8b440725e0 0x7f8b44074aa0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8b680b4100 tx=0x7f8b50009040 comp rx=0 tx=0).stop 2024-09-10T21:18:28.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.884+0000 7f8b6f000640 1 -- 172.21.15.16:0/1014214828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b680ca170 msgr2=0x7f8b6815b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:28.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.884+0000 7f8b6f000640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b680ca170 0x7f8b6815b270 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800e7f0 tx=0x7f8b5800ecb0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.884+0000 7f8b6f000640 1 -- 172.21.15.16:0/1014214828 shutdown_connections 2024-09-10T21:18:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.884+0000 7f8b6f000640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8b440725e0 0x7f8b44074aa0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.884+0000 7f8b6f000640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8b6815a270 0x7f8b680b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.884+0000 7f8b6f000640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b680d06b0 0x7f8b68156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.884+0000 7f8b6f000640 1 --2- 172.21.15.16:0/1014214828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b680ca170 0x7f8b6815b270 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.884+0000 7f8b6f000640 1 -- 172.21.15.16:0/1014214828 >> 172.21.15.16:0/1014214828 conn(0x7f8b680cfb20 msgr2=0x7f8b68157af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.884+0000 7f8b6f000640 1 -- 172.21.15.16:0/1014214828 shutdown_connections 2024-09-10T21:18:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:28.884+0000 7f8b6f000640 1 -- 172.21.15.16:0/1014214828 wait complete. 2024-09-10T21:18:28.910 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-10T21:18:29.049 INFO:teuthology.orchestra.run.smithi016.stderr:waiting with handles open 2024-09-10T21:18:29.049 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:18:29.081 INFO:teuthology.orchestra.run.smithi052.stdout:[{"id":7426,"entity":{"name":{"type":"client","num":7426},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":827843831}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":3.4039605690000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9898199121645055,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7426 192.168.0.1:0/827843831","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi052","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":7414,"entity":{"name":{"type":"client","num":7414},"addr":{"type":"any","addr":"172.21.15.16:0","nonce":2073476252}},"state":"open","num_leases":400,"num_caps":401,"request_load_avg":424,"uptime":4.5639471010000001,"requests_in_flight":399,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1201.2599248866677,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7414 172.21.15.16:0/2073476252","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-10T21:18:29.082 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi052", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 827843831, "type": "any" }, "name": { "num": 7426, "type": "client" } }, "id": 7426, "inst": "client.7426 192.168.0.1:0/827843831", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9898199121645055 }, "state": "open", "uptime": 3.403960569 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 2073476252, "type": "any" }, "name": { "num": 7414, "type": "client" } }, "id": 7414, "inst": "client.7414 172.21.15.16:0/2073476252", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 400, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 424, "requests_in_flight": 399, "session_cache_liveness": { "halflife": 300, "value": 1201.2599248866677 }, "state": "open", "uptime": 4.563947101 } ] 2024-09-10T21:18:29.082 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-09-10T21:18:29.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:18:29.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 -- 172.21.15.16:0/27446107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f00d06b0 msgr2=0x7ff4f015a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:29.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 --2- 172.21.15.16:0/27446107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f00d06b0 0x7ff4f015a270 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0074fa0 tx=0x7ff4e401c830 comp rx=0 tx=0).stop 2024-09-10T21:18:29.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 -- 172.21.15.16:0/27446107 shutdown_connections 2024-09-10T21:18:29.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 --2- 172.21.15.16:0/27446107 >> v2:172.21.15.52:3300/0 conn(0x7ff4f00700a0 0x7ff4f00ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:29.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 --2- 172.21.15.16:0/27446107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f00d06b0 0x7ff4f015a270 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:29.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 -- 172.21.15.16:0/27446107 >> 172.21.15.16:0/27446107 conn(0x7ff4f00cfb20 msgr2=0x7ff4f00cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:29.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 -- 172.21.15.16:0/27446107 shutdown_connections 2024-09-10T21:18:29.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 -- 172.21.15.16:0/27446107 wait complete. 2024-09-10T21:18:29.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 Processor -- start 2024-09-10T21:18:29.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 -- start start 2024-09-10T21:18:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4f00700a0 0x7ff4f015d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f00ce610 0x7ff4f01fa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff4f00d06b0 0x7ff4f0158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f00ca6b0 con 0x7ff4f00ce610 2024-09-10T21:18:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f00707a0 con 0x7ff4f00700a0 2024-09-10T21:18:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f7600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f00ca830 con 0x7ff4f00d06b0 2024-09-10T21:18:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f4c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4f00700a0 0x7ff4f015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f00ce610 0x7ff4f01fa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f4c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4f00700a0 0x7ff4f015d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:41078/0 (socket says 172.21.15.16:41078) 2024-09-10T21:18:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f4c00640 1 -- 172.21.15.16:0/3794880712 learned_addr learned my addr 172.21.15.16:0/3794880712 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f5600640 1 --2- 172.21.15.16:0/3794880712 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff4f00d06b0 0x7ff4f0158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:29.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f4c00640 1 -- 172.21.15.16:0/3794880712 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff4f00d06b0 msgr2=0x7ff4f0158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:29.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f4c00640 1 --2- 172.21.15.16:0/3794880712 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff4f00d06b0 0x7ff4f0158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:29.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f4c00640 1 -- 172.21.15.16:0/3794880712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f00ce610 msgr2=0x7ff4f01fa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:29.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.224+0000 7ff4f4c00640 1 --2- 172.21.15.16:0/3794880712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f00ce610 0x7ff4f01fa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:29.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.228+0000 7ff4f4c00640 1 -- 172.21.15.16:0/3794880712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f01fadb0 con 0x7ff4f00700a0 2024-09-10T21:18:29.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.228+0000 7ff4f4c00640 1 --2- 172.21.15.16:0/3794880712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4f00700a0 0x7ff4f015d700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40b0040 tx=0x7ff4e40a5800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:29.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.228+0000 7ff4ecc00640 1 -- 172.21.15.16:0/3794880712 <== mon.1 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e40b2040 con 0x7ff4f00700a0 2024-09-10T21:18:29.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.228+0000 7ff4ecc00640 1 -- 172.21.15.16:0/3794880712 <== mon.1 v2:172.21.15.52:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4e4004430 con 0x7ff4f00700a0 2024-09-10T21:18:29.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.228+0000 7ff4ecc00640 1 -- 172.21.15.16:0/3794880712 <== mon.1 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4005280 con 0x7ff4f00700a0 2024-09-10T21:18:29.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.228+0000 7ff4f7600640 1 -- 172.21.15.16:0/3794880712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f01fb090 con 0x7ff4f00700a0 2024-09-10T21:18:29.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.228+0000 7ff4f7600640 1 -- 172.21.15.16:0/3794880712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f01fb650 con 0x7ff4f00700a0 2024-09-10T21:18:29.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.228+0000 7ff4f7600640 1 -- 172.21.15.16:0/3794880712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f00ca170 con 0x7ff4f00700a0 2024-09-10T21:18:29.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.228+0000 7ff4ecc00640 1 -- 172.21.15.16:0/3794880712 <== mon.1 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff4e4005420 con 0x7ff4f00700a0 2024-09-10T21:18:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.228+0000 7ff4ecc00640 1 --2- 172.21.15.16:0/3794880712 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff4d00725e0 0x7ff4d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.228+0000 7ff4ecc00640 1 -- 172.21.15.16:0/3794880712 <== mon.1 v2:172.21.15.52:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7ff4e412cb70 con 0x7ff4f00700a0 2024-09-10T21:18:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.232+0000 7ff4efe00640 1 --2- 172.21.15.16:0/3794880712 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff4d00725e0 0x7ff4d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.232+0000 7ff4efe00640 1 --2- 172.21.15.16:0/3794880712 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff4d00725e0 0x7ff4d0074aa0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc007fd0 tx=0x7ff4dc009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.236+0000 7ff4ecc00640 1 -- 172.21.15.16:0/3794880712 <== mon.1 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff4e40164b0 con 0x7ff4f00700a0 2024-09-10T21:18:29.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.444+0000 7ff4f7600640 1 -- 172.21.15.16:0/3794880712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff4f00b5aa0 con 0x7ff4f00700a0 2024-09-10T21:18:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.444+0000 7ff4ecc00640 1 -- 172.21.15.16:0/3794880712 <== mon.1 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff4e40f7bf0 con 0x7ff4f00700a0 2024-09-10T21:18:29.451 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:29.452 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T21:18:29.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.448+0000 7ff4f7600640 1 -- 172.21.15.16:0/3794880712 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff4d00725e0 msgr2=0x7ff4d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:29.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.448+0000 7ff4f7600640 1 --2- 172.21.15.16:0/3794880712 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff4d00725e0 0x7ff4d0074aa0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc007fd0 tx=0x7ff4dc009040 comp rx=0 tx=0).stop 2024-09-10T21:18:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.448+0000 7ff4f7600640 1 -- 172.21.15.16:0/3794880712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4f00700a0 msgr2=0x7ff4f015d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.448+0000 7ff4f7600640 1 --2- 172.21.15.16:0/3794880712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4f00700a0 0x7ff4f015d700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40b0040 tx=0x7ff4e40a5800 comp rx=0 tx=0).stop 2024-09-10T21:18:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.448+0000 7ff4f7600640 1 -- 172.21.15.16:0/3794880712 shutdown_connections 2024-09-10T21:18:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.448+0000 7ff4f7600640 1 --2- 172.21.15.16:0/3794880712 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff4d00725e0 0x7ff4d0074aa0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.448+0000 7ff4f7600640 1 --2- 172.21.15.16:0/3794880712 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff4f00d06b0 0x7ff4f0158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.448+0000 7ff4f7600640 1 --2- 172.21.15.16:0/3794880712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f00ce610 0x7ff4f01fa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.448+0000 7ff4f7600640 1 --2- 172.21.15.16:0/3794880712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4f00700a0 0x7ff4f015d700 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.448+0000 7ff4f7600640 1 -- 172.21.15.16:0/3794880712 >> 172.21.15.16:0/3794880712 conn(0x7ff4f00cfb20 msgr2=0x7ff4f00b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.448+0000 7ff4f7600640 1 -- 172.21.15.16:0/3794880712 shutdown_connections 2024-09-10T21:18:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:29.448+0000 7ff4f7600640 1 -- 172.21.15.16:0/3794880712 wait complete. 2024-09-10T21:18:29.473 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-10T21:18:29.473 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-10T21:18:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:18:33.053 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:18:34.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:18:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 -- 172.21.15.16:0/204944396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd00cb990 msgr2=0x7f8dd00737c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 --2- 172.21.15.16:0/204944396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd00cb990 0x7f8dd00737c0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009bc0 tx=0x7f8dc801ca80 comp rx=0 tx=0).stop 2024-09-10T21:18:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 -- 172.21.15.16:0/204944396 shutdown_connections 2024-09-10T21:18:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 --2- 172.21.15.16:0/204944396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd00cb990 0x7f8dd00737c0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 --2- 172.21.15.16:0/204944396 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8dd015f290 0x7f8dd00cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 -- 172.21.15.16:0/204944396 >> 172.21.15.16:0/204944396 conn(0x7f8dd00d0c30 msgr2=0x7f8dd00d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 -- 172.21.15.16:0/204944396 shutdown_connections 2024-09-10T21:18:34.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 -- 172.21.15.16:0/204944396 wait complete. 2024-09-10T21:18:34.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 Processor -- start 2024-09-10T21:18:34.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 -- start start 2024-09-10T21:18:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd00c41d0 0x7f8dd00ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8dd00cb990 0x7f8dd0166070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dd015f290 0x7f8dd01fba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01691a0 con 0x7f8dd00c41d0 2024-09-10T21:18:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd0169020 con 0x7f8dd015f290 2024-09-10T21:18:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd7000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd0169320 con 0x7f8dd00cb990 2024-09-10T21:18:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dcfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd00c41d0 0x7f8dd00ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dcfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd00c41d0 0x7f8dd00ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49880/0 (socket says 172.21.15.16:49880) 2024-09-10T21:18:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dcfe00640 1 -- 172.21.15.16:0/1322938320 learned_addr learned my addr 172.21.15.16:0/1322938320 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dc7e00640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8dd00cb990 0x7f8dd0166070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dd5000640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dd015f290 0x7f8dd01fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dcfe00640 1 -- 172.21.15.16:0/1322938320 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8dd00cb990 msgr2=0x7f8dd0166070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dcfe00640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8dd00cb990 0x7f8dd0166070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dcfe00640 1 -- 172.21.15.16:0/1322938320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dd015f290 msgr2=0x7f8dd01fba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dcfe00640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dd015f290 0x7f8dd01fba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dcfe00640 1 -- 172.21.15.16:0/1322938320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd00cf310 con 0x7f8dd00c41d0 2024-09-10T21:18:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dc7e00640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8dd00cb990 0x7f8dd0166070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dcfe00640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd00c41d0 0x7f8dd00ce800 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000e7a0 tx=0x7f8dc000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.628+0000 7f8dccc00640 1 -- 172.21.15.16:0/1322938320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc000ccc0 con 0x7f8dd00c41d0 2024-09-10T21:18:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.632+0000 7f8dccc00640 1 -- 172.21.15.16:0/1322938320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dc0010430 con 0x7f8dd00c41d0 2024-09-10T21:18:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.632+0000 7f8dccc00640 1 -- 172.21.15.16:0/1322938320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0010860 con 0x7f8dd00c41d0 2024-09-10T21:18:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.632+0000 7f8dd7000640 1 -- 172.21.15.16:0/1322938320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd00cf030 con 0x7f8dd00c41d0 2024-09-10T21:18:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.632+0000 7f8dd7000640 1 -- 172.21.15.16:0/1322938320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd00cf8c0 con 0x7f8dd00c41d0 2024-09-10T21:18:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.632+0000 7f8dd7000640 1 -- 172.21.15.16:0/1322938320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd00737c0 con 0x7f8dd00c41d0 2024-09-10T21:18:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.632+0000 7f8dccc00640 1 -- 172.21.15.16:0/1322938320 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8dc0010a00 con 0x7f8dd00c41d0 2024-09-10T21:18:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.632+0000 7f8dccc00640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8dac0725e0 0x7f8dac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.632+0000 7f8dccc00640 1 -- 172.21.15.16:0/1322938320 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f8dc0093c70 con 0x7f8dd00c41d0 2024-09-10T21:18:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.632+0000 7f8dc7e00640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8dac0725e0 0x7f8dac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.636+0000 7f8dc7e00640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8dac0725e0 0x7f8dac074aa0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8db8001230 tx=0x7f8db8009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.636+0000 7f8dccc00640 1 -- 172.21.15.16:0/1322938320 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8dc005f340 con 0x7f8dd00c41d0 2024-09-10T21:18:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dd7000640 1 -- 172.21.15.16:0/1322938320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8dd0152b00 con 0x7f8dd00c41d0 2024-09-10T21:18:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dccc00640 1 -- 172.21.15.16:0/1322938320 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f8dc005ec40 con 0x7f8dd00c41d0 2024-09-10T21:18:34.853 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:34.853 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:18:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dd7000640 1 -- 172.21.15.16:0/1322938320 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8dac0725e0 msgr2=0x7f8dac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dd7000640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8dac0725e0 0x7f8dac074aa0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8db8001230 tx=0x7f8db8009040 comp rx=0 tx=0).stop 2024-09-10T21:18:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dd7000640 1 -- 172.21.15.16:0/1322938320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd00c41d0 msgr2=0x7f8dd00ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dd7000640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd00c41d0 0x7f8dd00ce800 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000e7a0 tx=0x7f8dc000ec60 comp rx=0 tx=0).stop 2024-09-10T21:18:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dd7000640 1 -- 172.21.15.16:0/1322938320 shutdown_connections 2024-09-10T21:18:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dd7000640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8dac0725e0 0x7f8dac074aa0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dd7000640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dd015f290 0x7f8dd01fba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dd7000640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8dd00cb990 0x7f8dd0166070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dd7000640 1 --2- 172.21.15.16:0/1322938320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd00c41d0 0x7f8dd00ce800 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dd7000640 1 -- 172.21.15.16:0/1322938320 >> 172.21.15.16:0/1322938320 conn(0x7f8dd00d0c30 msgr2=0x7f8dd00c9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dd7000640 1 -- 172.21.15.16:0/1322938320 shutdown_connections 2024-09-10T21:18:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:34.848+0000 7f8dd7000640 1 -- 172.21.15.16:0/1322938320 wait complete. 2024-09-10T21:18:34.874 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:18:34.874 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-09-10T21:18:34.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:18:35.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.016+0000 7f8c0a800640 1 -- 172.21.15.16:0/1686321278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8003a60 msgr2=0x7f8be8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:35.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.016+0000 7f8c0a800640 1 --2- 172.21.15.16:0/1686321278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be8003a60 0x7f8be8003e80 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4006fc0 tx=0x7f8bf401ce50 comp rx=0 tx=0).stop 2024-09-10T21:18:35.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.016+0000 7f8c03e00640 1 -- 172.21.15.16:0/1686321278 reap_dead start 2024-09-10T21:18:35.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.016+0000 7f8c0a800640 1 -- 172.21.15.16:0/1686321278 shutdown_connections 2024-09-10T21:18:35.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.016+0000 7f8c0a800640 1 -- 172.21.15.16:0/1686321278 >> 172.21.15.16:0/1686321278 conn(0x7f8c040d0c30 msgr2=0x7f8c040d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.016+0000 7f8c0a800640 1 -- 172.21.15.16:0/1686321278 shutdown_connections 2024-09-10T21:18:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.016+0000 7f8c0a800640 1 -- 172.21.15.16:0/1686321278 wait complete. 2024-09-10T21:18:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.016+0000 7f8c0a800640 1 Processor -- start 2024-09-10T21:18:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c0a800640 1 -- start start 2024-09-10T21:18:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c0a800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be8003a60 0x7f8c04154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c0a800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c0415f700 0x7f8c0415fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c0a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c04155af0 0x7f8c04155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c0a800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04165b60 con 0x7f8c04155af0 2024-09-10T21:18:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c0a800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8c040d0850 con 0x7f8be8003a60 2024-09-10T21:18:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c0a800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8c04165ce0 con 0x7f8c0415f700 2024-09-10T21:18:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c03e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c04155af0 0x7f8c04155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c03400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be8003a60 0x7f8c04154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c03e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c04155af0 0x7f8c04155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49904/0 (socket says 172.21.15.16:49904) 2024-09-10T21:18:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c03e00640 1 -- 172.21.15.16:0/688728568 learned_addr learned my addr 172.21.15.16:0/688728568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8bfbe00640 1 --2- 172.21.15.16:0/688728568 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c0415f700 0x7f8c0415fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:35.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c03e00640 1 -- 172.21.15.16:0/688728568 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c0415f700 msgr2=0x7f8c0415fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:35.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c03e00640 1 --2- 172.21.15.16:0/688728568 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c0415f700 0x7f8c0415fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:35.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c03e00640 1 -- 172.21.15.16:0/688728568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be8003a60 msgr2=0x7f8c04154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:35.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c03e00640 1 --2- 172.21.15.16:0/688728568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be8003a60 0x7f8c04154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:35.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c03e00640 1 -- 172.21.15.16:0/688728568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c040cbc80 con 0x7f8c04155af0 2024-09-10T21:18:35.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c03e00640 1 --2- 172.21.15.16:0/688728568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c04155af0 0x7f8c04155f70 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc010910 tx=0x7f8bfc010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:35.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8bfb400640 1 -- 172.21.15.16:0/688728568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc003b90 con 0x7f8c04155af0 2024-09-10T21:18:35.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8bfb400640 1 -- 172.21.15.16:0/688728568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bfc003e20 con 0x7f8c04155af0 2024-09-10T21:18:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8bfb400640 1 -- 172.21.15.16:0/688728568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc004d70 con 0x7f8c04155af0 2024-09-10T21:18:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c0a800640 1 -- 172.21.15.16:0/688728568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c040c59d0 con 0x7f8c04155af0 2024-09-10T21:18:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c0a800640 1 -- 172.21.15.16:0/688728568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c041601a0 con 0x7f8c04155af0 2024-09-10T21:18:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.020+0000 7f8c0a800640 1 -- 172.21.15.16:0/688728568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c041564b0 con 0x7f8c04155af0 2024-09-10T21:18:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.024+0000 7f8bfb400640 1 -- 172.21.15.16:0/688728568 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8bfc00f040 con 0x7f8c04155af0 2024-09-10T21:18:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.024+0000 7f8bfb400640 1 --2- 172.21.15.16:0/688728568 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8be00725e0 0x7f8be0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.024+0000 7f8bfb400640 1 -- 172.21.15.16:0/688728568 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f8bfc093150 con 0x7f8c04155af0 2024-09-10T21:18:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.024+0000 7f8c03400640 1 --2- 172.21.15.16:0/688728568 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8be00725e0 0x7f8be0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.024+0000 7f8c03400640 1 --2- 172.21.15.16:0/688728568 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8be00725e0 0x7f8be0074aa0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40ab000 tx=0x7f8bf4019c90 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.028+0000 7f8bfb400640 1 -- 172.21.15.16:0/688728568 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8bfc05e790 con 0x7f8c04155af0 2024-09-10T21:18:35.055 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:18:35.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.240+0000 7f8c0a800640 1 -- 172.21.15.16:0/688728568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8c040cf530 con 0x7f8c04155af0 2024-09-10T21:18:35.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.240+0000 7f8bfb400640 1 -- 172.21.15.16:0/688728568 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f8bfc05e090 con 0x7f8c04155af0 2024-09-10T21:18:35.245 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:35.245 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:18:35.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.240+0000 7f8c0a800640 1 -- 172.21.15.16:0/688728568 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8be00725e0 msgr2=0x7f8be0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:35.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.240+0000 7f8c0a800640 1 --2- 172.21.15.16:0/688728568 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8be00725e0 0x7f8be0074aa0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40ab000 tx=0x7f8bf4019c90 comp rx=0 tx=0).stop 2024-09-10T21:18:35.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.240+0000 7f8c0a800640 1 -- 172.21.15.16:0/688728568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c04155af0 msgr2=0x7f8c04155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:35.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.240+0000 7f8c0a800640 1 --2- 172.21.15.16:0/688728568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c04155af0 0x7f8c04155f70 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc010910 tx=0x7f8bfc010d10 comp rx=0 tx=0).stop 2024-09-10T21:18:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.240+0000 7f8c0a800640 1 -- 172.21.15.16:0/688728568 shutdown_connections 2024-09-10T21:18:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.240+0000 7f8c0a800640 1 --2- 172.21.15.16:0/688728568 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8be00725e0 0x7f8be0074aa0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.240+0000 7f8c0a800640 1 --2- 172.21.15.16:0/688728568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c04155af0 0x7f8c04155f70 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.240+0000 7f8c0a800640 1 --2- 172.21.15.16:0/688728568 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8c0415f700 0x7f8c0415fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.240+0000 7f8c0a800640 1 --2- 172.21.15.16:0/688728568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8be8003a60 0x7f8c04154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.240+0000 7f8c0a800640 1 -- 172.21.15.16:0/688728568 >> 172.21.15.16:0/688728568 conn(0x7f8c040d0c30 msgr2=0x7f8c040c4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.244+0000 7f8c0a800640 1 -- 172.21.15.16:0/688728568 shutdown_connections 2024-09-10T21:18:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35.244+0000 7f8c0a800640 1 -- 172.21.15.16:0/688728568 wait complete. 2024-09-10T21:18:35.267 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:18:35.267 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-09-10T21:18:35.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:35 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-10T21:18:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-10T21:18:37.058 INFO:teuthology.orchestra.run.smithi016.stderr:stdin closed, goodbye! 2024-09-10T21:18:37.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:18:37.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.280+0000 7fa61aa00640 1 -- 172.21.15.16:0/3065273047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6141599d0 msgr2=0x7fa61415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:37.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.280+0000 7fa61aa00640 1 --2- 172.21.15.16:0/3065273047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6141599d0 0x7fa61415f830 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fa604009900 tx=0x7fa60401c850 comp rx=0 tx=0).stop 2024-09-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.280+0000 7fa61aa00640 1 -- 172.21.15.16:0/3065273047 shutdown_connections 2024-09-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.280+0000 7fa61aa00640 1 --2- 172.21.15.16:0/3065273047 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6140726b0 0x7fa614072b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.280+0000 7fa61aa00640 1 --2- 172.21.15.16:0/3065273047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6141599d0 0x7fa61415f830 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.280+0000 7fa61aa00640 1 -- 172.21.15.16:0/3065273047 >> 172.21.15.16:0/3065273047 conn(0x7fa6140cfb20 msgr2=0x7fa6140cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.280+0000 7fa61aa00640 1 -- 172.21.15.16:0/3065273047 shutdown_connections 2024-09-10T21:18:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.280+0000 7fa61aa00640 1 -- 172.21.15.16:0/3065273047 wait complete. 2024-09-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa61aa00640 1 Processor -- start 2024-09-10T21:18:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa61aa00640 1 -- start start 2024-09-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa61aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61406e640 0x7fa6141fb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa61aa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6140726b0 0x7fa614157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa61aa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6141599d0 0x7fa614165570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa61aa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa614168e60 con 0x7fa61406e640 2024-09-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa61aa00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa614168ce0 con 0x7fa6140726b0 2024-09-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa61aa00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa614168fe0 con 0x7fa6141599d0 2024-09-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa613e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61406e640 0x7fa6141fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa613e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61406e640 0x7fa6141fb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49932/0 (socket says 172.21.15.16:49932) 2024-09-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa613e00640 1 -- 172.21.15.16:0/3662385225 learned_addr learned my addr 172.21.15.16:0/3662385225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa618a00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6141599d0 0x7fa614165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa60be00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6140726b0 0x7fa614157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa613e00640 1 -- 172.21.15.16:0/3662385225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6141599d0 msgr2=0x7fa614165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa613e00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6141599d0 0x7fa614165570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa613e00640 1 -- 172.21.15.16:0/3662385225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6140726b0 msgr2=0x7fa614157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa613e00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6140726b0 0x7fa614157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa613e00640 1 -- 172.21.15.16:0/3662385225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6140ce310 con 0x7fa61406e640 2024-09-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa618a00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6141599d0 0x7fa614165570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa613e00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61406e640 0x7fa6141fb800 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa60401cd30 tx=0x7fa6040a6520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa610c00640 1 -- 172.21.15.16:0/3662385225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6040ac070 con 0x7fa61406e640 2024-09-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa610c00640 1 -- 172.21.15.16:0/3662385225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa604004340 con 0x7fa61406e640 2024-09-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa610c00640 1 -- 172.21.15.16:0/3662385225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604005320 con 0x7fa61406e640 2024-09-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.284+0000 7fa61aa00640 1 -- 172.21.15.16:0/3662385225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6140ce5d0 con 0x7fa61406e640 2024-09-10T21:18:37.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.288+0000 7fa61aa00640 1 -- 172.21.15.16:0/3662385225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6140ceb90 con 0x7fa61406e640 2024-09-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.288+0000 7fa61aa00640 1 -- 172.21.15.16:0/3662385225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa614072b40 con 0x7fa61406e640 2024-09-10T21:18:37.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.288+0000 7fa610c00640 1 -- 172.21.15.16:0/3662385225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa6040c3060 con 0x7fa61406e640 2024-09-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.288+0000 7fa610c00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5f00725e0 0x7fa5f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.288+0000 7fa610c00640 1 -- 172.21.15.16:0/3662385225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7fa60412d780 con 0x7fa61406e640 2024-09-10T21:18:37.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.288+0000 7fa60be00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5f00725e0 0x7fa5f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:37.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.292+0000 7fa60be00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5f00725e0 0x7fa5f0074aa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa6140ca2c0 tx=0x7fa5fc009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:37.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.292+0000 7fa610c00640 1 -- 172.21.15.16:0/3662385225 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa6040164d0 con 0x7fa61406e640 2024-09-10T21:18:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.504+0000 7fa61aa00640 1 -- 172.21.15.16:0/3662385225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa614072b40 con 0x7fa61406e640 2024-09-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.504+0000 7fa610c00640 1 -- 172.21.15.16:0/3662385225 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5227 (secure 0 0 0) 0x7fa6040f8dd0 con 0x7fa61406e640 2024-09-10T21:18:37.510 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":95,"btime":"2024-09-10T21:18:33:594669+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3971491960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3971491960},{"type":"v1","addr":"172.21.15.52:6834","nonce":3971491960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7854,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4224090659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4224090659},{"type":"v1","addr":"172.21.15.16:6836","nonce":4224090659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1749071285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1749071285},{"type":"v1","addr":"172.21.15.16:6837","nonce":1749071285}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:16.852256+0000","modified":"2024-09-10T21:18:33.478948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":7,"addr":"172.21.15.52:6835/3672156966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3672156966},{"type":"v1","addr":"172.21.15.52:6835","nonce":3672156966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7869,"qdb_cluster":[7869]},"id":10}]} 2024-09-10T21:18:37.511 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 95 2024-09-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.508+0000 7fa61aa00640 1 -- 172.21.15.16:0/3662385225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5f00725e0 msgr2=0x7fa5f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.508+0000 7fa61aa00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5f00725e0 0x7fa5f0074aa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa6140ca2c0 tx=0x7fa5fc009040 comp rx=0 tx=0).stop 2024-09-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.508+0000 7fa61aa00640 1 -- 172.21.15.16:0/3662385225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61406e640 msgr2=0x7fa6141fb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.508+0000 7fa61aa00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61406e640 0x7fa6141fb800 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa60401cd30 tx=0x7fa6040a6520 comp rx=0 tx=0).stop 2024-09-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.508+0000 7fa61aa00640 1 -- 172.21.15.16:0/3662385225 shutdown_connections 2024-09-10T21:18:37.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.508+0000 7fa61aa00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa5f00725e0 0x7fa5f0074aa0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.508+0000 7fa61aa00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6141599d0 0x7fa614165570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.508+0000 7fa61aa00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6140726b0 0x7fa614157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.508+0000 7fa61aa00640 1 --2- 172.21.15.16:0/3662385225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61406e640 0x7fa6141fb800 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.508+0000 7fa61aa00640 1 -- 172.21.15.16:0/3662385225 >> 172.21.15.16:0/3662385225 conn(0x7fa6140cfb20 msgr2=0x7fa6140d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.508+0000 7fa61aa00640 1 -- 172.21.15.16:0/3662385225 shutdown_connections 2024-09-10T21:18:37.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:37.508+0000 7fa61aa00640 1 -- 172.21.15.16:0/3662385225 wait complete. 2024-09-10T21:18:37.533 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-10T21:18:37.705 INFO:teuthology.orchestra.run.smithi052.stdout:[{"id":7426,"entity":{"name":{"type":"client","num":7426},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":827843831}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":12.027861259,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9308360219339731,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7426 192.168.0.1:0/827843831","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi052","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":7414,"entity":{"name":{"type":"client","num":7414},"addr":{"type":"any","addr":"172.21.15.16:0","nonce":2073476252}},"state":"open","num_leases":400,"num_caps":201,"request_load_avg":384,"uptime":13.187847790999999,"requests_in_flight":0,"num_completed_requests":399,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1515.2674606987187,"halflife":60},"release_caps":{"value":199.55964904303892,"halflife":60},"recall_caps_throttle":{"value":478.06939798237738,"halflife":1.5},"recall_caps_throttle2o":{"value":194.11678079965284,"halflife":0.5},"session_cache_liveness":{"value":1177.5611785977792,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7414 172.21.15.16:0/2073476252","completed_requests":[{"tid":29,"created_ino":"0x10000000001"},{"tid":30,"created_ino":"0x100000001f6"},{"tid":31,"created_ino":"0x100000001f7"},{"tid":32,"created_ino":"0x100000001f8"},{"tid":33,"created_ino":"0x100000001f9"},{"tid":34,"created_ino":"0x100000001fa"},{"tid":35,"created_ino":"0x100000001fb"},{"tid":36,"created_ino":"0x100000001fc"},{"tid":37,"created_ino":"0x100000001fd"},{"tid":38,"created_ino":"0x100000001fe"},{"tid":39,"created_ino":"0x100000001ff"},{"tid":40,"created_ino":"0x10000000200"},{"tid":41,"created_ino":"0x10000000201"},{"tid":42,"created_ino":"0x10000000202"},{"tid":43,"created_ino":"0x10000000203"},{"tid":44,"created_ino":"0x10000000204"},{"tid":45,"created_ino":"0x10000000205"},{"tid":46,"created_ino":"0x10000000206"},{"tid":47,"created_ino":"0x10000000207"},{"tid":48,"created_ino":"0x10000000208"},{"tid":49,"created_ino":"0x10000000209"},{"tid":50,"created_ino":"0x1000000020a"},{"tid":51,"created_ino":"0x1000000020b"},{"tid":52,"created_ino":"0x1000000020c"},{"tid":53,"created_ino":"0x1000000020d"},{"tid":54,"created_ino":"0x1000000020e"},{"tid":55,"created_ino":"0x1000000020f"},{"tid":56,"created_ino":"0x10000000210"},{"tid":57,"created_ino":"0x10000000211"},{"tid":58,"created_ino":"0x10000000212"},{"tid":59,"created_ino":"0x10000000213"},{"tid":60,"created_ino":"0x10000000214"},{"tid":61,"created_ino":"0x10000000215"},{"tid":62,"created_ino":"0x10000000216"},{"tid":63,"created_ino":"0x10000000217"},{"tid":64,"created_ino":"0x10000000218"},{"tid":65,"created_ino":"0x10000000219"},{"tid":66,"created_ino":"0x1000000021a"},{"tid":67,"created_ino":"0x1000000021b"},{"tid":68,"created_ino":"0x1000000021c"},{"tid":69,"created_ino":"0x1000000021d"},{"tid":70,"created_ino":"0x1000000021e"},{"tid":71,"created_ino":"0x1000000021f"},{"tid":72,"created_ino":"0x10000000220"},{"tid":73,"created_ino":"0x10000000221"},{"tid":74,"created_ino":"0x10000000222"},{"tid":75,"created_ino":"0x10000000223"},{"tid":76,"created_ino":"0x10000000224"},{"tid":77,"created_ino":"0x10000000225"},{"tid":78,"created_ino":"0x10000000226"},{"tid":79,"created_ino":"0x10000000227"},{"tid":80,"created_ino":"0x10000000228"},{"tid":81,"created_ino":"0x10000000229"},{"tid":82,"created_ino":"0x1000000022a"},{"tid":83,"created_ino":"0x1000000022b"},{"tid":84,"created_ino":"0x1000000022c"},{"tid":85,"created_ino":"0x1000000022d"},{"tid":86,"created_ino":"0x1000000022e"},{"tid":87,"created_ino":"0x1000000022f"},{"tid":88,"created_ino":"0x10000000230"},{"tid":89,"created_ino":"0x10000000231"},{"tid":90,"created_ino":"0x10000000232"},{"tid":91,"created_ino":"0x10000000233"},{"tid":92,"created_ino":"0x10000000234"},{"tid":93,"created_ino":"0x10000000235"},{"tid":94,"created_ino":"0x10000000236"},{"tid":95,"created_ino":"0x10000000237"},{"tid":96,"created_ino":"0x10000000238"},{"tid":97,"created_ino":"0x10000000239"},{"tid":98,"created_ino":"0x1000000023a"},{"tid":99,"created_ino":"0x1000000023b"},{"tid":100,"created_ino":"0x1000000023c"},{"tid":101,"created_ino":"0x1000000023d"},{"tid":102,"created_ino":"0x1000000023e"},{"tid":103,"created_ino":"0x1000000023f"},{"tid":104,"created_ino":"0x10000000240"},{"tid":105,"created_ino":"0x10000000241"},{"tid":106,"created_ino":"0x10000000242"},{"tid":107,"created_ino":"0x10000000243"},{"tid":108,"created_ino":"0x10000000244"},{"tid":109,"created_ino":"0x10000000245"},{"tid":110,"created_ino":"0x10000000246"},{"tid":111,"created_ino":"0x10000000247"},{"tid":112,"created_ino":"0x10000000248"},{"tid":113,"created_ino":"0x10000000249"},{"tid":114,"created_ino":"0x1000000024a"},{"tid":115,"created_ino":"0x1000000024b"},{"tid":116,"created_ino":"0x1000000024c"},{"tid":117,"created_ino":"0x1000000024d"},{"tid":118,"created_ino":"0x1000000024e"},{"tid":119,"created_ino":"0x1000000024f"},{"tid":120,"created_ino":"0x10000000250"},{"tid":121,"created_ino":"0x10000000251"},{"tid":122,"created_ino":"0x10000000252"},{"tid":123,"created_ino":"0x10000000253"},{"tid":124,"created_ino":"0x10000000254"},{"tid":125,"created_ino":"0x10000000255"},{"tid":126,"created_ino":"0x10000000256"},{"tid":127,"created_ino":"0x10000000257"},{"tid":128,"created_ino":"0x10000000258"},{"tid":129,"created_ino":"0x10000000259"},{"tid":130,"created_ino":"0x1000000025a"},{"tid":131,"created_ino":"0x1000000025b"},{"tid":132,"created_ino":"0x1000000025c"},{"tid":133,"created_ino":"0x1000000025d"},{"tid":134,"created_ino":"0x1000000025e"},{"tid":135,"created_ino":"0x1000000025f"},{"tid":136,"created_ino":"0x10000000260"},{"tid":137,"created_ino":"0x10000000261"},{"tid":138,"created_ino":"0x10000000262"},{"tid":139,"created_ino":"0x10000000263"},{"tid":140,"created_ino":"0x10000000264"},{"tid":141,"created_ino":"0x10000000265"},{"tid":142,"created_ino":"0x10000000266"},{"tid":143,"created_ino":"0x10000000267"},{"tid":144,"created_ino":"0x10000000268"},{"tid":145,"created_ino":"0x10000000269"},{"tid":146,"created_ino":"0x1000000026a"},{"tid":147,"created_ino":"0x1000000026b"},{"tid":148,"created_ino":"0x1000000026c"},{"tid":149,"created_ino":"0x1000000026d"},{"tid":150,"created_ino":"0x1000000026e"},{"tid":151,"created_ino":"0x1000000026f"},{"tid":152,"created_ino":"0x10000000270"},{"tid":153,"created_ino":"0x10000000271"},{"tid":154,"created_ino":"0x10000000272"},{"tid":155,"created_ino":"0x10000000273"},{"tid":156,"created_ino":"0x10000000274"},{"tid":157,"created_ino":"0x10000000275"},{"tid":158,"created_ino":"0x10000000276"},{"tid":159,"created_ino":"0x10000000277"},{"tid":160,"created_ino":"0x10000000278"},{"tid":161,"created_ino":"0x10000000279"},{"tid":162,"created_ino":"0x1000000027a"},{"tid":163,"created_ino":"0x1000000027b"},{"tid":164,"created_ino":"0x1000000027c"},{"tid":165,"created_ino":"0x1000000027d"},{"tid":166,"created_ino":"0x1000000027e"},{"tid":167,"created_ino":"0x1000000027f"},{"tid":168,"created_ino":"0x10000000280"},{"tid":169,"created_ino":"0x10000000281"},{"tid":170,"created_ino":"0x10000000282"},{"tid":171,"created_ino":"0x10000000283"},{"tid":172,"created_ino":"0x10000000284"},{"tid":173,"created_ino":"0x10000000285"},{"tid":174,"created_ino":"0x10000000286"},{"tid":175,"created_ino":"0x10000000287"},{"tid":176,"created_ino":"0x10000000288"},{"tid":177,"created_ino":"0x10000000289"},{"tid":178,"created_ino":"0x1000000028a"},{"tid":179,"created_ino":"0x1000000028b"},{"tid":180,"created_ino":"0x1000000028c"},{"tid":181,"created_ino":"0x1000000028d"},{"tid":182,"created_ino":"0x1000000028e"},{"tid":183,"created_ino":"0x1000000028f"},{"tid":184,"created_ino":"0x10000000290"},{"tid":185,"created_ino":"0x10000000291"},{"tid":186,"created_ino":"0x10000000292"},{"tid":187,"created_ino":"0x10000000293"},{"tid":188,"created_ino":"0x10000000294"},{"tid":189,"created_ino":"0x10000000295"},{"tid":190,"created_ino":"0x10000000296"},{"tid":191,"created_ino":"0x10000000297"},{"tid":192,"created_ino":"0x10000000298"},{"tid":193,"created_ino":"0x10000000299"},{"tid":194,"created_ino":"0x1000000029a"},{"tid":195,"created_ino":"0x1000000029b"},{"tid":196,"created_ino":"0x1000000029c"},{"tid":197,"created_ino":"0x1000000029d"},{"tid":198,"created_ino":"0x1000000029e"},{"tid":199,"created_ino":"0x1000000029f"},{"tid":200,"created_ino":"0x100000002a0"},{"tid":201,"created_ino":"0x100000002a1"},{"tid":202,"created_ino":"0x100000002a2"},{"tid":203,"created_ino":"0x100000002a3"},{"tid":204,"created_ino":"0x100000002a4"},{"tid":205,"created_ino":"0x100000002a5"},{"tid":206,"created_ino":"0x100000002a6"},{"tid":207,"created_ino":"0x100000002a7"},{"tid":208,"created_ino":"0x100000002a8"},{"tid":209,"created_ino":"0x100000002a9"},{"tid":210,"created_ino":"0x100000002aa"},{"tid":211,"created_ino":"0x100000002ab"},{"tid":212,"created_ino":"0x100000002ac"},{"tid":213,"created_ino":"0x100000002ad"},{"tid":214,"created_ino":"0x100000002ae"},{"tid":215,"created_ino":"0x100000002af"},{"tid":216,"created_ino":"0x100000002b0"},{"tid":217,"created_ino":"0x100000002b1"},{"tid":218,"created_ino":"0x100000002b2"},{"tid":219,"created_ino":"0x100000002b3"},{"tid":220,"created_ino":"0x100000002b4"},{"tid":221,"created_ino":"0x100000002b5"},{"tid":222,"created_ino":"0x100000002b6"},{"tid":223,"created_ino":"0x100000002b7"},{"tid":224,"created_ino":"0x100000002b8"},{"tid":225,"created_ino":"0x100000002b9"},{"tid":226,"created_ino":"0x100000002ba"},{"tid":227,"created_ino":"0x100000002bb"},{"tid":228,"created_ino":"0x100000002bc"},{"tid":229,"created_ino":"0x100000002bd"},{"tid":230,"created_ino":"0x100000002be"},{"tid":231,"created_ino":"0x100000002bf"},{"tid":232,"created_ino":"0x100000002c0"},{"tid":233,"created_ino":"0x100000002c1"},{"tid":234,"created_ino":"0x100000002c2"},{"tid":235,"created_ino":"0x100000002c3"},{"tid":236,"created_ino":"0x100000002c4"},{"tid":237,"created_ino":"0x100000002c5"},{"tid":238,"created_ino":"0x100000002c6"},{"tid":239,"created_ino":"0x100000002c7"},{"tid":240,"created_ino":"0x100000002c8"},{"tid":241,"created_ino":"0x100000002c9"},{"tid":242,"created_ino":"0x100000002ca"},{"tid":243,"created_ino":"0x100000002cb"},{"tid":244,"created_ino":"0x100000002cc"},{"tid":245,"created_ino":"0x100000002cd"},{"tid":246,"created_ino":"0x100000002ce"},{"tid":247,"created_ino":"0x100000002cf"},{"tid":248,"created_ino":"0x100000002d0"},{"tid":249,"created_ino":"0x100000002d1"},{"tid":250,"created_ino":"0x100000002d2"},{"tid":251,"created_ino":"0x100000002d3"},{"tid":252,"created_ino":"0x100000002d4"},{"tid":253,"created_ino":"0x100000002d5"},{"tid":254,"created_ino":"0x100000002d6"},{"tid":255,"created_ino":"0x100000002d7"},{"tid":256,"created_ino":"0x100000002d8"},{"tid":257,"created_ino":"0x100000002d9"},{"tid":258,"created_ino":"0x100000002da"},{"tid":259,"created_ino":"0x100000002db"},{"tid":260,"created_ino":"0x100000002dc"},{"tid":261,"created_ino":"0x100000002dd"},{"tid":262,"created_ino":"0x100000002de"},{"tid":263,"created_ino":"0x100000002df"},{"tid":264,"created_ino":"0x100000002e0"},{"tid":265,"created_ino":"0x100000002e1"},{"tid":266,"created_ino":"0x100000002e2"},{"tid":267,"created_ino":"0x100000002e3"},{"tid":268,"created_ino":"0x100000002e4"},{"tid":269,"created_ino":"0x100000002e5"},{"tid":270,"created_ino":"0x100000002e6"},{"tid":271,"created_ino":"0x100000002e7"},{"tid":272,"created_ino":"0x100000002e8"},{"tid":273,"created_ino":"0x100000002e9"},{"tid":274,"created_ino":"0x100000002ea"},{"tid":275,"created_ino":"0x100000002eb"},{"tid":276,"created_ino":"0x100000002ec"},{"tid":277,"created_ino":"0x100000002ed"},{"tid":278,"created_ino":"0x100000002ee"},{"tid":279,"created_ino":"0x100000002ef"},{"tid":280,"created_ino":"0x100000002f0"},{"tid":281,"created_ino":"0x100000002f1"},{"tid":282,"created_ino":"0x100000002f2"},{"tid":283,"created_ino":"0x100000002f3"},{"tid":284,"created_ino":"0x100000002f4"},{"tid":285,"created_ino":"0x100000002f5"},{"tid":286,"created_ino":"0x100000002f6"},{"tid":287,"created_ino":"0x100000002f7"},{"tid":288,"created_ino":"0x100000002f8"},{"tid":289,"created_ino":"0x100000002f9"},{"tid":290,"created_ino":"0x100000002fa"},{"tid":291,"created_ino":"0x100000002fb"},{"tid":292,"created_ino":"0x100000002fc"},{"tid":293,"created_ino":"0x100000002fd"},{"tid":294,"created_ino":"0x100000002fe"},{"tid":295,"created_ino":"0x100000002ff"},{"tid":296,"created_ino":"0x10000000300"},{"tid":297,"created_ino":"0x10000000301"},{"tid":298,"created_ino":"0x10000000302"},{"tid":299,"created_ino":"0x10000000303"},{"tid":300,"created_ino":"0x10000000304"},{"tid":301,"created_ino":"0x10000000305"},{"tid":302,"created_ino":"0x10000000306"},{"tid":303,"created_ino":"0x10000000307"},{"tid":304,"created_ino":"0x10000000308"},{"tid":305,"created_ino":"0x10000000309"},{"tid":306,"created_ino":"0x1000000030a"},{"tid":307,"created_ino":"0x1000000030b"},{"tid":308,"created_ino":"0x1000000030c"},{"tid":309,"created_ino":"0x1000000030d"},{"tid":310,"created_ino":"0x1000000030e"},{"tid":311,"created_ino":"0x1000000030f"},{"tid":312,"created_ino":"0x10000000310"},{"tid":313,"created_ino":"0x10000000311"},{"tid":314,"created_ino":"0x10000000312"},{"tid":315,"created_ino":"0x10000000313"},{"tid":316,"created_ino":"0x10000000314"},{"tid":317,"created_ino":"0x10000000315"},{"tid":318,"created_ino":"0x10000000316"},{"tid":319,"created_ino":"0x10000000317"},{"tid":320,"created_ino":"0x10000000318"},{"tid":321,"created_ino":"0x10000000319"},{"tid":322,"created_ino":"0x1000000031a"},{"tid":323,"created_ino":"0x1000000031b"},{"tid":324,"created_ino":"0x1000000031c"},{"tid":325,"created_ino":"0x1000000031d"},{"tid":326,"created_ino":"0x1000000031e"},{"tid":327,"created_ino":"0x1000000031f"},{"tid":328,"created_ino":"0x10000000320"},{"tid":329,"created_ino":"0x10000000321"},{"tid":330,"created_ino":"0x10000000322"},{"tid":331,"created_ino":"0x10000000323"},{"tid":332,"created_ino":"0x10000000324"},{"tid":333,"created_ino":"0x10000000325"},{"tid":334,"created_ino":"0x10000000326"},{"tid":335,"created_ino":"0x10000000327"},{"tid":336,"created_ino":"0x10000000328"},{"tid":337,"created_ino":"0x10000000329"},{"tid":338,"created_ino":"0x1000000032a"},{"tid":339,"created_ino":"0x1000000032b"},{"tid":340,"created_ino":"0x1000000032c"},{"tid":341,"created_ino":"0x1000000032d"},{"tid":342,"created_ino":"0x1000000032e"},{"tid":343,"created_ino":"0x1000000032f"},{"tid":344,"created_ino":"0x10000000330"},{"tid":345,"created_ino":"0x10000000331"},{"tid":346,"created_ino":"0x10000000332"},{"tid":347,"created_ino":"0x10000000333"},{"tid":348,"created_ino":"0x10000000334"},{"tid":349,"created_ino":"0x10000000335"},{"tid":350,"created_ino":"0x10000000336"},{"tid":351,"created_ino":"0x10000000337"},{"tid":352,"created_ino":"0x10000000338"},{"tid":353,"created_ino":"0x10000000339"},{"tid":354,"created_ino":"0x1000000033a"},{"tid":355,"created_ino":"0x1000000033b"},{"tid":356,"created_ino":"0x1000000033c"},{"tid":357,"created_ino":"0x1000000033d"},{"tid":358,"created_ino":"0x1000000033e"},{"tid":359,"created_ino":"0x1000000033f"},{"tid":360,"created_ino":"0x10000000340"},{"tid":361,"created_ino":"0x10000000341"},{"tid":362,"created_ino":"0x10000000342"},{"tid":363,"created_ino":"0x10000000343"},{"tid":364,"created_ino":"0x10000000344"},{"tid":365,"created_ino":"0x10000000345"},{"tid":366,"created_ino":"0x10000000346"},{"tid":367,"created_ino":"0x10000000347"},{"tid":368,"created_ino":"0x10000000348"},{"tid":369,"created_ino":"0x10000000349"},{"tid":370,"created_ino":"0x1000000034a"},{"tid":371,"created_ino":"0x1000000034b"},{"tid":372,"created_ino":"0x1000000034c"},{"tid":373,"created_ino":"0x1000000034d"},{"tid":374,"created_ino":"0x1000000034e"},{"tid":375,"created_ino":"0x1000000034f"},{"tid":376,"created_ino":"0x10000000350"},{"tid":377,"created_ino":"0x10000000351"},{"tid":378,"created_ino":"0x10000000352"},{"tid":379,"created_ino":"0x10000000353"},{"tid":380,"created_ino":"0x10000000354"},{"tid":381,"created_ino":"0x10000000355"},{"tid":382,"created_ino":"0x10000000356"},{"tid":383,"created_ino":"0x10000000357"},{"tid":384,"created_ino":"0x10000000358"},{"tid":385,"created_ino":"0x10000000359"},{"tid":386,"created_ino":"0x1000000035a"},{"tid":387,"created_ino":"0x1000000035b"},{"tid":388,"created_ino":"0x1000000035c"},{"tid":389,"created_ino":"0x1000000035d"},{"tid":390,"created_ino":"0x1000000035e"},{"tid":391,"created_ino":"0x1000000035f"},{"tid":392,"created_ino":"0x10000000360"},{"tid":393,"created_ino":"0x10000000361"},{"tid":394,"created_ino":"0x10000000362"},{"tid":395,"created_ino":"0x10000000363"},{"tid":396,"created_ino":"0x10000000364"},{"tid":397,"created_ino":"0x10000000365"},{"tid":398,"created_ino":"0x10000000366"},{"tid":399,"created_ino":"0x10000000367"},{"tid":400,"created_ino":"0x10000000368"},{"tid":401,"created_ino":"0x10000000369"},{"tid":402,"created_ino":"0x1000000036a"},{"tid":403,"created_ino":"0x1000000036b"},{"tid":404,"created_ino":"0x1000000036c"},{"tid":405,"created_ino":"0x1000000036d"},{"tid":406,"created_ino":"0x1000000036e"},{"tid":407,"created_ino":"0x1000000036f"},{"tid":408,"created_ino":"0x10000000370"},{"tid":409,"created_ino":"0x10000000371"},{"tid":410,"created_ino":"0x10000000372"},{"tid":411,"created_ino":"0x10000000373"},{"tid":412,"created_ino":"0x10000000374"},{"tid":413,"created_ino":"0x10000000375"},{"tid":414,"created_ino":"0x10000000376"},{"tid":415,"created_ino":"0x10000000377"},{"tid":416,"created_ino":"0x10000000378"},{"tid":417,"created_ino":"0x10000000379"},{"tid":418,"created_ino":"0x1000000037a"},{"tid":419,"created_ino":"0x1000000037b"},{"tid":420,"created_ino":"0x1000000037c"},{"tid":421,"created_ino":"0x1000000037d"},{"tid":422,"created_ino":"0x1000000037e"},{"tid":423,"created_ino":"0x1000000037f"},{"tid":424,"created_ino":"0x10000000380"},{"tid":425,"created_ino":"0x10000000381"},{"tid":426,"created_ino":"0x10000000382"},{"tid":427,"created_ino":"0x10000000383"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000384","length":603}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-10T21:18:37.708 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi052", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 827843831, "type": "any" }, "name": { "num": 7426, "type": "client" } }, "id": 7426, "inst": "client.7426 192.168.0.1:0/827843831", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.930836021933973 }, "state": "open", "uptime": 12.027861259 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 29 }, { "created_ino": "0x100000001f6", "tid": 30 }, { "created_ino": "0x100000001f7", "tid": 31 }, { "created_ino": "0x100000001f8", "tid": 32 }, { "created_ino": "0x100000001f9", "tid": 33 }, { "created_ino": "0x100000001fa", "tid": 34 }, { "created_ino": "0x100000001fb", "tid": 35 }, { "created_ino": "0x100000001fc", "tid": 36 }, { "created_ino": "0x100000001fd", "tid": 37 }, { "created_ino": "0x100000001fe", "tid": 38 }, { "created_ino": "0x100000001ff", "tid": 39 }, { "created_ino": "0x10000000200", "tid": 40 }, { "created_ino": "0x10000000201", "tid": 41 }, { "created_ino": "0x10000000202", "tid": 42 }, { "created_ino": "0x10000000203", "tid": 43 }, { "created_ino": "0x10000000204", "tid": 44 }, { "created_ino": "0x10000000205", "tid": 45 }, { "created_ino": "0x10000000206", "tid": 46 }, { "created_ino": "0x10000000207", "tid": 47 }, { "created_ino": "0x10000000208", "tid": 48 }, { "created_ino": "0x10000000209", "tid": 49 }, { "created_ino": "0x1000000020a", "tid": 50 }, { "created_ino": "0x1000000020b", "tid": 51 }, { "created_ino": "0x1000000020c", "tid": 52 }, { "created_ino": "0x1000000020d", "tid": 53 }, { "created_ino": "0x1000000020e", "tid": 54 }, { "created_ino": "0x1000000020f", "tid": 55 }, { "created_ino": "0x10000000210", "tid": 56 }, { "created_ino": "0x10000000211", "tid": 57 }, { "created_ino": "0x10000000212", "tid": 58 }, { "created_ino": "0x10000000213", "tid": 59 }, { "created_ino": "0x10000000214", "tid": 60 }, { "created_ino": "0x10000000215", "tid": 61 }, { "created_ino": "0x10000000216", "tid": 62 }, { "created_ino": "0x10000000217", "tid": 63 }, { "created_ino": "0x10000000218", "tid": 64 }, { "created_ino": "0x10000000219", "tid": 65 }, { "created_ino": "0x1000000021a", "tid": 66 }, { "created_ino": "0x1000000021b", "tid": 67 }, { "created_ino": "0x1000000021c", "tid": 68 }, { "created_ino": "0x1000000021d", "tid": 69 }, { "created_ino": "0x1000000021e", "tid": 70 }, { "created_ino": "0x1000000021f", "tid": 71 }, { "created_ino": "0x10000000220", "tid": 72 }, { "created_ino": "0x10000000221", "tid": 73 }, { "created_ino": "0x10000000222", "tid": 74 }, { "created_ino": "0x10000000223", "tid": 75 }, { "created_ino": "0x10000000224", "tid": 76 }, { "created_ino": "0x10000000225", "tid": 77 }, { "created_ino": "0x10000000226", "tid": 78 }, { "created_ino": "0x10000000227", "tid": 79 }, { "created_ino": "0x10000000228", "tid": 80 }, { "created_ino": "0x10000000229", "tid": 81 }, { "created_ino": "0x1000000022a", "tid": 82 }, { "created_ino": "0x1000000022b", "tid": 83 }, { "created_ino": "0x1000000022c", "tid": 84 }, { "created_ino": "0x1000000022d", "tid": 85 }, { "created_ino": "0x1000000022e", "tid": 86 }, { "created_ino": "0x1000000022f", "tid": 87 }, { "created_ino": "0x10000000230", "tid": 88 }, { "created_ino": "0x10000000231", "tid": 89 }, { "created_ino": "0x10000000232", "tid": 90 }, { "created_ino": "0x10000000233", "tid": 91 }, { "created_ino": "0x10000000234", "tid": 92 }, { "created_ino": "0x10000000235", "tid": 93 }, { "created_ino": "0x10000000236", "tid": 94 }, { "created_ino": "0x10000000237", "tid": 95 }, { "created_ino": "0x10000000238", "tid": 96 }, { "created_ino": "0x10000000239", "tid": 97 }, { "created_ino": "0x1000000023a", "tid": 98 }, { "created_ino": "0x1000000023b", "tid": 99 }, { "created_ino": "0x1000000023c", "tid": 100 }, { "created_ino": "0x1000000023d", "tid": 101 }, { "created_ino": "0x1000000023e", "tid": 102 }, { "created_ino": "0x1000000023f", "tid": 103 }, { "created_ino": "0x10000000240", "tid": 104 }, { "created_ino": "0x10000000241", "tid": 105 }, { "created_ino": "0x10000000242", "tid": 106 }, { "created_ino": "0x10000000243", "tid": 107 }, { "created_ino": "0x10000000244", "tid": 108 }, { "created_ino": "0x10000000245", "tid": 109 }, { "created_ino": "0x10000000246", "tid": 110 }, { "created_ino": "0x10000000247", "tid": 111 }, { "created_ino": "0x10000000248", "tid": 112 }, { "created_ino": "0x10000000249", "tid": 113 }, { "created_ino": "0x1000000024a", "tid": 114 }, { "created_ino": "0x1000000024b", "tid": 115 }, { "created_ino": "0x1000000024c", "tid": 116 }, { "created_ino": "0x1000000024d", "tid": 117 }, { "created_ino": "0x1000000024e", "tid": 118 }, { "created_ino": "0x1000000024f", "tid": 119 }, { "created_ino": "0x10000000250", "tid": 120 }, { "created_ino": "0x10000000251", "tid": 121 }, { "created_ino": "0x10000000252", "tid": 122 }, { "created_ino": "0x10000000253", "tid": 123 }, { "created_ino": "0x10000000254", "tid": 124 }, { "created_ino": "0x10000000255", "tid": 125 }, { "created_ino": "0x10000000256", "tid": 126 }, { "created_ino": "0x10000000257", "tid": 127 }, { "created_ino": "0x10000000258", "tid": 128 }, { "created_ino": "0x10000000259", "tid": 129 }, { "created_ino": "0x1000000025a", "tid": 130 }, { "created_ino": "0x1000000025b", "tid": 131 }, { "created_ino": "0x1000000025c", "tid": 132 }, { "created_ino": "0x1000000025d", "tid": 133 }, { "created_ino": "0x1000000025e", "tid": 134 }, { "created_ino": "0x1000000025f", "tid": 135 }, { "created_ino": "0x10000000260", "tid": 136 }, { "created_ino": "0x10000000261", "tid": 137 }, { "created_ino": "0x10000000262", "tid": 138 }, { "created_ino": "0x10000000263", "tid": 139 }, { "created_ino": "0x10000000264", "tid": 140 }, { "created_ino": "0x10000000265", "tid": 141 }, { "created_ino": "0x10000000266", "tid": 142 }, { "created_ino": "0x10000000267", "tid": 143 }, { "created_ino": "0x10000000268", "tid": 144 }, { "created_ino": "0x10000000269", "tid": 145 }, { "created_ino": "0x1000000026a", "tid": 146 }, { "created_ino": "0x1000000026b", "tid": 147 }, { "created_ino": "0x1000000026c", "tid": 148 }, { "created_ino": "0x1000000026d", "tid": 149 }, { "created_ino": "0x1000000026e", "tid": 150 }, { "created_ino": "0x1000000026f", "tid": 151 }, { "created_ino": "0x10000000270", "tid": 152 }, { "created_ino": "0x10000000271", "tid": 153 }, { "created_ino": "0x10000000272", "tid": 154 }, { "created_ino": "0x10000000273", "tid": 155 }, { "created_ino": "0x10000000274", "tid": 156 }, { "created_ino": "0x10000000275", "tid": 157 }, { "created_ino": "0x10000000276", "tid": 158 }, { "created_ino": "0x10000000277", "tid": 159 }, { "created_ino": "0x10000000278", "tid": 160 }, { "created_ino": "0x10000000279", "tid": 161 }, { "created_ino": "0x1000000027a", "tid": 162 }, { "created_ino": "0x1000000027b", "tid": 163 }, { "created_ino": "0x1000000027c", "tid": 164 }, { "created_ino": "0x1000000027d", "tid": 165 }, { "created_ino": "0x1000000027e", "tid": 166 }, { "created_ino": "0x1000000027f", "tid": 167 }, { "created_ino": "0x10000000280", "tid": 168 }, { "created_ino": "0x10000000281", "tid": 169 }, { "created_ino": "0x10000000282", "tid": 170 }, { "created_ino": "0x10000000283", "tid": 171 }, { "created_ino": "0x10000000284", "tid": 172 }, { "created_ino": "0x10000000285", "tid": 173 }, { "created_ino": "0x10000000286", "tid": 174 }, { "created_ino": "0x10000000287", "tid": 175 }, { "created_ino": "0x10000000288", "tid": 176 }, { "created_ino": "0x10000000289", "tid": 177 }, { "created_ino": "0x1000000028a", "tid": 178 }, { "created_ino": "0x1000000028b", "tid": 179 }, { "created_ino": "0x1000000028c", "tid": 180 }, { "created_ino": "0x1000000028d", "tid": 181 }, { "created_ino": "0x1000000028e", "tid": 182 }, { "created_ino": "0x1000000028f", "tid": 183 }, { "created_ino": "0x10000000290", "tid": 184 }, { "created_ino": "0x10000000291", "tid": 185 }, { "created_ino": "0x10000000292", "tid": 186 }, { "created_ino": "0x10000000293", "tid": 187 }, { "created_ino": "0x10000000294", "tid": 188 }, { "created_ino": "0x10000000295", "tid": 189 }, { "created_ino": "0x10000000296", "tid": 190 }, { "created_ino": "0x10000000297", "tid": 191 }, { "created_ino": "0x10000000298", "tid": 192 }, { "created_ino": "0x10000000299", "tid": 193 }, { "created_ino": "0x1000000029a", "tid": 194 }, { "created_ino": "0x1000000029b", "tid": 195 }, { "created_ino": "0x1000000029c", "tid": 196 }, { "created_ino": "0x1000000029d", "tid": 197 }, { "created_ino": "0x1000000029e", "tid": 198 }, { "created_ino": "0x1000000029f", "tid": 199 }, { "created_ino": "0x100000002a0", "tid": 200 }, { "created_ino": "0x100000002a1", "tid": 201 }, { "created_ino": "0x100000002a2", "tid": 202 }, { "created_ino": "0x100000002a3", "tid": 203 }, { "created_ino": "0x100000002a4", "tid": 204 }, { "created_ino": "0x100000002a5", "tid": 205 }, { "created_ino": "0x100000002a6", "tid": 206 }, { "created_ino": "0x100000002a7", "tid": 207 }, { "created_ino": "0x100000002a8", "tid": 208 }, { "created_ino": "0x100000002a9", "tid": 209 }, { "created_ino": "0x100000002aa", "tid": 210 }, { "created_ino": "0x100000002ab", "tid": 211 }, { "created_ino": "0x100000002ac", "tid": 212 }, { "created_ino": "0x100000002ad", "tid": 213 }, { "created_ino": "0x100000002ae", "tid": 214 }, { "created_ino": "0x100000002af", "tid": 215 }, { "created_ino": "0x100000002b0", "tid": 216 }, { "created_ino": "0x100000002b1", "tid": 217 }, { "created_ino": "0x100000002b2", "tid": 218 }, { "created_ino": "0x100000002b3", "tid": 219 }, { "created_ino": "0x100000002b4", "tid": 220 }, { "created_ino": "0x100000002b5", "tid": 221 }, { "created_ino": "0x100000002b6", "tid": 222 }, { "created_ino": "0x100000002b7", "tid": 223 }, { "created_ino": "0x100000002b8", "tid": 224 }, { "created_ino": "0x100000002b9", "tid": 225 }, { "created_ino": "0x100000002ba", "tid": 226 }, { "created_ino": "0x100000002bb", "tid": 227 }, { "created_ino": "0x100000002bc", "tid": 228 }, { "created_ino": "0x100000002bd", "tid": 229 }, { "created_ino": "0x100000002be", "tid": 230 }, { "created_ino": "0x100000002bf", "tid": 231 }, { "created_ino": "0x100000002c0", "tid": 232 }, { "created_ino": "0x100000002c1", "tid": 233 }, { "created_ino": "0x100000002c2", "tid": 234 }, { "created_ino": "0x100000002c3", "tid": 235 }, { "created_ino": "0x100000002c4", "tid": 236 }, { "created_ino": "0x100000002c5", "tid": 237 }, { "created_ino": "0x100000002c6", "tid": 238 }, { "created_ino": "0x100000002c7", "tid": 239 }, { "created_ino": "0x100000002c8", "tid": 240 }, { "created_ino": "0x100000002c9", "tid": 241 }, { "created_ino": "0x100000002ca", "tid": 242 }, { "created_ino": "0x100000002cb", "tid": 243 }, { "created_ino": "0x100000002cc", "tid": 244 }, { "created_ino": "0x100000002cd", "tid": 245 }, { "created_ino": "0x100000002ce", "tid": 246 }, { "created_ino": "0x100000002cf", "tid": 247 }, { "created_ino": "0x100000002d0", "tid": 248 }, { "created_ino": "0x100000002d1", "tid": 249 }, { "created_ino": "0x100000002d2", "tid": 250 }, { "created_ino": "0x100000002d3", "tid": 251 }, { "created_ino": "0x100000002d4", "tid": 252 }, { "created_ino": "0x100000002d5", "tid": 253 }, { "created_ino": "0x100000002d6", "tid": 254 }, { "created_ino": "0x100000002d7", "tid": 255 }, { "created_ino": "0x100000002d8", "tid": 256 }, { "created_ino": "0x100000002d9", "tid": 257 }, { "created_ino": "0x100000002da", "tid": 258 }, { "created_ino": "0x100000002db", "tid": 259 }, { "created_ino": "0x100000002dc", "tid": 260 }, { "created_ino": "0x100000002dd", "tid": 261 }, { "created_ino": "0x100000002de", "tid": 262 }, { "created_ino": "0x100000002df", "tid": 263 }, { "created_ino": "0x100000002e0", "tid": 264 }, { "created_ino": "0x100000002e1", "tid": 265 }, { "created_ino": "0x100000002e2", "tid": 266 }, { "created_ino": "0x100000002e3", "tid": 267 }, { "created_ino": "0x100000002e4", "tid": 268 }, { "created_ino": "0x100000002e5", "tid": 269 }, { "created_ino": "0x100000002e6", "tid": 270 }, { "created_ino": "0x100000002e7", "tid": 271 }, { "created_ino": "0x100000002e8", "tid": 272 }, { "created_ino": "0x100000002e9", "tid": 273 }, { "created_ino": "0x100000002ea", "tid": 274 }, { "created_ino": "0x100000002eb", "tid": 275 }, { "created_ino": "0x100000002ec", "tid": 276 }, { "created_ino": "0x100000002ed", "tid": 277 }, { "created_ino": "0x100000002ee", "tid": 278 }, { "created_ino": "0x100000002ef", "tid": 279 }, { "created_ino": "0x100000002f0", "tid": 280 }, { "created_ino": "0x100000002f1", "tid": 281 }, { "created_ino": "0x100000002f2", "tid": 282 }, { "created_ino": "0x100000002f3", "tid": 283 }, { "created_ino": "0x100000002f4", "tid": 284 }, { "created_ino": "0x100000002f5", "tid": 285 }, { "created_ino": "0x100000002f6", "tid": 286 }, { "created_ino": "0x100000002f7", "tid": 287 }, { "created_ino": "0x100000002f8", "tid": 288 }, { "created_ino": "0x100000002f9", "tid": 289 }, { "created_ino": "0x100000002fa", "tid": 290 }, { "created_ino": "0x100000002fb", "tid": 291 }, { "created_ino": "0x100000002fc", "tid": 292 }, { "created_ino": "0x100000002fd", "tid": 293 }, { "created_ino": "0x100000002fe", "tid": 294 }, { "created_ino": "0x100000002ff", "tid": 295 }, { "created_ino": "0x10000000300", "tid": 296 }, { "created_ino": "0x10000000301", "tid": 297 }, { "created_ino": "0x10000000302", "tid": 298 }, { "created_ino": "0x10000000303", "tid": 299 }, { "created_ino": "0x10000000304", "tid": 300 }, { "created_ino": "0x10000000305", "tid": 301 }, { "created_ino": "0x10000000306", "tid": 302 }, { "created_ino": "0x10000000307", "tid": 303 }, { "created_ino": "0x10000000308", "tid": 304 }, { "created_ino": "0x10000000309", "tid": 305 }, { "created_ino": "0x1000000030a", "tid": 306 }, { "created_ino": "0x1000000030b", "tid": 307 }, { "created_ino": "0x1000000030c", "tid": 308 }, { "created_ino": "0x1000000030d", "tid": 309 }, { "created_ino": "0x1000000030e", "tid": 310 }, { "created_ino": "0x1000000030f", "tid": 311 }, { "created_ino": "0x10000000310", "tid": 312 }, { "created_ino": "0x10000000311", "tid": 313 }, { "created_ino": "0x10000000312", "tid": 314 }, { "created_ino": "0x10000000313", "tid": 315 }, { "created_ino": "0x10000000314", "tid": 316 }, { "created_ino": "0x10000000315", "tid": 317 }, { "created_ino": "0x10000000316", "tid": 318 }, { "created_ino": "0x10000000317", "tid": 319 }, { "created_ino": "0x10000000318", "tid": 320 }, { "created_ino": "0x10000000319", "tid": 321 }, { "created_ino": "0x1000000031a", "tid": 322 }, { "created_ino": "0x1000000031b", "tid": 323 }, { "created_ino": "0x1000000031c", "tid": 324 }, { "created_ino": "0x1000000031d", "tid": 325 }, { "created_ino": "0x1000000031e", "tid": 326 }, { "created_ino": "0x1000000031f", "tid": 327 }, { "created_ino": "0x10000000320", "tid": 328 }, { "created_ino": "0x10000000321", "tid": 329 }, { "created_ino": "0x10000000322", "tid": 330 }, { "created_ino": "0x10000000323", "tid": 331 }, { "created_ino": "0x10000000324", "tid": 332 }, { "created_ino": "0x10000000325", "tid": 333 }, { "created_ino": "0x10000000326", "tid": 334 }, { "created_ino": "0x10000000327", "tid": 335 }, { "created_ino": "0x10000000328", "tid": 336 }, { "created_ino": "0x10000000329", "tid": 337 }, { "created_ino": "0x1000000032a", "tid": 338 }, { "created_ino": "0x1000000032b", "tid": 339 }, { "created_ino": "0x1000000032c", "tid": 340 }, { "created_ino": "0x1000000032d", "tid": 341 }, { "created_ino": "0x1000000032e", "tid": 342 }, { "created_ino": "0x1000000032f", "tid": 343 }, { "created_ino": "0x10000000330", "tid": 344 }, { "created_ino": "0x10000000331", "tid": 345 }, { "created_ino": "0x10000000332", "tid": 346 }, { "created_ino": "0x10000000333", "tid": 347 }, { "created_ino": "0x10000000334", "tid": 348 }, { "created_ino": "0x10000000335", "tid": 349 }, { "created_ino": "0x10000000336", "tid": 350 }, { "created_ino": "0x10000000337", "tid": 351 }, { "created_ino": "0x10000000338", "tid": 352 }, { "created_ino": "0x10000000339", "tid": 353 }, { "created_ino": "0x1000000033a", "tid": 354 }, { "created_ino": "0x1000000033b", "tid": 355 }, { "created_ino": "0x1000000033c", "tid": 356 }, { "created_ino": "0x1000000033d", "tid": 357 }, { "created_ino": "0x1000000033e", "tid": 358 }, { "created_ino": "0x1000000033f", "tid": 359 }, { "created_ino": "0x10000000340", "tid": 360 }, { "created_ino": "0x10000000341", "tid": 361 }, { "created_ino": "0x10000000342", "tid": 362 }, { "created_ino": "0x10000000343", "tid": 363 }, { "created_ino": "0x10000000344", "tid": 364 }, { "created_ino": "0x10000000345", "tid": 365 }, { "created_ino": "0x10000000346", "tid": 366 }, { "created_ino": "0x10000000347", "tid": 367 }, { "created_ino": "0x10000000348", "tid": 368 }, { "created_ino": "0x10000000349", "tid": 369 }, { "created_ino": "0x1000000034a", "tid": 370 }, { "created_ino": "0x1000000034b", "tid": 371 }, { "created_ino": "0x1000000034c", "tid": 372 }, { "created_ino": "0x1000000034d", "tid": 373 }, { "created_ino": "0x1000000034e", "tid": 374 }, { "created_ino": "0x1000000034f", "tid": 375 }, { "created_ino": "0x10000000350", "tid": 376 }, { "created_ino": "0x10000000351", "tid": 377 }, { "created_ino": "0x10000000352", "tid": 378 }, { "created_ino": "0x10000000353", "tid": 379 }, { "created_ino": "0x10000000354", "tid": 380 }, { "created_ino": "0x10000000355", "tid": 381 }, { "created_ino": "0x10000000356", "tid": 382 }, { "created_ino": "0x10000000357", "tid": 383 }, { "created_ino": "0x10000000358", "tid": 384 }, { "created_ino": "0x10000000359", "tid": 385 }, { "created_ino": "0x1000000035a", "tid": 386 }, { "created_ino": "0x1000000035b", "tid": 387 }, { "created_ino": "0x1000000035c", "tid": 388 }, { "created_ino": "0x1000000035d", "tid": 389 }, { "created_ino": "0x1000000035e", "tid": 390 }, { "created_ino": "0x1000000035f", "tid": 391 }, { "created_ino": "0x10000000360", "tid": 392 }, { "created_ino": "0x10000000361", "tid": 393 }, { "created_ino": "0x10000000362", "tid": 394 }, { "created_ino": "0x10000000363", "tid": 395 }, { "created_ino": "0x10000000364", "tid": 396 }, { "created_ino": "0x10000000365", "tid": 397 }, { "created_ino": "0x10000000366", "tid": 398 }, { "created_ino": "0x10000000367", "tid": 399 }, { "created_ino": "0x10000000368", "tid": 400 }, { "created_ino": "0x10000000369", "tid": 401 }, { "created_ino": "0x1000000036a", "tid": 402 }, { "created_ino": "0x1000000036b", "tid": 403 }, { "created_ino": "0x1000000036c", "tid": 404 }, { "created_ino": "0x1000000036d", "tid": 405 }, { "created_ino": "0x1000000036e", "tid": 406 }, { "created_ino": "0x1000000036f", "tid": 407 }, { "created_ino": "0x10000000370", "tid": 408 }, { "created_ino": "0x10000000371", "tid": 409 }, { "created_ino": "0x10000000372", "tid": 410 }, { "created_ino": "0x10000000373", "tid": 411 }, { "created_ino": "0x10000000374", "tid": 412 }, { "created_ino": "0x10000000375", "tid": 413 }, { "created_ino": "0x10000000376", "tid": 414 }, { "created_ino": "0x10000000377", "tid": 415 }, { "created_ino": "0x10000000378", "tid": 416 }, { "created_ino": "0x10000000379", "tid": 417 }, { "created_ino": "0x1000000037a", "tid": 418 }, { "created_ino": "0x1000000037b", "tid": 419 }, { "created_ino": "0x1000000037c", "tid": 420 }, { "created_ino": "0x1000000037d", "tid": 421 }, { "created_ino": "0x1000000037e", "tid": 422 }, { "created_ino": "0x1000000037f", "tid": 423 }, { "created_ino": "0x10000000380", "tid": 424 }, { "created_ino": "0x10000000381", "tid": 425 }, { "created_ino": "0x10000000382", "tid": 426 }, { "created_ino": "0x10000000383", "tid": 427 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 2073476252, "type": "any" }, "name": { "num": 7414, "type": "client" } }, "id": 7414, "inst": "client.7414 172.21.15.16:0/2073476252", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 201, "num_completed_flushes": 0, "num_completed_requests": 399, "num_leases": 400, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 603, "start": "0x10000000384" } ], "recall_caps": { "halflife": 60, "value": 1515.2674606987187 }, "recall_caps_throttle": { "halflife": 1.5, "value": 478.0693979823774 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 194.11678079965284 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 199.55964904303892 }, "request_load_avg": 384, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1177.5611785977792 }, "state": "open", "uptime": 13.187847791 } ] 2024-09-10T21:18:37.708 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:18:37.708 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:18:37.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:18:37.802 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:18:37.886 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:18:37.886 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:18:37.886 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 400\nabs_path = "/home/ubuntu/cephtest/mnt.0/."\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi016') 2024-09-10T21:18:37.886 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:18:37.887 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:37.895 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:18:37.896 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:37.951 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.136+0000 7fc7ff400640 1 -- 172.21.15.16:0/4080822620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f80b2860 msgr2=0x7fc7f8160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.136+0000 7fc7ff400640 1 --2- 172.21.15.16:0/4080822620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f80b2860 0x7fc7f8160510 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009c80 tx=0x7fc7f001cda0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.136+0000 7fc7ff400640 1 -- 172.21.15.16:0/4080822620 shutdown_connections 2024-09-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.136+0000 7fc7ff400640 1 --2- 172.21.15.16:0/4080822620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f80b2860 0x7fc7f8160510 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.136+0000 7fc7ff400640 1 --2- 172.21.15.16:0/4080822620 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7f80cffe0 0x7fc7f80ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.136+0000 7fc7ff400640 1 -- 172.21.15.16:0/4080822620 >> 172.21.15.16:0/4080822620 conn(0x7fc7f80d1d80 msgr2=0x7fc7f80d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:38.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.136+0000 7fc7ff400640 1 -- 172.21.15.16:0/4080822620 shutdown_connections 2024-09-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.136+0000 7fc7ff400640 1 -- 172.21.15.16:0/4080822620 wait complete. 2024-09-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7ff400640 1 Processor -- start 2024-09-10T21:18:38.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7ff400640 1 -- start start 2024-09-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7ff400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7f80b2860 0x7fc7f815b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7ff400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7f80cffe0 0x7fc7f8155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7ff400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f80d03c0 0x7fc7f806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7ff400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80cd910 con 0x7fc7f80d03c0 2024-09-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7ff400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80cd790 con 0x7fc7f80cffe0 2024-09-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7ff400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f80cda90 con 0x7fc7f80b2860 2024-09-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7fd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f80d03c0 0x7fc7f806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7fd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f80d03c0 0x7fc7f806ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49940/0 (socket says 172.21.15.16:49940) 2024-09-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7fd400640 1 -- 172.21.15.16:0/3352301583 learned_addr learned my addr 172.21.15.16:0/3352301583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7efe00640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7f80cffe0 0x7fc7f8155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7fca00640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7f80b2860 0x7fc7f815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:38.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7fd400640 1 -- 172.21.15.16:0/3352301583 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7f80b2860 msgr2=0x7fc7f815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7fd400640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7f80b2860 0x7fc7f815b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7fd400640 1 -- 172.21.15.16:0/3352301583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7f80cffe0 msgr2=0x7fc7f8155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7fd400640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7f80cffe0 0x7fc7f8155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7fd400640 1 -- 172.21.15.16:0/3352301583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f81fb150 con 0x7fc7f80d03c0 2024-09-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7efe00640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7f80cffe0 0x7fc7f8155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7fd400640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f80d03c0 0x7fc7f806ed40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fc7f001b450 tx=0x7fc7f00092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7f5600640 1 -- 172.21.15.16:0/3352301583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f00bb070 con 0x7fc7f80d03c0 2024-09-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7f5600640 1 -- 172.21.15.16:0/3352301583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7f0004c50 con 0x7fc7f80d03c0 2024-09-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7ff400640 1 -- 172.21.15.16:0/3352301583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f81fb3f0 con 0x7fc7f80d03c0 2024-09-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7f5600640 1 -- 172.21.15.16:0/3352301583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f00b2040 con 0x7fc7f80d03c0 2024-09-10T21:18:38.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.140+0000 7fc7ff400640 1 -- 172.21.15.16:0/3352301583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f815aed0 con 0x7fc7f80d03c0 2024-09-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.144+0000 7fc7ff400640 1 -- 172.21.15.16:0/3352301583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f80ce2b0 con 0x7fc7f80d03c0 2024-09-10T21:18:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.144+0000 7fc7f5600640 1 -- 172.21.15.16:0/3352301583 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc7f00b5030 con 0x7fc7f80d03c0 2024-09-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.144+0000 7fc7f5600640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7d40725e0 0x7fc7d4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.144+0000 7fc7f5600640 1 -- 172.21.15.16:0/3352301583 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7fc7f012cd80 con 0x7fc7f80d03c0 2024-09-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.144+0000 7fc7fca00640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7d40725e0 0x7fc7d4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.144+0000 7fc7fca00640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7d40725e0 0x7fc7d4074aa0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc7f81f56b0 tx=0x7fc7e80091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:38.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.148+0000 7fc7f5600640 1 -- 172.21.15.16:0/3352301583 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc7f0016990 con 0x7fc7f80d03c0 2024-09-10T21:18:38.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.332+0000 7fc7ff400640 1 -- 172.21.15.16:0/3352301583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc7f80cf950 con 0x7fc7f80d03c0 2024-09-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.332+0000 7fc7f5600640 1 -- 172.21.15.16:0/3352301583 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) ==== 80+0+0 (secure 0 0 0) 0x7fc7f00f83d0 con 0x7fc7f80d03c0 2024-09-10T21:18:38.339 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.336+0000 7fc7ff400640 1 -- 172.21.15.16:0/3352301583 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7d40725e0 msgr2=0x7fc7d4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.336+0000 7fc7ff400640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7d40725e0 0x7fc7d4074aa0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc7f81f56b0 tx=0x7fc7e80091e0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.336+0000 7fc7ff400640 1 -- 172.21.15.16:0/3352301583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f80d03c0 msgr2=0x7fc7f806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.336+0000 7fc7ff400640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f80d03c0 0x7fc7f806ed40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fc7f001b450 tx=0x7fc7f00092e0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.336+0000 7fc7ff400640 1 -- 172.21.15.16:0/3352301583 shutdown_connections 2024-09-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.336+0000 7fc7ff400640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc7d40725e0 0x7fc7d4074aa0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.336+0000 7fc7ff400640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7f80d03c0 0x7fc7f806ed40 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.336+0000 7fc7ff400640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7f80cffe0 0x7fc7f8155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.336+0000 7fc7ff400640 1 --2- 172.21.15.16:0/3352301583 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc7f80b2860 0x7fc7f815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.336+0000 7fc7ff400640 1 -- 172.21.15.16:0/3352301583 >> 172.21.15.16:0/3352301583 conn(0x7fc7f80d1d80 msgr2=0x7fc7f8070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.336+0000 7fc7ff400640 1 -- 172.21.15.16:0/3352301583 shutdown_connections 2024-09-10T21:18:38.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.336+0000 7fc7ff400640 1 -- 172.21.15.16:0/3352301583 wait complete. 2024-09-10T21:18:38.359 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:18:38.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:38.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:18:38.459 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:18:38.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:38.459 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:18:38.468 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:18:38.469 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:18:38.469 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:38.479 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:18:38.479 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:38.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:18:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 -- 172.21.15.16:0/989318564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c0d0ca0 msgr2=0x7f141c168d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 --2- 172.21.15.16:0/989318564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c0d0ca0 0x7f141c168d60 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f1414009bc0 tx=0x7f141401cb70 comp rx=0 tx=0).stop 2024-09-10T21:18:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 -- 172.21.15.16:0/989318564 shutdown_connections 2024-09-10T21:18:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 --2- 172.21.15.16:0/989318564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c0d0ca0 0x7f141c168d60 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 -- 172.21.15.16:0/989318564 >> 172.21.15.16:0/989318564 conn(0x7f141c09b130 msgr2=0x7f141c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 -- 172.21.15.16:0/989318564 shutdown_connections 2024-09-10T21:18:38.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 -- 172.21.15.16:0/989318564 wait complete. 2024-09-10T21:18:38.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 Processor -- start 2024-09-10T21:18:38.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 -- start start 2024-09-10T21:18:38.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c0b0b40 0x7f141c15ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:38.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f141c0d0ca0 0x7f141c14db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:38.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f141c161620 0x7f141c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:38.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f141c16bf60 con 0x7f141c0b0b40 2024-09-10T21:18:38.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f141c16bde0 con 0x7f141c0d0ca0 2024-09-10T21:18:38.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1421000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f141c16c0e0 con 0x7f141c161620 2024-09-10T21:18:38.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1419e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c0b0b40 0x7f141c15ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:38.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1419e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c0b0b40 0x7f141c15ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49960/0 (socket says 172.21.15.16:49960) 2024-09-10T21:18:38.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1419e00640 1 -- 172.21.15.16:0/1704107643 learned_addr learned my addr 172.21.15.16:0/1704107643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:38.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1413e00640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f141c0d0ca0 0x7f141c14db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:38.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f141a800640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f141c161620 0x7f141c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:38.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1419e00640 1 -- 172.21.15.16:0/1704107643 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f141c161620 msgr2=0x7f141c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:38.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1419e00640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f141c161620 0x7f141c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1419e00640 1 -- 172.21.15.16:0/1704107643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f141c0d0ca0 msgr2=0x7f141c14db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:38.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1419e00640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f141c0d0ca0 0x7f141c14db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f1419e00640 1 -- 172.21.15.16:0/1704107643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141c156d40 con 0x7f141c0b0b40 2024-09-10T21:18:38.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.672+0000 7f141a800640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f141c161620 0x7f141c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:38.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.676+0000 7f1419e00640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c0b0b40 0x7f141c15ac90 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c430 tx=0x7f140c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:38.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.676+0000 7f1412000640 1 -- 172.21.15.16:0/1704107643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c005230 con 0x7f141c0b0b40 2024-09-10T21:18:38.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.676+0000 7f1412000640 1 -- 172.21.15.16:0/1704107643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f140c0053f0 con 0x7f141c0b0b40 2024-09-10T21:18:38.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.676+0000 7f1412000640 1 -- 172.21.15.16:0/1704107643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c004900 con 0x7f141c0b0b40 2024-09-10T21:18:38.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.676+0000 7f1421000640 1 -- 172.21.15.16:0/1704107643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141c1faa40 con 0x7f141c0b0b40 2024-09-10T21:18:38.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.676+0000 7f1421000640 1 -- 172.21.15.16:0/1704107643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141c1fb000 con 0x7f141c0b0b40 2024-09-10T21:18:38.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.676+0000 7f1421000640 1 -- 172.21.15.16:0/1704107643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f141c1fb200 con 0x7f141c0b0b40 2024-09-10T21:18:38.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.676+0000 7f1412000640 1 -- 172.21.15.16:0/1704107643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f140c004aa0 con 0x7f141c0b0b40 2024-09-10T21:18:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.680+0000 7f1412000640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f13f8072610 0x7f13f8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.680+0000 7f1412000640 1 -- 172.21.15.16:0/1704107643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f140c092d20 con 0x7f141c0b0b40 2024-09-10T21:18:38.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.680+0000 7f1413e00640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f13f8072610 0x7f13f8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:38.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.680+0000 7f1413e00640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f13f8072610 0x7f13f8074ad0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1404000c10 tx=0x7f1404009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:38.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.684+0000 7f1412000640 1 -- 172.21.15.16:0/1704107643 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f141c1fb200 con 0x7f141c0b0b40 2024-09-10T21:18:38.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.872+0000 7f1421000640 1 -- 172.21.15.16:0/1704107643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f141c0d0590 con 0x7f141c0b0b40 2024-09-10T21:18:38.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.872+0000 7f1412000640 1 -- 172.21.15.16:0/1704107643 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) ==== 80+0+0 (secure 0 0 0) 0x7f140c05e3f0 con 0x7f141c0b0b40 2024-09-10T21:18:38.878 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:18:38.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.876+0000 7f1421000640 1 -- 172.21.15.16:0/1704107643 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f13f8072610 msgr2=0x7f13f8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:38.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.876+0000 7f1421000640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f13f8072610 0x7f13f8074ad0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1404000c10 tx=0x7f1404009040 comp rx=0 tx=0).stop 2024-09-10T21:18:38.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.876+0000 7f1421000640 1 -- 172.21.15.16:0/1704107643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c0b0b40 msgr2=0x7f141c15ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:38.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.876+0000 7f1421000640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c0b0b40 0x7f141c15ac90 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c430 tx=0x7f140c00c8f0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.876+0000 7f1421000640 1 -- 172.21.15.16:0/1704107643 shutdown_connections 2024-09-10T21:18:38.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.876+0000 7f1421000640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f13f8072610 0x7f13f8074ad0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.876+0000 7f1421000640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f141c161620 0x7f141c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.876+0000 7f1421000640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f141c0d0ca0 0x7f141c14db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.876+0000 7f1421000640 1 --2- 172.21.15.16:0/1704107643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141c0b0b40 0x7f141c15ac90 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:38.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.876+0000 7f1421000640 1 -- 172.21.15.16:0/1704107643 >> 172.21.15.16:0/1704107643 conn(0x7f141c09b130 msgr2=0x7f141c0d2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:38.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.876+0000 7f1421000640 1 -- 172.21.15.16:0/1704107643 shutdown_connections 2024-09-10T21:18:38.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:38.876+0000 7f1421000640 1 -- 172.21.15.16:0/1704107643 wait complete. 2024-09-10T21:18:38.899 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T21:18:38.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:38.899 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T21:18:38.925 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:18:38.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:18:38.926 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T21:18:38.980 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:18:38.980 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:18:38.980 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:38.989 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:18:38.989 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:18:38.989 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:18:38.989 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:39.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:18:39.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:39.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1375212630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6ac0053a0 msgr2=0x7fe6ac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 --2- 172.21.15.16:0/1375212630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6ac0053a0 0x7fe6ac00b9d0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00c560 tx=0x7fe6bc01d240 comp rx=0 tx=0).stop 2024-09-10T21:18:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1375212630 shutdown_connections 2024-09-10T21:18:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 --2- 172.21.15.16:0/1375212630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6ac0053a0 0x7fe6ac00b9d0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1375212630 >> 172.21.15.16:0/1375212630 conn(0x7fe6c409b130 msgr2=0x7fe6c40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1375212630 shutdown_connections 2024-09-10T21:18:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1375212630 wait complete. 2024-09-10T21:18:39.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 Processor -- start 2024-09-10T21:18:39.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 -- start start 2024-09-10T21:18:39.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe6ac0053a0 0x7fe6c40cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c415a5c0 0x7fe6c40c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6c40cd610 0x7fe6c40cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4071370 con 0x7fe6c415a5c0 2024-09-10T21:18:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c40711f0 con 0x7fe6c40cd610 2024-09-10T21:18:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6cbc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c40714f0 con 0x7fe6ac0053a0 2024-09-10T21:18:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c415a5c0 0x7fe6c40c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c415a5c0 0x7fe6c40c7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49980/0 (socket says 172.21.15.16:49980) 2024-09-10T21:18:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6c3e00640 1 -- 172.21.15.16:0/1239916518 learned_addr learned my addr 172.21.15.16:0/1239916518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6c9200640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe6ac0053a0 0x7fe6c40cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6c9c00640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6c40cd610 0x7fe6c40cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6c3e00640 1 -- 172.21.15.16:0/1239916518 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe6ac0053a0 msgr2=0x7fe6c40cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6c3e00640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe6ac0053a0 0x7fe6c40cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6c3e00640 1 -- 172.21.15.16:0/1239916518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6c40cd610 msgr2=0x7fe6c40cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6c3e00640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6c40cd610 0x7fe6c40cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6c3e00640 1 -- 172.21.15.16:0/1239916518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c40c8120 con 0x7fe6c415a5c0 2024-09-10T21:18:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6c3e00640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c415a5c0 0x7fe6c40c7be0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0011860 tx=0x7fe6b0011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.180+0000 7fe6c0c00640 1 -- 172.21.15.16:0/1239916518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b000f940 con 0x7fe6c415a5c0 2024-09-10T21:18:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.184+0000 7fe6c0c00640 1 -- 172.21.15.16:0/1239916518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6b000fae0 con 0x7fe6c415a5c0 2024-09-10T21:18:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.184+0000 7fe6c0c00640 1 -- 172.21.15.16:0/1239916518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0003600 con 0x7fe6c415a5c0 2024-09-10T21:18:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.184+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1239916518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c40ca170 con 0x7fe6c415a5c0 2024-09-10T21:18:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.184+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1239916518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c40ca730 con 0x7fe6c415a5c0 2024-09-10T21:18:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.184+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1239916518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c40ca970 con 0x7fe6c415a5c0 2024-09-10T21:18:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.184+0000 7fe6c0c00640 1 -- 172.21.15.16:0/1239916518 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe6b0018070 con 0x7fe6c415a5c0 2024-09-10T21:18:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.184+0000 7fe6c0c00640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6a4072610 0x7fe6a4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.184+0000 7fe6c0c00640 1 -- 172.21.15.16:0/1239916518 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7fe6b0093eb0 con 0x7fe6c415a5c0 2024-09-10T21:18:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.184+0000 7fe6c9200640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6a4072610 0x7fe6a4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.184+0000 7fe6c9200640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6a4072610 0x7fe6a4074ad0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80011c0 tx=0x7fe6b8006b70 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:39.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.188+0000 7fe6c0c00640 1 -- 172.21.15.16:0/1239916518 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe6b005f4f0 con 0x7fe6c415a5c0 2024-09-10T21:18:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.396+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1239916518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6c40af740 con 0x7fe6c415a5c0 2024-09-10T21:18:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.396+0000 7fe6c0c00640 1 -- 172.21.15.16:0/1239916518 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5227 (secure 0 0 0) 0x7fe6b005edf0 con 0x7fe6c415a5c0 2024-09-10T21:18:39.402 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":95,"btime":"2024-09-10T21:18:33:594669+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3971491960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3971491960},{"type":"v1","addr":"172.21.15.52:6834","nonce":3971491960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7854,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4224090659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4224090659},{"type":"v1","addr":"172.21.15.16:6836","nonce":4224090659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1749071285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1749071285},{"type":"v1","addr":"172.21.15.16:6837","nonce":1749071285}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:16.852256+0000","modified":"2024-09-10T21:18:33.478948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":7,"addr":"172.21.15.52:6835/3672156966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3672156966},{"type":"v1","addr":"172.21.15.52:6835","nonce":3672156966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7869,"qdb_cluster":[7869]},"id":10}]} 2024-09-10T21:18:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 95 2024-09-10T21:18:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.400+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1239916518 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6a4072610 msgr2=0x7fe6a4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.400+0000 7fe6cbc00640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6a4072610 0x7fe6a4074ad0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80011c0 tx=0x7fe6b8006b70 comp rx=0 tx=0).stop 2024-09-10T21:18:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.400+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1239916518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c415a5c0 msgr2=0x7fe6c40c7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.400+0000 7fe6cbc00640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c415a5c0 0x7fe6c40c7be0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0011860 tx=0x7fe6b0011c60 comp rx=0 tx=0).stop 2024-09-10T21:18:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.400+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1239916518 shutdown_connections 2024-09-10T21:18:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.400+0000 7fe6cbc00640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe6a4072610 0x7fe6a4074ad0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.400+0000 7fe6cbc00640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6c40cd610 0x7fe6c40cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.400+0000 7fe6cbc00640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c415a5c0 0x7fe6c40c7be0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.400+0000 7fe6cbc00640 1 --2- 172.21.15.16:0/1239916518 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe6ac0053a0 0x7fe6c40cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.400+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1239916518 >> 172.21.15.16:0/1239916518 conn(0x7fe6c409b130 msgr2=0x7fe6c415e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.400+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1239916518 shutdown_connections 2024-09-10T21:18:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.400+0000 7fe6cbc00640 1 -- 172.21.15.16:0/1239916518 wait complete. 2024-09-10T21:18:39.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:39.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.564+0000 7f68bb200640 1 -- 172.21.15.16:0/3484855687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68bc074280 msgr2=0x7f68bc0cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.564+0000 7f68bb200640 1 --2- 172.21.15.16:0/3484855687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68bc074280 0x7f68bc0cea10 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009bc0 tx=0x7f68b401cba0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.564+0000 7f68bb200640 1 -- 172.21.15.16:0/3484855687 shutdown_connections 2024-09-10T21:18:39.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.564+0000 7f68bb200640 1 --2- 172.21.15.16:0/3484855687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68bc074280 0x7f68bc0cea10 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.564+0000 7f68bb200640 1 --2- 172.21.15.16:0/3484855687 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f68bc0af780 0x7f68bc15f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.564+0000 7f68bb200640 1 -- 172.21.15.16:0/3484855687 >> 172.21.15.16:0/3484855687 conn(0x7f68bc0d0a60 msgr2=0x7f68bc0d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:39.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.564+0000 7f68bb200640 1 -- 172.21.15.16:0/3484855687 shutdown_connections 2024-09-10T21:18:39.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.564+0000 7f68bb200640 1 -- 172.21.15.16:0/3484855687 wait complete. 2024-09-10T21:18:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.564+0000 7f68bb200640 1 Processor -- start 2024-09-10T21:18:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.564+0000 7f68bb200640 1 -- start start 2024-09-10T21:18:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68bb200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68bc0737e0 0x7f68bc1f7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68bb200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f68bc074280 0x7f68bc0c4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68bb200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc0af780 0x7f68bc157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68bb200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc0cfcb0 con 0x7f68bc0737e0 2024-09-10T21:18:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68bb200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc0cfb30 con 0x7f68bc0af780 2024-09-10T21:18:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68bb200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f68bc06cbe0 con 0x7f68bc074280 2024-09-10T21:18:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68b9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68bc0737e0 0x7f68bc1f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68b9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68bc0737e0 0x7f68bc1f7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49984/0 (socket says 172.21.15.16:49984) 2024-09-10T21:18:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68b9e00640 1 -- 172.21.15.16:0/1862234086 learned_addr learned my addr 172.21.15.16:0/1862234086 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68ba800640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc0af780 0x7f68bc157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68b9400640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f68bc074280 0x7f68bc0c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68b9e00640 1 -- 172.21.15.16:0/1862234086 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f68bc074280 msgr2=0x7f68bc0c4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68b9e00640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f68bc074280 0x7f68bc0c4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68b9e00640 1 -- 172.21.15.16:0/1862234086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc0af780 msgr2=0x7f68bc157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68b9e00640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc0af780 0x7f68bc157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68b9e00640 1 -- 172.21.15.16:0/1862234086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68bc0c58d0 con 0x7f68bc0737e0 2024-09-10T21:18:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68b9e00640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68bc0737e0 0x7f68bc1f7980 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00c500 tx=0x7f68ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68b2000640 1 -- 172.21.15.16:0/1862234086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac017070 con 0x7f68bc0737e0 2024-09-10T21:18:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68b2000640 1 -- 172.21.15.16:0/1862234086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68ac003600 con 0x7f68bc0737e0 2024-09-10T21:18:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68b2000640 1 -- 172.21.15.16:0/1862234086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac003a90 con 0x7f68bc0737e0 2024-09-10T21:18:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68bb200640 1 -- 172.21.15.16:0/1862234086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68bc0c4de0 con 0x7f68bc0737e0 2024-09-10T21:18:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68bb200640 1 -- 172.21.15.16:0/1862234086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68bc0c5b70 con 0x7f68bc0737e0 2024-09-10T21:18:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.568+0000 7f68bb200640 1 -- 172.21.15.16:0/1862234086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68bc0cea10 con 0x7f68bc0737e0 2024-09-10T21:18:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.572+0000 7f68b2000640 1 -- 172.21.15.16:0/1862234086 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f68ac003c30 con 0x7f68bc0737e0 2024-09-10T21:18:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.572+0000 7f68b2000640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f689c072610 0x7f689c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.572+0000 7f68b2000640 1 -- 172.21.15.16:0/1862234086 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f68ac092920 con 0x7f68bc0737e0 2024-09-10T21:18:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.572+0000 7f68b9400640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f689c072610 0x7f689c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.572+0000 7f68b9400640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f689c072610 0x7f689c074ad0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f68bc159e50 tx=0x7f68a800b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.576+0000 7f68b2000640 1 -- 172.21.15.16:0/1862234086 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f68ac05da60 con 0x7f68bc0737e0 2024-09-10T21:18:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68bb200640 1 -- 172.21.15.16:0/1862234086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68bc0c4210 con 0x7f68bc0737e0 2024-09-10T21:18:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68b2000640 1 -- 172.21.15.16:0/1862234086 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5227 (secure 0 0 0) 0x7f68ac062490 con 0x7f68bc0737e0 2024-09-10T21:18:39.793 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:39.793 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":95,"btime":"2024-09-10T21:18:33:594669+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3971491960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3971491960},{"type":"v1","addr":"172.21.15.52:6834","nonce":3971491960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7854,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4224090659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4224090659},{"type":"v1","addr":"172.21.15.16:6836","nonce":4224090659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1749071285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1749071285},{"type":"v1","addr":"172.21.15.16:6837","nonce":1749071285}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:16.852256+0000","modified":"2024-09-10T21:18:33.478948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":7,"addr":"172.21.15.52:6835/3672156966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3672156966},{"type":"v1","addr":"172.21.15.52:6835","nonce":3672156966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7869,"qdb_cluster":[7869]},"id":10}]} 2024-09-10T21:18:39.793 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 95 2024-09-10T21:18:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68bb200640 1 -- 172.21.15.16:0/1862234086 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f689c072610 msgr2=0x7f689c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68bb200640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f689c072610 0x7f689c074ad0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f68bc159e50 tx=0x7f68a800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68bb200640 1 -- 172.21.15.16:0/1862234086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68bc0737e0 msgr2=0x7f68bc1f7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68bb200640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68bc0737e0 0x7f68bc1f7980 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00c500 tx=0x7f68ac00c9c0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68bb200640 1 -- 172.21.15.16:0/1862234086 shutdown_connections 2024-09-10T21:18:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68bb200640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f689c072610 0x7f689c074ad0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68bb200640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc0af780 0x7f68bc157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68bb200640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f68bc074280 0x7f68bc0c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68bb200640 1 --2- 172.21.15.16:0/1862234086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68bc0737e0 0x7f68bc1f7980 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68bb200640 1 -- 172.21.15.16:0/1862234086 >> 172.21.15.16:0/1862234086 conn(0x7f68bc0d0a60 msgr2=0x7f68bc0718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68bb200640 1 -- 172.21.15.16:0/1862234086 shutdown_connections 2024-09-10T21:18:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.788+0000 7f68bb200640 1 -- 172.21.15.16:0/1862234086 wait complete. 2024-09-10T21:18:39.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:18:39.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 -- 172.21.15.16:0/2480948333 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f51280d0890 msgr2=0x7f51280d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 --2- 172.21.15.16:0/2480948333 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f51280d0890 0x7f51280d01c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5120009c80 tx=0x7f512001f5a0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 -- 172.21.15.16:0/2480948333 shutdown_connections 2024-09-10T21:18:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 --2- 172.21.15.16:0/2480948333 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f51280d0890 0x7f51280d01c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 --2- 172.21.15.16:0/2480948333 >> v2:172.21.15.52:3300/0 conn(0x7f512815f450 0x7f51280cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 -- 172.21.15.16:0/2480948333 >> 172.21.15.16:0/2480948333 conn(0x7f51280d1d80 msgr2=0x7f51280d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 -- 172.21.15.16:0/2480948333 shutdown_connections 2024-09-10T21:18:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 -- 172.21.15.16:0/2480948333 wait complete. 2024-09-10T21:18:39.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 Processor -- start 2024-09-10T21:18:39.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 -- start start 2024-09-10T21:18:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51280d0890 0x7f51281f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f512815f450 0x7f5128071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51281f7fc0 0x7f51281f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f51280b2e30 con 0x7f51280d0890 2024-09-10T21:18:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f51280b2cb0 con 0x7f51281f7fc0 2024-09-10T21:18:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f512e200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f51280b2fb0 con 0x7f512815f450 2024-09-10T21:18:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f5127400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51280d0890 0x7f51281f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f5127400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51280d0890 0x7f51281f55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49996/0 (socket says 172.21.15.16:49996) 2024-09-10T21:18:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f5127400640 1 -- 172.21.15.16:0/464840732 learned_addr learned my addr 172.21.15.16:0/464840732 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f511fe00640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f512815f450 0x7f5128071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.956+0000 7f5127e00640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51281f7fc0 0x7f51281f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.960+0000 7f5127400640 1 -- 172.21.15.16:0/464840732 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f512815f450 msgr2=0x7f5128071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.960+0000 7f5127400640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f512815f450 0x7f5128071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.960+0000 7f5127400640 1 -- 172.21.15.16:0/464840732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51281f7fc0 msgr2=0x7f51281f83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.960+0000 7f5127400640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51281f7fc0 0x7f51281f83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.960+0000 7f5127400640 1 -- 172.21.15.16:0/464840732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51281f8ac0 con 0x7f51280d0890 2024-09-10T21:18:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.960+0000 7f5127400640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51280d0890 0x7f51281f55c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f511800e7d0 tx=0x7f511800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.960+0000 7f511f400640 1 -- 172.21.15.16:0/464840732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511800ccc0 con 0x7f51280d0890 2024-09-10T21:18:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.960+0000 7f511f400640 1 -- 172.21.15.16:0/464840732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5118010430 con 0x7f51280d0890 2024-09-10T21:18:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.960+0000 7f511f400640 1 -- 172.21.15.16:0/464840732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118010860 con 0x7f51280d0890 2024-09-10T21:18:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.960+0000 7f512e200640 1 -- 172.21.15.16:0/464840732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51281f4ae0 con 0x7f51280d0890 2024-09-10T21:18:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.960+0000 7f512e200640 1 -- 172.21.15.16:0/464840732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51281f50a0 con 0x7f51280d0890 2024-09-10T21:18:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.960+0000 7f512e200640 1 -- 172.21.15.16:0/464840732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51280cf540 con 0x7f51280d0890 2024-09-10T21:18:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.960+0000 7f511f400640 1 -- 172.21.15.16:0/464840732 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5118010a00 con 0x7f51280d0890 2024-09-10T21:18:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.964+0000 7f511f400640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f51040725e0 0x7f5104074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.964+0000 7f511f400640 1 -- 172.21.15.16:0/464840732 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f5118093c50 con 0x7f51280d0890 2024-09-10T21:18:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.964+0000 7f511fe00640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f51040725e0 0x7f5104074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.964+0000 7f511fe00640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f51040725e0 0x7f5104074aa0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5128159ca0 tx=0x7f5110009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:39.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:39.968+0000 7f511f400640 1 -- 172.21.15.16:0/464840732 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f511805ecd0 con 0x7f51280d0890 2024-09-10T21:18:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.140+0000 7f512e200640 1 -- 172.21.15.16:0/464840732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51280b1e20 con 0x7f51280d0890 2024-09-10T21:18:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.144+0000 7f511f400640 1 -- 172.21.15.16:0/464840732 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16691 (secure 0 0 0) 0x7f5118063700 con 0x7f51280d0890 2024-09-10T21:18:40.148 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:40.148 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":127,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:18:18.441972+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:18:14.559149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T21:18:15.813518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:18:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.144+0000 7f512e200640 1 -- 172.21.15.16:0/464840732 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f51040725e0 msgr2=0x7f5104074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.144+0000 7f512e200640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f51040725e0 0x7f5104074aa0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5128159ca0 tx=0x7f5110009040 comp rx=0 tx=0).stop 2024-09-10T21:18:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.144+0000 7f512e200640 1 -- 172.21.15.16:0/464840732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51280d0890 msgr2=0x7f51281f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:40.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.144+0000 7f512e200640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51280d0890 0x7f51281f55c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f511800e7d0 tx=0x7f511800ec90 comp rx=0 tx=0).stop 2024-09-10T21:18:40.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.144+0000 7f512e200640 1 -- 172.21.15.16:0/464840732 shutdown_connections 2024-09-10T21:18:40.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.144+0000 7f512e200640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f51040725e0 0x7f5104074aa0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.144+0000 7f512e200640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51281f7fc0 0x7f51281f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.144+0000 7f512e200640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f512815f450 0x7f5128071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.144+0000 7f512e200640 1 --2- 172.21.15.16:0/464840732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51280d0890 0x7f51281f55c0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.144+0000 7f512e200640 1 -- 172.21.15.16:0/464840732 >> 172.21.15.16:0/464840732 conn(0x7f51280d1d80 msgr2=0x7f5128070bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:40.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.144+0000 7f512e200640 1 -- 172.21.15.16:0/464840732 shutdown_connections 2024-09-10T21:18:40.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.144+0000 7f512e200640 1 -- 172.21.15.16:0/464840732 wait complete. 2024-09-10T21:18:40.169 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:18:40.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:40.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 -- 172.21.15.16:0/2286713155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f541599d0 msgr2=0x7f4f5415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:40.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 --2- 172.21.15.16:0/2286713155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f541599d0 0x7f4f5415f830 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f4f5406fd80 tx=0x7f4f4401c770 comp rx=0 tx=0).stop 2024-09-10T21:18:40.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 -- 172.21.15.16:0/2286713155 shutdown_connections 2024-09-10T21:18:40.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 --2- 172.21.15.16:0/2286713155 >> v2:172.21.15.52:3300/0 conn(0x7f4f5406e640 0x7f4f540ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 --2- 172.21.15.16:0/2286713155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f541599d0 0x7f4f5415f830 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 -- 172.21.15.16:0/2286713155 >> 172.21.15.16:0/2286713155 conn(0x7f4f540cfb20 msgr2=0x7f4f540cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:40.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 -- 172.21.15.16:0/2286713155 shutdown_connections 2024-09-10T21:18:40.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 -- 172.21.15.16:0/2286713155 wait complete. 2024-09-10T21:18:40.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 Processor -- start 2024-09-10T21:18:40.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 -- start start 2024-09-10T21:18:40.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f5406e640 0x7f4f540c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:40.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4f540ca6b0 0x7f4f54155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:40.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f541599d0 0x7f4f54158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:40.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54168bd0 con 0x7f4f5406e640 2024-09-10T21:18:40.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54073060 con 0x7f4f541599d0 2024-09-10T21:18:40.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f59c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4f54168d50 con 0x7f4f540ca6b0 2024-09-10T21:18:40.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f52a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f5406e640 0x7f4f540c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f52a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f5406e640 0x7f4f540c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50014/0 (socket says 172.21.15.16:50014) 2024-09-10T21:18:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f52a00640 1 -- 172.21.15.16:0/2066220082 learned_addr learned my addr 172.21.15.16:0/2066220082 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f4a000640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4f540ca6b0 0x7f4f54155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.312+0000 7f4f53400640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f541599d0 0x7f4f54158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.316+0000 7f4f52a00640 1 -- 172.21.15.16:0/2066220082 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4f540ca6b0 msgr2=0x7f4f54155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.316+0000 7f4f52a00640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4f540ca6b0 0x7f4f54155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.316+0000 7f4f52a00640 1 -- 172.21.15.16:0/2066220082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f541599d0 msgr2=0x7f4f54158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.316+0000 7f4f52a00640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f541599d0 0x7f4f54158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.316+0000 7f4f52a00640 1 -- 172.21.15.16:0/2066220082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f540c69a0 con 0x7f4f5406e640 2024-09-10T21:18:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.316+0000 7f4f52a00640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f5406e640 0x7f4f540c7d70 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f4f44000c10 tx=0x7f4f440a56a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.316+0000 7f4f4b400640 1 -- 172.21.15.16:0/2066220082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f440bb070 con 0x7f4f5406e640 2024-09-10T21:18:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.316+0000 7f4f4b400640 1 -- 172.21.15.16:0/2066220082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f440044b0 con 0x7f4f5406e640 2024-09-10T21:18:40.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.316+0000 7f4f4b400640 1 -- 172.21.15.16:0/2066220082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f440b2040 con 0x7f4f5406e640 2024-09-10T21:18:40.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.316+0000 7f4f59c00640 1 -- 172.21.15.16:0/2066220082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f540c8400 con 0x7f4f5406e640 2024-09-10T21:18:40.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.316+0000 7f4f59c00640 1 -- 172.21.15.16:0/2066220082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f540c89c0 con 0x7f4f5406e640 2024-09-10T21:18:40.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.316+0000 7f4f59c00640 1 -- 172.21.15.16:0/2066220082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f540ca170 con 0x7f4f5406e640 2024-09-10T21:18:40.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.316+0000 7f4f4b400640 1 -- 172.21.15.16:0/2066220082 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4f440b5030 con 0x7f4f5406e640 2024-09-10T21:18:40.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.320+0000 7f4f4b400640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4f30072610 0x7f4f30074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:40.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.320+0000 7f4f4b400640 1 -- 172.21.15.16:0/2066220082 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f4f4412c920 con 0x7f4f5406e640 2024-09-10T21:18:40.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.320+0000 7f4f4a000640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4f30072610 0x7f4f30074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:40.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.320+0000 7f4f4a000640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4f30072610 0x7f4f30074ad0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c005fd0 tx=0x7f4f3c005db0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:40.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.320+0000 7f4f4b400640 1 -- 172.21.15.16:0/2066220082 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4f440163f0 con 0x7f4f5406e640 2024-09-10T21:18:40.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.532+0000 7f4f59c00640 1 -- 172.21.15.16:0/2066220082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f540c33f0 con 0x7f4f5406e640 2024-09-10T21:18:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.532+0000 7f4f4b400640 1 -- 172.21.15.16:0/2066220082 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5227 (secure 0 0 0) 0x7f4f440f7ff0 con 0x7f4f5406e640 2024-09-10T21:18:40.539 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:40.539 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":95,"btime":"2024-09-10T21:18:33:594669+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3971491960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3971491960},{"type":"v1","addr":"172.21.15.52:6834","nonce":3971491960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7854,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4224090659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4224090659},{"type":"v1","addr":"172.21.15.16:6836","nonce":4224090659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1749071285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1749071285},{"type":"v1","addr":"172.21.15.16:6837","nonce":1749071285}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:16.852256+0000","modified":"2024-09-10T21:18:33.478948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":7,"addr":"172.21.15.52:6835/3672156966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3672156966},{"type":"v1","addr":"172.21.15.52:6835","nonce":3672156966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7869,"qdb_cluster":[7869]},"id":10}]} 2024-09-10T21:18:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 95 2024-09-10T21:18:40.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.536+0000 7f4f59c00640 1 -- 172.21.15.16:0/2066220082 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4f30072610 msgr2=0x7f4f30074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:40.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.536+0000 7f4f59c00640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4f30072610 0x7f4f30074ad0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c005fd0 tx=0x7f4f3c005db0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.536+0000 7f4f59c00640 1 -- 172.21.15.16:0/2066220082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f5406e640 msgr2=0x7f4f540c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:40.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.536+0000 7f4f59c00640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f5406e640 0x7f4f540c7d70 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f4f44000c10 tx=0x7f4f440a56a0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.536+0000 7f4f59c00640 1 -- 172.21.15.16:0/2066220082 shutdown_connections 2024-09-10T21:18:40.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.536+0000 7f4f59c00640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4f30072610 0x7f4f30074ad0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.536+0000 7f4f59c00640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f541599d0 0x7f4f54158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.536+0000 7f4f59c00640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4f540ca6b0 0x7f4f54155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.536+0000 7f4f59c00640 1 --2- 172.21.15.16:0/2066220082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f5406e640 0x7f4f540c7d70 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.536+0000 7f4f59c00640 1 -- 172.21.15.16:0/2066220082 >> 172.21.15.16:0/2066220082 conn(0x7f4f540cfb20 msgr2=0x7f4f5415e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:40.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.536+0000 7f4f59c00640 1 -- 172.21.15.16:0/2066220082 shutdown_connections 2024-09-10T21:18:40.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.536+0000 7f4f59c00640 1 -- 172.21.15.16:0/2066220082 wait complete. 2024-09-10T21:18:40.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:40.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.700+0000 7fe10b400640 1 -- 172.21.15.16:0/2797508850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1040d0890 msgr2=0x7fe1040b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:40.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.700+0000 7fe10b400640 1 --2- 172.21.15.16:0/2797508850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1040d0890 0x7fe1040b0020 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe1040b3810 tx=0x7fe0fc01cd60 comp rx=0 tx=0).stop 2024-09-10T21:18:40.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.700+0000 7fe10b400640 1 -- 172.21.15.16:0/2797508850 shutdown_connections 2024-09-10T21:18:40.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.700+0000 7fe10b400640 1 --2- 172.21.15.16:0/2797508850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1040d0890 0x7fe1040b0020 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.700+0000 7fe10b400640 1 --2- 172.21.15.16:0/2797508850 >> v2:172.21.15.52:3300/0 conn(0x7fe1040b3200 0x7fe1040afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.700+0000 7fe10b400640 1 -- 172.21.15.16:0/2797508850 >> 172.21.15.16:0/2797508850 conn(0x7fe1040d1d80 msgr2=0x7fe1040d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:40.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.700+0000 7fe10b400640 1 -- 172.21.15.16:0/2797508850 shutdown_connections 2024-09-10T21:18:40.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.700+0000 7fe10b400640 1 -- 172.21.15.16:0/2797508850 wait complete. 2024-09-10T21:18:40.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe10b400640 1 Processor -- start 2024-09-10T21:18:40.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe10b400640 1 -- start start 2024-09-10T21:18:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe10b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1040d0890 0x7fe1040ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe10b400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10415f450 0x7fe1041529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe10b400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe10414f350 0x7fe104152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe10b400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe104168d50 con 0x7fe1040d0890 2024-09-10T21:18:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe10b400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe104168bd0 con 0x7fe10415f450 2024-09-10T21:18:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe10b400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe104168ed0 con 0x7fe10414f350 2024-09-10T21:18:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe108a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1040d0890 0x7fe1040ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe108a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1040d0890 0x7fe1040ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50038/0 (socket says 172.21.15.16:50038) 2024-09-10T21:18:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe108a00640 1 -- 172.21.15.16:0/4134519336 learned_addr learned my addr 172.21.15.16:0/4134519336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe0fbe00640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10415f450 0x7fe1041529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe109400640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe10414f350 0x7fe104152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe108a00640 1 -- 172.21.15.16:0/4134519336 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe10414f350 msgr2=0x7fe104152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe108a00640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe10414f350 0x7fe104152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe108a00640 1 -- 172.21.15.16:0/4134519336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10415f450 msgr2=0x7fe1041529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe108a00640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10415f450 0x7fe1041529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe108a00640 1 -- 172.21.15.16:0/4134519336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1040cebc0 con 0x7fe1040d0890 2024-09-10T21:18:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe109400640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe10414f350 0x7fe104152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe0fbe00640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10415f450 0x7fe1041529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe108a00640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1040d0890 0x7fe1040ce680 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400c3f0 tx=0x7fe0f400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:40.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe101600640 1 -- 172.21.15.16:0/4134519336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40052d0 con 0x7fe1040d0890 2024-09-10T21:18:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe101600640 1 -- 172.21.15.16:0/4134519336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0f4007020 con 0x7fe1040d0890 2024-09-10T21:18:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe10b400640 1 -- 172.21.15.16:0/4134519336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1040ccbe0 con 0x7fe1040d0890 2024-09-10T21:18:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe101600640 1 -- 172.21.15.16:0/4134519336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4007470 con 0x7fe1040d0890 2024-09-10T21:18:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.704+0000 7fe10b400640 1 -- 172.21.15.16:0/4134519336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1040cd1a0 con 0x7fe1040d0890 2024-09-10T21:18:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.708+0000 7fe10b400640 1 -- 172.21.15.16:0/4134519336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1040cfc80 con 0x7fe1040d0890 2024-09-10T21:18:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.708+0000 7fe101600640 1 -- 172.21.15.16:0/4134519336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe0f40067e0 con 0x7fe1040d0890 2024-09-10T21:18:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.708+0000 7fe101600640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0e0072610 0x7fe0e0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.708+0000 7fe101600640 1 -- 172.21.15.16:0/4134519336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7fe0f4092680 con 0x7fe1040d0890 2024-09-10T21:18:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.708+0000 7fe0fbe00640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0e0072610 0x7fe0e0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.708+0000 7fe0fbe00640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0e0072610 0x7fe0e0074ad0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009c30 tx=0x7fe0ec0091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.712+0000 7fe101600640 1 -- 172.21.15.16:0/4134519336 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe1040cebc0 con 0x7fe1040d0890 2024-09-10T21:18:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe10b400640 1 -- 172.21.15.16:0/4134519336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1041562a0 con 0x7fe1040d0890 2024-09-10T21:18:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe101600640 1 -- 172.21.15.16:0/4134519336 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5227 (secure 0 0 0) 0x7fe0f405dcc0 con 0x7fe1040d0890 2024-09-10T21:18:40.925 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:40.925 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":95,"btime":"2024-09-10T21:18:33:594669+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3971491960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3971491960},{"type":"v1","addr":"172.21.15.52:6834","nonce":3971491960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7854,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4224090659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4224090659},{"type":"v1","addr":"172.21.15.16:6836","nonce":4224090659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1749071285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1749071285},{"type":"v1","addr":"172.21.15.16:6837","nonce":1749071285}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:16.852256+0000","modified":"2024-09-10T21:18:33.478948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"b","rank":0,"incarnation":89,"state":"up:active","state_seq":7,"addr":"172.21.15.52:6835/3672156966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":3672156966},{"type":"v1","addr":"172.21.15.52:6835","nonce":3672156966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7869,"qdb_cluster":[7869]},"id":10}]} 2024-09-10T21:18:40.925 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 95 2024-09-10T21:18:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe10b400640 1 -- 172.21.15.16:0/4134519336 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0e0072610 msgr2=0x7fe0e0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe10b400640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0e0072610 0x7fe0e0074ad0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009c30 tx=0x7fe0ec0091e0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe10b400640 1 -- 172.21.15.16:0/4134519336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1040d0890 msgr2=0x7fe1040ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe10b400640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1040d0890 0x7fe1040ce680 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400c3f0 tx=0x7fe0f400c8b0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe10b400640 1 -- 172.21.15.16:0/4134519336 shutdown_connections 2024-09-10T21:18:40.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe10b400640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0e0072610 0x7fe0e0074ad0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe10b400640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe10414f350 0x7fe104152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe10b400640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10415f450 0x7fe1041529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe10b400640 1 --2- 172.21.15.16:0/4134519336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1040d0890 0x7fe1040ce680 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe10b400640 1 -- 172.21.15.16:0/4134519336 >> 172.21.15.16:0/4134519336 conn(0x7fe1040d1d80 msgr2=0x7fe1041567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe10b400640 1 -- 172.21.15.16:0/4134519336 shutdown_connections 2024-09-10T21:18:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:40.920+0000 7fe10b400640 1 -- 172.21.15.16:0/4134519336 wait complete. 2024-09-10T21:18:40.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:18:41.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.088+0000 7f91f4200640 1 -- 172.21.15.16:0/287554843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91ec0b13c0 msgr2=0x7f91ec071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.088+0000 7f91f4200640 1 --2- 172.21.15.16:0/287554843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91ec0b13c0 0x7f91ec071520 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f91ec074cc0 tx=0x7f91d801c7a0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.088+0000 7f91f4200640 1 -- 172.21.15.16:0/287554843 shutdown_connections 2024-09-10T21:18:41.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.088+0000 7f91f4200640 1 --2- 172.21.15.16:0/287554843 >> v2:172.21.15.52:3300/0 conn(0x7f91ec0cfc80 0x7f91ec0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.088+0000 7f91f4200640 1 --2- 172.21.15.16:0/287554843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91ec0b13c0 0x7f91ec071520 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.088+0000 7f91f4200640 1 --2- 172.21.15.16:0/287554843 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91ec15f400 0x7f91ec070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.088+0000 7f91f4200640 1 -- 172.21.15.16:0/287554843 >> 172.21.15.16:0/287554843 conn(0x7f91ec09b130 msgr2=0x7f91ec0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:41.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.088+0000 7f91f4200640 1 -- 172.21.15.16:0/287554843 shutdown_connections 2024-09-10T21:18:41.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.088+0000 7f91f4200640 1 -- 172.21.15.16:0/287554843 wait complete. 2024-09-10T21:18:41.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.088+0000 7f91f4200640 1 Processor -- start 2024-09-10T21:18:41.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f4200640 1 -- start start 2024-09-10T21:18:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f4200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91ec0b13c0 0x7f91ec200170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f4200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91ec0cfc80 0x7f91ec1fc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f4200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91ec15f400 0x7f91ec14eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f4200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec0afd90 con 0x7f91ec0cfc80 2024-09-10T21:18:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f4200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec0d1260 con 0x7f91ec0b13c0 2024-09-10T21:18:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f4200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f91ec0aff10 con 0x7f91ec15f400 2024-09-10T21:18:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91ec0cfc80 0x7f91ec1fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91ec0cfc80 0x7f91ec1fc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50054/0 (socket says 172.21.15.16:50054) 2024-09-10T21:18:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f0e00640 1 -- 172.21.15.16:0/2084966119 learned_addr learned my addr 172.21.15.16:0/2084966119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f1800640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91ec0b13c0 0x7f91ec200170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f2200640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91ec15f400 0x7f91ec14eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f0e00640 1 -- 172.21.15.16:0/2084966119 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91ec15f400 msgr2=0x7f91ec14eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f0e00640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91ec15f400 0x7f91ec14eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f0e00640 1 -- 172.21.15.16:0/2084966119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91ec0b13c0 msgr2=0x7f91ec200170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f0e00640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91ec0b13c0 0x7f91ec200170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f0e00640 1 -- 172.21.15.16:0/2084966119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec1fc8c0 con 0x7f91ec0cfc80 2024-09-10T21:18:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f0e00640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91ec0cfc80 0x7f91ec1fc120 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f91d80b0040 tx=0x7f91d80a66e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91e9600640 1 -- 172.21.15.16:0/2084966119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d80bc020 con 0x7f91ec0cfc80 2024-09-10T21:18:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91e9600640 1 -- 172.21.15.16:0/2084966119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91d8004490 con 0x7f91ec0cfc80 2024-09-10T21:18:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91e9600640 1 -- 172.21.15.16:0/2084966119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8005290 con 0x7f91ec0cfc80 2024-09-10T21:18:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f4200640 1 -- 172.21.15.16:0/2084966119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ec1fcba0 con 0x7f91ec0cfc80 2024-09-10T21:18:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f4200640 1 -- 172.21.15.16:0/2084966119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ec06deb0 con 0x7f91ec0cfc80 2024-09-10T21:18:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.092+0000 7f91f4200640 1 -- 172.21.15.16:0/2084966119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ec0d00d0 con 0x7f91ec0cfc80 2024-09-10T21:18:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.096+0000 7f91e9600640 1 -- 172.21.15.16:0/2084966119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f91d8005430 con 0x7f91ec0cfc80 2024-09-10T21:18:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.096+0000 7f91e9600640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91cc0725e0 0x7f91cc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.096+0000 7f91e9600640 1 -- 172.21.15.16:0/2084966119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7f91d812dc00 con 0x7f91ec0cfc80 2024-09-10T21:18:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.096+0000 7f91f1800640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91cc0725e0 0x7f91cc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.096+0000 7f91f1800640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91cc0725e0 0x7f91cc074aa0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f91ec1f9cd0 tx=0x7f91e000b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:41.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.100+0000 7f91e9600640 1 -- 172.21.15.16:0/2084966119 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f91d80c6060 con 0x7f91ec0cfc80 2024-09-10T21:18:41.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.272+0000 7f91f4200640 1 -- 172.21.15.16:0/2084966119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91ec0aed10 con 0x7f91ec0cfc80 2024-09-10T21:18:41.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.276+0000 7f91e9600640 1 -- 172.21.15.16:0/2084966119 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16691 (secure 0 0 0) 0x7f91d80164b0 con 0x7f91ec0cfc80 2024-09-10T21:18:41.280 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:41.280 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":127,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:18:18.441972+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:18:14.559149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T21:18:15.813518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:18:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.276+0000 7f91f4200640 1 -- 172.21.15.16:0/2084966119 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91cc0725e0 msgr2=0x7f91cc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.276+0000 7f91f4200640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91cc0725e0 0x7f91cc074aa0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f91ec1f9cd0 tx=0x7f91e000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.276+0000 7f91f4200640 1 -- 172.21.15.16:0/2084966119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91ec0cfc80 msgr2=0x7f91ec1fc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.276+0000 7f91f4200640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91ec0cfc80 0x7f91ec1fc120 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f91d80b0040 tx=0x7f91d80a66e0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.276+0000 7f91f4200640 1 -- 172.21.15.16:0/2084966119 shutdown_connections 2024-09-10T21:18:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.276+0000 7f91f4200640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f91cc0725e0 0x7f91cc074aa0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.276+0000 7f91f4200640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f91ec15f400 0x7f91ec14eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.276+0000 7f91f4200640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91ec0cfc80 0x7f91ec1fc120 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.276+0000 7f91f4200640 1 --2- 172.21.15.16:0/2084966119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91ec0b13c0 0x7f91ec200170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.276+0000 7f91f4200640 1 -- 172.21.15.16:0/2084966119 >> 172.21.15.16:0/2084966119 conn(0x7f91ec09b130 msgr2=0x7f91ec15e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:41.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.276+0000 7f91f4200640 1 -- 172.21.15.16:0/2084966119 shutdown_connections 2024-09-10T21:18:41.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.276+0000 7f91f4200640 1 -- 172.21.15.16:0/2084966119 wait complete. 2024-09-10T21:18:41.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:18:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.444+0000 7fa383c00640 1 -- 172.21.15.16:0/2555414809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0cffe0 msgr2=0x7fa37c0ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.444+0000 7fa383c00640 1 --2- 172.21.15.16:0/2555414809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0cffe0 0x7fa37c0ce2b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fa370009840 tx=0x7fa37001c820 comp rx=0 tx=0).stop 2024-09-10T21:18:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.444+0000 7fa383c00640 1 -- 172.21.15.16:0/2555414809 shutdown_connections 2024-09-10T21:18:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.444+0000 7fa383c00640 1 --2- 172.21.15.16:0/2555414809 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa37c0d03c0 0x7fa37c15fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.444+0000 7fa383c00640 1 --2- 172.21.15.16:0/2555414809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0cffe0 0x7fa37c0ce2b0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.444+0000 7fa383c00640 1 -- 172.21.15.16:0/2555414809 >> 172.21.15.16:0/2555414809 conn(0x7fa37c0d1d80 msgr2=0x7fa37c0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa383c00640 1 -- 172.21.15.16:0/2555414809 shutdown_connections 2024-09-10T21:18:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa383c00640 1 -- 172.21.15.16:0/2555414809 wait complete. 2024-09-10T21:18:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa383c00640 1 Processor -- start 2024-09-10T21:18:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa383c00640 1 -- start start 2024-09-10T21:18:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa383c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0b2860 0x7fa37c1fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa383c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa37c0cffe0 0x7fa37c0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa383c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa37c0d03c0 0x7fa37c0b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa383c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c0cd890 con 0x7fa37c0b2860 2024-09-10T21:18:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa383c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c0cd710 con 0x7fa37c0d03c0 2024-09-10T21:18:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa383c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa37c0cda10 con 0x7fa37c0cffe0 2024-09-10T21:18:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa381200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0b2860 0x7fa37c1fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa381200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0b2860 0x7fa37c1fbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50066/0 (socket says 172.21.15.16:50066) 2024-09-10T21:18:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa381200640 1 -- 172.21.15.16:0/3307878743 learned_addr learned my addr 172.21.15.16:0/3307878743 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa381c00640 1 --2- 172.21.15.16:0/3307878743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa37c0d03c0 0x7fa37c0b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa37be00640 1 --2- 172.21.15.16:0/3307878743 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa37c0cffe0 0x7fa37c0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa381200640 1 -- 172.21.15.16:0/3307878743 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa37c0cffe0 msgr2=0x7fa37c0cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa381200640 1 --2- 172.21.15.16:0/3307878743 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa37c0cffe0 0x7fa37c0cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa381200640 1 -- 172.21.15.16:0/3307878743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa37c0d03c0 msgr2=0x7fa37c0b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa381200640 1 --2- 172.21.15.16:0/3307878743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa37c0d03c0 0x7fa37c0b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa381200640 1 -- 172.21.15.16:0/3307878743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c071190 con 0x7fa37c0b2860 2024-09-10T21:18:41.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa37be00640 1 --2- 172.21.15.16:0/3307878743 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa37c0cffe0 0x7fa37c0cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.448+0000 7fa381200640 1 --2- 172.21.15.16:0/3307878743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0b2860 0x7fa37c1fbd30 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa370009790 tx=0x7fa3700a5670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.452+0000 7fa378c00640 1 -- 172.21.15.16:0/3307878743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3700b2040 con 0x7fa37c0b2860 2024-09-10T21:18:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.452+0000 7fa378c00640 1 -- 172.21.15.16:0/3307878743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa370004380 con 0x7fa37c0b2860 2024-09-10T21:18:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.452+0000 7fa383c00640 1 -- 172.21.15.16:0/3307878743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37c0cbe40 con 0x7fa37c0b2860 2024-09-10T21:18:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.452+0000 7fa378c00640 1 -- 172.21.15.16:0/3307878743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370003860 con 0x7fa37c0b2860 2024-09-10T21:18:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.452+0000 7fa383c00640 1 -- 172.21.15.16:0/3307878743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37c1f81b0 con 0x7fa37c0b2860 2024-09-10T21:18:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.452+0000 7fa383c00640 1 -- 172.21.15.16:0/3307878743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa37c0cf950 con 0x7fa37c0b2860 2024-09-10T21:18:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.452+0000 7fa378c00640 1 -- 172.21.15.16:0/3307878743 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa3700b5070 con 0x7fa37c0b2860 2024-09-10T21:18:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.456+0000 7fa378c00640 1 --2- 172.21.15.16:0/3307878743 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa35c0725e0 0x7fa35c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.456+0000 7fa378c00640 1 -- 172.21.15.16:0/3307878743 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7fa37012d0e0 con 0x7fa37c0b2860 2024-09-10T21:18:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.456+0000 7fa37be00640 1 --2- 172.21.15.16:0/3307878743 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa35c0725e0 0x7fa35c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.456+0000 7fa37be00640 1 --2- 172.21.15.16:0/3307878743 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa35c0725e0 0x7fa35c074aa0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0b0f70 tx=0x7fa3680091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.460+0000 7fa378c00640 1 -- 172.21.15.16:0/3307878743 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa37c0cbe40 con 0x7fa37c0b2860 2024-09-10T21:18:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.668+0000 7fa383c00640 1 -- 172.21.15.16:0/3307878743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa37c166de0 con 0x7fa37c0b2860 2024-09-10T21:18:41.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.672+0000 7fa378c00640 1 -- 172.21.15.16:0/3307878743 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v96) ==== 316+0+0 (secure 0 0 0) 0x7fa370016410 con 0x7fa37c0b2860 2024-09-10T21:18:41.679 INFO:teuthology.orchestra.run.smithi016.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-09-10T21:18:41.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:18:41.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-09-10T21:18:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.832+0000 7efd43000640 1 -- 172.21.15.16:0/375496515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c0cfe30 msgr2=0x7efd3c0ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.832+0000 7efd43000640 1 --2- 172.21.15.16:0/375496515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c0cfe30 0x7efd3c0ce590 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009870 tx=0x7efd2c01f010 comp rx=0 tx=0).stop 2024-09-10T21:18:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.832+0000 7efd43000640 1 -- 172.21.15.16:0/375496515 shutdown_connections 2024-09-10T21:18:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.832+0000 7efd43000640 1 --2- 172.21.15.16:0/375496515 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd3c0ce040 0x7efd3c0b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.832+0000 7efd43000640 1 --2- 172.21.15.16:0/375496515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c0cfe30 0x7efd3c0ce590 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.832+0000 7efd43000640 1 -- 172.21.15.16:0/375496515 >> 172.21.15.16:0/375496515 conn(0x7efd3c0d1df0 msgr2=0x7efd3c0d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.832+0000 7efd43000640 1 -- 172.21.15.16:0/375496515 shutdown_connections 2024-09-10T21:18:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.832+0000 7efd43000640 1 -- 172.21.15.16:0/375496515 wait complete. 2024-09-10T21:18:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd43000640 1 Processor -- start 2024-09-10T21:18:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd43000640 1 -- start start 2024-09-10T21:18:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd43000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c0ce040 0x7efd3c06f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd43000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd3c0cfe30 0x7efd3c1f8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd43000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3c0d0210 0x7efd3c1f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd43000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c0d2b00 con 0x7efd3c0ce040 2024-09-10T21:18:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd43000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c0b14f0 con 0x7efd3c0d0210 2024-09-10T21:18:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd43000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7efd3c0cf1e0 con 0x7efd3c0cfe30 2024-09-10T21:18:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd3be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c0ce040 0x7efd3c06f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd3be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c0ce040 0x7efd3c06f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50086/0 (socket says 172.21.15.16:50086) 2024-09-10T21:18:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd3be00640 1 -- 172.21.15.16:0/1334043937 learned_addr learned my addr 172.21.15.16:0/1334043937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd41000640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3c0d0210 0x7efd3c1f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd33e00640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd3c0cfe30 0x7efd3c1f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd3be00640 1 -- 172.21.15.16:0/1334043937 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd3c0cfe30 msgr2=0x7efd3c1f8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd3be00640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd3c0cfe30 0x7efd3c1f8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd3be00640 1 -- 172.21.15.16:0/1334043937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3c0d0210 msgr2=0x7efd3c1f8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd3be00640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3c0d0210 0x7efd3c1f8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd3be00640 1 -- 172.21.15.16:0/1334043937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c1f5860 con 0x7efd3c0ce040 2024-09-10T21:18:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd3be00640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c0ce040 0x7efd3c06f4f0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7efd2c01d6c0 tx=0x7efd2c0a8ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd38c00640 1 -- 172.21.15.16:0/1334043937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0b5040 con 0x7efd3c0ce040 2024-09-10T21:18:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd38c00640 1 -- 172.21.15.16:0/1334043937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd2c005020 con 0x7efd3c0ce040 2024-09-10T21:18:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd38c00640 1 -- 172.21.15.16:0/1334043937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c005470 con 0x7efd3c0ce040 2024-09-10T21:18:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd43000640 1 -- 172.21.15.16:0/1334043937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3c1f4aa0 con 0x7efd3c0ce040 2024-09-10T21:18:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd43000640 1 -- 172.21.15.16:0/1334043937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3c1f5060 con 0x7efd3c0ce040 2024-09-10T21:18:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.836+0000 7efd43000640 1 -- 172.21.15.16:0/1334043937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3c0b1710 con 0x7efd3c0ce040 2024-09-10T21:18:41.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.840+0000 7efd38c00640 1 -- 172.21.15.16:0/1334043937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efd2c0b8050 con 0x7efd3c0ce040 2024-09-10T21:18:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.840+0000 7efd38c00640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd180725e0 0x7efd18074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.840+0000 7efd38c00640 1 -- 172.21.15.16:0/1334043937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5709+0+0 (secure 0 0 0) 0x7efd2c130740 con 0x7efd3c0ce040 2024-09-10T21:18:41.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.840+0000 7efd33e00640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd180725e0 0x7efd18074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.840+0000 7efd33e00640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd180725e0 0x7efd18074aa0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7efd3c070e60 tx=0x7efd24009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:41.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:41.844+0000 7efd38c00640 1 -- 172.21.15.16:0/1334043937 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efd2c018b70 con 0x7efd3c0ce040 2024-09-10T21:18:42.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.052+0000 7efd43000640 1 -- 172.21.15.16:0/1334043937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7efd3c06fc60 con 0x7efd3c0ce040 2024-09-10T21:18:42.059 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-10T21:18:42.052+0000 7ff429400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:18:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.056+0000 7efd38c00640 1 -- 172.21.15.16:0/1334043937 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) ==== 191+0+0 (secure 0 0 0) 0x7efd2c0fbd90 con 0x7efd3c0ce040 2024-09-10T21:18:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:18:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.060+0000 7efd43000640 1 -- 172.21.15.16:0/1334043937 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd180725e0 msgr2=0x7efd18074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.060+0000 7efd43000640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd180725e0 0x7efd18074aa0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7efd3c070e60 tx=0x7efd24009040 comp rx=0 tx=0).stop 2024-09-10T21:18:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.060+0000 7efd43000640 1 -- 172.21.15.16:0/1334043937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c0ce040 msgr2=0x7efd3c06f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.060+0000 7efd43000640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c0ce040 0x7efd3c06f4f0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7efd2c01d6c0 tx=0x7efd2c0a8ca0 comp rx=0 tx=0).stop 2024-09-10T21:18:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.060+0000 7efd43000640 1 -- 172.21.15.16:0/1334043937 shutdown_connections 2024-09-10T21:18:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.060+0000 7efd43000640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd180725e0 0x7efd18074aa0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.060+0000 7efd43000640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3c0d0210 0x7efd3c1f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.060+0000 7efd43000640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd3c0cfe30 0x7efd3c1f8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.060+0000 7efd43000640 1 --2- 172.21.15.16:0/1334043937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c0ce040 0x7efd3c06f4f0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.060+0000 7efd43000640 1 -- 172.21.15.16:0/1334043937 >> 172.21.15.16:0/1334043937 conn(0x7efd3c0d1df0 msgr2=0x7efd3c155f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.060+0000 7efd43000640 1 -- 172.21.15.16:0/1334043937 shutdown_connections 2024-09-10T21:18:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.060+0000 7efd43000640 1 -- 172.21.15.16:0/1334043937 wait complete. 2024-09-10T21:18:42.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:18:42.134 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:18:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 -- 172.21.15.16:0/1523534523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e815bdc0 msgr2=0x7f40e80d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 --2- 172.21.15.16:0/1523534523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e815bdc0 0x7f40e80d2800 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f40e80747c0 tx=0x7f40d801c7a0 comp rx=0 tx=0).stop 2024-09-10T21:18:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 -- 172.21.15.16:0/1523534523 shutdown_connections 2024-09-10T21:18:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 --2- 172.21.15.16:0/1523534523 >> v2:172.21.15.52:3300/0 conn(0x7f40e80b5050 0x7f40e815f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 --2- 172.21.15.16:0/1523534523 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40e80cd300 0x7f40e80b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 --2- 172.21.15.16:0/1523534523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e815bdc0 0x7f40e80d2800 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 -- 172.21.15.16:0/1523534523 >> 172.21.15.16:0/1523534523 conn(0x7f40e80d1ce0 msgr2=0x7f40e80d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 -- 172.21.15.16:0/1523534523 shutdown_connections 2024-09-10T21:18:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 -- 172.21.15.16:0/1523534523 wait complete. 2024-09-10T21:18:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 Processor -- start 2024-09-10T21:18:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 -- start start 2024-09-10T21:18:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40e80b5050 0x7f40e8150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40e80cd300 0x7f40e81fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e815bdc0 0x7f40e8151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40e80ceb10 con 0x7f40e815bdc0 2024-09-10T21:18:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40e80ce990 con 0x7f40e80b5050 2024-09-10T21:18:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40ee600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f40e80cec90 con 0x7f40e80cd300 2024-09-10T21:18:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e815bdc0 0x7f40e8151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e815bdc0 0x7f40e8151e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41338/0 (socket says 172.21.15.16:41338) 2024-09-10T21:18:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40e7e00640 1 -- 172.21.15.16:0/652409988 learned_addr learned my addr 172.21.15.16:0/652409988 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.228+0000 7f40dfe00640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40e80cd300 0x7f40e81fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40e7400640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40e80b5050 0x7f40e8150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40e7e00640 1 -- 172.21.15.16:0/652409988 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40e80cd300 msgr2=0x7f40e81fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40e7e00640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40e80cd300 0x7f40e81fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40e7e00640 1 -- 172.21.15.16:0/652409988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40e80b5050 msgr2=0x7f40e8150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40e7e00640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40e80b5050 0x7f40e8150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40e7e00640 1 -- 172.21.15.16:0/652409988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e815b820 con 0x7f40e815bdc0 2024-09-10T21:18:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40e7e00640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e815bdc0 0x7f40e8151e70 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f40e000e9c0 tx=0x7f40e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40df400640 1 -- 172.21.15.16:0/652409988 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e000cd80 con 0x7f40e815bdc0 2024-09-10T21:18:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40df400640 1 -- 172.21.15.16:0/652409988 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40e0004b30 con 0x7f40e815bdc0 2024-09-10T21:18:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40df400640 1 -- 172.21.15.16:0/652409988 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0005020 con 0x7f40e815bdc0 2024-09-10T21:18:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40ee600640 1 -- 172.21.15.16:0/652409988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e80c9270 con 0x7f40e815bdc0 2024-09-10T21:18:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40ee600640 1 -- 172.21.15.16:0/652409988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e80c9830 con 0x7f40e815bdc0 2024-09-10T21:18:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40ee600640 1 -- 172.21.15.16:0/652409988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e80cf1c0 con 0x7f40e815bdc0 2024-09-10T21:18:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.232+0000 7f40df400640 1 -- 172.21.15.16:0/652409988 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f40e000f040 con 0x7f40e815bdc0 2024-09-10T21:18:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.236+0000 7f40df400640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40c4072610 0x7f40c4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.236+0000 7f40df400640 1 -- 172.21.15.16:0/652409988 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5795+0+0 (secure 0 0 0) 0x7f40e00938a0 con 0x7f40e815bdc0 2024-09-10T21:18:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.236+0000 7f40e7400640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40c4072610 0x7f40c4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.236+0000 7f40e7400640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40c4072610 0x7f40c4074ad0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f40d80ae000 tx=0x7f40d801afe0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.236+0000 7f40df400640 1 -- 172.21.15.16:0/652409988 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f40e815b820 con 0x7f40e815bdc0 2024-09-10T21:18:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:42.448+0000 7f40ee600640 1 -- 172.21.15.16:0/652409988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f40e80704a0 con 0x7f40e815bdc0 2024-09-10T21:18:43.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.056+0000 7f40df400640 1 -- 172.21.15.16:0/652409988 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v98) ==== 104+0+0 (secure 0 0 0) 0x7f40e0016030 con 0x7f40e815bdc0 2024-09-10T21:18:43.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.060+0000 7f40ee600640 1 -- 172.21.15.16:0/652409988 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40c4072610 msgr2=0x7f40c4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:43.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.060+0000 7f40ee600640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40c4072610 0x7f40c4074ad0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f40d80ae000 tx=0x7f40d801afe0 comp rx=0 tx=0).stop 2024-09-10T21:18:43.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.060+0000 7f40ee600640 1 -- 172.21.15.16:0/652409988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e815bdc0 msgr2=0x7f40e8151e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:43.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.060+0000 7f40ee600640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e815bdc0 0x7f40e8151e70 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f40e000e9c0 tx=0x7f40e000ee80 comp rx=0 tx=0).stop 2024-09-10T21:18:43.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.060+0000 7f40ee600640 1 -- 172.21.15.16:0/652409988 shutdown_connections 2024-09-10T21:18:43.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.060+0000 7f40ee600640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f40c4072610 0x7f40c4074ad0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:43.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.060+0000 7f40ee600640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e815bdc0 0x7f40e8151e70 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:43.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.060+0000 7f40ee600640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f40e80cd300 0x7f40e81fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:43.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.060+0000 7f40ee600640 1 --2- 172.21.15.16:0/652409988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40e80b5050 0x7f40e8150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:43.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.060+0000 7f40ee600640 1 -- 172.21.15.16:0/652409988 >> 172.21.15.16:0/652409988 conn(0x7f40e80d1ce0 msgr2=0x7f40e80b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:43.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.060+0000 7f40ee600640 1 -- 172.21.15.16:0/652409988 shutdown_connections 2024-09-10T21:18:43.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.060+0000 7f40ee600640 1 -- 172.21.15.16:0/652409988 wait complete. 2024-09-10T21:18:43.092 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:18:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.244+0000 7f71ffa00640 1 -- 172.21.15.16:0/934358165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e00052f0 msgr2=0x7f71e000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.244+0000 7f71ffa00640 1 --2- 172.21.15.16:0/934358165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e00052f0 0x7f71e000b9d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009eb0 tx=0x7f71f001d5f0 comp rx=0 tx=0).stop 2024-09-10T21:18:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.244+0000 7f71ffa00640 1 -- 172.21.15.16:0/934358165 shutdown_connections 2024-09-10T21:18:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.244+0000 7f71ffa00640 1 --2- 172.21.15.16:0/934358165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e00052f0 0x7f71e000b9d0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.244+0000 7f71ffa00640 1 -- 172.21.15.16:0/934358165 >> 172.21.15.16:0/934358165 conn(0x7f71f80d1e40 msgr2=0x7f71f80d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.244+0000 7f71ffa00640 1 -- 172.21.15.16:0/934358165 shutdown_connections 2024-09-10T21:18:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.244+0000 7f71ffa00640 1 -- 172.21.15.16:0/934358165 wait complete. 2024-09-10T21:18:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.244+0000 7f71ffa00640 1 Processor -- start 2024-09-10T21:18:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.244+0000 7f71ffa00640 1 -- start start 2024-09-10T21:18:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71ffa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e00052f0 0x7f71f8156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71ffa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71f8157a60 0x7f71f8155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71ffa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f71f8159240 0x7f71f8071900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71ffa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f71f80ce850 con 0x7f71f8157a60 2024-09-10T21:18:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71ffa00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71f80d2b70 con 0x7f71e00052f0 2024-09-10T21:18:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71ffa00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f71f80ce9d0 con 0x7f71f8159240 2024-09-10T21:18:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71f7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71f8157a60 0x7f71f8155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71f7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71f8157a60 0x7f71f8155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41360/0 (socket says 172.21.15.16:41360) 2024-09-10T21:18:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71f7e00640 1 -- 172.21.15.16:0/1571765852 learned_addr learned my addr 172.21.15.16:0/1571765852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71fd000640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e00052f0 0x7f71f8156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71fda00640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f71f8159240 0x7f71f8071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71f7e00640 1 -- 172.21.15.16:0/1571765852 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f71f8159240 msgr2=0x7f71f8071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71f7e00640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f71f8159240 0x7f71f8071900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71f7e00640 1 -- 172.21.15.16:0/1571765852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e00052f0 msgr2=0x7f71f8156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71f7e00640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e00052f0 0x7f71f8156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71f7e00640 1 -- 172.21.15.16:0/1571765852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f80c8fb0 con 0x7f71f8157a60 2024-09-10T21:18:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71f7e00640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71f8157a60 0x7f71f8155270 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f71e4003f30 tx=0x7f71e4001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71f4c00640 1 -- 172.21.15.16:0/1571765852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4011640 con 0x7f71f8157a60 2024-09-10T21:18:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71f4c00640 1 -- 172.21.15.16:0/1571765852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71e4016020 con 0x7f71f8157a60 2024-09-10T21:18:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71f4c00640 1 -- 172.21.15.16:0/1571765852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4011a40 con 0x7f71f8157a60 2024-09-10T21:18:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71ffa00640 1 -- 172.21.15.16:0/1571765852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f80c9290 con 0x7f71f8157a60 2024-09-10T21:18:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71ffa00640 1 -- 172.21.15.16:0/1571765852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f8157500 con 0x7f71f8157a60 2024-09-10T21:18:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.248+0000 7f71ffa00640 1 -- 172.21.15.16:0/1571765852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f80c94d0 con 0x7f71f8157a60 2024-09-10T21:18:43.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.252+0000 7f71f4c00640 1 -- 172.21.15.16:0/1571765852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f71e4011be0 con 0x7f71f8157a60 2024-09-10T21:18:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.252+0000 7f71f4c00640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f71d80725e0 0x7f71d8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.252+0000 7f71f4c00640 1 -- 172.21.15.16:0/1571765852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5795+0+0 (secure 0 0 0) 0x7f71e4093fc0 con 0x7f71f8157a60 2024-09-10T21:18:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.252+0000 7f71fd000640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f71d80725e0 0x7f71d8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:43.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.252+0000 7f71fd000640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f71d80725e0 0x7f71d8074aa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f71ec012580 tx=0x7f71ec000ee0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:43.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.256+0000 7f71f4c00640 1 -- 172.21.15.16:0/1571765852 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f71f80c94d0 con 0x7f71f8157a60 2024-09-10T21:18:43.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:43.428+0000 7f71ffa00640 1 -- 172.21.15.16:0/1571765852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f71f80b1a60 con 0x7f71f8157a60 2024-09-10T21:18:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.064+0000 7f71f4c00640 1 -- 172.21.15.16:0/1571765852 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) ==== 181+0+0 (secure 0 0 0) 0x7f71e4020020 con 0x7f71f8157a60 2024-09-10T21:18:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:18:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.068+0000 7f71ffa00640 1 -- 172.21.15.16:0/1571765852 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f71d80725e0 msgr2=0x7f71d8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.068+0000 7f71ffa00640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f71d80725e0 0x7f71d8074aa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f71ec012580 tx=0x7f71ec000ee0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.068+0000 7f71ffa00640 1 -- 172.21.15.16:0/1571765852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71f8157a60 msgr2=0x7f71f8155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.068+0000 7f71ffa00640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71f8157a60 0x7f71f8155270 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f71e4003f30 tx=0x7f71e4001700 comp rx=0 tx=0).stop 2024-09-10T21:18:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.068+0000 7f71ffa00640 1 -- 172.21.15.16:0/1571765852 shutdown_connections 2024-09-10T21:18:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.068+0000 7f71ffa00640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f71d80725e0 0x7f71d8074aa0 secure :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f71ec012580 tx=0x7f71ec000ee0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.068+0000 7f71ffa00640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f71f8159240 0x7f71f8071900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.068+0000 7f71ffa00640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71f8157a60 0x7f71f8155270 secure :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f71e4003f30 tx=0x7f71e4001700 comp rx=0 tx=0).stop 2024-09-10T21:18:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.068+0000 7f71ffa00640 1 --2- 172.21.15.16:0/1571765852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e00052f0 0x7f71f8156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.068+0000 7f71ffa00640 1 -- 172.21.15.16:0/1571765852 >> 172.21.15.16:0/1571765852 conn(0x7f71f80d1e40 msgr2=0x7f71f80cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.068+0000 7f71ffa00640 1 -- 172.21.15.16:0/1571765852 shutdown_connections 2024-09-10T21:18:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.068+0000 7f71ffa00640 1 -- 172.21.15.16:0/1571765852 wait complete. 2024-09-10T21:18:44.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:18:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.248+0000 7f76f7c00640 1 -- 172.21.15.16:0/2334444022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f00d2960 msgr2=0x7f76f00ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.248+0000 7f76f7c00640 1 --2- 172.21.15.16:0/2334444022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f00d2960 0x7f76f00ced90 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f76f00737b0 tx=0x7f76d801c770 comp rx=0 tx=0).stop 2024-09-10T21:18:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.248+0000 7f76f7c00640 1 -- 172.21.15.16:0/2334444022 shutdown_connections 2024-09-10T21:18:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.248+0000 7f76f7c00640 1 --2- 172.21.15.16:0/2334444022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f00d2960 0x7f76f00ced90 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.248+0000 7f76f7c00640 1 --2- 172.21.15.16:0/2334444022 >> v2:172.21.15.52:3300/0 conn(0x7f76f015bcf0 0x7f76f00ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.248+0000 7f76f7c00640 1 -- 172.21.15.16:0/2334444022 >> 172.21.15.16:0/2334444022 conn(0x7f76f00d1e40 msgr2=0x7f76f00d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.248+0000 7f76f7c00640 1 -- 172.21.15.16:0/2334444022 shutdown_connections 2024-09-10T21:18:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.248+0000 7f76f7c00640 1 -- 172.21.15.16:0/2334444022 wait complete. 2024-09-10T21:18:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f7c00640 1 Processor -- start 2024-09-10T21:18:44.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f7c00640 1 -- start start 2024-09-10T21:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f7c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f00b1a60 0x7f76f01582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f7c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76f00d2960 0x7f76f0158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f7c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76f015bcf0 0x7f76f0158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f7c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01690b0 con 0x7f76f00b1a60 2024-09-10T21:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f7c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f76f0168f30 con 0x7f76f015bcf0 2024-09-10T21:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f7c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f76f0169230 con 0x7f76f00d2960 2024-09-10T21:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f00b1a60 0x7f76f01582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f00b1a60 0x7f76f01582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41366/0 (socket says 172.21.15.16:41366) 2024-09-10T21:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f5200640 1 -- 172.21.15.16:0/3985616489 learned_addr learned my addr 172.21.15.16:0/3985616489 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76e7e00640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76f00d2960 0x7f76f0158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f5c00640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76f015bcf0 0x7f76f0158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f5200640 1 -- 172.21.15.16:0/3985616489 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76f00d2960 msgr2=0x7f76f0158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f5200640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76f00d2960 0x7f76f0158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f5200640 1 -- 172.21.15.16:0/3985616489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76f015bcf0 msgr2=0x7f76f0158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f5200640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76f015bcf0 0x7f76f0158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f5200640 1 -- 172.21.15.16:0/3985616489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f014e940 con 0x7f76f00b1a60 2024-09-10T21:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f5200640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f00b1a60 0x7f76f01582f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f76e000ca20 tx=0x7f76e000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76ed600640 1 -- 172.21.15.16:0/3985616489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0007c70 con 0x7f76f00b1a60 2024-09-10T21:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76ed600640 1 -- 172.21.15.16:0/3985616489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76e000f450 con 0x7f76f00b1a60 2024-09-10T21:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76ed600640 1 -- 172.21.15.16:0/3985616489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e000f8a0 con 0x7f76f00b1a60 2024-09-10T21:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f7c00640 1 -- 172.21.15.16:0/3985616489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f014e6a0 con 0x7f76f00b1a60 2024-09-10T21:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f7c00640 1 -- 172.21.15.16:0/3985616489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f0156e50 con 0x7f76f00b1a60 2024-09-10T21:18:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.252+0000 7f76f7c00640 1 -- 172.21.15.16:0/3985616489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f0150ec0 con 0x7f76f00b1a60 2024-09-10T21:18:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.256+0000 7f76ed600640 1 -- 172.21.15.16:0/3985616489 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f76e000fa40 con 0x7f76f00b1a60 2024-09-10T21:18:44.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.256+0000 7f76ed600640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76cc0725e0 0x7f76cc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.256+0000 7f76ed600640 1 -- 172.21.15.16:0/3985616489 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5357+0+0 (secure 0 0 0) 0x7f76e00938b0 con 0x7f76f00b1a60 2024-09-10T21:18:44.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.256+0000 7f76e7e00640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76cc0725e0 0x7f76cc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.256+0000 7f76e7e00640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76cc0725e0 0x7f76cc074aa0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f76d80af000 tx=0x7f76d801afb0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.260+0000 7f76ed600640 1 -- 172.21.15.16:0/3985616489 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f76e0014030 con 0x7f76f00b1a60 2024-09-10T21:18:44.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:44.440+0000 7f76f7c00640 1 -- 172.21.15.16:0/3985616489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f76f00c8a50 con 0x7f76f00b1a60 2024-09-10T21:18:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.064+0000 7f76ed600640 1 -- 172.21.15.16:0/3985616489 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) ==== 169+0+0 (secure 0 0 0) 0x7f76e005f0e0 con 0x7f76f00b1a60 2024-09-10T21:18:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-10T21:18:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.072+0000 7f76f7c00640 1 -- 172.21.15.16:0/3985616489 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76cc0725e0 msgr2=0x7f76cc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.072+0000 7f76f7c00640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76cc0725e0 0x7f76cc074aa0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f76d80af000 tx=0x7f76d801afb0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.072+0000 7f76f7c00640 1 -- 172.21.15.16:0/3985616489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f00b1a60 msgr2=0x7f76f01582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.072+0000 7f76f7c00640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f00b1a60 0x7f76f01582f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f76e000ca20 tx=0x7f76e000cee0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.072+0000 7f76f7c00640 1 -- 172.21.15.16:0/3985616489 shutdown_connections 2024-09-10T21:18:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.072+0000 7f76f7c00640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f76cc0725e0 0x7f76cc074aa0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.072+0000 7f76f7c00640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76f015bcf0 0x7f76f0158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.072+0000 7f76f7c00640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f76f00d2960 0x7f76f0158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.072+0000 7f76f7c00640 1 --2- 172.21.15.16:0/3985616489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f00b1a60 0x7f76f01582f0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.072+0000 7f76f7c00640 1 -- 172.21.15.16:0/3985616489 >> 172.21.15.16:0/3985616489 conn(0x7f76f00d1e40 msgr2=0x7f76f00af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.072+0000 7f76f7c00640 1 -- 172.21.15.16:0/3985616489 shutdown_connections 2024-09-10T21:18:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.072+0000 7f76f7c00640 1 -- 172.21.15.16:0/3985616489 wait complete. 2024-09-10T21:18:45.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:18:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.244+0000 7f757d000640 1 -- 172.21.15.16:0/3787922672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75780b13c0 msgr2=0x7f7578071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.244+0000 7f757d000640 1 --2- 172.21.15.16:0/3787922672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75780b13c0 0x7f7578071520 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f7578073290 tx=0x7f756001c7a0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.244+0000 7f757d000640 1 -- 172.21.15.16:0/3787922672 shutdown_connections 2024-09-10T21:18:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.244+0000 7f757d000640 1 --2- 172.21.15.16:0/3787922672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75780b13c0 0x7f7578071520 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.244+0000 7f757d000640 1 --2- 172.21.15.16:0/3787922672 >> v2:172.21.15.52:3300/0 conn(0x7f757815f400 0x7f7578070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.244+0000 7f757d000640 1 -- 172.21.15.16:0/3787922672 >> 172.21.15.16:0/3787922672 conn(0x7f757809b130 msgr2=0x7f75780d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.244+0000 7f757d000640 1 -- 172.21.15.16:0/3787922672 shutdown_connections 2024-09-10T21:18:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.244+0000 7f757d000640 1 -- 172.21.15.16:0/3787922672 wait complete. 2024-09-10T21:18:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.244+0000 7f757d000640 1 Processor -- start 2024-09-10T21:18:45.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f757d000640 1 -- start start 2024-09-10T21:18:45.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f757d000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75780b13c0 0x7f75781f9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:45.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f757d000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75780cfc80 0x7f75780b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:45.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f757d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757815f400 0x7f7578158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:45.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f757d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75781695f0 con 0x7f757815f400 2024-09-10T21:18:45.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f757d000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f75780d1270 con 0x7f75780b13c0 2024-09-10T21:18:45.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f757d000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7578169770 con 0x7f75780cfc80 2024-09-10T21:18:45.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f7576800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757815f400 0x7f7578158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:45.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f7576800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757815f400 0x7f7578158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41390/0 (socket says 172.21.15.16:41390) 2024-09-10T21:18:45.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f7576800640 1 -- 172.21.15.16:0/3730595528 learned_addr learned my addr 172.21.15.16:0/3730595528 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:45.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f7575e00640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75780b13c0 0x7f75781f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f756fe00640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75780cfc80 0x7f75780b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f7576800640 1 -- 172.21.15.16:0/3730595528 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75780cfc80 msgr2=0x7f75780b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f7576800640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75780cfc80 0x7f75780b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f7576800640 1 -- 172.21.15.16:0/3730595528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75780b13c0 msgr2=0x7f75781f9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f7576800640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75780b13c0 0x7f75781f9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f7576800640 1 -- 172.21.15.16:0/3730595528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75780b50f0 con 0x7f757815f400 2024-09-10T21:18:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f7576800640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757815f400 0x7f7578158ca0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f757000c780 tx=0x7f757000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f756e000640 1 -- 172.21.15.16:0/3730595528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570004b70 con 0x7f757815f400 2024-09-10T21:18:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f756e000640 1 -- 172.21.15.16:0/3730595528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7570004e00 con 0x7f757815f400 2024-09-10T21:18:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f756e000640 1 -- 172.21.15.16:0/3730595528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757000aa90 con 0x7f757815f400 2024-09-10T21:18:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f757d000640 1 -- 172.21.15.16:0/3730595528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75781f9ce0 con 0x7f757815f400 2024-09-10T21:18:45.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f757d000640 1 -- 172.21.15.16:0/3730595528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75781fa2a0 con 0x7f757815f400 2024-09-10T21:18:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.248+0000 7f757d000640 1 -- 172.21.15.16:0/3730595528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75780af280 con 0x7f757815f400 2024-09-10T21:18:45.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.252+0000 7f756e000640 1 -- 172.21.15.16:0/3730595528 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f757000ac30 con 0x7f757815f400 2024-09-10T21:18:45.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.252+0000 7f756e000640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f75540725e0 0x7f7554074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:45.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.252+0000 7f756e000640 1 -- 172.21.15.16:0/3730595528 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7f7570093250 con 0x7f757815f400 2024-09-10T21:18:45.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.252+0000 7f7575e00640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f75540725e0 0x7f7554074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:45.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.252+0000 7f7575e00640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f75540725e0 0x7f7554074aa0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7568005d70 tx=0x7f7568005d00 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:45.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.256+0000 7f756e000640 1 -- 172.21.15.16:0/3730595528 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7570020070 con 0x7f757815f400 2024-09-10T21:18:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.432+0000 7f757d000640 1 -- 172.21.15.16:0/3730595528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f75780d00b0 con 0x7f757815f400 2024-09-10T21:18:45.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.436+0000 7f756e000640 1 -- 172.21.15.16:0/3730595528 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v64) ==== 71+0+0 (secure 0 0 0) 0x7f757005ec20 con 0x7f757815f400 2024-09-10T21:18:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.436+0000 7f757d000640 1 -- 172.21.15.16:0/3730595528 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f75540725e0 msgr2=0x7f7554074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.436+0000 7f757d000640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f75540725e0 0x7f7554074aa0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7568005d70 tx=0x7f7568005d00 comp rx=0 tx=0).stop 2024-09-10T21:18:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.436+0000 7f757d000640 1 -- 172.21.15.16:0/3730595528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757815f400 msgr2=0x7f7578158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.436+0000 7f757d000640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757815f400 0x7f7578158ca0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f757000c780 tx=0x7f757000cc40 comp rx=0 tx=0).stop 2024-09-10T21:18:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.440+0000 7f757d000640 1 -- 172.21.15.16:0/3730595528 shutdown_connections 2024-09-10T21:18:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.440+0000 7f757d000640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f75540725e0 0x7f7554074aa0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.440+0000 7f757d000640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757815f400 0x7f7578158ca0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.440+0000 7f757d000640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f75780cfc80 0x7f75780b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.440+0000 7f757d000640 1 --2- 172.21.15.16:0/3730595528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75780b13c0 0x7f75781f9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.440+0000 7f757d000640 1 -- 172.21.15.16:0/3730595528 >> 172.21.15.16:0/3730595528 conn(0x7f757809b130 msgr2=0x7f75780aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.440+0000 7f757d000640 1 -- 172.21.15.16:0/3730595528 shutdown_connections 2024-09-10T21:18:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.440+0000 7f757d000640 1 -- 172.21.15.16:0/3730595528 wait complete. 2024-09-10T21:18:45.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2024-09-10T21:18:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.612+0000 7fb18a200640 1 -- 172.21.15.16:0/1464011091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1680052f0 msgr2=0x7fb16800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.612+0000 7fb18a200640 1 --2- 172.21.15.16:0/1464011091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1680052f0 0x7fb16800b9d0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb17c006fd0 tx=0x7fb17c01d2a0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.612+0000 7fb18a200640 1 -- 172.21.15.16:0/1464011091 shutdown_connections 2024-09-10T21:18:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.612+0000 7fb18a200640 1 --2- 172.21.15.16:0/1464011091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1680052f0 0x7fb16800b9d0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.612+0000 7fb18a200640 1 -- 172.21.15.16:0/1464011091 >> 172.21.15.16:0/1464011091 conn(0x7fb1840b15c0 msgr2=0x7fb1840b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.612+0000 7fb18a200640 1 -- 172.21.15.16:0/1464011091 shutdown_connections 2024-09-10T21:18:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.612+0000 7fb18a200640 1 -- 172.21.15.16:0/1464011091 wait complete. 2024-09-10T21:18:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb18a200640 1 Processor -- start 2024-09-10T21:18:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb18a200640 1 -- start start 2024-09-10T21:18:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb18a200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1680052f0 0x7fb1840aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb18a200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1840c96e0 0x7fb1841557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb18a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb184155d10 0x7fb1841f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb18a200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb184074190 con 0x7fb184155d10 2024-09-10T21:18:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb18a200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1840c5310 con 0x7fb1680052f0 2024-09-10T21:18:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb18a200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb184074310 con 0x7fb1840c96e0 2024-09-10T21:18:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb189800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb184155d10 0x7fb1841f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb188e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1680052f0 0x7fb1840aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb17be00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1840c96e0 0x7fb1841557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb189800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb184155d10 0x7fb1841f7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41404/0 (socket says 172.21.15.16:41404) 2024-09-10T21:18:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb189800640 1 -- 172.21.15.16:0/1242373367 learned_addr learned my addr 172.21.15.16:0/1242373367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb189800640 1 -- 172.21.15.16:0/1242373367 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1840c96e0 msgr2=0x7fb1841557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb189800640 1 --2- 172.21.15.16:0/1242373367 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1840c96e0 0x7fb1841557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb189800640 1 -- 172.21.15.16:0/1242373367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1680052f0 msgr2=0x7fb1840aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb189800640 1 --2- 172.21.15.16:0/1242373367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1680052f0 0x7fb1840aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb189800640 1 -- 172.21.15.16:0/1242373367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1841f8910 con 0x7fb184155d10 2024-09-10T21:18:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb189800640 1 --2- 172.21.15.16:0/1242373367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb184155d10 0x7fb1841f7a90 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fb17c01db40 tx=0x7fb17c004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb181600640 1 -- 172.21.15.16:0/1242373367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c001a50 con 0x7fb184155d10 2024-09-10T21:18:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb181600640 1 -- 172.21.15.16:0/1242373367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb17c003980 con 0x7fb184155d10 2024-09-10T21:18:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb18a200640 1 -- 172.21.15.16:0/1242373367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1840adbe0 con 0x7fb184155d10 2024-09-10T21:18:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb181600640 1 -- 172.21.15.16:0/1242373367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c003dd0 con 0x7fb184155d10 2024-09-10T21:18:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.616+0000 7fb18a200640 1 -- 172.21.15.16:0/1242373367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1840ae1a0 con 0x7fb184155d10 2024-09-10T21:18:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.620+0000 7fb18a200640 1 -- 172.21.15.16:0/1242373367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb18415f510 con 0x7fb184155d10 2024-09-10T21:18:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.620+0000 7fb181600640 1 -- 172.21.15.16:0/1242373367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb17c006070 con 0x7fb184155d10 2024-09-10T21:18:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.620+0000 7fb181600640 1 --2- 172.21.15.16:0/1242373367 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb160072610 0x7fb160074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.620+0000 7fb181600640 1 -- 172.21.15.16:0/1242373367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7fb17c12ca00 con 0x7fb184155d10 2024-09-10T21:18:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.620+0000 7fb188e00640 1 --2- 172.21.15.16:0/1242373367 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb160072610 0x7fb160074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.620+0000 7fb188e00640 1 --2- 172.21.15.16:0/1242373367 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb160072610 0x7fb160074ad0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb174000c10 tx=0x7fb174006b80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.624+0000 7fb181600640 1 -- 172.21.15.16:0/1242373367 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb184072d30 con 0x7fb184155d10 2024-09-10T21:18:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:45.864+0000 7fb18a200640 1 -- 172.21.15.16:0/1242373367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) -- 0x7fb1840c6100 con 0x7fb184155d10 2024-09-10T21:18:46.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.068+0000 7fb181600640 1 -- 172.21.15.16:0/1242373367 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v425) ==== 148+0+0 (secure 0 0 0) 0x7fb17c0f7cf0 con 0x7fb184155d10 2024-09-10T21:18:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.072+0000 7fb18a200640 1 -- 172.21.15.16:0/1242373367 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb160072610 msgr2=0x7fb160074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.072+0000 7fb18a200640 1 --2- 172.21.15.16:0/1242373367 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb160072610 0x7fb160074ad0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb174000c10 tx=0x7fb174006b80 comp rx=0 tx=0).stop 2024-09-10T21:18:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.072+0000 7fb18a200640 1 -- 172.21.15.16:0/1242373367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb184155d10 msgr2=0x7fb1841f7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.072+0000 7fb18a200640 1 --2- 172.21.15.16:0/1242373367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb184155d10 0x7fb1841f7a90 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fb17c01db40 tx=0x7fb17c004f80 comp rx=0 tx=0).stop 2024-09-10T21:18:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.072+0000 7fb18a200640 1 -- 172.21.15.16:0/1242373367 shutdown_connections 2024-09-10T21:18:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.072+0000 7fb18a200640 1 --2- 172.21.15.16:0/1242373367 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb160072610 0x7fb160074ad0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.072+0000 7fb18a200640 1 --2- 172.21.15.16:0/1242373367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb184155d10 0x7fb1841f7a90 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.072+0000 7fb18a200640 1 --2- 172.21.15.16:0/1242373367 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb1840c96e0 0x7fb1841557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.072+0000 7fb18a200640 1 --2- 172.21.15.16:0/1242373367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1680052f0 0x7fb1840aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.072+0000 7fb18a200640 1 -- 172.21.15.16:0/1242373367 >> 172.21.15.16:0/1242373367 conn(0x7fb1840b15c0 msgr2=0x7fb18406e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.072+0000 7fb18a200640 1 -- 172.21.15.16:0/1242373367 shutdown_connections 2024-09-10T21:18:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.072+0000 7fb18a200640 1 -- 172.21.15.16:0/1242373367 wait complete. 2024-09-10T21:18:46.097 INFO:tasks.cephfs_test_runner:test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-09-10T21:18:46.097 INFO:tasks.cephfs_test_runner:Starting test: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) When a client has a bug (which we will simulate) preventing it from releasing caps, 2024-09-10T21:18:46.097 INFO:tasks.cephfs_test_runner:test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-10T21:18:46.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:18:46.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.240+0000 7f9066400640 1 -- 172.21.15.16:0/3419210273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9044003c70 msgr2=0x7f9044004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.240+0000 7f9066400640 1 --2- 172.21.15.16:0/3419210273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9044003c70 0x7f9044004070 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f9050001860 tx=0x7f905001d340 comp rx=0 tx=0).stop 2024-09-10T21:18:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.240+0000 7f905fe00640 1 -- 172.21.15.16:0/3419210273 reap_dead start 2024-09-10T21:18:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.240+0000 7f9066400640 1 -- 172.21.15.16:0/3419210273 shutdown_connections 2024-09-10T21:18:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.240+0000 7f9066400640 1 -- 172.21.15.16:0/3419210273 >> 172.21.15.16:0/3419210273 conn(0x7f90600cfb20 msgr2=0x7f90600cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.240+0000 7f9066400640 1 -- 172.21.15.16:0/3419210273 shutdown_connections 2024-09-10T21:18:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.240+0000 7f9066400640 1 -- 172.21.15.16:0/3419210273 wait complete. 2024-09-10T21:18:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.240+0000 7f9066400640 1 Processor -- start 2024-09-10T21:18:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.240+0000 7f9066400640 1 -- start start 2024-09-10T21:18:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9066400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9044003c70 0x7f9060151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9066400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90600c8970 0x7f90600c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9066400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f90600c6720 0x7f9060154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9066400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9060072a40 con 0x7f90600c8970 2024-09-10T21:18:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9066400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f90600728c0 con 0x7f9044003c70 2024-09-10T21:18:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9066400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9060072bc0 con 0x7f90600c6720 2024-09-10T21:18:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9057e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90600c8970 0x7f90600c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9057e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90600c8970 0x7f90600c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41414/0 (socket says 172.21.15.16:41414) 2024-09-10T21:18:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9057e00640 1 -- 172.21.15.16:0/3636753399 learned_addr learned my addr 172.21.15.16:0/3636753399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f905f400640 1 --2- 172.21.15.16:0/3636753399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9044003c70 0x7f9060151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9057e00640 1 -- 172.21.15.16:0/3636753399 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f90600c6720 msgr2=0x7f9060154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9057e00640 1 --2- 172.21.15.16:0/3636753399 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f90600c6720 0x7f9060154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9057e00640 1 -- 172.21.15.16:0/3636753399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9044003c70 msgr2=0x7f9060151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9057e00640 1 --2- 172.21.15.16:0/3636753399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9044003c70 0x7f9060151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9057e00640 1 -- 172.21.15.16:0/3636753399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90601fb000 con 0x7f90600c8970 2024-09-10T21:18:46.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9057e00640 1 --2- 172.21.15.16:0/3636753399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90600c8970 0x7f90600c61e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f904800ea70 tx=0x7f904800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:46.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9057400640 1 -- 172.21.15.16:0/3636753399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904800dc90 con 0x7f90600c8970 2024-09-10T21:18:46.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9057400640 1 -- 172.21.15.16:0/3636753399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f904800de30 con 0x7f90600c8970 2024-09-10T21:18:46.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9057400640 1 -- 172.21.15.16:0/3636753399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048004270 con 0x7f90600c8970 2024-09-10T21:18:46.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9066400640 1 -- 172.21.15.16:0/3636753399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90601fb2e0 con 0x7f90600c8970 2024-09-10T21:18:46.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9066400640 1 -- 172.21.15.16:0/3636753399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90601fb8a0 con 0x7f90600c8970 2024-09-10T21:18:46.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.244+0000 7f9066400640 1 -- 172.21.15.16:0/3636753399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90600ca170 con 0x7f90600c8970 2024-09-10T21:18:46.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.248+0000 7f9057400640 1 -- 172.21.15.16:0/3636753399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f904800b040 con 0x7f90600c8970 2024-09-10T21:18:46.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.248+0000 7f9057400640 1 --2- 172.21.15.16:0/3636753399 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f903c0725e0 0x7f903c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.248+0000 7f9057400640 1 -- 172.21.15.16:0/3636753399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7f9048093950 con 0x7f90600c8970 2024-09-10T21:18:46.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.248+0000 7f905f400640 1 --2- 172.21.15.16:0/3636753399 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f903c0725e0 0x7f903c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.248+0000 7f905f400640 1 --2- 172.21.15.16:0/3636753399 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f903c0725e0 0x7f903c074aa0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9050001830 tx=0x7f9050019720 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:46.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.252+0000 7f9057400640 1 -- 172.21.15.16:0/3636753399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f904805f290 con 0x7f90600c8970 2024-09-10T21:18:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.424+0000 7f9066400640 1 -- 172.21.15.16:0/3636753399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90600b5aa0 con 0x7f90600c8970 2024-09-10T21:18:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.424+0000 7f9057400640 1 -- 172.21.15.16:0/3636753399 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+12994 (secure 0 0 0) 0x7f904805eb90 con 0x7f90600c8970 2024-09-10T21:18:46.431 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:46.431 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":130,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:18:45.068619+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6835/3672156966":"2024-09-11T21:18:42.058529+0000","172.21.15.52:6833/3672156966":"2024-09-11T21:18:42.058529+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:18:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.428+0000 7f9066400640 1 -- 172.21.15.16:0/3636753399 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f903c0725e0 msgr2=0x7f903c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.428+0000 7f9066400640 1 --2- 172.21.15.16:0/3636753399 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f903c0725e0 0x7f903c074aa0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9050001830 tx=0x7f9050019720 comp rx=0 tx=0).stop 2024-09-10T21:18:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.428+0000 7f9066400640 1 -- 172.21.15.16:0/3636753399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90600c8970 msgr2=0x7f90600c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.428+0000 7f9066400640 1 --2- 172.21.15.16:0/3636753399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90600c8970 0x7f90600c61e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f904800ea70 tx=0x7f904800ef30 comp rx=0 tx=0).stop 2024-09-10T21:18:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.428+0000 7f9066400640 1 -- 172.21.15.16:0/3636753399 shutdown_connections 2024-09-10T21:18:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.428+0000 7f9066400640 1 --2- 172.21.15.16:0/3636753399 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f903c0725e0 0x7f903c074aa0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.428+0000 7f9066400640 1 --2- 172.21.15.16:0/3636753399 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f90600c6720 0x7f9060154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.428+0000 7f9066400640 1 --2- 172.21.15.16:0/3636753399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90600c8970 0x7f90600c61e0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.428+0000 7f9066400640 1 --2- 172.21.15.16:0/3636753399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9044003c70 0x7f9060151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.428+0000 7f9066400640 1 -- 172.21.15.16:0/3636753399 >> 172.21.15.16:0/3636753399 conn(0x7f90600cfb20 msgr2=0x7f90600b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.428+0000 7f9066400640 1 -- 172.21.15.16:0/3636753399 shutdown_connections 2024-09-10T21:18:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.428+0000 7f9066400640 1 -- 172.21.15.16:0/3636753399 wait complete. 2024-09-10T21:18:46.453 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:10:17.493770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:18:46.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:18:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 -- 172.21.15.16:0/2253472342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0c8820 msgr2=0x7f661c0c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 --2- 172.21.15.16:0/2253472342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0c8820 0x7f661c0c8c60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f661c068b10 tx=0x7f660401c790 comp rx=0 tx=0).stop 2024-09-10T21:18:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 -- 172.21.15.16:0/2253472342 shutdown_connections 2024-09-10T21:18:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 --2- 172.21.15.16:0/2253472342 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f661c0c91a0 0x7f661c165a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 --2- 172.21.15.16:0/2253472342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0c8820 0x7f661c0c8c60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 -- 172.21.15.16:0/2253472342 >> 172.21.15.16:0/2253472342 conn(0x7f661c0d0390 msgr2=0x7f661c0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 -- 172.21.15.16:0/2253472342 shutdown_connections 2024-09-10T21:18:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 -- 172.21.15.16:0/2253472342 wait complete. 2024-09-10T21:18:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 Processor -- start 2024-09-10T21:18:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 -- start start 2024-09-10T21:18:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f661c0c8820 0x7f661c0c2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0c91a0 0x7f661c0c2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f661c15f420 0x7f661c155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f661c168c70 con 0x7f661c15f420 2024-09-10T21:18:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f661c168af0 con 0x7f661c0c91a0 2024-09-10T21:18:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6622c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f661c168df0 con 0x7f661c0c8820 2024-09-10T21:18:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6620c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f661c15f420 0x7f661c155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6620c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f661c15f420 0x7f661c155440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41420/0 (socket says 172.21.15.16:41420) 2024-09-10T21:18:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6620c00640 1 -- 172.21.15.16:0/1504153985 learned_addr learned my addr 172.21.15.16:0/1504153985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6613e00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0c91a0 0x7f661c0c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f661be00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f661c0c8820 0x7f661c0c2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6620c00640 1 -- 172.21.15.16:0/1504153985 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f661c0c8820 msgr2=0x7f661c0c2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6620c00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f661c0c8820 0x7f661c0c2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6620c00640 1 -- 172.21.15.16:0/1504153985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0c91a0 msgr2=0x7f661c0c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6620c00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0c91a0 0x7f661c0c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.608+0000 7f6620c00640 1 -- 172.21.15.16:0/1504153985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661c0c35f0 con 0x7f661c15f420 2024-09-10T21:18:46.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.612+0000 7f6620c00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f661c15f420 0x7f661c155440 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f661400c420 tx=0x7f661400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:46.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.612+0000 7f6618c00640 1 -- 172.21.15.16:0/1504153985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614003980 con 0x7f661c15f420 2024-09-10T21:18:46.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.612+0000 7f6618c00640 1 -- 172.21.15.16:0/1504153985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6614003c10 con 0x7f661c15f420 2024-09-10T21:18:46.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.612+0000 7f6622c00640 1 -- 172.21.15.16:0/1504153985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661c0c38d0 con 0x7f661c15f420 2024-09-10T21:18:46.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.612+0000 7f6618c00640 1 -- 172.21.15.16:0/1504153985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661400b690 con 0x7f661c15f420 2024-09-10T21:18:46.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.612+0000 7f6622c00640 1 -- 172.21.15.16:0/1504153985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661c0c3e90 con 0x7f661c15f420 2024-09-10T21:18:46.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.612+0000 7f6622c00640 1 -- 172.21.15.16:0/1504153985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f661c0c8c60 con 0x7f661c15f420 2024-09-10T21:18:46.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.612+0000 7f6618c00640 1 -- 172.21.15.16:0/1504153985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f661400b040 con 0x7f661c15f420 2024-09-10T21:18:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.612+0000 7f6618c00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65f80725e0 0x7f65f8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.612+0000 7f6618c00640 1 -- 172.21.15.16:0/1504153985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7f6614093470 con 0x7f661c15f420 2024-09-10T21:18:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.612+0000 7f661be00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65f80725e0 0x7f65f8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.616+0000 7f661be00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65f80725e0 0x7f65f8074aa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f660c001780 tx=0x7f660c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.616+0000 7f6618c00640 1 -- 172.21.15.16:0/1504153985 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f661405e760 con 0x7f661c15f420 2024-09-10T21:18:46.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6622c00640 1 -- 172.21.15.16:0/1504153985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f661c070290 con 0x7f661c15f420 2024-09-10T21:18:46.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6618c00640 1 -- 172.21.15.16:0/1504153985 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) ==== 93+0+10 (secure 0 0 0) 0x7f6614063190 con 0x7f661c15f420 2024-09-10T21:18:46.796 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-10T21:18:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6622c00640 1 -- 172.21.15.16:0/1504153985 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65f80725e0 msgr2=0x7f65f8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6622c00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65f80725e0 0x7f65f8074aa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f660c001780 tx=0x7f660c009040 comp rx=0 tx=0).stop 2024-09-10T21:18:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6622c00640 1 -- 172.21.15.16:0/1504153985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f661c15f420 msgr2=0x7f661c155440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6622c00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f661c15f420 0x7f661c155440 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f661400c420 tx=0x7f661400c8e0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6622c00640 1 -- 172.21.15.16:0/1504153985 shutdown_connections 2024-09-10T21:18:46.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6622c00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65f80725e0 0x7f65f8074aa0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6622c00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f661c15f420 0x7f661c155440 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6622c00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f661c0c91a0 0x7f661c0c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6622c00640 1 --2- 172.21.15.16:0/1504153985 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f661c0c8820 0x7f661c0c2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6622c00640 1 -- 172.21.15.16:0/1504153985 >> 172.21.15.16:0/1504153985 conn(0x7f661c0d0390 msgr2=0x7f661c15da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:46.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6622c00640 1 -- 172.21.15.16:0/1504153985 shutdown_connections 2024-09-10T21:18:46.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.792+0000 7f6622c00640 1 -- 172.21.15.16:0/1504153985 wait complete. 2024-09-10T21:18:46.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2024-09-10T21:18:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 -- 172.21.15.16:0/3942076778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9fc0b3200 msgr2=0x7fd9fc0afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 --2- 172.21.15.16:0/3942076778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9fc0b3200 0x7fd9fc0afae0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009840 tx=0x7fd9e401c820 comp rx=0 tx=0).stop 2024-09-10T21:18:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 -- 172.21.15.16:0/3942076778 shutdown_connections 2024-09-10T21:18:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 --2- 172.21.15.16:0/3942076778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9fc0b3200 0x7fd9fc0afae0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 -- 172.21.15.16:0/3942076778 >> 172.21.15.16:0/3942076778 conn(0x7fd9fc0d1d80 msgr2=0x7fd9fc0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 -- 172.21.15.16:0/3942076778 shutdown_connections 2024-09-10T21:18:46.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 -- 172.21.15.16:0/3942076778 wait complete. 2024-09-10T21:18:46.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 Processor -- start 2024-09-10T21:18:46.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 -- start start 2024-09-10T21:18:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9fc0d0890 0x7fd9fc167500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc15f450 0x7fd9fc148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd9fc06f080 0x7fd9fc149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc0b0730 con 0x7fd9fc0d0890 2024-09-10T21:18:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc0b08b0 con 0x7fd9fc15f450 2024-09-10T21:18:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fda01600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd9fc0b0a30 con 0x7fd9fc06f080 2024-09-10T21:18:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fd9faa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9fc0d0890 0x7fd9fc167500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fd9faa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9fc0d0890 0x7fd9fc167500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41438/0 (socket says 172.21.15.16:41438) 2024-09-10T21:18:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fd9faa00640 1 -- 172.21.15.16:0/3330008734 learned_addr learned my addr 172.21.15.16:0/3330008734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fd9f3e00640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc15f450 0x7fd9fc148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fd9fb400640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd9fc06f080 0x7fd9fc149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fd9faa00640 1 -- 172.21.15.16:0/3330008734 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd9fc06f080 msgr2=0x7fd9fc149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fd9faa00640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd9fc06f080 0x7fd9fc149400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fd9faa00640 1 -- 172.21.15.16:0/3330008734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc15f450 msgr2=0x7fd9fc148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fd9faa00640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc15f450 0x7fd9fc148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fd9faa00640 1 -- 172.21.15.16:0/3330008734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc145cb0 con 0x7fd9fc0d0890 2024-09-10T21:18:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.960+0000 7fd9f3e00640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc15f450 0x7fd9fc148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.964+0000 7fd9faa00640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9fc0d0890 0x7fd9fc167500 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00c3c0 tx=0x7fd9ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.964+0000 7fd9f2a00640 1 -- 172.21.15.16:0/3330008734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec011790 con 0x7fd9fc0d0890 2024-09-10T21:18:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.964+0000 7fd9f2a00640 1 -- 172.21.15.16:0/3330008734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9ec017070 con 0x7fd9fc0d0890 2024-09-10T21:18:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.964+0000 7fd9f2a00640 1 -- 172.21.15.16:0/3330008734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec011d20 con 0x7fd9fc0d0890 2024-09-10T21:18:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.964+0000 7fda01600640 1 -- 172.21.15.16:0/3330008734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9fc145f90 con 0x7fd9fc0d0890 2024-09-10T21:18:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.964+0000 7fda01600640 1 -- 172.21.15.16:0/3330008734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9fc146550 con 0x7fd9fc0d0890 2024-09-10T21:18:46.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.964+0000 7fda01600640 1 -- 172.21.15.16:0/3330008734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9fc0cfc80 con 0x7fd9fc0d0890 2024-09-10T21:18:46.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.964+0000 7fd9f2a00640 1 -- 172.21.15.16:0/3330008734 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd9ec01c460 con 0x7fd9fc0d0890 2024-09-10T21:18:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.964+0000 7fd9f2a00640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9d80725e0 0x7fd9d8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.964+0000 7fd9f2a00640 1 -- 172.21.15.16:0/3330008734 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7fd9ec092e10 con 0x7fd9fc0d0890 2024-09-10T21:18:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.964+0000 7fd9f3e00640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9d80725e0 0x7fd9d8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.968+0000 7fd9f3e00640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9d80725e0 0x7fd9d8074aa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40af000 tx=0x7fd9e401b060 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:46.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:46.968+0000 7fd9f2a00640 1 -- 172.21.15.16:0/3330008734 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd9ec096030 con 0x7fd9fc0d0890 2024-09-10T21:18:47.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:47.204+0000 7fda01600640 1 -- 172.21.15.16:0/3330008734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) -- 0x7fd9fc074ba0 con 0x7fd9fc0d0890 2024-09-10T21:18:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.076+0000 7fd9f2a00640 1 -- 172.21.15.16:0/3330008734 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v427) ==== 154+0+0 (secure 0 0 0) 0x7fd9ec065e20 con 0x7fd9fc0d0890 2024-09-10T21:18:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.076+0000 7fda01600640 1 -- 172.21.15.16:0/3330008734 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9d80725e0 msgr2=0x7fd9d8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.076+0000 7fda01600640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9d80725e0 0x7fd9d8074aa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40af000 tx=0x7fd9e401b060 comp rx=0 tx=0).stop 2024-09-10T21:18:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.076+0000 7fda01600640 1 -- 172.21.15.16:0/3330008734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9fc0d0890 msgr2=0x7fd9fc167500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.076+0000 7fda01600640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9fc0d0890 0x7fd9fc167500 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00c3c0 tx=0x7fd9ec00c880 comp rx=0 tx=0).stop 2024-09-10T21:18:48.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.076+0000 7fda01600640 1 -- 172.21.15.16:0/3330008734 shutdown_connections 2024-09-10T21:18:48.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.076+0000 7fda01600640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd9d80725e0 0x7fd9d8074aa0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.076+0000 7fda01600640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd9fc06f080 0x7fd9fc149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.076+0000 7fda01600640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc15f450 0x7fd9fc148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.076+0000 7fda01600640 1 --2- 172.21.15.16:0/3330008734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9fc0d0890 0x7fd9fc167500 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.076+0000 7fda01600640 1 -- 172.21.15.16:0/3330008734 >> 172.21.15.16:0/3330008734 conn(0x7fd9fc0d1d80 msgr2=0x7fd9fc071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:48.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.080+0000 7fda01600640 1 -- 172.21.15.16:0/3330008734 shutdown_connections 2024-09-10T21:18:48.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.080+0000 7fda01600640 1 -- 172.21.15.16:0/3330008734 wait complete. 2024-09-10T21:18:48.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:18:48.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.252+0000 7f4708a00640 1 -- 172.21.15.16:0/2618645818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f470415a270 msgr2=0x7f47040ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.252+0000 7f4708a00640 1 --2- 172.21.15.16:0/2618645818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f470415a270 0x7f47040ce610 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009840 tx=0x7f46f401c760 comp rx=0 tx=0).stop 2024-09-10T21:18:48.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.252+0000 7f4708a00640 1 -- 172.21.15.16:0/2618645818 shutdown_connections 2024-09-10T21:18:48.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.252+0000 7f4708a00640 1 --2- 172.21.15.16:0/2618645818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f470415a270 0x7f47040ce610 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.252+0000 7f4708a00640 1 -- 172.21.15.16:0/2618645818 >> 172.21.15.16:0/2618645818 conn(0x7f47040cfb20 msgr2=0x7f47040cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:48.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.252+0000 7f4708a00640 1 -- 172.21.15.16:0/2618645818 shutdown_connections 2024-09-10T21:18:48.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.252+0000 7f4708a00640 1 -- 172.21.15.16:0/2618645818 wait complete. 2024-09-10T21:18:48.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.252+0000 7f4708a00640 1 Processor -- start 2024-09-10T21:18:48.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.252+0000 7f4708a00640 1 -- start start 2024-09-10T21:18:48.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f4708a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f47040ca170 0x7f47040b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f4708a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47040d06b0 0x7f47040740a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f4708a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f470415a270 0x7f47041fba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f4708a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4704168da0 con 0x7f47040d06b0 2024-09-10T21:18:48.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f4708a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4704168c20 con 0x7f470415a270 2024-09-10T21:18:48.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f4708a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4704168f20 con 0x7f47040ca170 2024-09-10T21:18:48.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f46f9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47040d06b0 0x7f47040740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f46f9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47040d06b0 0x7f47040740a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41458/0 (socket says 172.21.15.16:41458) 2024-09-10T21:18:48.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f46f9400640 1 -- 172.21.15.16:0/3774740190 learned_addr learned my addr 172.21.15.16:0/3774740190 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:48.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f4701e00640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f47040ca170 0x7f47040b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f4702800640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f470415a270 0x7f47041fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f46f9400640 1 -- 172.21.15.16:0/3774740190 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f47040ca170 msgr2=0x7f47040b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f46f9400640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f47040ca170 0x7f47040b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f46f9400640 1 -- 172.21.15.16:0/3774740190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f470415a270 msgr2=0x7f47041fba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f46f9400640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f470415a270 0x7f47041fba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f46f9400640 1 -- 172.21.15.16:0/3774740190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47040b2d70 con 0x7f47040d06b0 2024-09-10T21:18:48.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f4702800640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f470415a270 0x7f47041fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:48.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f46f9400640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47040d06b0 0x7f47040740a0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00c450 tx=0x7f46ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:48.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f46faa00640 1 -- 172.21.15.16:0/3774740190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec017070 con 0x7f47040d06b0 2024-09-10T21:18:48.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f46faa00640 1 -- 172.21.15.16:0/3774740190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46ec003600 con 0x7f47040d06b0 2024-09-10T21:18:48.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f46faa00640 1 -- 172.21.15.16:0/3774740190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec003a50 con 0x7f47040d06b0 2024-09-10T21:18:48.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f4708a00640 1 -- 172.21.15.16:0/3774740190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47040b3050 con 0x7f47040d06b0 2024-09-10T21:18:48.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f4708a00640 1 -- 172.21.15.16:0/3774740190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47041fb4f0 con 0x7f47040d06b0 2024-09-10T21:18:48.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f4708a00640 1 -- 172.21.15.16:0/3774740190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47041fb6f0 con 0x7f47040d06b0 2024-09-10T21:18:48.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.256+0000 7f46faa00640 1 -- 172.21.15.16:0/3774740190 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f46ec003bf0 con 0x7f47040d06b0 2024-09-10T21:18:48.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.260+0000 7f46faa00640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46e00725e0 0x7f46e0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.260+0000 7f46faa00640 1 -- 172.21.15.16:0/3774740190 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7f46ec0925d0 con 0x7f47040d06b0 2024-09-10T21:18:48.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.260+0000 7f4701e00640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46e00725e0 0x7f46e0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.260+0000 7f4701e00640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46e00725e0 0x7f46e0074aa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f46f40af000 tx=0x7f46f401afa0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:48.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.264+0000 7f46faa00640 1 -- 172.21.15.16:0/3774740190 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f46ec05df20 con 0x7f47040d06b0 2024-09-10T21:18:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.436+0000 7f4708a00640 1 -- 172.21.15.16:0/3774740190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f47040ce610 con 0x7f47040d06b0 2024-09-10T21:18:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.436+0000 7f46faa00640 1 -- 172.21.15.16:0/3774740190 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v64) ==== 59+0+1998 (secure 0 0 0) 0x7f46ec05d820 con 0x7f47040d06b0 2024-09-10T21:18:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:18:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:18:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:18:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:18:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4226.427518 2024-09-10T21:18:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5426.620519 2024-09-10T21:18:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4255.210203 2024-09-10T21:18:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4255.734871 2024-09-10T21:18:48.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4192.833008 2024-09-10T21:18:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4136.940759 2024-09-10T21:18:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4217.384701 2024-09-10T21:18:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5220.337972 2024-09-10T21:18:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-10T21:18:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:18:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-10T21:18:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-10T21:18:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:18:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:18:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-10T21:18:48.444 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:18:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:18:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:18:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-10T21:18:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-10T21:18:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-10T21:18:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:18:48.445 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:18:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.440+0000 7f4708a00640 1 -- 172.21.15.16:0/3774740190 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46e00725e0 msgr2=0x7f46e0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.440+0000 7f4708a00640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46e00725e0 0x7f46e0074aa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f46f40af000 tx=0x7f46f401afa0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.440+0000 7f4708a00640 1 -- 172.21.15.16:0/3774740190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47040d06b0 msgr2=0x7f47040740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.440+0000 7f4708a00640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47040d06b0 0x7f47040740a0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00c450 tx=0x7f46ec00c910 comp rx=0 tx=0).stop 2024-09-10T21:18:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.440+0000 7f4708a00640 1 -- 172.21.15.16:0/3774740190 shutdown_connections 2024-09-10T21:18:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.440+0000 7f4708a00640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f46e00725e0 0x7f46e0074aa0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.440+0000 7f4708a00640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f470415a270 0x7f47041fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.440+0000 7f4708a00640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47040d06b0 0x7f47040740a0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.440+0000 7f4708a00640 1 --2- 172.21.15.16:0/3774740190 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f47040ca170 0x7f47040b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.440+0000 7f4708a00640 1 -- 172.21.15.16:0/3774740190 >> 172.21.15.16:0/3774740190 conn(0x7f47040cfb20 msgr2=0x7f47040cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.440+0000 7f4708a00640 1 -- 172.21.15.16:0/3774740190 shutdown_connections 2024-09-10T21:18:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.440+0000 7f4708a00640 1 -- 172.21.15.16:0/3774740190 wait complete. 2024-09-10T21:18:48.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:18:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.612+0000 7fe105000640 1 -- 172.21.15.16:0/1906901844 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe10015a9a0 msgr2=0x7fe1000b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.612+0000 7fe105000640 1 --2- 172.21.15.16:0/1906901844 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe10015a9a0 0x7fe1000b3f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009870 tx=0x7fe0f001c7d0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.612+0000 7fe105000640 1 -- 172.21.15.16:0/1906901844 shutdown_connections 2024-09-10T21:18:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.612+0000 7fe105000640 1 --2- 172.21.15.16:0/1906901844 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe10015a9a0 0x7fe1000b3f80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.612+0000 7fe105000640 1 -- 172.21.15.16:0/1906901844 >> 172.21.15.16:0/1906901844 conn(0x7fe1000b16a0 msgr2=0x7fe1000b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.612+0000 7fe105000640 1 -- 172.21.15.16:0/1906901844 shutdown_connections 2024-09-10T21:18:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.612+0000 7fe105000640 1 -- 172.21.15.16:0/1906901844 wait complete. 2024-09-10T21:18:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe105000640 1 Processor -- start 2024-09-10T21:18:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe105000640 1 -- start start 2024-09-10T21:18:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe105000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1000b33b0 0x7fe1001f8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe105000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10015a9a0 0x7fe1001543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe105000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10015ad80 0x7fe100159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe105000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe100168dd0 con 0x7fe10015a9a0 2024-09-10T21:18:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe105000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe100168c50 con 0x7fe10015ad80 2024-09-10T21:18:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe105000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe100168f50 con 0x7fe1000b33b0 2024-09-10T21:18:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0f7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10015a9a0 0x7fe1001543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0fe800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10015ad80 0x7fe100159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0fde00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1000b33b0 0x7fe1001f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0f7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10015a9a0 0x7fe1001543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41468/0 (socket says 172.21.15.16:41468) 2024-09-10T21:18:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0f7e00640 1 -- 172.21.15.16:0/1447124314 learned_addr learned my addr 172.21.15.16:0/1447124314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0f7e00640 1 -- 172.21.15.16:0/1447124314 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1000b33b0 msgr2=0x7fe1001f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0f7e00640 1 --2- 172.21.15.16:0/1447124314 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1000b33b0 0x7fe1001f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0f7e00640 1 -- 172.21.15.16:0/1447124314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10015ad80 msgr2=0x7fe100159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0f7e00640 1 --2- 172.21.15.16:0/1447124314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10015ad80 0x7fe100159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0f7e00640 1 -- 172.21.15.16:0/1447124314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1000b45f0 con 0x7fe10015a9a0 2024-09-10T21:18:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0fde00640 1 --2- 172.21.15.16:0/1447124314 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1000b33b0 0x7fe1001f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0f7e00640 1 --2- 172.21.15.16:0/1447124314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10015a9a0 0x7fe1001543d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800c3e0 tx=0x7fe0e800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0f6000640 1 -- 172.21.15.16:0/1447124314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8011790 con 0x7fe10015a9a0 2024-09-10T21:18:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0f6000640 1 -- 172.21.15.16:0/1447124314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0e8017070 con 0x7fe10015a9a0 2024-09-10T21:18:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe0f6000640 1 -- 172.21.15.16:0/1447124314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8011d00 con 0x7fe10015a9a0 2024-09-10T21:18:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe105000640 1 -- 172.21.15.16:0/1447124314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe100165660 con 0x7fe10015a9a0 2024-09-10T21:18:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe105000640 1 -- 172.21.15.16:0/1447124314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10015c360 con 0x7fe10015a9a0 2024-09-10T21:18:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.616+0000 7fe105000640 1 -- 172.21.15.16:0/1447124314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10015e560 con 0x7fe10015a9a0 2024-09-10T21:18:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.620+0000 7fe0f6000640 1 -- 172.21.15.16:0/1447124314 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe0e801c460 con 0x7fe10015a9a0 2024-09-10T21:18:48.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.620+0000 7fe0f6000640 1 --2- 172.21.15.16:0/1447124314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0dc072610 0x7fe0dc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.620+0000 7fe0f6000640 1 -- 172.21.15.16:0/1447124314 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7fe0e8092b90 con 0x7fe10015a9a0 2024-09-10T21:18:48.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.620+0000 7fe0fde00640 1 --2- 172.21.15.16:0/1447124314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0dc072610 0x7fe0dc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.620+0000 7fe0fde00640 1 --2- 172.21.15.16:0/1447124314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0dc072610 0x7fe0dc074ad0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00af000 tx=0x7fe0f00b2040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:48.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.624+0000 7fe0f6000640 1 -- 172.21.15.16:0/1447124314 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe0e805df10 con 0x7fe10015a9a0 2024-09-10T21:18:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.800+0000 7fe105000640 1 -- 172.21.15.16:0/1447124314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe1000cea90 con 0x7fe10015a9a0 2024-09-10T21:18:48.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.804+0000 7fe0f6000640 1 -- 172.21.15.16:0/1447124314 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v65)=0 v65) ==== 122+0+0 (secure 0 0 0) 0x7fe0e8062940 con 0x7fe10015a9a0 2024-09-10T21:18:48.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.804+0000 7fe105000640 1 -- 172.21.15.16:0/1447124314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0dc072610 msgr2=0x7fe0dc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.804+0000 7fe105000640 1 --2- 172.21.15.16:0/1447124314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0dc072610 0x7fe0dc074ad0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00af000 tx=0x7fe0f00b2040 comp rx=0 tx=0).stop 2024-09-10T21:18:48.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.804+0000 7fe105000640 1 -- 172.21.15.16:0/1447124314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10015a9a0 msgr2=0x7fe1001543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.804+0000 7fe105000640 1 --2- 172.21.15.16:0/1447124314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10015a9a0 0x7fe1001543d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800c3e0 tx=0x7fe0e800c8a0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.804+0000 7fe105000640 1 -- 172.21.15.16:0/1447124314 shutdown_connections 2024-09-10T21:18:48.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.804+0000 7fe105000640 1 --2- 172.21.15.16:0/1447124314 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe0dc072610 0x7fe0dc074ad0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.804+0000 7fe105000640 1 --2- 172.21.15.16:0/1447124314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10015ad80 0x7fe100159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.804+0000 7fe105000640 1 --2- 172.21.15.16:0/1447124314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10015a9a0 0x7fe1001543d0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.804+0000 7fe105000640 1 --2- 172.21.15.16:0/1447124314 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe1000b33b0 0x7fe1001f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.804+0000 7fe105000640 1 -- 172.21.15.16:0/1447124314 >> 172.21.15.16:0/1447124314 conn(0x7fe1000b16a0 msgr2=0x7fe1000cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:48.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.804+0000 7fe105000640 1 -- 172.21.15.16:0/1447124314 shutdown_connections 2024-09-10T21:18:48.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:48.804+0000 7fe105000640 1 -- 172.21.15.16:0/1447124314 wait complete. 2024-09-10T21:18:48.829 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:18:48.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:18:48.918 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:18:49.010 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:18:49.010 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:18:49.010 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:49.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:18:49.019 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:18:49.019 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:49.029 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:18:49.029 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:18:49.029 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:49.071 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:18:49.071 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:18:49.072 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:18:49.081 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:18:49.081 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:18:49.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 -- 172.21.15.16:0/2372185368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe43815f350 msgr2=0x7fe4380cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 --2- 172.21.15.16:0/2372185368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe43815f350 0x7fe4380cd130 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fe4380ce4a0 tx=0x7fe42801c720 comp rx=0 tx=0).stop 2024-09-10T21:18:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 -- 172.21.15.16:0/2372185368 shutdown_connections 2024-09-10T21:18:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 --2- 172.21.15.16:0/2372185368 >> v2:172.21.15.52:3300/0 conn(0x7fe4380cfb60 0x7fe4380cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 --2- 172.21.15.16:0/2372185368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe43815f350 0x7fe4380cd130 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 -- 172.21.15.16:0/2372185368 >> 172.21.15.16:0/2372185368 conn(0x7fe4380d1d80 msgr2=0x7fe4380d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:49.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 -- 172.21.15.16:0/2372185368 shutdown_connections 2024-09-10T21:18:49.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 -- 172.21.15.16:0/2372185368 wait complete. 2024-09-10T21:18:49.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 Processor -- start 2024-09-10T21:18:49.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 -- start start 2024-09-10T21:18:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe4380cfb60 0x7fe4380c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4380d0530 0x7fe4381f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe43815f350 0x7fe4381576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe438168eb0 con 0x7fe4380d0530 2024-09-10T21:18:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe438168d30 con 0x7fe43815f350 2024-09-10T21:18:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe43e000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe438169030 con 0x7fe4380cfb60 2024-09-10T21:18:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe436a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4380d0530 0x7fe4381f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe436a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4380d0530 0x7fe4381f5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41486/0 (socket says 172.21.15.16:41486) 2024-09-10T21:18:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe436a00640 1 -- 172.21.15.16:0/2464068808 learned_addr learned my addr 172.21.15.16:0/2464068808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe437400640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe4380cfb60 0x7fe4380c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe437e00640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe43815f350 0x7fe4381576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe436a00640 1 -- 172.21.15.16:0/2464068808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe4380cfb60 msgr2=0x7fe4380c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe436a00640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe4380cfb60 0x7fe4380c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe436a00640 1 -- 172.21.15.16:0/2464068808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe43815f350 msgr2=0x7fe4381576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe436a00640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe43815f350 0x7fe4381576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.260+0000 7fe436a00640 1 -- 172.21.15.16:0/2464068808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4380c7c80 con 0x7fe4380d0530 2024-09-10T21:18:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.264+0000 7fe436a00640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4380d0530 0x7fe4381f5fe0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fe42400e9f0 tx=0x7fe42400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.264+0000 7fe42f400640 1 -- 172.21.15.16:0/2464068808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42400cd80 con 0x7fe4380d0530 2024-09-10T21:18:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.264+0000 7fe42f400640 1 -- 172.21.15.16:0/2464068808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe424010430 con 0x7fe4380d0530 2024-09-10T21:18:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.264+0000 7fe42f400640 1 -- 172.21.15.16:0/2464068808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424010860 con 0x7fe4380d0530 2024-09-10T21:18:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.264+0000 7fe43e000640 1 -- 172.21.15.16:0/2464068808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4380c7f60 con 0x7fe4380d0530 2024-09-10T21:18:49.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.264+0000 7fe43e000640 1 -- 172.21.15.16:0/2464068808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4380c8520 con 0x7fe4380d0530 2024-09-10T21:18:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.264+0000 7fe43e000640 1 -- 172.21.15.16:0/2464068808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4380cfff0 con 0x7fe4380d0530 2024-09-10T21:18:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.264+0000 7fe42f400640 1 -- 172.21.15.16:0/2464068808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe424010a00 con 0x7fe4380d0530 2024-09-10T21:18:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.264+0000 7fe42f400640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4180725e0 0x7fe418074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.264+0000 7fe42f400640 1 -- 172.21.15.16:0/2464068808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7fe424093b10 con 0x7fe4380d0530 2024-09-10T21:18:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.268+0000 7fe437400640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4180725e0 0x7fe418074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.268+0000 7fe437400640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4180725e0 0x7fe418074aa0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe4280af000 tx=0x7fe42801af60 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:49.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.268+0000 7fe42f400640 1 -- 172.21.15.16:0/2464068808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe42405ef40 con 0x7fe4380d0530 2024-09-10T21:18:49.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.476+0000 7fe43e000640 1 -- 172.21.15.16:0/2464068808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe43815eaf0 con 0x7fe4380d0530 2024-09-10T21:18:49.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.480+0000 7fe42f400640 1 -- 172.21.15.16:0/2464068808 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+3908 (secure 0 0 0) 0x7fe424063970 con 0x7fe4380d0530 2024-09-10T21:18:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:49.485 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":98,"btime":"2024-09-10T21:18:43:060691+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/3971491960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3971491960},{"type":"v1","addr":"172.21.15.52:6834","nonce":3971491960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7558,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/574639653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":574639653},{"type":"v1","addr":"172.21.15.52:6835","nonce":574639653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":7854,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/4224090659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4224090659},{"type":"v1","addr":"172.21.15.16:6836","nonce":4224090659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1749071285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1749071285},{"type":"v1","addr":"172.21.15.16:6837","nonce":1749071285}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88}],"filesystems":[]} 2024-09-10T21:18:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 98 2024-09-10T21:18:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.480+0000 7fe43e000640 1 -- 172.21.15.16:0/2464068808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4180725e0 msgr2=0x7fe418074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.480+0000 7fe43e000640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4180725e0 0x7fe418074aa0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe4280af000 tx=0x7fe42801af60 comp rx=0 tx=0).stop 2024-09-10T21:18:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.480+0000 7fe43e000640 1 -- 172.21.15.16:0/2464068808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4380d0530 msgr2=0x7fe4381f5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.480+0000 7fe43e000640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4380d0530 0x7fe4381f5fe0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fe42400e9f0 tx=0x7fe42400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.480+0000 7fe43e000640 1 -- 172.21.15.16:0/2464068808 shutdown_connections 2024-09-10T21:18:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.480+0000 7fe43e000640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe4180725e0 0x7fe418074aa0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.480+0000 7fe43e000640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe43815f350 0x7fe4381576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.480+0000 7fe43e000640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4380d0530 0x7fe4381f5fe0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.480+0000 7fe43e000640 1 --2- 172.21.15.16:0/2464068808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe4380cfb60 0x7fe4380c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.480+0000 7fe43e000640 1 -- 172.21.15.16:0/2464068808 >> 172.21.15.16:0/2464068808 conn(0x7fe4380d1d80 msgr2=0x7fe4380b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.480+0000 7fe43e000640 1 -- 172.21.15.16:0/2464068808 shutdown_connections 2024-09-10T21:18:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.480+0000 7fe43e000640 1 -- 172.21.15.16:0/2464068808 wait complete. 2024-09-10T21:18:49.505 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:18:49.505 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:18:49.505 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:18:49.506 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:18:49.506 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:18:49.506 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:18:49.506 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:18:49.506 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:18:49.506 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:18:49.506 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:18:49.507 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:18:49.507 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:18:49.507 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:18:49.507 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:18:49.507 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:18:49.507 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:18:49.518 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:18:49.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:18:49.519 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:18:49.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:18:49.523 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:18:49.523 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:18:49.524 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:18:49.524 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:18:49.562 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:18:49.562 DEBUG:teuthology.parallel:result is None 2024-09-10T21:18:49.563 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:18:49.564 DEBUG:teuthology.parallel:result is None 2024-09-10T21:18:49.570 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:18:49.570 DEBUG:teuthology.parallel:result is None 2024-09-10T21:18:49.572 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:18:49.572 DEBUG:teuthology.parallel:result is None 2024-09-10T21:18:49.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:18:49.649 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-10T21:18:49.649 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-10T21:18:49.667 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:18:49.667 INFO:tasks.ceph.mds.d.smithi052.stdout:starting mds.d at 2024-09-10T21:18:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.720+0000 7f2301c00640 1 -- 172.21.15.16:0/2492711594 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22fc070520 msgr2=0x7f22fc15e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.720+0000 7f2301c00640 1 --2- 172.21.15.16:0/2492711594 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22fc070520 0x7f22fc15e600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009bc0 tx=0x7f22f401c8a0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.720+0000 7f2301c00640 1 -- 172.21.15.16:0/2492711594 shutdown_connections 2024-09-10T21:18:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.720+0000 7f2301c00640 1 --2- 172.21.15.16:0/2492711594 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22fc070520 0x7f22fc15e600 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.720+0000 7f2301c00640 1 --2- 172.21.15.16:0/2492711594 >> v2:172.21.15.52:3300/0 conn(0x7f22fc072d40 0x7f22fc06ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.720+0000 7f2301c00640 1 --2- 172.21.15.16:0/2492711594 >> v2:172.21.15.16:3300/0 conn(0x7f22fc155000 0x7f22fc15bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.720+0000 7f2301c00640 1 -- 172.21.15.16:0/2492711594 >> 172.21.15.16:0/2492711594 conn(0x7f22fc0cfb20 msgr2=0x7f22fc0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.720+0000 7f2301c00640 1 -- 172.21.15.16:0/2492711594 shutdown_connections 2024-09-10T21:18:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.720+0000 7f2301c00640 1 -- 172.21.15.16:0/2492711594 wait complete. 2024-09-10T21:18:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f2301c00640 1 Processor -- start 2024-09-10T21:18:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f2301c00640 1 -- start start 2024-09-10T21:18:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f2301c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc070520 0x7f22fc0c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f2301c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22fc072d40 0x7f22fc1456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f2301c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22fc155000 0x7f22fc15d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f2301c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc0cac10 con 0x7f22fc070520 2024-09-10T21:18:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f2301c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc0caa90 con 0x7f22fc072d40 2024-09-10T21:18:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f2301c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f22fc0cad90 con 0x7f22fc155000 2024-09-10T21:18:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22faa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc070520 0x7f22fc0c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22faa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc070520 0x7f22fc0c2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41550/0 (socket says 172.21.15.16:41550) 2024-09-10T21:18:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22faa00640 1 -- 172.21.15.16:0/3366420354 learned_addr learned my addr 172.21.15.16:0/3366420354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22fb400640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22fc155000 0x7f22fc15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22f3e00640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22fc072d40 0x7f22fc1456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22faa00640 1 -- 172.21.15.16:0/3366420354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22fc155000 msgr2=0x7f22fc15d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22faa00640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22fc155000 0x7f22fc15d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22faa00640 1 -- 172.21.15.16:0/3366420354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22fc072d40 msgr2=0x7f22fc1456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22faa00640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22fc072d40 0x7f22fc1456d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22faa00640 1 -- 172.21.15.16:0/3366420354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22fc154580 con 0x7f22fc070520 2024-09-10T21:18:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22faa00640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc070520 0x7f22fc0c2970 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00ea20 tx=0x7f22ec00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22f2a00640 1 -- 172.21.15.16:0/3366420354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec00cd80 con 0x7f22fc070520 2024-09-10T21:18:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22f2a00640 1 -- 172.21.15.16:0/3366420354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22ec004020 con 0x7f22fc070520 2024-09-10T21:18:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f22f2a00640 1 -- 172.21.15.16:0/3366420354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec004450 con 0x7f22fc070520 2024-09-10T21:18:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f2301c00640 1 -- 172.21.15.16:0/3366420354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22fc145c10 con 0x7f22fc070520 2024-09-10T21:18:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.724+0000 7f2301c00640 1 -- 172.21.15.16:0/3366420354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22fc0c5910 con 0x7f22fc070520 2024-09-10T21:18:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.728+0000 7f2301c00640 1 -- 172.21.15.16:0/3366420354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22fc06ffe0 con 0x7f22fc070520 2024-09-10T21:18:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.728+0000 7f22f2a00640 1 -- 172.21.15.16:0/3366420354 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f22ec003600 con 0x7f22fc070520 2024-09-10T21:18:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.728+0000 7f22f2a00640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f22d80725e0 0x7f22d8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.728+0000 7f22f2a00640 1 -- 172.21.15.16:0/3366420354 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7f22ec0939d0 con 0x7f22fc070520 2024-09-10T21:18:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.728+0000 7f22f3e00640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f22d80725e0 0x7f22d8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.728+0000 7f22f3e00640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f22d80725e0 0x7f22d8074aa0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0b4930 tx=0x7f22e40091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.732+0000 7f22f2a00640 1 -- 172.21.15.16:0/3366420354 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f22ec05f310 con 0x7f22fc070520 2024-09-10T21:18:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:49.920+0000 7f2301c00640 1 -- 172.21.15.16:0/3366420354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f22fc0b3920 con 0x7f22fc070520 2024-09-10T21:18:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.088+0000 7f22f2a00640 1 -- 172.21.15.16:0/3366420354 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v132) ==== 97+0+0 (secure 0 0 0) 0x7f22ec05ec10 con 0x7f22fc070520 2024-09-10T21:18:50.096 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-10T21:18:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.092+0000 7f2301c00640 1 -- 172.21.15.16:0/3366420354 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f22d80725e0 msgr2=0x7f22d8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.092+0000 7f2301c00640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f22d80725e0 0x7f22d8074aa0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0b4930 tx=0x7f22e40091e0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.092+0000 7f2301c00640 1 -- 172.21.15.16:0/3366420354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc070520 msgr2=0x7f22fc0c2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.092+0000 7f2301c00640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc070520 0x7f22fc0c2970 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00ea20 tx=0x7f22ec00ee20 comp rx=0 tx=0).stop 2024-09-10T21:18:50.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.092+0000 7f2301c00640 1 -- 172.21.15.16:0/3366420354 shutdown_connections 2024-09-10T21:18:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.092+0000 7f2301c00640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f22d80725e0 0x7f22d8074aa0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.092+0000 7f2301c00640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f22fc155000 0x7f22fc15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.092+0000 7f2301c00640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22fc072d40 0x7f22fc1456d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.092+0000 7f2301c00640 1 --2- 172.21.15.16:0/3366420354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc070520 0x7f22fc0c2970 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.092+0000 7f2301c00640 1 -- 172.21.15.16:0/3366420354 >> 172.21.15.16:0/3366420354 conn(0x7f22fc0cfb20 msgr2=0x7f22fc0b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.092+0000 7f2301c00640 1 -- 172.21.15.16:0/3366420354 shutdown_connections 2024-09-10T21:18:50.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.092+0000 7f2301c00640 1 -- 172.21.15.16:0/3366420354 wait complete. 2024-09-10T21:18:50.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:18:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 -- 172.21.15.16:0/2948407265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f40d06b0 msgr2=0x7f15f40700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 --2- 172.21.15.16:0/2948407265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f40d06b0 0x7f15f40700a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f15e000aa10 tx=0x7f15e001dba0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 -- 172.21.15.16:0/2948407265 shutdown_connections 2024-09-10T21:18:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 --2- 172.21.15.16:0/2948407265 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15f40705e0 0x7f15f40ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 --2- 172.21.15.16:0/2948407265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f40d06b0 0x7f15f40700a0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 -- 172.21.15.16:0/2948407265 >> 172.21.15.16:0/2948407265 conn(0x7f15f40cfb20 msgr2=0x7f15f40cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 -- 172.21.15.16:0/2948407265 shutdown_connections 2024-09-10T21:18:50.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 -- 172.21.15.16:0/2948407265 wait complete. 2024-09-10T21:18:50.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 Processor -- start 2024-09-10T21:18:50.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 -- start start 2024-09-10T21:18:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15f40705e0 0x7f15f41fb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f40d06b0 0x7f15f4152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15f415a270 0x7f15f40b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15f41671f0 con 0x7f15f40d06b0 2024-09-10T21:18:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f15f4073070 con 0x7f15f40705e0 2024-09-10T21:18:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.260+0000 7f15fbc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f15f4167370 con 0x7f15f415a270 2024-09-10T21:18:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f40d06b0 0x7f15f4152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f40d06b0 0x7f15f4152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41566/0 (socket says 172.21.15.16:41566) 2024-09-10T21:18:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f3e00640 1 -- 172.21.15.16:0/355807439 learned_addr learned my addr 172.21.15.16:0/355807439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f9200640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15f40705e0 0x7f15f41fb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f9c00640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15f415a270 0x7f15f40b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f3e00640 1 -- 172.21.15.16:0/355807439 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15f415a270 msgr2=0x7f15f40b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f3e00640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15f415a270 0x7f15f40b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f3e00640 1 -- 172.21.15.16:0/355807439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15f40705e0 msgr2=0x7f15f41fb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f3e00640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15f40705e0 0x7f15f41fb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f3e00640 1 -- 172.21.15.16:0/355807439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f41fb9d0 con 0x7f15f40d06b0 2024-09-10T21:18:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f9200640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15f40705e0 0x7f15f41fb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f3e00640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f40d06b0 0x7f15f4152930 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f15e00b1040 tx=0x7f15e00a6870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f0c00640 1 -- 172.21.15.16:0/355807439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e00b3070 con 0x7f15f40d06b0 2024-09-10T21:18:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f0c00640 1 -- 172.21.15.16:0/355807439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15e00055f0 con 0x7f15f40d06b0 2024-09-10T21:18:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15fbc00640 1 -- 172.21.15.16:0/355807439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f4153550 con 0x7f15f40d06b0 2024-09-10T21:18:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f0c00640 1 -- 172.21.15.16:0/355807439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0004930 con 0x7f15f40d06b0 2024-09-10T21:18:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15fbc00640 1 -- 172.21.15.16:0/355807439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f40c9220 con 0x7f15f40d06b0 2024-09-10T21:18:50.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15fbc00640 1 -- 172.21.15.16:0/355807439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f40ca170 con 0x7f15f40d06b0 2024-09-10T21:18:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.264+0000 7f15f0c00640 1 -- 172.21.15.16:0/355807439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f15e000b040 con 0x7f15f40d06b0 2024-09-10T21:18:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.268+0000 7f15f0c00640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15d40725e0 0x7f15d4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.268+0000 7f15f0c00640 1 -- 172.21.15.16:0/355807439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4857+0+0 (secure 0 0 0) 0x7f15e012d2e0 con 0x7f15f40d06b0 2024-09-10T21:18:50.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.268+0000 7f15f9200640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15d40725e0 0x7f15d4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.268+0000 7f15f9200640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15d40725e0 0x7f15d4074aa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f15e80098b0 tx=0x7f15e8008040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.268+0000 7f15f0c00640 1 -- 172.21.15.16:0/355807439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f15f4153550 con 0x7f15f40d06b0 2024-09-10T21:18:50.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.500+0000 7f15fbc00640 1 -- 172.21.15.16:0/355807439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f15f40743a0 con 0x7f15f40d06b0 2024-09-10T21:18:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.500+0000 7f15f0c00640 1 -- 172.21.15.16:0/355807439 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) ==== 80+0+5304 (secure 0 0 0) 0x7f15e0017820 con 0x7f15f40d06b0 2024-09-10T21:18:50.507 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:50.507 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:18:50.507 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-10T21:18:50.507 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.508 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-10T21:18:50.508 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmhg+lJBAAWjuYW75BGo2dmyAUi7s9+A==", 2024-09-10T21:18:50.508 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.508 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:50.508 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:18:50.508 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:18:50.508 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.508 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.508 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.508 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-10T21:18:50.508 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmQr7+MBAAOzXhWIirBOoi/7wKnrFgmg==", 2024-09-10T21:18:50.508 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.509 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:50.509 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:18:50.509 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:18:50.509 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.509 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.509 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.509 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-10T21:18:50.509 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmxxoALBAAgkmw91/VP1Uv2WMZsJKAyA==", 2024-09-10T21:18:50.509 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.509 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:50.509 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:18:50.510 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:18:50.510 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.510 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.510 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.510 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-10T21:18:50.510 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmA4tzOBAAiS+8n+Gokz6yXapVf2ecYg==", 2024-09-10T21:18:50.510 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.510 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:50.510 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:18:50.510 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:18:50.510 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.510 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.511 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.511 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-10T21:18:50.511 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHteBmRoahLRAA32PsUkqtGkeI7tMFRwas5Q==", 2024-09-10T21:18:50.511 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.511 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:50.511 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:50.511 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.511 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.511 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.511 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-10T21:18:50.511 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCLteBmGmEMBhAAr1yQEf/NkiN4+ezAQ9VR2A==", 2024-09-10T21:18:50.512 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.512 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:50.512 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:50.512 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.512 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.512 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.512 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-10T21:18:50.512 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCOteBmmMuAHBAAt+G/KApsZymjTLRSmyRbdw==", 2024-09-10T21:18:50.512 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.512 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:50.512 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:50.512 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-10T21:18:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCRteBmoQfNMBAA0vnIqlYoI13KNAAfNmJg3w==", 2024-09-10T21:18:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.513 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-10T21:18:50.514 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCWteBmc/7lDxAADlSi4UpcFJVNdp6YTkpdSw==", 2024-09-10T21:18:50.514 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.514 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:50.514 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:50.514 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.514 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.514 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.514 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-10T21:18:50.514 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCZteBmBZ1LJhAAPcbZa1LQu4kNbSsJ/8Z81A==", 2024-09-10T21:18:50.514 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.514 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:50.514 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:50.515 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.515 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.515 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.515 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-10T21:18:50.515 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCcteBm7/czOxAAqREnY84+LF1N9dRXOhzz6g==", 2024-09-10T21:18:50.515 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.515 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:50.515 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:50.515 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.515 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.515 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.516 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-10T21:18:50.516 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCgteBmG2J8FRAAB+cMFCnf9wJg1R3RD5SUJw==", 2024-09-10T21:18:50.516 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.516 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-10T21:18:50.516 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-10T21:18:50.516 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.516 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.516 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.516 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-10T21:18:50.516 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA==", 2024-09-10T21:18:50.516 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.516 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:50.517 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-10T21:18:50.517 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:18:50.517 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.517 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.517 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-10T21:18:50.517 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmuEm4ChAAe08QvhwIcN8ZHa3AoVcYww==", 2024-09-10T21:18:50.517 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.517 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:50.517 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:18:50.517 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:18:50.517 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:18:50.517 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.518 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.518 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-10T21:18:50.518 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBm50o1BxAAIFbVxVqU4rU8eKYpuZjTHA==", 2024-09-10T21:18:50.518 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.518 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:50.518 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:18:50.518 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:18:50.518 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:18:50.518 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.518 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.518 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGteBmxtu0EBAAmN7+07lDiwjsIC/pNYUlSw==", 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmfqgYCBAAm00tBe45phGieCcp1q46cw==", 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:18:50.519 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-10T21:18:50.520 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-10T21:18:50.520 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.520 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.520 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.520 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-10T21:18:50.520 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmMVABIBAAoB72mPyXyUO6Y6k+fMqQQg==", 2024-09-10T21:18:50.520 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.520 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:18:50.520 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:18:50.520 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.520 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-10T21:18:50.521 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.521 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-10T21:18:50.521 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFteBmeyYxHBAAvBkpTzjPQKAOlHzI1jQ46g==", 2024-09-10T21:18:50.521 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-10T21:18:50.521 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-10T21:18:50.521 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-10T21:18:50.521 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-10T21:18:50.521 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.521 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.521 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-10T21:18:50.521 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:18:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.504+0000 7f15fbc00640 1 -- 172.21.15.16:0/355807439 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15d40725e0 msgr2=0x7f15d4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.504+0000 7f15fbc00640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15d40725e0 0x7f15d4074aa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f15e80098b0 tx=0x7f15e8008040 comp rx=0 tx=0).stop 2024-09-10T21:18:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.504+0000 7f15fbc00640 1 -- 172.21.15.16:0/355807439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f40d06b0 msgr2=0x7f15f4152930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.504+0000 7f15fbc00640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f40d06b0 0x7f15f4152930 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f15e00b1040 tx=0x7f15e00a6870 comp rx=0 tx=0).stop 2024-09-10T21:18:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.504+0000 7f15fbc00640 1 -- 172.21.15.16:0/355807439 shutdown_connections 2024-09-10T21:18:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.504+0000 7f15fbc00640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f15d40725e0 0x7f15d4074aa0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.504+0000 7f15fbc00640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f15f415a270 0x7f15f40b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.504+0000 7f15fbc00640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f40d06b0 0x7f15f4152930 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.504+0000 7f15fbc00640 1 --2- 172.21.15.16:0/355807439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15f40705e0 0x7f15f41fb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.504+0000 7f15fbc00640 1 -- 172.21.15.16:0/355807439 >> 172.21.15.16:0/355807439 conn(0x7f15f40cfb20 msgr2=0x7f15f40731f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.504+0000 7f15fbc00640 1 -- 172.21.15.16:0/355807439 shutdown_connections 2024-09-10T21:18:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.504+0000 7f15fbc00640 1 -- 172.21.15.16:0/355807439 wait complete. 2024-09-10T21:18:50.529 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:18:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.668+0000 7fc3c2200640 1 -- 172.21.15.16:0/60873551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3bc15f400 msgr2=0x7fc3bc070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 --2- 172.21.15.16:0/60873551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3bc15f400 0x7fc3bc070fe0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac005fd0 tx=0x7fc3ac01c700 comp rx=0 tx=0).stop 2024-09-10T21:18:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 -- 172.21.15.16:0/60873551 shutdown_connections 2024-09-10T21:18:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 --2- 172.21.15.16:0/60873551 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3bc0b13c0 0x7fc3bc071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 --2- 172.21.15.16:0/60873551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3bc15f400 0x7fc3bc070fe0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 -- 172.21.15.16:0/60873551 >> 172.21.15.16:0/60873551 conn(0x7fc3bc09b130 msgr2=0x7fc3bc0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 -- 172.21.15.16:0/60873551 shutdown_connections 2024-09-10T21:18:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 -- 172.21.15.16:0/60873551 wait complete. 2024-09-10T21:18:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 Processor -- start 2024-09-10T21:18:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 -- start start 2024-09-10T21:18:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3bc0b13c0 0x7fc3bc14eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3bc0cfc80 0x7fc3bc15aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3bc15f400 0x7fc3bc15b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc169770 con 0x7fc3bc0b13c0 2024-09-10T21:18:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc1695f0 con 0x7fc3bc0cfc80 2024-09-10T21:18:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3c2200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc3bc1698f0 con 0x7fc3bc15f400 2024-09-10T21:18:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3bb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3bc0b13c0 0x7fc3bc14eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3bb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3bc0b13c0 0x7fc3bc14eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41586/0 (socket says 172.21.15.16:41586) 2024-09-10T21:18:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3bb400640 1 -- 172.21.15.16:0/630590546 learned_addr learned my addr 172.21.15.16:0/630590546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3b3e00640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3bc0cfc80 0x7fc3bc15aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3bbe00640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3bc15f400 0x7fc3bc15b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3bb400640 1 -- 172.21.15.16:0/630590546 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3bc15f400 msgr2=0x7fc3bc15b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3bb400640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3bc15f400 0x7fc3bc15b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3bb400640 1 -- 172.21.15.16:0/630590546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3bc0cfc80 msgr2=0x7fc3bc15aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3bb400640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3bc0cfc80 0x7fc3bc15aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3bb400640 1 -- 172.21.15.16:0/630590546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3bc15c1b0 con 0x7fc3bc0b13c0 2024-09-10T21:18:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.672+0000 7fc3bb400640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3bc0b13c0 0x7fc3bc14eb60 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac004660 tx=0x7fc3ac003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.676+0000 7fc3b3400640 1 -- 172.21.15.16:0/630590546 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac004030 con 0x7fc3bc0b13c0 2024-09-10T21:18:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.676+0000 7fc3b3400640 1 -- 172.21.15.16:0/630590546 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3ac005030 con 0x7fc3bc0b13c0 2024-09-10T21:18:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.676+0000 7fc3b3400640 1 -- 172.21.15.16:0/630590546 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac005200 con 0x7fc3bc0b13c0 2024-09-10T21:18:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.676+0000 7fc3c2200640 1 -- 172.21.15.16:0/630590546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3bc159d20 con 0x7fc3bc0b13c0 2024-09-10T21:18:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.676+0000 7fc3c2200640 1 -- 172.21.15.16:0/630590546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3bc159fc0 con 0x7fc3bc0b13c0 2024-09-10T21:18:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.676+0000 7fc3c2200640 1 -- 172.21.15.16:0/630590546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3bc0d00d0 con 0x7fc3bc0b13c0 2024-09-10T21:18:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.676+0000 7fc3b3400640 1 -- 172.21.15.16:0/630590546 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc3ac0053a0 con 0x7fc3bc0b13c0 2024-09-10T21:18:50.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.676+0000 7fc3b3400640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc398072610 0x7fc398074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:50.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.680+0000 7fc3b3400640 1 -- 172.21.15.16:0/630590546 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4857+0+0 (secure 0 0 0) 0x7fc3ac12d0b0 con 0x7fc3bc0b13c0 2024-09-10T21:18:50.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.680+0000 7fc3b3e00640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc398072610 0x7fc398074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:50.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.680+0000 7fc3b3e00640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc398072610 0x7fc398074ad0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4005fd0 tx=0x7fc3a4005db0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.680+0000 7fc3b3400640 1 -- 172.21.15.16:0/630590546 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc3ac0c4020 con 0x7fc3bc0b13c0 2024-09-10T21:18:50.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.856+0000 7fc3c2200640 1 -- 172.21.15.16:0/630590546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc3bc0cdc10 con 0x7fc3bc0b13c0 2024-09-10T21:18:50.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.856+0000 7fc3b3400640 1 -- 172.21.15.16:0/630590546 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v134) ==== 84+0+65 (secure 0 0 0) 0x7fc3ac016410 con 0x7fc3bc0b13c0 2024-09-10T21:18:50.863 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:50.863 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-10T21:18:50.863 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-10T21:18:50.863 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-10T21:18:50.863 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-10T21:18:50.864 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-10T21:18:50.864 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-10T21:18:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.860+0000 7fc3c2200640 1 -- 172.21.15.16:0/630590546 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc398072610 msgr2=0x7fc398074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.860+0000 7fc3c2200640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc398072610 0x7fc398074ad0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4005fd0 tx=0x7fc3a4005db0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.860+0000 7fc3c2200640 1 -- 172.21.15.16:0/630590546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3bc0b13c0 msgr2=0x7fc3bc14eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.860+0000 7fc3c2200640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3bc0b13c0 0x7fc3bc14eb60 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac004660 tx=0x7fc3ac003df0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.860+0000 7fc3c2200640 1 -- 172.21.15.16:0/630590546 shutdown_connections 2024-09-10T21:18:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.860+0000 7fc3c2200640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc398072610 0x7fc398074ad0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.860+0000 7fc3c2200640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc3bc15f400 0x7fc3bc15b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.860+0000 7fc3c2200640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc3bc0cfc80 0x7fc3bc15aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.860+0000 7fc3c2200640 1 --2- 172.21.15.16:0/630590546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3bc0b13c0 0x7fc3bc14eb60 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.860+0000 7fc3c2200640 1 -- 172.21.15.16:0/630590546 >> 172.21.15.16:0/630590546 conn(0x7fc3bc09b130 msgr2=0x7fc3bc158b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.860+0000 7fc3c2200640 1 -- 172.21.15.16:0/630590546 shutdown_connections 2024-09-10T21:18:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:50.860+0000 7fc3c2200640 1 -- 172.21.15.16:0/630590546 wait complete. 2024-09-10T21:18:50.885 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:18:50.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:18:51.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.032+0000 7f9fc9400640 1 -- 172.21.15.16:0/2464373558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40cfa90 msgr2=0x7f9fc40c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:51.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.032+0000 7f9fc9400640 1 --2- 172.21.15.16:0/2464373558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40cfa90 0x7f9fc40c63e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009cb0 tx=0x7f9fbc01cd70 comp rx=0 tx=0).stop 2024-09-10T21:18:51.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.032+0000 7f9fc9400640 1 -- 172.21.15.16:0/2464373558 shutdown_connections 2024-09-10T21:18:51.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.032+0000 7f9fc9400640 1 --2- 172.21.15.16:0/2464373558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40cfa90 0x7f9fc40c63e0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.032+0000 7f9fc9400640 1 --2- 172.21.15.16:0/2464373558 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fc40ced80 0x7f9fc40c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.032+0000 7f9fc9400640 1 -- 172.21.15.16:0/2464373558 >> 172.21.15.16:0/2464373558 conn(0x7f9fc40d1ec0 msgr2=0x7f9fc40d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:51.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.032+0000 7f9fc9400640 1 -- 172.21.15.16:0/2464373558 shutdown_connections 2024-09-10T21:18:51.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.032+0000 7f9fc9400640 1 -- 172.21.15.16:0/2464373558 wait complete. 2024-09-10T21:18:51.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.032+0000 7f9fc9400640 1 Processor -- start 2024-09-10T21:18:51.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.032+0000 7f9fc9400640 1 -- start start 2024-09-10T21:18:51.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc9400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fc40ced80 0x7f9fc409f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc9400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc40cfa90 0x7f9fc40cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc415bd40 0x7f9fc40b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc9400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc4169280 con 0x7f9fc415bd40 2024-09-10T21:18:51.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc9400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc4169100 con 0x7f9fc40cfa90 2024-09-10T21:18:51.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc9400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc4169400 con 0x7f9fc40ced80 2024-09-10T21:18:51.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc415bd40 0x7f9fc40b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc415bd40 0x7f9fc40b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41604/0 (socket says 172.21.15.16:41604) 2024-09-10T21:18:51.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc3200640 1 -- 172.21.15.16:0/3796490117 learned_addr learned my addr 172.21.15.16:0/3796490117 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:51.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc2800640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fc40ced80 0x7f9fc409f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fbbe00640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc40cfa90 0x7f9fc40cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc3200640 1 -- 172.21.15.16:0/3796490117 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fc40ced80 msgr2=0x7f9fc409f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:51.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc3200640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fc40ced80 0x7f9fc409f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc3200640 1 -- 172.21.15.16:0/3796490117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc40cfa90 msgr2=0x7f9fc40cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:51.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc3200640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc40cfa90 0x7f9fc40cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc3200640 1 -- 172.21.15.16:0/3796490117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc40cb740 con 0x7f9fc415bd40 2024-09-10T21:18:51.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc3200640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc415bd40 0x7f9fc40b5cd0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc01a9a0 tx=0x7f9fbc0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:51.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fbaa00640 1 -- 172.21.15.16:0/3796490117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0bb070 con 0x7f9fc415bd40 2024-09-10T21:18:51.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fbaa00640 1 -- 172.21.15.16:0/3796490117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fbc004d40 con 0x7f9fc415bd40 2024-09-10T21:18:51.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fbaa00640 1 -- 172.21.15.16:0/3796490117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0b2040 con 0x7f9fc415bd40 2024-09-10T21:18:51.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc9400640 1 -- 172.21.15.16:0/3796490117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc41f96e0 con 0x7f9fc415bd40 2024-09-10T21:18:51.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc9400640 1 -- 172.21.15.16:0/3796490117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc41f9ca0 con 0x7f9fc415bd40 2024-09-10T21:18:51.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fc9400640 1 -- 172.21.15.16:0/3796490117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc40ce670 con 0x7f9fc415bd40 2024-09-10T21:18:51.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.036+0000 7f9fbaa00640 1 -- 172.21.15.16:0/3796490117 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9fbc0b5030 con 0x7f9fc415bd40 2024-09-10T21:18:51.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.040+0000 7f9fbaa00640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fa00725e0 0x7f9fa0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.040+0000 7f9fbaa00640 1 -- 172.21.15.16:0/3796490117 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4857+0+0 (secure 0 0 0) 0x7f9fbc12caa0 con 0x7f9fc415bd40 2024-09-10T21:18:51.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.040+0000 7f9fc2800640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fa00725e0 0x7f9fa0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.040+0000 7f9fc2800640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fa00725e0 0x7f9fa0074aa0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4000f70 tx=0x7f9fb4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.044+0000 7f9fbaa00640 1 -- 172.21.15.16:0/3796490117 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9fbc016990 con 0x7f9fc415bd40 2024-09-10T21:18:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:51.232+0000 7f9fc9400640 1 -- 172.21.15.16:0/3796490117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9fc40b2ee0 con 0x7f9fc415bd40 2024-09-10T21:18:52.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.096+0000 7f9fbaa00640 1 -- 172.21.15.16:0/3796490117 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v135) ==== 138+0+0 (secure 0 0 0) 0x7f9fbc0f7de0 con 0x7f9fc415bd40 2024-09-10T21:18:52.103 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-10T21:18:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.100+0000 7f9fc9400640 1 -- 172.21.15.16:0/3796490117 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fa00725e0 msgr2=0x7f9fa0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.100+0000 7f9fc9400640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fa00725e0 0x7f9fa0074aa0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4000f70 tx=0x7f9fb4009040 comp rx=0 tx=0).stop 2024-09-10T21:18:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.100+0000 7f9fc9400640 1 -- 172.21.15.16:0/3796490117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc415bd40 msgr2=0x7f9fc40b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.100+0000 7f9fc9400640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc415bd40 0x7f9fc40b5cd0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc01a9a0 tx=0x7f9fbc0092c0 comp rx=0 tx=0).stop 2024-09-10T21:18:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.100+0000 7f9fc9400640 1 -- 172.21.15.16:0/3796490117 shutdown_connections 2024-09-10T21:18:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.100+0000 7f9fc9400640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f9fa00725e0 0x7f9fa0074aa0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.100+0000 7f9fc9400640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc415bd40 0x7f9fc40b5cd0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.100+0000 7f9fc9400640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc40cfa90 0x7f9fc40cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.100+0000 7f9fc9400640 1 --2- 172.21.15.16:0/3796490117 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9fc40ced80 0x7f9fc409f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.100+0000 7f9fc9400640 1 -- 172.21.15.16:0/3796490117 >> 172.21.15.16:0/3796490117 conn(0x7f9fc40d1ec0 msgr2=0x7f9fc40b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.100+0000 7f9fc9400640 1 -- 172.21.15.16:0/3796490117 shutdown_connections 2024-09-10T21:18:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.100+0000 7f9fc9400640 1 -- 172.21.15.16:0/3796490117 wait complete. 2024-09-10T21:18:52.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:18:52.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.288+0000 7f7a0b800640 1 -- 172.21.15.16:0/3167895601 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a040d0210 msgr2=0x7f7a040ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:52.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.288+0000 7f7a0b800640 1 --2- 172.21.15.16:0/3167895601 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a040d0210 0x7f7a040ceca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009900 tx=0x7f79f001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:18:52.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.288+0000 7f7a0b800640 1 -- 172.21.15.16:0/3167895601 shutdown_connections 2024-09-10T21:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.288+0000 7f7a0b800640 1 --2- 172.21.15.16:0/3167895601 >> v2:172.21.15.52:3300/0 conn(0x7f7a040ce040 0x7f7a040b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.288+0000 7f7a0b800640 1 --2- 172.21.15.16:0/3167895601 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a040d0210 0x7f7a040ceca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.288+0000 7f7a0b800640 1 -- 172.21.15.16:0/3167895601 >> 172.21.15.16:0/3167895601 conn(0x7f7a040d1df0 msgr2=0x7f7a040d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.288+0000 7f7a0b800640 1 -- 172.21.15.16:0/3167895601 shutdown_connections 2024-09-10T21:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.288+0000 7f7a0b800640 1 -- 172.21.15.16:0/3167895601 wait complete. 2024-09-10T21:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.288+0000 7f7a0b800640 1 Processor -- start 2024-09-10T21:18:52.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.288+0000 7f7a0b800640 1 -- start start 2024-09-10T21:18:52.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a0b800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a040ce040 0x7f7a04155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:52.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a0b800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a040cfe30 0x7f7a040b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:52.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a0b800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a040d0210 0x7f7a0406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:52.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a0b800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a040b14f0 con 0x7f7a040d0210 2024-09-10T21:18:52.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a0b800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a040d2b20 con 0x7f7a040cfe30 2024-09-10T21:18:52.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a0b800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7a040cf1e0 con 0x7f7a040ce040 2024-09-10T21:18:52.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a09800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a040d0210 0x7f7a0406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:52.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a03e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a040cfe30 0x7f7a040b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:52.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a08e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a040ce040 0x7f7a04155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:52.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a09800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a040d0210 0x7f7a0406ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45358/0 (socket says 172.21.15.16:45358) 2024-09-10T21:18:52.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a09800640 1 -- 172.21.15.16:0/2018105472 learned_addr learned my addr 172.21.15.16:0/2018105472 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:52.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a03e00640 1 --2- 172.21.15.16:0/2018105472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a040cfe30 0x7f7a040b5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:39014/0 (socket says 172.21.15.16:39014) 2024-09-10T21:18:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a09800640 1 -- 172.21.15.16:0/2018105472 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a040ce040 msgr2=0x7f7a04155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a09800640 1 --2- 172.21.15.16:0/2018105472 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a040ce040 0x7f7a04155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a09800640 1 -- 172.21.15.16:0/2018105472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a040cfe30 msgr2=0x7f7a040b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a09800640 1 --2- 172.21.15.16:0/2018105472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a040cfe30 0x7f7a040b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a09800640 1 -- 172.21.15.16:0/2018105472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a041f8910 con 0x7f7a040d0210 2024-09-10T21:18:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a09800640 1 --2- 172.21.15.16:0/2018105472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a040d0210 0x7f7a0406ed40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00efd0 tx=0x7f79fc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a00c00640 1 -- 172.21.15.16:0/2018105472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc003b80 con 0x7f7a040d0210 2024-09-10T21:18:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a00c00640 1 -- 172.21.15.16:0/2018105472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79fc003d20 con 0x7f7a040d0210 2024-09-10T21:18:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a00c00640 1 -- 172.21.15.16:0/2018105472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc004280 con 0x7f7a040d0210 2024-09-10T21:18:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a0b800640 1 -- 172.21.15.16:0/2018105472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a040c8450 con 0x7f7a040d0210 2024-09-10T21:18:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a0b800640 1 -- 172.21.15.16:0/2018105472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a040c5ac0 con 0x7f7a040d0210 2024-09-10T21:18:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.292+0000 7f7a0b800640 1 -- 172.21.15.16:0/2018105472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a040c8970 con 0x7f7a040d0210 2024-09-10T21:18:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.296+0000 7f7a00c00640 1 -- 172.21.15.16:0/2018105472 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f79fc004420 con 0x7f7a040d0210 2024-09-10T21:18:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.296+0000 7f7a00c00640 1 --2- 172.21.15.16:0/2018105472 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f79e40725e0 0x7f79e4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.296+0000 7f7a00c00640 1 -- 172.21.15.16:0/2018105472 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5227+0+0 (secure 0 0 0) 0x7f79fc093dc0 con 0x7f7a040d0210 2024-09-10T21:18:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.296+0000 7f7a08e00640 1 --2- 172.21.15.16:0/2018105472 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f79e40725e0 0x7f79e4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.296+0000 7f7a08e00640 1 --2- 172.21.15.16:0/2018105472 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f79e40725e0 0x7f79e4074aa0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f79f8001820 tx=0x7f79f8009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:52.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.300+0000 7f7a00c00640 1 -- 172.21.15.16:0/2018105472 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7a040c8970 con 0x7f7a040d0210 2024-09-10T21:18:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:52.472+0000 7f7a0b800640 1 -- 172.21.15.16:0/2018105472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7a040b1710 con 0x7f7a040d0210 2024-09-10T21:18:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.104+0000 7f7a00c00640 1 -- 172.21.15.16:0/2018105472 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v136) ==== 170+0+0 (secure 0 0 0) 0x7f79fc05f0a0 con 0x7f7a040d0210 2024-09-10T21:18:53.111 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-10T21:18:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.108+0000 7f7a0b800640 1 -- 172.21.15.16:0/2018105472 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f79e40725e0 msgr2=0x7f79e4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.108+0000 7f7a0b800640 1 --2- 172.21.15.16:0/2018105472 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f79e40725e0 0x7f79e4074aa0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f79f8001820 tx=0x7f79f8009040 comp rx=0 tx=0).stop 2024-09-10T21:18:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.108+0000 7f7a0b800640 1 -- 172.21.15.16:0/2018105472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a040d0210 msgr2=0x7f7a0406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:53.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.108+0000 7f7a0b800640 1 --2- 172.21.15.16:0/2018105472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a040d0210 0x7f7a0406ed40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00efd0 tx=0x7f79fc00c6a0 comp rx=0 tx=0).stop 2024-09-10T21:18:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.112+0000 7f7a0b800640 1 -- 172.21.15.16:0/2018105472 shutdown_connections 2024-09-10T21:18:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.112+0000 7f7a0b800640 1 --2- 172.21.15.16:0/2018105472 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f79e40725e0 0x7f79e4074aa0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.112+0000 7f7a0b800640 1 --2- 172.21.15.16:0/2018105472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a040d0210 0x7f7a0406ed40 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.112+0000 7f7a0b800640 1 --2- 172.21.15.16:0/2018105472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a040cfe30 0x7f7a040b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.112+0000 7f7a0b800640 1 --2- 172.21.15.16:0/2018105472 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7a040ce040 0x7f7a04155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.112+0000 7f7a0b800640 1 -- 172.21.15.16:0/2018105472 >> 172.21.15.16:0/2018105472 conn(0x7f7a040d1df0 msgr2=0x7f7a040aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.112+0000 7f7a0b800640 1 -- 172.21.15.16:0/2018105472 shutdown_connections 2024-09-10T21:18:53.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.112+0000 7f7a0b800640 1 -- 172.21.15.16:0/2018105472 wait complete. 2024-09-10T21:18:53.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:18:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 -- 172.21.15.16:0/1766235905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a500cfe30 msgr2=0x7f8a500ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 --2- 172.21.15.16:0/1766235905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a500cfe30 0x7f8a500ce590 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009870 tx=0x7f8a4001f010 comp rx=0 tx=0).stop 2024-09-10T21:18:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 -- 172.21.15.16:0/1766235905 shutdown_connections 2024-09-10T21:18:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 --2- 172.21.15.16:0/1766235905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a500cfe30 0x7f8a500ce590 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 -- 172.21.15.16:0/1766235905 >> 172.21.15.16:0/1766235905 conn(0x7f8a500d1df0 msgr2=0x7f8a500d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 -- 172.21.15.16:0/1766235905 shutdown_connections 2024-09-10T21:18:53.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 -- 172.21.15.16:0/1766235905 wait complete. 2024-09-10T21:18:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 Processor -- start 2024-09-10T21:18:53.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 -- start start 2024-09-10T21:18:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a500ce040 0x7f8a500c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a500cfe30 0x7f8a50151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a500d0210 0x7f8a500b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a500d2ad0 con 0x7f8a500d0210 2024-09-10T21:18:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a500b14f0 con 0x7f8a500cfe30 2024-09-10T21:18:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a55600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8a500cf1e0 con 0x7f8a500ce040 2024-09-10T21:18:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a4f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a500d0210 0x7f8a500b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a4f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a500d0210 0x7f8a500b6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45366/0 (socket says 172.21.15.16:45366) 2024-09-10T21:18:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a4f400640 1 -- 172.21.15.16:0/3918136355 learned_addr learned my addr 172.21.15.16:0/3918136355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a47e00640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a500cfe30 0x7f8a50151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a4ea00640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a500ce040 0x7f8a500c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a4f400640 1 -- 172.21.15.16:0/3918136355 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a500ce040 msgr2=0x7f8a500c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a4f400640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a500ce040 0x7f8a500c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a4f400640 1 -- 172.21.15.16:0/3918136355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a500cfe30 msgr2=0x7f8a50151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a4f400640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a500cfe30 0x7f8a50151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a4f400640 1 -- 172.21.15.16:0/3918136355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a500c8ec0 con 0x7f8a500d0210 2024-09-10T21:18:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.304+0000 7f8a4ea00640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a500ce040 0x7f8a500c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.308+0000 7f8a4f400640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a500d0210 0x7f8a500b6b30 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800c7f0 tx=0x7f8a4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.308+0000 7f8a46a00640 1 -- 172.21.15.16:0/3918136355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48003b80 con 0x7f8a500d0210 2024-09-10T21:18:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.308+0000 7f8a46a00640 1 -- 172.21.15.16:0/3918136355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a48003d20 con 0x7f8a500d0210 2024-09-10T21:18:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.308+0000 7f8a46a00640 1 -- 172.21.15.16:0/3918136355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800aab0 con 0x7f8a500d0210 2024-09-10T21:18:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.308+0000 7f8a55600640 1 -- 172.21.15.16:0/3918136355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a500c9160 con 0x7f8a500d0210 2024-09-10T21:18:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.308+0000 7f8a55600640 1 -- 172.21.15.16:0/3918136355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a500cbe60 con 0x7f8a500d0210 2024-09-10T21:18:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.308+0000 7f8a55600640 1 -- 172.21.15.16:0/3918136355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a500b1710 con 0x7f8a500d0210 2024-09-10T21:18:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.308+0000 7f8a46a00640 1 -- 172.21.15.16:0/3918136355 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8a4800ac50 con 0x7f8a500d0210 2024-09-10T21:18:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.312+0000 7f8a46a00640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8a2c0725e0 0x7f8a2c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.312+0000 7f8a46a00640 1 -- 172.21.15.16:0/3918136355 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5609+0+0 (secure 0 0 0) 0x7f8a48093a70 con 0x7f8a500d0210 2024-09-10T21:18:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.312+0000 7f8a4ea00640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8a2c0725e0 0x7f8a2c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.312+0000 7f8a4ea00640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8a2c0725e0 0x7f8a2c074aa0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8a400b2000 tx=0x7f8a4001d850 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.316+0000 7f8a46a00640 1 -- 172.21.15.16:0/3918136355 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8a4805f220 con 0x7f8a500d0210 2024-09-10T21:18:53.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:53.520+0000 7f8a55600640 1 -- 172.21.15.16:0/3918136355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8a500c4af0 con 0x7f8a500d0210 2024-09-10T21:18:54.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.108+0000 7f8a46a00640 1 -- 172.21.15.16:0/3918136355 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v101) ==== 174+0+0 (secure 0 0 0) 0x7f8a4805eb20 con 0x7f8a500d0210 2024-09-10T21:18:54.113 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 22 and data pool 23 2024-09-10T21:18:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.112+0000 7f8a55600640 1 -- 172.21.15.16:0/3918136355 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8a2c0725e0 msgr2=0x7f8a2c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.112+0000 7f8a55600640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8a2c0725e0 0x7f8a2c074aa0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8a400b2000 tx=0x7f8a4001d850 comp rx=0 tx=0).stop 2024-09-10T21:18:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.112+0000 7f8a55600640 1 -- 172.21.15.16:0/3918136355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a500d0210 msgr2=0x7f8a500b6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.112+0000 7f8a55600640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a500d0210 0x7f8a500b6b30 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800c7f0 tx=0x7f8a4800ccb0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.112+0000 7f8a55600640 1 -- 172.21.15.16:0/3918136355 shutdown_connections 2024-09-10T21:18:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.112+0000 7f8a55600640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8a2c0725e0 0x7f8a2c074aa0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.112+0000 7f8a55600640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a500d0210 0x7f8a500b6b30 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.112+0000 7f8a55600640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a500cfe30 0x7f8a50151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.112+0000 7f8a55600640 1 --2- 172.21.15.16:0/3918136355 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8a500ce040 0x7f8a500c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.112+0000 7f8a55600640 1 -- 172.21.15.16:0/3918136355 >> 172.21.15.16:0/3918136355 conn(0x7f8a500d1df0 msgr2=0x7f8a5015f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.112+0000 7f8a55600640 1 -- 172.21.15.16:0/3918136355 shutdown_connections 2024-09-10T21:18:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.112+0000 7f8a55600640 1 -- 172.21.15.16:0/3918136355 wait complete. 2024-09-10T21:18:54.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:18:54.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:18:54.237 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.304+0000 7f9925a00640 1 -- 172.21.15.16:0/1130132956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9920161620 msgr2=0x7f9920070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.304+0000 7f9925a00640 1 --2- 172.21.15.16:0/1130132956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9920161620 0x7f9920070cf0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f99100098f0 tx=0x7f991001c8b0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.304+0000 7f9925a00640 1 -- 172.21.15.16:0/1130132956 shutdown_connections 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.304+0000 7f9925a00640 1 --2- 172.21.15.16:0/1130132956 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f9920071230 0x7f99200d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.304+0000 7f9925a00640 1 --2- 172.21.15.16:0/1130132956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9920161620 0x7f9920070cf0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.304+0000 7f9925a00640 1 -- 172.21.15.16:0/1130132956 >> 172.21.15.16:0/1130132956 conn(0x7f992009b130 msgr2=0x7f99200d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.304+0000 7f9925a00640 1 -- 172.21.15.16:0/1130132956 shutdown_connections 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.304+0000 7f9925a00640 1 -- 172.21.15.16:0/1130132956 wait complete. 2024-09-10T21:18:54.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9925a00640 1 Processor -- start 2024-09-10T21:18:54.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9925a00640 1 -- start start 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9925a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9920071230 0x7f992014ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9925a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f99200b0b40 0x7f992015bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9925a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9920161620 0x7f992015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9925a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f992016b310 con 0x7f9920161620 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9925a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f992016b190 con 0x7f9920071230 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9925a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f992016b490 con 0x7f99200b0b40 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f991f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9920161620 0x7f992015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f991f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9920161620 0x7f992015c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45376/0 (socket says 172.21.15.16:45376) 2024-09-10T21:18:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f991f400640 1 -- 172.21.15.16:0/1638810918 learned_addr learned my addr 172.21.15.16:0/1638810918 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f991ea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9920071230 0x7f992014ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9917e00640 1 --2- 172.21.15.16:0/1638810918 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f99200b0b40 0x7f992015bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f991f400640 1 -- 172.21.15.16:0/1638810918 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f99200b0b40 msgr2=0x7f992015bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f991f400640 1 --2- 172.21.15.16:0/1638810918 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f99200b0b40 0x7f992015bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f991f400640 1 -- 172.21.15.16:0/1638810918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9920071230 msgr2=0x7f992014ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f991f400640 1 --2- 172.21.15.16:0/1638810918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9920071230 0x7f992014ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f991f400640 1 -- 172.21.15.16:0/1638810918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992014bf50 con 0x7f9920161620 2024-09-10T21:18:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f991f400640 1 --2- 172.21.15.16:0/1638810918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9920161620 0x7f992015c1e0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f991800c7b0 tx=0x7f991800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9916a00640 1 -- 172.21.15.16:0/1638810918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918017070 con 0x7f9920161620 2024-09-10T21:18:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9916a00640 1 -- 172.21.15.16:0/1638810918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9918005070 con 0x7f9920161620 2024-09-10T21:18:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9916a00640 1 -- 172.21.15.16:0/1638810918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991800ce60 con 0x7f9920161620 2024-09-10T21:18:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9925a00640 1 -- 172.21.15.16:0/1638810918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99201fb2d0 con 0x7f9920161620 2024-09-10T21:18:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9925a00640 1 -- 172.21.15.16:0/1638810918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99201fb890 con 0x7f9920161620 2024-09-10T21:18:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.308+0000 7f9925a00640 1 -- 172.21.15.16:0/1638810918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99200d0590 con 0x7f9920161620 2024-09-10T21:18:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.312+0000 7f9916a00640 1 -- 172.21.15.16:0/1638810918 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f99180054c0 con 0x7f9920161620 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.312+0000 7f9916a00640 1 --2- 172.21.15.16:0/1638810918 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f98fc072610 0x7f98fc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.312+0000 7f9916a00640 1 -- 172.21.15.16:0/1638810918 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5709+0+0 (secure 0 0 0) 0x7f9918092af0 con 0x7f9920161620 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.312+0000 7f991ea00640 1 --2- 172.21.15.16:0/1638810918 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f98fc072610 0x7f98fc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.312+0000 7f991ea00640 1 --2- 172.21.15.16:0/1638810918 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f98fc072610 0x7f98fc074ad0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f99100af000 tx=0x7f991001b0f0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:54.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.316+0000 7f9916a00640 1 -- 172.21.15.16:0/1638810918 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f991805dae0 con 0x7f9920161620 2024-09-10T21:18:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.492+0000 7f9925a00640 1 -- 172.21.15.16:0/1638810918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9920158fa0 con 0x7f9920161620 2024-09-10T21:18:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.492+0000 7f9916a00640 1 -- 172.21.15.16:0/1638810918 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16704 (secure 0 0 0) 0x7f9918062510 con 0x7f9920161620 2024-09-10T21:18:54.499 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:54.499 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":137,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:18:53.527318+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:18:51.247076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T21:18:52.488695+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:18:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.496+0000 7f9925a00640 1 -- 172.21.15.16:0/1638810918 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f98fc072610 msgr2=0x7f98fc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.496+0000 7f9925a00640 1 --2- 172.21.15.16:0/1638810918 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f98fc072610 0x7f98fc074ad0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f99100af000 tx=0x7f991001b0f0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.496+0000 7f9925a00640 1 -- 172.21.15.16:0/1638810918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9920161620 msgr2=0x7f992015c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.496+0000 7f9925a00640 1 --2- 172.21.15.16:0/1638810918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9920161620 0x7f992015c1e0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f991800c7b0 tx=0x7f991800cc70 comp rx=0 tx=0).stop 2024-09-10T21:18:54.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.496+0000 7f9925a00640 1 -- 172.21.15.16:0/1638810918 shutdown_connections 2024-09-10T21:18:54.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.496+0000 7f9925a00640 1 --2- 172.21.15.16:0/1638810918 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f98fc072610 0x7f98fc074ad0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.496+0000 7f9925a00640 1 --2- 172.21.15.16:0/1638810918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9920161620 0x7f992015c1e0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.496+0000 7f9925a00640 1 --2- 172.21.15.16:0/1638810918 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f99200b0b40 0x7f992015bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.496+0000 7f9925a00640 1 --2- 172.21.15.16:0/1638810918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9920071230 0x7f992014ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.496+0000 7f9925a00640 1 -- 172.21.15.16:0/1638810918 >> 172.21.15.16:0/1638810918 conn(0x7f992009b130 msgr2=0x7f992014eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:54.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.496+0000 7f9925a00640 1 -- 172.21.15.16:0/1638810918 shutdown_connections 2024-09-10T21:18:54.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.496+0000 7f9925a00640 1 -- 172.21.15.16:0/1638810918 wait complete. 2024-09-10T21:18:54.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:18:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.704+0000 7fb6b1e00640 1 -- 172.21.15.16:0/1594477948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0716f0 msgr2=0x7fb6ac0af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.704+0000 7fb6b1e00640 1 --2- 172.21.15.16:0/1594477948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0716f0 0x7fb6ac0af580 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0cc360 tx=0x7fb69801c7a0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.704+0000 7fb6b1e00640 1 -- 172.21.15.16:0/1594477948 shutdown_connections 2024-09-10T21:18:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.704+0000 7fb6b1e00640 1 --2- 172.21.15.16:0/1594477948 >> v2:172.21.15.52:3300/0 conn(0x7fb6ac0b13c0 0x7fb6ac0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.704+0000 7fb6b1e00640 1 --2- 172.21.15.16:0/1594477948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0716f0 0x7fb6ac0af580 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.704+0000 7fb6b1e00640 1 -- 172.21.15.16:0/1594477948 >> 172.21.15.16:0/1594477948 conn(0x7fb6ac09b130 msgr2=0x7fb6ac0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.704+0000 7fb6b1e00640 1 -- 172.21.15.16:0/1594477948 shutdown_connections 2024-09-10T21:18:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.704+0000 7fb6b1e00640 1 -- 172.21.15.16:0/1594477948 wait complete. 2024-09-10T21:18:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.704+0000 7fb6b1e00640 1 Processor -- start 2024-09-10T21:18:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.704+0000 7fb6b1e00640 1 -- start start 2024-09-10T21:18:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6b1e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ac0716f0 0x7fb6ac155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6b1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0b13c0 0x7fb6ac158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6b1e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6ac15f400 0x7fb6ac0b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6b1e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac168f60 con 0x7fb6ac0b13c0 2024-09-10T21:18:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6b1e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac168de0 con 0x7fb6ac0716f0 2024-09-10T21:18:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6b1e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb6ac1690e0 con 0x7fb6ac15f400 2024-09-10T21:18:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6aa000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0b13c0 0x7fb6ac158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6aa000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0b13c0 0x7fb6ac158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45382/0 (socket says 172.21.15.16:45382) 2024-09-10T21:18:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6aa000640 1 -- 172.21.15.16:0/734822062 learned_addr learned my addr 172.21.15.16:0/734822062 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6aaa00640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ac0716f0 0x7fb6ac155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6ab400640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6ac15f400 0x7fb6ac0b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6aa000640 1 -- 172.21.15.16:0/734822062 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6ac15f400 msgr2=0x7fb6ac0b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6aa000640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6ac15f400 0x7fb6ac0b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6aa000640 1 -- 172.21.15.16:0/734822062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ac0716f0 msgr2=0x7fb6ac155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6aa000640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ac0716f0 0x7fb6ac155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6aa000640 1 -- 172.21.15.16:0/734822062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ac0b6ae0 con 0x7fb6ac0b13c0 2024-09-10T21:18:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6aaa00640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ac0716f0 0x7fb6ac155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6aa000640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0b13c0 0x7fb6ac158810 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb6980b0040 tx=0x7fb6980a55f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6a3400640 1 -- 172.21.15.16:0/734822062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980b2040 con 0x7fb6ac0b13c0 2024-09-10T21:18:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6a3400640 1 -- 172.21.15.16:0/734822062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb698004490 con 0x7fb6ac0b13c0 2024-09-10T21:18:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6a3400640 1 -- 172.21.15.16:0/734822062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb698005280 con 0x7fb6ac0b13c0 2024-09-10T21:18:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6b1e00640 1 -- 172.21.15.16:0/734822062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ac0b6dc0 con 0x7fb6ac0b13c0 2024-09-10T21:18:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6b1e00640 1 -- 172.21.15.16:0/734822062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ac1582f0 con 0x7fb6ac0b13c0 2024-09-10T21:18:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.708+0000 7fb6b1e00640 1 -- 172.21.15.16:0/734822062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ac070fe0 con 0x7fb6ac0b13c0 2024-09-10T21:18:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.712+0000 7fb6a3400640 1 -- 172.21.15.16:0/734822062 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb698005420 con 0x7fb6ac0b13c0 2024-09-10T21:18:54.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.712+0000 7fb6a3400640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb68c0725e0 0x7fb68c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.712+0000 7fb6a3400640 1 -- 172.21.15.16:0/734822062 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5709+0+0 (secure 0 0 0) 0x7fb69812ccd0 con 0x7fb6ac0b13c0 2024-09-10T21:18:54.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.712+0000 7fb6aaa00640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb68c0725e0 0x7fb68c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.712+0000 7fb6aaa00640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb68c0725e0 0x7fb68c074aa0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb69c001700 tx=0x7fb69c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.716+0000 7fb6a3400640 1 -- 172.21.15.16:0/734822062 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb6980164b0 con 0x7fb6ac0b13c0 2024-09-10T21:18:54.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.892+0000 7fb6b1e00640 1 -- 172.21.15.16:0/734822062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6ac156340 con 0x7fb6ac0b13c0 2024-09-10T21:18:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.892+0000 7fb6a3400640 1 -- 172.21.15.16:0/734822062 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16704 (secure 0 0 0) 0x7fb6980f8320 con 0x7fb6ac0b13c0 2024-09-10T21:18:54.898 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:54.898 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":137,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:18:53.527318+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:18:51.247076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T21:18:52.488695+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:18:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.896+0000 7fb6b1e00640 1 -- 172.21.15.16:0/734822062 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb68c0725e0 msgr2=0x7fb68c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.896+0000 7fb6b1e00640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb68c0725e0 0x7fb68c074aa0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb69c001700 tx=0x7fb69c009040 comp rx=0 tx=0).stop 2024-09-10T21:18:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.896+0000 7fb6b1e00640 1 -- 172.21.15.16:0/734822062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0b13c0 msgr2=0x7fb6ac158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.896+0000 7fb6b1e00640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0b13c0 0x7fb6ac158810 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb6980b0040 tx=0x7fb6980a55f0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.896+0000 7fb6b1e00640 1 -- 172.21.15.16:0/734822062 shutdown_connections 2024-09-10T21:18:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.896+0000 7fb6b1e00640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb68c0725e0 0x7fb68c074aa0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.896+0000 7fb6b1e00640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb6ac15f400 0x7fb6ac0b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.896+0000 7fb6b1e00640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0b13c0 0x7fb6ac158810 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.896+0000 7fb6b1e00640 1 --2- 172.21.15.16:0/734822062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ac0716f0 0x7fb6ac155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.896+0000 7fb6b1e00640 1 -- 172.21.15.16:0/734822062 >> 172.21.15.16:0/734822062 conn(0x7fb6ac09b130 msgr2=0x7fb6ac0cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:54.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.896+0000 7fb6b1e00640 1 -- 172.21.15.16:0/734822062 shutdown_connections 2024-09-10T21:18:54.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:54.896+0000 7fb6b1e00640 1 -- 172.21.15.16:0/734822062 wait complete. 2024-09-10T21:18:54.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:18:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.064+0000 7f98cac00640 1 -- 172.21.15.16:0/600226840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40ce040 msgr2=0x7f98c40b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.064+0000 7f98cac00640 1 --2- 172.21.15.16:0/600226840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40ce040 0x7f98c40b1710 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009900 tx=0x7f98bc01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:18:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.064+0000 7f98cac00640 1 -- 172.21.15.16:0/600226840 shutdown_connections 2024-09-10T21:18:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.064+0000 7f98cac00640 1 --2- 172.21.15.16:0/600226840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40ce040 0x7f98c40b1710 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98cac00640 1 --2- 172.21.15.16:0/600226840 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f98c40d0210 0x7f98c40ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98cac00640 1 -- 172.21.15.16:0/600226840 >> 172.21.15.16:0/600226840 conn(0x7f98c40d1df0 msgr2=0x7f98c40d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98cac00640 1 -- 172.21.15.16:0/600226840 shutdown_connections 2024-09-10T21:18:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98cac00640 1 -- 172.21.15.16:0/600226840 wait complete. 2024-09-10T21:18:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98cac00640 1 Processor -- start 2024-09-10T21:18:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98cac00640 1 -- start start 2024-09-10T21:18:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98cac00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98c40ce040 0x7f98c4150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98cac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40cfe30 0x7f98c40b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98cac00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f98c40d0210 0x7f98c41f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98cac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98c40d2aa0 con 0x7f98c40cfe30 2024-09-10T21:18:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98cac00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f98c40b14f0 con 0x7f98c40ce040 2024-09-10T21:18:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98cac00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f98c40cf1e0 con 0x7f98c40d0210 2024-09-10T21:18:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98bb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40cfe30 0x7f98c40b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98bb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40cfe30 0x7f98c40b7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45414/0 (socket says 172.21.15.16:45414) 2024-09-10T21:18:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98bb400640 1 -- 172.21.15.16:0/11676885 learned_addr learned my addr 172.21.15.16:0/11676885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98c3e00640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98c40ce040 0x7f98c4150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98c8c00640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f98c40d0210 0x7f98c41f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98bb400640 1 -- 172.21.15.16:0/11676885 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f98c40d0210 msgr2=0x7f98c41f7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98bb400640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f98c40d0210 0x7f98c41f7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98bb400640 1 -- 172.21.15.16:0/11676885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98c40ce040 msgr2=0x7f98c4150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98bb400640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98c40ce040 0x7f98c4150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98bb400640 1 -- 172.21.15.16:0/11676885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c40b5cf0 con 0x7f98c40cfe30 2024-09-10T21:18:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98c3e00640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98c40ce040 0x7f98c4150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.068+0000 7f98bb400640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40cfe30 0x7f98c40b7430 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00d950 tx=0x7f98ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.072+0000 7f98c0c00640 1 -- 172.21.15.16:0/11676885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac017070 con 0x7f98c40cfe30 2024-09-10T21:18:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.072+0000 7f98c0c00640 1 -- 172.21.15.16:0/11676885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98ac009d20 con 0x7f98c40cfe30 2024-09-10T21:18:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.072+0000 7f98c0c00640 1 -- 172.21.15.16:0/11676885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0106c0 con 0x7f98c40cfe30 2024-09-10T21:18:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.072+0000 7f98cac00640 1 -- 172.21.15.16:0/11676885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c40b6550 con 0x7f98c40cfe30 2024-09-10T21:18:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.072+0000 7f98cac00640 1 -- 172.21.15.16:0/11676885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c40b6270 con 0x7f98c40cfe30 2024-09-10T21:18:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.072+0000 7f98cac00640 1 -- 172.21.15.16:0/11676885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c40b1710 con 0x7f98c40cfe30 2024-09-10T21:18:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.072+0000 7f98c0c00640 1 -- 172.21.15.16:0/11676885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f98ac010860 con 0x7f98c40cfe30 2024-09-10T21:18:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.072+0000 7f98c0c00640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f98a00725e0 0x7f98a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.072+0000 7f98c0c00640 1 -- 172.21.15.16:0/11676885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5709+0+0 (secure 0 0 0) 0x7f98ac093750 con 0x7f98c40cfe30 2024-09-10T21:18:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.072+0000 7f98c3e00640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f98a00725e0 0x7f98a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:55.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.076+0000 7f98c3e00640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f98a00725e0 0x7f98a0074aa0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f98b4007fd0 tx=0x7f98b4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.076+0000 7f98c0c00640 1 -- 172.21.15.16:0/11676885 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f98ac05ee20 con 0x7f98c40cfe30 2024-09-10T21:18:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:55.288+0000 7f98cac00640 1 -- 172.21.15.16:0/11676885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f98c41540d0 con 0x7f98c40cfe30 2024-09-10T21:18:56.104 INFO:tasks.ceph.mds.b.smithi052.stderr:2024-09-10T21:18:56.099+0000 7fc47e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:18:56.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.140+0000 7f98c0c00640 1 -- 172.21.15.16:0/11676885 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v104) ==== 118+0+0 (secure 0 0 0) 0x7f98ac05e720 con 0x7f98c40cfe30 2024-09-10T21:18:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.144+0000 7f98cac00640 1 -- 172.21.15.16:0/11676885 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f98a00725e0 msgr2=0x7f98a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.144+0000 7f98cac00640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f98a00725e0 0x7f98a0074aa0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f98b4007fd0 tx=0x7f98b4009040 comp rx=0 tx=0).stop 2024-09-10T21:18:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.144+0000 7f98cac00640 1 -- 172.21.15.16:0/11676885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40cfe30 msgr2=0x7f98c40b7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.144+0000 7f98cac00640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40cfe30 0x7f98c40b7430 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00d950 tx=0x7f98ac00de10 comp rx=0 tx=0).stop 2024-09-10T21:18:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.144+0000 7f98cac00640 1 -- 172.21.15.16:0/11676885 shutdown_connections 2024-09-10T21:18:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.144+0000 7f98cac00640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f98a00725e0 0x7f98a0074aa0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.144+0000 7f98cac00640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f98c40d0210 0x7f98c41f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.144+0000 7f98cac00640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c40cfe30 0x7f98c40b7430 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.144+0000 7f98cac00640 1 --2- 172.21.15.16:0/11676885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98c40ce040 0x7f98c4150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.144+0000 7f98cac00640 1 -- 172.21.15.16:0/11676885 >> 172.21.15.16:0/11676885 conn(0x7f98c40d1df0 msgr2=0x7f98c40af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.144+0000 7f98cac00640 1 -- 172.21.15.16:0/11676885 shutdown_connections 2024-09-10T21:18:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.144+0000 7f98cac00640 1 -- 172.21.15.16:0/11676885 wait complete. 2024-09-10T21:18:56.173 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:18:56.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:18:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 -- 172.21.15.16:0/1713749507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7840cca00 msgr2=0x7fa7840ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 --2- 172.21.15.16:0/1713749507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7840cca00 0x7fa7840ce340 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fa784072600 tx=0x7fa77001c810 comp rx=0 tx=0).stop 2024-09-10T21:18:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 -- 172.21.15.16:0/1713749507 shutdown_connections 2024-09-10T21:18:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 --2- 172.21.15.16:0/1713749507 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7840b04a0 0x7fa7840b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 --2- 172.21.15.16:0/1713749507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7840cca00 0x7fa7840ce340 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 --2- 172.21.15.16:0/1713749507 >> v2:172.21.15.52:3300/0 conn(0x7fa7840cdf60 0x7fa7840cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 -- 172.21.15.16:0/1713749507 >> 172.21.15.16:0/1713749507 conn(0x7fa7840d0390 msgr2=0x7fa7840d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 -- 172.21.15.16:0/1713749507 shutdown_connections 2024-09-10T21:18:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 -- 172.21.15.16:0/1713749507 wait complete. 2024-09-10T21:18:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 Processor -- start 2024-09-10T21:18:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 -- start start 2024-09-10T21:18:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7840b04a0 0x7fa784156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7840cca00 0x7fa7840b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7840cdf60 0x7fa784156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7840cc630 con 0x7fa7840cca00 2024-09-10T21:18:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa7840cc4b0 con 0x7fa7840b04a0 2024-09-10T21:18:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.324+0000 7fa78c200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa7840ce880 con 0x7fa7840cdf60 2024-09-10T21:18:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa788e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7840cca00 0x7fa7840b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa788e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7840cca00 0x7fa7840b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45442/0 (socket says 172.21.15.16:45442) 2024-09-10T21:18:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa788e00640 1 -- 172.21.15.16:0/1007918011 learned_addr learned my addr 172.21.15.16:0/1007918011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa789800640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7840b04a0 0x7fa784156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa78a200640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7840cdf60 0x7fa784156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa788e00640 1 -- 172.21.15.16:0/1007918011 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7840cdf60 msgr2=0x7fa784156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa788e00640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7840cdf60 0x7fa784156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa788e00640 1 -- 172.21.15.16:0/1007918011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7840b04a0 msgr2=0x7fa784156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa788e00640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7840b04a0 0x7fa784156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa788e00640 1 -- 172.21.15.16:0/1007918011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7840b4340 con 0x7fa7840cca00 2024-09-10T21:18:56.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa789800640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7840b04a0 0x7fa784156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:56.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa788e00640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7840cca00 0x7fa7840b50c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fa77001ccf0 tx=0x7fa7700a6520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:56.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa781600640 1 -- 172.21.15.16:0/1007918011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7700ac070 con 0x7fa7840cca00 2024-09-10T21:18:56.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa78c200640 1 -- 172.21.15.16:0/1007918011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7841f61e0 con 0x7fa7840cca00 2024-09-10T21:18:56.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa781600640 1 -- 172.21.15.16:0/1007918011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa770004340 con 0x7fa7840cca00 2024-09-10T21:18:56.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa781600640 1 -- 172.21.15.16:0/1007918011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770005320 con 0x7fa7840cca00 2024-09-10T21:18:56.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa78c200640 1 -- 172.21.15.16:0/1007918011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7840b48c0 con 0x7fa7840cca00 2024-09-10T21:18:56.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa78c200640 1 -- 172.21.15.16:0/1007918011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7840ce340 con 0x7fa7840cca00 2024-09-10T21:18:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.328+0000 7fa781600640 1 -- 172.21.15.16:0/1007918011 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa7700054c0 con 0x7fa7840cca00 2024-09-10T21:18:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.332+0000 7fa781600640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa7640725e0 0x7fa764074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.332+0000 7fa781600640 1 -- 172.21.15.16:0/1007918011 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fa77012dc00 con 0x7fa7840cca00 2024-09-10T21:18:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.332+0000 7fa789800640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa7640725e0 0x7fa764074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.332+0000 7fa789800640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa7640725e0 0x7fa764074aa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa778005fd0 tx=0x7fa778005f20 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.336+0000 7fa781600640 1 -- 172.21.15.16:0/1007918011 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa770016490 con 0x7fa7840cca00 2024-09-10T21:18:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:56.544+0000 7fa78c200640 1 -- 172.21.15.16:0/1007918011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa78406ec50 con 0x7fa7840cca00 2024-09-10T21:18:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.148+0000 7fa781600640 1 -- 172.21.15.16:0/1007918011 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v105) ==== 122+0+0 (secure 0 0 0) 0x7fa7700f92d0 con 0x7fa7840cca00 2024-09-10T21:18:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.152+0000 7fa78c200640 1 -- 172.21.15.16:0/1007918011 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa7640725e0 msgr2=0x7fa764074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.152+0000 7fa78c200640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa7640725e0 0x7fa764074aa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa778005fd0 tx=0x7fa778005f20 comp rx=0 tx=0).stop 2024-09-10T21:18:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.152+0000 7fa78c200640 1 -- 172.21.15.16:0/1007918011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7840cca00 msgr2=0x7fa7840b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.152+0000 7fa78c200640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7840cca00 0x7fa7840b50c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fa77001ccf0 tx=0x7fa7700a6520 comp rx=0 tx=0).stop 2024-09-10T21:18:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.152+0000 7fa78c200640 1 -- 172.21.15.16:0/1007918011 shutdown_connections 2024-09-10T21:18:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.152+0000 7fa78c200640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa7640725e0 0x7fa764074aa0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.152+0000 7fa78c200640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa7840cdf60 0x7fa784156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.152+0000 7fa78c200640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7840cca00 0x7fa7840b50c0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.152+0000 7fa78c200640 1 --2- 172.21.15.16:0/1007918011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7840b04a0 0x7fa784156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.152+0000 7fa78c200640 1 -- 172.21.15.16:0/1007918011 >> 172.21.15.16:0/1007918011 conn(0x7fa7840d0390 msgr2=0x7fa784070750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.152+0000 7fa78c200640 1 -- 172.21.15.16:0/1007918011 shutdown_connections 2024-09-10T21:18:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.152+0000 7fa78c200640 1 -- 172.21.15.16:0/1007918011 wait complete. 2024-09-10T21:18:57.180 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:18:57.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:18:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.324+0000 7fd616200640 1 -- 172.21.15.16:0/4048434894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6100d0890 msgr2=0x7fd6100b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:57.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.324+0000 7fd616200640 1 --2- 172.21.15.16:0/4048434894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6100d0890 0x7fd6100b0020 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fd608009bc0 tx=0x7fd60801cb40 comp rx=0 tx=0).stop 2024-09-10T21:18:57.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 -- 172.21.15.16:0/4048434894 shutdown_connections 2024-09-10T21:18:57.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 --2- 172.21.15.16:0/4048434894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6100d0890 0x7fd6100b0020 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 --2- 172.21.15.16:0/4048434894 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6100b3200 0x7fd6100afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 -- 172.21.15.16:0/4048434894 >> 172.21.15.16:0/4048434894 conn(0x7fd6100d1d80 msgr2=0x7fd6100d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:57.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 -- 172.21.15.16:0/4048434894 shutdown_connections 2024-09-10T21:18:57.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 -- 172.21.15.16:0/4048434894 wait complete. 2024-09-10T21:18:57.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 Processor -- start 2024-09-10T21:18:57.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 -- start start 2024-09-10T21:18:57.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6100d0890 0x7fd61006e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:57.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61015f450 0x7fd610151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:57.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6100cb8f0 0x7fd6100c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:57.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd610168d50 con 0x7fd61015f450 2024-09-10T21:18:57.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd610168bd0 con 0x7fd6100d0890 2024-09-10T21:18:57.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd616200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd610168ed0 con 0x7fd6100cb8f0 2024-09-10T21:18:57.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd607e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61015f450 0x7fd610151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:57.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd607e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61015f450 0x7fd610151590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45460/0 (socket says 172.21.15.16:45460) 2024-09-10T21:18:57.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd607e00640 1 -- 172.21.15.16:0/3414579477 learned_addr learned my addr 172.21.15.16:0/3414579477 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:57.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd60f400640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6100d0890 0x7fd61006e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:57.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd60fe00640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6100cb8f0 0x7fd6100c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:57.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd607e00640 1 -- 172.21.15.16:0/3414579477 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6100cb8f0 msgr2=0x7fd6100c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:57.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd607e00640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6100cb8f0 0x7fd6100c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd607e00640 1 -- 172.21.15.16:0/3414579477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6100d0890 msgr2=0x7fd61006e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:57.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd607e00640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6100d0890 0x7fd61006e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd607e00640 1 -- 172.21.15.16:0/3414579477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd610070960 con 0x7fd61015f450 2024-09-10T21:18:57.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.328+0000 7fd607e00640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61015f450 0x7fd610151590 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800c9d0 tx=0x7fd5f800ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:57.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.332+0000 7fd607400640 1 -- 172.21.15.16:0/3414579477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8007c70 con 0x7fd61015f450 2024-09-10T21:18:57.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.332+0000 7fd607400640 1 -- 172.21.15.16:0/3414579477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5f8005020 con 0x7fd61015f450 2024-09-10T21:18:57.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.332+0000 7fd616200640 1 -- 172.21.15.16:0/3414579477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61006f1a0 con 0x7fd61015f450 2024-09-10T21:18:57.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.332+0000 7fd607400640 1 -- 172.21.15.16:0/3414579477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8005470 con 0x7fd61015f450 2024-09-10T21:18:57.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.332+0000 7fd616200640 1 -- 172.21.15.16:0/3414579477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61006f760 con 0x7fd61015f450 2024-09-10T21:18:57.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.332+0000 7fd616200640 1 -- 172.21.15.16:0/3414579477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6100cfc80 con 0x7fd61015f450 2024-09-10T21:18:57.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.332+0000 7fd607400640 1 -- 172.21.15.16:0/3414579477 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd5f80047b0 con 0x7fd61015f450 2024-09-10T21:18:57.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.332+0000 7fd607400640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd5ec072640 0x7fd5ec074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:57.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.332+0000 7fd607400640 1 -- 172.21.15.16:0/3414579477 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fd5f8093720 con 0x7fd61015f450 2024-09-10T21:18:57.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.332+0000 7fd60f400640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd5ec072640 0x7fd5ec074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:57.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.336+0000 7fd60f400640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd5ec072640 0x7fd5ec074b00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd6000046c0 tx=0x7fd600007ec0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.336+0000 7fd607400640 1 -- 172.21.15.16:0/3414579477 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd5f8014030 con 0x7fd61015f450 2024-09-10T21:18:57.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.544+0000 7fd616200640 1 -- 172.21.15.16:0/3414579477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd610074ba0 con 0x7fd61015f450 2024-09-10T21:18:57.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.544+0000 7fd607400640 1 -- 172.21.15.16:0/3414579477 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5232 (secure 0 0 0) 0x7fd5f805e710 con 0x7fd61015f450 2024-09-10T21:18:57.551 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:57.551 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":105,"btime":"2024-09-10T21:18:57:146722+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1121375070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1121375070},{"type":"v1","addr":"172.21.15.52:6835","nonce":1121375070}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":8247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3443454578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3443454578},{"type":"v1","addr":"172.21.15.16:6836","nonce":3443454578}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":8250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1848135983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1848135983},{"type":"v1","addr":"172.21.15.16:6837","nonce":1848135983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:53.527283+0000","modified":"2024-09-10T21:18:56.547852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"b","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1948696903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1948696903},{"type":"v1","addr":"172.21.15.52:6834","nonce":1948696903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8259,"qdb_cluster":[8259]},"id":11}]} 2024-09-10T21:18:57.551 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 105 2024-09-10T21:18:57.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.548+0000 7fd616200640 1 -- 172.21.15.16:0/3414579477 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd5ec072640 msgr2=0x7fd5ec074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:57.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.548+0000 7fd616200640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd5ec072640 0x7fd5ec074b00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd6000046c0 tx=0x7fd600007ec0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.548+0000 7fd616200640 1 -- 172.21.15.16:0/3414579477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61015f450 msgr2=0x7fd610151590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:57.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.548+0000 7fd616200640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61015f450 0x7fd610151590 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800c9d0 tx=0x7fd5f800ce90 comp rx=0 tx=0).stop 2024-09-10T21:18:57.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.548+0000 7fd616200640 1 -- 172.21.15.16:0/3414579477 shutdown_connections 2024-09-10T21:18:57.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.548+0000 7fd616200640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd5ec072640 0x7fd5ec074b00 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.548+0000 7fd616200640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd6100cb8f0 0x7fd6100c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.548+0000 7fd616200640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61015f450 0x7fd610151590 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.548+0000 7fd616200640 1 --2- 172.21.15.16:0/3414579477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6100d0890 0x7fd61006e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.548+0000 7fd616200640 1 -- 172.21.15.16:0/3414579477 >> 172.21.15.16:0/3414579477 conn(0x7fd6100d1d80 msgr2=0x7fd610152d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:57.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.548+0000 7fd616200640 1 -- 172.21.15.16:0/3414579477 shutdown_connections 2024-09-10T21:18:57.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.548+0000 7fd616200640 1 -- 172.21.15.16:0/3414579477 wait complete. 2024-09-10T21:18:57.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:18:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.716+0000 7f6a89400640 1 -- 172.21.15.16:0/1417744333 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a84151ee0 msgr2=0x7f6a8415bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.716+0000 7f6a89400640 1 --2- 172.21.15.16:0/1417744333 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a84151ee0 0x7f6a8415bec0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009870 tx=0x7f6a7401c830 comp rx=0 tx=0).stop 2024-09-10T21:18:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.716+0000 7f6a89400640 1 -- 172.21.15.16:0/1417744333 shutdown_connections 2024-09-10T21:18:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.716+0000 7f6a89400640 1 --2- 172.21.15.16:0/1417744333 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a84151ee0 0x7f6a8415bec0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.716+0000 7f6a89400640 1 -- 172.21.15.16:0/1417744333 >> 172.21.15.16:0/1417744333 conn(0x7f6a840d04e0 msgr2=0x7f6a840d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.716+0000 7f6a89400640 1 -- 172.21.15.16:0/1417744333 shutdown_connections 2024-09-10T21:18:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.716+0000 7f6a89400640 1 -- 172.21.15.16:0/1417744333 wait complete. 2024-09-10T21:18:57.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.716+0000 7f6a89400640 1 Processor -- start 2024-09-10T21:18:57.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a89400640 1 -- start start 2024-09-10T21:18:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a89400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a840737b0 0x7f6a84151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a89400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a840af6a0 0x7f6a84142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a89400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a84151ee0 0x7f6a84150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a89400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a84169430 con 0x7f6a840af6a0 2024-09-10T21:18:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a89400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a841692b0 con 0x7f6a840737b0 2024-09-10T21:18:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a89400640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6a841695b0 con 0x7f6a84151ee0 2024-09-10T21:18:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a81e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a840af6a0 0x7f6a84142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a81e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a840af6a0 0x7f6a84142030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45472/0 (socket says 172.21.15.16:45472) 2024-09-10T21:18:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a81e00640 1 -- 172.21.15.16:0/413535883 learned_addr learned my addr 172.21.15.16:0/413535883 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a82800640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a840737b0 0x7f6a84151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a83200640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a84151ee0 0x7f6a84150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a81e00640 1 -- 172.21.15.16:0/413535883 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a84151ee0 msgr2=0x7f6a84150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a81e00640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a84151ee0 0x7f6a84150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a81e00640 1 -- 172.21.15.16:0/413535883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a840737b0 msgr2=0x7f6a84151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a81e00640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a840737b0 0x7f6a84151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a81e00640 1 -- 172.21.15.16:0/413535883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a840c7e10 con 0x7f6a840af6a0 2024-09-10T21:18:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a81e00640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a840af6a0 0x7f6a84142030 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c490 tx=0x7f6a7000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a7aa00640 1 -- 172.21.15.16:0/413535883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70016020 con 0x7f6a840af6a0 2024-09-10T21:18:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a7aa00640 1 -- 172.21.15.16:0/413535883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a70012920 con 0x7f6a840af6a0 2024-09-10T21:18:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a7aa00640 1 -- 172.21.15.16:0/413535883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70012d50 con 0x7f6a840af6a0 2024-09-10T21:18:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a89400640 1 -- 172.21.15.16:0/413535883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a84150f60 con 0x7f6a840af6a0 2024-09-10T21:18:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a89400640 1 -- 172.21.15.16:0/413535883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a840b0560 con 0x7f6a840af6a0 2024-09-10T21:18:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.720+0000 7f6a89400640 1 -- 172.21.15.16:0/413535883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a8415bec0 con 0x7f6a840af6a0 2024-09-10T21:18:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.724+0000 7f6a7aa00640 1 -- 172.21.15.16:0/413535883 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6a7001b400 con 0x7f6a840af6a0 2024-09-10T21:18:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.724+0000 7f6a7aa00640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6a64072610 0x7f6a64074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.724+0000 7f6a7aa00640 1 -- 172.21.15.16:0/413535883 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f6a70093440 con 0x7f6a840af6a0 2024-09-10T21:18:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.724+0000 7f6a82800640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6a64072610 0x7f6a64074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.724+0000 7f6a82800640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6a64072610 0x7f6a64074ad0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6a740af000 tx=0x7f6a7401b070 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.728+0000 7f6a7aa00640 1 -- 172.21.15.16:0/413535883 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6a70020020 con 0x7f6a840af6a0 2024-09-10T21:18:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a89400640 1 -- 172.21.15.16:0/413535883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a8415b000 con 0x7f6a840af6a0 2024-09-10T21:18:57.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a7aa00640 1 -- 172.21.15.16:0/413535883 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+16691 (secure 0 0 0) 0x7f6a7005e430 con 0x7f6a840af6a0 2024-09-10T21:18:57.909 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:57.909 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":138,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:18:55.112658+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:18:51.247076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T21:18:52.488695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:18:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a89400640 1 -- 172.21.15.16:0/413535883 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6a64072610 msgr2=0x7f6a64074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a89400640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6a64072610 0x7f6a64074ad0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6a740af000 tx=0x7f6a7401b070 comp rx=0 tx=0).stop 2024-09-10T21:18:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a89400640 1 -- 172.21.15.16:0/413535883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a840af6a0 msgr2=0x7f6a84142030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a89400640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a840af6a0 0x7f6a84142030 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c490 tx=0x7f6a7000c950 comp rx=0 tx=0).stop 2024-09-10T21:18:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a89400640 1 -- 172.21.15.16:0/413535883 shutdown_connections 2024-09-10T21:18:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a89400640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6a64072610 0x7f6a64074ad0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a89400640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a84151ee0 0x7f6a84150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a89400640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a840af6a0 0x7f6a84142030 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a89400640 1 --2- 172.21.15.16:0/413535883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a840737b0 0x7f6a84151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a89400640 1 -- 172.21.15.16:0/413535883 >> 172.21.15.16:0/413535883 conn(0x7f6a840d04e0 msgr2=0x7f6a8406f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a89400640 1 -- 172.21.15.16:0/413535883 shutdown_connections 2024-09-10T21:18:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:57.904+0000 7f6a89400640 1 -- 172.21.15.16:0/413535883 wait complete. 2024-09-10T21:18:57.932 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:18:57.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:18:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.072+0000 7fa64b000640 1 -- 172.21.15.16:0/4291273237 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6440cc5e0 msgr2=0x7fa6440d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.072+0000 7fa64b000640 1 --2- 172.21.15.16:0/4291273237 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6440cc5e0 0x7fa6440d1cb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009c80 tx=0x7fa63c01cd10 comp rx=0 tx=0).stop 2024-09-10T21:18:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.072+0000 7fa64b000640 1 -- 172.21.15.16:0/4291273237 shutdown_connections 2024-09-10T21:18:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.072+0000 7fa64b000640 1 --2- 172.21.15.16:0/4291273237 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6440cc5e0 0x7fa6440d1cb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.072+0000 7fa64b000640 1 --2- 172.21.15.16:0/4291273237 >> v2:172.21.15.52:3300/0 conn(0x7fa6440cfc40 0x7fa64415fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.072+0000 7fa64b000640 1 -- 172.21.15.16:0/4291273237 >> 172.21.15.16:0/4291273237 conn(0x7fa644074c50 msgr2=0x7fa64409a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.072+0000 7fa64b000640 1 -- 172.21.15.16:0/4291273237 shutdown_connections 2024-09-10T21:18:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.072+0000 7fa64b000640 1 -- 172.21.15.16:0/4291273237 wait complete. 2024-09-10T21:18:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa64b000640 1 Processor -- start 2024-09-10T21:18:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa64b000640 1 -- start start 2024-09-10T21:18:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa64b000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6440cc5e0 0x7fa644154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa64b000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6440cfc40 0x7fa644154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa64b000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa64415bfc0 0x7fa6441f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa64b000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa644168b30 con 0x7fa6440cc5e0 2024-09-10T21:18:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa64b000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441689b0 con 0x7fa64415bfc0 2024-09-10T21:18:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa64b000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa644168cb0 con 0x7fa6440cfc40 2024-09-10T21:18:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa649c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6440cc5e0 0x7fa644154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa649c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6440cc5e0 0x7fa644154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45486/0 (socket says 172.21.15.16:45486) 2024-09-10T21:18:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa649c00640 1 -- 172.21.15.16:0/1295989674 learned_addr learned my addr 172.21.15.16:0/1295989674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa649200640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6440cfc40 0x7fa644154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa64a600640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa64415bfc0 0x7fa6441f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa649c00640 1 -- 172.21.15.16:0/1295989674 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6440cfc40 msgr2=0x7fa644154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa649c00640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6440cfc40 0x7fa644154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa649c00640 1 -- 172.21.15.16:0/1295989674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa64415bfc0 msgr2=0x7fa6441f8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa649c00640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa64415bfc0 0x7fa6441f8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa649c00640 1 -- 172.21.15.16:0/1295989674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa644152ac0 con 0x7fa6440cc5e0 2024-09-10T21:18:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa649200640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6440cfc40 0x7fa644154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa64a600640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa64415bfc0 0x7fa6441f8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa649c00640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6440cc5e0 0x7fa644154a90 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fa63800ea20 tx=0x7fa63800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa641600640 1 -- 172.21.15.16:0/1295989674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63800cd80 con 0x7fa6440cc5e0 2024-09-10T21:18:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa641600640 1 -- 172.21.15.16:0/1295989674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa638010430 con 0x7fa6440cc5e0 2024-09-10T21:18:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa641600640 1 -- 172.21.15.16:0/1295989674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638010860 con 0x7fa6440cc5e0 2024-09-10T21:18:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa64b000640 1 -- 172.21.15.16:0/1295989674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64414b4e0 con 0x7fa6440cc5e0 2024-09-10T21:18:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.076+0000 7fa64b000640 1 -- 172.21.15.16:0/1295989674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64415a960 con 0x7fa6440cc5e0 2024-09-10T21:18:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.080+0000 7fa64b000640 1 -- 172.21.15.16:0/1295989674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6440d1cb0 con 0x7fa6440cc5e0 2024-09-10T21:18:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.080+0000 7fa641600640 1 -- 172.21.15.16:0/1295989674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa638010a00 con 0x7fa6440cc5e0 2024-09-10T21:18:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.080+0000 7fa641600640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa624072610 0x7fa624074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.080+0000 7fa641600640 1 -- 172.21.15.16:0/1295989674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fa638093e70 con 0x7fa6440cc5e0 2024-09-10T21:18:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.080+0000 7fa649200640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa624072610 0x7fa624074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.080+0000 7fa649200640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa624072610 0x7fa624074ad0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa6441525d0 tx=0x7fa630009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.084+0000 7fa641600640 1 -- 172.21.15.16:0/1295989674 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa64414b4e0 con 0x7fa6440cc5e0 2024-09-10T21:18:58.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.256+0000 7fa64b000640 1 -- 172.21.15.16:0/1295989674 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa64415ebe0 con 0x7fa624072610 2024-09-10T21:18:58.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.260+0000 7fa641600640 1 -- 172.21.15.16:0/1295989674 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245305 (secure 0 0 0) 0x7fa64415ebe0 con 0x7fa624072610 2024-09-10T21:18:58.270 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:18:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.264+0000 7fa64b000640 1 -- 172.21.15.16:0/1295989674 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa624072610 msgr2=0x7fa624074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.264+0000 7fa64b000640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa624072610 0x7fa624074ad0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa6441525d0 tx=0x7fa630009040 comp rx=0 tx=0).stop 2024-09-10T21:18:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.264+0000 7fa64b000640 1 -- 172.21.15.16:0/1295989674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6440cc5e0 msgr2=0x7fa644154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.264+0000 7fa64b000640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6440cc5e0 0x7fa644154a90 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fa63800ea20 tx=0x7fa63800eee0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.264+0000 7fa64b000640 1 -- 172.21.15.16:0/1295989674 shutdown_connections 2024-09-10T21:18:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.264+0000 7fa64b000640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa624072610 0x7fa624074ad0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.264+0000 7fa64b000640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa64415bfc0 0x7fa6441f8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.264+0000 7fa64b000640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa6440cfc40 0x7fa644154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.264+0000 7fa64b000640 1 --2- 172.21.15.16:0/1295989674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6440cc5e0 0x7fa644154a90 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.264+0000 7fa64b000640 1 -- 172.21.15.16:0/1295989674 >> 172.21.15.16:0/1295989674 conn(0x7fa644074c50 msgr2=0x7fa644158d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.264+0000 7fa64b000640 1 -- 172.21.15.16:0/1295989674 shutdown_connections 2024-09-10T21:18:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.264+0000 7fa64b000640 1 -- 172.21.15.16:0/1295989674 wait complete. 2024-09-10T21:18:58.291 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-10T21:18:57.575293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":166,"acting":166,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":293096,"kb_used_data":14448,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754681624,"statfs":{"total":773094113280,"available":772793982976,"internally_reserved":0,"allocated":14794752,"data_stored":7034506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.472130"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118441+0000","last_change":"2024-09-10T21:18:53.118441+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118441+0000","last_undegraded":"2024-09-10T21:18:53.118441+0000","last_fullsized":"2024-09-10T21:18:53.118441+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117949+0000","last_change":"2024-09-10T21:18:53.117949+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117949+0000","last_undegraded":"2024-09-10T21:18:53.117949+0000","last_fullsized":"2024-09-10T21:18:53.117949+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118237+0000","last_change":"2024-09-10T21:18:53.118237+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118237+0000","last_undegraded":"2024-09-10T21:18:53.118237+0000","last_fullsized":"2024-09-10T21:18:53.118237+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118754+0000","last_change":"2024-09-10T21:18:53.118754+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118754+0000","last_undegraded":"2024-09-10T21:18:53.118754+0000","last_fullsized":"2024-09-10T21:18:53.118754+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121298+0000","last_change":"2024-09-10T21:18:53.121298+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121298+0000","last_undegraded":"2024-09-10T21:18:53.121298+0000","last_fullsized":"2024-09-10T21:18:53.121298+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124732+0000","last_change":"2024-09-10T21:18:53.124732+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124732+0000","last_undegraded":"2024-09-10T21:18:53.124732+0000","last_fullsized":"2024-09-10T21:18:53.124732+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124460+0000","last_change":"2024-09-10T21:18:53.124460+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124460+0000","last_undegraded":"2024-09-10T21:18:53.124460+0000","last_fullsized":"2024-09-10T21:18:53.124460+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123934+0000","last_change":"2024-09-10T21:18:53.123934+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123934+0000","last_undegraded":"2024-09-10T21:18:53.123934+0000","last_fullsized":"2024-09-10T21:18:53.123934+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117758+0000","last_change":"2024-09-10T21:18:53.117758+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117758+0000","last_undegraded":"2024-09-10T21:18:53.117758+0000","last_fullsized":"2024-09-10T21:18:53.117758+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117647+0000","last_change":"2024-09-10T21:18:53.117647+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117647+0000","last_undegraded":"2024-09-10T21:18:53.117647+0000","last_fullsized":"2024-09-10T21:18:53.117647+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124781+0000","last_change":"2024-09-10T21:18:53.124781+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124781+0000","last_undegraded":"2024-09-10T21:18:53.124781+0000","last_fullsized":"2024-09-10T21:18:53.124781+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123669+0000","last_change":"2024-09-10T21:18:53.123669+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123669+0000","last_undegraded":"2024-09-10T21:18:53.123669+0000","last_fullsized":"2024-09-10T21:18:53.123669+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123065+0000","last_change":"2024-09-10T21:18:53.123065+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123065+0000","last_undegraded":"2024-09-10T21:18:53.123065+0000","last_fullsized":"2024-09-10T21:18:53.123065+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.115101+0000","last_change":"2024-09-10T21:18:53.115101+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.115101+0000","last_undegraded":"2024-09-10T21:18:53.115101+0000","last_fullsized":"2024-09-10T21:18:53.115101+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124613+0000","last_change":"2024-09-10T21:18:53.124613+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124613+0000","last_undegraded":"2024-09-10T21:18:53.124613+0000","last_fullsized":"2024-09-10T21:18:53.124613+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117230+0000","last_change":"2024-09-10T21:18:53.117230+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117230+0000","last_undegraded":"2024-09-10T21:18:53.117230+0000","last_fullsized":"2024-09-10T21:18:53.117230+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.115573+0000","last_change":"2024-09-10T21:18:53.115573+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.115573+0000","last_undegraded":"2024-09-10T21:18:53.115573+0000","last_fullsized":"2024-09-10T21:18:53.115573+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118327+0000","last_change":"2024-09-10T21:18:53.118327+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118327+0000","last_undegraded":"2024-09-10T21:18:53.118327+0000","last_fullsized":"2024-09-10T21:18:53.118327+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.122414+0000","last_change":"2024-09-10T21:18:53.122414+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.122414+0000","last_undegraded":"2024-09-10T21:18:53.122414+0000","last_fullsized":"2024-09-10T21:18:53.122414+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117885+0000","last_change":"2024-09-10T21:18:53.117885+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117885+0000","last_undegraded":"2024-09-10T21:18:53.117885+0000","last_fullsized":"2024-09-10T21:18:53.117885+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124230+0000","last_change":"2024-09-10T21:18:53.124230+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124230+0000","last_undegraded":"2024-09-10T21:18:53.124230+0000","last_fullsized":"2024-09-10T21:18:53.124230+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118139+0000","last_change":"2024-09-10T21:18:53.118139+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118139+0000","last_undegraded":"2024-09-10T21:18:53.118139+0000","last_fullsized":"2024-09-10T21:18:53.118139+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114559+0000","last_change":"2024-09-10T21:18:53.114559+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114559+0000","last_undegraded":"2024-09-10T21:18:53.114559+0000","last_fullsized":"2024-09-10T21:18:53.114559+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.130551+0000","last_change":"2024-09-10T21:18:53.122058+0000","last_active":"2024-09-10T21:18:53.130551+0000","last_peered":"2024-09-10T21:18:53.130551+0000","last_clean":"2024-09-10T21:18:53.130551+0000","last_became_active":"2024-09-10T21:18:53.121768+0000","last_became_peered":"2024-09-10T21:18:53.121768+0000","last_unstale":"2024-09-10T21:18:53.130551+0000","last_undegraded":"2024-09-10T21:18:53.130551+0000","last_fullsized":"2024-09-10T21:18:53.130551+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:46.151037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116889+0000","last_change":"2024-09-10T21:18:53.116889+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116889+0000","last_undegraded":"2024-09-10T21:18:53.116889+0000","last_fullsized":"2024-09-10T21:18:53.116889+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.140097+0000","last_change":"2024-09-10T21:18:53.134253+0000","last_active":"2024-09-10T21:18:53.140097+0000","last_peered":"2024-09-10T21:18:53.140097+0000","last_clean":"2024-09-10T21:18:53.140097+0000","last_became_active":"2024-09-10T21:18:53.133971+0000","last_became_peered":"2024-09-10T21:18:53.133971+0000","last_unstale":"2024-09-10T21:18:53.140097+0000","last_undegraded":"2024-09-10T21:18:53.140097+0000","last_fullsized":"2024-09-10T21:18:53.140097+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:05.811497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.104469+0000","last_change":"2024-09-10T21:18:52.104469+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.104469+0000","last_undegraded":"2024-09-10T21:18:52.104469+0000","last_fullsized":"2024-09-10T21:18:52.104469+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123461+0000","last_change":"2024-09-10T21:18:53.123461+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123461+0000","last_undegraded":"2024-09-10T21:18:53.123461+0000","last_fullsized":"2024-09-10T21:18:53.123461+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.134759+0000","last_change":"2024-09-10T21:18:53.132095+0000","last_active":"2024-09-10T21:18:53.134759+0000","last_peered":"2024-09-10T21:18:53.134759+0000","last_clean":"2024-09-10T21:18:53.134759+0000","last_became_active":"2024-09-10T21:18:53.131751+0000","last_became_peered":"2024-09-10T21:18:53.131751+0000","last_unstale":"2024-09-10T21:18:53.134759+0000","last_undegraded":"2024-09-10T21:18:53.134759+0000","last_fullsized":"2024-09-10T21:18:53.134759+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:03.376294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116079+0000","last_change":"2024-09-10T21:18:53.116079+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116079+0000","last_undegraded":"2024-09-10T21:18:53.116079+0000","last_fullsized":"2024-09-10T21:18:53.116079+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.138579+0000","last_change":"2024-09-10T21:18:53.135048+0000","last_active":"2024-09-10T21:18:53.138579+0000","last_peered":"2024-09-10T21:18:53.138579+0000","last_clean":"2024-09-10T21:18:53.138579+0000","last_became_active":"2024-09-10T21:18:53.134366+0000","last_became_peered":"2024-09-10T21:18:53.134366+0000","last_unstale":"2024-09-10T21:18:53.138579+0000","last_undegraded":"2024-09-10T21:18:53.138579+0000","last_fullsized":"2024-09-10T21:18:53.138579+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:53.969257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.132642+0000","last_change":"2024-09-10T21:18:53.130356+0000","last_active":"2024-09-10T21:18:53.132642+0000","last_peered":"2024-09-10T21:18:53.132642+0000","last_clean":"2024-09-10T21:18:53.132642+0000","last_became_active":"2024-09-10T21:18:53.130043+0000","last_became_peered":"2024-09-10T21:18:53.130043+0000","last_unstale":"2024-09-10T21:18:53.132642+0000","last_undegraded":"2024-09-10T21:18:53.132642+0000","last_fullsized":"2024-09-10T21:18:53.132642+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:02.691227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114902+0000","last_change":"2024-09-10T21:18:53.114902+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114902+0000","last_undegraded":"2024-09-10T21:18:53.114902+0000","last_fullsized":"2024-09-10T21:18:53.114902+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.134829+0000","last_change":"2024-09-10T21:18:53.129464+0000","last_active":"2024-09-10T21:18:53.134829+0000","last_peered":"2024-09-10T21:18:53.134829+0000","last_clean":"2024-09-10T21:18:53.134829+0000","last_became_active":"2024-09-10T21:18:53.129179+0000","last_became_peered":"2024-09-10T21:18:53.129179+0000","last_unstale":"2024-09-10T21:18:53.134829+0000","last_undegraded":"2024-09-10T21:18:53.134829+0000","last_fullsized":"2024-09-10T21:18:53.134829+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:10.823382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114044+0000","last_change":"2024-09-10T21:18:53.114044+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114044+0000","last_undegraded":"2024-09-10T21:18:53.114044+0000","last_fullsized":"2024-09-10T21:18:53.114044+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.132295+0000","last_change":"2024-09-10T21:18:53.127488+0000","last_active":"2024-09-10T21:18:53.132295+0000","last_peered":"2024-09-10T21:18:53.132295+0000","last_clean":"2024-09-10T21:18:53.132295+0000","last_became_active":"2024-09-10T21:18:53.125698+0000","last_became_peered":"2024-09-10T21:18:53.125698+0000","last_unstale":"2024-09-10T21:18:53.132295+0000","last_undegraded":"2024-09-10T21:18:53.132295+0000","last_fullsized":"2024-09-10T21:18:53.132295+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:47.324904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121885+0000","last_change":"2024-09-10T21:18:53.121885+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121885+0000","last_undegraded":"2024-09-10T21:18:53.121885+0000","last_fullsized":"2024-09-10T21:18:53.121885+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.104899+0000","last_change":"2024-09-10T21:18:52.104899+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.104899+0000","last_undegraded":"2024-09-10T21:18:52.104899+0000","last_fullsized":"2024-09-10T21:18:52.104899+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121944+0000","last_change":"2024-09-10T21:18:53.121944+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121944+0000","last_undegraded":"2024-09-10T21:18:53.121944+0000","last_fullsized":"2024-09-10T21:18:53.121944+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.136282+0000","last_change":"2024-09-10T21:18:53.134405+0000","last_active":"2024-09-10T21:18:53.136282+0000","last_peered":"2024-09-10T21:18:53.136282+0000","last_clean":"2024-09-10T21:18:53.136282+0000","last_became_active":"2024-09-10T21:18:53.134012+0000","last_became_peered":"2024-09-10T21:18:53.134012+0000","last_unstale":"2024-09-10T21:18:53.136282+0000","last_undegraded":"2024-09-10T21:18:53.136282+0000","last_fullsized":"2024-09-10T21:18:53.136282+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:09.018998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116619+0000","last_change":"2024-09-10T21:18:53.116619+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116619+0000","last_undegraded":"2024-09-10T21:18:53.116619+0000","last_fullsized":"2024-09-10T21:18:53.116619+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111870+0000","last_change":"2024-09-10T21:18:52.111870+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111870+0000","last_undegraded":"2024-09-10T21:18:52.111870+0000","last_fullsized":"2024-09-10T21:18:52.111870+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121517+0000","last_change":"2024-09-10T21:18:53.121517+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121517+0000","last_undegraded":"2024-09-10T21:18:53.121517+0000","last_fullsized":"2024-09-10T21:18:53.121517+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.131783+0000","last_change":"2024-09-10T21:18:53.127007+0000","last_active":"2024-09-10T21:18:53.131783+0000","last_peered":"2024-09-10T21:18:53.131783+0000","last_clean":"2024-09-10T21:18:53.131783+0000","last_became_active":"2024-09-10T21:18:53.126673+0000","last_became_peered":"2024-09-10T21:18:53.126673+0000","last_unstale":"2024-09-10T21:18:53.131783+0000","last_undegraded":"2024-09-10T21:18:53.131783+0000","last_fullsized":"2024-09-10T21:18:53.131783+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:39.746760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116547+0000","last_change":"2024-09-10T21:18:53.116547+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116547+0000","last_undegraded":"2024-09-10T21:18:53.116547+0000","last_fullsized":"2024-09-10T21:18:53.116547+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.104198+0000","last_change":"2024-09-10T21:18:52.104198+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.104198+0000","last_undegraded":"2024-09-10T21:18:52.104198+0000","last_fullsized":"2024-09-10T21:18:52.104198+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.138049+0000","last_change":"2024-09-10T21:18:53.137417+0000","last_active":"2024-09-10T21:18:53.138049+0000","last_peered":"2024-09-10T21:18:53.138049+0000","last_clean":"2024-09-10T21:18:53.138049+0000","last_became_active":"2024-09-10T21:18:53.137042+0000","last_became_peered":"2024-09-10T21:18:53.137042+0000","last_unstale":"2024-09-10T21:18:53.138049+0000","last_undegraded":"2024-09-10T21:18:53.138049+0000","last_fullsized":"2024-09-10T21:18:53.138049+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:45.885394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123123+0000","last_change":"2024-09-10T21:18:53.123123+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123123+0000","last_undegraded":"2024-09-10T21:18:53.123123+0000","last_fullsized":"2024-09-10T21:18:53.123123+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111728+0000","last_change":"2024-09-10T21:18:52.111728+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111728+0000","last_undegraded":"2024-09-10T21:18:52.111728+0000","last_fullsized":"2024-09-10T21:18:52.111728+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.122766+0000","last_change":"2024-09-10T21:18:53.122766+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.122766+0000","last_undegraded":"2024-09-10T21:18:53.122766+0000","last_fullsized":"2024-09-10T21:18:53.122766+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111758+0000","last_change":"2024-09-10T21:18:52.111758+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111758+0000","last_undegraded":"2024-09-10T21:18:52.111758+0000","last_fullsized":"2024-09-10T21:18:52.111758+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117371+0000","last_change":"2024-09-10T21:18:53.117371+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117371+0000","last_undegraded":"2024-09-10T21:18:53.117371+0000","last_fullsized":"2024-09-10T21:18:53.117371+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.140124+0000","last_change":"2024-09-10T21:18:53.136538+0000","last_active":"2024-09-10T21:18:53.140124+0000","last_peered":"2024-09-10T21:18:53.140124+0000","last_clean":"2024-09-10T21:18:53.140124+0000","last_became_active":"2024-09-10T21:18:53.135006+0000","last_became_peered":"2024-09-10T21:18:53.135006+0000","last_unstale":"2024-09-10T21:18:53.140124+0000","last_undegraded":"2024-09-10T21:18:53.140124+0000","last_fullsized":"2024-09-10T21:18:53.140124+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:58.524531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117304+0000","last_change":"2024-09-10T21:18:53.117304+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117304+0000","last_undegraded":"2024-09-10T21:18:53.117304+0000","last_fullsized":"2024-09-10T21:18:53.117304+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111997+0000","last_change":"2024-09-10T21:18:52.111997+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111997+0000","last_undegraded":"2024-09-10T21:18:52.111997+0000","last_fullsized":"2024-09-10T21:18:52.111997+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.115916+0000","last_change":"2024-09-10T21:18:53.115916+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.115916+0000","last_undegraded":"2024-09-10T21:18:53.115916+0000","last_fullsized":"2024-09-10T21:18:53.115916+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.133308+0000","last_change":"2024-09-10T21:18:53.126832+0000","last_active":"2024-09-10T21:18:53.133308+0000","last_peered":"2024-09-10T21:18:53.133308+0000","last_clean":"2024-09-10T21:18:53.133308+0000","last_became_active":"2024-09-10T21:18:53.126580+0000","last_became_peered":"2024-09-10T21:18:53.126580+0000","last_unstale":"2024-09-10T21:18:53.133308+0000","last_undegraded":"2024-09-10T21:18:53.133308+0000","last_fullsized":"2024-09-10T21:18:53.133308+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:42.773987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116655+0000","last_change":"2024-09-10T21:18:53.116655+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116655+0000","last_undegraded":"2024-09-10T21:18:53.116655+0000","last_fullsized":"2024-09-10T21:18:53.116655+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.134062+0000","last_change":"2024-09-10T21:18:53.129128+0000","last_active":"2024-09-10T21:18:53.134062+0000","last_peered":"2024-09-10T21:18:53.134062+0000","last_clean":"2024-09-10T21:18:53.134062+0000","last_became_active":"2024-09-10T21:18:53.128826+0000","last_became_peered":"2024-09-10T21:18:53.128826+0000","last_unstale":"2024-09-10T21:18:53.134062+0000","last_undegraded":"2024-09-10T21:18:53.134062+0000","last_fullsized":"2024-09-10T21:18:53.134062+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:14.312782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116046+0000","last_change":"2024-09-10T21:18:53.116046+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116046+0000","last_undegraded":"2024-09-10T21:18:53.116046+0000","last_fullsized":"2024-09-10T21:18:53.116046+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.133416+0000","last_change":"2024-09-10T21:18:53.128814+0000","last_active":"2024-09-10T21:18:53.133416+0000","last_peered":"2024-09-10T21:18:53.133416+0000","last_clean":"2024-09-10T21:18:53.133416+0000","last_became_active":"2024-09-10T21:18:53.128474+0000","last_became_peered":"2024-09-10T21:18:53.128474+0000","last_unstale":"2024-09-10T21:18:53.133416+0000","last_undegraded":"2024-09-10T21:18:53.133416+0000","last_fullsized":"2024-09-10T21:18:53.133416+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:50.343267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116419+0000","last_change":"2024-09-10T21:18:53.116419+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116419+0000","last_undegraded":"2024-09-10T21:18:53.116419+0000","last_fullsized":"2024-09-10T21:18:53.116419+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.135855+0000","last_change":"2024-09-10T21:18:53.132527+0000","last_active":"2024-09-10T21:18:53.135855+0000","last_peered":"2024-09-10T21:18:53.135855+0000","last_clean":"2024-09-10T21:18:53.135855+0000","last_became_active":"2024-09-10T21:18:53.130251+0000","last_became_peered":"2024-09-10T21:18:53.130251+0000","last_unstale":"2024-09-10T21:18:53.135855+0000","last_undegraded":"2024-09-10T21:18:53.135855+0000","last_fullsized":"2024-09-10T21:18:53.135855+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:12.376207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123636+0000","last_change":"2024-09-10T21:18:53.123636+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123636+0000","last_undegraded":"2024-09-10T21:18:53.123636+0000","last_fullsized":"2024-09-10T21:18:53.123636+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.137844+0000","last_change":"2024-09-10T21:18:53.133231+0000","last_active":"2024-09-10T21:18:53.137844+0000","last_peered":"2024-09-10T21:18:53.137844+0000","last_clean":"2024-09-10T21:18:53.137844+0000","last_became_active":"2024-09-10T21:18:53.132930+0000","last_became_peered":"2024-09-10T21:18:53.132930+0000","last_unstale":"2024-09-10T21:18:53.137844+0000","last_undegraded":"2024-09-10T21:18:53.137844+0000","last_fullsized":"2024-09-10T21:18:53.137844+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:26.790852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.115124+0000","last_change":"2024-09-10T21:18:53.115124+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.115124+0000","last_undegraded":"2024-09-10T21:18:53.115124+0000","last_fullsized":"2024-09-10T21:18:53.115124+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.105072+0000","last_change":"2024-09-10T21:18:52.105072+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.105072+0000","last_undegraded":"2024-09-10T21:18:52.105072+0000","last_fullsized":"2024-09-10T21:18:52.105072+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116808+0000","last_change":"2024-09-10T21:18:53.116808+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116808+0000","last_undegraded":"2024-09-10T21:18:53.116808+0000","last_fullsized":"2024-09-10T21:18:53.116808+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.134459+0000","last_change":"2024-09-10T21:18:53.130476+0000","last_active":"2024-09-10T21:18:53.134459+0000","last_peered":"2024-09-10T21:18:53.134459+0000","last_clean":"2024-09-10T21:18:53.134459+0000","last_became_active":"2024-09-10T21:18:53.130213+0000","last_became_peered":"2024-09-10T21:18:53.130213+0000","last_unstale":"2024-09-10T21:18:53.134459+0000","last_undegraded":"2024-09-10T21:18:53.134459+0000","last_fullsized":"2024-09-10T21:18:53.134459+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:40.694947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114244+0000","last_change":"2024-09-10T21:18:53.114244+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114244+0000","last_undegraded":"2024-09-10T21:18:53.114244+0000","last_fullsized":"2024-09-10T21:18:53.114244+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.135183+0000","last_change":"2024-09-10T21:18:53.130268+0000","last_active":"2024-09-10T21:18:53.135183+0000","last_peered":"2024-09-10T21:18:53.135183+0000","last_clean":"2024-09-10T21:18:53.135183+0000","last_became_active":"2024-09-10T21:18:53.129955+0000","last_became_peered":"2024-09-10T21:18:53.129955+0000","last_unstale":"2024-09-10T21:18:53.135183+0000","last_undegraded":"2024-09-10T21:18:53.135183+0000","last_fullsized":"2024-09-10T21:18:53.135183+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:03.568523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114041+0000","last_change":"2024-09-10T21:18:53.114041+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114041+0000","last_undegraded":"2024-09-10T21:18:53.114041+0000","last_fullsized":"2024-09-10T21:18:53.114041+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.135542+0000","last_change":"2024-09-10T21:18:53.133483+0000","last_active":"2024-09-10T21:18:53.135542+0000","last_peered":"2024-09-10T21:18:53.135542+0000","last_clean":"2024-09-10T21:18:53.135542+0000","last_became_active":"2024-09-10T21:18:53.133196+0000","last_became_peered":"2024-09-10T21:18:53.133196+0000","last_unstale":"2024-09-10T21:18:53.135542+0000","last_undegraded":"2024-09-10T21:18:53.135542+0000","last_fullsized":"2024-09-10T21:18:53.135542+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:42:00.777546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111911+0000","last_change":"2024-09-10T21:18:52.111911+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111911+0000","last_undegraded":"2024-09-10T21:18:52.111911+0000","last_fullsized":"2024-09-10T21:18:52.111911+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116699+0000","last_change":"2024-09-10T21:18:53.116699+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116699+0000","last_undegraded":"2024-09-10T21:18:53.116699+0000","last_fullsized":"2024-09-10T21:18:53.116699+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.135056+0000","last_change":"2024-09-10T21:18:53.133724+0000","last_active":"2024-09-10T21:18:53.135056+0000","last_peered":"2024-09-10T21:18:53.135056+0000","last_clean":"2024-09-10T21:18:53.135056+0000","last_became_active":"2024-09-10T21:18:53.133442+0000","last_became_peered":"2024-09-10T21:18:53.133442+0000","last_unstale":"2024-09-10T21:18:53.135056+0000","last_undegraded":"2024-09-10T21:18:53.135056+0000","last_fullsized":"2024-09-10T21:18:53.135056+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:12.220728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121788+0000","last_change":"2024-09-10T21:18:53.121788+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121788+0000","last_undegraded":"2024-09-10T21:18:53.121788+0000","last_fullsized":"2024-09-10T21:18:53.121788+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.136783+0000","last_change":"2024-09-10T21:18:53.132411+0000","last_active":"2024-09-10T21:18:53.136783+0000","last_peered":"2024-09-10T21:18:53.136783+0000","last_clean":"2024-09-10T21:18:53.136783+0000","last_became_active":"2024-09-10T21:18:53.131949+0000","last_became_peered":"2024-09-10T21:18:53.131949+0000","last_unstale":"2024-09-10T21:18:53.136783+0000","last_undegraded":"2024-09-10T21:18:53.136783+0000","last_fullsized":"2024-09-10T21:18:53.136783+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:59.498416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116457+0000","last_change":"2024-09-10T21:18:53.116457+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116457+0000","last_undegraded":"2024-09-10T21:18:53.116457+0000","last_fullsized":"2024-09-10T21:18:53.116457+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.131780+0000","last_change":"2024-09-10T21:18:53.127615+0000","last_active":"2024-09-10T21:18:53.131780+0000","last_peered":"2024-09-10T21:18:53.131780+0000","last_clean":"2024-09-10T21:18:53.131780+0000","last_became_active":"2024-09-10T21:18:53.127317+0000","last_became_peered":"2024-09-10T21:18:53.127317+0000","last_unstale":"2024-09-10T21:18:53.131780+0000","last_undegraded":"2024-09-10T21:18:53.131780+0000","last_fullsized":"2024-09-10T21:18:53.131780+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:48.915819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123286+0000","last_change":"2024-09-10T21:18:53.123286+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123286+0000","last_undegraded":"2024-09-10T21:18:53.123286+0000","last_fullsized":"2024-09-10T21:18:53.123286+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.114087+0000","last_change":"2024-09-10T21:18:52.114087+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.114087+0000","last_undegraded":"2024-09-10T21:18:52.114087+0000","last_fullsized":"2024-09-10T21:18:52.114087+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":313,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-10T21:18:52.113900+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:18:52.113900+0000","last_peered":"2024-09-10T21:18:52.113900+0000","last_clean":"2024-09-10T21:18:52.113900+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:18:52.113900+0000","last_undegraded":"2024-09-10T21:18:52.113900+0000","last_fullsized":"2024-09-10T21:18:52.113900+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":6},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37016,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509548,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37596,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334244,"statfs":{"total":96636764160,"available":96598265856,"internally_reserved":0,"allocated":2297856,"data_stored":1323658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509546,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37016,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509547,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37016,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542250,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41160,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330680,"statfs":{"total":96636764160,"available":96594616320,"internally_reserved":0,"allocated":1691648,"data_stored":727015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542249,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37580,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334260,"statfs":{"total":96636764160,"available":96598282240,"internally_reserved":0,"allocated":2285568,"data_stored":1317383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:18:58.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:18:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.476+0000 7fd868200640 1 -- 172.21.15.16:0/3076293048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8600b0590 msgr2=0x7fd860071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.476+0000 7fd868200640 1 --2- 172.21.15.16:0/3076293048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8600b0590 0x7fd860071520 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fd860073290 tx=0x7fd85801cc60 comp rx=0 tx=0).stop 2024-09-10T21:18:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.476+0000 7fd868200640 1 -- 172.21.15.16:0/3076293048 shutdown_connections 2024-09-10T21:18:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.476+0000 7fd868200640 1 --2- 172.21.15.16:0/3076293048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8600b0590 0x7fd860071520 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.476+0000 7fd868200640 1 --2- 172.21.15.16:0/3076293048 >> v2:172.21.15.52:3300/0 conn(0x7fd86015f400 0x7fd860070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.476+0000 7fd868200640 1 -- 172.21.15.16:0/3076293048 >> 172.21.15.16:0/3076293048 conn(0x7fd86009b130 msgr2=0x7fd8600d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.476+0000 7fd868200640 1 -- 172.21.15.16:0/3076293048 shutdown_connections 2024-09-10T21:18:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.476+0000 7fd868200640 1 -- 172.21.15.16:0/3076293048 wait complete. 2024-09-10T21:18:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd868200640 1 Processor -- start 2024-09-10T21:18:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd868200640 1 -- start start 2024-09-10T21:18:58.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd868200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd8600b0590 0x7fd86015e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:58.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd868200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8600b13c0 0x7fd8601592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:58.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd868200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd86015f400 0x7fd8600cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:58.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd868200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd860168f60 con 0x7fd8600b13c0 2024-09-10T21:18:58.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd868200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd860168de0 con 0x7fd86015f400 2024-09-10T21:18:58.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd868200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd8601690e0 con 0x7fd8600b0590 2024-09-10T21:18:58.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd864e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8600b13c0 0x7fd8601592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd864e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8600b13c0 0x7fd8601592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45496/0 (socket says 172.21.15.16:45496) 2024-09-10T21:18:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd864e00640 1 -- 172.21.15.16:0/3507091948 learned_addr learned my addr 172.21.15.16:0/3507091948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd865800640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd8600b0590 0x7fd86015e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd866200640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd86015f400 0x7fd8600cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd864e00640 1 -- 172.21.15.16:0/3507091948 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd8600b0590 msgr2=0x7fd86015e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd864e00640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd8600b0590 0x7fd86015e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd864e00640 1 -- 172.21.15.16:0/3507091948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd86015f400 msgr2=0x7fd8600cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd864e00640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd86015f400 0x7fd8600cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd864e00640 1 -- 172.21.15.16:0/3507091948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8600b7b60 con 0x7fd8600b13c0 2024-09-10T21:18:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd864e00640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8600b13c0 0x7fd8601592f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c490 tx=0x7fd84c001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd85d600640 1 -- 172.21.15.16:0/3507091948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c017070 con 0x7fd8600b13c0 2024-09-10T21:18:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd85d600640 1 -- 172.21.15.16:0/3507091948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd84c003600 con 0x7fd8600b13c0 2024-09-10T21:18:58.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd85d600640 1 -- 172.21.15.16:0/3507091948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c003a50 con 0x7fd8600b13c0 2024-09-10T21:18:58.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd868200640 1 -- 172.21.15.16:0/3507091948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8601f8060 con 0x7fd8600b13c0 2024-09-10T21:18:58.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd868200640 1 -- 172.21.15.16:0/3507091948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8601f8620 con 0x7fd8600b13c0 2024-09-10T21:18:58.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.480+0000 7fd868200640 1 -- 172.21.15.16:0/3507091948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8600cfc80 con 0x7fd8600b13c0 2024-09-10T21:18:58.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.484+0000 7fd85d600640 1 -- 172.21.15.16:0/3507091948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd84c003bf0 con 0x7fd8600b13c0 2024-09-10T21:18:58.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.484+0000 7fd85d600640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd840072610 0x7fd840074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:58.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.484+0000 7fd85d600640 1 -- 172.21.15.16:0/3507091948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fd84c092390 con 0x7fd8600b13c0 2024-09-10T21:18:58.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.484+0000 7fd865800640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd840072610 0x7fd840074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:58.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.484+0000 7fd865800640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd840072610 0x7fd840074ad0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd8540098e0 tx=0x7fd854009060 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.488+0000 7fd85d600640 1 -- 172.21.15.16:0/3507091948 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd84c05d410 con 0x7fd8600b13c0 2024-09-10T21:18:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.656+0000 7fd868200640 1 -- 172.21.15.16:0/3507091948 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd8600b1d60 con 0x7fd840072610 2024-09-10T21:18:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.660+0000 7fd85d600640 1 -- 172.21.15.16:0/3507091948 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245305 (secure 0 0 0) 0x7fd8600b1d60 con 0x7fd840072610 2024-09-10T21:18:58.669 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:18:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.664+0000 7fd868200640 1 -- 172.21.15.16:0/3507091948 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd840072610 msgr2=0x7fd840074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.664+0000 7fd868200640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd840072610 0x7fd840074ad0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd8540098e0 tx=0x7fd854009060 comp rx=0 tx=0).stop 2024-09-10T21:18:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.664+0000 7fd868200640 1 -- 172.21.15.16:0/3507091948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8600b13c0 msgr2=0x7fd8601592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.664+0000 7fd868200640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8600b13c0 0x7fd8601592f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c490 tx=0x7fd84c001e80 comp rx=0 tx=0).stop 2024-09-10T21:18:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.664+0000 7fd868200640 1 -- 172.21.15.16:0/3507091948 shutdown_connections 2024-09-10T21:18:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.664+0000 7fd868200640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd840072610 0x7fd840074ad0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.664+0000 7fd868200640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd86015f400 0x7fd8600cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.664+0000 7fd868200640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8600b13c0 0x7fd8601592f0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.664+0000 7fd868200640 1 --2- 172.21.15.16:0/3507091948 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd8600b0590 0x7fd86015e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.664+0000 7fd868200640 1 -- 172.21.15.16:0/3507091948 >> 172.21.15.16:0/3507091948 conn(0x7fd86009b130 msgr2=0x7fd86006dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.664+0000 7fd868200640 1 -- 172.21.15.16:0/3507091948 shutdown_connections 2024-09-10T21:18:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.664+0000 7fd868200640 1 -- 172.21.15.16:0/3507091948 wait complete. 2024-09-10T21:18:58.691 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-10T21:18:57.575293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":166,"acting":166,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":293096,"kb_used_data":14448,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754681624,"statfs":{"total":773094113280,"available":772793982976,"internally_reserved":0,"allocated":14794752,"data_stored":7034506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.472130"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118441+0000","last_change":"2024-09-10T21:18:53.118441+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118441+0000","last_undegraded":"2024-09-10T21:18:53.118441+0000","last_fullsized":"2024-09-10T21:18:53.118441+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117949+0000","last_change":"2024-09-10T21:18:53.117949+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117949+0000","last_undegraded":"2024-09-10T21:18:53.117949+0000","last_fullsized":"2024-09-10T21:18:53.117949+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118237+0000","last_change":"2024-09-10T21:18:53.118237+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118237+0000","last_undegraded":"2024-09-10T21:18:53.118237+0000","last_fullsized":"2024-09-10T21:18:53.118237+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118754+0000","last_change":"2024-09-10T21:18:53.118754+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118754+0000","last_undegraded":"2024-09-10T21:18:53.118754+0000","last_fullsized":"2024-09-10T21:18:53.118754+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121298+0000","last_change":"2024-09-10T21:18:53.121298+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121298+0000","last_undegraded":"2024-09-10T21:18:53.121298+0000","last_fullsized":"2024-09-10T21:18:53.121298+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124732+0000","last_change":"2024-09-10T21:18:53.124732+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124732+0000","last_undegraded":"2024-09-10T21:18:53.124732+0000","last_fullsized":"2024-09-10T21:18:53.124732+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124460+0000","last_change":"2024-09-10T21:18:53.124460+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124460+0000","last_undegraded":"2024-09-10T21:18:53.124460+0000","last_fullsized":"2024-09-10T21:18:53.124460+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123934+0000","last_change":"2024-09-10T21:18:53.123934+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123934+0000","last_undegraded":"2024-09-10T21:18:53.123934+0000","last_fullsized":"2024-09-10T21:18:53.123934+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117758+0000","last_change":"2024-09-10T21:18:53.117758+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117758+0000","last_undegraded":"2024-09-10T21:18:53.117758+0000","last_fullsized":"2024-09-10T21:18:53.117758+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117647+0000","last_change":"2024-09-10T21:18:53.117647+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117647+0000","last_undegraded":"2024-09-10T21:18:53.117647+0000","last_fullsized":"2024-09-10T21:18:53.117647+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124781+0000","last_change":"2024-09-10T21:18:53.124781+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124781+0000","last_undegraded":"2024-09-10T21:18:53.124781+0000","last_fullsized":"2024-09-10T21:18:53.124781+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123669+0000","last_change":"2024-09-10T21:18:53.123669+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123669+0000","last_undegraded":"2024-09-10T21:18:53.123669+0000","last_fullsized":"2024-09-10T21:18:53.123669+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123065+0000","last_change":"2024-09-10T21:18:53.123065+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123065+0000","last_undegraded":"2024-09-10T21:18:53.123065+0000","last_fullsized":"2024-09-10T21:18:53.123065+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.115101+0000","last_change":"2024-09-10T21:18:53.115101+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.115101+0000","last_undegraded":"2024-09-10T21:18:53.115101+0000","last_fullsized":"2024-09-10T21:18:53.115101+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124613+0000","last_change":"2024-09-10T21:18:53.124613+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124613+0000","last_undegraded":"2024-09-10T21:18:53.124613+0000","last_fullsized":"2024-09-10T21:18:53.124613+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117230+0000","last_change":"2024-09-10T21:18:53.117230+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117230+0000","last_undegraded":"2024-09-10T21:18:53.117230+0000","last_fullsized":"2024-09-10T21:18:53.117230+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.115573+0000","last_change":"2024-09-10T21:18:53.115573+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.115573+0000","last_undegraded":"2024-09-10T21:18:53.115573+0000","last_fullsized":"2024-09-10T21:18:53.115573+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118327+0000","last_change":"2024-09-10T21:18:53.118327+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118327+0000","last_undegraded":"2024-09-10T21:18:53.118327+0000","last_fullsized":"2024-09-10T21:18:53.118327+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.122414+0000","last_change":"2024-09-10T21:18:53.122414+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.122414+0000","last_undegraded":"2024-09-10T21:18:53.122414+0000","last_fullsized":"2024-09-10T21:18:53.122414+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117885+0000","last_change":"2024-09-10T21:18:53.117885+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117885+0000","last_undegraded":"2024-09-10T21:18:53.117885+0000","last_fullsized":"2024-09-10T21:18:53.117885+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124230+0000","last_change":"2024-09-10T21:18:53.124230+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124230+0000","last_undegraded":"2024-09-10T21:18:53.124230+0000","last_fullsized":"2024-09-10T21:18:53.124230+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118139+0000","last_change":"2024-09-10T21:18:53.118139+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118139+0000","last_undegraded":"2024-09-10T21:18:53.118139+0000","last_fullsized":"2024-09-10T21:18:53.118139+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114559+0000","last_change":"2024-09-10T21:18:53.114559+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114559+0000","last_undegraded":"2024-09-10T21:18:53.114559+0000","last_fullsized":"2024-09-10T21:18:53.114559+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.130551+0000","last_change":"2024-09-10T21:18:53.122058+0000","last_active":"2024-09-10T21:18:53.130551+0000","last_peered":"2024-09-10T21:18:53.130551+0000","last_clean":"2024-09-10T21:18:53.130551+0000","last_became_active":"2024-09-10T21:18:53.121768+0000","last_became_peered":"2024-09-10T21:18:53.121768+0000","last_unstale":"2024-09-10T21:18:53.130551+0000","last_undegraded":"2024-09-10T21:18:53.130551+0000","last_fullsized":"2024-09-10T21:18:53.130551+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:46.151037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116889+0000","last_change":"2024-09-10T21:18:53.116889+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116889+0000","last_undegraded":"2024-09-10T21:18:53.116889+0000","last_fullsized":"2024-09-10T21:18:53.116889+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.140097+0000","last_change":"2024-09-10T21:18:53.134253+0000","last_active":"2024-09-10T21:18:53.140097+0000","last_peered":"2024-09-10T21:18:53.140097+0000","last_clean":"2024-09-10T21:18:53.140097+0000","last_became_active":"2024-09-10T21:18:53.133971+0000","last_became_peered":"2024-09-10T21:18:53.133971+0000","last_unstale":"2024-09-10T21:18:53.140097+0000","last_undegraded":"2024-09-10T21:18:53.140097+0000","last_fullsized":"2024-09-10T21:18:53.140097+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:05.811497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.104469+0000","last_change":"2024-09-10T21:18:52.104469+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.104469+0000","last_undegraded":"2024-09-10T21:18:52.104469+0000","last_fullsized":"2024-09-10T21:18:52.104469+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123461+0000","last_change":"2024-09-10T21:18:53.123461+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123461+0000","last_undegraded":"2024-09-10T21:18:53.123461+0000","last_fullsized":"2024-09-10T21:18:53.123461+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.134759+0000","last_change":"2024-09-10T21:18:53.132095+0000","last_active":"2024-09-10T21:18:53.134759+0000","last_peered":"2024-09-10T21:18:53.134759+0000","last_clean":"2024-09-10T21:18:53.134759+0000","last_became_active":"2024-09-10T21:18:53.131751+0000","last_became_peered":"2024-09-10T21:18:53.131751+0000","last_unstale":"2024-09-10T21:18:53.134759+0000","last_undegraded":"2024-09-10T21:18:53.134759+0000","last_fullsized":"2024-09-10T21:18:53.134759+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:03.376294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116079+0000","last_change":"2024-09-10T21:18:53.116079+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116079+0000","last_undegraded":"2024-09-10T21:18:53.116079+0000","last_fullsized":"2024-09-10T21:18:53.116079+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.138579+0000","last_change":"2024-09-10T21:18:53.135048+0000","last_active":"2024-09-10T21:18:53.138579+0000","last_peered":"2024-09-10T21:18:53.138579+0000","last_clean":"2024-09-10T21:18:53.138579+0000","last_became_active":"2024-09-10T21:18:53.134366+0000","last_became_peered":"2024-09-10T21:18:53.134366+0000","last_unstale":"2024-09-10T21:18:53.138579+0000","last_undegraded":"2024-09-10T21:18:53.138579+0000","last_fullsized":"2024-09-10T21:18:53.138579+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:53.969257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.132642+0000","last_change":"2024-09-10T21:18:53.130356+0000","last_active":"2024-09-10T21:18:53.132642+0000","last_peered":"2024-09-10T21:18:53.132642+0000","last_clean":"2024-09-10T21:18:53.132642+0000","last_became_active":"2024-09-10T21:18:53.130043+0000","last_became_peered":"2024-09-10T21:18:53.130043+0000","last_unstale":"2024-09-10T21:18:53.132642+0000","last_undegraded":"2024-09-10T21:18:53.132642+0000","last_fullsized":"2024-09-10T21:18:53.132642+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:02.691227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114902+0000","last_change":"2024-09-10T21:18:53.114902+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114902+0000","last_undegraded":"2024-09-10T21:18:53.114902+0000","last_fullsized":"2024-09-10T21:18:53.114902+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.134829+0000","last_change":"2024-09-10T21:18:53.129464+0000","last_active":"2024-09-10T21:18:53.134829+0000","last_peered":"2024-09-10T21:18:53.134829+0000","last_clean":"2024-09-10T21:18:53.134829+0000","last_became_active":"2024-09-10T21:18:53.129179+0000","last_became_peered":"2024-09-10T21:18:53.129179+0000","last_unstale":"2024-09-10T21:18:53.134829+0000","last_undegraded":"2024-09-10T21:18:53.134829+0000","last_fullsized":"2024-09-10T21:18:53.134829+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:10.823382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114044+0000","last_change":"2024-09-10T21:18:53.114044+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114044+0000","last_undegraded":"2024-09-10T21:18:53.114044+0000","last_fullsized":"2024-09-10T21:18:53.114044+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.132295+0000","last_change":"2024-09-10T21:18:53.127488+0000","last_active":"2024-09-10T21:18:53.132295+0000","last_peered":"2024-09-10T21:18:53.132295+0000","last_clean":"2024-09-10T21:18:53.132295+0000","last_became_active":"2024-09-10T21:18:53.125698+0000","last_became_peered":"2024-09-10T21:18:53.125698+0000","last_unstale":"2024-09-10T21:18:53.132295+0000","last_undegraded":"2024-09-10T21:18:53.132295+0000","last_fullsized":"2024-09-10T21:18:53.132295+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:47.324904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121885+0000","last_change":"2024-09-10T21:18:53.121885+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121885+0000","last_undegraded":"2024-09-10T21:18:53.121885+0000","last_fullsized":"2024-09-10T21:18:53.121885+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.104899+0000","last_change":"2024-09-10T21:18:52.104899+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.104899+0000","last_undegraded":"2024-09-10T21:18:52.104899+0000","last_fullsized":"2024-09-10T21:18:52.104899+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121944+0000","last_change":"2024-09-10T21:18:53.121944+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121944+0000","last_undegraded":"2024-09-10T21:18:53.121944+0000","last_fullsized":"2024-09-10T21:18:53.121944+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.136282+0000","last_change":"2024-09-10T21:18:53.134405+0000","last_active":"2024-09-10T21:18:53.136282+0000","last_peered":"2024-09-10T21:18:53.136282+0000","last_clean":"2024-09-10T21:18:53.136282+0000","last_became_active":"2024-09-10T21:18:53.134012+0000","last_became_peered":"2024-09-10T21:18:53.134012+0000","last_unstale":"2024-09-10T21:18:53.136282+0000","last_undegraded":"2024-09-10T21:18:53.136282+0000","last_fullsized":"2024-09-10T21:18:53.136282+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:09.018998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116619+0000","last_change":"2024-09-10T21:18:53.116619+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116619+0000","last_undegraded":"2024-09-10T21:18:53.116619+0000","last_fullsized":"2024-09-10T21:18:53.116619+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111870+0000","last_change":"2024-09-10T21:18:52.111870+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111870+0000","last_undegraded":"2024-09-10T21:18:52.111870+0000","last_fullsized":"2024-09-10T21:18:52.111870+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121517+0000","last_change":"2024-09-10T21:18:53.121517+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121517+0000","last_undegraded":"2024-09-10T21:18:53.121517+0000","last_fullsized":"2024-09-10T21:18:53.121517+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.131783+0000","last_change":"2024-09-10T21:18:53.127007+0000","last_active":"2024-09-10T21:18:53.131783+0000","last_peered":"2024-09-10T21:18:53.131783+0000","last_clean":"2024-09-10T21:18:53.131783+0000","last_became_active":"2024-09-10T21:18:53.126673+0000","last_became_peered":"2024-09-10T21:18:53.126673+0000","last_unstale":"2024-09-10T21:18:53.131783+0000","last_undegraded":"2024-09-10T21:18:53.131783+0000","last_fullsized":"2024-09-10T21:18:53.131783+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:39.746760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116547+0000","last_change":"2024-09-10T21:18:53.116547+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116547+0000","last_undegraded":"2024-09-10T21:18:53.116547+0000","last_fullsized":"2024-09-10T21:18:53.116547+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.104198+0000","last_change":"2024-09-10T21:18:52.104198+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.104198+0000","last_undegraded":"2024-09-10T21:18:52.104198+0000","last_fullsized":"2024-09-10T21:18:52.104198+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.138049+0000","last_change":"2024-09-10T21:18:53.137417+0000","last_active":"2024-09-10T21:18:53.138049+0000","last_peered":"2024-09-10T21:18:53.138049+0000","last_clean":"2024-09-10T21:18:53.138049+0000","last_became_active":"2024-09-10T21:18:53.137042+0000","last_became_peered":"2024-09-10T21:18:53.137042+0000","last_unstale":"2024-09-10T21:18:53.138049+0000","last_undegraded":"2024-09-10T21:18:53.138049+0000","last_fullsized":"2024-09-10T21:18:53.138049+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:45.885394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123123+0000","last_change":"2024-09-10T21:18:53.123123+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123123+0000","last_undegraded":"2024-09-10T21:18:53.123123+0000","last_fullsized":"2024-09-10T21:18:53.123123+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111728+0000","last_change":"2024-09-10T21:18:52.111728+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111728+0000","last_undegraded":"2024-09-10T21:18:52.111728+0000","last_fullsized":"2024-09-10T21:18:52.111728+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.122766+0000","last_change":"2024-09-10T21:18:53.122766+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.122766+0000","last_undegraded":"2024-09-10T21:18:53.122766+0000","last_fullsized":"2024-09-10T21:18:53.122766+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111758+0000","last_change":"2024-09-10T21:18:52.111758+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111758+0000","last_undegraded":"2024-09-10T21:18:52.111758+0000","last_fullsized":"2024-09-10T21:18:52.111758+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117371+0000","last_change":"2024-09-10T21:18:53.117371+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117371+0000","last_undegraded":"2024-09-10T21:18:53.117371+0000","last_fullsized":"2024-09-10T21:18:53.117371+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.140124+0000","last_change":"2024-09-10T21:18:53.136538+0000","last_active":"2024-09-10T21:18:53.140124+0000","last_peered":"2024-09-10T21:18:53.140124+0000","last_clean":"2024-09-10T21:18:53.140124+0000","last_became_active":"2024-09-10T21:18:53.135006+0000","last_became_peered":"2024-09-10T21:18:53.135006+0000","last_unstale":"2024-09-10T21:18:53.140124+0000","last_undegraded":"2024-09-10T21:18:53.140124+0000","last_fullsized":"2024-09-10T21:18:53.140124+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:58.524531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117304+0000","last_change":"2024-09-10T21:18:53.117304+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117304+0000","last_undegraded":"2024-09-10T21:18:53.117304+0000","last_fullsized":"2024-09-10T21:18:53.117304+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111997+0000","last_change":"2024-09-10T21:18:52.111997+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111997+0000","last_undegraded":"2024-09-10T21:18:52.111997+0000","last_fullsized":"2024-09-10T21:18:52.111997+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.115916+0000","last_change":"2024-09-10T21:18:53.115916+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.115916+0000","last_undegraded":"2024-09-10T21:18:53.115916+0000","last_fullsized":"2024-09-10T21:18:53.115916+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.133308+0000","last_change":"2024-09-10T21:18:53.126832+0000","last_active":"2024-09-10T21:18:53.133308+0000","last_peered":"2024-09-10T21:18:53.133308+0000","last_clean":"2024-09-10T21:18:53.133308+0000","last_became_active":"2024-09-10T21:18:53.126580+0000","last_became_peered":"2024-09-10T21:18:53.126580+0000","last_unstale":"2024-09-10T21:18:53.133308+0000","last_undegraded":"2024-09-10T21:18:53.133308+0000","last_fullsized":"2024-09-10T21:18:53.133308+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:42.773987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116655+0000","last_change":"2024-09-10T21:18:53.116655+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116655+0000","last_undegraded":"2024-09-10T21:18:53.116655+0000","last_fullsized":"2024-09-10T21:18:53.116655+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.134062+0000","last_change":"2024-09-10T21:18:53.129128+0000","last_active":"2024-09-10T21:18:53.134062+0000","last_peered":"2024-09-10T21:18:53.134062+0000","last_clean":"2024-09-10T21:18:53.134062+0000","last_became_active":"2024-09-10T21:18:53.128826+0000","last_became_peered":"2024-09-10T21:18:53.128826+0000","last_unstale":"2024-09-10T21:18:53.134062+0000","last_undegraded":"2024-09-10T21:18:53.134062+0000","last_fullsized":"2024-09-10T21:18:53.134062+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:14.312782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116046+0000","last_change":"2024-09-10T21:18:53.116046+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116046+0000","last_undegraded":"2024-09-10T21:18:53.116046+0000","last_fullsized":"2024-09-10T21:18:53.116046+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.133416+0000","last_change":"2024-09-10T21:18:53.128814+0000","last_active":"2024-09-10T21:18:53.133416+0000","last_peered":"2024-09-10T21:18:53.133416+0000","last_clean":"2024-09-10T21:18:53.133416+0000","last_became_active":"2024-09-10T21:18:53.128474+0000","last_became_peered":"2024-09-10T21:18:53.128474+0000","last_unstale":"2024-09-10T21:18:53.133416+0000","last_undegraded":"2024-09-10T21:18:53.133416+0000","last_fullsized":"2024-09-10T21:18:53.133416+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:50.343267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116419+0000","last_change":"2024-09-10T21:18:53.116419+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116419+0000","last_undegraded":"2024-09-10T21:18:53.116419+0000","last_fullsized":"2024-09-10T21:18:53.116419+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.135855+0000","last_change":"2024-09-10T21:18:53.132527+0000","last_active":"2024-09-10T21:18:53.135855+0000","last_peered":"2024-09-10T21:18:53.135855+0000","last_clean":"2024-09-10T21:18:53.135855+0000","last_became_active":"2024-09-10T21:18:53.130251+0000","last_became_peered":"2024-09-10T21:18:53.130251+0000","last_unstale":"2024-09-10T21:18:53.135855+0000","last_undegraded":"2024-09-10T21:18:53.135855+0000","last_fullsized":"2024-09-10T21:18:53.135855+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:12.376207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123636+0000","last_change":"2024-09-10T21:18:53.123636+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123636+0000","last_undegraded":"2024-09-10T21:18:53.123636+0000","last_fullsized":"2024-09-10T21:18:53.123636+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.137844+0000","last_change":"2024-09-10T21:18:53.133231+0000","last_active":"2024-09-10T21:18:53.137844+0000","last_peered":"2024-09-10T21:18:53.137844+0000","last_clean":"2024-09-10T21:18:53.137844+0000","last_became_active":"2024-09-10T21:18:53.132930+0000","last_became_peered":"2024-09-10T21:18:53.132930+0000","last_unstale":"2024-09-10T21:18:53.137844+0000","last_undegraded":"2024-09-10T21:18:53.137844+0000","last_fullsized":"2024-09-10T21:18:53.137844+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:26.790852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.115124+0000","last_change":"2024-09-10T21:18:53.115124+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.115124+0000","last_undegraded":"2024-09-10T21:18:53.115124+0000","last_fullsized":"2024-09-10T21:18:53.115124+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.105072+0000","last_change":"2024-09-10T21:18:52.105072+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.105072+0000","last_undegraded":"2024-09-10T21:18:52.105072+0000","last_fullsized":"2024-09-10T21:18:52.105072+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116808+0000","last_change":"2024-09-10T21:18:53.116808+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116808+0000","last_undegraded":"2024-09-10T21:18:53.116808+0000","last_fullsized":"2024-09-10T21:18:53.116808+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.134459+0000","last_change":"2024-09-10T21:18:53.130476+0000","last_active":"2024-09-10T21:18:53.134459+0000","last_peered":"2024-09-10T21:18:53.134459+0000","last_clean":"2024-09-10T21:18:53.134459+0000","last_became_active":"2024-09-10T21:18:53.130213+0000","last_became_peered":"2024-09-10T21:18:53.130213+0000","last_unstale":"2024-09-10T21:18:53.134459+0000","last_undegraded":"2024-09-10T21:18:53.134459+0000","last_fullsized":"2024-09-10T21:18:53.134459+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:40.694947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114244+0000","last_change":"2024-09-10T21:18:53.114244+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114244+0000","last_undegraded":"2024-09-10T21:18:53.114244+0000","last_fullsized":"2024-09-10T21:18:53.114244+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.135183+0000","last_change":"2024-09-10T21:18:53.130268+0000","last_active":"2024-09-10T21:18:53.135183+0000","last_peered":"2024-09-10T21:18:53.135183+0000","last_clean":"2024-09-10T21:18:53.135183+0000","last_became_active":"2024-09-10T21:18:53.129955+0000","last_became_peered":"2024-09-10T21:18:53.129955+0000","last_unstale":"2024-09-10T21:18:53.135183+0000","last_undegraded":"2024-09-10T21:18:53.135183+0000","last_fullsized":"2024-09-10T21:18:53.135183+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:03.568523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114041+0000","last_change":"2024-09-10T21:18:53.114041+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114041+0000","last_undegraded":"2024-09-10T21:18:53.114041+0000","last_fullsized":"2024-09-10T21:18:53.114041+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.135542+0000","last_change":"2024-09-10T21:18:53.133483+0000","last_active":"2024-09-10T21:18:53.135542+0000","last_peered":"2024-09-10T21:18:53.135542+0000","last_clean":"2024-09-10T21:18:53.135542+0000","last_became_active":"2024-09-10T21:18:53.133196+0000","last_became_peered":"2024-09-10T21:18:53.133196+0000","last_unstale":"2024-09-10T21:18:53.135542+0000","last_undegraded":"2024-09-10T21:18:53.135542+0000","last_fullsized":"2024-09-10T21:18:53.135542+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:42:00.777546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111911+0000","last_change":"2024-09-10T21:18:52.111911+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111911+0000","last_undegraded":"2024-09-10T21:18:52.111911+0000","last_fullsized":"2024-09-10T21:18:52.111911+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116699+0000","last_change":"2024-09-10T21:18:53.116699+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116699+0000","last_undegraded":"2024-09-10T21:18:53.116699+0000","last_fullsized":"2024-09-10T21:18:53.116699+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.135056+0000","last_change":"2024-09-10T21:18:53.133724+0000","last_active":"2024-09-10T21:18:53.135056+0000","last_peered":"2024-09-10T21:18:53.135056+0000","last_clean":"2024-09-10T21:18:53.135056+0000","last_became_active":"2024-09-10T21:18:53.133442+0000","last_became_peered":"2024-09-10T21:18:53.133442+0000","last_unstale":"2024-09-10T21:18:53.135056+0000","last_undegraded":"2024-09-10T21:18:53.135056+0000","last_fullsized":"2024-09-10T21:18:53.135056+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:12.220728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121788+0000","last_change":"2024-09-10T21:18:53.121788+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121788+0000","last_undegraded":"2024-09-10T21:18:53.121788+0000","last_fullsized":"2024-09-10T21:18:53.121788+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.136783+0000","last_change":"2024-09-10T21:18:53.132411+0000","last_active":"2024-09-10T21:18:53.136783+0000","last_peered":"2024-09-10T21:18:53.136783+0000","last_clean":"2024-09-10T21:18:53.136783+0000","last_became_active":"2024-09-10T21:18:53.131949+0000","last_became_peered":"2024-09-10T21:18:53.131949+0000","last_unstale":"2024-09-10T21:18:53.136783+0000","last_undegraded":"2024-09-10T21:18:53.136783+0000","last_fullsized":"2024-09-10T21:18:53.136783+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:59.498416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116457+0000","last_change":"2024-09-10T21:18:53.116457+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116457+0000","last_undegraded":"2024-09-10T21:18:53.116457+0000","last_fullsized":"2024-09-10T21:18:53.116457+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.131780+0000","last_change":"2024-09-10T21:18:53.127615+0000","last_active":"2024-09-10T21:18:53.131780+0000","last_peered":"2024-09-10T21:18:53.131780+0000","last_clean":"2024-09-10T21:18:53.131780+0000","last_became_active":"2024-09-10T21:18:53.127317+0000","last_became_peered":"2024-09-10T21:18:53.127317+0000","last_unstale":"2024-09-10T21:18:53.131780+0000","last_undegraded":"2024-09-10T21:18:53.131780+0000","last_fullsized":"2024-09-10T21:18:53.131780+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:48.915819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123286+0000","last_change":"2024-09-10T21:18:53.123286+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123286+0000","last_undegraded":"2024-09-10T21:18:53.123286+0000","last_fullsized":"2024-09-10T21:18:53.123286+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.114087+0000","last_change":"2024-09-10T21:18:52.114087+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.114087+0000","last_undegraded":"2024-09-10T21:18:52.114087+0000","last_fullsized":"2024-09-10T21:18:52.114087+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":313,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-10T21:18:52.113900+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:18:52.113900+0000","last_peered":"2024-09-10T21:18:52.113900+0000","last_clean":"2024-09-10T21:18:52.113900+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:18:52.113900+0000","last_undegraded":"2024-09-10T21:18:52.113900+0000","last_fullsized":"2024-09-10T21:18:52.113900+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":6},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37016,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509548,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37596,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334244,"statfs":{"total":96636764160,"available":96598265856,"internally_reserved":0,"allocated":2297856,"data_stored":1323658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509546,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37016,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509547,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37016,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542250,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41160,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330680,"statfs":{"total":96636764160,"available":96594616320,"internally_reserved":0,"allocated":1691648,"data_stored":727015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542249,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37580,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334260,"statfs":{"total":96636764160,"available":96598282240,"internally_reserved":0,"allocated":2285568,"data_stored":1317383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:18:58.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:18:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.876+0000 7efd27e00640 1 -- 172.21.15.16:0/4289588410 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd28161590 msgr2=0x7efd28068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.876+0000 7efd27e00640 1 --2- 172.21.15.16:0/4289588410 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd28161590 0x7efd28068c70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efd18009870 tx=0x7efd1801c6a0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.876+0000 7efd27e00640 1 -- 172.21.15.16:0/4289588410 shutdown_connections 2024-09-10T21:18:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.876+0000 7efd27e00640 1 --2- 172.21.15.16:0/4289588410 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd28161590 0x7efd28068c70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.876+0000 7efd27e00640 1 -- 172.21.15.16:0/4289588410 >> 172.21.15.16:0/4289588410 conn(0x7efd280b1d40 msgr2=0x7efd280b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.876+0000 7efd27e00640 1 -- 172.21.15.16:0/4289588410 shutdown_connections 2024-09-10T21:18:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.876+0000 7efd27e00640 1 -- 172.21.15.16:0/4289588410 wait complete. 2024-09-10T21:18:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd27e00640 1 Processor -- start 2024-09-10T21:18:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd27e00640 1 -- start start 2024-09-10T21:18:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd27e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd28074860 0x7efd281560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd27e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cc4f0 0x7efd2815b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd27e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd28161590 0x7efd28158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd27e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd2816c320 con 0x7efd280cc4f0 2024-09-10T21:18:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd27e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efd2816c1a0 con 0x7efd28161590 2024-09-10T21:18:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd27e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7efd2816c4a0 con 0x7efd28074860 2024-09-10T21:18:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd26000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cc4f0 0x7efd2815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd26000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cc4f0 0x7efd2815b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45508/0 (socket says 172.21.15.16:45508) 2024-09-10T21:18:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd26000640 1 -- 172.21.15.16:0/4084561929 learned_addr learned my addr 172.21.15.16:0/4084561929 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd26a00640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd28074860 0x7efd281560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd27400640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd28161590 0x7efd28158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd26000640 1 -- 172.21.15.16:0/4084561929 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd28074860 msgr2=0x7efd281560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd26000640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd28074860 0x7efd281560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd26000640 1 -- 172.21.15.16:0/4084561929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd28161590 msgr2=0x7efd28158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd26000640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd28161590 0x7efd28158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd26000640 1 -- 172.21.15.16:0/4084561929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd28156ab0 con 0x7efd280cc4f0 2024-09-10T21:18:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd27400640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd28161590 0x7efd28158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd26000640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cc4f0 0x7efd2815b160 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7efd1400caa0 tx=0x7efd1400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd1ea00640 1 -- 172.21.15.16:0/4084561929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14016020 con 0x7efd280cc4f0 2024-09-10T21:18:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd1ea00640 1 -- 172.21.15.16:0/4084561929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd1400ab10 con 0x7efd280cc4f0 2024-09-10T21:18:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd1ea00640 1 -- 172.21.15.16:0/4084561929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14012420 con 0x7efd280cc4f0 2024-09-10T21:18:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd27e00640 1 -- 172.21.15.16:0/4084561929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd28156d90 con 0x7efd280cc4f0 2024-09-10T21:18:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd27e00640 1 -- 172.21.15.16:0/4084561929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2815ac40 con 0x7efd280cc4f0 2024-09-10T21:18:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.880+0000 7efd27e00640 1 -- 172.21.15.16:0/4084561929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd2815ae40 con 0x7efd280cc4f0 2024-09-10T21:18:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.884+0000 7efd1ea00640 1 -- 172.21.15.16:0/4084561929 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efd140125c0 con 0x7efd280cc4f0 2024-09-10T21:18:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.884+0000 7efd1ea00640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd080725e0 0x7efd08074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.884+0000 7efd1ea00640 1 -- 172.21.15.16:0/4084561929 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7efd140937f0 con 0x7efd280cc4f0 2024-09-10T21:18:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.884+0000 7efd26a00640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd080725e0 0x7efd08074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.884+0000 7efd26a00640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd080725e0 0x7efd08074aa0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7efd180ae000 tx=0x7efd180aa040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:58.888+0000 7efd1ea00640 1 -- 172.21.15.16:0/4084561929 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efd2815ae40 con 0x7efd280cc4f0 2024-09-10T21:18:59.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.064+0000 7efd27e00640 1 -- 172.21.15.16:0/4084561929 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd28156f60 con 0x7efd080725e0 2024-09-10T21:18:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.068+0000 7efd1ea00640 1 -- 172.21.15.16:0/4084561929 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245305 (secure 0 0 0) 0x7efd28156f60 con 0x7efd080725e0 2024-09-10T21:18:59.074 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:18:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.072+0000 7efd27e00640 1 -- 172.21.15.16:0/4084561929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd080725e0 msgr2=0x7efd08074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.072+0000 7efd27e00640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd080725e0 0x7efd08074aa0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7efd180ae000 tx=0x7efd180aa040 comp rx=0 tx=0).stop 2024-09-10T21:18:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.072+0000 7efd27e00640 1 -- 172.21.15.16:0/4084561929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cc4f0 msgr2=0x7efd2815b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.072+0000 7efd27e00640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cc4f0 0x7efd2815b160 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7efd1400caa0 tx=0x7efd1400cf60 comp rx=0 tx=0).stop 2024-09-10T21:18:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.072+0000 7efd27e00640 1 -- 172.21.15.16:0/4084561929 shutdown_connections 2024-09-10T21:18:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.072+0000 7efd27e00640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd080725e0 0x7efd08074aa0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.072+0000 7efd27e00640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd28161590 0x7efd28158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.072+0000 7efd27e00640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd280cc4f0 0x7efd2815b160 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.072+0000 7efd27e00640 1 --2- 172.21.15.16:0/4084561929 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd28074860 0x7efd281560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.072+0000 7efd27e00640 1 -- 172.21.15.16:0/4084561929 >> 172.21.15.16:0/4084561929 conn(0x7efd280b1d40 msgr2=0x7efd2815ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.072+0000 7efd27e00640 1 -- 172.21.15.16:0/4084561929 shutdown_connections 2024-09-10T21:18:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.072+0000 7efd27e00640 1 -- 172.21.15.16:0/4084561929 wait complete. 2024-09-10T21:18:59.096 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-10T21:18:57.575293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":166,"acting":166,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":293096,"kb_used_data":14448,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754681624,"statfs":{"total":773094113280,"available":772793982976,"internally_reserved":0,"allocated":14794752,"data_stored":7034506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":285134384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.472130"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118441+0000","last_change":"2024-09-10T21:18:53.118441+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118441+0000","last_undegraded":"2024-09-10T21:18:53.118441+0000","last_fullsized":"2024-09-10T21:18:53.118441+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117949+0000","last_change":"2024-09-10T21:18:53.117949+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117949+0000","last_undegraded":"2024-09-10T21:18:53.117949+0000","last_fullsized":"2024-09-10T21:18:53.117949+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118237+0000","last_change":"2024-09-10T21:18:53.118237+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118237+0000","last_undegraded":"2024-09-10T21:18:53.118237+0000","last_fullsized":"2024-09-10T21:18:53.118237+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118754+0000","last_change":"2024-09-10T21:18:53.118754+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118754+0000","last_undegraded":"2024-09-10T21:18:53.118754+0000","last_fullsized":"2024-09-10T21:18:53.118754+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121298+0000","last_change":"2024-09-10T21:18:53.121298+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121298+0000","last_undegraded":"2024-09-10T21:18:53.121298+0000","last_fullsized":"2024-09-10T21:18:53.121298+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124732+0000","last_change":"2024-09-10T21:18:53.124732+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124732+0000","last_undegraded":"2024-09-10T21:18:53.124732+0000","last_fullsized":"2024-09-10T21:18:53.124732+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124460+0000","last_change":"2024-09-10T21:18:53.124460+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124460+0000","last_undegraded":"2024-09-10T21:18:53.124460+0000","last_fullsized":"2024-09-10T21:18:53.124460+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123934+0000","last_change":"2024-09-10T21:18:53.123934+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123934+0000","last_undegraded":"2024-09-10T21:18:53.123934+0000","last_fullsized":"2024-09-10T21:18:53.123934+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117758+0000","last_change":"2024-09-10T21:18:53.117758+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117758+0000","last_undegraded":"2024-09-10T21:18:53.117758+0000","last_fullsized":"2024-09-10T21:18:53.117758+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117647+0000","last_change":"2024-09-10T21:18:53.117647+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117647+0000","last_undegraded":"2024-09-10T21:18:53.117647+0000","last_fullsized":"2024-09-10T21:18:53.117647+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124781+0000","last_change":"2024-09-10T21:18:53.124781+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124781+0000","last_undegraded":"2024-09-10T21:18:53.124781+0000","last_fullsized":"2024-09-10T21:18:53.124781+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123669+0000","last_change":"2024-09-10T21:18:53.123669+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123669+0000","last_undegraded":"2024-09-10T21:18:53.123669+0000","last_fullsized":"2024-09-10T21:18:53.123669+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123065+0000","last_change":"2024-09-10T21:18:53.123065+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123065+0000","last_undegraded":"2024-09-10T21:18:53.123065+0000","last_fullsized":"2024-09-10T21:18:53.123065+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.115101+0000","last_change":"2024-09-10T21:18:53.115101+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.115101+0000","last_undegraded":"2024-09-10T21:18:53.115101+0000","last_fullsized":"2024-09-10T21:18:53.115101+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124613+0000","last_change":"2024-09-10T21:18:53.124613+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124613+0000","last_undegraded":"2024-09-10T21:18:53.124613+0000","last_fullsized":"2024-09-10T21:18:53.124613+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117230+0000","last_change":"2024-09-10T21:18:53.117230+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117230+0000","last_undegraded":"2024-09-10T21:18:53.117230+0000","last_fullsized":"2024-09-10T21:18:53.117230+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.115573+0000","last_change":"2024-09-10T21:18:53.115573+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.115573+0000","last_undegraded":"2024-09-10T21:18:53.115573+0000","last_fullsized":"2024-09-10T21:18:53.115573+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118327+0000","last_change":"2024-09-10T21:18:53.118327+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118327+0000","last_undegraded":"2024-09-10T21:18:53.118327+0000","last_fullsized":"2024-09-10T21:18:53.118327+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.122414+0000","last_change":"2024-09-10T21:18:53.122414+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.122414+0000","last_undegraded":"2024-09-10T21:18:53.122414+0000","last_fullsized":"2024-09-10T21:18:53.122414+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117885+0000","last_change":"2024-09-10T21:18:53.117885+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117885+0000","last_undegraded":"2024-09-10T21:18:53.117885+0000","last_fullsized":"2024-09-10T21:18:53.117885+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.124230+0000","last_change":"2024-09-10T21:18:53.124230+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.124230+0000","last_undegraded":"2024-09-10T21:18:53.124230+0000","last_fullsized":"2024-09-10T21:18:53.124230+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.118139+0000","last_change":"2024-09-10T21:18:53.118139+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.118139+0000","last_undegraded":"2024-09-10T21:18:53.118139+0000","last_fullsized":"2024-09-10T21:18:53.118139+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114559+0000","last_change":"2024-09-10T21:18:53.114559+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114559+0000","last_undegraded":"2024-09-10T21:18:53.114559+0000","last_fullsized":"2024-09-10T21:18:53.114559+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.130551+0000","last_change":"2024-09-10T21:18:53.122058+0000","last_active":"2024-09-10T21:18:53.130551+0000","last_peered":"2024-09-10T21:18:53.130551+0000","last_clean":"2024-09-10T21:18:53.130551+0000","last_became_active":"2024-09-10T21:18:53.121768+0000","last_became_peered":"2024-09-10T21:18:53.121768+0000","last_unstale":"2024-09-10T21:18:53.130551+0000","last_undegraded":"2024-09-10T21:18:53.130551+0000","last_fullsized":"2024-09-10T21:18:53.130551+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:46.151037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116889+0000","last_change":"2024-09-10T21:18:53.116889+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116889+0000","last_undegraded":"2024-09-10T21:18:53.116889+0000","last_fullsized":"2024-09-10T21:18:53.116889+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.140097+0000","last_change":"2024-09-10T21:18:53.134253+0000","last_active":"2024-09-10T21:18:53.140097+0000","last_peered":"2024-09-10T21:18:53.140097+0000","last_clean":"2024-09-10T21:18:53.140097+0000","last_became_active":"2024-09-10T21:18:53.133971+0000","last_became_peered":"2024-09-10T21:18:53.133971+0000","last_unstale":"2024-09-10T21:18:53.140097+0000","last_undegraded":"2024-09-10T21:18:53.140097+0000","last_fullsized":"2024-09-10T21:18:53.140097+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:05.811497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.104469+0000","last_change":"2024-09-10T21:18:52.104469+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.104469+0000","last_undegraded":"2024-09-10T21:18:52.104469+0000","last_fullsized":"2024-09-10T21:18:52.104469+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123461+0000","last_change":"2024-09-10T21:18:53.123461+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123461+0000","last_undegraded":"2024-09-10T21:18:53.123461+0000","last_fullsized":"2024-09-10T21:18:53.123461+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.134759+0000","last_change":"2024-09-10T21:18:53.132095+0000","last_active":"2024-09-10T21:18:53.134759+0000","last_peered":"2024-09-10T21:18:53.134759+0000","last_clean":"2024-09-10T21:18:53.134759+0000","last_became_active":"2024-09-10T21:18:53.131751+0000","last_became_peered":"2024-09-10T21:18:53.131751+0000","last_unstale":"2024-09-10T21:18:53.134759+0000","last_undegraded":"2024-09-10T21:18:53.134759+0000","last_fullsized":"2024-09-10T21:18:53.134759+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:03.376294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116079+0000","last_change":"2024-09-10T21:18:53.116079+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116079+0000","last_undegraded":"2024-09-10T21:18:53.116079+0000","last_fullsized":"2024-09-10T21:18:53.116079+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.138579+0000","last_change":"2024-09-10T21:18:53.135048+0000","last_active":"2024-09-10T21:18:53.138579+0000","last_peered":"2024-09-10T21:18:53.138579+0000","last_clean":"2024-09-10T21:18:53.138579+0000","last_became_active":"2024-09-10T21:18:53.134366+0000","last_became_peered":"2024-09-10T21:18:53.134366+0000","last_unstale":"2024-09-10T21:18:53.138579+0000","last_undegraded":"2024-09-10T21:18:53.138579+0000","last_fullsized":"2024-09-10T21:18:53.138579+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:53.969257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.132642+0000","last_change":"2024-09-10T21:18:53.130356+0000","last_active":"2024-09-10T21:18:53.132642+0000","last_peered":"2024-09-10T21:18:53.132642+0000","last_clean":"2024-09-10T21:18:53.132642+0000","last_became_active":"2024-09-10T21:18:53.130043+0000","last_became_peered":"2024-09-10T21:18:53.130043+0000","last_unstale":"2024-09-10T21:18:53.132642+0000","last_undegraded":"2024-09-10T21:18:53.132642+0000","last_fullsized":"2024-09-10T21:18:53.132642+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:02.691227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114902+0000","last_change":"2024-09-10T21:18:53.114902+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114902+0000","last_undegraded":"2024-09-10T21:18:53.114902+0000","last_fullsized":"2024-09-10T21:18:53.114902+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.134829+0000","last_change":"2024-09-10T21:18:53.129464+0000","last_active":"2024-09-10T21:18:53.134829+0000","last_peered":"2024-09-10T21:18:53.134829+0000","last_clean":"2024-09-10T21:18:53.134829+0000","last_became_active":"2024-09-10T21:18:53.129179+0000","last_became_peered":"2024-09-10T21:18:53.129179+0000","last_unstale":"2024-09-10T21:18:53.134829+0000","last_undegraded":"2024-09-10T21:18:53.134829+0000","last_fullsized":"2024-09-10T21:18:53.134829+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:10.823382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114044+0000","last_change":"2024-09-10T21:18:53.114044+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114044+0000","last_undegraded":"2024-09-10T21:18:53.114044+0000","last_fullsized":"2024-09-10T21:18:53.114044+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.132295+0000","last_change":"2024-09-10T21:18:53.127488+0000","last_active":"2024-09-10T21:18:53.132295+0000","last_peered":"2024-09-10T21:18:53.132295+0000","last_clean":"2024-09-10T21:18:53.132295+0000","last_became_active":"2024-09-10T21:18:53.125698+0000","last_became_peered":"2024-09-10T21:18:53.125698+0000","last_unstale":"2024-09-10T21:18:53.132295+0000","last_undegraded":"2024-09-10T21:18:53.132295+0000","last_fullsized":"2024-09-10T21:18:53.132295+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:47.324904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121885+0000","last_change":"2024-09-10T21:18:53.121885+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121885+0000","last_undegraded":"2024-09-10T21:18:53.121885+0000","last_fullsized":"2024-09-10T21:18:53.121885+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.104899+0000","last_change":"2024-09-10T21:18:52.104899+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.104899+0000","last_undegraded":"2024-09-10T21:18:52.104899+0000","last_fullsized":"2024-09-10T21:18:52.104899+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121944+0000","last_change":"2024-09-10T21:18:53.121944+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121944+0000","last_undegraded":"2024-09-10T21:18:53.121944+0000","last_fullsized":"2024-09-10T21:18:53.121944+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.136282+0000","last_change":"2024-09-10T21:18:53.134405+0000","last_active":"2024-09-10T21:18:53.136282+0000","last_peered":"2024-09-10T21:18:53.136282+0000","last_clean":"2024-09-10T21:18:53.136282+0000","last_became_active":"2024-09-10T21:18:53.134012+0000","last_became_peered":"2024-09-10T21:18:53.134012+0000","last_unstale":"2024-09-10T21:18:53.136282+0000","last_undegraded":"2024-09-10T21:18:53.136282+0000","last_fullsized":"2024-09-10T21:18:53.136282+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:09.018998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116619+0000","last_change":"2024-09-10T21:18:53.116619+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116619+0000","last_undegraded":"2024-09-10T21:18:53.116619+0000","last_fullsized":"2024-09-10T21:18:53.116619+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111870+0000","last_change":"2024-09-10T21:18:52.111870+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111870+0000","last_undegraded":"2024-09-10T21:18:52.111870+0000","last_fullsized":"2024-09-10T21:18:52.111870+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121517+0000","last_change":"2024-09-10T21:18:53.121517+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121517+0000","last_undegraded":"2024-09-10T21:18:53.121517+0000","last_fullsized":"2024-09-10T21:18:53.121517+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.131783+0000","last_change":"2024-09-10T21:18:53.127007+0000","last_active":"2024-09-10T21:18:53.131783+0000","last_peered":"2024-09-10T21:18:53.131783+0000","last_clean":"2024-09-10T21:18:53.131783+0000","last_became_active":"2024-09-10T21:18:53.126673+0000","last_became_peered":"2024-09-10T21:18:53.126673+0000","last_unstale":"2024-09-10T21:18:53.131783+0000","last_undegraded":"2024-09-10T21:18:53.131783+0000","last_fullsized":"2024-09-10T21:18:53.131783+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:39.746760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116547+0000","last_change":"2024-09-10T21:18:53.116547+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116547+0000","last_undegraded":"2024-09-10T21:18:53.116547+0000","last_fullsized":"2024-09-10T21:18:53.116547+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.104198+0000","last_change":"2024-09-10T21:18:52.104198+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.104198+0000","last_undegraded":"2024-09-10T21:18:52.104198+0000","last_fullsized":"2024-09-10T21:18:52.104198+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.138049+0000","last_change":"2024-09-10T21:18:53.137417+0000","last_active":"2024-09-10T21:18:53.138049+0000","last_peered":"2024-09-10T21:18:53.138049+0000","last_clean":"2024-09-10T21:18:53.138049+0000","last_became_active":"2024-09-10T21:18:53.137042+0000","last_became_peered":"2024-09-10T21:18:53.137042+0000","last_unstale":"2024-09-10T21:18:53.138049+0000","last_undegraded":"2024-09-10T21:18:53.138049+0000","last_fullsized":"2024-09-10T21:18:53.138049+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:45.885394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123123+0000","last_change":"2024-09-10T21:18:53.123123+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123123+0000","last_undegraded":"2024-09-10T21:18:53.123123+0000","last_fullsized":"2024-09-10T21:18:53.123123+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111728+0000","last_change":"2024-09-10T21:18:52.111728+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111728+0000","last_undegraded":"2024-09-10T21:18:52.111728+0000","last_fullsized":"2024-09-10T21:18:52.111728+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.122766+0000","last_change":"2024-09-10T21:18:53.122766+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.122766+0000","last_undegraded":"2024-09-10T21:18:53.122766+0000","last_fullsized":"2024-09-10T21:18:53.122766+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111758+0000","last_change":"2024-09-10T21:18:52.111758+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111758+0000","last_undegraded":"2024-09-10T21:18:52.111758+0000","last_fullsized":"2024-09-10T21:18:52.111758+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117371+0000","last_change":"2024-09-10T21:18:53.117371+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117371+0000","last_undegraded":"2024-09-10T21:18:53.117371+0000","last_fullsized":"2024-09-10T21:18:53.117371+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.140124+0000","last_change":"2024-09-10T21:18:53.136538+0000","last_active":"2024-09-10T21:18:53.140124+0000","last_peered":"2024-09-10T21:18:53.140124+0000","last_clean":"2024-09-10T21:18:53.140124+0000","last_became_active":"2024-09-10T21:18:53.135006+0000","last_became_peered":"2024-09-10T21:18:53.135006+0000","last_unstale":"2024-09-10T21:18:53.140124+0000","last_undegraded":"2024-09-10T21:18:53.140124+0000","last_fullsized":"2024-09-10T21:18:53.140124+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:58.524531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.117304+0000","last_change":"2024-09-10T21:18:53.117304+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.117304+0000","last_undegraded":"2024-09-10T21:18:53.117304+0000","last_fullsized":"2024-09-10T21:18:53.117304+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111997+0000","last_change":"2024-09-10T21:18:52.111997+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111997+0000","last_undegraded":"2024-09-10T21:18:52.111997+0000","last_fullsized":"2024-09-10T21:18:52.111997+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.115916+0000","last_change":"2024-09-10T21:18:53.115916+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.115916+0000","last_undegraded":"2024-09-10T21:18:53.115916+0000","last_fullsized":"2024-09-10T21:18:53.115916+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.133308+0000","last_change":"2024-09-10T21:18:53.126832+0000","last_active":"2024-09-10T21:18:53.133308+0000","last_peered":"2024-09-10T21:18:53.133308+0000","last_clean":"2024-09-10T21:18:53.133308+0000","last_became_active":"2024-09-10T21:18:53.126580+0000","last_became_peered":"2024-09-10T21:18:53.126580+0000","last_unstale":"2024-09-10T21:18:53.133308+0000","last_undegraded":"2024-09-10T21:18:53.133308+0000","last_fullsized":"2024-09-10T21:18:53.133308+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:42.773987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116655+0000","last_change":"2024-09-10T21:18:53.116655+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116655+0000","last_undegraded":"2024-09-10T21:18:53.116655+0000","last_fullsized":"2024-09-10T21:18:53.116655+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.134062+0000","last_change":"2024-09-10T21:18:53.129128+0000","last_active":"2024-09-10T21:18:53.134062+0000","last_peered":"2024-09-10T21:18:53.134062+0000","last_clean":"2024-09-10T21:18:53.134062+0000","last_became_active":"2024-09-10T21:18:53.128826+0000","last_became_peered":"2024-09-10T21:18:53.128826+0000","last_unstale":"2024-09-10T21:18:53.134062+0000","last_undegraded":"2024-09-10T21:18:53.134062+0000","last_fullsized":"2024-09-10T21:18:53.134062+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:14.312782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116046+0000","last_change":"2024-09-10T21:18:53.116046+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116046+0000","last_undegraded":"2024-09-10T21:18:53.116046+0000","last_fullsized":"2024-09-10T21:18:53.116046+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.133416+0000","last_change":"2024-09-10T21:18:53.128814+0000","last_active":"2024-09-10T21:18:53.133416+0000","last_peered":"2024-09-10T21:18:53.133416+0000","last_clean":"2024-09-10T21:18:53.133416+0000","last_became_active":"2024-09-10T21:18:53.128474+0000","last_became_peered":"2024-09-10T21:18:53.128474+0000","last_unstale":"2024-09-10T21:18:53.133416+0000","last_undegraded":"2024-09-10T21:18:53.133416+0000","last_fullsized":"2024-09-10T21:18:53.133416+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:50.343267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116419+0000","last_change":"2024-09-10T21:18:53.116419+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116419+0000","last_undegraded":"2024-09-10T21:18:53.116419+0000","last_fullsized":"2024-09-10T21:18:53.116419+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.135855+0000","last_change":"2024-09-10T21:18:53.132527+0000","last_active":"2024-09-10T21:18:53.135855+0000","last_peered":"2024-09-10T21:18:53.135855+0000","last_clean":"2024-09-10T21:18:53.135855+0000","last_became_active":"2024-09-10T21:18:53.130251+0000","last_became_peered":"2024-09-10T21:18:53.130251+0000","last_unstale":"2024-09-10T21:18:53.135855+0000","last_undegraded":"2024-09-10T21:18:53.135855+0000","last_fullsized":"2024-09-10T21:18:53.135855+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:12.376207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123636+0000","last_change":"2024-09-10T21:18:53.123636+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123636+0000","last_undegraded":"2024-09-10T21:18:53.123636+0000","last_fullsized":"2024-09-10T21:18:53.123636+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.137844+0000","last_change":"2024-09-10T21:18:53.133231+0000","last_active":"2024-09-10T21:18:53.137844+0000","last_peered":"2024-09-10T21:18:53.137844+0000","last_clean":"2024-09-10T21:18:53.137844+0000","last_became_active":"2024-09-10T21:18:53.132930+0000","last_became_peered":"2024-09-10T21:18:53.132930+0000","last_unstale":"2024-09-10T21:18:53.137844+0000","last_undegraded":"2024-09-10T21:18:53.137844+0000","last_fullsized":"2024-09-10T21:18:53.137844+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:26.790852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.115124+0000","last_change":"2024-09-10T21:18:53.115124+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.115124+0000","last_undegraded":"2024-09-10T21:18:53.115124+0000","last_fullsized":"2024-09-10T21:18:53.115124+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.105072+0000","last_change":"2024-09-10T21:18:52.105072+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.105072+0000","last_undegraded":"2024-09-10T21:18:52.105072+0000","last_fullsized":"2024-09-10T21:18:52.105072+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116808+0000","last_change":"2024-09-10T21:18:53.116808+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116808+0000","last_undegraded":"2024-09-10T21:18:53.116808+0000","last_fullsized":"2024-09-10T21:18:53.116808+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.134459+0000","last_change":"2024-09-10T21:18:53.130476+0000","last_active":"2024-09-10T21:18:53.134459+0000","last_peered":"2024-09-10T21:18:53.134459+0000","last_clean":"2024-09-10T21:18:53.134459+0000","last_became_active":"2024-09-10T21:18:53.130213+0000","last_became_peered":"2024-09-10T21:18:53.130213+0000","last_unstale":"2024-09-10T21:18:53.134459+0000","last_undegraded":"2024-09-10T21:18:53.134459+0000","last_fullsized":"2024-09-10T21:18:53.134459+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:40.694947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114244+0000","last_change":"2024-09-10T21:18:53.114244+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114244+0000","last_undegraded":"2024-09-10T21:18:53.114244+0000","last_fullsized":"2024-09-10T21:18:53.114244+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.135183+0000","last_change":"2024-09-10T21:18:53.130268+0000","last_active":"2024-09-10T21:18:53.135183+0000","last_peered":"2024-09-10T21:18:53.135183+0000","last_clean":"2024-09-10T21:18:53.135183+0000","last_became_active":"2024-09-10T21:18:53.129955+0000","last_became_peered":"2024-09-10T21:18:53.129955+0000","last_unstale":"2024-09-10T21:18:53.135183+0000","last_undegraded":"2024-09-10T21:18:53.135183+0000","last_fullsized":"2024-09-10T21:18:53.135183+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:03.568523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.114041+0000","last_change":"2024-09-10T21:18:53.114041+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.114041+0000","last_undegraded":"2024-09-10T21:18:53.114041+0000","last_fullsized":"2024-09-10T21:18:53.114041+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.135542+0000","last_change":"2024-09-10T21:18:53.133483+0000","last_active":"2024-09-10T21:18:53.135542+0000","last_peered":"2024-09-10T21:18:53.135542+0000","last_clean":"2024-09-10T21:18:53.135542+0000","last_became_active":"2024-09-10T21:18:53.133196+0000","last_became_peered":"2024-09-10T21:18:53.133196+0000","last_unstale":"2024-09-10T21:18:53.135542+0000","last_undegraded":"2024-09-10T21:18:53.135542+0000","last_fullsized":"2024-09-10T21:18:53.135542+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:42:00.777546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:18:53.102675+0000","last_change":"2024-09-10T21:18:53.102675+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.102675+0000","last_undegraded":"2024-09-10T21:18:53.102675+0000","last_fullsized":"2024-09-10T21:18:53.102675+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.111911+0000","last_change":"2024-09-10T21:18:52.111911+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.111911+0000","last_undegraded":"2024-09-10T21:18:52.111911+0000","last_fullsized":"2024-09-10T21:18:52.111911+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116699+0000","last_change":"2024-09-10T21:18:53.116699+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116699+0000","last_undegraded":"2024-09-10T21:18:53.116699+0000","last_fullsized":"2024-09-10T21:18:53.116699+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.135056+0000","last_change":"2024-09-10T21:18:53.133724+0000","last_active":"2024-09-10T21:18:53.135056+0000","last_peered":"2024-09-10T21:18:53.135056+0000","last_clean":"2024-09-10T21:18:53.135056+0000","last_became_active":"2024-09-10T21:18:53.133442+0000","last_became_peered":"2024-09-10T21:18:53.133442+0000","last_unstale":"2024-09-10T21:18:53.135056+0000","last_undegraded":"2024-09-10T21:18:53.135056+0000","last_fullsized":"2024-09-10T21:18:53.135056+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:12.220728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.121788+0000","last_change":"2024-09-10T21:18:53.121788+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.121788+0000","last_undegraded":"2024-09-10T21:18:53.121788+0000","last_fullsized":"2024-09-10T21:18:53.121788+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.136783+0000","last_change":"2024-09-10T21:18:53.132411+0000","last_active":"2024-09-10T21:18:53.136783+0000","last_peered":"2024-09-10T21:18:53.136783+0000","last_clean":"2024-09-10T21:18:53.136783+0000","last_became_active":"2024-09-10T21:18:53.131949+0000","last_became_peered":"2024-09-10T21:18:53.131949+0000","last_unstale":"2024-09-10T21:18:53.136783+0000","last_undegraded":"2024-09-10T21:18:53.136783+0000","last_fullsized":"2024-09-10T21:18:53.136783+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:59.498416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.116457+0000","last_change":"2024-09-10T21:18:53.116457+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.116457+0000","last_undegraded":"2024-09-10T21:18:53.116457+0000","last_fullsized":"2024-09-10T21:18:53.116457+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-10T21:18:53.131780+0000","last_change":"2024-09-10T21:18:53.127615+0000","last_active":"2024-09-10T21:18:53.131780+0000","last_peered":"2024-09-10T21:18:53.131780+0000","last_clean":"2024-09-10T21:18:53.131780+0000","last_became_active":"2024-09-10T21:18:53.127317+0000","last_became_peered":"2024-09-10T21:18:53.127317+0000","last_unstale":"2024-09-10T21:18:53.131780+0000","last_undegraded":"2024-09-10T21:18:53.131780+0000","last_fullsized":"2024-09-10T21:18:53.131780+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:48.915819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-10T21:18:53.123286+0000","last_change":"2024-09-10T21:18:53.123286+0000","last_active":"2024-09-10T21:18:53.102675+0000","last_peered":"2024-09-10T21:18:53.102675+0000","last_clean":"2024-09-10T21:18:53.102675+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:53.123286+0000","last_undegraded":"2024-09-10T21:18:53.123286+0000","last_fullsized":"2024-09-10T21:18:53.123286+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T21:18:52.114087+0000","last_change":"2024-09-10T21:18:52.114087+0000","last_active":"2024-09-10T21:18:52.095934+0000","last_peered":"2024-09-10T21:18:52.095934+0000","last_clean":"2024-09-10T21:18:52.095934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:18:52.114087+0000","last_undegraded":"2024-09-10T21:18:52.114087+0000","last_fullsized":"2024-09-10T21:18:52.114087+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":313,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-10T21:18:52.113900+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:18:52.113900+0000","last_peered":"2024-09-10T21:18:52.113900+0000","last_clean":"2024-09-10T21:18:52.113900+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:18:52.113900+0000","last_undegraded":"2024-09-10T21:18:52.113900+0000","last_fullsized":"2024-09-10T21:18:52.113900+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":6},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37016,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509548,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37596,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334244,"statfs":{"total":96636764160,"available":96598265856,"internally_reserved":0,"allocated":2297856,"data_stored":1323658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509546,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37016,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509547,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37016,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":1703936,"data_stored":733290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542250,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41160,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330680,"statfs":{"total":96636764160,"available":96594616320,"internally_reserved":0,"allocated":1691648,"data_stored":727015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542249,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37580,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334260,"statfs":{"total":96636764160,"available":96598282240,"internally_reserved":0,"allocated":2285568,"data_stored":1317383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:18:59.101 INFO:tasks.ceph.ceph_manager.ceph:PG 23.3f is not active+clean 2024-09-10T21:18:59.101 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.118441+0000', 'last_change': '2024-09-10T21:18:53.118441+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.118441+0000', 'last_undegraded': '2024-09-10T21:18:53.118441+0000', 'last_fullsized': '2024-09-10T21:18:53.118441+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:18:59.101 INFO:tasks.ceph.ceph_manager.ceph:PG 23.3e is not active+clean 2024-09-10T21:18:59.101 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.117949+0000', 'last_change': '2024-09-10T21:18:53.117949+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.117949+0000', 'last_undegraded': '2024-09-10T21:18:53.117949+0000', 'last_fullsized': '2024-09-10T21:18:53.117949+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:18:59.101 INFO:tasks.ceph.ceph_manager.ceph:PG 23.3d is not active+clean 2024-09-10T21:18:59.101 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.118237+0000', 'last_change': '2024-09-10T21:18:53.118237+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.118237+0000', 'last_undegraded': '2024-09-10T21:18:53.118237+0000', 'last_fullsized': '2024-09-10T21:18:53.118237+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:18:59.101 INFO:tasks.ceph.ceph_manager.ceph:PG 23.3c is not active+clean 2024-09-10T21:18:59.101 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.118754+0000', 'last_change': '2024-09-10T21:18:53.118754+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.118754+0000', 'last_undegraded': '2024-09-10T21:18:53.118754+0000', 'last_fullsized': '2024-09-10T21:18:53.118754+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:18:59.101 INFO:tasks.ceph.ceph_manager.ceph:PG 23.3b is not active+clean 2024-09-10T21:18:59.102 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.102 INFO:tasks.ceph.ceph_manager.ceph:PG 23.3a is not active+clean 2024-09-10T21:18:59.102 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.102 INFO:tasks.ceph.ceph_manager.ceph:PG 23.39 is not active+clean 2024-09-10T21:18:59.102 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.121298+0000', 'last_change': '2024-09-10T21:18:53.121298+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.121298+0000', 'last_undegraded': '2024-09-10T21:18:53.121298+0000', 'last_fullsized': '2024-09-10T21:18:53.121298+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:18:59.102 INFO:tasks.ceph.ceph_manager.ceph:PG 23.38 is not active+clean 2024-09-10T21:18:59.102 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.124732+0000', 'last_change': '2024-09-10T21:18:53.124732+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.124732+0000', 'last_undegraded': '2024-09-10T21:18:53.124732+0000', 'last_fullsized': '2024-09-10T21:18:53.124732+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:18:59.102 INFO:tasks.ceph.ceph_manager.ceph:PG 23.37 is not active+clean 2024-09-10T21:18:59.102 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.124460+0000', 'last_change': '2024-09-10T21:18:53.124460+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.124460+0000', 'last_undegraded': '2024-09-10T21:18:53.124460+0000', 'last_fullsized': '2024-09-10T21:18:53.124460+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:18:59.102 INFO:tasks.ceph.ceph_manager.ceph:PG 23.36 is not active+clean 2024-09-10T21:18:59.103 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.123934+0000', 'last_change': '2024-09-10T21:18:53.123934+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.123934+0000', 'last_undegraded': '2024-09-10T21:18:53.123934+0000', 'last_fullsized': '2024-09-10T21:18:53.123934+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:18:59.103 INFO:tasks.ceph.ceph_manager.ceph:PG 23.35 is not active+clean 2024-09-10T21:18:59.103 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.103 INFO:tasks.ceph.ceph_manager.ceph:PG 23.34 is not active+clean 2024-09-10T21:18:59.103 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.117758+0000', 'last_change': '2024-09-10T21:18:53.117758+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.117758+0000', 'last_undegraded': '2024-09-10T21:18:53.117758+0000', 'last_fullsized': '2024-09-10T21:18:53.117758+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:18:59.103 INFO:tasks.ceph.ceph_manager.ceph:PG 23.33 is not active+clean 2024-09-10T21:18:59.103 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.117647+0000', 'last_change': '2024-09-10T21:18:53.117647+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.117647+0000', 'last_undegraded': '2024-09-10T21:18:53.117647+0000', 'last_fullsized': '2024-09-10T21:18:53.117647+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:18:59.103 INFO:tasks.ceph.ceph_manager.ceph:PG 23.32 is not active+clean 2024-09-10T21:18:59.103 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.124781+0000', 'last_change': '2024-09-10T21:18:53.124781+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.124781+0000', 'last_undegraded': '2024-09-10T21:18:53.124781+0000', 'last_fullsized': '2024-09-10T21:18:53.124781+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:18:59.103 INFO:tasks.ceph.ceph_manager.ceph:PG 23.31 is not active+clean 2024-09-10T21:18:59.103 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.123669+0000', 'last_change': '2024-09-10T21:18:53.123669+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.123669+0000', 'last_undegraded': '2024-09-10T21:18:53.123669+0000', 'last_fullsized': '2024-09-10T21:18:53.123669+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:18:59.104 INFO:tasks.ceph.ceph_manager.ceph:PG 23.30 is not active+clean 2024-09-10T21:18:59.104 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.123065+0000', 'last_change': '2024-09-10T21:18:53.123065+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.123065+0000', 'last_undegraded': '2024-09-10T21:18:53.123065+0000', 'last_fullsized': '2024-09-10T21:18:53.123065+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:18:59.104 INFO:tasks.ceph.ceph_manager.ceph:PG 23.2f is not active+clean 2024-09-10T21:18:59.104 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.104 INFO:tasks.ceph.ceph_manager.ceph:PG 23.2e is not active+clean 2024-09-10T21:18:59.104 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.115101+0000', 'last_change': '2024-09-10T21:18:53.115101+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.115101+0000', 'last_undegraded': '2024-09-10T21:18:53.115101+0000', 'last_fullsized': '2024-09-10T21:18:53.115101+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:18:59.104 INFO:tasks.ceph.ceph_manager.ceph:PG 23.2d is not active+clean 2024-09-10T21:18:59.104 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.124613+0000', 'last_change': '2024-09-10T21:18:53.124613+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.124613+0000', 'last_undegraded': '2024-09-10T21:18:53.124613+0000', 'last_fullsized': '2024-09-10T21:18:53.124613+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:18:59.104 INFO:tasks.ceph.ceph_manager.ceph:PG 23.2c is not active+clean 2024-09-10T21:18:59.104 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.117230+0000', 'last_change': '2024-09-10T21:18:53.117230+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.117230+0000', 'last_undegraded': '2024-09-10T21:18:53.117230+0000', 'last_fullsized': '2024-09-10T21:18:53.117230+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:18:59.105 INFO:tasks.ceph.ceph_manager.ceph:PG 23.2b is not active+clean 2024-09-10T21:18:59.105 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.105 INFO:tasks.ceph.ceph_manager.ceph:PG 23.2a is not active+clean 2024-09-10T21:18:59.105 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.105 INFO:tasks.ceph.ceph_manager.ceph:PG 23.29 is not active+clean 2024-09-10T21:18:59.105 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.115573+0000', 'last_change': '2024-09-10T21:18:53.115573+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.115573+0000', 'last_undegraded': '2024-09-10T21:18:53.115573+0000', 'last_fullsized': '2024-09-10T21:18:53.115573+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:18:59.105 INFO:tasks.ceph.ceph_manager.ceph:PG 23.28 is not active+clean 2024-09-10T21:18:59.105 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.105 INFO:tasks.ceph.ceph_manager.ceph:PG 23.27 is not active+clean 2024-09-10T21:18:59.105 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.105 INFO:tasks.ceph.ceph_manager.ceph:PG 23.26 is not active+clean 2024-09-10T21:18:59.106 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.118327+0000', 'last_change': '2024-09-10T21:18:53.118327+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.118327+0000', 'last_undegraded': '2024-09-10T21:18:53.118327+0000', 'last_fullsized': '2024-09-10T21:18:53.118327+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:18:59.106 INFO:tasks.ceph.ceph_manager.ceph:PG 23.25 is not active+clean 2024-09-10T21:18:59.106 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.106 INFO:tasks.ceph.ceph_manager.ceph:PG 23.24 is not active+clean 2024-09-10T21:18:59.106 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.122414+0000', 'last_change': '2024-09-10T21:18:53.122414+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.122414+0000', 'last_undegraded': '2024-09-10T21:18:53.122414+0000', 'last_fullsized': '2024-09-10T21:18:53.122414+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:18:59.106 INFO:tasks.ceph.ceph_manager.ceph:PG 23.23 is not active+clean 2024-09-10T21:18:59.106 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.117885+0000', 'last_change': '2024-09-10T21:18:53.117885+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.117885+0000', 'last_undegraded': '2024-09-10T21:18:53.117885+0000', 'last_fullsized': '2024-09-10T21:18:53.117885+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:18:59.106 INFO:tasks.ceph.ceph_manager.ceph:PG 23.22 is not active+clean 2024-09-10T21:18:59.106 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.124230+0000', 'last_change': '2024-09-10T21:18:53.124230+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.124230+0000', 'last_undegraded': '2024-09-10T21:18:53.124230+0000', 'last_fullsized': '2024-09-10T21:18:53.124230+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:18:59.106 INFO:tasks.ceph.ceph_manager.ceph:PG 23.21 is not active+clean 2024-09-10T21:18:59.107 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.118139+0000', 'last_change': '2024-09-10T21:18:53.118139+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.118139+0000', 'last_undegraded': '2024-09-10T21:18:53.118139+0000', 'last_fullsized': '2024-09-10T21:18:53.118139+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:18:59.107 INFO:tasks.ceph.ceph_manager.ceph:PG 23.20 is not active+clean 2024-09-10T21:18:59.107 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.107 INFO:tasks.ceph.ceph_manager.ceph:PG 23.1e is not active+clean 2024-09-10T21:18:59.107 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.114559+0000', 'last_change': '2024-09-10T21:18:53.114559+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.114559+0000', 'last_undegraded': '2024-09-10T21:18:53.114559+0000', 'last_fullsized': '2024-09-10T21:18:53.114559+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:18:59.107 INFO:tasks.ceph.ceph_manager.ceph:PG 23.1f is not active+clean 2024-09-10T21:18:59.107 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.116889+0000', 'last_change': '2024-09-10T21:18:53.116889+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.116889+0000', 'last_undegraded': '2024-09-10T21:18:53.116889+0000', 'last_fullsized': '2024-09-10T21:18:53.116889+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:18:59.107 INFO:tasks.ceph.ceph_manager.ceph:PG 23.1c is not active+clean 2024-09-10T21:18:59.107 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.107 INFO:tasks.ceph.ceph_manager.ceph:PG 22.1d is not active+clean 2024-09-10T21:18:59.107 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '22.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 135, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:52.104469+0000', 'last_change': '2024-09-10T21:18:52.104469+0000', 'last_active': '2024-09-10T21:18:52.095934+0000', 'last_peered': '2024-09-10T21:18:52.095934+0000', 'last_clean': '2024-09-10T21:18:52.095934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:52.104469+0000', 'last_undegraded': '2024-09-10T21:18:52.104469+0000', 'last_fullsized': '2024-09-10T21:18:52.104469+0000', 'mapping_epoch': 135, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 135, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:18:59.108 INFO:tasks.ceph.ceph_manager.ceph:PG 23.1d is not active+clean 2024-09-10T21:18:59.108 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.123461+0000', 'last_change': '2024-09-10T21:18:53.123461+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.123461+0000', 'last_undegraded': '2024-09-10T21:18:53.123461+0000', 'last_fullsized': '2024-09-10T21:18:53.123461+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:18:59.108 INFO:tasks.ceph.ceph_manager.ceph:PG 23.1a is not active+clean 2024-09-10T21:18:59.108 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.116079+0000', 'last_change': '2024-09-10T21:18:53.116079+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.116079+0000', 'last_undegraded': '2024-09-10T21:18:53.116079+0000', 'last_fullsized': '2024-09-10T21:18:53.116079+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:18:59.108 INFO:tasks.ceph.ceph_manager.ceph:PG 23.1b is not active+clean 2024-09-10T21:18:59.108 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.108 INFO:tasks.ceph.ceph_manager.ceph:PG 23.18 is not active+clean 2024-09-10T21:18:59.108 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.114902+0000', 'last_change': '2024-09-10T21:18:53.114902+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.114902+0000', 'last_undegraded': '2024-09-10T21:18:53.114902+0000', 'last_fullsized': '2024-09-10T21:18:53.114902+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:18:59.108 INFO:tasks.ceph.ceph_manager.ceph:PG 23.19 is not active+clean 2024-09-10T21:18:59.108 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.114044+0000', 'last_change': '2024-09-10T21:18:53.114044+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.114044+0000', 'last_undegraded': '2024-09-10T21:18:53.114044+0000', 'last_fullsized': '2024-09-10T21:18:53.114044+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:18:59.109 INFO:tasks.ceph.ceph_manager.ceph:PG 23.17 is not active+clean 2024-09-10T21:18:59.109 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.121885+0000', 'last_change': '2024-09-10T21:18:53.121885+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.121885+0000', 'last_undegraded': '2024-09-10T21:18:53.121885+0000', 'last_fullsized': '2024-09-10T21:18:53.121885+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:18:59.109 INFO:tasks.ceph.ceph_manager.ceph:PG 22.16 is not active+clean 2024-09-10T21:18:59.109 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '22.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 135, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:52.104899+0000', 'last_change': '2024-09-10T21:18:52.104899+0000', 'last_active': '2024-09-10T21:18:52.095934+0000', 'last_peered': '2024-09-10T21:18:52.095934+0000', 'last_clean': '2024-09-10T21:18:52.095934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:52.104899+0000', 'last_undegraded': '2024-09-10T21:18:52.104899+0000', 'last_fullsized': '2024-09-10T21:18:52.104899+0000', 'mapping_epoch': 135, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 135, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:18:59.109 INFO:tasks.ceph.ceph_manager.ceph:PG 23.14 is not active+clean 2024-09-10T21:18:59.109 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.121944+0000', 'last_change': '2024-09-10T21:18:53.121944+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.121944+0000', 'last_undegraded': '2024-09-10T21:18:53.121944+0000', 'last_fullsized': '2024-09-10T21:18:53.121944+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:18:59.109 INFO:tasks.ceph.ceph_manager.ceph:PG 23.15 is not active+clean 2024-09-10T21:18:59.109 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.116619+0000', 'last_change': '2024-09-10T21:18:53.116619+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.116619+0000', 'last_undegraded': '2024-09-10T21:18:53.116619+0000', 'last_fullsized': '2024-09-10T21:18:53.116619+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:18:59.109 INFO:tasks.ceph.ceph_manager.ceph:PG 22.14 is not active+clean 2024-09-10T21:18:59.109 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '22.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 135, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:52.111870+0000', 'last_change': '2024-09-10T21:18:52.111870+0000', 'last_active': '2024-09-10T21:18:52.095934+0000', 'last_peered': '2024-09-10T21:18:52.095934+0000', 'last_clean': '2024-09-10T21:18:52.095934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:52.111870+0000', 'last_undegraded': '2024-09-10T21:18:52.111870+0000', 'last_fullsized': '2024-09-10T21:18:52.111870+0000', 'mapping_epoch': 135, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 135, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:18:59.110 INFO:tasks.ceph.ceph_manager.ceph:PG 23.12 is not active+clean 2024-09-10T21:18:59.110 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.121517+0000', 'last_change': '2024-09-10T21:18:53.121517+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.121517+0000', 'last_undegraded': '2024-09-10T21:18:53.121517+0000', 'last_fullsized': '2024-09-10T21:18:53.121517+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:18:59.110 INFO:tasks.ceph.ceph_manager.ceph:PG 23.13 is not active+clean 2024-09-10T21:18:59.110 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.116547+0000', 'last_change': '2024-09-10T21:18:53.116547+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.116547+0000', 'last_undegraded': '2024-09-10T21:18:53.116547+0000', 'last_fullsized': '2024-09-10T21:18:53.116547+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:18:59.110 INFO:tasks.ceph.ceph_manager.ceph:PG 22.12 is not active+clean 2024-09-10T21:18:59.110 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '22.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 135, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:52.104198+0000', 'last_change': '2024-09-10T21:18:52.104198+0000', 'last_active': '2024-09-10T21:18:52.095934+0000', 'last_peered': '2024-09-10T21:18:52.095934+0000', 'last_clean': '2024-09-10T21:18:52.095934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:52.104198+0000', 'last_undegraded': '2024-09-10T21:18:52.104198+0000', 'last_fullsized': '2024-09-10T21:18:52.104198+0000', 'mapping_epoch': 135, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 135, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:18:59.110 INFO:tasks.ceph.ceph_manager.ceph:PG 23.10 is not active+clean 2024-09-10T21:18:59.110 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.110 INFO:tasks.ceph.ceph_manager.ceph:PG 23.11 is not active+clean 2024-09-10T21:18:59.110 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.123123+0000', 'last_change': '2024-09-10T21:18:53.123123+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.123123+0000', 'last_undegraded': '2024-09-10T21:18:53.123123+0000', 'last_fullsized': '2024-09-10T21:18:53.123123+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:18:59.110 INFO:tasks.ceph.ceph_manager.ceph:PG 22.10 is not active+clean 2024-09-10T21:18:59.111 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '22.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 135, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:52.111728+0000', 'last_change': '2024-09-10T21:18:52.111728+0000', 'last_active': '2024-09-10T21:18:52.095934+0000', 'last_peered': '2024-09-10T21:18:52.095934+0000', 'last_clean': '2024-09-10T21:18:52.095934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:52.111728+0000', 'last_undegraded': '2024-09-10T21:18:52.111728+0000', 'last_fullsized': '2024-09-10T21:18:52.111728+0000', 'mapping_epoch': 135, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 135, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:18:59.111 INFO:tasks.ceph.ceph_manager.ceph:PG 23.e is not active+clean 2024-09-10T21:18:59.111 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.122766+0000', 'last_change': '2024-09-10T21:18:53.122766+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.122766+0000', 'last_undegraded': '2024-09-10T21:18:53.122766+0000', 'last_fullsized': '2024-09-10T21:18:53.122766+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:18:59.111 INFO:tasks.ceph.ceph_manager.ceph:PG 22.f is not active+clean 2024-09-10T21:18:59.111 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '22.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 135, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:52.111758+0000', 'last_change': '2024-09-10T21:18:52.111758+0000', 'last_active': '2024-09-10T21:18:52.095934+0000', 'last_peered': '2024-09-10T21:18:52.095934+0000', 'last_clean': '2024-09-10T21:18:52.095934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:52.111758+0000', 'last_undegraded': '2024-09-10T21:18:52.111758+0000', 'last_fullsized': '2024-09-10T21:18:52.111758+0000', 'mapping_epoch': 135, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 135, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:18:59.111 INFO:tasks.ceph.ceph_manager.ceph:PG 23.f is not active+clean 2024-09-10T21:18:59.111 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.117371+0000', 'last_change': '2024-09-10T21:18:53.117371+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.117371+0000', 'last_undegraded': '2024-09-10T21:18:53.117371+0000', 'last_fullsized': '2024-09-10T21:18:53.117371+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:18:59.111 INFO:tasks.ceph.ceph_manager.ceph:PG 23.c is not active+clean 2024-09-10T21:18:59.111 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.117304+0000', 'last_change': '2024-09-10T21:18:53.117304+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.117304+0000', 'last_undegraded': '2024-09-10T21:18:53.117304+0000', 'last_fullsized': '2024-09-10T21:18:53.117304+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:18:59.111 INFO:tasks.ceph.ceph_manager.ceph:PG 22.d is not active+clean 2024-09-10T21:18:59.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '22.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 135, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:52.111997+0000', 'last_change': '2024-09-10T21:18:52.111997+0000', 'last_active': '2024-09-10T21:18:52.095934+0000', 'last_peered': '2024-09-10T21:18:52.095934+0000', 'last_clean': '2024-09-10T21:18:52.095934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:52.111997+0000', 'last_undegraded': '2024-09-10T21:18:52.111997+0000', 'last_fullsized': '2024-09-10T21:18:52.111997+0000', 'mapping_epoch': 135, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 135, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:18:59.112 INFO:tasks.ceph.ceph_manager.ceph:PG 23.d is not active+clean 2024-09-10T21:18:59.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.115916+0000', 'last_change': '2024-09-10T21:18:53.115916+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.115916+0000', 'last_undegraded': '2024-09-10T21:18:53.115916+0000', 'last_fullsized': '2024-09-10T21:18:53.115916+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:18:59.112 INFO:tasks.ceph.ceph_manager.ceph:PG 23.a is not active+clean 2024-09-10T21:18:59.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.116655+0000', 'last_change': '2024-09-10T21:18:53.116655+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.116655+0000', 'last_undegraded': '2024-09-10T21:18:53.116655+0000', 'last_fullsized': '2024-09-10T21:18:53.116655+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:18:59.112 INFO:tasks.ceph.ceph_manager.ceph:PG 23.b is not active+clean 2024-09-10T21:18:59.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.116046+0000', 'last_change': '2024-09-10T21:18:53.116046+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.116046+0000', 'last_undegraded': '2024-09-10T21:18:53.116046+0000', 'last_fullsized': '2024-09-10T21:18:53.116046+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:18:59.112 INFO:tasks.ceph.ceph_manager.ceph:PG 23.8 is not active+clean 2024-09-10T21:18:59.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.116419+0000', 'last_change': '2024-09-10T21:18:53.116419+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.116419+0000', 'last_undegraded': '2024-09-10T21:18:53.116419+0000', 'last_fullsized': '2024-09-10T21:18:53.116419+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:18:59.112 INFO:tasks.ceph.ceph_manager.ceph:PG 23.9 is not active+clean 2024-09-10T21:18:59.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.123636+0000', 'last_change': '2024-09-10T21:18:53.123636+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.123636+0000', 'last_undegraded': '2024-09-10T21:18:53.123636+0000', 'last_fullsized': '2024-09-10T21:18:53.123636+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:18:59.113 INFO:tasks.ceph.ceph_manager.ceph:PG 23.6 is not active+clean 2024-09-10T21:18:59.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.115124+0000', 'last_change': '2024-09-10T21:18:53.115124+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.115124+0000', 'last_undegraded': '2024-09-10T21:18:53.115124+0000', 'last_fullsized': '2024-09-10T21:18:53.115124+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:18:59.113 INFO:tasks.ceph.ceph_manager.ceph:PG 22.7 is not active+clean 2024-09-10T21:18:59.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '22.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 135, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:52.105072+0000', 'last_change': '2024-09-10T21:18:52.105072+0000', 'last_active': '2024-09-10T21:18:52.095934+0000', 'last_peered': '2024-09-10T21:18:52.095934+0000', 'last_clean': '2024-09-10T21:18:52.095934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:52.105072+0000', 'last_undegraded': '2024-09-10T21:18:52.105072+0000', 'last_fullsized': '2024-09-10T21:18:52.105072+0000', 'mapping_epoch': 135, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 135, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:18:59.113 INFO:tasks.ceph.ceph_manager.ceph:PG 23.7 is not active+clean 2024-09-10T21:18:59.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.116808+0000', 'last_change': '2024-09-10T21:18:53.116808+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.116808+0000', 'last_undegraded': '2024-09-10T21:18:53.116808+0000', 'last_fullsized': '2024-09-10T21:18:53.116808+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:18:59.113 INFO:tasks.ceph.ceph_manager.ceph:PG 23.4 is not active+clean 2024-09-10T21:18:59.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.114244+0000', 'last_change': '2024-09-10T21:18:53.114244+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.114244+0000', 'last_undegraded': '2024-09-10T21:18:53.114244+0000', 'last_fullsized': '2024-09-10T21:18:53.114244+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:18:59.113 INFO:tasks.ceph.ceph_manager.ceph:PG 23.5 is not active+clean 2024-09-10T21:18:59.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.114041+0000', 'last_change': '2024-09-10T21:18:53.114041+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.114041+0000', 'last_undegraded': '2024-09-10T21:18:53.114041+0000', 'last_fullsized': '2024-09-10T21:18:53.114041+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:18:59.114 INFO:tasks.ceph.ceph_manager.ceph:PG 23.2 is not active+clean 2024-09-10T21:18:59.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T21:18:53.102675+0000', 'last_change': '2024-09-10T21:18:53.102675+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.102675+0000', 'last_undegraded': '2024-09-10T21:18:53.102675+0000', 'last_fullsized': '2024-09-10T21:18:53.102675+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T21:18:59.114 INFO:tasks.ceph.ceph_manager.ceph:PG 22.3 is not active+clean 2024-09-10T21:18:59.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '22.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 135, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:52.111911+0000', 'last_change': '2024-09-10T21:18:52.111911+0000', 'last_active': '2024-09-10T21:18:52.095934+0000', 'last_peered': '2024-09-10T21:18:52.095934+0000', 'last_clean': '2024-09-10T21:18:52.095934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:52.111911+0000', 'last_undegraded': '2024-09-10T21:18:52.111911+0000', 'last_fullsized': '2024-09-10T21:18:52.111911+0000', 'mapping_epoch': 135, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 135, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:18:59.114 INFO:tasks.ceph.ceph_manager.ceph:PG 23.3 is not active+clean 2024-09-10T21:18:59.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.116699+0000', 'last_change': '2024-09-10T21:18:53.116699+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.116699+0000', 'last_undegraded': '2024-09-10T21:18:53.116699+0000', 'last_fullsized': '2024-09-10T21:18:53.116699+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:18:59.114 INFO:tasks.ceph.ceph_manager.ceph:PG 23.0 is not active+clean 2024-09-10T21:18:59.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.121788+0000', 'last_change': '2024-09-10T21:18:53.121788+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.121788+0000', 'last_undegraded': '2024-09-10T21:18:53.121788+0000', 'last_fullsized': '2024-09-10T21:18:53.121788+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:18:59.114 INFO:tasks.ceph.ceph_manager.ceph:PG 23.1 is not active+clean 2024-09-10T21:18:59.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.116457+0000', 'last_change': '2024-09-10T21:18:53.116457+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.116457+0000', 'last_undegraded': '2024-09-10T21:18:53.116457+0000', 'last_fullsized': '2024-09-10T21:18:53.116457+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:18:59.115 INFO:tasks.ceph.ceph_manager.ceph:PG 23.16 is not active+clean 2024-09-10T21:18:59.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '23.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:53.123286+0000', 'last_change': '2024-09-10T21:18:53.123286+0000', 'last_active': '2024-09-10T21:18:53.102675+0000', 'last_peered': '2024-09-10T21:18:53.102675+0000', 'last_clean': '2024-09-10T21:18:53.102675+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:53.123286+0000', 'last_undegraded': '2024-09-10T21:18:53.123286+0000', 'last_fullsized': '2024-09-10T21:18:53.123286+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:53.102675+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:18:59.115 INFO:tasks.ceph.ceph_manager.ceph:PG 22.17 is not active+clean 2024-09-10T21:18:59.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '22.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 135, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:18:52.114087+0000', 'last_change': '2024-09-10T21:18:52.114087+0000', 'last_active': '2024-09-10T21:18:52.095934+0000', 'last_peered': '2024-09-10T21:18:52.095934+0000', 'last_clean': '2024-09-10T21:18:52.095934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:18:52.114087+0000', 'last_undegraded': '2024-09-10T21:18:52.114087+0000', 'last_fullsized': '2024-09-10T21:18:52.114087+0000', 'mapping_epoch': 135, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 135, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'last_clean_scrub_stamp': '2024-09-10T21:18:52.095934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:18:59.115 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T21:18:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.284+0000 7f3409800640 1 -- 172.21.15.16:0/1138421823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33e8004810 msgr2=0x7f33e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.284+0000 7f3409800640 1 --2- 172.21.15.16:0/1138421823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33e8004810 0x7f33e8004c90 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009b60 tx=0x7f33ec01d250 comp rx=0 tx=0).stop 2024-09-10T21:18:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.284+0000 7f3403400640 1 -- 172.21.15.16:0/1138421823 reap_dead start 2024-09-10T21:18:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.284+0000 7f3409800640 1 -- 172.21.15.16:0/1138421823 shutdown_connections 2024-09-10T21:18:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.284+0000 7f3409800640 1 -- 172.21.15.16:0/1138421823 >> 172.21.15.16:0/1138421823 conn(0x7f34040cfb20 msgr2=0x7f34040cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.284+0000 7f3409800640 1 -- 172.21.15.16:0/1138421823 shutdown_connections 2024-09-10T21:18:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.284+0000 7f3409800640 1 -- 172.21.15.16:0/1138421823 wait complete. 2024-09-10T21:18:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3409800640 1 Processor -- start 2024-09-10T21:18:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3409800640 1 -- start start 2024-09-10T21:18:59.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3409800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33e8004810 0x7f34040c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:59.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3409800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34041f5250 0x7f34040b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3409800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f34040c6d10 0x7f34040b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3409800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3404072b50 con 0x7f34041f5250 2024-09-10T21:18:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3409800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f34040729d0 con 0x7f33e8004810 2024-09-10T21:18:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3409800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3404072cd0 con 0x7f34040c6d10 2024-09-10T21:18:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34041f5250 0x7f34040b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34041f5250 0x7f34040b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45538/0 (socket says 172.21.15.16:45538) 2024-09-10T21:18:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f33fbe00640 1 -- 172.21.15.16:0/1205434522 learned_addr learned my addr 172.21.15.16:0/1205434522 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3402a00640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33e8004810 0x7f34040c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3403400640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f34040c6d10 0x7f34040b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f33fbe00640 1 -- 172.21.15.16:0/1205434522 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f34040c6d10 msgr2=0x7f34040b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f33fbe00640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f34040c6d10 0x7f34040b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f33fbe00640 1 -- 172.21.15.16:0/1205434522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33e8004810 msgr2=0x7f34040c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f33fbe00640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33e8004810 0x7f34040c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f33fbe00640 1 -- 172.21.15.16:0/1205434522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34040b0aa0 con 0x7f34041f5250 2024-09-10T21:18:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3402a00640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33e8004810 0x7f34040c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f33fbe00640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34041f5250 0x7f34040b0560 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f33ec01d080 tx=0x7f33ec005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f33faa00640 1 -- 172.21.15.16:0/1205434522 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec0bd070 con 0x7f34041f5250 2024-09-10T21:18:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f33faa00640 1 -- 172.21.15.16:0/1205434522 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33ec0049c0 con 0x7f34041f5250 2024-09-10T21:18:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f33faa00640 1 -- 172.21.15.16:0/1205434522 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec0b1040 con 0x7f34041f5250 2024-09-10T21:18:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3409800640 1 -- 172.21.15.16:0/1205434522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34040c76e0 con 0x7f34041f5250 2024-09-10T21:18:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3409800640 1 -- 172.21.15.16:0/1205434522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34040c93f0 con 0x7f34041f5250 2024-09-10T21:18:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.288+0000 7f3409800640 1 -- 172.21.15.16:0/1205434522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34040ca170 con 0x7f34041f5250 2024-09-10T21:18:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.292+0000 7f33faa00640 1 -- 172.21.15.16:0/1205434522 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f33ec0c2440 con 0x7f34041f5250 2024-09-10T21:18:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.292+0000 7f33faa00640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33e0072610 0x7f33e0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.292+0000 7f33faa00640 1 -- 172.21.15.16:0/1205434522 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f33ec12d800 con 0x7f34041f5250 2024-09-10T21:18:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.292+0000 7f3402a00640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33e0072610 0x7f33e0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.292+0000 7f3402a00640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33e0072610 0x7f33e0074ad0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f33f4006fd0 tx=0x7f33f40067b0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:59.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.296+0000 7f33faa00640 1 -- 172.21.15.16:0/1205434522 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f33ec016fc0 con 0x7f34041f5250 2024-09-10T21:18:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f3409800640 1 -- 172.21.15.16:0/1205434522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f34040b5a50 con 0x7f34041f5250 2024-09-10T21:18:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f33faa00640 1 -- 172.21.15.16:0/1205434522 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1575 (secure 0 0 0) 0x7f33ec0f8880 con 0x7f34041f5250 2024-09-10T21:18:59.536 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:59.536 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":534,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":138,"num_osds":8,"num_up_osds":8,"osd_up_since":1726002618,"num_in_osds":8,"osd_in_since":1726002608,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":60},{"state_name":"active+clean","count":23},{"state_name":"unknown","count":14}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":300130304,"bytes_avail":772793982976,"bytes_total":773094113280,"unknown_pgs_ratio":0.14432989060878754,"inactive_pgs_ratio":0.6185566782951355},"fsmap":{"epoch":105,"btime":"2024-09-10T21:18:57:146722+0000","id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"b","status":"up:active","gid":8259}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-10T21:18:05.561892+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T21:18:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f3409800640 1 -- 172.21.15.16:0/1205434522 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33e0072610 msgr2=0x7f33e0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f3409800640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33e0072610 0x7f33e0074ad0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f33f4006fd0 tx=0x7f33f40067b0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f3409800640 1 -- 172.21.15.16:0/1205434522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34041f5250 msgr2=0x7f34040b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f3409800640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34041f5250 0x7f34040b0560 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f33ec01d080 tx=0x7f33ec005640 comp rx=0 tx=0).stop 2024-09-10T21:18:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f3409800640 1 -- 172.21.15.16:0/1205434522 shutdown_connections 2024-09-10T21:18:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f3409800640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33e0072610 0x7f33e0074ad0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f3409800640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f34040c6d10 0x7f34040b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f3409800640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34041f5250 0x7f34040b0560 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f3409800640 1 --2- 172.21.15.16:0/1205434522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33e8004810 0x7f34040c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f3409800640 1 -- 172.21.15.16:0/1205434522 >> 172.21.15.16:0/1205434522 conn(0x7f34040cfb20 msgr2=0x7f340415b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f3409800640 1 -- 172.21.15.16:0/1205434522 shutdown_connections 2024-09-10T21:18:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.532+0000 7f3409800640 1 -- 172.21.15.16:0/1205434522 wait complete. 2024-09-10T21:18:59.558 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T21:18:59.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:18:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.700+0000 7f77e0200640 1 -- 172.21.15.16:0/2735750495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0005920 msgr2=0x7f77c0005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.700+0000 7f77e0200640 1 --2- 172.21.15.16:0/2735750495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0005920 0x7f77c0005da0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009b30 tx=0x7f77c401d1e0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.700+0000 7f77e0200640 1 -- 172.21.15.16:0/2735750495 shutdown_connections 2024-09-10T21:18:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.700+0000 7f77e0200640 1 --2- 172.21.15.16:0/2735750495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0005920 0x7f77c0005da0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.700+0000 7f77e0200640 1 -- 172.21.15.16:0/2735750495 >> 172.21.15.16:0/2735750495 conn(0x7f77d80cfb20 msgr2=0x7f77d80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.700+0000 7f77e0200640 1 -- 172.21.15.16:0/2735750495 shutdown_connections 2024-09-10T21:18:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.700+0000 7f77e0200640 1 -- 172.21.15.16:0/2735750495 wait complete. 2024-09-10T21:18:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.700+0000 7f77e0200640 1 Processor -- start 2024-09-10T21:18:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.700+0000 7f77e0200640 1 -- start start 2024-09-10T21:18:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77e0200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0005920 0x7f77d81f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77e0200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77d81473e0 0x7f77d8157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77e0200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77d8157940 0x7f77d80b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77e0200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80ca2f0 con 0x7f77c0005920 2024-09-10T21:18:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77e0200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80ca170 con 0x7f77d8157940 2024-09-10T21:18:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77e0200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f77d80ca470 con 0x7f77d81473e0 2024-09-10T21:18:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77dd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0005920 0x7f77d81f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77dd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0005920 0x7f77d81f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45558/0 (socket says 172.21.15.16:45558) 2024-09-10T21:18:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77dd800640 1 -- 172.21.15.16:0/4145629966 learned_addr learned my addr 172.21.15.16:0/4145629966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:18:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77dce00640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77d81473e0 0x7f77d8157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77de200640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77d8157940 0x7f77d80b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77dd800640 1 -- 172.21.15.16:0/4145629966 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77d81473e0 msgr2=0x7f77d8157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77dd800640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77d81473e0 0x7f77d8157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77dd800640 1 -- 172.21.15.16:0/4145629966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77d8157940 msgr2=0x7f77d80b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77dd800640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77d8157940 0x7f77d80b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77dd800640 1 -- 172.21.15.16:0/4145629966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d8156360 con 0x7f77c0005920 2024-09-10T21:18:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77dd800640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0005920 0x7f77d81f87c0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f77cc003f30 tx=0x7f77cc001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77d5600640 1 -- 172.21.15.16:0/4145629966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc011640 con 0x7f77c0005920 2024-09-10T21:18:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77d5600640 1 -- 172.21.15.16:0/4145629966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77cc016020 con 0x7f77c0005920 2024-09-10T21:18:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77d5600640 1 -- 172.21.15.16:0/4145629966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc011a40 con 0x7f77c0005920 2024-09-10T21:18:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77e0200640 1 -- 172.21.15.16:0/4145629966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d8156640 con 0x7f77c0005920 2024-09-10T21:18:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77e0200640 1 -- 172.21.15.16:0/4145629966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d8156c00 con 0x7f77c0005920 2024-09-10T21:18:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77e0200640 1 -- 172.21.15.16:0/4145629966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d80ca9d0 con 0x7f77c0005920 2024-09-10T21:18:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.704+0000 7f77d5600640 1 -- 172.21.15.16:0/4145629966 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f77cc011be0 con 0x7f77c0005920 2024-09-10T21:18:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.708+0000 7f77d5600640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77b80725e0 0x7f77b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.708+0000 7f77d5600640 1 -- 172.21.15.16:0/4145629966 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f77cc093f70 con 0x7f77c0005920 2024-09-10T21:18:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.708+0000 7f77dce00640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77b80725e0 0x7f77b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.708+0000 7f77dce00640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77b80725e0 0x7f77b8074aa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f77c40ae000 tx=0x7f77c4019f80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.712+0000 7f77d5600640 1 -- 172.21.15.16:0/4145629966 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f77cc020020 con 0x7f77c0005920 2024-09-10T21:18:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.880+0000 7f77e0200640 1 -- 172.21.15.16:0/4145629966 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77d80c33f0 con 0x7f77b80725e0 2024-09-10T21:18:59.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.884+0000 7f77d5600640 1 -- 172.21.15.16:0/4145629966 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252474 (secure 0 0 0) 0x7f77d80c33f0 con 0x7f77b80725e0 2024-09-10T21:18:59.892 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:18:59.893 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:18:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.888+0000 7f77e0200640 1 -- 172.21.15.16:0/4145629966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77b80725e0 msgr2=0x7f77b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.888+0000 7f77e0200640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77b80725e0 0x7f77b8074aa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f77c40ae000 tx=0x7f77c4019f80 comp rx=0 tx=0).stop 2024-09-10T21:18:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.888+0000 7f77e0200640 1 -- 172.21.15.16:0/4145629966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0005920 msgr2=0x7f77d81f87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.888+0000 7f77e0200640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0005920 0x7f77d81f87c0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f77cc003f30 tx=0x7f77cc001700 comp rx=0 tx=0).stop 2024-09-10T21:18:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.888+0000 7f77e0200640 1 -- 172.21.15.16:0/4145629966 shutdown_connections 2024-09-10T21:18:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.888+0000 7f77e0200640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77b80725e0 0x7f77b8074aa0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.888+0000 7f77e0200640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77d8157940 0x7f77d80b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.888+0000 7f77e0200640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77d81473e0 0x7f77d8157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.888+0000 7f77e0200640 1 --2- 172.21.15.16:0/4145629966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0005920 0x7f77d81f87c0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.888+0000 7f77e0200640 1 -- 172.21.15.16:0/4145629966 >> 172.21.15.16:0/4145629966 conn(0x7f77d80cfb20 msgr2=0x7f77d80cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.888+0000 7f77e0200640 1 -- 172.21.15.16:0/4145629966 shutdown_connections 2024-09-10T21:18:59.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:18:59.888+0000 7f77e0200640 1 -- 172.21.15.16:0/4145629966 wait complete. 2024-09-10T21:18:59.913 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-10T21:18:59.575597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":297516,"kb_used_data":14716,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754677204,"statfs":{"total":773094113280,"available":772789456896,"internally_reserved":0,"allocated":15069184,"data_stored":7150176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.463373"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124518+0000","last_change":"2024-09-10T21:18:54.148585+0000","last_active":"2024-09-10T21:18:56.124518+0000","last_peered":"2024-09-10T21:18:56.124518+0000","last_clean":"2024-09-10T21:18:56.124518+0000","last_became_active":"2024-09-10T21:18:54.145516+0000","last_became_peered":"2024-09-10T21:18:54.145516+0000","last_unstale":"2024-09-10T21:18:56.124518+0000","last_undegraded":"2024-09-10T21:18:56.124518+0000","last_fullsized":"2024-09-10T21:18:56.124518+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:04.643193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124482+0000","last_change":"2024-09-10T21:18:54.143896+0000","last_active":"2024-09-10T21:18:56.124482+0000","last_peered":"2024-09-10T21:18:56.124482+0000","last_clean":"2024-09-10T21:18:56.124482+0000","last_became_active":"2024-09-10T21:18:54.143634+0000","last_became_peered":"2024-09-10T21:18:54.143634+0000","last_unstale":"2024-09-10T21:18:56.124482+0000","last_undegraded":"2024-09-10T21:18:56.124482+0000","last_fullsized":"2024-09-10T21:18:56.124482+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:36.226183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124299+0000","last_change":"2024-09-10T21:18:54.137010+0000","last_active":"2024-09-10T21:18:56.124299+0000","last_peered":"2024-09-10T21:18:56.124299+0000","last_clean":"2024-09-10T21:18:56.124299+0000","last_became_active":"2024-09-10T21:18:54.132490+0000","last_became_peered":"2024-09-10T21:18:54.132490+0000","last_unstale":"2024-09-10T21:18:56.124299+0000","last_undegraded":"2024-09-10T21:18:56.124299+0000","last_fullsized":"2024-09-10T21:18:56.124299+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:28.746632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118536+0000","last_change":"2024-09-10T21:18:54.151199+0000","last_active":"2024-09-10T21:18:55.118536+0000","last_peered":"2024-09-10T21:18:55.118536+0000","last_clean":"2024-09-10T21:18:55.118536+0000","last_became_active":"2024-09-10T21:18:54.150935+0000","last_became_peered":"2024-09-10T21:18:54.150935+0000","last_unstale":"2024-09-10T21:18:55.118536+0000","last_undegraded":"2024-09-10T21:18:55.118536+0000","last_fullsized":"2024-09-10T21:18:55.118536+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:11:43.194385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536453+0000","last_change":"2024-09-10T21:18:54.149942+0000","last_active":"2024-09-10T21:18:57.536453+0000","last_peered":"2024-09-10T21:18:57.536453+0000","last_clean":"2024-09-10T21:18:57.536453+0000","last_became_active":"2024-09-10T21:18:54.149641+0000","last_became_peered":"2024-09-10T21:18:54.149641+0000","last_unstale":"2024-09-10T21:18:57.536453+0000","last_undegraded":"2024-09-10T21:18:57.536453+0000","last_fullsized":"2024-09-10T21:18:57.536453+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:48.061606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.713961+0000","last_change":"2024-09-10T21:18:54.149259+0000","last_active":"2024-09-10T21:18:57.713961+0000","last_peered":"2024-09-10T21:18:57.713961+0000","last_clean":"2024-09-10T21:18:57.713961+0000","last_became_active":"2024-09-10T21:18:54.148593+0000","last_became_peered":"2024-09-10T21:18:54.148593+0000","last_unstale":"2024-09-10T21:18:57.713961+0000","last_undegraded":"2024-09-10T21:18:57.713961+0000","last_fullsized":"2024-09-10T21:18:57.713961+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:55.391340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412719+0000","last_change":"2024-09-10T21:18:54.141633+0000","last_active":"2024-09-10T21:18:55.412719+0000","last_peered":"2024-09-10T21:18:55.412719+0000","last_clean":"2024-09-10T21:18:55.412719+0000","last_became_active":"2024-09-10T21:18:54.141358+0000","last_became_peered":"2024-09-10T21:18:54.141358+0000","last_unstale":"2024-09-10T21:18:55.412719+0000","last_undegraded":"2024-09-10T21:18:55.412719+0000","last_fullsized":"2024-09-10T21:18:55.412719+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:13.382947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714433+0000","last_change":"2024-09-10T21:18:54.141560+0000","last_active":"2024-09-10T21:18:57.714433+0000","last_peered":"2024-09-10T21:18:57.714433+0000","last_clean":"2024-09-10T21:18:57.714433+0000","last_became_active":"2024-09-10T21:18:54.141005+0000","last_became_peered":"2024-09-10T21:18:54.141005+0000","last_unstale":"2024-09-10T21:18:57.714433+0000","last_undegraded":"2024-09-10T21:18:57.714433+0000","last_fullsized":"2024-09-10T21:18:57.714433+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:09.504837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714839+0000","last_change":"2024-09-10T21:18:54.145288+0000","last_active":"2024-09-10T21:18:57.714839+0000","last_peered":"2024-09-10T21:18:57.714839+0000","last_clean":"2024-09-10T21:18:57.714839+0000","last_became_active":"2024-09-10T21:18:54.145003+0000","last_became_peered":"2024-09-10T21:18:54.145003+0000","last_unstale":"2024-09-10T21:18:57.714839+0000","last_undegraded":"2024-09-10T21:18:57.714839+0000","last_fullsized":"2024-09-10T21:18:57.714839+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:38.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714735+0000","last_change":"2024-09-10T21:18:54.151547+0000","last_active":"2024-09-10T21:18:57.714735+0000","last_peered":"2024-09-10T21:18:57.714735+0000","last_clean":"2024-09-10T21:18:57.714735+0000","last_became_active":"2024-09-10T21:18:54.151254+0000","last_became_peered":"2024-09-10T21:18:54.151254+0000","last_unstale":"2024-09-10T21:18:57.714735+0000","last_undegraded":"2024-09-10T21:18:57.714735+0000","last_fullsized":"2024-09-10T21:18:57.714735+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:18.902652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536518+0000","last_change":"2024-09-10T21:18:54.138443+0000","last_active":"2024-09-10T21:18:57.536518+0000","last_peered":"2024-09-10T21:18:57.536518+0000","last_clean":"2024-09-10T21:18:57.536518+0000","last_became_active":"2024-09-10T21:18:54.138053+0000","last_became_peered":"2024-09-10T21:18:54.138053+0000","last_unstale":"2024-09-10T21:18:57.536518+0000","last_undegraded":"2024-09-10T21:18:57.536518+0000","last_fullsized":"2024-09-10T21:18:57.536518+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:41.987074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116858+0000","last_change":"2024-09-10T21:18:54.134839+0000","last_active":"2024-09-10T21:18:55.116858+0000","last_peered":"2024-09-10T21:18:55.116858+0000","last_clean":"2024-09-10T21:18:55.116858+0000","last_became_active":"2024-09-10T21:18:54.134552+0000","last_became_peered":"2024-09-10T21:18:54.134552+0000","last_unstale":"2024-09-10T21:18:55.116858+0000","last_undegraded":"2024-09-10T21:18:55.116858+0000","last_fullsized":"2024-09-10T21:18:55.116858+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:53.785552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116627+0000","last_change":"2024-09-10T21:18:54.129992+0000","last_active":"2024-09-10T21:18:55.116627+0000","last_peered":"2024-09-10T21:18:55.116627+0000","last_clean":"2024-09-10T21:18:55.116627+0000","last_became_active":"2024-09-10T21:18:54.129578+0000","last_became_peered":"2024-09-10T21:18:54.129578+0000","last_unstale":"2024-09-10T21:18:55.116627+0000","last_undegraded":"2024-09-10T21:18:55.116627+0000","last_fullsized":"2024-09-10T21:18:55.116627+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:39.550472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714963+0000","last_change":"2024-09-10T21:18:54.146240+0000","last_active":"2024-09-10T21:18:57.714963+0000","last_peered":"2024-09-10T21:18:57.714963+0000","last_clean":"2024-09-10T21:18:57.714963+0000","last_became_active":"2024-09-10T21:18:54.145952+0000","last_became_peered":"2024-09-10T21:18:54.145952+0000","last_unstale":"2024-09-10T21:18:57.714963+0000","last_undegraded":"2024-09-10T21:18:57.714963+0000","last_fullsized":"2024-09-10T21:18:57.714963+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:27.820474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714511+0000","last_change":"2024-09-10T21:18:54.149220+0000","last_active":"2024-09-10T21:18:57.714511+0000","last_peered":"2024-09-10T21:18:57.714511+0000","last_clean":"2024-09-10T21:18:57.714511+0000","last_became_active":"2024-09-10T21:18:54.143941+0000","last_became_peered":"2024-09-10T21:18:54.143941+0000","last_unstale":"2024-09-10T21:18:57.714511+0000","last_undegraded":"2024-09-10T21:18:57.714511+0000","last_fullsized":"2024-09-10T21:18:57.714511+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:03.393936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412492+0000","last_change":"2024-09-10T21:18:54.128306+0000","last_active":"2024-09-10T21:18:55.412492+0000","last_peered":"2024-09-10T21:18:55.412492+0000","last_clean":"2024-09-10T21:18:55.412492+0000","last_became_active":"2024-09-10T21:18:54.127837+0000","last_became_peered":"2024-09-10T21:18:54.127837+0000","last_unstale":"2024-09-10T21:18:55.412492+0000","last_undegraded":"2024-09-10T21:18:55.412492+0000","last_fullsized":"2024-09-10T21:18:55.412492+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:40.753501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714238+0000","last_change":"2024-09-10T21:18:54.150068+0000","last_active":"2024-09-10T21:18:57.714238+0000","last_peered":"2024-09-10T21:18:57.714238+0000","last_clean":"2024-09-10T21:18:57.714238+0000","last_became_active":"2024-09-10T21:18:54.149774+0000","last_became_peered":"2024-09-10T21:18:54.149774+0000","last_unstale":"2024-09-10T21:18:57.714238+0000","last_undegraded":"2024-09-10T21:18:57.714238+0000","last_fullsized":"2024-09-10T21:18:57.714238+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:44.793301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118759+0000","last_change":"2024-09-10T21:18:54.129484+0000","last_active":"2024-09-10T21:18:55.118759+0000","last_peered":"2024-09-10T21:18:55.118759+0000","last_clean":"2024-09-10T21:18:55.118759+0000","last_became_active":"2024-09-10T21:18:54.128927+0000","last_became_peered":"2024-09-10T21:18:54.128927+0000","last_unstale":"2024-09-10T21:18:55.118759+0000","last_undegraded":"2024-09-10T21:18:55.118759+0000","last_fullsized":"2024-09-10T21:18:55.118759+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:49.960348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412845+0000","last_change":"2024-09-10T21:18:54.135461+0000","last_active":"2024-09-10T21:18:55.412845+0000","last_peered":"2024-09-10T21:18:55.412845+0000","last_clean":"2024-09-10T21:18:55.412845+0000","last_became_active":"2024-09-10T21:18:54.135129+0000","last_became_peered":"2024-09-10T21:18:54.135129+0000","last_unstale":"2024-09-10T21:18:55.412845+0000","last_undegraded":"2024-09-10T21:18:55.412845+0000","last_fullsized":"2024-09-10T21:18:55.412845+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:27.802270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116954+0000","last_change":"2024-09-10T21:18:54.131703+0000","last_active":"2024-09-10T21:18:55.116954+0000","last_peered":"2024-09-10T21:18:55.116954+0000","last_clean":"2024-09-10T21:18:55.116954+0000","last_became_active":"2024-09-10T21:18:54.131469+0000","last_became_peered":"2024-09-10T21:18:54.131469+0000","last_unstale":"2024-09-10T21:18:55.116954+0000","last_undegraded":"2024-09-10T21:18:55.116954+0000","last_fullsized":"2024-09-10T21:18:55.116954+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:08.975945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714308+0000","last_change":"2024-09-10T21:18:54.155216+0000","last_active":"2024-09-10T21:18:57.714308+0000","last_peered":"2024-09-10T21:18:57.714308+0000","last_clean":"2024-09-10T21:18:57.714308+0000","last_became_active":"2024-09-10T21:18:54.154821+0000","last_became_peered":"2024-09-10T21:18:54.154821+0000","last_unstale":"2024-09-10T21:18:57.714308+0000","last_undegraded":"2024-09-10T21:18:57.714308+0000","last_fullsized":"2024-09-10T21:18:57.714308+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:18.510060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536449+0000","last_change":"2024-09-10T21:18:54.136990+0000","last_active":"2024-09-10T21:18:57.536449+0000","last_peered":"2024-09-10T21:18:57.536449+0000","last_clean":"2024-09-10T21:18:57.536449+0000","last_became_active":"2024-09-10T21:18:54.136700+0000","last_became_peered":"2024-09-10T21:18:54.136700+0000","last_unstale":"2024-09-10T21:18:57.536449+0000","last_undegraded":"2024-09-10T21:18:57.536449+0000","last_fullsized":"2024-09-10T21:18:57.536449+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:58.842730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117448+0000","last_change":"2024-09-10T21:18:54.130109+0000","last_active":"2024-09-10T21:18:55.117448+0000","last_peered":"2024-09-10T21:18:55.117448+0000","last_clean":"2024-09-10T21:18:55.117448+0000","last_became_active":"2024-09-10T21:18:54.129818+0000","last_became_peered":"2024-09-10T21:18:54.129818+0000","last_unstale":"2024-09-10T21:18:55.117448+0000","last_undegraded":"2024-09-10T21:18:55.117448+0000","last_fullsized":"2024-09-10T21:18:55.117448+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:06.371859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536746+0000","last_change":"2024-09-10T21:18:54.151570+0000","last_active":"2024-09-10T21:18:57.536746+0000","last_peered":"2024-09-10T21:18:57.536746+0000","last_clean":"2024-09-10T21:18:57.536746+0000","last_became_active":"2024-09-10T21:18:54.151323+0000","last_became_peered":"2024-09-10T21:18:54.151323+0000","last_unstale":"2024-09-10T21:18:57.536746+0000","last_undegraded":"2024-09-10T21:18:57.536746+0000","last_fullsized":"2024-09-10T21:18:57.536746+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:55.961963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536558+0000","last_change":"2024-09-10T21:18:54.150942+0000","last_active":"2024-09-10T21:18:57.536558+0000","last_peered":"2024-09-10T21:18:57.536558+0000","last_clean":"2024-09-10T21:18:57.536558+0000","last_became_active":"2024-09-10T21:18:54.147390+0000","last_became_peered":"2024-09-10T21:18:54.147390+0000","last_unstale":"2024-09-10T21:18:57.536558+0000","last_undegraded":"2024-09-10T21:18:57.536558+0000","last_fullsized":"2024-09-10T21:18:57.536558+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:47.191704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117961+0000","last_change":"2024-09-10T21:18:54.144696+0000","last_active":"2024-09-10T21:18:55.117961+0000","last_peered":"2024-09-10T21:18:55.117961+0000","last_clean":"2024-09-10T21:18:55.117961+0000","last_became_active":"2024-09-10T21:18:54.144114+0000","last_became_peered":"2024-09-10T21:18:54.144114+0000","last_unstale":"2024-09-10T21:18:55.117961+0000","last_undegraded":"2024-09-10T21:18:55.117961+0000","last_fullsized":"2024-09-10T21:18:55.117961+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:13.343963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536821+0000","last_change":"2024-09-10T21:18:54.135462+0000","last_active":"2024-09-10T21:18:57.536821+0000","last_peered":"2024-09-10T21:18:57.536821+0000","last_clean":"2024-09-10T21:18:57.536821+0000","last_became_active":"2024-09-10T21:18:54.135193+0000","last_became_peered":"2024-09-10T21:18:54.135193+0000","last_unstale":"2024-09-10T21:18:57.536821+0000","last_undegraded":"2024-09-10T21:18:57.536821+0000","last_fullsized":"2024-09-10T21:18:57.536821+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:13.577636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412739+0000","last_change":"2024-09-10T21:18:54.140632+0000","last_active":"2024-09-10T21:18:55.412739+0000","last_peered":"2024-09-10T21:18:55.412739+0000","last_clean":"2024-09-10T21:18:55.412739+0000","last_became_active":"2024-09-10T21:18:54.140193+0000","last_became_peered":"2024-09-10T21:18:54.140193+0000","last_unstale":"2024-09-10T21:18:55.412739+0000","last_undegraded":"2024-09-10T21:18:55.412739+0000","last_fullsized":"2024-09-10T21:18:55.412739+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:00.133528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124200+0000","last_change":"2024-09-10T21:18:54.137981+0000","last_active":"2024-09-10T21:18:56.124200+0000","last_peered":"2024-09-10T21:18:56.124200+0000","last_clean":"2024-09-10T21:18:56.124200+0000","last_became_active":"2024-09-10T21:18:54.136200+0000","last_became_peered":"2024-09-10T21:18:54.136200+0000","last_unstale":"2024-09-10T21:18:56.124200+0000","last_undegraded":"2024-09-10T21:18:56.124200+0000","last_fullsized":"2024-09-10T21:18:56.124200+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:49.986865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714458+0000","last_change":"2024-09-10T21:18:54.136050+0000","last_active":"2024-09-10T21:18:57.714458+0000","last_peered":"2024-09-10T21:18:57.714458+0000","last_clean":"2024-09-10T21:18:57.714458+0000","last_became_active":"2024-09-10T21:18:54.134760+0000","last_became_peered":"2024-09-10T21:18:54.134760+0000","last_unstale":"2024-09-10T21:18:57.714458+0000","last_undegraded":"2024-09-10T21:18:57.714458+0000","last_fullsized":"2024-09-10T21:18:57.714458+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:55.225422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.123953+0000","last_change":"2024-09-10T21:18:54.128530+0000","last_active":"2024-09-10T21:18:56.123953+0000","last_peered":"2024-09-10T21:18:56.123953+0000","last_clean":"2024-09-10T21:18:56.123953+0000","last_became_active":"2024-09-10T21:18:54.128263+0000","last_became_peered":"2024-09-10T21:18:54.128263+0000","last_unstale":"2024-09-10T21:18:56.123953+0000","last_undegraded":"2024-09-10T21:18:56.123953+0000","last_fullsized":"2024-09-10T21:18:56.123953+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:09.259466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714226+0000","last_change":"2024-09-10T21:18:54.146925+0000","last_active":"2024-09-10T21:18:57.714226+0000","last_peered":"2024-09-10T21:18:57.714226+0000","last_clean":"2024-09-10T21:18:57.714226+0000","last_became_active":"2024-09-10T21:18:54.146621+0000","last_became_peered":"2024-09-10T21:18:54.146621+0000","last_unstale":"2024-09-10T21:18:57.714226+0000","last_undegraded":"2024-09-10T21:18:57.714226+0000","last_fullsized":"2024-09-10T21:18:57.714226+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:21.444679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118922+0000","last_change":"2024-09-10T21:18:54.132032+0000","last_active":"2024-09-10T21:18:55.118922+0000","last_peered":"2024-09-10T21:18:55.118922+0000","last_clean":"2024-09-10T21:18:55.118922+0000","last_became_active":"2024-09-10T21:18:54.131736+0000","last_became_peered":"2024-09-10T21:18:54.131736+0000","last_unstale":"2024-09-10T21:18:55.118922+0000","last_undegraded":"2024-09-10T21:18:55.118922+0000","last_fullsized":"2024-09-10T21:18:55.118922+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:50.696171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"137'4","reported_seq":19,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124672+0000","last_change":"2024-09-10T21:18:53.122058+0000","last_active":"2024-09-10T21:18:56.124672+0000","last_peered":"2024-09-10T21:18:56.124672+0000","last_clean":"2024-09-10T21:18:56.124672+0000","last_became_active":"2024-09-10T21:18:53.121768+0000","last_became_peered":"2024-09-10T21:18:53.121768+0000","last_unstale":"2024-09-10T21:18:56.124672+0000","last_undegraded":"2024-09-10T21:18:56.124672+0000","last_fullsized":"2024-09-10T21:18:56.124672+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:52.042309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124591+0000","last_change":"2024-09-10T21:18:54.128927+0000","last_active":"2024-09-10T21:18:56.124591+0000","last_peered":"2024-09-10T21:18:56.124591+0000","last_clean":"2024-09-10T21:18:56.124591+0000","last_became_active":"2024-09-10T21:18:54.128107+0000","last_became_peered":"2024-09-10T21:18:54.128107+0000","last_unstale":"2024-09-10T21:18:56.124591+0000","last_undegraded":"2024-09-10T21:18:56.124591+0000","last_fullsized":"2024-09-10T21:18:56.124591+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:01.643551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714790+0000","last_change":"2024-09-10T21:18:53.134253+0000","last_active":"2024-09-10T21:18:57.714790+0000","last_peered":"2024-09-10T21:18:57.714790+0000","last_clean":"2024-09-10T21:18:57.714790+0000","last_became_active":"2024-09-10T21:18:53.133971+0000","last_became_peered":"2024-09-10T21:18:53.133971+0000","last_unstale":"2024-09-10T21:18:57.714790+0000","last_undegraded":"2024-09-10T21:18:57.714790+0000","last_fullsized":"2024-09-10T21:18:57.714790+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:22.649998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714499+0000","last_change":"2024-09-10T21:18:54.149088+0000","last_active":"2024-09-10T21:18:57.714499+0000","last_peered":"2024-09-10T21:18:57.714499+0000","last_clean":"2024-09-10T21:18:57.714499+0000","last_became_active":"2024-09-10T21:18:54.148557+0000","last_became_peered":"2024-09-10T21:18:54.148557+0000","last_unstale":"2024-09-10T21:18:57.714499+0000","last_undegraded":"2024-09-10T21:18:57.714499+0000","last_fullsized":"2024-09-10T21:18:57.714499+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:37.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.713801+0000","last_change":"2024-09-10T21:18:53.138535+0000","last_active":"2024-09-10T21:18:57.713801+0000","last_peered":"2024-09-10T21:18:57.713801+0000","last_clean":"2024-09-10T21:18:57.713801+0000","last_became_active":"2024-09-10T21:18:53.138234+0000","last_became_peered":"2024-09-10T21:18:53.138234+0000","last_unstale":"2024-09-10T21:18:57.713801+0000","last_undegraded":"2024-09-10T21:18:57.713801+0000","last_fullsized":"2024-09-10T21:18:57.713801+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:03.695792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714773+0000","last_change":"2024-09-10T21:18:54.142121+0000","last_active":"2024-09-10T21:18:57.714773+0000","last_peered":"2024-09-10T21:18:57.714773+0000","last_clean":"2024-09-10T21:18:57.714773+0000","last_became_active":"2024-09-10T21:18:54.141515+0000","last_became_peered":"2024-09-10T21:18:54.141515+0000","last_unstale":"2024-09-10T21:18:57.714773+0000","last_undegraded":"2024-09-10T21:18:57.714773+0000","last_fullsized":"2024-09-10T21:18:57.714773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:53.049049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117745+0000","last_change":"2024-09-10T21:18:53.132095+0000","last_active":"2024-09-10T21:18:55.117745+0000","last_peered":"2024-09-10T21:18:55.117745+0000","last_clean":"2024-09-10T21:18:55.117745+0000","last_became_active":"2024-09-10T21:18:53.131751+0000","last_became_peered":"2024-09-10T21:18:53.131751+0000","last_unstale":"2024-09-10T21:18:55.117745+0000","last_undegraded":"2024-09-10T21:18:55.117745+0000","last_fullsized":"2024-09-10T21:18:55.117745+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:39.746760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118528+0000","last_change":"2024-09-10T21:18:54.132329+0000","last_active":"2024-09-10T21:18:55.118528+0000","last_peered":"2024-09-10T21:18:55.118528+0000","last_clean":"2024-09-10T21:18:55.118528+0000","last_became_active":"2024-09-10T21:18:54.132042+0000","last_became_peered":"2024-09-10T21:18:54.132042+0000","last_unstale":"2024-09-10T21:18:55.118528+0000","last_undegraded":"2024-09-10T21:18:55.118528+0000","last_fullsized":"2024-09-10T21:18:55.118528+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:34.600677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124219+0000","last_change":"2024-09-10T21:18:53.135048+0000","last_active":"2024-09-10T21:18:56.124219+0000","last_peered":"2024-09-10T21:18:56.124219+0000","last_clean":"2024-09-10T21:18:56.124219+0000","last_became_active":"2024-09-10T21:18:53.134366+0000","last_became_peered":"2024-09-10T21:18:53.134366+0000","last_unstale":"2024-09-10T21:18:56.124219+0000","last_undegraded":"2024-09-10T21:18:56.124219+0000","last_fullsized":"2024-09-10T21:18:56.124219+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:17.895912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714022+0000","last_change":"2024-09-10T21:18:54.146108+0000","last_active":"2024-09-10T21:18:57.714022+0000","last_peered":"2024-09-10T21:18:57.714022+0000","last_clean":"2024-09-10T21:18:57.714022+0000","last_became_active":"2024-09-10T21:18:54.145835+0000","last_became_peered":"2024-09-10T21:18:54.145835+0000","last_unstale":"2024-09-10T21:18:57.714022+0000","last_undegraded":"2024-09-10T21:18:57.714022+0000","last_fullsized":"2024-09-10T21:18:57.714022+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:42.657250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118681+0000","last_change":"2024-09-10T21:18:53.130356+0000","last_active":"2024-09-10T21:18:55.118681+0000","last_peered":"2024-09-10T21:18:55.118681+0000","last_clean":"2024-09-10T21:18:55.118681+0000","last_became_active":"2024-09-10T21:18:53.130043+0000","last_became_peered":"2024-09-10T21:18:53.130043+0000","last_unstale":"2024-09-10T21:18:55.118681+0000","last_undegraded":"2024-09-10T21:18:55.118681+0000","last_fullsized":"2024-09-10T21:18:55.118681+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:49.589673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117030+0000","last_change":"2024-09-10T21:18:54.136233+0000","last_active":"2024-09-10T21:18:55.117030+0000","last_peered":"2024-09-10T21:18:55.117030+0000","last_clean":"2024-09-10T21:18:55.117030+0000","last_became_active":"2024-09-10T21:18:54.135939+0000","last_became_peered":"2024-09-10T21:18:54.135939+0000","last_unstale":"2024-09-10T21:18:55.117030+0000","last_undegraded":"2024-09-10T21:18:55.117030+0000","last_fullsized":"2024-09-10T21:18:55.117030+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:22.677132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124139+0000","last_change":"2024-09-10T21:18:53.129464+0000","last_active":"2024-09-10T21:18:56.124139+0000","last_peered":"2024-09-10T21:18:56.124139+0000","last_clean":"2024-09-10T21:18:56.124139+0000","last_became_active":"2024-09-10T21:18:53.129179+0000","last_became_peered":"2024-09-10T21:18:53.129179+0000","last_unstale":"2024-09-10T21:18:56.124139+0000","last_undegraded":"2024-09-10T21:18:56.124139+0000","last_fullsized":"2024-09-10T21:18:56.124139+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:58.176727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117072+0000","last_change":"2024-09-10T21:18:54.130958+0000","last_active":"2024-09-10T21:18:55.117072+0000","last_peered":"2024-09-10T21:18:55.117072+0000","last_clean":"2024-09-10T21:18:55.117072+0000","last_became_active":"2024-09-10T21:18:54.130681+0000","last_became_peered":"2024-09-10T21:18:54.130681+0000","last_unstale":"2024-09-10T21:18:55.117072+0000","last_undegraded":"2024-09-10T21:18:55.117072+0000","last_fullsized":"2024-09-10T21:18:55.117072+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:23.939236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116973+0000","last_change":"2024-09-10T21:18:53.127488+0000","last_active":"2024-09-10T21:18:55.116973+0000","last_peered":"2024-09-10T21:18:55.116973+0000","last_clean":"2024-09-10T21:18:55.116973+0000","last_became_active":"2024-09-10T21:18:53.125698+0000","last_became_peered":"2024-09-10T21:18:53.125698+0000","last_unstale":"2024-09-10T21:18:55.116973+0000","last_undegraded":"2024-09-10T21:18:55.116973+0000","last_fullsized":"2024-09-10T21:18:55.116973+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:29.175487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412963+0000","last_change":"2024-09-10T21:18:54.143008+0000","last_active":"2024-09-10T21:18:55.412963+0000","last_peered":"2024-09-10T21:18:55.412963+0000","last_clean":"2024-09-10T21:18:55.412963+0000","last_became_active":"2024-09-10T21:18:54.142077+0000","last_became_peered":"2024-09-10T21:18:54.142077+0000","last_unstale":"2024-09-10T21:18:55.412963+0000","last_undegraded":"2024-09-10T21:18:55.412963+0000","last_fullsized":"2024-09-10T21:18:55.412963+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:14.772219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714227+0000","last_change":"2024-09-10T21:18:53.137421+0000","last_active":"2024-09-10T21:18:57.714227+0000","last_peered":"2024-09-10T21:18:57.714227+0000","last_clean":"2024-09-10T21:18:57.714227+0000","last_became_active":"2024-09-10T21:18:53.136850+0000","last_became_peered":"2024-09-10T21:18:53.136850+0000","last_unstale":"2024-09-10T21:18:57.714227+0000","last_undegraded":"2024-09-10T21:18:57.714227+0000","last_fullsized":"2024-09-10T21:18:57.714227+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:54.955222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412863+0000","last_change":"2024-09-10T21:18:54.142684+0000","last_active":"2024-09-10T21:18:55.412863+0000","last_peered":"2024-09-10T21:18:55.412863+0000","last_clean":"2024-09-10T21:18:55.412863+0000","last_became_active":"2024-09-10T21:18:54.142361+0000","last_became_peered":"2024-09-10T21:18:54.142361+0000","last_unstale":"2024-09-10T21:18:55.412863+0000","last_undegraded":"2024-09-10T21:18:55.412863+0000","last_fullsized":"2024-09-10T21:18:55.412863+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:37.685415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"137'3","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117466+0000","last_change":"2024-09-10T21:18:53.134405+0000","last_active":"2024-09-10T21:18:55.117466+0000","last_peered":"2024-09-10T21:18:55.117466+0000","last_clean":"2024-09-10T21:18:55.117466+0000","last_became_active":"2024-09-10T21:18:53.134012+0000","last_became_peered":"2024-09-10T21:18:53.134012+0000","last_unstale":"2024-09-10T21:18:55.117466+0000","last_undegraded":"2024-09-10T21:18:55.117466+0000","last_fullsized":"2024-09-10T21:18:55.117466+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:59.126787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124304+0000","last_change":"2024-09-10T21:18:54.137307+0000","last_active":"2024-09-10T21:18:56.124304+0000","last_peered":"2024-09-10T21:18:56.124304+0000","last_clean":"2024-09-10T21:18:56.124304+0000","last_became_active":"2024-09-10T21:18:54.135312+0000","last_became_peered":"2024-09-10T21:18:54.135312+0000","last_unstale":"2024-09-10T21:18:56.124304+0000","last_undegraded":"2024-09-10T21:18:56.124304+0000","last_fullsized":"2024-09-10T21:18:56.124304+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:02.586421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"137'6","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536576+0000","last_change":"2024-09-10T21:18:53.135320+0000","last_active":"2024-09-10T21:18:57.536576+0000","last_peered":"2024-09-10T21:18:57.536576+0000","last_clean":"2024-09-10T21:18:57.536576+0000","last_became_active":"2024-09-10T21:18:53.134966+0000","last_became_peered":"2024-09-10T21:18:53.134966+0000","last_unstale":"2024-09-10T21:18:57.536576+0000","last_undegraded":"2024-09-10T21:18:57.536576+0000","last_fullsized":"2024-09-10T21:18:57.536576+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:43.118277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412589+0000","last_change":"2024-09-10T21:18:54.135436+0000","last_active":"2024-09-10T21:18:55.412589+0000","last_peered":"2024-09-10T21:18:55.412589+0000","last_clean":"2024-09-10T21:18:55.412589+0000","last_became_active":"2024-09-10T21:18:54.135110+0000","last_became_peered":"2024-09-10T21:18:54.135110+0000","last_unstale":"2024-09-10T21:18:55.412589+0000","last_undegraded":"2024-09-10T21:18:55.412589+0000","last_fullsized":"2024-09-10T21:18:55.412589+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:50.596414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118490+0000","last_change":"2024-09-10T21:18:53.127007+0000","last_active":"2024-09-10T21:18:55.118490+0000","last_peered":"2024-09-10T21:18:55.118490+0000","last_clean":"2024-09-10T21:18:55.118490+0000","last_became_active":"2024-09-10T21:18:53.126673+0000","last_became_peered":"2024-09-10T21:18:53.126673+0000","last_unstale":"2024-09-10T21:18:55.118490+0000","last_undegraded":"2024-09-10T21:18:55.118490+0000","last_fullsized":"2024-09-10T21:18:55.118490+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:59.126787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117775+0000","last_change":"2024-09-10T21:18:54.147288+0000","last_active":"2024-09-10T21:18:55.117775+0000","last_peered":"2024-09-10T21:18:55.117775+0000","last_clean":"2024-09-10T21:18:55.117775+0000","last_became_active":"2024-09-10T21:18:54.145370+0000","last_became_peered":"2024-09-10T21:18:54.145370+0000","last_unstale":"2024-09-10T21:18:55.117775+0000","last_undegraded":"2024-09-10T21:18:55.117775+0000","last_fullsized":"2024-09-10T21:18:55.117775+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:30.693628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714096+0000","last_change":"2024-09-10T21:18:53.133200+0000","last_active":"2024-09-10T21:18:57.714096+0000","last_peered":"2024-09-10T21:18:57.714096+0000","last_clean":"2024-09-10T21:18:57.714096+0000","last_became_active":"2024-09-10T21:18:53.132906+0000","last_became_peered":"2024-09-10T21:18:53.132906+0000","last_unstale":"2024-09-10T21:18:57.714096+0000","last_undegraded":"2024-09-10T21:18:57.714096+0000","last_fullsized":"2024-09-10T21:18:57.714096+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:02.410211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.713910+0000","last_change":"2024-09-10T21:18:54.148718+0000","last_active":"2024-09-10T21:18:57.713910+0000","last_peered":"2024-09-10T21:18:57.713910+0000","last_clean":"2024-09-10T21:18:57.713910+0000","last_became_active":"2024-09-10T21:18:54.148438+0000","last_became_peered":"2024-09-10T21:18:54.148438+0000","last_unstale":"2024-09-10T21:18:57.713910+0000","last_undegraded":"2024-09-10T21:18:57.713910+0000","last_fullsized":"2024-09-10T21:18:57.713910+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:53.066340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412710+0000","last_change":"2024-09-10T21:18:53.137417+0000","last_active":"2024-09-10T21:18:55.412710+0000","last_peered":"2024-09-10T21:18:55.412710+0000","last_clean":"2024-09-10T21:18:55.412710+0000","last_became_active":"2024-09-10T21:18:53.137042+0000","last_became_peered":"2024-09-10T21:18:53.137042+0000","last_unstale":"2024-09-10T21:18:55.412710+0000","last_undegraded":"2024-09-10T21:18:55.412710+0000","last_fullsized":"2024-09-10T21:18:55.412710+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:00.755770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714511+0000","last_change":"2024-09-10T21:18:54.137500+0000","last_active":"2024-09-10T21:18:57.714511+0000","last_peered":"2024-09-10T21:18:57.714511+0000","last_clean":"2024-09-10T21:18:57.714511+0000","last_became_active":"2024-09-10T21:18:54.136137+0000","last_became_peered":"2024-09-10T21:18:54.136137+0000","last_unstale":"2024-09-10T21:18:57.714511+0000","last_undegraded":"2024-09-10T21:18:57.714511+0000","last_fullsized":"2024-09-10T21:18:57.714511+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:59.183467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"137'8","reported_seq":23,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536249+0000","last_change":"2024-09-10T21:18:53.135369+0000","last_active":"2024-09-10T21:18:57.536249+0000","last_peered":"2024-09-10T21:18:57.536249+0000","last_clean":"2024-09-10T21:18:57.536249+0000","last_became_active":"2024-09-10T21:18:53.133318+0000","last_became_peered":"2024-09-10T21:18:53.133318+0000","last_unstale":"2024-09-10T21:18:57.536249+0000","last_undegraded":"2024-09-10T21:18:57.536249+0000","last_fullsized":"2024-09-10T21:18:57.536249+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:52.963942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.413070+0000","last_change":"2024-09-10T21:18:54.126929+0000","last_active":"2024-09-10T21:18:55.413070+0000","last_peered":"2024-09-10T21:18:55.413070+0000","last_clean":"2024-09-10T21:18:55.413070+0000","last_became_active":"2024-09-10T21:18:54.126664+0000","last_became_peered":"2024-09-10T21:18:54.126664+0000","last_unstale":"2024-09-10T21:18:55.413070+0000","last_undegraded":"2024-09-10T21:18:55.413070+0000","last_fullsized":"2024-09-10T21:18:55.413070+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:08.370813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536672+0000","last_change":"2024-09-10T21:18:53.139526+0000","last_active":"2024-09-10T21:18:57.536672+0000","last_peered":"2024-09-10T21:18:57.536672+0000","last_clean":"2024-09-10T21:18:57.536672+0000","last_became_active":"2024-09-10T21:18:53.139137+0000","last_became_peered":"2024-09-10T21:18:53.139137+0000","last_unstale":"2024-09-10T21:18:57.536672+0000","last_undegraded":"2024-09-10T21:18:57.536672+0000","last_fullsized":"2024-09-10T21:18:57.536672+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:30.445921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.119156+0000","last_change":"2024-09-10T21:18:54.140068+0000","last_active":"2024-09-10T21:18:55.119156+0000","last_peered":"2024-09-10T21:18:55.119156+0000","last_clean":"2024-09-10T21:18:55.119156+0000","last_became_active":"2024-09-10T21:18:54.139359+0000","last_became_peered":"2024-09-10T21:18:54.139359+0000","last_unstale":"2024-09-10T21:18:55.119156+0000","last_undegraded":"2024-09-10T21:18:55.119156+0000","last_fullsized":"2024-09-10T21:18:55.119156+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:24.562500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714907+0000","last_change":"2024-09-10T21:18:53.136538+0000","last_active":"2024-09-10T21:18:57.714907+0000","last_peered":"2024-09-10T21:18:57.714907+0000","last_clean":"2024-09-10T21:18:57.714907+0000","last_became_active":"2024-09-10T21:18:53.135006+0000","last_became_peered":"2024-09-10T21:18:53.135006+0000","last_unstale":"2024-09-10T21:18:57.714907+0000","last_undegraded":"2024-09-10T21:18:57.714907+0000","last_fullsized":"2024-09-10T21:18:57.714907+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:25.364169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118736+0000","last_change":"2024-09-10T21:18:54.131993+0000","last_active":"2024-09-10T21:18:55.118736+0000","last_peered":"2024-09-10T21:18:55.118736+0000","last_clean":"2024-09-10T21:18:55.118736+0000","last_became_active":"2024-09-10T21:18:54.131682+0000","last_became_peered":"2024-09-10T21:18:54.131682+0000","last_unstale":"2024-09-10T21:18:55.118736+0000","last_undegraded":"2024-09-10T21:18:55.118736+0000","last_fullsized":"2024-09-10T21:18:55.118736+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:47:49.255213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536749+0000","last_change":"2024-09-10T21:18:53.138468+0000","last_active":"2024-09-10T21:18:57.536749+0000","last_peered":"2024-09-10T21:18:57.536749+0000","last_clean":"2024-09-10T21:18:57.536749+0000","last_became_active":"2024-09-10T21:18:53.138185+0000","last_became_peered":"2024-09-10T21:18:53.138185+0000","last_unstale":"2024-09-10T21:18:57.536749+0000","last_undegraded":"2024-09-10T21:18:57.536749+0000","last_fullsized":"2024-09-10T21:18:57.536749+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:27:45.345741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117457+0000","last_change":"2024-09-10T21:18:54.130564+0000","last_active":"2024-09-10T21:18:55.117457+0000","last_peered":"2024-09-10T21:18:55.117457+0000","last_clean":"2024-09-10T21:18:55.117457+0000","last_became_active":"2024-09-10T21:18:54.130232+0000","last_became_peered":"2024-09-10T21:18:54.130232+0000","last_unstale":"2024-09-10T21:18:55.117457+0000","last_undegraded":"2024-09-10T21:18:55.117457+0000","last_fullsized":"2024-09-10T21:18:55.117457+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:00.051016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117000+0000","last_change":"2024-09-10T21:18:53.126832+0000","last_active":"2024-09-10T21:18:55.117000+0000","last_peered":"2024-09-10T21:18:55.117000+0000","last_clean":"2024-09-10T21:18:55.117000+0000","last_became_active":"2024-09-10T21:18:53.126580+0000","last_became_peered":"2024-09-10T21:18:53.126580+0000","last_unstale":"2024-09-10T21:18:55.117000+0000","last_undegraded":"2024-09-10T21:18:55.117000+0000","last_fullsized":"2024-09-10T21:18:55.117000+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:02.216397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.123947+0000","last_change":"2024-09-10T21:18:54.129175+0000","last_active":"2024-09-10T21:18:56.123947+0000","last_peered":"2024-09-10T21:18:56.123947+0000","last_clean":"2024-09-10T21:18:56.123947+0000","last_became_active":"2024-09-10T21:18:54.128849+0000","last_became_peered":"2024-09-10T21:18:54.128849+0000","last_unstale":"2024-09-10T21:18:56.123947+0000","last_undegraded":"2024-09-10T21:18:56.123947+0000","last_fullsized":"2024-09-10T21:18:56.123947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:40.590604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118066+0000","last_change":"2024-09-10T21:18:53.129128+0000","last_active":"2024-09-10T21:18:55.118066+0000","last_peered":"2024-09-10T21:18:55.118066+0000","last_clean":"2024-09-10T21:18:55.118066+0000","last_became_active":"2024-09-10T21:18:53.128826+0000","last_became_peered":"2024-09-10T21:18:53.128826+0000","last_unstale":"2024-09-10T21:18:55.118066+0000","last_undegraded":"2024-09-10T21:18:55.118066+0000","last_fullsized":"2024-09-10T21:18:55.118066+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:12.376207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118060+0000","last_change":"2024-09-10T21:18:54.148318+0000","last_active":"2024-09-10T21:18:55.118060+0000","last_peered":"2024-09-10T21:18:55.118060+0000","last_clean":"2024-09-10T21:18:55.118060+0000","last_became_active":"2024-09-10T21:18:54.145050+0000","last_became_peered":"2024-09-10T21:18:54.145050+0000","last_unstale":"2024-09-10T21:18:55.118060+0000","last_undegraded":"2024-09-10T21:18:55.118060+0000","last_fullsized":"2024-09-10T21:18:55.118060+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:14.320404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116471+0000","last_change":"2024-09-10T21:18:53.128814+0000","last_active":"2024-09-10T21:18:55.116471+0000","last_peered":"2024-09-10T21:18:55.116471+0000","last_clean":"2024-09-10T21:18:55.116471+0000","last_became_active":"2024-09-10T21:18:53.128474+0000","last_became_peered":"2024-09-10T21:18:53.128474+0000","last_unstale":"2024-09-10T21:18:55.116471+0000","last_undegraded":"2024-09-10T21:18:55.116471+0000","last_fullsized":"2024-09-10T21:18:55.116471+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:43.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.123874+0000","last_change":"2024-09-10T21:18:54.123449+0000","last_active":"2024-09-10T21:18:56.123874+0000","last_peered":"2024-09-10T21:18:56.123874+0000","last_clean":"2024-09-10T21:18:56.123874+0000","last_became_active":"2024-09-10T21:18:54.123152+0000","last_became_peered":"2024-09-10T21:18:54.123152+0000","last_unstale":"2024-09-10T21:18:56.123874+0000","last_undegraded":"2024-09-10T21:18:56.123874+0000","last_fullsized":"2024-09-10T21:18:56.123874+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:14.790647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117929+0000","last_change":"2024-09-10T21:18:53.132527+0000","last_active":"2024-09-10T21:18:55.117929+0000","last_peered":"2024-09-10T21:18:55.117929+0000","last_clean":"2024-09-10T21:18:55.117929+0000","last_became_active":"2024-09-10T21:18:53.130251+0000","last_became_peered":"2024-09-10T21:18:53.130251+0000","last_unstale":"2024-09-10T21:18:55.117929+0000","last_undegraded":"2024-09-10T21:18:55.117929+0000","last_fullsized":"2024-09-10T21:18:55.117929+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:36.678675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412524+0000","last_change":"2024-09-10T21:18:54.137449+0000","last_active":"2024-09-10T21:18:55.412524+0000","last_peered":"2024-09-10T21:18:55.412524+0000","last_clean":"2024-09-10T21:18:55.412524+0000","last_became_active":"2024-09-10T21:18:54.137183+0000","last_became_peered":"2024-09-10T21:18:54.137183+0000","last_unstale":"2024-09-10T21:18:55.412524+0000","last_undegraded":"2024-09-10T21:18:55.412524+0000","last_fullsized":"2024-09-10T21:18:55.412524+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:03.697968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116541+0000","last_change":"2024-09-10T21:18:53.133231+0000","last_active":"2024-09-10T21:18:55.116541+0000","last_peered":"2024-09-10T21:18:55.116541+0000","last_clean":"2024-09-10T21:18:55.116541+0000","last_became_active":"2024-09-10T21:18:53.132930+0000","last_became_peered":"2024-09-10T21:18:53.132930+0000","last_unstale":"2024-09-10T21:18:55.116541+0000","last_undegraded":"2024-09-10T21:18:55.116541+0000","last_fullsized":"2024-09-10T21:18:55.116541+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:08.667229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118234+0000","last_change":"2024-09-10T21:18:54.144376+0000","last_active":"2024-09-10T21:18:55.118234+0000","last_peered":"2024-09-10T21:18:55.118234+0000","last_clean":"2024-09-10T21:18:55.118234+0000","last_became_active":"2024-09-10T21:18:54.143762+0000","last_became_peered":"2024-09-10T21:18:54.143762+0000","last_unstale":"2024-09-10T21:18:55.118234+0000","last_undegraded":"2024-09-10T21:18:55.118234+0000","last_fullsized":"2024-09-10T21:18:55.118234+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:18.195215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714325+0000","last_change":"2024-09-10T21:18:53.134435+0000","last_active":"2024-09-10T21:18:57.714325+0000","last_peered":"2024-09-10T21:18:57.714325+0000","last_clean":"2024-09-10T21:18:57.714325+0000","last_became_active":"2024-09-10T21:18:53.133087+0000","last_became_peered":"2024-09-10T21:18:53.133087+0000","last_unstale":"2024-09-10T21:18:57.714325+0000","last_undegraded":"2024-09-10T21:18:57.714325+0000","last_fullsized":"2024-09-10T21:18:57.714325+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:12.570896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.119187+0000","last_change":"2024-09-10T21:18:54.131428+0000","last_active":"2024-09-10T21:18:55.119187+0000","last_peered":"2024-09-10T21:18:55.119187+0000","last_clean":"2024-09-10T21:18:55.119187+0000","last_became_active":"2024-09-10T21:18:54.131238+0000","last_became_peered":"2024-09-10T21:18:54.131238+0000","last_unstale":"2024-09-10T21:18:55.119187+0000","last_undegraded":"2024-09-10T21:18:55.119187+0000","last_fullsized":"2024-09-10T21:18:55.119187+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:17.093434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"138'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116884+0000","last_change":"2024-09-10T21:18:53.130476+0000","last_active":"2024-09-10T21:18:55.116884+0000","last_peered":"2024-09-10T21:18:55.116884+0000","last_clean":"2024-09-10T21:18:55.116884+0000","last_became_active":"2024-09-10T21:18:53.130213+0000","last_became_peered":"2024-09-10T21:18:53.130213+0000","last_unstale":"2024-09-10T21:18:55.116884+0000","last_undegraded":"2024-09-10T21:18:55.116884+0000","last_fullsized":"2024-09-10T21:18:55.116884+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:02.172443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118851+0000","last_change":"2024-09-10T21:18:54.132591+0000","last_active":"2024-09-10T21:18:55.118851+0000","last_peered":"2024-09-10T21:18:55.118851+0000","last_clean":"2024-09-10T21:18:55.118851+0000","last_became_active":"2024-09-10T21:18:54.132251+0000","last_became_peered":"2024-09-10T21:18:54.132251+0000","last_unstale":"2024-09-10T21:18:55.118851+0000","last_undegraded":"2024-09-10T21:18:55.118851+0000","last_fullsized":"2024-09-10T21:18:55.118851+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:38.405472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124603+0000","last_change":"2024-09-10T21:18:53.130268+0000","last_active":"2024-09-10T21:18:56.124603+0000","last_peered":"2024-09-10T21:18:56.124603+0000","last_clean":"2024-09-10T21:18:56.124603+0000","last_became_active":"2024-09-10T21:18:53.129955+0000","last_became_peered":"2024-09-10T21:18:53.129955+0000","last_unstale":"2024-09-10T21:18:56.124603+0000","last_undegraded":"2024-09-10T21:18:56.124603+0000","last_fullsized":"2024-09-10T21:18:56.124603+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:37.865123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118634+0000","last_change":"2024-09-10T21:18:54.143312+0000","last_active":"2024-09-10T21:18:55.118634+0000","last_peered":"2024-09-10T21:18:55.118634+0000","last_clean":"2024-09-10T21:18:55.118634+0000","last_became_active":"2024-09-10T21:18:54.143013+0000","last_became_peered":"2024-09-10T21:18:54.143013+0000","last_unstale":"2024-09-10T21:18:55.118634+0000","last_undegraded":"2024-09-10T21:18:55.118634+0000","last_fullsized":"2024-09-10T21:18:55.118634+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:40.688104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117862+0000","last_change":"2024-09-10T21:18:53.133483+0000","last_active":"2024-09-10T21:18:55.117862+0000","last_peered":"2024-09-10T21:18:55.117862+0000","last_clean":"2024-09-10T21:18:55.117862+0000","last_became_active":"2024-09-10T21:18:53.133196+0000","last_became_peered":"2024-09-10T21:18:53.133196+0000","last_unstale":"2024-09-10T21:18:55.117862+0000","last_undegraded":"2024-09-10T21:18:55.117862+0000","last_fullsized":"2024-09-10T21:18:55.117862+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:02.691227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714013+0000","last_change":"2024-09-10T21:18:54.149512+0000","last_active":"2024-09-10T21:18:57.714013+0000","last_peered":"2024-09-10T21:18:57.714013+0000","last_clean":"2024-09-10T21:18:57.714013+0000","last_became_active":"2024-09-10T21:18:54.148990+0000","last_became_peered":"2024-09-10T21:18:54.148990+0000","last_unstale":"2024-09-10T21:18:57.714013+0000","last_undegraded":"2024-09-10T21:18:57.714013+0000","last_fullsized":"2024-09-10T21:18:57.714013+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:49.480369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536517+0000","last_change":"2024-09-10T21:18:53.137039+0000","last_active":"2024-09-10T21:18:57.536517+0000","last_peered":"2024-09-10T21:18:57.536517+0000","last_clean":"2024-09-10T21:18:57.536517+0000","last_became_active":"2024-09-10T21:18:53.136763+0000","last_became_peered":"2024-09-10T21:18:53.136763+0000","last_unstale":"2024-09-10T21:18:57.536517+0000","last_undegraded":"2024-09-10T21:18:57.536517+0000","last_fullsized":"2024-09-10T21:18:57.536517+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:32.308582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124008+0000","last_change":"2024-09-10T21:18:54.130766+0000","last_active":"2024-09-10T21:18:56.124008+0000","last_peered":"2024-09-10T21:18:56.124008+0000","last_clean":"2024-09-10T21:18:56.124008+0000","last_became_active":"2024-09-10T21:18:54.130480+0000","last_became_peered":"2024-09-10T21:18:54.130480+0000","last_unstale":"2024-09-10T21:18:56.124008+0000","last_undegraded":"2024-09-10T21:18:56.124008+0000","last_fullsized":"2024-09-10T21:18:56.124008+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:48.359201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412532+0000","last_change":"2024-09-10T21:18:53.133724+0000","last_active":"2024-09-10T21:18:55.412532+0000","last_peered":"2024-09-10T21:18:55.412532+0000","last_clean":"2024-09-10T21:18:55.412532+0000","last_became_active":"2024-09-10T21:18:53.133442+0000","last_became_peered":"2024-09-10T21:18:53.133442+0000","last_unstale":"2024-09-10T21:18:55.412532+0000","last_undegraded":"2024-09-10T21:18:55.412532+0000","last_fullsized":"2024-09-10T21:18:55.412532+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:04.295889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.413079+0000","last_change":"2024-09-10T21:18:54.137184+0000","last_active":"2024-09-10T21:18:55.413079+0000","last_peered":"2024-09-10T21:18:55.413079+0000","last_clean":"2024-09-10T21:18:55.413079+0000","last_became_active":"2024-09-10T21:18:54.136895+0000","last_became_peered":"2024-09-10T21:18:54.136895+0000","last_unstale":"2024-09-10T21:18:55.413079+0000","last_undegraded":"2024-09-10T21:18:55.413079+0000","last_fullsized":"2024-09-10T21:18:55.413079+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:53.555070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124264+0000","last_change":"2024-09-10T21:18:53.132411+0000","last_active":"2024-09-10T21:18:56.124264+0000","last_peered":"2024-09-10T21:18:56.124264+0000","last_clean":"2024-09-10T21:18:56.124264+0000","last_became_active":"2024-09-10T21:18:53.131949+0000","last_became_peered":"2024-09-10T21:18:53.131949+0000","last_unstale":"2024-09-10T21:18:56.124264+0000","last_undegraded":"2024-09-10T21:18:56.124264+0000","last_fullsized":"2024-09-10T21:18:56.124264+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:02.387195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.123873+0000","last_change":"2024-09-10T21:18:54.144243+0000","last_active":"2024-09-10T21:18:56.123873+0000","last_peered":"2024-09-10T21:18:56.123873+0000","last_clean":"2024-09-10T21:18:56.123873+0000","last_became_active":"2024-09-10T21:18:54.143449+0000","last_became_peered":"2024-09-10T21:18:54.143449+0000","last_unstale":"2024-09-10T21:18:56.123873+0000","last_undegraded":"2024-09-10T21:18:56.123873+0000","last_fullsized":"2024-09-10T21:18:56.123873+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:01.161281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.123801+0000","last_change":"2024-09-10T21:18:53.127615+0000","last_active":"2024-09-10T21:18:56.123801+0000","last_peered":"2024-09-10T21:18:56.123801+0000","last_clean":"2024-09-10T21:18:56.123801+0000","last_became_active":"2024-09-10T21:18:53.127317+0000","last_became_peered":"2024-09-10T21:18:53.127317+0000","last_unstale":"2024-09-10T21:18:56.123801+0000","last_undegraded":"2024-09-10T21:18:56.123801+0000","last_fullsized":"2024-09-10T21:18:56.123801+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:54.218681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.413032+0000","last_change":"2024-09-10T21:18:54.134957+0000","last_active":"2024-09-10T21:18:55.413032+0000","last_peered":"2024-09-10T21:18:55.413032+0000","last_clean":"2024-09-10T21:18:55.413032+0000","last_became_active":"2024-09-10T21:18:54.128460+0000","last_became_peered":"2024-09-10T21:18:54.128460+0000","last_unstale":"2024-09-10T21:18:55.413032+0000","last_undegraded":"2024-09-10T21:18:55.413032+0000","last_fullsized":"2024-09-10T21:18:55.413032+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:49.710726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536622+0000","last_change":"2024-09-10T21:18:53.136422+0000","last_active":"2024-09-10T21:18:57.536622+0000","last_peered":"2024-09-10T21:18:57.536622+0000","last_clean":"2024-09-10T21:18:57.536622+0000","last_became_active":"2024-09-10T21:18:53.136132+0000","last_became_peered":"2024-09-10T21:18:53.136132+0000","last_unstale":"2024-09-10T21:18:57.536622+0000","last_undegraded":"2024-09-10T21:18:57.536622+0000","last_fullsized":"2024-09-10T21:18:57.536622+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:30.203255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":319,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536713+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:18:57.536713+0000","last_peered":"2024-09-10T21:18:57.536713+0000","last_clean":"2024-09-10T21:18:57.536713+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:18:57.536713+0000","last_undegraded":"2024-09-10T21:18:57.536713+0000","last_fullsized":"2024-09-10T21:18:57.536713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37040,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334800,"statfs":{"total":96636764160,"available":96598835200,"internally_reserved":0,"allocated":1732608,"data_stored":746399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509549,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37624,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334216,"statfs":{"total":96636764160,"available":96598237184,"internally_reserved":0,"allocated":2330624,"data_stored":1337405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509547,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37040,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334800,"statfs":{"total":96636764160,"available":96598835200,"internally_reserved":0,"allocated":1732608,"data_stored":746399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509546,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37056,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334784,"statfs":{"total":96636764160,"available":96598818816,"internally_reserved":0,"allocated":1744896,"data_stored":747999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509548,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1732608,"data_stored":746947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509547,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37040,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334800,"statfs":{"total":96636764160,"available":96598835200,"internally_reserved":0,"allocated":1732608,"data_stored":746435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542251,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41192,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330648,"statfs":{"total":96636764160,"available":96594583552,"internally_reserved":0,"allocated":1716224,"data_stored":740065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542250,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37640,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334200,"statfs":{"total":96636764160,"available":96598220800,"internally_reserved":0,"allocated":2347008,"data_stored":1338527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:19:02.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:19:03.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.072+0000 7f0db9a00640 1 -- 172.21.15.16:0/2259399398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db415f400 msgr2=0x7f0db4070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:03.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.072+0000 7f0db9a00640 1 --2- 172.21.15.16:0/2259399398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db415f400 0x7f0db4070fe0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f0db4074cc0 tx=0x7f0da401c780 comp rx=0 tx=0).stop 2024-09-10T21:19:03.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.072+0000 7f0db9a00640 1 -- 172.21.15.16:0/2259399398 shutdown_connections 2024-09-10T21:19:03.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.072+0000 7f0db9a00640 1 --2- 172.21.15.16:0/2259399398 >> v2:172.21.15.52:3300/0 conn(0x7f0db40cfc80 0x7f0db40d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.072+0000 7f0db9a00640 1 --2- 172.21.15.16:0/2259399398 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0db40b13c0 0x7f0db4071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.072+0000 7f0db9a00640 1 --2- 172.21.15.16:0/2259399398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db415f400 0x7f0db4070fe0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.072+0000 7f0db9a00640 1 -- 172.21.15.16:0/2259399398 >> 172.21.15.16:0/2259399398 conn(0x7f0db409b130 msgr2=0x7f0db40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:03.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.072+0000 7f0db9a00640 1 -- 172.21.15.16:0/2259399398 shutdown_connections 2024-09-10T21:19:03.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.072+0000 7f0db9a00640 1 -- 172.21.15.16:0/2259399398 wait complete. 2024-09-10T21:19:03.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db9a00640 1 Processor -- start 2024-09-10T21:19:03.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db9a00640 1 -- start start 2024-09-10T21:19:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db9a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0db40b13c0 0x7f0db4155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db9a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0db40cfc80 0x7f0db40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db415f400 0x7f0db4155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db9a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0db40afd90 con 0x7f0db415f400 2024-09-10T21:19:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db9a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0db40d1260 con 0x7f0db40cfc80 2024-09-10T21:19:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db9a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0db40aff10 con 0x7f0db40b13c0 2024-09-10T21:19:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db415f400 0x7f0db4155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db415f400 0x7f0db4155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38052/0 (socket says 172.21.15.16:38052) 2024-09-10T21:19:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db3400640 1 -- 172.21.15.16:0/1774332908 learned_addr learned my addr 172.21.15.16:0/1774332908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0dabe00640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0db40cfc80 0x7f0db40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db3400640 1 -- 172.21.15.16:0/1774332908 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0db40b13c0 msgr2=0x7f0db4155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db3400640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0db40b13c0 0x7f0db4155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db3400640 1 -- 172.21.15.16:0/1774332908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0db40cfc80 msgr2=0x7f0db40b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db3400640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0db40cfc80 0x7f0db40b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db3400640 1 -- 172.21.15.16:0/1774332908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db415a6d0 con 0x7f0db415f400 2024-09-10T21:19:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0dabe00640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0db40cfc80 0x7f0db40b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:19:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db3400640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db415f400 0x7f0db4155c90 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00eb20 tx=0x7f0dac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0daaa00640 1 -- 172.21.15.16:0/1774332908 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac004b30 con 0x7f0db415f400 2024-09-10T21:19:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0daaa00640 1 -- 172.21.15.16:0/1774332908 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dac004cf0 con 0x7f0db415f400 2024-09-10T21:19:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db9a00640 1 -- 172.21.15.16:0/1774332908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db406e7a0 con 0x7f0db415f400 2024-09-10T21:19:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0daaa00640 1 -- 172.21.15.16:0/1774332908 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac005280 con 0x7f0db415f400 2024-09-10T21:19:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.076+0000 7f0db9a00640 1 -- 172.21.15.16:0/1774332908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db415a970 con 0x7f0db415f400 2024-09-10T21:19:03.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.080+0000 7f0db9a00640 1 -- 172.21.15.16:0/1774332908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db40d00d0 con 0x7f0db415f400 2024-09-10T21:19:03.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.080+0000 7f0daaa00640 1 -- 172.21.15.16:0/1774332908 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0dac005420 con 0x7f0db415f400 2024-09-10T21:19:03.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.080+0000 7f0daaa00640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0d90072610 0x7f0d90074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:03.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.080+0000 7f0daaa00640 1 -- 172.21.15.16:0/1774332908 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f0dac094070 con 0x7f0db415f400 2024-09-10T21:19:03.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.080+0000 7f0db2a00640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0d90072610 0x7f0d90074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:03.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.080+0000 7f0db2a00640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0d90072610 0x7f0d90074ad0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0da40af000 tx=0x7f0da4019520 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.084+0000 7f0daaa00640 1 -- 172.21.15.16:0/1774332908 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0dac05f6b0 con 0x7f0db415f400 2024-09-10T21:19:03.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.256+0000 7f0db9a00640 1 -- 172.21.15.16:0/1774332908 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0db40b2630 con 0x7f0d90072610 2024-09-10T21:19:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.260+0000 7f0daaa00640 1 -- 172.21.15.16:0/1774332908 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252474 (secure 0 0 0) 0x7f0db40b2630 con 0x7f0d90072610 2024-09-10T21:19:03.268 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:19:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.264+0000 7f0db9a00640 1 -- 172.21.15.16:0/1774332908 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0d90072610 msgr2=0x7f0d90074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.264+0000 7f0db9a00640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0d90072610 0x7f0d90074ad0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0da40af000 tx=0x7f0da4019520 comp rx=0 tx=0).stop 2024-09-10T21:19:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.264+0000 7f0db9a00640 1 -- 172.21.15.16:0/1774332908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db415f400 msgr2=0x7f0db4155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.264+0000 7f0db9a00640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db415f400 0x7f0db4155c90 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00eb20 tx=0x7f0dac00c2d0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.264+0000 7f0db9a00640 1 -- 172.21.15.16:0/1774332908 shutdown_connections 2024-09-10T21:19:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.264+0000 7f0db9a00640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0d90072610 0x7f0d90074ad0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.264+0000 7f0db9a00640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db415f400 0x7f0db4155c90 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.264+0000 7f0db9a00640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0db40cfc80 0x7f0db40b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.264+0000 7f0db9a00640 1 --2- 172.21.15.16:0/1774332908 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0db40b13c0 0x7f0db4155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.264+0000 7f0db9a00640 1 -- 172.21.15.16:0/1774332908 >> 172.21.15.16:0/1774332908 conn(0x7f0db409b130 msgr2=0x7f0db40cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.264+0000 7f0db9a00640 1 -- 172.21.15.16:0/1774332908 shutdown_connections 2024-09-10T21:19:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.264+0000 7f0db9a00640 1 -- 172.21.15.16:0/1774332908 wait complete. 2024-09-10T21:19:03.288 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-10T21:19:01.576126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":297516,"kb_used_data":14716,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754677204,"statfs":{"total":773094113280,"available":772789456896,"internally_reserved":0,"allocated":15069184,"data_stored":7150176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001841"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124518+0000","last_change":"2024-09-10T21:18:54.148585+0000","last_active":"2024-09-10T21:18:56.124518+0000","last_peered":"2024-09-10T21:18:56.124518+0000","last_clean":"2024-09-10T21:18:56.124518+0000","last_became_active":"2024-09-10T21:18:54.145516+0000","last_became_peered":"2024-09-10T21:18:54.145516+0000","last_unstale":"2024-09-10T21:18:56.124518+0000","last_undegraded":"2024-09-10T21:18:56.124518+0000","last_fullsized":"2024-09-10T21:18:56.124518+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:04.643193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124482+0000","last_change":"2024-09-10T21:18:54.143896+0000","last_active":"2024-09-10T21:18:56.124482+0000","last_peered":"2024-09-10T21:18:56.124482+0000","last_clean":"2024-09-10T21:18:56.124482+0000","last_became_active":"2024-09-10T21:18:54.143634+0000","last_became_peered":"2024-09-10T21:18:54.143634+0000","last_unstale":"2024-09-10T21:18:56.124482+0000","last_undegraded":"2024-09-10T21:18:56.124482+0000","last_fullsized":"2024-09-10T21:18:56.124482+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:36.226183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124299+0000","last_change":"2024-09-10T21:18:54.137010+0000","last_active":"2024-09-10T21:18:56.124299+0000","last_peered":"2024-09-10T21:18:56.124299+0000","last_clean":"2024-09-10T21:18:56.124299+0000","last_became_active":"2024-09-10T21:18:54.132490+0000","last_became_peered":"2024-09-10T21:18:54.132490+0000","last_unstale":"2024-09-10T21:18:56.124299+0000","last_undegraded":"2024-09-10T21:18:56.124299+0000","last_fullsized":"2024-09-10T21:18:56.124299+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:28.746632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118536+0000","last_change":"2024-09-10T21:18:54.151199+0000","last_active":"2024-09-10T21:18:55.118536+0000","last_peered":"2024-09-10T21:18:55.118536+0000","last_clean":"2024-09-10T21:18:55.118536+0000","last_became_active":"2024-09-10T21:18:54.150935+0000","last_became_peered":"2024-09-10T21:18:54.150935+0000","last_unstale":"2024-09-10T21:18:55.118536+0000","last_undegraded":"2024-09-10T21:18:55.118536+0000","last_fullsized":"2024-09-10T21:18:55.118536+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:11:43.194385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536453+0000","last_change":"2024-09-10T21:18:54.149942+0000","last_active":"2024-09-10T21:18:57.536453+0000","last_peered":"2024-09-10T21:18:57.536453+0000","last_clean":"2024-09-10T21:18:57.536453+0000","last_became_active":"2024-09-10T21:18:54.149641+0000","last_became_peered":"2024-09-10T21:18:54.149641+0000","last_unstale":"2024-09-10T21:18:57.536453+0000","last_undegraded":"2024-09-10T21:18:57.536453+0000","last_fullsized":"2024-09-10T21:18:57.536453+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:48.061606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.713961+0000","last_change":"2024-09-10T21:18:54.149259+0000","last_active":"2024-09-10T21:18:57.713961+0000","last_peered":"2024-09-10T21:18:57.713961+0000","last_clean":"2024-09-10T21:18:57.713961+0000","last_became_active":"2024-09-10T21:18:54.148593+0000","last_became_peered":"2024-09-10T21:18:54.148593+0000","last_unstale":"2024-09-10T21:18:57.713961+0000","last_undegraded":"2024-09-10T21:18:57.713961+0000","last_fullsized":"2024-09-10T21:18:57.713961+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:55.391340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412719+0000","last_change":"2024-09-10T21:18:54.141633+0000","last_active":"2024-09-10T21:18:55.412719+0000","last_peered":"2024-09-10T21:18:55.412719+0000","last_clean":"2024-09-10T21:18:55.412719+0000","last_became_active":"2024-09-10T21:18:54.141358+0000","last_became_peered":"2024-09-10T21:18:54.141358+0000","last_unstale":"2024-09-10T21:18:55.412719+0000","last_undegraded":"2024-09-10T21:18:55.412719+0000","last_fullsized":"2024-09-10T21:18:55.412719+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:13.382947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714433+0000","last_change":"2024-09-10T21:18:54.141560+0000","last_active":"2024-09-10T21:18:57.714433+0000","last_peered":"2024-09-10T21:18:57.714433+0000","last_clean":"2024-09-10T21:18:57.714433+0000","last_became_active":"2024-09-10T21:18:54.141005+0000","last_became_peered":"2024-09-10T21:18:54.141005+0000","last_unstale":"2024-09-10T21:18:57.714433+0000","last_undegraded":"2024-09-10T21:18:57.714433+0000","last_fullsized":"2024-09-10T21:18:57.714433+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:09.504837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714839+0000","last_change":"2024-09-10T21:18:54.145288+0000","last_active":"2024-09-10T21:18:57.714839+0000","last_peered":"2024-09-10T21:18:57.714839+0000","last_clean":"2024-09-10T21:18:57.714839+0000","last_became_active":"2024-09-10T21:18:54.145003+0000","last_became_peered":"2024-09-10T21:18:54.145003+0000","last_unstale":"2024-09-10T21:18:57.714839+0000","last_undegraded":"2024-09-10T21:18:57.714839+0000","last_fullsized":"2024-09-10T21:18:57.714839+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:38.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714735+0000","last_change":"2024-09-10T21:18:54.151547+0000","last_active":"2024-09-10T21:18:57.714735+0000","last_peered":"2024-09-10T21:18:57.714735+0000","last_clean":"2024-09-10T21:18:57.714735+0000","last_became_active":"2024-09-10T21:18:54.151254+0000","last_became_peered":"2024-09-10T21:18:54.151254+0000","last_unstale":"2024-09-10T21:18:57.714735+0000","last_undegraded":"2024-09-10T21:18:57.714735+0000","last_fullsized":"2024-09-10T21:18:57.714735+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:18.902652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536518+0000","last_change":"2024-09-10T21:18:54.138443+0000","last_active":"2024-09-10T21:18:57.536518+0000","last_peered":"2024-09-10T21:18:57.536518+0000","last_clean":"2024-09-10T21:18:57.536518+0000","last_became_active":"2024-09-10T21:18:54.138053+0000","last_became_peered":"2024-09-10T21:18:54.138053+0000","last_unstale":"2024-09-10T21:18:57.536518+0000","last_undegraded":"2024-09-10T21:18:57.536518+0000","last_fullsized":"2024-09-10T21:18:57.536518+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:41.987074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116858+0000","last_change":"2024-09-10T21:18:54.134839+0000","last_active":"2024-09-10T21:18:55.116858+0000","last_peered":"2024-09-10T21:18:55.116858+0000","last_clean":"2024-09-10T21:18:55.116858+0000","last_became_active":"2024-09-10T21:18:54.134552+0000","last_became_peered":"2024-09-10T21:18:54.134552+0000","last_unstale":"2024-09-10T21:18:55.116858+0000","last_undegraded":"2024-09-10T21:18:55.116858+0000","last_fullsized":"2024-09-10T21:18:55.116858+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:53.785552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116627+0000","last_change":"2024-09-10T21:18:54.129992+0000","last_active":"2024-09-10T21:18:55.116627+0000","last_peered":"2024-09-10T21:18:55.116627+0000","last_clean":"2024-09-10T21:18:55.116627+0000","last_became_active":"2024-09-10T21:18:54.129578+0000","last_became_peered":"2024-09-10T21:18:54.129578+0000","last_unstale":"2024-09-10T21:18:55.116627+0000","last_undegraded":"2024-09-10T21:18:55.116627+0000","last_fullsized":"2024-09-10T21:18:55.116627+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:39.550472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714963+0000","last_change":"2024-09-10T21:18:54.146240+0000","last_active":"2024-09-10T21:18:57.714963+0000","last_peered":"2024-09-10T21:18:57.714963+0000","last_clean":"2024-09-10T21:18:57.714963+0000","last_became_active":"2024-09-10T21:18:54.145952+0000","last_became_peered":"2024-09-10T21:18:54.145952+0000","last_unstale":"2024-09-10T21:18:57.714963+0000","last_undegraded":"2024-09-10T21:18:57.714963+0000","last_fullsized":"2024-09-10T21:18:57.714963+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:27.820474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714511+0000","last_change":"2024-09-10T21:18:54.149220+0000","last_active":"2024-09-10T21:18:57.714511+0000","last_peered":"2024-09-10T21:18:57.714511+0000","last_clean":"2024-09-10T21:18:57.714511+0000","last_became_active":"2024-09-10T21:18:54.143941+0000","last_became_peered":"2024-09-10T21:18:54.143941+0000","last_unstale":"2024-09-10T21:18:57.714511+0000","last_undegraded":"2024-09-10T21:18:57.714511+0000","last_fullsized":"2024-09-10T21:18:57.714511+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:03.393936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412492+0000","last_change":"2024-09-10T21:18:54.128306+0000","last_active":"2024-09-10T21:18:55.412492+0000","last_peered":"2024-09-10T21:18:55.412492+0000","last_clean":"2024-09-10T21:18:55.412492+0000","last_became_active":"2024-09-10T21:18:54.127837+0000","last_became_peered":"2024-09-10T21:18:54.127837+0000","last_unstale":"2024-09-10T21:18:55.412492+0000","last_undegraded":"2024-09-10T21:18:55.412492+0000","last_fullsized":"2024-09-10T21:18:55.412492+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:40.753501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714238+0000","last_change":"2024-09-10T21:18:54.150068+0000","last_active":"2024-09-10T21:18:57.714238+0000","last_peered":"2024-09-10T21:18:57.714238+0000","last_clean":"2024-09-10T21:18:57.714238+0000","last_became_active":"2024-09-10T21:18:54.149774+0000","last_became_peered":"2024-09-10T21:18:54.149774+0000","last_unstale":"2024-09-10T21:18:57.714238+0000","last_undegraded":"2024-09-10T21:18:57.714238+0000","last_fullsized":"2024-09-10T21:18:57.714238+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:44.793301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118759+0000","last_change":"2024-09-10T21:18:54.129484+0000","last_active":"2024-09-10T21:18:55.118759+0000","last_peered":"2024-09-10T21:18:55.118759+0000","last_clean":"2024-09-10T21:18:55.118759+0000","last_became_active":"2024-09-10T21:18:54.128927+0000","last_became_peered":"2024-09-10T21:18:54.128927+0000","last_unstale":"2024-09-10T21:18:55.118759+0000","last_undegraded":"2024-09-10T21:18:55.118759+0000","last_fullsized":"2024-09-10T21:18:55.118759+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:49.960348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412845+0000","last_change":"2024-09-10T21:18:54.135461+0000","last_active":"2024-09-10T21:18:55.412845+0000","last_peered":"2024-09-10T21:18:55.412845+0000","last_clean":"2024-09-10T21:18:55.412845+0000","last_became_active":"2024-09-10T21:18:54.135129+0000","last_became_peered":"2024-09-10T21:18:54.135129+0000","last_unstale":"2024-09-10T21:18:55.412845+0000","last_undegraded":"2024-09-10T21:18:55.412845+0000","last_fullsized":"2024-09-10T21:18:55.412845+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:27.802270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116954+0000","last_change":"2024-09-10T21:18:54.131703+0000","last_active":"2024-09-10T21:18:55.116954+0000","last_peered":"2024-09-10T21:18:55.116954+0000","last_clean":"2024-09-10T21:18:55.116954+0000","last_became_active":"2024-09-10T21:18:54.131469+0000","last_became_peered":"2024-09-10T21:18:54.131469+0000","last_unstale":"2024-09-10T21:18:55.116954+0000","last_undegraded":"2024-09-10T21:18:55.116954+0000","last_fullsized":"2024-09-10T21:18:55.116954+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:08.975945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714308+0000","last_change":"2024-09-10T21:18:54.155216+0000","last_active":"2024-09-10T21:18:57.714308+0000","last_peered":"2024-09-10T21:18:57.714308+0000","last_clean":"2024-09-10T21:18:57.714308+0000","last_became_active":"2024-09-10T21:18:54.154821+0000","last_became_peered":"2024-09-10T21:18:54.154821+0000","last_unstale":"2024-09-10T21:18:57.714308+0000","last_undegraded":"2024-09-10T21:18:57.714308+0000","last_fullsized":"2024-09-10T21:18:57.714308+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:18.510060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536449+0000","last_change":"2024-09-10T21:18:54.136990+0000","last_active":"2024-09-10T21:18:57.536449+0000","last_peered":"2024-09-10T21:18:57.536449+0000","last_clean":"2024-09-10T21:18:57.536449+0000","last_became_active":"2024-09-10T21:18:54.136700+0000","last_became_peered":"2024-09-10T21:18:54.136700+0000","last_unstale":"2024-09-10T21:18:57.536449+0000","last_undegraded":"2024-09-10T21:18:57.536449+0000","last_fullsized":"2024-09-10T21:18:57.536449+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:58.842730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117448+0000","last_change":"2024-09-10T21:18:54.130109+0000","last_active":"2024-09-10T21:18:55.117448+0000","last_peered":"2024-09-10T21:18:55.117448+0000","last_clean":"2024-09-10T21:18:55.117448+0000","last_became_active":"2024-09-10T21:18:54.129818+0000","last_became_peered":"2024-09-10T21:18:54.129818+0000","last_unstale":"2024-09-10T21:18:55.117448+0000","last_undegraded":"2024-09-10T21:18:55.117448+0000","last_fullsized":"2024-09-10T21:18:55.117448+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:06.371859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536746+0000","last_change":"2024-09-10T21:18:54.151570+0000","last_active":"2024-09-10T21:18:57.536746+0000","last_peered":"2024-09-10T21:18:57.536746+0000","last_clean":"2024-09-10T21:18:57.536746+0000","last_became_active":"2024-09-10T21:18:54.151323+0000","last_became_peered":"2024-09-10T21:18:54.151323+0000","last_unstale":"2024-09-10T21:18:57.536746+0000","last_undegraded":"2024-09-10T21:18:57.536746+0000","last_fullsized":"2024-09-10T21:18:57.536746+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:55.961963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536558+0000","last_change":"2024-09-10T21:18:54.150942+0000","last_active":"2024-09-10T21:18:57.536558+0000","last_peered":"2024-09-10T21:18:57.536558+0000","last_clean":"2024-09-10T21:18:57.536558+0000","last_became_active":"2024-09-10T21:18:54.147390+0000","last_became_peered":"2024-09-10T21:18:54.147390+0000","last_unstale":"2024-09-10T21:18:57.536558+0000","last_undegraded":"2024-09-10T21:18:57.536558+0000","last_fullsized":"2024-09-10T21:18:57.536558+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:47.191704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117961+0000","last_change":"2024-09-10T21:18:54.144696+0000","last_active":"2024-09-10T21:18:55.117961+0000","last_peered":"2024-09-10T21:18:55.117961+0000","last_clean":"2024-09-10T21:18:55.117961+0000","last_became_active":"2024-09-10T21:18:54.144114+0000","last_became_peered":"2024-09-10T21:18:54.144114+0000","last_unstale":"2024-09-10T21:18:55.117961+0000","last_undegraded":"2024-09-10T21:18:55.117961+0000","last_fullsized":"2024-09-10T21:18:55.117961+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:13.343963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536821+0000","last_change":"2024-09-10T21:18:54.135462+0000","last_active":"2024-09-10T21:18:57.536821+0000","last_peered":"2024-09-10T21:18:57.536821+0000","last_clean":"2024-09-10T21:18:57.536821+0000","last_became_active":"2024-09-10T21:18:54.135193+0000","last_became_peered":"2024-09-10T21:18:54.135193+0000","last_unstale":"2024-09-10T21:18:57.536821+0000","last_undegraded":"2024-09-10T21:18:57.536821+0000","last_fullsized":"2024-09-10T21:18:57.536821+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:13.577636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412739+0000","last_change":"2024-09-10T21:18:54.140632+0000","last_active":"2024-09-10T21:18:55.412739+0000","last_peered":"2024-09-10T21:18:55.412739+0000","last_clean":"2024-09-10T21:18:55.412739+0000","last_became_active":"2024-09-10T21:18:54.140193+0000","last_became_peered":"2024-09-10T21:18:54.140193+0000","last_unstale":"2024-09-10T21:18:55.412739+0000","last_undegraded":"2024-09-10T21:18:55.412739+0000","last_fullsized":"2024-09-10T21:18:55.412739+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:00.133528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124200+0000","last_change":"2024-09-10T21:18:54.137981+0000","last_active":"2024-09-10T21:18:56.124200+0000","last_peered":"2024-09-10T21:18:56.124200+0000","last_clean":"2024-09-10T21:18:56.124200+0000","last_became_active":"2024-09-10T21:18:54.136200+0000","last_became_peered":"2024-09-10T21:18:54.136200+0000","last_unstale":"2024-09-10T21:18:56.124200+0000","last_undegraded":"2024-09-10T21:18:56.124200+0000","last_fullsized":"2024-09-10T21:18:56.124200+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:49.986865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714458+0000","last_change":"2024-09-10T21:18:54.136050+0000","last_active":"2024-09-10T21:18:57.714458+0000","last_peered":"2024-09-10T21:18:57.714458+0000","last_clean":"2024-09-10T21:18:57.714458+0000","last_became_active":"2024-09-10T21:18:54.134760+0000","last_became_peered":"2024-09-10T21:18:54.134760+0000","last_unstale":"2024-09-10T21:18:57.714458+0000","last_undegraded":"2024-09-10T21:18:57.714458+0000","last_fullsized":"2024-09-10T21:18:57.714458+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:55.225422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.123953+0000","last_change":"2024-09-10T21:18:54.128530+0000","last_active":"2024-09-10T21:18:56.123953+0000","last_peered":"2024-09-10T21:18:56.123953+0000","last_clean":"2024-09-10T21:18:56.123953+0000","last_became_active":"2024-09-10T21:18:54.128263+0000","last_became_peered":"2024-09-10T21:18:54.128263+0000","last_unstale":"2024-09-10T21:18:56.123953+0000","last_undegraded":"2024-09-10T21:18:56.123953+0000","last_fullsized":"2024-09-10T21:18:56.123953+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:09.259466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714226+0000","last_change":"2024-09-10T21:18:54.146925+0000","last_active":"2024-09-10T21:18:57.714226+0000","last_peered":"2024-09-10T21:18:57.714226+0000","last_clean":"2024-09-10T21:18:57.714226+0000","last_became_active":"2024-09-10T21:18:54.146621+0000","last_became_peered":"2024-09-10T21:18:54.146621+0000","last_unstale":"2024-09-10T21:18:57.714226+0000","last_undegraded":"2024-09-10T21:18:57.714226+0000","last_fullsized":"2024-09-10T21:18:57.714226+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:21.444679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118922+0000","last_change":"2024-09-10T21:18:54.132032+0000","last_active":"2024-09-10T21:18:55.118922+0000","last_peered":"2024-09-10T21:18:55.118922+0000","last_clean":"2024-09-10T21:18:55.118922+0000","last_became_active":"2024-09-10T21:18:54.131736+0000","last_became_peered":"2024-09-10T21:18:54.131736+0000","last_unstale":"2024-09-10T21:18:55.118922+0000","last_undegraded":"2024-09-10T21:18:55.118922+0000","last_fullsized":"2024-09-10T21:18:55.118922+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:50.696171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"137'4","reported_seq":19,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124672+0000","last_change":"2024-09-10T21:18:53.122058+0000","last_active":"2024-09-10T21:18:56.124672+0000","last_peered":"2024-09-10T21:18:56.124672+0000","last_clean":"2024-09-10T21:18:56.124672+0000","last_became_active":"2024-09-10T21:18:53.121768+0000","last_became_peered":"2024-09-10T21:18:53.121768+0000","last_unstale":"2024-09-10T21:18:56.124672+0000","last_undegraded":"2024-09-10T21:18:56.124672+0000","last_fullsized":"2024-09-10T21:18:56.124672+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:52.042309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124591+0000","last_change":"2024-09-10T21:18:54.128927+0000","last_active":"2024-09-10T21:18:56.124591+0000","last_peered":"2024-09-10T21:18:56.124591+0000","last_clean":"2024-09-10T21:18:56.124591+0000","last_became_active":"2024-09-10T21:18:54.128107+0000","last_became_peered":"2024-09-10T21:18:54.128107+0000","last_unstale":"2024-09-10T21:18:56.124591+0000","last_undegraded":"2024-09-10T21:18:56.124591+0000","last_fullsized":"2024-09-10T21:18:56.124591+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:01.643551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714790+0000","last_change":"2024-09-10T21:18:53.134253+0000","last_active":"2024-09-10T21:18:57.714790+0000","last_peered":"2024-09-10T21:18:57.714790+0000","last_clean":"2024-09-10T21:18:57.714790+0000","last_became_active":"2024-09-10T21:18:53.133971+0000","last_became_peered":"2024-09-10T21:18:53.133971+0000","last_unstale":"2024-09-10T21:18:57.714790+0000","last_undegraded":"2024-09-10T21:18:57.714790+0000","last_fullsized":"2024-09-10T21:18:57.714790+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:22.649998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714499+0000","last_change":"2024-09-10T21:18:54.149088+0000","last_active":"2024-09-10T21:18:57.714499+0000","last_peered":"2024-09-10T21:18:57.714499+0000","last_clean":"2024-09-10T21:18:57.714499+0000","last_became_active":"2024-09-10T21:18:54.148557+0000","last_became_peered":"2024-09-10T21:18:54.148557+0000","last_unstale":"2024-09-10T21:18:57.714499+0000","last_undegraded":"2024-09-10T21:18:57.714499+0000","last_fullsized":"2024-09-10T21:18:57.714499+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:37.650822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.713801+0000","last_change":"2024-09-10T21:18:53.138535+0000","last_active":"2024-09-10T21:18:57.713801+0000","last_peered":"2024-09-10T21:18:57.713801+0000","last_clean":"2024-09-10T21:18:57.713801+0000","last_became_active":"2024-09-10T21:18:53.138234+0000","last_became_peered":"2024-09-10T21:18:53.138234+0000","last_unstale":"2024-09-10T21:18:57.713801+0000","last_undegraded":"2024-09-10T21:18:57.713801+0000","last_fullsized":"2024-09-10T21:18:57.713801+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:03.695792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714773+0000","last_change":"2024-09-10T21:18:54.142121+0000","last_active":"2024-09-10T21:18:57.714773+0000","last_peered":"2024-09-10T21:18:57.714773+0000","last_clean":"2024-09-10T21:18:57.714773+0000","last_became_active":"2024-09-10T21:18:54.141515+0000","last_became_peered":"2024-09-10T21:18:54.141515+0000","last_unstale":"2024-09-10T21:18:57.714773+0000","last_undegraded":"2024-09-10T21:18:57.714773+0000","last_fullsized":"2024-09-10T21:18:57.714773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:53.049049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117745+0000","last_change":"2024-09-10T21:18:53.132095+0000","last_active":"2024-09-10T21:18:55.117745+0000","last_peered":"2024-09-10T21:18:55.117745+0000","last_clean":"2024-09-10T21:18:55.117745+0000","last_became_active":"2024-09-10T21:18:53.131751+0000","last_became_peered":"2024-09-10T21:18:53.131751+0000","last_unstale":"2024-09-10T21:18:55.117745+0000","last_undegraded":"2024-09-10T21:18:55.117745+0000","last_fullsized":"2024-09-10T21:18:55.117745+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:39.746760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118528+0000","last_change":"2024-09-10T21:18:54.132329+0000","last_active":"2024-09-10T21:18:55.118528+0000","last_peered":"2024-09-10T21:18:55.118528+0000","last_clean":"2024-09-10T21:18:55.118528+0000","last_became_active":"2024-09-10T21:18:54.132042+0000","last_became_peered":"2024-09-10T21:18:54.132042+0000","last_unstale":"2024-09-10T21:18:55.118528+0000","last_undegraded":"2024-09-10T21:18:55.118528+0000","last_fullsized":"2024-09-10T21:18:55.118528+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:34.600677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124219+0000","last_change":"2024-09-10T21:18:53.135048+0000","last_active":"2024-09-10T21:18:56.124219+0000","last_peered":"2024-09-10T21:18:56.124219+0000","last_clean":"2024-09-10T21:18:56.124219+0000","last_became_active":"2024-09-10T21:18:53.134366+0000","last_became_peered":"2024-09-10T21:18:53.134366+0000","last_unstale":"2024-09-10T21:18:56.124219+0000","last_undegraded":"2024-09-10T21:18:56.124219+0000","last_fullsized":"2024-09-10T21:18:56.124219+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:17.895912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714022+0000","last_change":"2024-09-10T21:18:54.146108+0000","last_active":"2024-09-10T21:18:57.714022+0000","last_peered":"2024-09-10T21:18:57.714022+0000","last_clean":"2024-09-10T21:18:57.714022+0000","last_became_active":"2024-09-10T21:18:54.145835+0000","last_became_peered":"2024-09-10T21:18:54.145835+0000","last_unstale":"2024-09-10T21:18:57.714022+0000","last_undegraded":"2024-09-10T21:18:57.714022+0000","last_fullsized":"2024-09-10T21:18:57.714022+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:42.657250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118681+0000","last_change":"2024-09-10T21:18:53.130356+0000","last_active":"2024-09-10T21:18:55.118681+0000","last_peered":"2024-09-10T21:18:55.118681+0000","last_clean":"2024-09-10T21:18:55.118681+0000","last_became_active":"2024-09-10T21:18:53.130043+0000","last_became_peered":"2024-09-10T21:18:53.130043+0000","last_unstale":"2024-09-10T21:18:55.118681+0000","last_undegraded":"2024-09-10T21:18:55.118681+0000","last_fullsized":"2024-09-10T21:18:55.118681+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:49.589673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117030+0000","last_change":"2024-09-10T21:18:54.136233+0000","last_active":"2024-09-10T21:18:55.117030+0000","last_peered":"2024-09-10T21:18:55.117030+0000","last_clean":"2024-09-10T21:18:55.117030+0000","last_became_active":"2024-09-10T21:18:54.135939+0000","last_became_peered":"2024-09-10T21:18:54.135939+0000","last_unstale":"2024-09-10T21:18:55.117030+0000","last_undegraded":"2024-09-10T21:18:55.117030+0000","last_fullsized":"2024-09-10T21:18:55.117030+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:22.677132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124139+0000","last_change":"2024-09-10T21:18:53.129464+0000","last_active":"2024-09-10T21:18:56.124139+0000","last_peered":"2024-09-10T21:18:56.124139+0000","last_clean":"2024-09-10T21:18:56.124139+0000","last_became_active":"2024-09-10T21:18:53.129179+0000","last_became_peered":"2024-09-10T21:18:53.129179+0000","last_unstale":"2024-09-10T21:18:56.124139+0000","last_undegraded":"2024-09-10T21:18:56.124139+0000","last_fullsized":"2024-09-10T21:18:56.124139+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:58.176727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117072+0000","last_change":"2024-09-10T21:18:54.130958+0000","last_active":"2024-09-10T21:18:55.117072+0000","last_peered":"2024-09-10T21:18:55.117072+0000","last_clean":"2024-09-10T21:18:55.117072+0000","last_became_active":"2024-09-10T21:18:54.130681+0000","last_became_peered":"2024-09-10T21:18:54.130681+0000","last_unstale":"2024-09-10T21:18:55.117072+0000","last_undegraded":"2024-09-10T21:18:55.117072+0000","last_fullsized":"2024-09-10T21:18:55.117072+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:23.939236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116973+0000","last_change":"2024-09-10T21:18:53.127488+0000","last_active":"2024-09-10T21:18:55.116973+0000","last_peered":"2024-09-10T21:18:55.116973+0000","last_clean":"2024-09-10T21:18:55.116973+0000","last_became_active":"2024-09-10T21:18:53.125698+0000","last_became_peered":"2024-09-10T21:18:53.125698+0000","last_unstale":"2024-09-10T21:18:55.116973+0000","last_undegraded":"2024-09-10T21:18:55.116973+0000","last_fullsized":"2024-09-10T21:18:55.116973+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:29.175487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412963+0000","last_change":"2024-09-10T21:18:54.143008+0000","last_active":"2024-09-10T21:18:55.412963+0000","last_peered":"2024-09-10T21:18:55.412963+0000","last_clean":"2024-09-10T21:18:55.412963+0000","last_became_active":"2024-09-10T21:18:54.142077+0000","last_became_peered":"2024-09-10T21:18:54.142077+0000","last_unstale":"2024-09-10T21:18:55.412963+0000","last_undegraded":"2024-09-10T21:18:55.412963+0000","last_fullsized":"2024-09-10T21:18:55.412963+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:14.772219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714227+0000","last_change":"2024-09-10T21:18:53.137421+0000","last_active":"2024-09-10T21:18:57.714227+0000","last_peered":"2024-09-10T21:18:57.714227+0000","last_clean":"2024-09-10T21:18:57.714227+0000","last_became_active":"2024-09-10T21:18:53.136850+0000","last_became_peered":"2024-09-10T21:18:53.136850+0000","last_unstale":"2024-09-10T21:18:57.714227+0000","last_undegraded":"2024-09-10T21:18:57.714227+0000","last_fullsized":"2024-09-10T21:18:57.714227+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:54.955222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412863+0000","last_change":"2024-09-10T21:18:54.142684+0000","last_active":"2024-09-10T21:18:55.412863+0000","last_peered":"2024-09-10T21:18:55.412863+0000","last_clean":"2024-09-10T21:18:55.412863+0000","last_became_active":"2024-09-10T21:18:54.142361+0000","last_became_peered":"2024-09-10T21:18:54.142361+0000","last_unstale":"2024-09-10T21:18:55.412863+0000","last_undegraded":"2024-09-10T21:18:55.412863+0000","last_fullsized":"2024-09-10T21:18:55.412863+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:37.685415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"137'3","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117466+0000","last_change":"2024-09-10T21:18:53.134405+0000","last_active":"2024-09-10T21:18:55.117466+0000","last_peered":"2024-09-10T21:18:55.117466+0000","last_clean":"2024-09-10T21:18:55.117466+0000","last_became_active":"2024-09-10T21:18:53.134012+0000","last_became_peered":"2024-09-10T21:18:53.134012+0000","last_unstale":"2024-09-10T21:18:55.117466+0000","last_undegraded":"2024-09-10T21:18:55.117466+0000","last_fullsized":"2024-09-10T21:18:55.117466+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:59.126787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124304+0000","last_change":"2024-09-10T21:18:54.137307+0000","last_active":"2024-09-10T21:18:56.124304+0000","last_peered":"2024-09-10T21:18:56.124304+0000","last_clean":"2024-09-10T21:18:56.124304+0000","last_became_active":"2024-09-10T21:18:54.135312+0000","last_became_peered":"2024-09-10T21:18:54.135312+0000","last_unstale":"2024-09-10T21:18:56.124304+0000","last_undegraded":"2024-09-10T21:18:56.124304+0000","last_fullsized":"2024-09-10T21:18:56.124304+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:02.586421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"137'6","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536576+0000","last_change":"2024-09-10T21:18:53.135320+0000","last_active":"2024-09-10T21:18:57.536576+0000","last_peered":"2024-09-10T21:18:57.536576+0000","last_clean":"2024-09-10T21:18:57.536576+0000","last_became_active":"2024-09-10T21:18:53.134966+0000","last_became_peered":"2024-09-10T21:18:53.134966+0000","last_unstale":"2024-09-10T21:18:57.536576+0000","last_undegraded":"2024-09-10T21:18:57.536576+0000","last_fullsized":"2024-09-10T21:18:57.536576+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:43.118277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412589+0000","last_change":"2024-09-10T21:18:54.135436+0000","last_active":"2024-09-10T21:18:55.412589+0000","last_peered":"2024-09-10T21:18:55.412589+0000","last_clean":"2024-09-10T21:18:55.412589+0000","last_became_active":"2024-09-10T21:18:54.135110+0000","last_became_peered":"2024-09-10T21:18:54.135110+0000","last_unstale":"2024-09-10T21:18:55.412589+0000","last_undegraded":"2024-09-10T21:18:55.412589+0000","last_fullsized":"2024-09-10T21:18:55.412589+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:50.596414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118490+0000","last_change":"2024-09-10T21:18:53.127007+0000","last_active":"2024-09-10T21:18:55.118490+0000","last_peered":"2024-09-10T21:18:55.118490+0000","last_clean":"2024-09-10T21:18:55.118490+0000","last_became_active":"2024-09-10T21:18:53.126673+0000","last_became_peered":"2024-09-10T21:18:53.126673+0000","last_unstale":"2024-09-10T21:18:55.118490+0000","last_undegraded":"2024-09-10T21:18:55.118490+0000","last_fullsized":"2024-09-10T21:18:55.118490+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:59.126787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117775+0000","last_change":"2024-09-10T21:18:54.147288+0000","last_active":"2024-09-10T21:18:55.117775+0000","last_peered":"2024-09-10T21:18:55.117775+0000","last_clean":"2024-09-10T21:18:55.117775+0000","last_became_active":"2024-09-10T21:18:54.145370+0000","last_became_peered":"2024-09-10T21:18:54.145370+0000","last_unstale":"2024-09-10T21:18:55.117775+0000","last_undegraded":"2024-09-10T21:18:55.117775+0000","last_fullsized":"2024-09-10T21:18:55.117775+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:30.693628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714096+0000","last_change":"2024-09-10T21:18:53.133200+0000","last_active":"2024-09-10T21:18:57.714096+0000","last_peered":"2024-09-10T21:18:57.714096+0000","last_clean":"2024-09-10T21:18:57.714096+0000","last_became_active":"2024-09-10T21:18:53.132906+0000","last_became_peered":"2024-09-10T21:18:53.132906+0000","last_unstale":"2024-09-10T21:18:57.714096+0000","last_undegraded":"2024-09-10T21:18:57.714096+0000","last_fullsized":"2024-09-10T21:18:57.714096+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:02.410211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.713910+0000","last_change":"2024-09-10T21:18:54.148718+0000","last_active":"2024-09-10T21:18:57.713910+0000","last_peered":"2024-09-10T21:18:57.713910+0000","last_clean":"2024-09-10T21:18:57.713910+0000","last_became_active":"2024-09-10T21:18:54.148438+0000","last_became_peered":"2024-09-10T21:18:54.148438+0000","last_unstale":"2024-09-10T21:18:57.713910+0000","last_undegraded":"2024-09-10T21:18:57.713910+0000","last_fullsized":"2024-09-10T21:18:57.713910+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:53.066340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412710+0000","last_change":"2024-09-10T21:18:53.137417+0000","last_active":"2024-09-10T21:18:55.412710+0000","last_peered":"2024-09-10T21:18:55.412710+0000","last_clean":"2024-09-10T21:18:55.412710+0000","last_became_active":"2024-09-10T21:18:53.137042+0000","last_became_peered":"2024-09-10T21:18:53.137042+0000","last_unstale":"2024-09-10T21:18:55.412710+0000","last_undegraded":"2024-09-10T21:18:55.412710+0000","last_fullsized":"2024-09-10T21:18:55.412710+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:00.755770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714511+0000","last_change":"2024-09-10T21:18:54.137500+0000","last_active":"2024-09-10T21:18:57.714511+0000","last_peered":"2024-09-10T21:18:57.714511+0000","last_clean":"2024-09-10T21:18:57.714511+0000","last_became_active":"2024-09-10T21:18:54.136137+0000","last_became_peered":"2024-09-10T21:18:54.136137+0000","last_unstale":"2024-09-10T21:18:57.714511+0000","last_undegraded":"2024-09-10T21:18:57.714511+0000","last_fullsized":"2024-09-10T21:18:57.714511+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:59.183467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"137'8","reported_seq":23,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536249+0000","last_change":"2024-09-10T21:18:53.135369+0000","last_active":"2024-09-10T21:18:57.536249+0000","last_peered":"2024-09-10T21:18:57.536249+0000","last_clean":"2024-09-10T21:18:57.536249+0000","last_became_active":"2024-09-10T21:18:53.133318+0000","last_became_peered":"2024-09-10T21:18:53.133318+0000","last_unstale":"2024-09-10T21:18:57.536249+0000","last_undegraded":"2024-09-10T21:18:57.536249+0000","last_fullsized":"2024-09-10T21:18:57.536249+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:52.963942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.413070+0000","last_change":"2024-09-10T21:18:54.126929+0000","last_active":"2024-09-10T21:18:55.413070+0000","last_peered":"2024-09-10T21:18:55.413070+0000","last_clean":"2024-09-10T21:18:55.413070+0000","last_became_active":"2024-09-10T21:18:54.126664+0000","last_became_peered":"2024-09-10T21:18:54.126664+0000","last_unstale":"2024-09-10T21:18:55.413070+0000","last_undegraded":"2024-09-10T21:18:55.413070+0000","last_fullsized":"2024-09-10T21:18:55.413070+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:08.370813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536672+0000","last_change":"2024-09-10T21:18:53.139526+0000","last_active":"2024-09-10T21:18:57.536672+0000","last_peered":"2024-09-10T21:18:57.536672+0000","last_clean":"2024-09-10T21:18:57.536672+0000","last_became_active":"2024-09-10T21:18:53.139137+0000","last_became_peered":"2024-09-10T21:18:53.139137+0000","last_unstale":"2024-09-10T21:18:57.536672+0000","last_undegraded":"2024-09-10T21:18:57.536672+0000","last_fullsized":"2024-09-10T21:18:57.536672+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:30.445921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.119156+0000","last_change":"2024-09-10T21:18:54.140068+0000","last_active":"2024-09-10T21:18:55.119156+0000","last_peered":"2024-09-10T21:18:55.119156+0000","last_clean":"2024-09-10T21:18:55.119156+0000","last_became_active":"2024-09-10T21:18:54.139359+0000","last_became_peered":"2024-09-10T21:18:54.139359+0000","last_unstale":"2024-09-10T21:18:55.119156+0000","last_undegraded":"2024-09-10T21:18:55.119156+0000","last_fullsized":"2024-09-10T21:18:55.119156+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:24.562500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714907+0000","last_change":"2024-09-10T21:18:53.136538+0000","last_active":"2024-09-10T21:18:57.714907+0000","last_peered":"2024-09-10T21:18:57.714907+0000","last_clean":"2024-09-10T21:18:57.714907+0000","last_became_active":"2024-09-10T21:18:53.135006+0000","last_became_peered":"2024-09-10T21:18:53.135006+0000","last_unstale":"2024-09-10T21:18:57.714907+0000","last_undegraded":"2024-09-10T21:18:57.714907+0000","last_fullsized":"2024-09-10T21:18:57.714907+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:25.364169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118736+0000","last_change":"2024-09-10T21:18:54.131993+0000","last_active":"2024-09-10T21:18:55.118736+0000","last_peered":"2024-09-10T21:18:55.118736+0000","last_clean":"2024-09-10T21:18:55.118736+0000","last_became_active":"2024-09-10T21:18:54.131682+0000","last_became_peered":"2024-09-10T21:18:54.131682+0000","last_unstale":"2024-09-10T21:18:55.118736+0000","last_undegraded":"2024-09-10T21:18:55.118736+0000","last_fullsized":"2024-09-10T21:18:55.118736+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:47:49.255213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536749+0000","last_change":"2024-09-10T21:18:53.138468+0000","last_active":"2024-09-10T21:18:57.536749+0000","last_peered":"2024-09-10T21:18:57.536749+0000","last_clean":"2024-09-10T21:18:57.536749+0000","last_became_active":"2024-09-10T21:18:53.138185+0000","last_became_peered":"2024-09-10T21:18:53.138185+0000","last_unstale":"2024-09-10T21:18:57.536749+0000","last_undegraded":"2024-09-10T21:18:57.536749+0000","last_fullsized":"2024-09-10T21:18:57.536749+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:27:45.345741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117457+0000","last_change":"2024-09-10T21:18:54.130564+0000","last_active":"2024-09-10T21:18:55.117457+0000","last_peered":"2024-09-10T21:18:55.117457+0000","last_clean":"2024-09-10T21:18:55.117457+0000","last_became_active":"2024-09-10T21:18:54.130232+0000","last_became_peered":"2024-09-10T21:18:54.130232+0000","last_unstale":"2024-09-10T21:18:55.117457+0000","last_undegraded":"2024-09-10T21:18:55.117457+0000","last_fullsized":"2024-09-10T21:18:55.117457+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:00.051016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117000+0000","last_change":"2024-09-10T21:18:53.126832+0000","last_active":"2024-09-10T21:18:55.117000+0000","last_peered":"2024-09-10T21:18:55.117000+0000","last_clean":"2024-09-10T21:18:55.117000+0000","last_became_active":"2024-09-10T21:18:53.126580+0000","last_became_peered":"2024-09-10T21:18:53.126580+0000","last_unstale":"2024-09-10T21:18:55.117000+0000","last_undegraded":"2024-09-10T21:18:55.117000+0000","last_fullsized":"2024-09-10T21:18:55.117000+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:02.216397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.123947+0000","last_change":"2024-09-10T21:18:54.129175+0000","last_active":"2024-09-10T21:18:56.123947+0000","last_peered":"2024-09-10T21:18:56.123947+0000","last_clean":"2024-09-10T21:18:56.123947+0000","last_became_active":"2024-09-10T21:18:54.128849+0000","last_became_peered":"2024-09-10T21:18:54.128849+0000","last_unstale":"2024-09-10T21:18:56.123947+0000","last_undegraded":"2024-09-10T21:18:56.123947+0000","last_fullsized":"2024-09-10T21:18:56.123947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:40.590604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118066+0000","last_change":"2024-09-10T21:18:53.129128+0000","last_active":"2024-09-10T21:18:55.118066+0000","last_peered":"2024-09-10T21:18:55.118066+0000","last_clean":"2024-09-10T21:18:55.118066+0000","last_became_active":"2024-09-10T21:18:53.128826+0000","last_became_peered":"2024-09-10T21:18:53.128826+0000","last_unstale":"2024-09-10T21:18:55.118066+0000","last_undegraded":"2024-09-10T21:18:55.118066+0000","last_fullsized":"2024-09-10T21:18:55.118066+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:12.376207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118060+0000","last_change":"2024-09-10T21:18:54.148318+0000","last_active":"2024-09-10T21:18:55.118060+0000","last_peered":"2024-09-10T21:18:55.118060+0000","last_clean":"2024-09-10T21:18:55.118060+0000","last_became_active":"2024-09-10T21:18:54.145050+0000","last_became_peered":"2024-09-10T21:18:54.145050+0000","last_unstale":"2024-09-10T21:18:55.118060+0000","last_undegraded":"2024-09-10T21:18:55.118060+0000","last_fullsized":"2024-09-10T21:18:55.118060+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:14.320404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116471+0000","last_change":"2024-09-10T21:18:53.128814+0000","last_active":"2024-09-10T21:18:55.116471+0000","last_peered":"2024-09-10T21:18:55.116471+0000","last_clean":"2024-09-10T21:18:55.116471+0000","last_became_active":"2024-09-10T21:18:53.128474+0000","last_became_peered":"2024-09-10T21:18:53.128474+0000","last_unstale":"2024-09-10T21:18:55.116471+0000","last_undegraded":"2024-09-10T21:18:55.116471+0000","last_fullsized":"2024-09-10T21:18:55.116471+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:43.988080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.123874+0000","last_change":"2024-09-10T21:18:54.123449+0000","last_active":"2024-09-10T21:18:56.123874+0000","last_peered":"2024-09-10T21:18:56.123874+0000","last_clean":"2024-09-10T21:18:56.123874+0000","last_became_active":"2024-09-10T21:18:54.123152+0000","last_became_peered":"2024-09-10T21:18:54.123152+0000","last_unstale":"2024-09-10T21:18:56.123874+0000","last_undegraded":"2024-09-10T21:18:56.123874+0000","last_fullsized":"2024-09-10T21:18:56.123874+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:14.790647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117929+0000","last_change":"2024-09-10T21:18:53.132527+0000","last_active":"2024-09-10T21:18:55.117929+0000","last_peered":"2024-09-10T21:18:55.117929+0000","last_clean":"2024-09-10T21:18:55.117929+0000","last_became_active":"2024-09-10T21:18:53.130251+0000","last_became_peered":"2024-09-10T21:18:53.130251+0000","last_unstale":"2024-09-10T21:18:55.117929+0000","last_undegraded":"2024-09-10T21:18:55.117929+0000","last_fullsized":"2024-09-10T21:18:55.117929+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:36.678675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412524+0000","last_change":"2024-09-10T21:18:54.137449+0000","last_active":"2024-09-10T21:18:55.412524+0000","last_peered":"2024-09-10T21:18:55.412524+0000","last_clean":"2024-09-10T21:18:55.412524+0000","last_became_active":"2024-09-10T21:18:54.137183+0000","last_became_peered":"2024-09-10T21:18:54.137183+0000","last_unstale":"2024-09-10T21:18:55.412524+0000","last_undegraded":"2024-09-10T21:18:55.412524+0000","last_fullsized":"2024-09-10T21:18:55.412524+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:03.697968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116541+0000","last_change":"2024-09-10T21:18:53.133231+0000","last_active":"2024-09-10T21:18:55.116541+0000","last_peered":"2024-09-10T21:18:55.116541+0000","last_clean":"2024-09-10T21:18:55.116541+0000","last_became_active":"2024-09-10T21:18:53.132930+0000","last_became_peered":"2024-09-10T21:18:53.132930+0000","last_unstale":"2024-09-10T21:18:55.116541+0000","last_undegraded":"2024-09-10T21:18:55.116541+0000","last_fullsized":"2024-09-10T21:18:55.116541+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:08.667229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118234+0000","last_change":"2024-09-10T21:18:54.144376+0000","last_active":"2024-09-10T21:18:55.118234+0000","last_peered":"2024-09-10T21:18:55.118234+0000","last_clean":"2024-09-10T21:18:55.118234+0000","last_became_active":"2024-09-10T21:18:54.143762+0000","last_became_peered":"2024-09-10T21:18:54.143762+0000","last_unstale":"2024-09-10T21:18:55.118234+0000","last_undegraded":"2024-09-10T21:18:55.118234+0000","last_fullsized":"2024-09-10T21:18:55.118234+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:18.195215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714325+0000","last_change":"2024-09-10T21:18:53.134435+0000","last_active":"2024-09-10T21:18:57.714325+0000","last_peered":"2024-09-10T21:18:57.714325+0000","last_clean":"2024-09-10T21:18:57.714325+0000","last_became_active":"2024-09-10T21:18:53.133087+0000","last_became_peered":"2024-09-10T21:18:53.133087+0000","last_unstale":"2024-09-10T21:18:57.714325+0000","last_undegraded":"2024-09-10T21:18:57.714325+0000","last_fullsized":"2024-09-10T21:18:57.714325+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:12.570896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.119187+0000","last_change":"2024-09-10T21:18:54.131428+0000","last_active":"2024-09-10T21:18:55.119187+0000","last_peered":"2024-09-10T21:18:55.119187+0000","last_clean":"2024-09-10T21:18:55.119187+0000","last_became_active":"2024-09-10T21:18:54.131238+0000","last_became_peered":"2024-09-10T21:18:54.131238+0000","last_unstale":"2024-09-10T21:18:55.119187+0000","last_undegraded":"2024-09-10T21:18:55.119187+0000","last_fullsized":"2024-09-10T21:18:55.119187+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:17.093434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"138'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.116884+0000","last_change":"2024-09-10T21:18:53.130476+0000","last_active":"2024-09-10T21:18:55.116884+0000","last_peered":"2024-09-10T21:18:55.116884+0000","last_clean":"2024-09-10T21:18:55.116884+0000","last_became_active":"2024-09-10T21:18:53.130213+0000","last_became_peered":"2024-09-10T21:18:53.130213+0000","last_unstale":"2024-09-10T21:18:55.116884+0000","last_undegraded":"2024-09-10T21:18:55.116884+0000","last_fullsized":"2024-09-10T21:18:55.116884+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:02.172443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118851+0000","last_change":"2024-09-10T21:18:54.132591+0000","last_active":"2024-09-10T21:18:55.118851+0000","last_peered":"2024-09-10T21:18:55.118851+0000","last_clean":"2024-09-10T21:18:55.118851+0000","last_became_active":"2024-09-10T21:18:54.132251+0000","last_became_peered":"2024-09-10T21:18:54.132251+0000","last_unstale":"2024-09-10T21:18:55.118851+0000","last_undegraded":"2024-09-10T21:18:55.118851+0000","last_fullsized":"2024-09-10T21:18:55.118851+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:38.405472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124603+0000","last_change":"2024-09-10T21:18:53.130268+0000","last_active":"2024-09-10T21:18:56.124603+0000","last_peered":"2024-09-10T21:18:56.124603+0000","last_clean":"2024-09-10T21:18:56.124603+0000","last_became_active":"2024-09-10T21:18:53.129955+0000","last_became_peered":"2024-09-10T21:18:53.129955+0000","last_unstale":"2024-09-10T21:18:56.124603+0000","last_undegraded":"2024-09-10T21:18:56.124603+0000","last_fullsized":"2024-09-10T21:18:56.124603+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:37.865123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.118634+0000","last_change":"2024-09-10T21:18:54.143312+0000","last_active":"2024-09-10T21:18:55.118634+0000","last_peered":"2024-09-10T21:18:55.118634+0000","last_clean":"2024-09-10T21:18:55.118634+0000","last_became_active":"2024-09-10T21:18:54.143013+0000","last_became_peered":"2024-09-10T21:18:54.143013+0000","last_unstale":"2024-09-10T21:18:55.118634+0000","last_undegraded":"2024-09-10T21:18:55.118634+0000","last_fullsized":"2024-09-10T21:18:55.118634+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:40.688104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.117862+0000","last_change":"2024-09-10T21:18:53.133483+0000","last_active":"2024-09-10T21:18:55.117862+0000","last_peered":"2024-09-10T21:18:55.117862+0000","last_clean":"2024-09-10T21:18:55.117862+0000","last_became_active":"2024-09-10T21:18:53.133196+0000","last_became_peered":"2024-09-10T21:18:53.133196+0000","last_unstale":"2024-09-10T21:18:55.117862+0000","last_undegraded":"2024-09-10T21:18:55.117862+0000","last_fullsized":"2024-09-10T21:18:55.117862+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:02.691227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.714013+0000","last_change":"2024-09-10T21:18:54.149512+0000","last_active":"2024-09-10T21:18:57.714013+0000","last_peered":"2024-09-10T21:18:57.714013+0000","last_clean":"2024-09-10T21:18:57.714013+0000","last_became_active":"2024-09-10T21:18:54.148990+0000","last_became_peered":"2024-09-10T21:18:54.148990+0000","last_unstale":"2024-09-10T21:18:57.714013+0000","last_undegraded":"2024-09-10T21:18:57.714013+0000","last_fullsized":"2024-09-10T21:18:57.714013+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:49.480369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536517+0000","last_change":"2024-09-10T21:18:53.137039+0000","last_active":"2024-09-10T21:18:57.536517+0000","last_peered":"2024-09-10T21:18:57.536517+0000","last_clean":"2024-09-10T21:18:57.536517+0000","last_became_active":"2024-09-10T21:18:53.136763+0000","last_became_peered":"2024-09-10T21:18:53.136763+0000","last_unstale":"2024-09-10T21:18:57.536517+0000","last_undegraded":"2024-09-10T21:18:57.536517+0000","last_fullsized":"2024-09-10T21:18:57.536517+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:32.308582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124008+0000","last_change":"2024-09-10T21:18:54.130766+0000","last_active":"2024-09-10T21:18:56.124008+0000","last_peered":"2024-09-10T21:18:56.124008+0000","last_clean":"2024-09-10T21:18:56.124008+0000","last_became_active":"2024-09-10T21:18:54.130480+0000","last_became_peered":"2024-09-10T21:18:54.130480+0000","last_unstale":"2024-09-10T21:18:56.124008+0000","last_undegraded":"2024-09-10T21:18:56.124008+0000","last_fullsized":"2024-09-10T21:18:56.124008+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:48.359201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.412532+0000","last_change":"2024-09-10T21:18:53.133724+0000","last_active":"2024-09-10T21:18:55.412532+0000","last_peered":"2024-09-10T21:18:55.412532+0000","last_clean":"2024-09-10T21:18:55.412532+0000","last_became_active":"2024-09-10T21:18:53.133442+0000","last_became_peered":"2024-09-10T21:18:53.133442+0000","last_unstale":"2024-09-10T21:18:55.412532+0000","last_undegraded":"2024-09-10T21:18:55.412532+0000","last_fullsized":"2024-09-10T21:18:55.412532+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:04.295889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.413079+0000","last_change":"2024-09-10T21:18:54.137184+0000","last_active":"2024-09-10T21:18:55.413079+0000","last_peered":"2024-09-10T21:18:55.413079+0000","last_clean":"2024-09-10T21:18:55.413079+0000","last_became_active":"2024-09-10T21:18:54.136895+0000","last_became_peered":"2024-09-10T21:18:54.136895+0000","last_unstale":"2024-09-10T21:18:55.413079+0000","last_undegraded":"2024-09-10T21:18:55.413079+0000","last_fullsized":"2024-09-10T21:18:55.413079+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:53.555070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.124264+0000","last_change":"2024-09-10T21:18:53.132411+0000","last_active":"2024-09-10T21:18:56.124264+0000","last_peered":"2024-09-10T21:18:56.124264+0000","last_clean":"2024-09-10T21:18:56.124264+0000","last_became_active":"2024-09-10T21:18:53.131949+0000","last_became_peered":"2024-09-10T21:18:53.131949+0000","last_unstale":"2024-09-10T21:18:56.124264+0000","last_undegraded":"2024-09-10T21:18:56.124264+0000","last_fullsized":"2024-09-10T21:18:56.124264+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:02.387195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.123873+0000","last_change":"2024-09-10T21:18:54.144243+0000","last_active":"2024-09-10T21:18:56.123873+0000","last_peered":"2024-09-10T21:18:56.123873+0000","last_clean":"2024-09-10T21:18:56.123873+0000","last_became_active":"2024-09-10T21:18:54.143449+0000","last_became_peered":"2024-09-10T21:18:54.143449+0000","last_unstale":"2024-09-10T21:18:56.123873+0000","last_undegraded":"2024-09-10T21:18:56.123873+0000","last_fullsized":"2024-09-10T21:18:56.123873+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:01.161281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:56.123801+0000","last_change":"2024-09-10T21:18:53.127615+0000","last_active":"2024-09-10T21:18:56.123801+0000","last_peered":"2024-09-10T21:18:56.123801+0000","last_clean":"2024-09-10T21:18:56.123801+0000","last_became_active":"2024-09-10T21:18:53.127317+0000","last_became_peered":"2024-09-10T21:18:53.127317+0000","last_unstale":"2024-09-10T21:18:56.123801+0000","last_undegraded":"2024-09-10T21:18:56.123801+0000","last_fullsized":"2024-09-10T21:18:56.123801+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:54.218681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:55.413032+0000","last_change":"2024-09-10T21:18:54.134957+0000","last_active":"2024-09-10T21:18:55.413032+0000","last_peered":"2024-09-10T21:18:55.413032+0000","last_clean":"2024-09-10T21:18:55.413032+0000","last_became_active":"2024-09-10T21:18:54.128460+0000","last_became_peered":"2024-09-10T21:18:54.128460+0000","last_unstale":"2024-09-10T21:18:55.413032+0000","last_undegraded":"2024-09-10T21:18:55.413032+0000","last_fullsized":"2024-09-10T21:18:55.413032+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:53.102675+0000","last_clean_scrub_stamp":"2024-09-10T21:18:53.102675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:49.710726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536622+0000","last_change":"2024-09-10T21:18:53.136422+0000","last_active":"2024-09-10T21:18:57.536622+0000","last_peered":"2024-09-10T21:18:57.536622+0000","last_clean":"2024-09-10T21:18:57.536622+0000","last_became_active":"2024-09-10T21:18:53.136132+0000","last_became_peered":"2024-09-10T21:18:53.136132+0000","last_unstale":"2024-09-10T21:18:57.536622+0000","last_undegraded":"2024-09-10T21:18:57.536622+0000","last_fullsized":"2024-09-10T21:18:57.536622+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:18:52.095934+0000","last_clean_scrub_stamp":"2024-09-10T21:18:52.095934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:30.203255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":319,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T21:18:57.536713+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:18:57.536713+0000","last_peered":"2024-09-10T21:18:57.536713+0000","last_clean":"2024-09-10T21:18:57.536713+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:18:57.536713+0000","last_undegraded":"2024-09-10T21:18:57.536713+0000","last_fullsized":"2024-09-10T21:18:57.536713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37040,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334800,"statfs":{"total":96636764160,"available":96598835200,"internally_reserved":0,"allocated":1732608,"data_stored":746399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509549,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37624,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334216,"statfs":{"total":96636764160,"available":96598237184,"internally_reserved":0,"allocated":2330624,"data_stored":1337405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509547,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37040,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334800,"statfs":{"total":96636764160,"available":96598835200,"internally_reserved":0,"allocated":1732608,"data_stored":746399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509546,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37056,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334784,"statfs":{"total":96636764160,"available":96598818816,"internally_reserved":0,"allocated":1744896,"data_stored":747999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509548,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1732608,"data_stored":746947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509547,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37040,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334800,"statfs":{"total":96636764160,"available":96598835200,"internally_reserved":0,"allocated":1732608,"data_stored":746435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542251,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41192,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330648,"statfs":{"total":96636764160,"available":96594583552,"internally_reserved":0,"allocated":1716224,"data_stored":740065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542250,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37640,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334200,"statfs":{"total":96636764160,"available":96598220800,"internally_reserved":0,"allocated":2347008,"data_stored":1338527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:19:03.292 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:19:03.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:19:03.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 -- 172.21.15.16:0/1182713288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5380d0210 msgr2=0x7fc5380cead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:03.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 --2- 172.21.15.16:0/1182713288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5380d0210 0x7fc5380cead0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc5380cda10 tx=0x7fc52001f090 comp rx=0 tx=0).stop 2024-09-10T21:19:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 -- 172.21.15.16:0/1182713288 shutdown_connections 2024-09-10T21:19:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 --2- 172.21.15.16:0/1182713288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5380d0210 0x7fc5380cead0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 --2- 172.21.15.16:0/1182713288 >> v2:172.21.15.52:3300/0 conn(0x7fc5380cfe30 0x7fc5380ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 -- 172.21.15.16:0/1182713288 >> 172.21.15.16:0/1182713288 conn(0x7fc5380d1df0 msgr2=0x7fc5380d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 -- 172.21.15.16:0/1182713288 shutdown_connections 2024-09-10T21:19:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 -- 172.21.15.16:0/1182713288 wait complete. 2024-09-10T21:19:03.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 Processor -- start 2024-09-10T21:19:03.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 -- start start 2024-09-10T21:19:03.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc5380ce040 0x7fc538157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:03.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5380cfe30 0x7fc5381e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5380d0210 0x7fc5380b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5380b12e0 con 0x7fc5380d0210 2024-09-10T21:19:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc5380d2b20 con 0x7fc5380cfe30 2024-09-10T21:19:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ee00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc5380b1460 con 0x7fc5380ce040 2024-09-10T21:19:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5380d0210 0x7fc5380b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5380d0210 0x7fc5380b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38082/0 (socket says 172.21.15.16:38082) 2024-09-10T21:19:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc53ce00640 1 -- 172.21.15.16:0/1668103376 learned_addr learned my addr 172.21.15.16:0/1668103376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc52fe00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5380cfe30 0x7fc5381e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.484+0000 7fc537e00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc5380ce040 0x7fc538157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc53ce00640 1 -- 172.21.15.16:0/1668103376 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc5380ce040 msgr2=0x7fc538157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc53ce00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc5380ce040 0x7fc538157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc53ce00640 1 -- 172.21.15.16:0/1668103376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5380cfe30 msgr2=0x7fc5381e4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc53ce00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5380cfe30 0x7fc5381e4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc53ce00640 1 -- 172.21.15.16:0/1668103376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5381f5710 con 0x7fc5380d0210 2024-09-10T21:19:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc53ce00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5380d0210 0x7fc5380b68b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc53000c7b0 tx=0x7fc53000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc534c00640 1 -- 172.21.15.16:0/1668103376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530004b70 con 0x7fc5380d0210 2024-09-10T21:19:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc534c00640 1 -- 172.21.15.16:0/1668103376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc530004e00 con 0x7fc5380d0210 2024-09-10T21:19:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc534c00640 1 -- 172.21.15.16:0/1668103376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53000aab0 con 0x7fc5380d0210 2024-09-10T21:19:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc53ee00640 1 -- 172.21.15.16:0/1668103376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5381f59f0 con 0x7fc5380d0210 2024-09-10T21:19:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc53ee00640 1 -- 172.21.15.16:0/1668103376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5381e78b0 con 0x7fc5380d0210 2024-09-10T21:19:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc53ee00640 1 -- 172.21.15.16:0/1668103376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5380cf2b0 con 0x7fc5380d0210 2024-09-10T21:19:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc534c00640 1 -- 172.21.15.16:0/1668103376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc53000ac50 con 0x7fc5380d0210 2024-09-10T21:19:03.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.488+0000 7fc534c00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc5140725e0 0x7fc514074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:03.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.492+0000 7fc534c00640 1 -- 172.21.15.16:0/1668103376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fc5300933e0 con 0x7fc5380d0210 2024-09-10T21:19:03.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.492+0000 7fc537e00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc5140725e0 0x7fc514074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:03.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.492+0000 7fc537e00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc5140725e0 0x7fc514074aa0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc52800bc00 tx=0x7fc52800b390 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.492+0000 7fc534c00640 1 -- 172.21.15.16:0/1668103376 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc530020070 con 0x7fc5380d0210 2024-09-10T21:19:03.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.732+0000 7fc53ee00640 1 -- 172.21.15.16:0/1668103376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc53806f500 con 0x7fc5380d0210 2024-09-10T21:19:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.736+0000 7fc534c00640 1 -- 172.21.15.16:0/1668103376 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v32) ==== 190+0+147 (secure 0 0 0) 0x7fc53005e460 con 0x7fc5380d0210 2024-09-10T21:19:03.740 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-10T21:19:03.740 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGteBmF9AhARAAnWapaIBMGCOGbJc/p8LLjA== 2024-09-10T21:19:03.740 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-10T21:19:03.740 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-10T21:19:03.740 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:19:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-10T21:19:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.736+0000 7fc53ee00640 1 -- 172.21.15.16:0/1668103376 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc5140725e0 msgr2=0x7fc514074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.736+0000 7fc53ee00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc5140725e0 0x7fc514074aa0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc52800bc00 tx=0x7fc52800b390 comp rx=0 tx=0).stop 2024-09-10T21:19:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.736+0000 7fc53ee00640 1 -- 172.21.15.16:0/1668103376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5380d0210 msgr2=0x7fc5380b68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.736+0000 7fc53ee00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5380d0210 0x7fc5380b68b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc53000c7b0 tx=0x7fc53000cc70 comp rx=0 tx=0).stop 2024-09-10T21:19:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.736+0000 7fc53ee00640 1 -- 172.21.15.16:0/1668103376 shutdown_connections 2024-09-10T21:19:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.736+0000 7fc53ee00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc5140725e0 0x7fc514074aa0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.736+0000 7fc53ee00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5380d0210 0x7fc5380b68b0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.736+0000 7fc53ee00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5380cfe30 0x7fc5381e4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.736+0000 7fc53ee00640 1 --2- 172.21.15.16:0/1668103376 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc5380ce040 0x7fc538157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.736+0000 7fc53ee00640 1 -- 172.21.15.16:0/1668103376 >> 172.21.15.16:0/1668103376 conn(0x7fc5380d1df0 msgr2=0x7fc5380d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.736+0000 7fc53ee00640 1 -- 172.21.15.16:0/1668103376 shutdown_connections 2024-09-10T21:19:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.736+0000 7fc53ee00640 1 -- 172.21.15.16:0/1668103376 wait complete. 2024-09-10T21:19:03.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:19:03.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.904+0000 7fe9b1a00640 1 -- 172.21.15.16:0/2448039751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990004d50 msgr2=0x7fe9900051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:03.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.904+0000 7fe9b1a00640 1 --2- 172.21.15.16:0/2448039751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990004d50 0x7fe9900051d0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009ee0 tx=0x7fe9a401d5f0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.904+0000 7fe9ab400640 1 -- 172.21.15.16:0/2448039751 reap_dead start 2024-09-10T21:19:03.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.904+0000 7fe9b1a00640 1 -- 172.21.15.16:0/2448039751 shutdown_connections 2024-09-10T21:19:03.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.904+0000 7fe9b1a00640 1 -- 172.21.15.16:0/2448039751 >> 172.21.15.16:0/2448039751 conn(0x7fe9ac0d1d80 msgr2=0x7fe9ac0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:03.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.904+0000 7fe9b1a00640 1 -- 172.21.15.16:0/2448039751 shutdown_connections 2024-09-10T21:19:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.904+0000 7fe9b1a00640 1 -- 172.21.15.16:0/2448039751 wait complete. 2024-09-10T21:19:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9b1a00640 1 Processor -- start 2024-09-10T21:19:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9b1a00640 1 -- start start 2024-09-10T21:19:03.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9b1a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe990004d50 0x7fe9ac071980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:03.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9b1a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ac1521d0 0x7fe9ac152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:03.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9b1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ac0cec20 0x7fe9ac0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9b1a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac0b2e30 con 0x7fe9ac0cec20 2024-09-10T21:19:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9b1a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac0b2cb0 con 0x7fe9ac1521d0 2024-09-10T21:19:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9b1a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ac0b2fb0 con 0x7fe990004d50 2024-09-10T21:19:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9ab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ac0cec20 0x7fe9ac0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9aaa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe990004d50 0x7fe9ac071980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9ab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ac0cec20 0x7fe9ac0cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38102/0 (socket says 172.21.15.16:38102) 2024-09-10T21:19:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9ab400640 1 -- 172.21.15.16:0/1906476402 learned_addr learned my addr 172.21.15.16:0/1906476402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9aa000640 1 --2- 172.21.15.16:0/1906476402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ac1521d0 0x7fe9ac152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9ab400640 1 -- 172.21.15.16:0/1906476402 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe990004d50 msgr2=0x7fe9ac071980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9ab400640 1 --2- 172.21.15.16:0/1906476402 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe990004d50 0x7fe9ac071980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9ab400640 1 -- 172.21.15.16:0/1906476402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ac1521d0 msgr2=0x7fe9ac152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9ab400640 1 --2- 172.21.15.16:0/1906476402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ac1521d0 0x7fe9ac152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9ab400640 1 -- 172.21.15.16:0/1906476402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac06dd90 con 0x7fe9ac0cec20 2024-09-10T21:19:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9ab400640 1 --2- 172.21.15.16:0/1906476402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ac0cec20 0x7fe9ac0cf0d0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009ee0 tx=0x7fe9a4004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9a2a00640 1 -- 172.21.15.16:0/1906476402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40ad050 con 0x7fe9ac0cec20 2024-09-10T21:19:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9a2a00640 1 -- 172.21.15.16:0/1906476402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9a40a7a90 con 0x7fe9ac0cec20 2024-09-10T21:19:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9a2a00640 1 -- 172.21.15.16:0/1906476402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4009200 con 0x7fe9ac0cec20 2024-09-10T21:19:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9b1a00640 1 -- 172.21.15.16:0/1906476402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac06e070 con 0x7fe9ac0cec20 2024-09-10T21:19:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9b1a00640 1 -- 172.21.15.16:0/1906476402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac06e630 con 0x7fe9ac0cec20 2024-09-10T21:19:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.908+0000 7fe9b1a00640 1 -- 172.21.15.16:0/1906476402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ac0afae0 con 0x7fe9ac0cec20 2024-09-10T21:19:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.912+0000 7fe9a2a00640 1 -- 172.21.15.16:0/1906476402 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe9a4003980 con 0x7fe9ac0cec20 2024-09-10T21:19:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.912+0000 7fe9a2a00640 1 --2- 172.21.15.16:0/1906476402 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe98c0725e0 0x7fe98c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.912+0000 7fe9a2a00640 1 -- 172.21.15.16:0/1906476402 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fe9a412e1a0 con 0x7fe9ac0cec20 2024-09-10T21:19:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.912+0000 7fe9aaa00640 1 --2- 172.21.15.16:0/1906476402 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe98c0725e0 0x7fe98c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:03.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.912+0000 7fe9aaa00640 1 --2- 172.21.15.16:0/1906476402 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe98c0725e0 0x7fe98c074aa0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac072520 tx=0x7fe99c006040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:03.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:03.916+0000 7fe9a2a00640 1 -- 172.21.15.16:0/1906476402 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe9a4017300 con 0x7fe9ac0cec20 2024-09-10T21:19:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9b1a00640 1 -- 172.21.15.16:0/1906476402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9ac074ba0 con 0x7fe9ac0cec20 2024-09-10T21:19:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9a2a00640 1 -- 172.21.15.16:0/1906476402 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5232 (secure 0 0 0) 0x7fe9a40f9190 con 0x7fe9ac0cec20 2024-09-10T21:19:04.133 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:04.133 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":105,"btime":"2024-09-10T21:18:57:146722+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1121375070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1121375070},{"type":"v1","addr":"172.21.15.52:6835","nonce":1121375070}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":8247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3443454578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3443454578},{"type":"v1","addr":"172.21.15.16:6836","nonce":3443454578}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":8250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1848135983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1848135983},{"type":"v1","addr":"172.21.15.16:6837","nonce":1848135983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:53.527283+0000","modified":"2024-09-10T21:18:56.547852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"b","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1948696903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1948696903},{"type":"v1","addr":"172.21.15.52:6834","nonce":1948696903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8259,"qdb_cluster":[8259]},"id":11}]} 2024-09-10T21:19:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 105 2024-09-10T21:19:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9b1a00640 1 -- 172.21.15.16:0/1906476402 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe98c0725e0 msgr2=0x7fe98c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9b1a00640 1 --2- 172.21.15.16:0/1906476402 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe98c0725e0 0x7fe98c074aa0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac072520 tx=0x7fe99c006040 comp rx=0 tx=0).stop 2024-09-10T21:19:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9b1a00640 1 -- 172.21.15.16:0/1906476402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ac0cec20 msgr2=0x7fe9ac0cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9b1a00640 1 --2- 172.21.15.16:0/1906476402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ac0cec20 0x7fe9ac0cf0d0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009ee0 tx=0x7fe9a4004f50 comp rx=0 tx=0).stop 2024-09-10T21:19:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9b1a00640 1 -- 172.21.15.16:0/1906476402 shutdown_connections 2024-09-10T21:19:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9b1a00640 1 --2- 172.21.15.16:0/1906476402 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fe98c0725e0 0x7fe98c074aa0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9b1a00640 1 --2- 172.21.15.16:0/1906476402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9ac0cec20 0x7fe9ac0cf0d0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9b1a00640 1 --2- 172.21.15.16:0/1906476402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9ac1521d0 0x7fe9ac152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9b1a00640 1 --2- 172.21.15.16:0/1906476402 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fe990004d50 0x7fe9ac071980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9b1a00640 1 -- 172.21.15.16:0/1906476402 >> 172.21.15.16:0/1906476402 conn(0x7fe9ac0d1d80 msgr2=0x7fe9ac15a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9b1a00640 1 -- 172.21.15.16:0/1906476402 shutdown_connections 2024-09-10T21:19:04.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.128+0000 7fe9b1a00640 1 -- 172.21.15.16:0/1906476402 wait complete. 2024-09-10T21:19:04.557 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:18:53.527283+0000', 'modified': '2024-09-10T21:18:56.547852+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8259}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8259': {'gid': 8259, 'name': 'b', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.52:6834/1948696903', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6832', 'nonce': 1948696903}, {'type': 'v1', 'addr': '172.21.15.52:6834', 'nonce': 1948696903}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8259, 'qdb_cluster': [8259]} 2024-09-10T21:19:04.557 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:19:04.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.704+0000 7faa84600640 1 -- 172.21.15.16:0/823319195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c0cd8a0 msgr2=0x7faa7c0ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.704+0000 7faa84600640 1 --2- 172.21.15.16:0/823319195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c0cd8a0 0x7faa7c0ce2b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7faa74009c80 tx=0x7faa7401cda0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.704+0000 7faa84600640 1 -- 172.21.15.16:0/823319195 shutdown_connections 2024-09-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.704+0000 7faa84600640 1 --2- 172.21.15.16:0/823319195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c0cd8a0 0x7faa7c0ce2b0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.704+0000 7faa84600640 1 -- 172.21.15.16:0/823319195 >> 172.21.15.16:0/823319195 conn(0x7faa7c0d1d80 msgr2=0x7faa7c0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.704+0000 7faa84600640 1 -- 172.21.15.16:0/823319195 shutdown_connections 2024-09-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.704+0000 7faa84600640 1 -- 172.21.15.16:0/823319195 wait complete. 2024-09-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.704+0000 7faa84600640 1 Processor -- start 2024-09-10T21:19:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.704+0000 7faa84600640 1 -- start start 2024-09-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa84600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c0cd8a0 0x7faa7c1f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa84600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa7c0cffe0 0x7faa7c0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa84600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faa7c0d03c0 0x7faa7c1fb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa84600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c16a1e0 con 0x7faa7c0cd8a0 2024-09-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa84600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c16a060 con 0x7faa7c0cffe0 2024-09-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa84600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7faa7c16a360 con 0x7faa7c0d03c0 2024-09-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa81c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c0cd8a0 0x7faa7c1f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa81c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c0cd8a0 0x7faa7c1f7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38128/0 (socket says 172.21.15.16:38128) 2024-09-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa81c00640 1 -- 172.21.15.16:0/2930724327 learned_addr learned my addr 172.21.15.16:0/2930724327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa81200640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa7c0cffe0 0x7faa7c0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa82600640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faa7c0d03c0 0x7faa7c1fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa81c00640 1 -- 172.21.15.16:0/2930724327 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faa7c0d03c0 msgr2=0x7faa7c1fb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa81c00640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faa7c0d03c0 0x7faa7c1fb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa81c00640 1 -- 172.21.15.16:0/2930724327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa7c0cffe0 msgr2=0x7faa7c0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa81c00640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa7c0cffe0 0x7faa7c0ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa81c00640 1 -- 172.21.15.16:0/2930724327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7c157410 con 0x7faa7c0cd8a0 2024-09-10T21:19:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa81200640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa7c0cffe0 0x7faa7c0ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa81c00640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c0cd8a0 0x7faa7c1f7a70 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7faa7000c500 tx=0x7faa7000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa79600640 1 -- 172.21.15.16:0/2930724327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70017070 con 0x7faa7c0cd8a0 2024-09-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa79600640 1 -- 172.21.15.16:0/2930724327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa700053c0 con 0x7faa7c0cd8a0 2024-09-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa79600640 1 -- 172.21.15.16:0/2930724327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70012890 con 0x7faa7c0cd8a0 2024-09-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa84600640 1 -- 172.21.15.16:0/2930724327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7c1f8170 con 0x7faa7c0cd8a0 2024-09-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa84600640 1 -- 172.21.15.16:0/2930724327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa7c1f8730 con 0x7faa7c0cd8a0 2024-09-10T21:19:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.708+0000 7faa79600640 1 -- 172.21.15.16:0/2930724327 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faa70003600 con 0x7faa7c0cd8a0 2024-09-10T21:19:04.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa79600640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faa5c0725e0 0x7faa5c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa87ac7000 1 Processor -- start 2024-09-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa79600640 1 -- 172.21.15.16:0/2930724327 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7faa70092890 con 0x7faa7c0cd8a0 2024-09-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa81200640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faa5c0725e0 0x7faa5c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa87ac7000 1 -- start start 2024-09-10T21:19:04.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa81200640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faa5c0725e0 0x7faa5c074aa0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7faa7c1579d0 tx=0x7faa68009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:04.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa87ac7000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae9219d0 0x5579ae921db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa87ac7000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 0x5579ae8d58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa87ac7000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 0x5579ae8d61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa87ac7000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5579ae7f63f0 con 0x5579ae8d5e10 2024-09-10T21:19:04.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa87ac7000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5579ae733b20 con 0x5579ae9222f0 2024-09-10T21:19:04.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa87ac7000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x5579ae722f50 con 0x5579ae9219d0 2024-09-10T21:19:04.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa82600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 0x5579ae8d61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa82600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 0x5579ae8d61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38144/0 (socket says 172.21.15.16:38144) 2024-09-10T21:19:04.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa82600640 1 -- 172.21.15.16:0/2251869430 learned_addr learned my addr 172.21.15.16:0/2251869430 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa81c00640 1 --2- 172.21.15.16:0/2251869430 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae9219d0 0x5579ae921db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa81200640 1 --2- 172.21.15.16:0/2251869430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 0x5579ae8d58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa82600640 1 -- 172.21.15.16:0/2251869430 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae9219d0 msgr2=0x5579ae921db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa82600640 1 --2- 172.21.15.16:0/2251869430 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae9219d0 0x5579ae921db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa82600640 1 -- 172.21.15.16:0/2251869430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 msgr2=0x5579ae8d58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa82600640 1 --2- 172.21.15.16:0/2251869430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 0x5579ae8d58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa82600640 1 -- 172.21.15.16:0/2251869430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579ae6ea590 con 0x5579ae8d5e10 2024-09-10T21:19:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa81200640 1 --2- 172.21.15.16:0/2251869430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 0x5579ae8d58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa82600640 1 --2- 172.21.15.16:0/2251869430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 0x5579ae8d61f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7faa74009bd0 tx=0x7faa74019b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8d479e4c7e9752 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa6e000640 1 -- 172.21.15.16:0/2251869430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa740bb020 con 0x5579ae8d5e10 2024-09-10T21:19:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa6e000640 1 -- 172.21.15.16:0/2251869430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa74004d40 con 0x5579ae8d5e10 2024-09-10T21:19:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.712+0000 7faa6e000640 1 -- 172.21.15.16:0/2251869430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa740b2040 con 0x5579ae8d5e10 2024-09-10T21:19:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 -- 172.21.15.16:0/2251869430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 msgr2=0x5579ae8d61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 --2- 172.21.15.16:0/2251869430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 0x5579ae8d61f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7faa74009bd0 tx=0x7faa74019b40 comp rx=0 tx=0).stop 2024-09-10T21:19:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 -- 172.21.15.16:0/2251869430 shutdown_connections 2024-09-10T21:19:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 --2- 172.21.15.16:0/2251869430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 0x5579ae8d61f0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 --2- 172.21.15.16:0/2251869430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 0x5579ae8d58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 --2- 172.21.15.16:0/2251869430 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae9219d0 0x5579ae921db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 -- 172.21.15.16:0/2251869430 >> 172.21.15.16:0/2251869430 conn(0x5579ae7bf130 msgr2=0x5579ae904e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:04.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 -- 172.21.15.16:0/2251869430 shutdown_connections 2024-09-10T21:19:04.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 -- 172.21.15.16:0/2251869430 wait complete. 2024-09-10T21:19:04.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 -- start start 2024-09-10T21:19:04.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 Processor -- start 2024-09-10T21:19:04.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 0x5579ae9026a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae9219d0 0x5579ae902e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 0x5579ae9033b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5579ae6bd5d0 con 0x5579ae8d5e10 2024-09-10T21:19:04.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5579ae726d60 con 0x5579ae9222f0 2024-09-10T21:19:04.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x5579ae6c9500 con 0x5579ae9219d0 2024-09-10T21:19:04.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 10 client.0 _ll_register_callbacks cb 0x5579ae6cd810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:04.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa81c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 0x5579ae9026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa81c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 0x5579ae9026a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38152/0 (socket says 172.21.15.16:38152) 2024-09-10T21:19:04.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa81c00640 1 -- 172.21.15.16:0/1851594748 learned_addr learned my addr 172.21.15.16:0/1851594748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:04.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa82600640 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 0x5579ae9033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa81200640 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae9219d0 0x5579ae902e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa81c00640 1 -- 172.21.15.16:0/1851594748 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae9219d0 msgr2=0x5579ae902e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:04.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa81c00640 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae9219d0 0x5579ae902e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa81c00640 1 -- 172.21.15.16:0/1851594748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 msgr2=0x5579ae9033b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:04.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa81c00640 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 0x5579ae9033b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa81c00640 1 -- 172.21.15.16:0/1851594748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579ae6c1170 con 0x5579ae8d5e10 2024-09-10T21:19:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa81200640 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae9219d0 0x5579ae902e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa81c00640 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 0x5579ae9026a0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7faa7005d590 tx=0x7faa70000fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d3d6e702359ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa62a00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:19:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa62a00640 1 -- 172.21.15.16:0/1851594748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7005ccf0 con 0x5579ae8d5e10 2024-09-10T21:19:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa62a00640 1 -- 172.21.15.16:0/1851594748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7faa7005eb10 con 0x5579ae8d5e10 2024-09-10T21:19:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 -- 172.21.15.16:0/1851594748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5579ae6bb0f0 con 0x5579ae8d5e10 2024-09-10T21:19:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa62a00640 1 -- 172.21.15.16:0/1851594748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7faa70066ce0 con 0x5579ae8d5e10 2024-09-10T21:19:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 10 client.8403 fetch_fsmap learned FSMap version 105 2024-09-10T21:19:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa87ac7000 1 -- 172.21.15.16:0/1851594748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x5579ae7a7ef0 con 0x5579ae8d5e10 2024-09-10T21:19:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa62a00640 1 -- 172.21.15.16:0/1851594748 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5579ae6bb0f0 con 0x5579ae8d5e10 2024-09-10T21:19:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.716+0000 7faa62a00640 1 -- 172.21.15.16:0/1851594748 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 105) ==== 3502+0+0 (secure 0 0 0) 0x7faa700591e0 con 0x5579ae8d5e10 2024-09-10T21:19:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 10 client.8403 fetch_fsmap finished waiting for FSMap version 105 2024-09-10T21:19:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 10 client.8403 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:19:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 20 client.8403 populate_metadata read hostname 'smithi016' 2024-09-10T21:19:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.52:6832/1948696903,v1:172.21.15.52:6834/1948696903] conn(0x5579ae9040f0 0x5579ae933c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 4 client.8403 mds_command: new command op to 8259 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:19:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 1 -- 172.21.15.16:0/1851594748 --> [v2:172.21.15.52:6832/1948696903,v1:172.21.15.52:6834/1948696903] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579ae735a70 con 0x5579ae9040f0 2024-09-10T21:19:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa81200640 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.52:6832/1948696903,v1:172.21.15.52:6834/1948696903] conn(0x5579ae9040f0 0x5579ae933c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa81200640 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.52:6832/1948696903,v1:172.21.15.52:6834/1948696903] conn(0x5579ae9040f0 0x5579ae933c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71164de29a4f2737 server_cookie=efe98367d7c1fc4a in_seq=0 out_seq=0 2024-09-10T21:19:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa62a00640 10 client.8403 ms_handle_connect on v2:172.21.15.52:6832/1948696903 2024-09-10T21:19:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa62a00640 1 -- 172.21.15.16:0/1851594748 <== mds.0 v2:172.21.15.52:6832/1948696903 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5579ae735a70 con 0x5579ae9040f0 2024-09-10T21:19:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa62a00640 10 client.8403 handle_command_reply: tid=0 2024-09-10T21:19:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 1 client.8403 shutdown 2024-09-10T21:19:04.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 1 -- 172.21.15.16:0/1851594748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 msgr2=0x5579ae9026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 0x5579ae9026a0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7faa7005d590 tx=0x7faa70000fa0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 1 -- 172.21.15.16:0/1851594748 shutdown_connections 2024-09-10T21:19:04.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.52:6832/1948696903,v1:172.21.15.52:6834/1948696903] conn(0x5579ae9040f0 0x5579ae933c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 0x5579ae9033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae9219d0 0x5579ae902e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1851594748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae8d5e10 0x5579ae9026a0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 1 -- 172.21.15.16:0/1851594748 >> 172.21.15.16:0/1851594748 conn(0x5579ae7bf130 msgr2=0x5579ae8d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:04.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa62a00640 0 client.8403 ms_handle_reset on v2:172.21.15.52:6832/1948696903 2024-09-10T21:19:04.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 1 -- 172.21.15.16:0/1851594748 shutdown_connections 2024-09-10T21:19:04.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 1 -- 172.21.15.16:0/1851594748 wait complete. 2024-09-10T21:19:04.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.720+0000 7faa87ac7000 20 client.8403 trim_cache size 0 max 16384 2024-09-10T21:19:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa87ac7000 1 Processor -- start 2024-09-10T21:19:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa87ac7000 1 -- start start 2024-09-10T21:19:04.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa87ac7000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae8d5e10 0x5579ae993dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa87ac7000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9219d0 0x5579ae994310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa87ac7000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9222f0 0x5579ae994850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa87ac7000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c2042d0 con 0x5579ae9222f0 2024-09-10T21:19:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa87ac7000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c2033d0 con 0x5579ae9219d0 2024-09-10T21:19:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa87ac7000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7faa7c070ca0 con 0x5579ae8d5e10 2024-09-10T21:19:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa82600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9222f0 0x5579ae994850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa82600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9222f0 0x5579ae994850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38154/0 (socket says 172.21.15.16:38154) 2024-09-10T21:19:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa82600640 1 -- 172.21.15.16:0/1772775269 learned_addr learned my addr 172.21.15.16:0/1772775269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa82600640 1 -- 172.21.15.16:0/1772775269 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae8d5e10 msgr2=0x5579ae993dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa82600640 1 --2- 172.21.15.16:0/1772775269 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae8d5e10 0x5579ae993dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa82600640 1 -- 172.21.15.16:0/1772775269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9219d0 msgr2=0x5579ae994310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa82600640 1 --2- 172.21.15.16:0/1772775269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9219d0 0x5579ae994310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa82600640 1 -- 172.21.15.16:0/1772775269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579ae77ed90 con 0x5579ae9222f0 2024-09-10T21:19:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa82600640 1 --2- 172.21.15.16:0/1772775269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9222f0 0x5579ae994850 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7faa74000f90 tx=0x7faa740095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0431c25cafbad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa6d600640 1 -- 172.21.15.16:0/1772775269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa740bb070 con 0x5579ae9222f0 2024-09-10T21:19:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa6d600640 1 -- 172.21.15.16:0/1772775269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa740c5070 con 0x5579ae9222f0 2024-09-10T21:19:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa6d600640 1 -- 172.21.15.16:0/1772775269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa740b2040 con 0x5579ae9222f0 2024-09-10T21:19:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa87ac7000 1 -- 172.21.15.16:0/1772775269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9222f0 msgr2=0x5579ae994850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.740+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1772775269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9222f0 0x5579ae994850 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7faa74000f90 tx=0x7faa740095c0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 -- 172.21.15.16:0/1772775269 shutdown_connections 2024-09-10T21:19:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1772775269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9222f0 0x5579ae994850 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1772775269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9219d0 0x5579ae994310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1772775269 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae8d5e10 0x5579ae993dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 -- 172.21.15.16:0/1772775269 >> 172.21.15.16:0/1772775269 conn(0x5579ae7bf130 msgr2=0x5579ae8ddad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 -- 172.21.15.16:0/1772775269 shutdown_connections 2024-09-10T21:19:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 -- 172.21.15.16:0/1772775269 wait complete. 2024-09-10T21:19:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 -- start start 2024-09-10T21:19:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 Processor -- start 2024-09-10T21:19:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae8d5e10 0x5579ae9977e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9219d0 0x5579ae997d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 0x5579ae998260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5579ae726070 con 0x5579ae9219d0 2024-09-10T21:19:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5579ae7a7a60 con 0x5579ae9222f0 2024-09-10T21:19:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x5579ae7e2730 con 0x5579ae8d5e10 2024-09-10T21:19:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 10 client.0 _ll_register_callbacks cb 0x7faa7005d0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa81200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9219d0 0x5579ae997d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa81200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9219d0 0x5579ae997d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38168/0 (socket says 172.21.15.16:38168) 2024-09-10T21:19:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa81200640 1 -- 172.21.15.16:0/1907769700 learned_addr learned my addr 172.21.15.16:0/1907769700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa81c00640 1 --2- 172.21.15.16:0/1907769700 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae8d5e10 0x5579ae9977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa82600640 1 --2- 172.21.15.16:0/1907769700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 0x5579ae998260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa81200640 1 -- 172.21.15.16:0/1907769700 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae8d5e10 msgr2=0x5579ae9977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa81200640 1 --2- 172.21.15.16:0/1907769700 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae8d5e10 0x5579ae9977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa81200640 1 -- 172.21.15.16:0/1907769700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 msgr2=0x5579ae998260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa81200640 1 --2- 172.21.15.16:0/1907769700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 0x5579ae998260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa81200640 1 -- 172.21.15.16:0/1907769700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579ae6e9a90 con 0x5579ae9219d0 2024-09-10T21:19:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa81200640 1 --2- 172.21.15.16:0/1907769700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9219d0 0x5579ae997d20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7faa68000f70 tx=0x7faa68001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57cad40a2b5d3ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa62a00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-10T21:19:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa62a00640 1 -- 172.21.15.16:0/1907769700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68003ad0 con 0x5579ae9219d0 2024-09-10T21:19:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa62a00640 1 -- 172.21.15.16:0/1907769700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7faa6801e030 con 0x5579ae9219d0 2024-09-10T21:19:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 -- 172.21.15.16:0/1907769700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa740c5070 con 0x5579ae9219d0 2024-09-10T21:19:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa62a00640 1 -- 172.21.15.16:0/1907769700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7faa68003c70 con 0x5579ae9219d0 2024-09-10T21:19:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 10 client.8409 fetch_fsmap learned FSMap version 105 2024-09-10T21:19:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 -- 172.21.15.16:0/1907769700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x5579ae6c8050 con 0x5579ae9219d0 2024-09-10T21:19:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa62a00640 1 -- 172.21.15.16:0/1907769700 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa740c5070 con 0x5579ae9219d0 2024-09-10T21:19:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa62a00640 1 -- 172.21.15.16:0/1907769700 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 105) ==== 3502+0+0 (secure 0 0 0) 0x7faa68029070 con 0x5579ae9219d0 2024-09-10T21:19:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 10 client.8409 fetch_fsmap finished waiting for FSMap version 105 2024-09-10T21:19:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 10 client.8409 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:19:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 20 client.8409 populate_metadata read hostname 'smithi016' 2024-09-10T21:19:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1907769700 >> [v2:172.21.15.52:6832/1948696903,v1:172.21.15.52:6834/1948696903] conn(0x5579ae9375b0 0x5579ae9366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 4 client.8409 mds_command: new command op to 8259 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:19:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.744+0000 7faa87ac7000 1 -- 172.21.15.16:0/1907769700 --> [v2:172.21.15.52:6832/1948696903,v1:172.21.15.52:6834/1948696903] -- command(tid 0: {"prefix": "status"}) -- 0x5579ae731520 con 0x5579ae9375b0 2024-09-10T21:19:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa81c00640 1 --2- 172.21.15.16:0/1907769700 >> [v2:172.21.15.52:6832/1948696903,v1:172.21.15.52:6834/1948696903] conn(0x5579ae9375b0 0x5579ae9366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa81c00640 1 --2- 172.21.15.16:0/1907769700 >> [v2:172.21.15.52:6832/1948696903,v1:172.21.15.52:6834/1948696903] conn(0x5579ae9375b0 0x5579ae9366d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36d248e67417c1a9 server_cookie=79cbe03408ab3035 in_seq=0 out_seq=0 2024-09-10T21:19:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa62a00640 10 client.8409 ms_handle_connect on v2:172.21.15.52:6832/1948696903 2024-09-10T21:19:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa62a00640 1 -- 172.21.15.16:0/1907769700 <== mds.0 v2:172.21.15.52:6832/1948696903 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5579ae731520 con 0x5579ae9375b0 2024-09-10T21:19:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa62a00640 10 client.8409 handle_command_reply: tid=0 2024-09-10T21:19:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa87ac7000 1 client.8409 shutdown 2024-09-10T21:19:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa87ac7000 1 -- 172.21.15.16:0/1907769700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9219d0 msgr2=0x5579ae997d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1907769700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9219d0 0x5579ae997d20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7faa68000f70 tx=0x7faa68001090 comp rx=0 tx=0).stop 2024-09-10T21:19:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa87ac7000 1 -- 172.21.15.16:0/1907769700 shutdown_connections 2024-09-10T21:19:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1907769700 >> [v2:172.21.15.52:6832/1948696903,v1:172.21.15.52:6834/1948696903] conn(0x5579ae9375b0 0x5579ae9366d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1907769700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5579ae9222f0 0x5579ae998260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1907769700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579ae9219d0 0x5579ae997d20 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa87ac7000 1 --2- 172.21.15.16:0/1907769700 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x5579ae8d5e10 0x5579ae9977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa62a00640 0 client.8409 ms_handle_reset on v2:172.21.15.52:6832/1948696903 2024-09-10T21:19:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa87ac7000 1 -- 172.21.15.16:0/1907769700 >> 172.21.15.16:0/1907769700 conn(0x5579ae7bf130 msgr2=0x5579ae9205b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa87ac7000 1 -- 172.21.15.16:0/1907769700 shutdown_connections 2024-09-10T21:19:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa87ac7000 1 -- 172.21.15.16:0/1907769700 wait complete. 2024-09-10T21:19:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa87ac7000 20 client.8409 trim_cache size 0 max 16384 2024-09-10T21:19:04.758 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:19:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "eb19be60-6d8f-4816-b778-3d7d6698304d", 2024-09-10T21:19:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-10T21:19:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8259, 2024-09-10T21:19:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-10T21:19:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-10T21:19:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-10T21:19:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 10.638026712, 2024-09-10T21:19:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 105, 2024-09-10T21:19:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 138, 2024-09-10T21:19:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 138, 2024-09-10T21:19:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 15.081479043 2024-09-10T21:19:04.760 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:19:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa84600640 1 -- 172.21.15.16:0/2930724327 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faa5c0725e0 msgr2=0x7faa5c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa84600640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faa5c0725e0 0x7faa5c074aa0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7faa7c1579d0 tx=0x7faa68009040 comp rx=0 tx=0).stop 2024-09-10T21:19:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa84600640 1 -- 172.21.15.16:0/2930724327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c0cd8a0 msgr2=0x7faa7c1f7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa84600640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c0cd8a0 0x7faa7c1f7a70 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7faa7000c500 tx=0x7faa7000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa84600640 1 -- 172.21.15.16:0/2930724327 shutdown_connections 2024-09-10T21:19:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa84600640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7faa5c0725e0 0x7faa5c074aa0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa84600640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faa7c0d03c0 0x7faa7c1fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa84600640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa7c0cffe0 0x7faa7c0ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa84600640 1 --2- 172.21.15.16:0/2930724327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c0cd8a0 0x7faa7c1f7a70 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa84600640 1 -- 172.21.15.16:0/2930724327 >> 172.21.15.16:0/2930724327 conn(0x7faa7c0d1d80 msgr2=0x7faa7c15e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa84600640 1 -- 172.21.15.16:0/2930724327 shutdown_connections 2024-09-10T21:19:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.748+0000 7faa84600640 1 -- 172.21.15.16:0/2930724327 wait complete. 2024-09-10T21:19:04.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:19:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.912+0000 7fdcea000640 1 -- 172.21.15.16:0/852292202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdce40b0590 msgr2=0x7fdce4071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.912+0000 7fdcea000640 1 --2- 172.21.15.16:0/852292202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdce40b0590 0x7fdce4071520 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009bc0 tx=0x7fdcdc01ca80 comp rx=0 tx=0).stop 2024-09-10T21:19:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.912+0000 7fdcea000640 1 -- 172.21.15.16:0/852292202 shutdown_connections 2024-09-10T21:19:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.912+0000 7fdcea000640 1 --2- 172.21.15.16:0/852292202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdce40b0590 0x7fdce4071520 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.912+0000 7fdcea000640 1 --2- 172.21.15.16:0/852292202 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdce415f400 0x7fdce4070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.912+0000 7fdcea000640 1 -- 172.21.15.16:0/852292202 >> 172.21.15.16:0/852292202 conn(0x7fdce409b130 msgr2=0x7fdce40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.912+0000 7fdcea000640 1 -- 172.21.15.16:0/852292202 shutdown_connections 2024-09-10T21:19:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.912+0000 7fdcea000640 1 -- 172.21.15.16:0/852292202 wait complete. 2024-09-10T21:19:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcea000640 1 Processor -- start 2024-09-10T21:19:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcea000640 1 -- start start 2024-09-10T21:19:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcea000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdce40b0590 0x7fdce414fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcea000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdce40b13c0 0x7fdce406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcea000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce415f400 0x7fdce4152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcea000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4168f60 con 0x7fdce415f400 2024-09-10T21:19:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcea000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4168de0 con 0x7fdce40b0590 2024-09-10T21:19:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcea000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fdce41690e0 con 0x7fdce40b13c0 2024-09-10T21:19:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdce3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce415f400 0x7fdce4152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcdaa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdce40b13c0 0x7fdce406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdce3400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdce40b0590 0x7fdce414fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcdaa00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdce40b13c0 0x7fdce406edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3301/0 says I am v2:172.21.15.16:38918/0 (socket says 172.21.15.16:38918) 2024-09-10T21:19:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcdaa00640 1 -- 172.21.15.16:0/2450104941 learned_addr learned my addr 172.21.15.16:0/2450104941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdce3e00640 1 -- 172.21.15.16:0/2450104941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdce40b13c0 msgr2=0x7fdce406edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdce3e00640 1 --2- 172.21.15.16:0/2450104941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdce40b13c0 0x7fdce406edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdce3e00640 1 -- 172.21.15.16:0/2450104941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdce40b0590 msgr2=0x7fdce414fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdce3e00640 1 --2- 172.21.15.16:0/2450104941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdce40b0590 0x7fdce414fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdce3e00640 1 -- 172.21.15.16:0/2450104941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce41fb1f0 con 0x7fdce415f400 2024-09-10T21:19:04.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdce3e00640 1 --2- 172.21.15.16:0/2450104941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce415f400 0x7fdce4152f20 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0ac000 tx=0x7fdcdc0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:04.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcdbe00640 1 -- 172.21.15.16:0/2450104941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc0bb070 con 0x7fdce415f400 2024-09-10T21:19:04.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcea000640 1 -- 172.21.15.16:0/2450104941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce41fb490 con 0x7fdce415f400 2024-09-10T21:19:04.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcdbe00640 1 -- 172.21.15.16:0/2450104941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcdc004e00 con 0x7fdce415f400 2024-09-10T21:19:04.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcdbe00640 1 -- 172.21.15.16:0/2450104941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc0b2040 con 0x7fdce415f400 2024-09-10T21:19:04.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.916+0000 7fdcea000640 1 -- 172.21.15.16:0/2450104941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce41fba50 con 0x7fdce415f400 2024-09-10T21:19:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.920+0000 7fdcea000640 1 -- 172.21.15.16:0/2450104941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce40cfc80 con 0x7fdce415f400 2024-09-10T21:19:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.920+0000 7fdcdbe00640 1 -- 172.21.15.16:0/2450104941 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdcdc0b5030 con 0x7fdce415f400 2024-09-10T21:19:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.920+0000 7fdcdbe00640 1 --2- 172.21.15.16:0/2450104941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdcc0072610 0x7fdcc0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.920+0000 7fdcdbe00640 1 -- 172.21.15.16:0/2450104941 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fdcdc12cc00 con 0x7fdce415f400 2024-09-10T21:19:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.920+0000 7fdce3400640 1 --2- 172.21.15.16:0/2450104941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdcc0072610 0x7fdcc0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.920+0000 7fdce3400640 1 --2- 172.21.15.16:0/2450104941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdcc0072610 0x7fdcc0074ad0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009c10 tx=0x7fdcd40091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:04.924+0000 7fdcdbe00640 1 -- 172.21.15.16:0/2450104941 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdcdc0f1de0 con 0x7fdce415f400 2024-09-10T21:19:05.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.136+0000 7fdcea000640 1 -- 172.21.15.16:0/2450104941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdce40ce050 con 0x7fdce415f400 2024-09-10T21:19:05.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.136+0000 7fdcdbe00640 1 -- 172.21.15.16:0/2450104941 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) ==== 53+0+73 (secure 0 0 0) 0x7fdcdc016790 con 0x7fdce415f400 2024-09-10T21:19:05.141 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:19:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.136+0000 7fdcea000640 1 -- 172.21.15.16:0/2450104941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdcc0072610 msgr2=0x7fdcc0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.136+0000 7fdcea000640 1 --2- 172.21.15.16:0/2450104941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdcc0072610 0x7fdcc0074ad0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009c10 tx=0x7fdcd40091e0 comp rx=0 tx=0).stop 2024-09-10T21:19:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.136+0000 7fdcea000640 1 -- 172.21.15.16:0/2450104941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce415f400 msgr2=0x7fdce4152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.136+0000 7fdcea000640 1 --2- 172.21.15.16:0/2450104941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce415f400 0x7fdce4152f20 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0ac000 tx=0x7fdcdc0092d0 comp rx=0 tx=0).stop 2024-09-10T21:19:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.136+0000 7fdcea000640 1 -- 172.21.15.16:0/2450104941 shutdown_connections 2024-09-10T21:19:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.136+0000 7fdcea000640 1 --2- 172.21.15.16:0/2450104941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fdcc0072610 0x7fdcc0074ad0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.136+0000 7fdcea000640 1 --2- 172.21.15.16:0/2450104941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce415f400 0x7fdce4152f20 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.136+0000 7fdcea000640 1 --2- 172.21.15.16:0/2450104941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fdce40b13c0 0x7fdce406edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.136+0000 7fdcea000640 1 --2- 172.21.15.16:0/2450104941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdce40b0590 0x7fdce414fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.140+0000 7fdcea000640 1 -- 172.21.15.16:0/2450104941 >> 172.21.15.16:0/2450104941 conn(0x7fdce409b130 msgr2=0x7fdce40ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.140+0000 7fdcea000640 1 -- 172.21.15.16:0/2450104941 shutdown_connections 2024-09-10T21:19:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05.140+0000 7fdcea000640 1 -- 172.21.15.16:0/2450104941 wait complete. 2024-09-10T21:19:05.163 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:19:05.163 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:19:05.163 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-10T21:19:05.163 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:19:05.163 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:19:05.163 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:19:05.163 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:19:05.163 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:19:05.163 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:19:05.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:05.164 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-10T21:19:05.173 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:19:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:19:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:19:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-10T21:19:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:05.174 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:19:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:19:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-10T21:19:05.174 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:19:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:19:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-10T21:19:05.174 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:19:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:19:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-10T21:19:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:19:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41904sec preferred_lft 41904sec 2024-09-10T21:19:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-10T21:19:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:05.175 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:19:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:19:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-10T21:19:05.175 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:19:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:30:fb:74:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:19:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:19:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:05.176 INFO:teuthology.orchestra.run.smithi016.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:19:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:19:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:19:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:19:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:05.176 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:19:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:19:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:19:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:05.177 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:19:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:19:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:19:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:05.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:05.177 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:19:05.233 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-10T21:19:05.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:05.237 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-10T21:19:05.288 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-10T21:19:05.288 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:19:05.288 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:19:05.288 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:19:05.288 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:19:05.288 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:19:05.289 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:19:05.289 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-10T21:19:05.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:19:05.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:19:05.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:19:05.289 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-10T21:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:19:05.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:19:05.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:05.427 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:19:05.436 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:19:05.436 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:19:05.440 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:19:05.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:05.440 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-10T21:19:05.503 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:19:05.503 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:19:05.559 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:19:05.559 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:19:05.559 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:19:05.611 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:19:05.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:05.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:19:05.704 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:19:05.704 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:19:05.717 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:19:05.717 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:19:05.718 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:19:05.718 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:19:05.718 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:19:05.721 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:19:05.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:05.722 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:19:05.742 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:19:05.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:19:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:19:05.852 INFO:teuthology.orchestra.run.smithi016.stdout:client8415 2024-09-10T21:19:05.853 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:19:05.853 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:19:05.862 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:19:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.048+0000 7fb851a00640 1 -- 172.21.15.16:0/913552209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c0cb610 msgr2=0x7fb84c0cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.048+0000 7fb851a00640 1 --2- 172.21.15.16:0/913552209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c0cb610 0x7fb84c0cf530 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0d0b40 tx=0x7fb84401cd50 comp rx=0 tx=0).stop 2024-09-10T21:19:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.048+0000 7fb851a00640 1 -- 172.21.15.16:0/913552209 shutdown_connections 2024-09-10T21:19:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.048+0000 7fb851a00640 1 --2- 172.21.15.16:0/913552209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c0cb610 0x7fb84c0cf530 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.048+0000 7fb851a00640 1 --2- 172.21.15.16:0/913552209 >> v2:172.21.15.52:3300/0 conn(0x7fb84c0cc1e0 0x7fb84c0741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:06.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.048+0000 7fb851a00640 1 -- 172.21.15.16:0/913552209 >> 172.21.15.16:0/913552209 conn(0x7fb84c0d0c30 msgr2=0x7fb84c0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:06.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.048+0000 7fb851a00640 1 -- 172.21.15.16:0/913552209 shutdown_connections 2024-09-10T21:19:06.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.048+0000 7fb851a00640 1 -- 172.21.15.16:0/913552209 wait complete. 2024-09-10T21:19:06.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.048+0000 7fb851a00640 1 Processor -- start 2024-09-10T21:19:06.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.048+0000 7fb851a00640 1 -- start start 2024-09-10T21:19:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb851a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c0cb610 0x7fb84c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb851a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb84c0cc1e0 0x7fb84c1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb851a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb84c1f7f70 0x7fb84c0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb851a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c167cf0 con 0x7fb84c0cb610 2024-09-10T21:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb851a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c167b70 con 0x7fb84c0cc1e0 2024-09-10T21:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb851a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb84c167e70 con 0x7fb84c1f7f70 2024-09-10T21:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb84aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c0cb610 0x7fb84c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb84aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c0cb610 0x7fb84c156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38188/0 (socket says 172.21.15.16:38188) 2024-09-10T21:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb84aa00640 1 -- 172.21.15.16:0/3580662940 learned_addr learned my addr 172.21.15.16:0/3580662940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb842000640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb84c0cc1e0 0x7fb84c1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb84aa00640 1 -- 172.21.15.16:0/3580662940 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb84c1f7f70 msgr2=0x7fb84c0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb84b400640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb84c1f7f70 0x7fb84c0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb84aa00640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb84c1f7f70 0x7fb84c0b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb84aa00640 1 -- 172.21.15.16:0/3580662940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb84c0cc1e0 msgr2=0x7fb84c1594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb84aa00640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb84c0cc1e0 0x7fb84c1594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb84aa00640 1 -- 172.21.15.16:0/3580662940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84c156f00 con 0x7fb84c0cb610 2024-09-10T21:19:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb84b400640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb84c1f7f70 0x7fb84c0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:19:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb84aa00640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c0cb610 0x7fb84c156650 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00c460 tx=0x7fb83c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb843400640 1 -- 172.21.15.16:0/3580662940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0052d0 con 0x7fb84c0cb610 2024-09-10T21:19:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb843400640 1 -- 172.21.15.16:0/3580662940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb83c005490 con 0x7fb84c0cb610 2024-09-10T21:19:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb843400640 1 -- 172.21.15.16:0/3580662940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c012890 con 0x7fb84c0cb610 2024-09-10T21:19:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb851a00640 1 -- 172.21.15.16:0/3580662940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84c0b1c90 con 0x7fb84c0cb610 2024-09-10T21:19:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb851a00640 1 -- 172.21.15.16:0/3580662940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84c0b2250 con 0x7fb84c0cb610 2024-09-10T21:19:06.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb851a00640 1 -- 172.21.15.16:0/3580662940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb84c0b2450 con 0x7fb84c0cb610 2024-09-10T21:19:06.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.052+0000 7fb843400640 1 -- 172.21.15.16:0/3580662940 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb83c012a30 con 0x7fb84c0cb610 2024-09-10T21:19:06.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.056+0000 7fb843400640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb828072610 0x7fb828074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:06.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.056+0000 7fb843400640 1 -- 172.21.15.16:0/3580662940 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fb83c092d80 con 0x7fb84c0cb610 2024-09-10T21:19:06.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.056+0000 7fb842000640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb828072610 0x7fb828074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:06.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.056+0000 7fb842000640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb828072610 0x7fb828074ad0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0c7b80 tx=0x7fb8340091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:06.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.060+0000 7fb843400640 1 -- 172.21.15.16:0/3580662940 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb84c156f00 con 0x7fb84c0cb610 2024-09-10T21:19:06.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.268+0000 7fb851a00640 1 -- 172.21.15.16:0/3580662940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb84c0cf530 con 0x7fb84c0cb610 2024-09-10T21:19:06.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.268+0000 7fb843400640 1 -- 172.21.15.16:0/3580662940 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) ==== 53+0+73 (secure 0 0 0) 0x7fb83c05e480 con 0x7fb84c0cb610 2024-09-10T21:19:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:19:06.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.272+0000 7fb851a00640 1 -- 172.21.15.16:0/3580662940 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb828072610 msgr2=0x7fb828074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:06.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.272+0000 7fb851a00640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb828072610 0x7fb828074ad0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0c7b80 tx=0x7fb8340091e0 comp rx=0 tx=0).stop 2024-09-10T21:19:06.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.272+0000 7fb851a00640 1 -- 172.21.15.16:0/3580662940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c0cb610 msgr2=0x7fb84c156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:06.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.272+0000 7fb851a00640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c0cb610 0x7fb84c156650 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00c460 tx=0x7fb83c00c920 comp rx=0 tx=0).stop 2024-09-10T21:19:06.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.272+0000 7fb851a00640 1 -- 172.21.15.16:0/3580662940 shutdown_connections 2024-09-10T21:19:06.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.272+0000 7fb851a00640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb828072610 0x7fb828074ad0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:06.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.272+0000 7fb851a00640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb84c1f7f70 0x7fb84c0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:06.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.272+0000 7fb851a00640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb84c0cc1e0 0x7fb84c1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:06.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.272+0000 7fb851a00640 1 --2- 172.21.15.16:0/3580662940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c0cb610 0x7fb84c156650 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:06.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.272+0000 7fb851a00640 1 -- 172.21.15.16:0/3580662940 >> 172.21.15.16:0/3580662940 conn(0x7fb84c0d0c30 msgr2=0x7fb84c071760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:06.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.272+0000 7fb851a00640 1 -- 172.21.15.16:0/3580662940 shutdown_connections 2024-09-10T21:19:06.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:06.272+0000 7fb851a00640 1 -- 172.21.15.16:0/3580662940 wait complete. 2024-09-10T21:19:06.295 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:19:06.295 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:19:06.295 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-09-10T21:19:06.295 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T21:19:06.295 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:19:06.295 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:19:06.295 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:19:06.295 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:19:06.295 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:19:06.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:06.295 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-09-10T21:19:06.306 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:19:06.306 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:19:06.306 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:19:06.306 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:06.306 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-09-10T21:19:06.306 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:06.306 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:19:06.306 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:19:06.306 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-09-10T21:19:06.307 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:19:06.307 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:19:06.307 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-09-10T21:19:06.307 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:19:06.307 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-09-10T21:19:06.307 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-09-10T21:19:06.307 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:19:06.307 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 41904sec preferred_lft 41904sec 2024-09-10T21:19:06.307 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2024-09-10T21:19:06.307 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:06.307 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:19:06.307 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-09-10T21:19:06.307 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-09-10T21:19:06.308 INFO:teuthology.orchestra.run.smithi052.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:19:06.308 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 02:42:7e:b4:71:a2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:19:06.308 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:19:06.308 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:06.308 INFO:teuthology.orchestra.run.smithi052.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:19:06.308 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:19:06.308 INFO:teuthology.orchestra.run.smithi052.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:19:06.308 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:06.308 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:19:06.308 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:06.308 INFO:teuthology.orchestra.run.smithi052.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:19:06.308 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:19:06.308 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:19:06.309 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:06.309 INFO:teuthology.orchestra.run.smithi052.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:19:06.309 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:19:06.309 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:19:06.309 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:19:06.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:06.310 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:19:06.371 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-09-10T21:19:06.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:06.374 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-09-10T21:19:06.425 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-09-10T21:19:06.425 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:19:06.426 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:19:06.426 INFO:teuthology.orchestra.run.smithi052.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:19:06.426 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:19:06.426 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:19:06.426 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:19:06.426 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-09-10T21:19:06.426 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:19:06.426 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:19:06.426 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:19:06.426 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-09-10T21:19:06.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:19:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:19:06.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:19:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:19:06.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:06.573 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:19:06.583 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:19:06.584 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:19:06.584 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:19:06.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:06.584 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-09-10T21:19:06.651 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:19:06.652 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T21:19:06.707 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:19:06.708 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:19:06.708 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:19:06.759 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T21:19:06.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:06.759 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:19:06.857 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:19:06.857 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:19:06.872 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:19:06.872 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:19:06.873 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.52:3300,172.21.15.52:3301" 2024-09-10T21:19:06.873 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 1@eb19be60-6d8f-4816-b778-3d7d6698304d.cephfs=/ 2024-09-10T21:19:06.873 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.52:3300/172.21.15.52:3301" will pass to kernel 2024-09-10T21:19:06.875 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:19:06.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:06.875 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:19:06.895 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:19:06.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:19:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:19:06.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-09-10T21:19:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:19:06.997 INFO:teuthology.orchestra.run.smithi052.stdout:client7786 2024-09-10T21:19:06.997 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:19:06.997 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:19:07.006 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:19:07.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:19:07.074 DEBUG:teuthology.orchestra.run.smithi052:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:19:07.178 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:19:07.178 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:19:07.178 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:19:07.178 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:19:07.188 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:19:07.188 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:19:07.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:19:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.436+0000 7fa2f9a00640 1 -- 172.21.15.16:0/902558762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2f40cf510 msgr2=0x7fa2f415e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.436+0000 7fa2f9a00640 1 --2- 172.21.15.16:0/902558762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2f40cf510 0x7fa2f415e960 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009900 tx=0x7fa2dc01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.436+0000 7fa2f9a00640 1 -- 172.21.15.16:0/902558762 shutdown_connections 2024-09-10T21:19:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.436+0000 7fa2f9a00640 1 --2- 172.21.15.16:0/902558762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2f40cf510 0x7fa2f415e960 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.436+0000 7fa2f9a00640 1 --2- 172.21.15.16:0/902558762 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa2f415e580 0x7fa2f4074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.436+0000 7fa2f9a00640 1 -- 172.21.15.16:0/902558762 >> 172.21.15.16:0/902558762 conn(0x7fa2f40d0a80 msgr2=0x7fa2f40d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.436+0000 7fa2f9a00640 1 -- 172.21.15.16:0/902558762 shutdown_connections 2024-09-10T21:19:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.436+0000 7fa2f9a00640 1 -- 172.21.15.16:0/902558762 wait complete. 2024-09-10T21:19:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f9a00640 1 Processor -- start 2024-09-10T21:19:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f9a00640 1 -- start start 2024-09-10T21:19:07.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f40cf510 0x7fa2f40c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f9a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2f415e580 0x7fa2f41f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f9a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa2f415eea0 0x7fa2f4155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f9a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f40cbf30 con 0x7fa2f40cf510 2024-09-10T21:19:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f9a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f415be10 con 0x7fa2f415e580 2024-09-10T21:19:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f9a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f40cc0b0 con 0x7fa2f415eea0 2024-09-10T21:19:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f40cf510 0x7fa2f40c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f40cf510 0x7fa2f40c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38204/0 (socket says 172.21.15.16:38204) 2024-09-10T21:19:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f3e00640 1 -- 172.21.15.16:0/2921678167 learned_addr learned my addr 172.21.15.16:0/2921678167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2eb400640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2f415e580 0x7fa2f41f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f9000640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa2f415eea0 0x7fa2f4155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f3e00640 1 -- 172.21.15.16:0/2921678167 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa2f415eea0 msgr2=0x7fa2f4155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f3e00640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa2f415eea0 0x7fa2f4155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f3e00640 1 -- 172.21.15.16:0/2921678167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2f415e580 msgr2=0x7fa2f41f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f3e00640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2f415e580 0x7fa2f41f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f3e00640 1 -- 172.21.15.16:0/2921678167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f40c9ec0 con 0x7fa2f40cf510 2024-09-10T21:19:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2eb400640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2f415e580 0x7fa2f41f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:19:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f3e00640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f40cf510 0x7fa2f40c9830 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400ec00 tx=0x7fa2e400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f0c00640 1 -- 172.21.15.16:0/2921678167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4010630 con 0x7fa2f40cf510 2024-09-10T21:19:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f0c00640 1 -- 172.21.15.16:0/2921678167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2e40108c0 con 0x7fa2f40cf510 2024-09-10T21:19:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f9a00640 1 -- 172.21.15.16:0/2921678167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f40ca1a0 con 0x7fa2f40cf510 2024-09-10T21:19:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f0c00640 1 -- 172.21.15.16:0/2921678167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4010cf0 con 0x7fa2f40cf510 2024-09-10T21:19:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.440+0000 7fa2f9a00640 1 -- 172.21.15.16:0/2921678167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f41f4f00 con 0x7fa2f40cf510 2024-09-10T21:19:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.444+0000 7fa2f9a00640 1 -- 172.21.15.16:0/2921678167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f415e960 con 0x7fa2f40cf510 2024-09-10T21:19:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.444+0000 7fa2f0c00640 1 -- 172.21.15.16:0/2921678167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa2e4018420 con 0x7fa2f40cf510 2024-09-10T21:19:07.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.444+0000 7fa2f0c00640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa2d00725e0 0x7fa2d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:07.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.444+0000 7fa2f0c00640 1 -- 172.21.15.16:0/2921678167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fa2e4093f70 con 0x7fa2f40cf510 2024-09-10T21:19:07.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.444+0000 7fa2eb400640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa2d00725e0 0x7fa2d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:07.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.444+0000 7fa2eb400640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa2d00725e0 0x7fa2d0074aa0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0b2000 tx=0x7fa2dc0aa040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:07.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.448+0000 7fa2f0c00640 1 -- 172.21.15.16:0/2921678167 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa2e405f070 con 0x7fa2f40cf510 2024-09-10T21:19:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.632+0000 7fa2f9a00640 1 -- 172.21.15.16:0/2921678167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa2f40d17c0 con 0x7fa2f40cf510 2024-09-10T21:19:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.632+0000 7fa2f0c00640 1 -- 172.21.15.16:0/2921678167 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v138) ==== 80+0+0 (secure 0 0 0) 0x7fa2e4063aa0 con 0x7fa2f40cf510 2024-09-10T21:19:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:19:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.636+0000 7fa2f9a00640 1 -- 172.21.15.16:0/2921678167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa2d00725e0 msgr2=0x7fa2d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.636+0000 7fa2f9a00640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa2d00725e0 0x7fa2d0074aa0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0b2000 tx=0x7fa2dc0aa040 comp rx=0 tx=0).stop 2024-09-10T21:19:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.636+0000 7fa2f9a00640 1 -- 172.21.15.16:0/2921678167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f40cf510 msgr2=0x7fa2f40c9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.636+0000 7fa2f9a00640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f40cf510 0x7fa2f40c9830 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400ec00 tx=0x7fa2e400c6a0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.636+0000 7fa2f9a00640 1 -- 172.21.15.16:0/2921678167 shutdown_connections 2024-09-10T21:19:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.636+0000 7fa2f9a00640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa2d00725e0 0x7fa2d0074aa0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.636+0000 7fa2f9a00640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa2f415eea0 0x7fa2f4155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.636+0000 7fa2f9a00640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2f415e580 0x7fa2f41f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.636+0000 7fa2f9a00640 1 --2- 172.21.15.16:0/2921678167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f40cf510 0x7fa2f40c9830 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.636+0000 7fa2f9a00640 1 -- 172.21.15.16:0/2921678167 >> 172.21.15.16:0/2921678167 conn(0x7fa2f40d0a80 msgr2=0x7fa2f4070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.636+0000 7fa2f9a00640 1 -- 172.21.15.16:0/2921678167 shutdown_connections 2024-09-10T21:19:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.636+0000 7fa2f9a00640 1 -- 172.21.15.16:0/2921678167 wait complete. 2024-09-10T21:19:07.662 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:19:07.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:07.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:19:07.689 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:19:07.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:07.689 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:19:07.744 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:19:07.744 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:19:07.744 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:19:07.754 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:19:07.754 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:19:07.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:19:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 -- 172.21.15.16:0/3303098397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77dc0cd4c0 msgr2=0x7f77dc0b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 --2- 172.21.15.16:0/3303098397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77dc0cd4c0 0x7f77dc0b4aa0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f77dc072320 tx=0x7f77d401f580 comp rx=0 tx=0).stop 2024-09-10T21:19:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 -- 172.21.15.16:0/3303098397 shutdown_connections 2024-09-10T21:19:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 --2- 172.21.15.16:0/3303098397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77dc0cd4c0 0x7f77dc0b4aa0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 --2- 172.21.15.16:0/3303098397 >> v2:172.21.15.52:3300/0 conn(0x7f77dc15c040 0x7f77dc0cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 -- 172.21.15.16:0/3303098397 >> 172.21.15.16:0/3303098397 conn(0x7f77dc0d03a0 msgr2=0x7f77dc0d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:07.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 -- 172.21.15.16:0/3303098397 shutdown_connections 2024-09-10T21:19:07.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 -- 172.21.15.16:0/3303098397 wait complete. 2024-09-10T21:19:07.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 Processor -- start 2024-09-10T21:19:07.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 -- start start 2024-09-10T21:19:07.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77dc0c3a30 0x7f77dc0c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:07.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77dc0cd4c0 0x7f77dc09dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:07.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77dc15c040 0x7f77dc09bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:07.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc0b5370 con 0x7f77dc15c040 2024-09-10T21:19:07.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc0b51f0 con 0x7f77dc0cd4c0 2024-09-10T21:19:07.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77e2000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f77dc0b54f0 con 0x7f77dc0c3a30 2024-09-10T21:19:07.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77dbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77dc15c040 0x7f77dc09bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:07.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77dbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77dc15c040 0x7f77dc09bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38220/0 (socket says 172.21.15.16:38220) 2024-09-10T21:19:07.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77dbe00640 1 -- 172.21.15.16:0/515969051 learned_addr learned my addr 172.21.15.16:0/515969051 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:07.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77db400640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77dc0c3a30 0x7f77dc0c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:07.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77d3e00640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77dc0cd4c0 0x7f77dc09dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:07.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77dbe00640 1 -- 172.21.15.16:0/515969051 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77dc0c3a30 msgr2=0x7f77dc0c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:07.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77dbe00640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77dc0c3a30 0x7f77dc0c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77dbe00640 1 -- 172.21.15.16:0/515969051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77dc0cd4c0 msgr2=0x7f77dc09dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:07.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77dbe00640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77dc0cd4c0 0x7f77dc09dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:07.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.948+0000 7f77dbe00640 1 -- 172.21.15.16:0/515969051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc1f2380 con 0x7f77dc15c040 2024-09-10T21:19:07.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.952+0000 7f77dbe00640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77dc15c040 0x7f77dc09bce0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f77d40b3040 tx=0x7f77d40093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:07.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.952+0000 7f77d3400640 1 -- 172.21.15.16:0/515969051 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d40bf070 con 0x7f77dc15c040 2024-09-10T21:19:07.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.952+0000 7f77d3400640 1 -- 172.21.15.16:0/515969051 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77d4004b40 con 0x7f77dc15c040 2024-09-10T21:19:07.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.952+0000 7f77d3400640 1 -- 172.21.15.16:0/515969051 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d40b5040 con 0x7f77dc15c040 2024-09-10T21:19:07.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.952+0000 7f77e2000640 1 -- 172.21.15.16:0/515969051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77dc1f2640 con 0x7f77dc15c040 2024-09-10T21:19:07.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.952+0000 7f77e2000640 1 -- 172.21.15.16:0/515969051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77dc1f2c00 con 0x7f77dc15c040 2024-09-10T21:19:07.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.952+0000 7f77e2000640 1 -- 172.21.15.16:0/515969051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77dc0b4aa0 con 0x7f77dc15c040 2024-09-10T21:19:07.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.952+0000 7f77d3400640 1 -- 172.21.15.16:0/515969051 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f77d40b8070 con 0x7f77dc15c040 2024-09-10T21:19:07.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.952+0000 7f77d3400640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77b8072610 0x7f77b8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:07.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.952+0000 7f77d3400640 1 -- 172.21.15.16:0/515969051 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f77d41301e0 con 0x7f77dc15c040 2024-09-10T21:19:07.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.952+0000 7f77db400640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77b8072610 0x7f77b8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:07.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.956+0000 7f77db400640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77b8072610 0x7f77b8074ad0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00ab40 tx=0x7f77cc00a250 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:07.956+0000 7f77d3400640 1 -- 172.21.15.16:0/515969051 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f77d40190e0 con 0x7f77dc15c040 2024-09-10T21:19:08.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.140+0000 7f77e2000640 1 -- 172.21.15.16:0/515969051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f77dc15b870 con 0x7f77dc15c040 2024-09-10T21:19:08.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.144+0000 7f77d3400640 1 -- 172.21.15.16:0/515969051 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v138) ==== 80+0+0 (secure 0 0 0) 0x7f77d4100e20 con 0x7f77dc15c040 2024-09-10T21:19:08.148 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-10T21:19:08.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.144+0000 7f77e2000640 1 -- 172.21.15.16:0/515969051 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77b8072610 msgr2=0x7f77b8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.144+0000 7f77e2000640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77b8072610 0x7f77b8074ad0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00ab40 tx=0x7f77cc00a250 comp rx=0 tx=0).stop 2024-09-10T21:19:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.144+0000 7f77e2000640 1 -- 172.21.15.16:0/515969051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77dc15c040 msgr2=0x7f77dc09bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.144+0000 7f77e2000640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77dc15c040 0x7f77dc09bce0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f77d40b3040 tx=0x7f77d40093b0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.144+0000 7f77e2000640 1 -- 172.21.15.16:0/515969051 shutdown_connections 2024-09-10T21:19:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.144+0000 7f77e2000640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f77b8072610 0x7f77b8074ad0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.144+0000 7f77e2000640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77dc15c040 0x7f77dc09bce0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.144+0000 7f77e2000640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77dc0cd4c0 0x7f77dc09dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.144+0000 7f77e2000640 1 --2- 172.21.15.16:0/515969051 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f77dc0c3a30 0x7f77dc0c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.144+0000 7f77e2000640 1 -- 172.21.15.16:0/515969051 >> 172.21.15.16:0/515969051 conn(0x7f77dc0d03a0 msgr2=0x7f77dc15c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.144+0000 7f77e2000640 1 -- 172.21.15.16:0/515969051 shutdown_connections 2024-09-10T21:19:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.144+0000 7f77e2000640 1 -- 172.21.15.16:0/515969051 wait complete. 2024-09-10T21:19:08.169 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T21:19:08.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:08.170 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-10T21:19:08.198 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:19:08.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:08.198 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T21:19:08.253 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:19:08.253 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:19:08.253 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:19:08.262 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:19:08.262 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:19:08.262 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:19:08.263 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:19:08.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:19:08.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:19:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.444+0000 7fba9c200640 1 -- 172.21.15.16:0/3278140572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba940b3200 msgr2=0x7fba940afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.444+0000 7fba9c200640 1 --2- 172.21.15.16:0/3278140572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba940b3200 0x7fba940afae0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fba940b3810 tx=0x7fba8001c8a0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.444+0000 7fba9c200640 1 -- 172.21.15.16:0/3278140572 shutdown_connections 2024-09-10T21:19:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.444+0000 7fba9c200640 1 --2- 172.21.15.16:0/3278140572 >> v2:172.21.15.52:3300/0 conn(0x7fba940d0890 0x7fba940b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.444+0000 7fba9c200640 1 --2- 172.21.15.16:0/3278140572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba940b3200 0x7fba940afae0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.444+0000 7fba9c200640 1 -- 172.21.15.16:0/3278140572 >> 172.21.15.16:0/3278140572 conn(0x7fba940d1d80 msgr2=0x7fba940d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.444+0000 7fba9c200640 1 -- 172.21.15.16:0/3278140572 shutdown_connections 2024-09-10T21:19:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.444+0000 7fba9c200640 1 -- 172.21.15.16:0/3278140572 wait complete. 2024-09-10T21:19:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba9c200640 1 Processor -- start 2024-09-10T21:19:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba9c200640 1 -- start start 2024-09-10T21:19:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba9c200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba940d0890 0x7fba9406f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba9c200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9415f450 0x7fba9415cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba9c200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba940cc870 0x7fba94150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba9c200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba94168d50 con 0x7fba9415f450 2024-09-10T21:19:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba9c200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fba94168bd0 con 0x7fba940cc870 2024-09-10T21:19:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba9c200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fba94168ed0 con 0x7fba940d0890 2024-09-10T21:19:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba98e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9415f450 0x7fba9415cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba98e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9415f450 0x7fba9415cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38230/0 (socket says 172.21.15.16:38230) 2024-09-10T21:19:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba98e00640 1 -- 172.21.15.16:0/3806609514 learned_addr learned my addr 172.21.15.16:0/3806609514 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba9a200640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba940cc870 0x7fba94150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba99800640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba940d0890 0x7fba9406f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba98e00640 1 -- 172.21.15.16:0/3806609514 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba940d0890 msgr2=0x7fba9406f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba98e00640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba940d0890 0x7fba9406f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba98e00640 1 -- 172.21.15.16:0/3806609514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba940cc870 msgr2=0x7fba94150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba98e00640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba940cc870 0x7fba94150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba98e00640 1 -- 172.21.15.16:0/3806609514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba940ccdc0 con 0x7fba9415f450 2024-09-10T21:19:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba98e00640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9415f450 0x7fba9415cb90 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fba800b0040 tx=0x7fba800a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba91600640 1 -- 172.21.15.16:0/3806609514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba800b2040 con 0x7fba9415f450 2024-09-10T21:19:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba91600640 1 -- 172.21.15.16:0/3806609514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba80005020 con 0x7fba9415f450 2024-09-10T21:19:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba91600640 1 -- 172.21.15.16:0/3806609514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80005470 con 0x7fba9415f450 2024-09-10T21:19:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba9c200640 1 -- 172.21.15.16:0/3806609514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba940cd0a0 con 0x7fba9415f450 2024-09-10T21:19:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.448+0000 7fba9c200640 1 -- 172.21.15.16:0/3806609514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba940cd660 con 0x7fba9415f450 2024-09-10T21:19:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.452+0000 7fba9c200640 1 -- 172.21.15.16:0/3806609514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba940cd8a0 con 0x7fba9415f450 2024-09-10T21:19:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.452+0000 7fba91600640 1 -- 172.21.15.16:0/3806609514 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fba800046d0 con 0x7fba9415f450 2024-09-10T21:19:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.452+0000 7fba91600640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba740725e0 0x7fba74074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.452+0000 7fba91600640 1 -- 172.21.15.16:0/3806609514 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fba8012cf10 con 0x7fba9415f450 2024-09-10T21:19:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.452+0000 7fba99800640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba740725e0 0x7fba74074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.456+0000 7fba99800640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba740725e0 0x7fba74074aa0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fba940cc380 tx=0x7fba88009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.456+0000 7fba91600640 1 -- 172.21.15.16:0/3806609514 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fba80016490 con 0x7fba9415f450 2024-09-10T21:19:08.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.668+0000 7fba9c200640 1 -- 172.21.15.16:0/3806609514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba940cfc80 con 0x7fba9415f450 2024-09-10T21:19:08.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.668+0000 7fba91600640 1 -- 172.21.15.16:0/3806609514 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5232 (secure 0 0 0) 0x7fba800f8550 con 0x7fba9415f450 2024-09-10T21:19:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:08.675 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":105,"btime":"2024-09-10T21:18:57:146722+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1121375070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1121375070},{"type":"v1","addr":"172.21.15.52:6835","nonce":1121375070}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":8247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3443454578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3443454578},{"type":"v1","addr":"172.21.15.16:6836","nonce":3443454578}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":8250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1848135983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1848135983},{"type":"v1","addr":"172.21.15.16:6837","nonce":1848135983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:53.527283+0000","modified":"2024-09-10T21:18:56.547852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"b","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1948696903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1948696903},{"type":"v1","addr":"172.21.15.52:6834","nonce":1948696903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8259,"qdb_cluster":[8259]},"id":11}]} 2024-09-10T21:19:08.675 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 105 2024-09-10T21:19:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.672+0000 7fba9c200640 1 -- 172.21.15.16:0/3806609514 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba740725e0 msgr2=0x7fba74074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.672+0000 7fba9c200640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba740725e0 0x7fba74074aa0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fba940cc380 tx=0x7fba88009040 comp rx=0 tx=0).stop 2024-09-10T21:19:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.672+0000 7fba9c200640 1 -- 172.21.15.16:0/3806609514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9415f450 msgr2=0x7fba9415cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.672+0000 7fba9c200640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9415f450 0x7fba9415cb90 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fba800b0040 tx=0x7fba800a57b0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.672+0000 7fba9c200640 1 -- 172.21.15.16:0/3806609514 shutdown_connections 2024-09-10T21:19:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.672+0000 7fba9c200640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fba740725e0 0x7fba74074aa0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.672+0000 7fba9c200640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba940cc870 0x7fba94150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.672+0000 7fba9c200640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9415f450 0x7fba9415cb90 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.672+0000 7fba9c200640 1 --2- 172.21.15.16:0/3806609514 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fba940d0890 0x7fba9406f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.672+0000 7fba9c200640 1 -- 172.21.15.16:0/3806609514 >> 172.21.15.16:0/3806609514 conn(0x7fba940d1d80 msgr2=0x7fba9415da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:08.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.672+0000 7fba9c200640 1 -- 172.21.15.16:0/3806609514 shutdown_connections 2024-09-10T21:19:08.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.672+0000 7fba9c200640 1 -- 172.21.15.16:0/3806609514 wait complete. 2024-09-10T21:19:08.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:19:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 -- 172.21.15.16:0/3787594907 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd00cfd40 msgr2=0x7f2fd00d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 --2- 172.21.15.16:0/3787594907 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd00cfd40 0x7f2fd00d0120 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009840 tx=0x7f2fc001c790 comp rx=0 tx=0).stop 2024-09-10T21:19:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 -- 172.21.15.16:0/3787594907 shutdown_connections 2024-09-10T21:19:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 --2- 172.21.15.16:0/3787594907 >> v2:172.21.15.52:3300/0 conn(0x7f2fd00d2820 0x7f2fd015fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 --2- 172.21.15.16:0/3787594907 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd00cfd40 0x7f2fd00d0120 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 -- 172.21.15.16:0/3787594907 >> 172.21.15.16:0/3787594907 conn(0x7f2fd00d1d00 msgr2=0x7f2fd00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 -- 172.21.15.16:0/3787594907 shutdown_connections 2024-09-10T21:19:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 -- 172.21.15.16:0/3787594907 wait complete. 2024-09-10T21:19:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 Processor -- start 2024-09-10T21:19:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 -- start start 2024-09-10T21:19:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd00cfd40 0x7f2fd0071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fd00d2820 0x7f2fd006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd0160120 0x7f2fd0070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd0169360 con 0x7f2fd00cfd40 2024-09-10T21:19:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd01691e0 con 0x7f2fd00d2820 2024-09-10T21:19:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.840+0000 7f2fd5a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd01694e0 con 0x7f2fd0160120 2024-09-10T21:19:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fcfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd00cfd40 0x7f2fd0071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fcfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd00cfd40 0x7f2fd0071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38232/0 (socket says 172.21.15.16:38232) 2024-09-10T21:19:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fcfe00640 1 -- 172.21.15.16:0/1677584052 learned_addr learned my addr 172.21.15.16:0/1677584052 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fc7e00640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fd00d2820 0x7f2fd006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fd5000640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd0160120 0x7f2fd0070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fcfe00640 1 -- 172.21.15.16:0/1677584052 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd0160120 msgr2=0x7f2fd0070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fcfe00640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd0160120 0x7f2fd0070af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fcfe00640 1 -- 172.21.15.16:0/1677584052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fd00d2820 msgr2=0x7f2fd006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fcfe00640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fd00d2820 0x7f2fd006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fcfe00640 1 -- 172.21.15.16:0/1677584052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd01fb6c0 con 0x7f2fd00cfd40 2024-09-10T21:19:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fcfe00640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd00cfd40 0x7f2fd0071ca0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009790 tx=0x7f2fc0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fccc00640 1 -- 172.21.15.16:0/1677584052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00b2040 con 0x7f2fd00cfd40 2024-09-10T21:19:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fccc00640 1 -- 172.21.15.16:0/1677584052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fc0004320 con 0x7f2fd00cfd40 2024-09-10T21:19:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fccc00640 1 -- 172.21.15.16:0/1677584052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0005280 con 0x7f2fd00cfd40 2024-09-10T21:19:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fd5a00640 1 -- 172.21.15.16:0/1677584052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd01fb9a0 con 0x7f2fd00cfd40 2024-09-10T21:19:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fd5a00640 1 -- 172.21.15.16:0/1677584052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd01fbf60 con 0x7f2fd00cfd40 2024-09-10T21:19:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fd5a00640 1 -- 172.21.15.16:0/1677584052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd01fc570 con 0x7f2fd00cfd40 2024-09-10T21:19:08.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.844+0000 7f2fccc00640 1 -- 172.21.15.16:0/1677584052 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2fc00b5030 con 0x7f2fd00cfd40 2024-09-10T21:19:08.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.848+0000 7f2fccc00640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2fac0725e0 0x7f2fac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:08.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.848+0000 7f2fccc00640 1 -- 172.21.15.16:0/1677584052 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f2fc012c840 con 0x7f2fd00cfd40 2024-09-10T21:19:08.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.848+0000 7f2fc7e00640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2fac0725e0 0x7f2fac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:08.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.848+0000 7f2fc7e00640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2fac0725e0 0x7f2fac074aa0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009600 tx=0x7f2fb80091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:08.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:08.852+0000 7f2fccc00640 1 -- 172.21.15.16:0/1677584052 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2fc0016410 con 0x7f2fd00cfd40 2024-09-10T21:19:09.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.060+0000 7f2fd5a00640 1 -- 172.21.15.16:0/1677584052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fd00ce200 con 0x7f2fd00cfd40 2024-09-10T21:19:09.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.060+0000 7f2fccc00640 1 -- 172.21.15.16:0/1677584052 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5232 (secure 0 0 0) 0x7f2fc00f78c0 con 0x7f2fd00cfd40 2024-09-10T21:19:09.067 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:09.067 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":105,"btime":"2024-09-10T21:18:57:146722+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1121375070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1121375070},{"type":"v1","addr":"172.21.15.52:6835","nonce":1121375070}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":8247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3443454578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3443454578},{"type":"v1","addr":"172.21.15.16:6836","nonce":3443454578}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":8250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1848135983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1848135983},{"type":"v1","addr":"172.21.15.16:6837","nonce":1848135983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:53.527283+0000","modified":"2024-09-10T21:18:56.547852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"b","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1948696903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1948696903},{"type":"v1","addr":"172.21.15.52:6834","nonce":1948696903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8259,"qdb_cluster":[8259]},"id":11}]} 2024-09-10T21:19:09.067 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 105 2024-09-10T21:19:09.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.064+0000 7f2fd5a00640 1 -- 172.21.15.16:0/1677584052 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2fac0725e0 msgr2=0x7f2fac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.064+0000 7f2fd5a00640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2fac0725e0 0x7f2fac074aa0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009600 tx=0x7f2fb80091e0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.064+0000 7f2fd5a00640 1 -- 172.21.15.16:0/1677584052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd00cfd40 msgr2=0x7f2fd0071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.064+0000 7f2fd5a00640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd00cfd40 0x7f2fd0071ca0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009790 tx=0x7f2fc0005dc0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.064+0000 7f2fd5a00640 1 -- 172.21.15.16:0/1677584052 shutdown_connections 2024-09-10T21:19:09.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.064+0000 7f2fd5a00640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2fac0725e0 0x7f2fac074aa0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.064+0000 7f2fd5a00640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2fd0160120 0x7f2fd0070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.064+0000 7f2fd5a00640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fd00d2820 0x7f2fd006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.064+0000 7f2fd5a00640 1 --2- 172.21.15.16:0/1677584052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd00cfd40 0x7f2fd0071ca0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.064+0000 7f2fd5a00640 1 -- 172.21.15.16:0/1677584052 >> 172.21.15.16:0/1677584052 conn(0x7f2fd00d1d00 msgr2=0x7f2fd00ae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:09.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.064+0000 7f2fd5a00640 1 -- 172.21.15.16:0/1677584052 shutdown_connections 2024-09-10T21:19:09.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.064+0000 7f2fd5a00640 1 -- 172.21.15.16:0/1677584052 wait complete. 2024-09-10T21:19:09.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:19:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.232+0000 7feeea800640 1 -- 172.21.15.16:0/375104032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee40d0890 msgr2=0x7feee40b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.232+0000 7feeea800640 1 --2- 172.21.15.16:0/375104032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee40d0890 0x7feee40b0020 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7feedc009c80 tx=0x7feedc01cd40 comp rx=0 tx=0).stop 2024-09-10T21:19:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.232+0000 7feeea800640 1 -- 172.21.15.16:0/375104032 shutdown_connections 2024-09-10T21:19:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.232+0000 7feeea800640 1 --2- 172.21.15.16:0/375104032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee40d0890 0x7feee40b0020 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.232+0000 7feeea800640 1 --2- 172.21.15.16:0/375104032 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee40b3200 0x7feee40afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.232+0000 7feeea800640 1 -- 172.21.15.16:0/375104032 >> 172.21.15.16:0/375104032 conn(0x7feee40d1d80 msgr2=0x7feee40d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.232+0000 7feeea800640 1 -- 172.21.15.16:0/375104032 shutdown_connections 2024-09-10T21:19:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.232+0000 7feeea800640 1 -- 172.21.15.16:0/375104032 wait complete. 2024-09-10T21:19:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feeea800640 1 Processor -- start 2024-09-10T21:19:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feeea800640 1 -- start start 2024-09-10T21:19:09.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feeea800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee40d0890 0x7feee40cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:09.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feeea800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee415f450 0x7feee41526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:09.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feeea800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee41fb620 0x7feee41fa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:09.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feeea800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feee4168d50 con 0x7feee40d0890 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feeea800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feee4168bd0 con 0x7feee41fb620 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feeea800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7feee4168ed0 con 0x7feee415f450 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feee3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee40d0890 0x7feee40cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feee3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee40d0890 0x7feee40cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38238/0 (socket says 172.21.15.16:38238) 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feee3400640 1 -- 172.21.15.16:0/725932558 learned_addr learned my addr 172.21.15.16:0/725932558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feedaa00640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee415f450 0x7feee41526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feee3e00640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee41fb620 0x7feee41fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feee3400640 1 -- 172.21.15.16:0/725932558 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee415f450 msgr2=0x7feee41526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feee3400640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee415f450 0x7feee41526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feee3400640 1 -- 172.21.15.16:0/725932558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee41fb620 msgr2=0x7feee41fa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feee3400640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee41fb620 0x7feee41fa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feee3400640 1 -- 172.21.15.16:0/725932558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee40cd110 con 0x7feee40d0890 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feee3400640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee40d0890 0x7feee40cca80 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7feed400c3c0 tx=0x7feed400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feedbe00640 1 -- 172.21.15.16:0/725932558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40052d0 con 0x7feee40d0890 2024-09-10T21:19:09.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feedbe00640 1 -- 172.21.15.16:0/725932558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feed4007020 con 0x7feee40d0890 2024-09-10T21:19:09.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feedbe00640 1 -- 172.21.15.16:0/725932558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4007470 con 0x7feee40d0890 2024-09-10T21:19:09.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feeea800640 1 -- 172.21.15.16:0/725932558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee40cd3f0 con 0x7feee40d0890 2024-09-10T21:19:09.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.236+0000 7feeea800640 1 -- 172.21.15.16:0/725932558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee41fb100 con 0x7feee40d0890 2024-09-10T21:19:09.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.240+0000 7feeea800640 1 -- 172.21.15.16:0/725932558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee40cfc80 con 0x7feee40d0890 2024-09-10T21:19:09.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.240+0000 7feedbe00640 1 -- 172.21.15.16:0/725932558 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feed40067e0 con 0x7feee40d0890 2024-09-10T21:19:09.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.240+0000 7feedbe00640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec0072610 0x7feec0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:09.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.240+0000 7feedbe00640 1 -- 172.21.15.16:0/725932558 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7feed4092610 con 0x7feee40d0890 2024-09-10T21:19:09.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.240+0000 7feedaa00640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec0072610 0x7feec0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:09.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.240+0000 7feedaa00640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec0072610 0x7feec0074ad0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7feee40cb8d0 tx=0x7feecc00b390 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:09.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.244+0000 7feedbe00640 1 -- 172.21.15.16:0/725932558 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feed4001090 con 0x7feee40d0890 2024-09-10T21:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.420+0000 7feeea800640 1 -- 172.21.15.16:0/725932558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feee41fb2d0 con 0x7feee40d0890 2024-09-10T21:19:09.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.424+0000 7feedbe00640 1 -- 172.21.15.16:0/725932558 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+16691 (secure 0 0 0) 0x7feed405d690 con 0x7feee40d0890 2024-09-10T21:19:09.428 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:09.428 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":138,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:18:55.112658+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:18:51.247076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T21:18:52.488695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:19:09.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.424+0000 7feeea800640 1 -- 172.21.15.16:0/725932558 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec0072610 msgr2=0x7feec0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.424+0000 7feeea800640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec0072610 0x7feec0074ad0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7feee40cb8d0 tx=0x7feecc00b390 comp rx=0 tx=0).stop 2024-09-10T21:19:09.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.424+0000 7feeea800640 1 -- 172.21.15.16:0/725932558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee40d0890 msgr2=0x7feee40cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.424+0000 7feeea800640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee40d0890 0x7feee40cca80 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7feed400c3c0 tx=0x7feed400c880 comp rx=0 tx=0).stop 2024-09-10T21:19:09.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.424+0000 7feeea800640 1 -- 172.21.15.16:0/725932558 shutdown_connections 2024-09-10T21:19:09.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.424+0000 7feeea800640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec0072610 0x7feec0074ad0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.424+0000 7feeea800640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee41fb620 0x7feee41fa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.424+0000 7feeea800640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee415f450 0x7feee41526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.424+0000 7feeea800640 1 --2- 172.21.15.16:0/725932558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee40d0890 0x7feee40cca80 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.424+0000 7feeea800640 1 -- 172.21.15.16:0/725932558 >> 172.21.15.16:0/725932558 conn(0x7feee40d1d80 msgr2=0x7feee4150180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:09.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.424+0000 7feeea800640 1 -- 172.21.15.16:0/725932558 shutdown_connections 2024-09-10T21:19:09.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.424+0000 7feeea800640 1 -- 172.21.15.16:0/725932558 wait complete. 2024-09-10T21:19:09.450 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:19:09.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:19:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.592+0000 7f6ebd000640 1 -- 172.21.15.16:0/1113947073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eb80b0b40 msgr2=0x7f6eb806e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.592+0000 7f6ebd000640 1 --2- 172.21.15.16:0/1113947073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eb80b0b40 0x7f6eb806e020 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009bc0 tx=0x7f6eb001cb30 comp rx=0 tx=0).stop 2024-09-10T21:19:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.592+0000 7f6ebd000640 1 -- 172.21.15.16:0/1113947073 shutdown_connections 2024-09-10T21:19:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.592+0000 7f6ebd000640 1 --2- 172.21.15.16:0/1113947073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eb80b0b40 0x7f6eb806e020 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.592+0000 7f6ebd000640 1 --2- 172.21.15.16:0/1113947073 >> v2:172.21.15.52:3300/0 conn(0x7f6eb8161620 0x7f6eb8070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.592+0000 7f6ebd000640 1 -- 172.21.15.16:0/1113947073 >> 172.21.15.16:0/1113947073 conn(0x7f6eb809b130 msgr2=0x7f6eb80d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.592+0000 7f6ebd000640 1 -- 172.21.15.16:0/1113947073 shutdown_connections 2024-09-10T21:19:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.592+0000 7f6ebd000640 1 -- 172.21.15.16:0/1113947073 wait complete. 2024-09-10T21:19:09.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6ebd000640 1 Processor -- start 2024-09-10T21:19:09.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6ebd000640 1 -- start start 2024-09-10T21:19:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6ebd000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6eb8071230 0x7f6eb8156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6ebd000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb80b0b40 0x7f6eb8155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6ebd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eb8161620 0x7f6eb8155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6ebd000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb816b580 con 0x7f6eb8161620 2024-09-10T21:19:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6ebd000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb816b400 con 0x7f6eb80b0b40 2024-09-10T21:19:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6ebd000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb816b700 con 0x7f6eb8071230 2024-09-10T21:19:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eb6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eb8161620 0x7f6eb8155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eb6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eb8161620 0x7f6eb8155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38246/0 (socket says 172.21.15.16:38246) 2024-09-10T21:19:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eb6800640 1 -- 172.21.15.16:0/385494813 learned_addr learned my addr 172.21.15.16:0/385494813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eb5e00640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6eb8071230 0x7f6eb8156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eb5400640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb80b0b40 0x7f6eb8155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eb6800640 1 -- 172.21.15.16:0/385494813 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6eb8071230 msgr2=0x7f6eb8156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eb6800640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6eb8071230 0x7f6eb8156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eb6800640 1 -- 172.21.15.16:0/385494813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb80b0b40 msgr2=0x7f6eb8155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eb6800640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb80b0b40 0x7f6eb8155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eb6800640 1 -- 172.21.15.16:0/385494813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb80b0950 con 0x7f6eb8161620 2024-09-10T21:19:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eb5400640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb80b0b40 0x7f6eb8155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:19:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eb6800640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eb8161620 0x7f6eb8155580 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00b0040 tx=0x7f6eb0009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:09.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eae000640 1 -- 172.21.15.16:0/385494813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00b2040 con 0x7f6eb8161620 2024-09-10T21:19:09.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eae000640 1 -- 172.21.15.16:0/385494813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6eb0004040 con 0x7f6eb8161620 2024-09-10T21:19:09.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6eae000640 1 -- 172.21.15.16:0/385494813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0004470 con 0x7f6eb8161620 2024-09-10T21:19:09.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6ebd000640 1 -- 172.21.15.16:0/385494813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb80afd40 con 0x7f6eb8161620 2024-09-10T21:19:09.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6ebd000640 1 -- 172.21.15.16:0/385494813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb80b0300 con 0x7f6eb8161620 2024-09-10T21:19:09.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.596+0000 7f6ebd000640 1 -- 172.21.15.16:0/385494813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb806e020 con 0x7f6eb8161620 2024-09-10T21:19:09.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.600+0000 7f6eae000640 1 -- 172.21.15.16:0/385494813 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6eb00b5070 con 0x7f6eb8161620 2024-09-10T21:19:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.600+0000 7f6eae000640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6e98072610 0x7f6e98074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.600+0000 7f6eae000640 1 -- 172.21.15.16:0/385494813 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f6eb012d200 con 0x7f6eb8161620 2024-09-10T21:19:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.600+0000 7f6eb5e00640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6e98072610 0x7f6e98074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.600+0000 7f6eb5e00640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6e98072610 0x7f6e98074ad0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800bb00 tx=0x7f6ea8005dd0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.604+0000 7f6eae000640 1 -- 172.21.15.16:0/385494813 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6eb00c4060 con 0x7f6eb8161620 2024-09-10T21:19:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.812+0000 7f6ebd000640 1 -- 172.21.15.16:0/385494813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6eb8151ac0 con 0x7f6eb8161620 2024-09-10T21:19:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.812+0000 7f6eae000640 1 -- 172.21.15.16:0/385494813 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5232 (secure 0 0 0) 0x7f6eb0016750 con 0x7f6eb8161620 2024-09-10T21:19:09.818 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:09.818 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":105,"btime":"2024-09-10T21:18:57:146722+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1121375070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1121375070},{"type":"v1","addr":"172.21.15.52:6835","nonce":1121375070}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":8247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3443454578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3443454578},{"type":"v1","addr":"172.21.15.16:6836","nonce":3443454578}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":8250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1848135983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1848135983},{"type":"v1","addr":"172.21.15.16:6837","nonce":1848135983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:53.527283+0000","modified":"2024-09-10T21:18:56.547852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"b","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1948696903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1948696903},{"type":"v1","addr":"172.21.15.52:6834","nonce":1948696903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8259,"qdb_cluster":[8259]},"id":11}]} 2024-09-10T21:19:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 105 2024-09-10T21:19:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.816+0000 7f6ebd000640 1 -- 172.21.15.16:0/385494813 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6e98072610 msgr2=0x7f6e98074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.816+0000 7f6ebd000640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6e98072610 0x7f6e98074ad0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800bb00 tx=0x7f6ea8005dd0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.816+0000 7f6ebd000640 1 -- 172.21.15.16:0/385494813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eb8161620 msgr2=0x7f6eb8155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.816+0000 7f6ebd000640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eb8161620 0x7f6eb8155580 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00b0040 tx=0x7f6eb0009280 comp rx=0 tx=0).stop 2024-09-10T21:19:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.816+0000 7f6ebd000640 1 -- 172.21.15.16:0/385494813 shutdown_connections 2024-09-10T21:19:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.816+0000 7f6ebd000640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6e98072610 0x7f6e98074ad0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.816+0000 7f6ebd000640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eb8161620 0x7f6eb8155580 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.816+0000 7f6ebd000640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb80b0b40 0x7f6eb8155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.816+0000 7f6ebd000640 1 --2- 172.21.15.16:0/385494813 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6eb8071230 0x7f6eb8156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.816+0000 7f6ebd000640 1 -- 172.21.15.16:0/385494813 >> 172.21.15.16:0/385494813 conn(0x7f6eb809b130 msgr2=0x7f6eb80ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.816+0000 7f6ebd000640 1 -- 172.21.15.16:0/385494813 shutdown_connections 2024-09-10T21:19:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.816+0000 7f6ebd000640 1 -- 172.21.15.16:0/385494813 wait complete. 2024-09-10T21:19:09.840 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:19:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 -- 172.21.15.16:0/3903809346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b2815a270 msgr2=0x7f4b280ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 --2- 172.21.15.16:0/3903809346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b2815a270 0x7f4b280ce610 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f4b280c6b80 tx=0x7f4b1801c800 comp rx=0 tx=0).stop 2024-09-10T21:19:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 -- 172.21.15.16:0/3903809346 shutdown_connections 2024-09-10T21:19:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 --2- 172.21.15.16:0/3903809346 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4b280705e0 0x7f4b280ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 --2- 172.21.15.16:0/3903809346 >> v2:172.21.15.52:3300/0 conn(0x7f4b280d06b0 0x7f4b280700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 --2- 172.21.15.16:0/3903809346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b2815a270 0x7f4b280ce610 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 -- 172.21.15.16:0/3903809346 >> 172.21.15.16:0/3903809346 conn(0x7f4b280cfb20 msgr2=0x7f4b280cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 -- 172.21.15.16:0/3903809346 shutdown_connections 2024-09-10T21:19:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 -- 172.21.15.16:0/3903809346 wait complete. 2024-09-10T21:19:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 Processor -- start 2024-09-10T21:19:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 -- start start 2024-09-10T21:19:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b280705e0 0x7f4b280c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b280d06b0 0x7f4b281fb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4b2815a270 0x7f4b281fbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28072ed0 con 0x7f4b280d06b0 2024-09-10T21:19:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28072d50 con 0x7f4b280705e0 2024-09-10T21:19:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b2d200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4b28073050 con 0x7f4b2815a270 2024-09-10T21:19:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b1d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b280d06b0 0x7f4b281fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b1d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b280d06b0 0x7f4b281fb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38250/0 (socket says 172.21.15.16:38250) 2024-09-10T21:19:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b1d400640 1 -- 172.21.15.16:0/4129302676 learned_addr learned my addr 172.21.15.16:0/4129302676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b25e00640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b280705e0 0x7f4b280c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b26800640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4b2815a270 0x7f4b281fbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b1d400640 1 -- 172.21.15.16:0/4129302676 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4b2815a270 msgr2=0x7f4b281fbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b1d400640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4b2815a270 0x7f4b281fbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b1d400640 1 -- 172.21.15.16:0/4129302676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b280705e0 msgr2=0x7f4b280c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b1d400640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b280705e0 0x7f4b280c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b1d400640 1 -- 172.21.15.16:0/4129302676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2815bac0 con 0x7f4b280d06b0 2024-09-10T21:19:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.984+0000 7f4b26800640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4b2815a270 0x7f4b281fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.988+0000 7f4b1d400640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b280d06b0 0x7f4b281fb570 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000dbc0 tx=0x7f4b1000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:09.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.988+0000 7f4b1ea00640 1 -- 172.21.15.16:0/4129302676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10004220 con 0x7f4b280d06b0 2024-09-10T21:19:09.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.988+0000 7f4b1ea00640 1 -- 172.21.15.16:0/4129302676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b1000ae50 con 0x7f4b280d06b0 2024-09-10T21:19:09.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.988+0000 7f4b2d200640 1 -- 172.21.15.16:0/4129302676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b280c7170 con 0x7f4b280d06b0 2024-09-10T21:19:09.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.988+0000 7f4b1ea00640 1 -- 172.21.15.16:0/4129302676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10003860 con 0x7f4b280d06b0 2024-09-10T21:19:09.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.988+0000 7f4b2d200640 1 -- 172.21.15.16:0/4129302676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2815bd60 con 0x7f4b280d06b0 2024-09-10T21:19:09.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.988+0000 7f4b2d200640 1 -- 172.21.15.16:0/4129302676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b280700a0 con 0x7f4b280d06b0 2024-09-10T21:19:09.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.988+0000 7f4b1ea00640 1 -- 172.21.15.16:0/4129302676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4b1000b040 con 0x7f4b280d06b0 2024-09-10T21:19:09.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.988+0000 7f4b1ea00640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4b04072610 0x7f4b04074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:09.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.988+0000 7f4b1ea00640 1 -- 172.21.15.16:0/4129302676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f4b10093520 con 0x7f4b280d06b0 2024-09-10T21:19:09.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.988+0000 7f4b25e00640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4b04072610 0x7f4b04074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:09.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.992+0000 7f4b25e00640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4b04072610 0x7f4b04074ad0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4b180af000 tx=0x7f4b180a7040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:09.992+0000 7f4b1ea00640 1 -- 172.21.15.16:0/4129302676 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4b1005e5a0 con 0x7f4b280d06b0 2024-09-10T21:19:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.204+0000 7f4b2d200640 1 -- 172.21.15.16:0/4129302676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b280b5aa0 con 0x7f4b280d06b0 2024-09-10T21:19:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.208+0000 7f4b1ea00640 1 -- 172.21.15.16:0/4129302676 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5232 (secure 0 0 0) 0x7f4b10062fd0 con 0x7f4b280d06b0 2024-09-10T21:19:10.212 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:10.212 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":105,"btime":"2024-09-10T21:18:57:146722+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1121375070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1121375070},{"type":"v1","addr":"172.21.15.52:6835","nonce":1121375070}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":8247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3443454578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3443454578},{"type":"v1","addr":"172.21.15.16:6836","nonce":3443454578}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":8250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1848135983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1848135983},{"type":"v1","addr":"172.21.15.16:6837","nonce":1848135983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:18:53.527283+0000","modified":"2024-09-10T21:18:56.547852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8259":{"gid":8259,"name":"b","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.52:6834/1948696903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1948696903},{"type":"v1","addr":"172.21.15.52:6834","nonce":1948696903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8259,"qdb_cluster":[8259]},"id":11}]} 2024-09-10T21:19:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 105 2024-09-10T21:19:10.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.208+0000 7f4b2d200640 1 -- 172.21.15.16:0/4129302676 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4b04072610 msgr2=0x7f4b04074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:10.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.208+0000 7f4b2d200640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4b04072610 0x7f4b04074ad0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4b180af000 tx=0x7f4b180a7040 comp rx=0 tx=0).stop 2024-09-10T21:19:10.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.208+0000 7f4b2d200640 1 -- 172.21.15.16:0/4129302676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b280d06b0 msgr2=0x7f4b281fb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:10.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.208+0000 7f4b2d200640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b280d06b0 0x7f4b281fb570 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000dbc0 tx=0x7f4b1000a620 comp rx=0 tx=0).stop 2024-09-10T21:19:10.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.208+0000 7f4b2d200640 1 -- 172.21.15.16:0/4129302676 shutdown_connections 2024-09-10T21:19:10.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.208+0000 7f4b2d200640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f4b04072610 0x7f4b04074ad0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.208+0000 7f4b2d200640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4b2815a270 0x7f4b281fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.208+0000 7f4b2d200640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b280d06b0 0x7f4b281fb570 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.208+0000 7f4b2d200640 1 --2- 172.21.15.16:0/4129302676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b280705e0 0x7f4b280c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.208+0000 7f4b2d200640 1 -- 172.21.15.16:0/4129302676 >> 172.21.15.16:0/4129302676 conn(0x7f4b280cfb20 msgr2=0x7f4b2815e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.208+0000 7f4b2d200640 1 -- 172.21.15.16:0/4129302676 shutdown_connections 2024-09-10T21:19:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.208+0000 7f4b2d200640 1 -- 172.21.15.16:0/4129302676 wait complete. 2024-09-10T21:19:10.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:19:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 -- 172.21.15.16:0/334034791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0adc15f450 msgr2=0x7f0adc0cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 --2- 172.21.15.16:0/334034791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0adc15f450 0x7f0adc0cfc80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009900 tx=0x7f0ad001f070 comp rx=0 tx=0).stop 2024-09-10T21:19:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 -- 172.21.15.16:0/334034791 shutdown_connections 2024-09-10T21:19:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 --2- 172.21.15.16:0/334034791 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0adc0b3200 0x7f0adc0d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 --2- 172.21.15.16:0/334034791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0adc15f450 0x7f0adc0cfc80 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 -- 172.21.15.16:0/334034791 >> 172.21.15.16:0/334034791 conn(0x7f0adc0d1d80 msgr2=0x7f0adc0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 -- 172.21.15.16:0/334034791 shutdown_connections 2024-09-10T21:19:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 -- 172.21.15.16:0/334034791 wait complete. 2024-09-10T21:19:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 Processor -- start 2024-09-10T21:19:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 -- start start 2024-09-10T21:19:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0adc0d0890 0x7f0adc071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0adc15f450 0x7f0adc0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0adc0b69a0 0x7f0adc0c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc0b2e70 con 0x7f0adc0d0890 2024-09-10T21:19:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc0b2cf0 con 0x7f0adc15f450 2024-09-10T21:19:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae3a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f0adc0b2ff0 con 0x7f0adc0b69a0 2024-09-10T21:19:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0adc0d0890 0x7f0adc071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0adc0d0890 0x7f0adc071ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38270/0 (socket says 172.21.15.16:38270) 2024-09-10T21:19:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae1000640 1 -- 172.21.15.16:0/166339128 learned_addr learned my addr 172.21.15.16:0/166339128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0adbe00640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0adc15f450 0x7f0adc0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:10.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae1a00640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0adc0b69a0 0x7f0adc0c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:10.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae1000640 1 -- 172.21.15.16:0/166339128 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0adc0b69a0 msgr2=0x7f0adc0c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:10.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae1000640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0adc0b69a0 0x7f0adc0c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae1000640 1 -- 172.21.15.16:0/166339128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0adc15f450 msgr2=0x7f0adc0b6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:10.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae1000640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0adc15f450 0x7f0adc0b6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.376+0000 7f0ae1000640 1 -- 172.21.15.16:0/166339128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0adc1f5a60 con 0x7f0adc0d0890 2024-09-10T21:19:10.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.380+0000 7f0ae1000640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0adc0d0890 0x7f0adc071ef0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009850 tx=0x7f0ad0005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:10.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.380+0000 7f0ad8c00640 1 -- 172.21.15.16:0/166339128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad00b5040 con 0x7f0adc0d0890 2024-09-10T21:19:10.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.380+0000 7f0ad8c00640 1 -- 172.21.15.16:0/166339128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ad00038f0 con 0x7f0adc0d0890 2024-09-10T21:19:10.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.380+0000 7f0ad8c00640 1 -- 172.21.15.16:0/166339128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0003d00 con 0x7f0adc0d0890 2024-09-10T21:19:10.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.380+0000 7f0ae3a00640 1 -- 172.21.15.16:0/166339128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0adc0c6ba0 con 0x7f0adc0d0890 2024-09-10T21:19:10.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.380+0000 7f0ae3a00640 1 -- 172.21.15.16:0/166339128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0adc0c7160 con 0x7f0adc0d0890 2024-09-10T21:19:10.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.380+0000 7f0ae3a00640 1 -- 172.21.15.16:0/166339128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0adc0b1e90 con 0x7f0adc0d0890 2024-09-10T21:19:10.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.380+0000 7f0ad8c00640 1 -- 172.21.15.16:0/166339128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0ad00b8070 con 0x7f0adc0d0890 2024-09-10T21:19:10.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.380+0000 7f0ad8c00640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0abc0725e0 0x7f0abc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:10.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.380+0000 7f0ad8c00640 1 -- 172.21.15.16:0/166339128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f0ad01300e0 con 0x7f0adc0d0890 2024-09-10T21:19:10.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.380+0000 7f0adbe00640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0abc0725e0 0x7f0abc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:10.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.384+0000 7f0adbe00640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0abc0725e0 0x7f0abc074aa0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80058d0 tx=0x7f0ac8005860 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.384+0000 7f0ad8c00640 1 -- 172.21.15.16:0/166339128 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0ad0018c30 con 0x7f0adc0d0890 2024-09-10T21:19:10.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.560+0000 7f0ae3a00640 1 -- 172.21.15.16:0/166339128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0adc074ba0 con 0x7f0adc0d0890 2024-09-10T21:19:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.560+0000 7f0ad8c00640 1 -- 172.21.15.16:0/166339128 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+16691 (secure 0 0 0) 0x7f0ad00fb7b0 con 0x7f0adc0d0890 2024-09-10T21:19:10.567 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:10.567 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":138,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:18:55.112658+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:18:51.247076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T21:18:52.488695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:19:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.564+0000 7f0ae3a00640 1 -- 172.21.15.16:0/166339128 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0abc0725e0 msgr2=0x7f0abc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.564+0000 7f0ae3a00640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0abc0725e0 0x7f0abc074aa0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80058d0 tx=0x7f0ac8005860 comp rx=0 tx=0).stop 2024-09-10T21:19:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.564+0000 7f0ae3a00640 1 -- 172.21.15.16:0/166339128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0adc0d0890 msgr2=0x7f0adc071ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.564+0000 7f0ae3a00640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0adc0d0890 0x7f0adc071ef0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009850 tx=0x7f0ad0005e70 comp rx=0 tx=0).stop 2024-09-10T21:19:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.564+0000 7f0ae3a00640 1 -- 172.21.15.16:0/166339128 shutdown_connections 2024-09-10T21:19:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.564+0000 7f0ae3a00640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f0abc0725e0 0x7f0abc074aa0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.564+0000 7f0ae3a00640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f0adc0b69a0 0x7f0adc0c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.564+0000 7f0ae3a00640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0adc15f450 0x7f0adc0b6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.564+0000 7f0ae3a00640 1 --2- 172.21.15.16:0/166339128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0adc0d0890 0x7f0adc071ef0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.564+0000 7f0ae3a00640 1 -- 172.21.15.16:0/166339128 >> 172.21.15.16:0/166339128 conn(0x7f0adc0d1d80 msgr2=0x7f0adc15d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.564+0000 7f0ae3a00640 1 -- 172.21.15.16:0/166339128 shutdown_connections 2024-09-10T21:19:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.564+0000 7f0ae3a00640 1 -- 172.21.15.16:0/166339128 wait complete. 2024-09-10T21:19:10.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:19:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 -- 172.21.15.16:0/2913161749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7044070520 msgr2=0x7f70440ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 --2- 172.21.15.16:0/2913161749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7044070520 0x7f70440ca4c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f703c009c80 tx=0x7f703c01cda0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 -- 172.21.15.16:0/2913161749 shutdown_connections 2024-09-10T21:19:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 --2- 172.21.15.16:0/2913161749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7044070520 0x7f70440ca4c0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 --2- 172.21.15.16:0/2913161749 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f704415bbb0 0x7f704406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 -- 172.21.15.16:0/2913161749 >> 172.21.15.16:0/2913161749 conn(0x7f70440cfb20 msgr2=0x7f70440cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 -- 172.21.15.16:0/2913161749 shutdown_connections 2024-09-10T21:19:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 -- 172.21.15.16:0/2913161749 wait complete. 2024-09-10T21:19:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 Processor -- start 2024-09-10T21:19:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 -- start start 2024-09-10T21:19:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7044070520 0x7f7044152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7044155000 0x7f70441578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704415bbb0 0x7f70440cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7044167b20 con 0x7f704415bbb0 2024-09-10T21:19:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f70441679a0 con 0x7f7044155000 2024-09-10T21:19:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7049600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f7044167ca0 con 0x7f7044070520 2024-09-10T21:19:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7043400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704415bbb0 0x7f70440cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7043400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704415bbb0 0x7f70440cec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38288/0 (socket says 172.21.15.16:38288) 2024-09-10T21:19:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7043400640 1 -- 172.21.15.16:0/1390995966 learned_addr learned my addr 172.21.15.16:0/1390995966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f703a000640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7044155000 0x7f70441578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.732+0000 7f7042a00640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7044070520 0x7f7044152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f7043400640 1 -- 172.21.15.16:0/1390995966 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7044070520 msgr2=0x7f7044152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f7043400640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7044070520 0x7f7044152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f7043400640 1 -- 172.21.15.16:0/1390995966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7044155000 msgr2=0x7f70441578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f7043400640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7044155000 0x7f70441578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f7043400640 1 -- 172.21.15.16:0/1390995966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70441581a0 con 0x7f704415bbb0 2024-09-10T21:19:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f7043400640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704415bbb0 0x7f70440cec80 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f703c009bd0 tx=0x7f703c009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f703b400640 1 -- 172.21.15.16:0/1390995966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c0bb070 con 0x7f704415bbb0 2024-09-10T21:19:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f703b400640 1 -- 172.21.15.16:0/1390995966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f703c004e00 con 0x7f704415bbb0 2024-09-10T21:19:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f703b400640 1 -- 172.21.15.16:0/1390995966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c0b2040 con 0x7f704415bbb0 2024-09-10T21:19:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f7049600640 1 -- 172.21.15.16:0/1390995966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7044158480 con 0x7f704415bbb0 2024-09-10T21:19:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f7049600640 1 -- 172.21.15.16:0/1390995966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70440cdbe0 con 0x7f704415bbb0 2024-09-10T21:19:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f7049600640 1 -- 172.21.15.16:0/1390995966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70440ca4c0 con 0x7f704415bbb0 2024-09-10T21:19:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f703b400640 1 -- 172.21.15.16:0/1390995966 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f703c0b5030 con 0x7f704415bbb0 2024-09-10T21:19:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.736+0000 7f703b400640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f70200725e0 0x7f7020074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.740+0000 7f703b400640 1 -- 172.21.15.16:0/1390995966 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f703c12cd20 con 0x7f704415bbb0 2024-09-10T21:19:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.740+0000 7f7042a00640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f70200725e0 0x7f7020074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.740+0000 7f7042a00640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f70200725e0 0x7f7020074aa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f70440c9ca0 tx=0x7f7034009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.740+0000 7f703b400640 1 -- 172.21.15.16:0/1390995966 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f703c016990 con 0x7f704415bbb0 2024-09-10T21:19:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.948+0000 7f7049600640 1 -- 172.21.15.16:0/1390995966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f70440c6640 con 0x7f704415bbb0 2024-09-10T21:19:11.114 INFO:tasks.ceph.mds.b.smithi052.stderr: -2296> 2024-09-10T21:18:56.099+0000 7fc47e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:19:11.115 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-10T21:19:10.948+0000 7ff429400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:19:11.115 INFO:tasks.ceph.mds.b.smithi052.stdout:starting mds.b at 2024-09-10T21:19:11.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.952+0000 7f703b400640 1 -- 172.21.15.16:0/1390995966 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) ==== 161+0+0 (secure 0 0 0) 0x7f703c0f7da0 con 0x7f704415bbb0 2024-09-10T21:19:11.116 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:19:11.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.952+0000 7f7049600640 1 -- 172.21.15.16:0/1390995966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f70200725e0 msgr2=0x7f7020074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:11.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.952+0000 7f7049600640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f70200725e0 0x7f7020074aa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f70440c9ca0 tx=0x7f7034009040 comp rx=0 tx=0).stop 2024-09-10T21:19:11.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.952+0000 7f7049600640 1 -- 172.21.15.16:0/1390995966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704415bbb0 msgr2=0x7f70440cec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:11.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.952+0000 7f7049600640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704415bbb0 0x7f70440cec80 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f703c009bd0 tx=0x7f703c009390 comp rx=0 tx=0).stop 2024-09-10T21:19:11.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.952+0000 7f7049600640 1 -- 172.21.15.16:0/1390995966 shutdown_connections 2024-09-10T21:19:11.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.956+0000 7f7049600640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f70200725e0 0x7f7020074aa0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:11.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.956+0000 7f7049600640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704415bbb0 0x7f70440cec80 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:11.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.956+0000 7f7049600640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7044155000 0x7f70441578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:11.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.956+0000 7f7049600640 1 --2- 172.21.15.16:0/1390995966 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f7044070520 0x7f7044152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:11.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.956+0000 7f7049600640 1 -- 172.21.15.16:0/1390995966 >> 172.21.15.16:0/1390995966 conn(0x7f70440cfb20 msgr2=0x7f704406eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:11.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.956+0000 7f7049600640 1 -- 172.21.15.16:0/1390995966 shutdown_connections 2024-09-10T21:19:11.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:10.956+0000 7f7049600640 1 -- 172.21.15.16:0/1390995966 wait complete. 2024-09-10T21:19:11.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:19:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 -- 172.21.15.16:0/1713471178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c15bc10 msgr2=0x7f426c0b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 --2- 172.21.15.16:0/1713471178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c15bc10 0x7f426c0b2f60 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f426c073160 tx=0x7f425c01c770 comp rx=0 tx=0).stop 2024-09-10T21:19:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 -- 172.21.15.16:0/1713471178 shutdown_connections 2024-09-10T21:19:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 --2- 172.21.15.16:0/1713471178 >> v2:172.21.15.52:3300/0 conn(0x7f426c15bff0 0x7f426c0cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 --2- 172.21.15.16:0/1713471178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c15bc10 0x7f426c0b2f60 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 -- 172.21.15.16:0/1713471178 >> 172.21.15.16:0/1713471178 conn(0x7f426c09a9d0 msgr2=0x7f426c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:11.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 -- 172.21.15.16:0/1713471178 shutdown_connections 2024-09-10T21:19:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 -- 172.21.15.16:0/1713471178 wait complete. 2024-09-10T21:19:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 Processor -- start 2024-09-10T21:19:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 -- start start 2024-09-10T21:19:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0cc8b0 0x7f426c1658e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f426c15bc10 0x7f426c150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f426c15bff0 0x7f426c154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f426c168ed0 con 0x7f426c0cc8b0 2024-09-10T21:19:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f426c168d50 con 0x7f426c15bc10 2024-09-10T21:19:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426b200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f426c169050 con 0x7f426c15bff0 2024-09-10T21:19:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f4269e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0cc8b0 0x7f426c1658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:11.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f4269e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0cc8b0 0x7f426c1658e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38296/0 (socket says 172.21.15.16:38296) 2024-09-10T21:19:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f4269e00640 1 -- 172.21.15.16:0/3750723040 learned_addr learned my addr 172.21.15.16:0/3750723040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f4263e00640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f426c15bc10 0x7f426c150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426a800640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f426c15bff0 0x7f426c154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f4269e00640 1 -- 172.21.15.16:0/3750723040 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f426c15bff0 msgr2=0x7f426c154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f4269e00640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f426c15bff0 0x7f426c154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f4269e00640 1 -- 172.21.15.16:0/3750723040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f426c15bc10 msgr2=0x7f426c150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f4269e00640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f426c15bc10 0x7f426c150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f4269e00640 1 -- 172.21.15.16:0/3750723040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426c06f770 con 0x7f426c0cc8b0 2024-09-10T21:19:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.264+0000 7f426a800640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f426c15bff0 0x7f426c154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.268+0000 7f4269e00640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0cc8b0 0x7f426c1658e0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f425c000c10 tx=0x7f425c0a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.268+0000 7f4262000640 1 -- 172.21.15.16:0/3750723040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c0b2040 con 0x7f426c0cc8b0 2024-09-10T21:19:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.268+0000 7f4262000640 1 -- 172.21.15.16:0/3750723040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f425c004340 con 0x7f426c0cc8b0 2024-09-10T21:19:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.268+0000 7f426b200640 1 -- 172.21.15.16:0/3750723040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426c14fa90 con 0x7f426c0cc8b0 2024-09-10T21:19:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.268+0000 7f4262000640 1 -- 172.21.15.16:0/3750723040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c003860 con 0x7f426c0cc8b0 2024-09-10T21:19:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.268+0000 7f426b200640 1 -- 172.21.15.16:0/3750723040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f426c0b7860 con 0x7f426c0cc8b0 2024-09-10T21:19:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.268+0000 7f426b200640 1 -- 172.21.15.16:0/3750723040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426c14f880 con 0x7f426c0cc8b0 2024-09-10T21:19:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.268+0000 7f4262000640 1 -- 172.21.15.16:0/3750723040 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f425c0b5070 con 0x7f426c0cc8b0 2024-09-10T21:19:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.268+0000 7f4262000640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42480725e0 0x7f4248074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.268+0000 7f4262000640 1 -- 172.21.15.16:0/3750723040 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5824+0+0 (secure 0 0 0) 0x7f425c12d150 con 0x7f426c0cc8b0 2024-09-10T21:19:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.268+0000 7f4263e00640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42480725e0 0x7f4248074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.272+0000 7f4263e00640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42480725e0 0x7f4248074aa0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f426c0b5ad0 tx=0x7f425400b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:11.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.272+0000 7f4262000640 1 -- 172.21.15.16:0/3750723040 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f425c0163f0 con 0x7f426c0cc8b0 2024-09-10T21:19:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.484+0000 7f426b200640 1 -- 172.21.15.16:0/3750723040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f426c152570 con 0x7f426c0cc8b0 2024-09-10T21:19:11.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.956+0000 7f4262000640 1 -- 172.21.15.16:0/3750723040 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v107) ==== 104+0+0 (secure 0 0 0) 0x7f425c0f87d0 con 0x7f426c0cc8b0 2024-09-10T21:19:11.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.960+0000 7f426b200640 1 -- 172.21.15.16:0/3750723040 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42480725e0 msgr2=0x7f4248074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:11.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.960+0000 7f426b200640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42480725e0 0x7f4248074aa0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f426c0b5ad0 tx=0x7f425400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:19:11.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.960+0000 7f426b200640 1 -- 172.21.15.16:0/3750723040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0cc8b0 msgr2=0x7f426c1658e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.960+0000 7f426b200640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0cc8b0 0x7f426c1658e0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f425c000c10 tx=0x7f425c0a5630 comp rx=0 tx=0).stop 2024-09-10T21:19:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.960+0000 7f426b200640 1 -- 172.21.15.16:0/3750723040 shutdown_connections 2024-09-10T21:19:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.960+0000 7f426b200640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f42480725e0 0x7f4248074aa0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.960+0000 7f426b200640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f426c15bff0 0x7f426c154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.960+0000 7f426b200640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f426c15bc10 0x7f426c150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.960+0000 7f426b200640 1 --2- 172.21.15.16:0/3750723040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426c0cc8b0 0x7f426c1658e0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.960+0000 7f426b200640 1 -- 172.21.15.16:0/3750723040 >> 172.21.15.16:0/3750723040 conn(0x7f426c09a9d0 msgr2=0x7f426c155bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.960+0000 7f426b200640 1 -- 172.21.15.16:0/3750723040 shutdown_connections 2024-09-10T21:19:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:11.960+0000 7f426b200640 1 -- 172.21.15.16:0/3750723040 wait complete. 2024-09-10T21:19:11.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:19:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.132+0000 7fd441a00640 1 -- 172.21.15.16:0/3717695481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43c0ce9f0 msgr2=0x7fd43c0cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.132+0000 7fd441a00640 1 --2- 172.21.15.16:0/3717695481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43c0ce9f0 0x7fd43c0cfe50 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fd43c069970 tx=0x7fd43401cd50 comp rx=0 tx=0).stop 2024-09-10T21:19:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.132+0000 7fd441a00640 1 -- 172.21.15.16:0/3717695481 shutdown_connections 2024-09-10T21:19:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.132+0000 7fd441a00640 1 --2- 172.21.15.16:0/3717695481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43c0ce9f0 0x7fd43c0cfe50 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.132+0000 7fd441a00640 1 --2- 172.21.15.16:0/3717695481 >> v2:172.21.15.52:3300/0 conn(0x7fd43c0ce610 0x7fd43c0d2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.132+0000 7fd441a00640 1 -- 172.21.15.16:0/3717695481 >> 172.21.15.16:0/3717695481 conn(0x7fd43c0d1b70 msgr2=0x7fd43c0d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.132+0000 7fd441a00640 1 -- 172.21.15.16:0/3717695481 shutdown_connections 2024-09-10T21:19:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.132+0000 7fd441a00640 1 -- 172.21.15.16:0/3717695481 wait complete. 2024-09-10T21:19:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441a00640 1 Processor -- start 2024-09-10T21:19:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441a00640 1 -- start start 2024-09-10T21:19:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd43c0b2a10 0x7fd43c0b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd43c0ce610 0x7fd43c14dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43c0ce9f0 0x7fd43c14a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c0b21c0 con 0x7fd43c0ce9f0 2024-09-10T21:19:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c0b2040 con 0x7fd43c0b2a10 2024-09-10T21:19:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fd43c0b2340 con 0x7fd43c0ce610 2024-09-10T21:19:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43c0ce9f0 0x7fd43c14a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43c0ce9f0 0x7fd43c14a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54916/0 (socket says 172.21.15.16:54916) 2024-09-10T21:19:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441000640 1 -- 172.21.15.16:0/928497816 learned_addr learned my addr 172.21.15.16:0/928497816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd433400640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd43c0ce610 0x7fd43c14dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd43be00640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd43c0b2a10 0x7fd43c0b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441000640 1 -- 172.21.15.16:0/928497816 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd43c0ce610 msgr2=0x7fd43c14dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441000640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd43c0ce610 0x7fd43c14dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441000640 1 -- 172.21.15.16:0/928497816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd43c0b2a10 msgr2=0x7fd43c0b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441000640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd43c0b2a10 0x7fd43c0b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441000640 1 -- 172.21.15.16:0/928497816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c1fb470 con 0x7fd43c0ce9f0 2024-09-10T21:19:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd43be00640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd43c0b2a10 0x7fd43c0b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441000640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43c0ce9f0 0x7fd43c14a6b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fd43401b400 tx=0x7fd434003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd438c00640 1 -- 172.21.15.16:0/928497816 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4340b2040 con 0x7fd43c0ce9f0 2024-09-10T21:19:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd438c00640 1 -- 172.21.15.16:0/928497816 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4340ad7f0 con 0x7fd43c0ce9f0 2024-09-10T21:19:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd438c00640 1 -- 172.21.15.16:0/928497816 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4340adc20 con 0x7fd43c0ce9f0 2024-09-10T21:19:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441a00640 1 -- 172.21.15.16:0/928497816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c1fa6b0 con 0x7fd43c0ce9f0 2024-09-10T21:19:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.136+0000 7fd441a00640 1 -- 172.21.15.16:0/928497816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c1fac70 con 0x7fd43c0ce9f0 2024-09-10T21:19:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.140+0000 7fd441a00640 1 -- 172.21.15.16:0/928497816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43c0d2690 con 0x7fd43c0ce9f0 2024-09-10T21:19:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.140+0000 7fd438c00640 1 -- 172.21.15.16:0/928497816 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd4340b5070 con 0x7fd43c0ce9f0 2024-09-10T21:19:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.140+0000 7fd438c00640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd4180725e0 0x7fd418074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.140+0000 7fd438c00640 1 -- 172.21.15.16:0/928497816 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7fd43412d8d0 con 0x7fd43c0ce9f0 2024-09-10T21:19:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.140+0000 7fd43be00640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd4180725e0 0x7fd418074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.140+0000 7fd43be00640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd4180725e0 0x7fd418074aa0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0cbf30 tx=0x7fd42c005f50 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.144+0000 7fd438c00640 1 -- 172.21.15.16:0/928497816 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd434016970 con 0x7fd43c0ce9f0 2024-09-10T21:19:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.320+0000 7fd441a00640 1 -- 172.21.15.16:0/928497816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd43c06eba0 con 0x7fd43c0ce9f0 2024-09-10T21:19:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.960+0000 7fd438c00640 1 -- 172.21.15.16:0/928497816 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v141) ==== 181+0+0 (secure 0 0 0) 0x7fd4340f8f50 con 0x7fd43c0ce9f0 2024-09-10T21:19:12.964 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:19:12.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.960+0000 7fd441a00640 1 -- 172.21.15.16:0/928497816 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd4180725e0 msgr2=0x7fd418074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:12.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.960+0000 7fd441a00640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd4180725e0 0x7fd418074aa0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0cbf30 tx=0x7fd42c005f50 comp rx=0 tx=0).stop 2024-09-10T21:19:12.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.960+0000 7fd441a00640 1 -- 172.21.15.16:0/928497816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43c0ce9f0 msgr2=0x7fd43c14a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:12.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.960+0000 7fd441a00640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43c0ce9f0 0x7fd43c14a6b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fd43401b400 tx=0x7fd434003f80 comp rx=0 tx=0).stop 2024-09-10T21:19:12.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.964+0000 7fd441a00640 1 -- 172.21.15.16:0/928497816 shutdown_connections 2024-09-10T21:19:12.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.964+0000 7fd441a00640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fd4180725e0 0x7fd418074aa0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:12.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.964+0000 7fd441a00640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43c0ce9f0 0x7fd43c14a6b0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:12.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.964+0000 7fd441a00640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fd43c0ce610 0x7fd43c14dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:12.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.964+0000 7fd441a00640 1 --2- 172.21.15.16:0/928497816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd43c0b2a10 0x7fd43c0b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:12.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.964+0000 7fd441a00640 1 -- 172.21.15.16:0/928497816 >> 172.21.15.16:0/928497816 conn(0x7fd43c0d1b70 msgr2=0x7fd43c15edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:12.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.964+0000 7fd441a00640 1 -- 172.21.15.16:0/928497816 shutdown_connections 2024-09-10T21:19:12.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:12.964+0000 7fd441a00640 1 -- 172.21.15.16:0/928497816 wait complete. 2024-09-10T21:19:12.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:19:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 -- 172.21.15.16:0/1208286046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd680cfce0 msgr2=0x7efd680d1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 --2- 172.21.15.16:0/1208286046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd680cfce0 0x7efd680d1ef0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7efd6808d430 tx=0x7efd6001cd80 comp rx=0 tx=0).stop 2024-09-10T21:19:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 -- 172.21.15.16:0/1208286046 shutdown_connections 2024-09-10T21:19:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 --2- 172.21.15.16:0/1208286046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd680cfce0 0x7efd680d1ef0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 --2- 172.21.15.16:0/1208286046 >> v2:172.21.15.52:3300/0 conn(0x7efd68161a10 0x7efd680d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 -- 172.21.15.16:0/1208286046 >> 172.21.15.16:0/1208286046 conn(0x7efd680d3ff0 msgr2=0x7efd680d4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 -- 172.21.15.16:0/1208286046 shutdown_connections 2024-09-10T21:19:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 -- 172.21.15.16:0/1208286046 wait complete. 2024-09-10T21:19:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 Processor -- start 2024-09-10T21:19:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 -- start start 2024-09-10T21:19:13.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd680cfce0 0x7efd6814efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:13.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd68161630 0x7efd680b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:13.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd68161a10 0x7efd6815f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd6816b2e0 con 0x7efd68161a10 2024-09-10T21:19:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efd6816b160 con 0x7efd68161630 2024-09-10T21:19:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd70000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7efd6816b460 con 0x7efd680cfce0 2024-09-10T21:19:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd6e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd68161a10 0x7efd6815f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd6cc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd68161630 0x7efd680b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd6d600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd680cfce0 0x7efd6814efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd6e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd68161a10 0x7efd6815f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54938/0 (socket says 172.21.15.16:54938) 2024-09-10T21:19:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.144+0000 7efd6e000640 1 -- 172.21.15.16:0/625228794 learned_addr learned my addr 172.21.15.16:0/625228794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd6e000640 1 -- 172.21.15.16:0/625228794 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd680cfce0 msgr2=0x7efd6814efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd6e000640 1 --2- 172.21.15.16:0/625228794 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd680cfce0 0x7efd6814efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd6e000640 1 -- 172.21.15.16:0/625228794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd68161630 msgr2=0x7efd680b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd6e000640 1 --2- 172.21.15.16:0/625228794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd68161630 0x7efd680b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd6e000640 1 -- 172.21.15.16:0/625228794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd680b4bb0 con 0x7efd68161a10 2024-09-10T21:19:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd6e000640 1 --2- 172.21.15.16:0/625228794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd68161a10 0x7efd6815f520 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7efd60019990 tx=0x7efd60004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd65600640 1 -- 172.21.15.16:0/625228794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd600bb070 con 0x7efd68161a10 2024-09-10T21:19:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd65600640 1 -- 172.21.15.16:0/625228794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd60004b40 con 0x7efd68161a10 2024-09-10T21:19:13.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd70000640 1 -- 172.21.15.16:0/625228794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd680b4ef0 con 0x7efd68161a10 2024-09-10T21:19:13.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd65600640 1 -- 172.21.15.16:0/625228794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd600b2040 con 0x7efd68161a10 2024-09-10T21:19:13.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd70000640 1 -- 172.21.15.16:0/625228794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd680b5730 con 0x7efd68161a10 2024-09-10T21:19:13.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd70000640 1 -- 172.21.15.16:0/625228794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd680d1ef0 con 0x7efd68161a10 2024-09-10T21:19:13.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd65600640 1 -- 172.21.15.16:0/625228794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efd600b5030 con 0x7efd68161a10 2024-09-10T21:19:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.148+0000 7efd65600640 1 --2- 172.21.15.16:0/625228794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd480725e0 0x7efd48074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.152+0000 7efd65600640 1 -- 172.21.15.16:0/625228794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5386+0+0 (secure 0 0 0) 0x7efd6012cdf0 con 0x7efd68161a10 2024-09-10T21:19:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.152+0000 7efd6d600640 1 --2- 172.21.15.16:0/625228794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd480725e0 0x7efd48074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.152+0000 7efd6d600640 1 --2- 172.21.15.16:0/625228794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd480725e0 0x7efd48074aa0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7efd5c007fd0 tx=0x7efd5c009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.152+0000 7efd65600640 1 -- 172.21.15.16:0/625228794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efd600f1de0 con 0x7efd68161a10 2024-09-10T21:19:13.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.328+0000 7efd70000640 1 -- 172.21.15.16:0/625228794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7efd680cae80 con 0x7efd68161a10 2024-09-10T21:19:13.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.964+0000 7efd65600640 1 -- 172.21.15.16:0/625228794 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v142) ==== 169+0+0 (secure 0 0 0) 0x7efd680b4ef0 con 0x7efd68161a10 2024-09-10T21:19:13.970 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-10T21:19:13.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.968+0000 7efd70000640 1 -- 172.21.15.16:0/625228794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd480725e0 msgr2=0x7efd48074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:13.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.968+0000 7efd70000640 1 --2- 172.21.15.16:0/625228794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd480725e0 0x7efd48074aa0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7efd5c007fd0 tx=0x7efd5c009040 comp rx=0 tx=0).stop 2024-09-10T21:19:13.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.968+0000 7efd70000640 1 -- 172.21.15.16:0/625228794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd68161a10 msgr2=0x7efd6815f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:13.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.968+0000 7efd70000640 1 --2- 172.21.15.16:0/625228794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd68161a10 0x7efd6815f520 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7efd60019990 tx=0x7efd60004270 comp rx=0 tx=0).stop 2024-09-10T21:19:13.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.968+0000 7efd70000640 1 -- 172.21.15.16:0/625228794 shutdown_connections 2024-09-10T21:19:13.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.968+0000 7efd70000640 1 --2- 172.21.15.16:0/625228794 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efd480725e0 0x7efd48074aa0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:13.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.968+0000 7efd70000640 1 --2- 172.21.15.16:0/625228794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd68161a10 0x7efd6815f520 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:13.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.968+0000 7efd70000640 1 --2- 172.21.15.16:0/625228794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd68161630 0x7efd680b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:13.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.968+0000 7efd70000640 1 --2- 172.21.15.16:0/625228794 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efd680cfce0 0x7efd6814efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:13.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.968+0000 7efd70000640 1 -- 172.21.15.16:0/625228794 >> 172.21.15.16:0/625228794 conn(0x7efd680d3ff0 msgr2=0x7efd68151f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:13.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.968+0000 7efd70000640 1 -- 172.21.15.16:0/625228794 shutdown_connections 2024-09-10T21:19:13.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:13.968+0000 7efd70000640 1 -- 172.21.15.16:0/625228794 wait complete. 2024-09-10T21:19:13.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:19:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.140+0000 7f190e000640 1 -- 172.21.15.16:0/1188639876 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f19080b5050 msgr2=0x7f190815f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.140+0000 7f190e000640 1 --2- 172.21.15.16:0/1188639876 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f19080b5050 0x7f190815f850 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009bc0 tx=0x7f18f801cba0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.140+0000 7f190e000640 1 -- 172.21.15.16:0/1188639876 shutdown_connections 2024-09-10T21:19:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.140+0000 7f190e000640 1 --2- 172.21.15.16:0/1188639876 >> v2:172.21.15.52:3300/0 conn(0x7f190806e640 0x7f190815c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.140+0000 7f190e000640 1 --2- 172.21.15.16:0/1188639876 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f19080b5050 0x7f190815f850 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.140+0000 7f190e000640 1 -- 172.21.15.16:0/1188639876 >> 172.21.15.16:0/1188639876 conn(0x7f19080cfb20 msgr2=0x7f19080cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.140+0000 7f190e000640 1 -- 172.21.15.16:0/1188639876 shutdown_connections 2024-09-10T21:19:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.140+0000 7f190e000640 1 -- 172.21.15.16:0/1188639876 wait complete. 2024-09-10T21:19:14.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.140+0000 7f190e000640 1 Processor -- start 2024-09-10T21:19:14.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f190e000640 1 -- start start 2024-09-10T21:19:14.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f190e000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f190806e640 0x7f190815af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:14.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f190e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080b5050 0x7f190815b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f190e000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190815ced0 0x7f19081539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f190e000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1908168dd0 con 0x7f19080b5050 2024-09-10T21:19:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f190e000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1908168c50 con 0x7f190815ced0 2024-09-10T21:19:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f190e000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f1908168f50 con 0x7f190806e640 2024-09-10T21:19:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f18fea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080b5050 0x7f190815b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f1907400640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f190806e640 0x7f190815af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f18fea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080b5050 0x7f190815b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54940/0 (socket says 172.21.15.16:54940) 2024-09-10T21:19:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f18fea00640 1 -- 172.21.15.16:0/2029828199 learned_addr learned my addr 172.21.15.16:0/2029828199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:14.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f1907e00640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190815ced0 0x7f19081539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:14.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f1907400640 1 -- 172.21.15.16:0/2029828199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190815ced0 msgr2=0x7f19081539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:14.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f1907400640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190815ced0 0x7f19081539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f1907400640 1 -- 172.21.15.16:0/2029828199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080b5050 msgr2=0x7f190815b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f1907400640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080b5050 0x7f190815b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f1907400640 1 -- 172.21.15.16:0/2029828199 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1908159e80 con 0x7f190806e640 2024-09-10T21:19:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f18fea00640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080b5050 0x7f190815b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:19:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f1907400640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f190806e640 0x7f190815af20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f18f80ac000 tx=0x7f18f8005a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f18ffe00640 1 -- 172.21.15.16:0/2029828199 <== mon.2 v2:172.21.15.52:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f80b2040 con 0x7f190806e640 2024-09-10T21:19:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f18ffe00640 1 -- 172.21.15.16:0/2029828199 <== mon.2 v2:172.21.15.52:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18f8004130 con 0x7f190806e640 2024-09-10T21:19:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f18ffe00640 1 -- 172.21.15.16:0/2029828199 <== mon.2 v2:172.21.15.52:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8004560 con 0x7f190806e640 2024-09-10T21:19:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f190e000640 1 -- 172.21.15.16:0/2029828199 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190815a160 con 0x7f190806e640 2024-09-10T21:19:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f190e000640 1 -- 172.21.15.16:0/2029828199 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f190815a720 con 0x7f190806e640 2024-09-10T21:19:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.144+0000 7f190e000640 1 -- 172.21.15.16:0/2029828199 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190815c990 con 0x7f190806e640 2024-09-10T21:19:14.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.148+0000 7f18ffe00640 1 -- 172.21.15.16:0/2029828199 <== mon.2 v2:172.21.15.52:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f18f80b5070 con 0x7f190806e640 2024-09-10T21:19:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.148+0000 7f18ffe00640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18e4072610 0x7f18e4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.148+0000 7f18ffe00640 1 -- 172.21.15.16:0/2029828199 <== mon.2 v2:172.21.15.52:3301/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f18f812c620 con 0x7f190806e640 2024-09-10T21:19:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.148+0000 7f18fea00640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18e4072610 0x7f18e4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:14.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.148+0000 7f18fea00640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18e4072610 0x7f18e4074ad0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f19080c5d90 tx=0x7f18f0009260 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:14.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.152+0000 7f18ffe00640 1 -- 172.21.15.16:0/2029828199 <== mon.2 v2:172.21.15.52:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f18f80a5e20 con 0x7f190806e640 2024-09-10T21:19:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.332+0000 7f190e000640 1 -- 172.21.15.16:0/2029828199 --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f1908069970 con 0x7f190806e640 2024-09-10T21:19:14.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.340+0000 7f18ffe00640 1 -- 172.21.15.16:0/2029828199 <== mon.2 v2:172.21.15.52:3301/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v66) ==== 71+0+0 (secure 0 0 0) 0x7f18f8016790 con 0x7f190806e640 2024-09-10T21:19:14.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.344+0000 7f190e000640 1 -- 172.21.15.16:0/2029828199 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18e4072610 msgr2=0x7f18e4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:14.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.344+0000 7f190e000640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18e4072610 0x7f18e4074ad0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f19080c5d90 tx=0x7f18f0009260 comp rx=0 tx=0).stop 2024-09-10T21:19:14.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.344+0000 7f190e000640 1 -- 172.21.15.16:0/2029828199 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f190806e640 msgr2=0x7f190815af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:14.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.344+0000 7f190e000640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f190806e640 0x7f190815af20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f18f80ac000 tx=0x7f18f8005a80 comp rx=0 tx=0).stop 2024-09-10T21:19:14.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.344+0000 7f190e000640 1 -- 172.21.15.16:0/2029828199 shutdown_connections 2024-09-10T21:19:14.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.344+0000 7f190e000640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f18e4072610 0x7f18e4074ad0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.344+0000 7f190e000640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190815ced0 0x7f19081539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.344+0000 7f190e000640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080b5050 0x7f190815b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.344+0000 7f190e000640 1 --2- 172.21.15.16:0/2029828199 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f190806e640 0x7f190815af20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.344+0000 7f190e000640 1 -- 172.21.15.16:0/2029828199 >> 172.21.15.16:0/2029828199 conn(0x7f19080cfb20 msgr2=0x7f190815d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:14.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.344+0000 7f190e000640 1 -- 172.21.15.16:0/2029828199 shutdown_connections 2024-09-10T21:19:14.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.344+0000 7f190e000640 1 -- 172.21.15.16:0/2029828199 wait complete. 2024-09-10T21:19:14.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2024-09-10T21:19:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.512+0000 7f68b4600640 1 -- 172.21.15.16:0/2317128751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6894004810 msgr2=0x7f6894004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:14.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.512+0000 7f68b4600640 1 --2- 172.21.15.16:0/2317128751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6894004810 0x7f6894004c90 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6898001ec0 tx=0x7f689801cf80 comp rx=0 tx=0).stop 2024-09-10T21:19:14.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.512+0000 7f68b2600640 1 -- 172.21.15.16:0/2317128751 reap_dead start 2024-09-10T21:19:14.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.512+0000 7f68b4600640 1 -- 172.21.15.16:0/2317128751 shutdown_connections 2024-09-10T21:19:14.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.512+0000 7f68b4600640 1 -- 172.21.15.16:0/2317128751 >> 172.21.15.16:0/2317128751 conn(0x7f68ac0d0c30 msgr2=0x7f68ac0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:14.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.512+0000 7f68b4600640 1 -- 172.21.15.16:0/2317128751 shutdown_connections 2024-09-10T21:19:14.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.512+0000 7f68b4600640 1 -- 172.21.15.16:0/2317128751 wait complete. 2024-09-10T21:19:14.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.512+0000 7f68b4600640 1 Processor -- start 2024-09-10T21:19:14.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.512+0000 7f68b4600640 1 -- start start 2024-09-10T21:19:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b4600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894004810 0x7f68ac156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b4600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68ac15c6b0 0x7f68ac15de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b4600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f68ac15e3d0 0x7f68ac15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b4600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac1670c0 con 0x7f68ac15c6b0 2024-09-10T21:19:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b4600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac166f40 con 0x7f6894004810 2024-09-10T21:19:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b4600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f68ac167240 con 0x7f68ac15e3d0 2024-09-10T21:19:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68ac15c6b0 0x7f68ac15de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68ac15c6b0 0x7f68ac15de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54948/0 (socket says 172.21.15.16:54948) 2024-09-10T21:19:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b1200640 1 -- 172.21.15.16:0/4103146968 learned_addr learned my addr 172.21.15.16:0/4103146968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b1c00640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894004810 0x7f68ac156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b2600640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f68ac15e3d0 0x7f68ac15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b1200640 1 -- 172.21.15.16:0/4103146968 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f68ac15e3d0 msgr2=0x7f68ac15aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b1200640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f68ac15e3d0 0x7f68ac15aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b1200640 1 -- 172.21.15.16:0/4103146968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894004810 msgr2=0x7f68ac156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b1200640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894004810 0x7f68ac156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b1200640 1 -- 172.21.15.16:0/4103146968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ac162b40 con 0x7f68ac15c6b0 2024-09-10T21:19:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b1200640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68ac15c6b0 0x7f68ac15de90 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f6898098a40 tx=0x7f68980010b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68a9600640 1 -- 172.21.15.16:0/4103146968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898006090 con 0x7f68ac15c6b0 2024-09-10T21:19:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68a9600640 1 -- 172.21.15.16:0/4103146968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6898003600 con 0x7f68ac15c6b0 2024-09-10T21:19:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b4600640 1 -- 172.21.15.16:0/4103146968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ac162db0 con 0x7f68ac15c6b0 2024-09-10T21:19:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68a9600640 1 -- 172.21.15.16:0/4103146968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898003a50 con 0x7f68ac15c6b0 2024-09-10T21:19:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b4600640 1 -- 172.21.15.16:0/4103146968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ac163370 con 0x7f68ac15c6b0 2024-09-10T21:19:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.515+0000 7f68b4600640 1 -- 172.21.15.16:0/4103146968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68ac0737c0 con 0x7f68ac15c6b0 2024-09-10T21:19:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.519+0000 7f68a9600640 1 -- 172.21.15.16:0/4103146968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6898003bf0 con 0x7f68ac15c6b0 2024-09-10T21:19:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.519+0000 7f68a9600640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f688c072610 0x7f688c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.519+0000 7f68a9600640 1 -- 172.21.15.16:0/4103146968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f689812d7d0 con 0x7f68ac15c6b0 2024-09-10T21:19:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.519+0000 7f68b1c00640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f688c072610 0x7f688c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.519+0000 7f68b1c00640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f688c072610 0x7f688c074ad0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009b00 tx=0x7f68a00091e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:14.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.523+0000 7f68a9600640 1 -- 172.21.15.16:0/4103146968 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f68980bf060 con 0x7f68ac15c6b0 2024-09-10T21:19:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.755+0000 7f68b4600640 1 -- 172.21.15.16:0/4103146968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) -- 0x7f68ac0b6bb0 con 0x7f68ac15c6b0 2024-09-10T21:19:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.963+0000 7f68a9600640 1 -- 172.21.15.16:0/4103146968 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v451) ==== 151+0+0 (secure 0 0 0) 0x7f6898016d30 con 0x7f68ac15c6b0 2024-09-10T21:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.967+0000 7f68b4600640 1 -- 172.21.15.16:0/4103146968 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f688c072610 msgr2=0x7f688c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.967+0000 7f68b4600640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f688c072610 0x7f688c074ad0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009b00 tx=0x7f68a00091e0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.967+0000 7f68b4600640 1 -- 172.21.15.16:0/4103146968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68ac15c6b0 msgr2=0x7f68ac15de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.967+0000 7f68b4600640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68ac15c6b0 0x7f68ac15de90 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f6898098a40 tx=0x7f68980010b0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.967+0000 7f68b4600640 1 -- 172.21.15.16:0/4103146968 shutdown_connections 2024-09-10T21:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.967+0000 7f68b4600640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f688c072610 0x7f688c074ad0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.967+0000 7f68b4600640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f68ac15e3d0 0x7f68ac15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.967+0000 7f68b4600640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68ac15c6b0 0x7f68ac15de90 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.967+0000 7f68b4600640 1 --2- 172.21.15.16:0/4103146968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894004810 0x7f68ac156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.967+0000 7f68b4600640 1 -- 172.21.15.16:0/4103146968 >> 172.21.15.16:0/4103146968 conn(0x7f68ac0d0c30 msgr2=0x7f68ac0c9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.967+0000 7f68b4600640 1 -- 172.21.15.16:0/4103146968 shutdown_connections 2024-09-10T21:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:14.967+0000 7f68b4600640 1 -- 172.21.15.16:0/4103146968 wait complete. 2024-09-10T21:19:14.993 INFO:tasks.cephfs_test_runner:When a client has a bug (which we will simulate) preventing it from releasing caps, ... skipped 'Require FUSE client to inject client release failure' 2024-09-10T21:19:14.994 INFO:tasks.cephfs_test_runner: 2024-09-10T21:19:14.994 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-10T21:19:14.994 INFO:tasks.cephfs_test_runner:Ran 10 tests in 500.391s 2024-09-10T21:19:14.994 INFO:tasks.cephfs_test_runner: 2024-09-10T21:19:14.994 INFO:tasks.cephfs_test_runner:OK (skipped=4) 2024-09-10T21:19:14.994 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-10T21:19:15.003 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T21:19:15.011 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T21:19:15.012 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:19:15.012 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:19:15.021 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:19:15.021 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-10T21:19:15.021 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:19:15.075 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6525 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:19:15.075 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:19:15.075 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:19:15.085 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:19:15.085 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-09-10T21:19:15.086 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:19:15.136 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:19:15.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:15.136 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:19:15.145 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:19:15.145 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:19:15.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:15.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:19:15.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:15.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:19:15.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:15.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-10T21:19:15.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:15.822 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-10T21:19:15.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:15.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-10T21:19:16.291 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-09-10T21:19:16.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:19:16.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:16.294 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:19:16.305 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:19:16.305 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:19:16.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:16.305 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:19:16.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:16.372 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:19:16.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:16.834 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-09-10T21:19:16.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:16.962 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-09-10T21:19:16.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:19:16.972 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-09-10T21:19:17.437 INFO:teuthology.orchestra.run.smithi052.stderr:Cannot find device "ceph-brx" 2024-09-10T21:19:17.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:19:17.439 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T21:19:17.449 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:19:17.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:19:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 -- 172.21.15.16:0/1908030088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5580ca6b0 msgr2=0x7fc5580726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 --2- 172.21.15.16:0/1908030088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5580ca6b0 0x7fc5580726b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fc55806daf0 tx=0x7fc55001ccf0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 -- 172.21.15.16:0/1908030088 shutdown_connections 2024-09-10T21:19:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 --2- 172.21.15.16:0/1908030088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5580ca6b0 0x7fc5580726b0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 --2- 172.21.15.16:0/1908030088 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc55806e640 0x7fc5580ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 --2- 172.21.15.16:0/1908030088 >> v2:172.21.15.52:3300/0 conn(0x7fc5581599d0 0x7fc55815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 -- 172.21.15.16:0/1908030088 >> 172.21.15.16:0/1908030088 conn(0x7fc5580cfb20 msgr2=0x7fc5580cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 -- 172.21.15.16:0/1908030088 shutdown_connections 2024-09-10T21:19:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 -- 172.21.15.16:0/1908030088 wait complete. 2024-09-10T21:19:17.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 Processor -- start 2024-09-10T21:19:17.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 -- start start 2024-09-10T21:19:17.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc55806e640 0x7fc5580c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:17.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc5580ca6b0 0x7fc5581450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:17.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5581599d0 0x7fc55815b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:17.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc558074300 con 0x7fc5581599d0 2024-09-10T21:19:17.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc558074180 con 0x7fc55806e640 2024-09-10T21:19:17.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc55de00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fc558074480 con 0x7fc5580ca6b0 2024-09-10T21:19:17.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.603+0000 7fc557400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5581599d0 0x7fc55815b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:17.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc557400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5581599d0 0x7fc55815b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54968/0 (socket says 172.21.15.16:54968) 2024-09-10T21:19:17.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc557400640 1 -- 172.21.15.16:0/191983015 learned_addr learned my addr 172.21.15.16:0/191983015 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:17.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc556a00640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc55806e640 0x7fc5580c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:17.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc54fe00640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc5580ca6b0 0x7fc5581450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc557400640 1 -- 172.21.15.16:0/191983015 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc5580ca6b0 msgr2=0x7fc5581450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc557400640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc5580ca6b0 0x7fc5581450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc557400640 1 -- 172.21.15.16:0/191983015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc55806e640 msgr2=0x7fc5580c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc557400640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc55806e640 0x7fc5580c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc557400640 1 -- 172.21.15.16:0/191983015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc558144030 con 0x7fc5581599d0 2024-09-10T21:19:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc556a00640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc55806e640 0x7fc5580c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc54fe00640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc5580ca6b0 0x7fc5581450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc557400640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5581599d0 0x7fc55815b030 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fc550009bd0 tx=0x7fc550003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc54f400640 1 -- 172.21.15.16:0/191983015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5500bb070 con 0x7fc5581599d0 2024-09-10T21:19:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc54f400640 1 -- 172.21.15.16:0/191983015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc550004e00 con 0x7fc5581599d0 2024-09-10T21:19:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc54f400640 1 -- 172.21.15.16:0/191983015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5500b2040 con 0x7fc5581599d0 2024-09-10T21:19:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc55de00640 1 -- 172.21.15.16:0/191983015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc558144310 con 0x7fc5581599d0 2024-09-10T21:19:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc55de00640 1 -- 172.21.15.16:0/191983015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5581448d0 con 0x7fc5581599d0 2024-09-10T21:19:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc55de00640 1 -- 172.21.15.16:0/191983015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5580726b0 con 0x7fc5581599d0 2024-09-10T21:19:17.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.607+0000 7fc54f400640 1 -- 172.21.15.16:0/191983015 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc5500b5030 con 0x7fc5581599d0 2024-09-10T21:19:17.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.611+0000 7fc54f400640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc5340725e0 0x7fc534074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:17.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.611+0000 7fc54f400640 1 -- 172.21.15.16:0/191983015 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7fc55012caf0 con 0x7fc5581599d0 2024-09-10T21:19:17.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.611+0000 7fc556a00640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc5340725e0 0x7fc534074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.611+0000 7fc556a00640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc5340725e0 0x7fc534074aa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc54800bbd0 tx=0x7fc54800b5b0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.615+0000 7fc54f400640 1 -- 172.21.15.16:0/191983015 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc5500a5e20 con 0x7fc5581599d0 2024-09-10T21:19:17.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc55de00640 1 -- 172.21.15.16:0/191983015 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc558159330 con 0x7fc5340725e0 2024-09-10T21:19:17.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc54f400640 1 -- 172.21.15.16:0/191983015 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13466 (secure 0 0 0) 0x7fc558159330 con 0x7fc5340725e0 2024-09-10T21:19:17.797 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:17.797 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:19:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc55de00640 1 -- 172.21.15.16:0/191983015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc5340725e0 msgr2=0x7fc534074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc55de00640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc5340725e0 0x7fc534074aa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc54800bbd0 tx=0x7fc54800b5b0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc55de00640 1 -- 172.21.15.16:0/191983015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5581599d0 msgr2=0x7fc55815b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc55de00640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5581599d0 0x7fc55815b030 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fc550009bd0 tx=0x7fc550003f60 comp rx=0 tx=0).stop 2024-09-10T21:19:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc55de00640 1 -- 172.21.15.16:0/191983015 shutdown_connections 2024-09-10T21:19:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc55de00640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fc5340725e0 0x7fc534074aa0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc55de00640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5581599d0 0x7fc55815b030 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc55de00640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fc5580ca6b0 0x7fc5581450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc55de00640 1 --2- 172.21.15.16:0/191983015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc55806e640 0x7fc5580c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc55de00640 1 -- 172.21.15.16:0/191983015 >> 172.21.15.16:0/191983015 conn(0x7fc5580cfb20 msgr2=0x7fc558159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:17.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc55de00640 1 -- 172.21.15.16:0/191983015 shutdown_connections 2024-09-10T21:19:17.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.791+0000 7fc55de00640 1 -- 172.21.15.16:0/191983015 wait complete. 2024-09-10T21:19:17.819 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-10T21:19:17.579420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":297760,"kb_used_data":14964,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754676960,"statfs":{"total":773094113280,"available":772789207040,"internally_reserved":0,"allocated":15323136,"data_stored":7282765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4056,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-48,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001131"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":325,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T21:19:12.966651+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:19:12.966651+0000","last_peered":"2024-09-10T21:19:12.966651+0000","last_clean":"2024-09-10T21:19:12.966651+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:19:12.966651+0000","last_undegraded":"2024-09-10T21:19:12.966651+0000","last_fullsized":"2024-09-10T21:19:12.966651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509550,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37076,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509552,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37656,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334184,"statfs":{"total":96636764160,"available":96598204416,"internally_reserved":0,"allocated":2363392,"data_stored":1354936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509550,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509549,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32920,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338920,"statfs":{"total":96636764160,"available":96603054080,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509550,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542254,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41208,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330632,"statfs":{"total":96636764160,"available":96594567168,"internally_reserved":0,"allocated":1740800,"data_stored":752504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542253,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37664,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334176,"statfs":{"total":96636764160,"available":96598196224,"internally_reserved":0,"allocated":2371584,"data_stored":1352485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:19:17.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:19:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.959+0000 7fbf76c00640 1 -- 172.21.15.16:0/1437851595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf70154f90 msgr2=0x7fbf70072d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.959+0000 7fbf76c00640 1 --2- 172.21.15.16:0/1437851595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf70154f90 0x7fbf70072d40 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fbf70072040 tx=0x7fbf6001cd30 comp rx=0 tx=0).stop 2024-09-10T21:19:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.959+0000 7fbf76c00640 1 -- 172.21.15.16:0/1437851595 shutdown_connections 2024-09-10T21:19:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.959+0000 7fbf76c00640 1 --2- 172.21.15.16:0/1437851595 >> v2:172.21.15.52:3300/0 conn(0x7fbf7015bc70 0x7fbf7006ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.959+0000 7fbf76c00640 1 --2- 172.21.15.16:0/1437851595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf70154f90 0x7fbf70072d40 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.959+0000 7fbf76c00640 1 -- 172.21.15.16:0/1437851595 >> 172.21.15.16:0/1437851595 conn(0x7fbf700cfb20 msgr2=0x7fbf700cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.959+0000 7fbf76c00640 1 -- 172.21.15.16:0/1437851595 shutdown_connections 2024-09-10T21:19:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.959+0000 7fbf76c00640 1 -- 172.21.15.16:0/1437851595 wait complete. 2024-09-10T21:19:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf76c00640 1 Processor -- start 2024-09-10T21:19:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf76c00640 1 -- start start 2024-09-10T21:19:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf76c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf70070520 0x7fbf701f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf76c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf70154f90 0x7fbf700c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf76c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf7015bc70 0x7fbf70159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf76c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbf70167980 con 0x7fbf7015bc70 2024-09-10T21:19:17.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf76c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf700733f0 con 0x7fbf70070520 2024-09-10T21:19:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf76c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fbf70167b00 con 0x7fbf70154f90 2024-09-10T21:19:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf74c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf7015bc70 0x7fbf70159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf74c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf7015bc70 0x7fbf70159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54982/0 (socket says 172.21.15.16:54982) 2024-09-10T21:19:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf74c00640 1 -- 172.21.15.16:0/279479797 learned_addr learned my addr 172.21.15.16:0/279479797 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf67400640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf70154f90 0x7fbf700c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf6fe00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf70070520 0x7fbf701f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf74c00640 1 -- 172.21.15.16:0/279479797 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf70154f90 msgr2=0x7fbf700c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf74c00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf70154f90 0x7fbf700c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf74c00640 1 -- 172.21.15.16:0/279479797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf70070520 msgr2=0x7fbf701f75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf74c00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf70070520 0x7fbf701f75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf74c00640 1 -- 172.21.15.16:0/279479797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf700b2bf0 con 0x7fbf7015bc70 2024-09-10T21:19:17.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf6fe00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf70070520 0x7fbf701f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf74c00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf7015bc70 0x7fbf70159a50 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800c3c0 tx=0x7fbf6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf6cc00640 1 -- 172.21.15.16:0/279479797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68011790 con 0x7fbf7015bc70 2024-09-10T21:19:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf6cc00640 1 -- 172.21.15.16:0/279479797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf68017070 con 0x7fbf7015bc70 2024-09-10T21:19:17.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf6cc00640 1 -- 172.21.15.16:0/279479797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68011cc0 con 0x7fbf7015bc70 2024-09-10T21:19:17.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf76c00640 1 -- 172.21.15.16:0/279479797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf700c76e0 con 0x7fbf7015bc70 2024-09-10T21:19:17.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf76c00640 1 -- 172.21.15.16:0/279479797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf700c9450 con 0x7fbf7015bc70 2024-09-10T21:19:17.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.963+0000 7fbf76c00640 1 -- 172.21.15.16:0/279479797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf70072d40 con 0x7fbf7015bc70 2024-09-10T21:19:17.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.967+0000 7fbf6cc00640 1 -- 172.21.15.16:0/279479797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbf6801c460 con 0x7fbf7015bc70 2024-09-10T21:19:17.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.967+0000 7fbf6cc00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbf4c0725e0 0x7fbf4c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:17.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.967+0000 7fbf6cc00640 1 -- 172.21.15.16:0/279479797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7fbf68092e80 con 0x7fbf7015bc70 2024-09-10T21:19:17.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.967+0000 7fbf6fe00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbf4c0725e0 0x7fbf4c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:17.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.967+0000 7fbf6fe00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbf4c0725e0 0x7fbf4c074aa0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbf600af000 tx=0x7fbf6001b570 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:17.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:17.971+0000 7fbf6cc00640 1 -- 172.21.15.16:0/279479797 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbf700c76e0 con 0x7fbf7015bc70 2024-09-10T21:19:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf76c00640 1 -- 172.21.15.16:0/279479797 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf7015d600 con 0x7fbf4c0725e0 2024-09-10T21:19:18.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf6cc00640 1 -- 172.21.15.16:0/279479797 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13466 (secure 0 0 0) 0x7fbf7015d600 con 0x7fbf4c0725e0 2024-09-10T21:19:18.148 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:18.149 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:19:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf76c00640 1 -- 172.21.15.16:0/279479797 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbf4c0725e0 msgr2=0x7fbf4c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf76c00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbf4c0725e0 0x7fbf4c074aa0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbf600af000 tx=0x7fbf6001b570 comp rx=0 tx=0).stop 2024-09-10T21:19:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf76c00640 1 -- 172.21.15.16:0/279479797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf7015bc70 msgr2=0x7fbf70159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf76c00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf7015bc70 0x7fbf70159a50 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800c3c0 tx=0x7fbf6800c880 comp rx=0 tx=0).stop 2024-09-10T21:19:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf76c00640 1 -- 172.21.15.16:0/279479797 shutdown_connections 2024-09-10T21:19:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf76c00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fbf4c0725e0 0x7fbf4c074aa0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf76c00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf7015bc70 0x7fbf70159a50 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf76c00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fbf70154f90 0x7fbf700c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf76c00640 1 --2- 172.21.15.16:0/279479797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf70070520 0x7fbf701f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf76c00640 1 -- 172.21.15.16:0/279479797 >> 172.21.15.16:0/279479797 conn(0x7fbf700cfb20 msgr2=0x7fbf7006eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf76c00640 1 -- 172.21.15.16:0/279479797 shutdown_connections 2024-09-10T21:19:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.143+0000 7fbf76c00640 1 -- 172.21.15.16:0/279479797 wait complete. 2024-09-10T21:19:18.170 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-10T21:19:17.579420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":297760,"kb_used_data":14964,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754676960,"statfs":{"total":773094113280,"available":772789207040,"internally_reserved":0,"allocated":15323136,"data_stored":7282765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4056,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-48,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001131"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":325,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T21:19:12.966651+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:19:12.966651+0000","last_peered":"2024-09-10T21:19:12.966651+0000","last_clean":"2024-09-10T21:19:12.966651+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:19:12.966651+0000","last_undegraded":"2024-09-10T21:19:12.966651+0000","last_fullsized":"2024-09-10T21:19:12.966651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509550,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37076,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509552,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37656,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334184,"statfs":{"total":96636764160,"available":96598204416,"internally_reserved":0,"allocated":2363392,"data_stored":1354936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509550,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509549,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32920,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338920,"statfs":{"total":96636764160,"available":96603054080,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509550,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542254,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41208,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330632,"statfs":{"total":96636764160,"available":96594567168,"internally_reserved":0,"allocated":1740800,"data_stored":752504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542253,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37664,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334176,"statfs":{"total":96636764160,"available":96598196224,"internally_reserved":0,"allocated":2371584,"data_stored":1352485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:19:18.171 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:19:18.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:19:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.311+0000 7f37a5600640 1 -- 172.21.15.16:0/3371421862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a00ce110 msgr2=0x7f37a00cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.311+0000 7f37a5600640 1 --2- 172.21.15.16:0/3371421862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a00ce110 0x7f37a00cf110 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f37a00b6c70 tx=0x7f378801c830 comp rx=0 tx=0).stop 2024-09-10T21:19:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.311+0000 7f37a5600640 1 -- 172.21.15.16:0/3371421862 shutdown_connections 2024-09-10T21:19:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.311+0000 7f37a5600640 1 --2- 172.21.15.16:0/3371421862 >> v2:172.21.15.52:3300/0 conn(0x7f37a00cf650 0x7f37a006cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.311+0000 7f37a5600640 1 --2- 172.21.15.16:0/3371421862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a00ce110 0x7f37a00cf110 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.311+0000 7f37a5600640 1 -- 172.21.15.16:0/3371421862 >> 172.21.15.16:0/3371421862 conn(0x7f37a00cd5f0 msgr2=0x7f37a00cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.311+0000 7f37a5600640 1 -- 172.21.15.16:0/3371421862 shutdown_connections 2024-09-10T21:19:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.311+0000 7f37a5600640 1 -- 172.21.15.16:0/3371421862 wait complete. 2024-09-10T21:19:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a5600640 1 Processor -- start 2024-09-10T21:19:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a5600640 1 -- start start 2024-09-10T21:19:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a5600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37a00ce110 0x7f37a00c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a5600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37a00cf650 0x7f37a006f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a5600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015f200 0x7f37a0157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a5600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37a0168da0 con 0x7f37a015f200 2024-09-10T21:19:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a5600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f37a0168c20 con 0x7f37a00cf650 2024-09-10T21:19:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a5600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f37a0168f20 con 0x7f37a00ce110 2024-09-10T21:19:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015f200 0x7f37a0157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015f200 0x7f37a0157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55012/0 (socket says 172.21.15.16:55012) 2024-09-10T21:19:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a4c00640 1 -- 172.21.15.16:0/2678011493 learned_addr learned my addr 172.21.15.16:0/2678011493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f379fe00640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37a00ce110 0x7f37a00c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f3797400640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37a00cf650 0x7f37a006f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a4c00640 1 -- 172.21.15.16:0/2678011493 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37a00ce110 msgr2=0x7f37a00c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a4c00640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37a00ce110 0x7f37a00c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a4c00640 1 -- 172.21.15.16:0/2678011493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37a00cf650 msgr2=0x7f37a006f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a4c00640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37a00cf650 0x7f37a006f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a4c00640 1 -- 172.21.15.16:0/2678011493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a01fb300 con 0x7f37a015f200 2024-09-10T21:19:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f3797400640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37a00cf650 0x7f37a006f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a4c00640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015f200 0x7f37a0157920 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f379800eb20 tx=0x7f379800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f379cc00640 1 -- 172.21.15.16:0/2678011493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798004b30 con 0x7f37a015f200 2024-09-10T21:19:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f379cc00640 1 -- 172.21.15.16:0/2678011493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3798004cf0 con 0x7f37a015f200 2024-09-10T21:19:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f379cc00640 1 -- 172.21.15.16:0/2678011493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798005280 con 0x7f37a015f200 2024-09-10T21:19:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a5600640 1 -- 172.21.15.16:0/2678011493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a01fb5e0 con 0x7f37a015f200 2024-09-10T21:19:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a5600640 1 -- 172.21.15.16:0/2678011493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a01fbba0 con 0x7f37a015f200 2024-09-10T21:19:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.315+0000 7f37a5600640 1 -- 172.21.15.16:0/2678011493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a00cf110 con 0x7f37a015f200 2024-09-10T21:19:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.319+0000 7f379cc00640 1 -- 172.21.15.16:0/2678011493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3798005420 con 0x7f37a015f200 2024-09-10T21:19:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.319+0000 7f379cc00640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f377c0725e0 0x7f377c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.319+0000 7f379cc00640 1 -- 172.21.15.16:0/2678011493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f3798093cc0 con 0x7f37a015f200 2024-09-10T21:19:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.319+0000 7f379fe00640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f377c0725e0 0x7f377c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.319+0000 7f379fe00640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f377c0725e0 0x7f377c074aa0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f37a00cb590 tx=0x7f3790009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.323+0000 7f379cc00640 1 -- 172.21.15.16:0/2678011493 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f379809a080 con 0x7f37a015f200 2024-09-10T21:19:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f37a5600640 1 -- 172.21.15.16:0/2678011493 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37a0050f80 con 0x7f377c0725e0 2024-09-10T21:19:18.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f379cc00640 1 -- 172.21.15.16:0/2678011493 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13466 (secure 0 0 0) 0x7f37a0050f80 con 0x7f377c0725e0 2024-09-10T21:19:18.500 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:18.501 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:19:18.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f37a5600640 1 -- 172.21.15.16:0/2678011493 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f377c0725e0 msgr2=0x7f377c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:18.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f37a5600640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f377c0725e0 0x7f377c074aa0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f37a00cb590 tx=0x7f3790009040 comp rx=0 tx=0).stop 2024-09-10T21:19:18.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f37a5600640 1 -- 172.21.15.16:0/2678011493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015f200 msgr2=0x7f37a0157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f37a5600640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015f200 0x7f37a0157920 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f379800eb20 tx=0x7f379800c2d0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f37a5600640 1 -- 172.21.15.16:0/2678011493 shutdown_connections 2024-09-10T21:19:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f37a5600640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f377c0725e0 0x7f377c074aa0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f37a5600640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a015f200 0x7f37a0157920 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f37a5600640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37a00cf650 0x7f37a006f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f37a5600640 1 --2- 172.21.15.16:0/2678011493 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f37a00ce110 0x7f37a00c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f37a5600640 1 -- 172.21.15.16:0/2678011493 >> 172.21.15.16:0/2678011493 conn(0x7f37a00cd5f0 msgr2=0x7f37a00ca3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f37a5600640 1 -- 172.21.15.16:0/2678011493 shutdown_connections 2024-09-10T21:19:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.495+0000 7f37a5600640 1 -- 172.21.15.16:0/2678011493 wait complete. 2024-09-10T21:19:18.521 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-10T21:19:17.579420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":297760,"kb_used_data":14964,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754676960,"statfs":{"total":773094113280,"available":772789207040,"internally_reserved":0,"allocated":15323136,"data_stored":7282765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4056,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-48,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001131"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":325,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-10T21:19:12.966651+0000","last_change":"2024-09-10T21:10:19.082442+0000","last_active":"2024-09-10T21:19:12.966651+0000","last_peered":"2024-09-10T21:19:12.966651+0000","last_clean":"2024-09-10T21:19:12.966651+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:19:12.966651+0000","last_undegraded":"2024-09-10T21:19:12.966651+0000","last_fullsized":"2024-09-10T21:19:12.966651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:10:18.064016+0000","last_clean_scrub_stamp":"2024-09-10T21:10:18.064016+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:56.377677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509550,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37076,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509552,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37656,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334184,"statfs":{"total":96636764160,"available":96598204416,"internally_reserved":0,"allocated":2363392,"data_stored":1354936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509550,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509549,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32920,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338920,"statfs":{"total":96636764160,"available":96603054080,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509550,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1769472,"data_stored":764568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542254,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41208,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330632,"statfs":{"total":96636764160,"available":96594567168,"internally_reserved":0,"allocated":1740800,"data_stored":752504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542253,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37664,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334176,"statfs":{"total":96636764160,"available":96598196224,"internally_reserved":0,"allocated":2371584,"data_stored":1352485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:19:18.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:19:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.663+0000 7efc77e00640 1 -- 172.21.15.16:0/2829735236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc780d2820 msgr2=0x7efc780b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.663+0000 7efc77e00640 1 --2- 172.21.15.16:0/2829735236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc780d2820 0x7efc780b1600 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7efc78069920 tx=0x7efc6401c830 comp rx=0 tx=0).stop 2024-09-10T21:19:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.663+0000 7efc77e00640 1 -- 172.21.15.16:0/2829735236 shutdown_connections 2024-09-10T21:19:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.663+0000 7efc77e00640 1 --2- 172.21.15.16:0/2829735236 >> v2:172.21.15.52:3300/0 conn(0x7efc780b1b40 0x7efc780b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.663+0000 7efc77e00640 1 --2- 172.21.15.16:0/2829735236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc780d2820 0x7efc780b1600 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.663+0000 7efc77e00640 1 -- 172.21.15.16:0/2829735236 >> 172.21.15.16:0/2829735236 conn(0x7efc780d1d00 msgr2=0x7efc780d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.663+0000 7efc77e00640 1 -- 172.21.15.16:0/2829735236 shutdown_connections 2024-09-10T21:19:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.663+0000 7efc77e00640 1 -- 172.21.15.16:0/2829735236 wait complete. 2024-09-10T21:19:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.663+0000 7efc77e00640 1 Processor -- start 2024-09-10T21:19:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.663+0000 7efc77e00640 1 -- start start 2024-09-10T21:19:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc77e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc780b1b40 0x7efc7806f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc77e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc780cfd40 0x7efc781fad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc77e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efc780d2820 0x7efc781fb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc77e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc780b0580 con 0x7efc780b1b40 2024-09-10T21:19:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc77e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efc780b0400 con 0x7efc780cfd40 2024-09-10T21:19:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc77e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7efc780b0700 con 0x7efc780d2820 2024-09-10T21:19:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc76a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc780b1b40 0x7efc7806f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc76a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc780b1b40 0x7efc7806f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55032/0 (socket says 172.21.15.16:55032) 2024-09-10T21:19:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc76a00640 1 -- 172.21.15.16:0/886395026 learned_addr learned my addr 172.21.15.16:0/886395026 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc76000640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc780cfd40 0x7efc781fad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc77400640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efc780d2820 0x7efc781fb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc76a00640 1 -- 172.21.15.16:0/886395026 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efc780d2820 msgr2=0x7efc781fb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc76a00640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efc780d2820 0x7efc781fb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc76a00640 1 -- 172.21.15.16:0/886395026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc780cfd40 msgr2=0x7efc781fad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc76a00640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc780cfd40 0x7efc781fad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc76a00640 1 -- 172.21.15.16:0/886395026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc780c8ad0 con 0x7efc780b1b40 2024-09-10T21:19:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc76a00640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc780b1b40 0x7efc7806f700 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7efc6800c450 tx=0x7efc6800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:18.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc6ea00640 1 -- 172.21.15.16:0/886395026 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68011790 con 0x7efc780b1b40 2024-09-10T21:19:18.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc6ea00640 1 -- 172.21.15.16:0/886395026 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc68017070 con 0x7efc780b1b40 2024-09-10T21:19:18.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc6ea00640 1 -- 172.21.15.16:0/886395026 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68011ce0 con 0x7efc780b1b40 2024-09-10T21:19:18.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc77e00640 1 -- 172.21.15.16:0/886395026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc780cbdd0 con 0x7efc780b1b40 2024-09-10T21:19:18.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc77e00640 1 -- 172.21.15.16:0/886395026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7815b170 con 0x7efc780b1b40 2024-09-10T21:19:18.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc77e00640 1 -- 172.21.15.16:0/886395026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc780b1fb0 con 0x7efc780b1b40 2024-09-10T21:19:18.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.667+0000 7efc6ea00640 1 -- 172.21.15.16:0/886395026 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efc6801c460 con 0x7efc780b1b40 2024-09-10T21:19:18.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.671+0000 7efc6ea00640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efc580725e0 0x7efc58074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:18.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.671+0000 7efc6ea00640 1 -- 172.21.15.16:0/886395026 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7efc68092d60 con 0x7efc780b1b40 2024-09-10T21:19:18.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.671+0000 7efc76000640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efc580725e0 0x7efc58074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:18.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.671+0000 7efc76000640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efc580725e0 0x7efc58074aa0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efc640af000 tx=0x7efc640a7040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:18.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.675+0000 7efc6ea00640 1 -- 172.21.15.16:0/886395026 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efc780c8ad0 con 0x7efc780b1b40 2024-09-10T21:19:18.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc77e00640 1 -- 172.21.15.16:0/886395026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc780b1920 con 0x7efc780b1b40 2024-09-10T21:19:18.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc6ea00640 1 -- 172.21.15.16:0/886395026 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+12994 (secure 0 0 0) 0x7efc6805e730 con 0x7efc780b1b40 2024-09-10T21:19:18.856 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:18.856 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":142,"fsid":"eb19be60-6d8f-4816-b778-3d7d6698304d","created":"2024-09-10T21:10:03.679589+0000","modified":"2024-09-10T21:19:13.963630+0000","last_up_change":"2024-09-10T21:10:18.064016+0000","last_in_change":"2024-09-10T21:10:08.139455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:10:17.493770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"30a917bd-7358-4830-b4fc-946e2f5ff128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6811","nonce":1824652127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6813","nonce":1824652127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6817","nonce":1824652127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1824652127},{"type":"v1","addr":"172.21.15.16:6815","nonce":1824652127}]},"public_addr":"172.21.15.16:6811/1824652127","cluster_addr":"172.21.15.16:6813/1824652127","heartbeat_back_addr":"172.21.15.16:6817/1824652127","heartbeat_front_addr":"172.21.15.16:6815/1824652127","state":["exists","up"]},{"osd":1,"uuid":"d0eef88e-6006-46df-a6d6-59a745737c54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6803","nonce":4097098606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6805","nonce":4097098606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6809","nonce":4097098606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4097098606},{"type":"v1","addr":"172.21.15.16:6807","nonce":4097098606}]},"public_addr":"172.21.15.16:6803/4097098606","cluster_addr":"172.21.15.16:6805/4097098606","heartbeat_back_addr":"172.21.15.16:6809/4097098606","heartbeat_front_addr":"172.21.15.16:6807/4097098606","state":["exists","up"]},{"osd":2,"uuid":"81d386aa-827c-4249-9805-b7e752a7918f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6827","nonce":3506045643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6829","nonce":3506045643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6833","nonce":3506045643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3506045643},{"type":"v1","addr":"172.21.15.16:6831","nonce":3506045643}]},"public_addr":"172.21.15.16:6827/3506045643","cluster_addr":"172.21.15.16:6829/3506045643","heartbeat_back_addr":"172.21.15.16:6833/3506045643","heartbeat_front_addr":"172.21.15.16:6831/3506045643","state":["exists","up"]},{"osd":3,"uuid":"eaaf4ff6-2335-43b1-9d7d-34360073abe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6819","nonce":2694071396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6821","nonce":2694071396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6825","nonce":2694071396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2694071396},{"type":"v1","addr":"172.21.15.16:6823","nonce":2694071396}]},"public_addr":"172.21.15.16:6819/2694071396","cluster_addr":"172.21.15.16:6821/2694071396","heartbeat_back_addr":"172.21.15.16:6825/2694071396","heartbeat_front_addr":"172.21.15.16:6823/2694071396","state":["exists","up"]},{"osd":4,"uuid":"cb466bfa-3956-40f8-9a8f-89f479318524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6825","nonce":447485120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6827","nonce":447485120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6831","nonce":447485120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":447485120},{"type":"v1","addr":"172.21.15.52:6829","nonce":447485120}]},"public_addr":"172.21.15.52:6825/447485120","cluster_addr":"172.21.15.52:6827/447485120","heartbeat_back_addr":"172.21.15.52:6831/447485120","heartbeat_front_addr":"172.21.15.52:6829/447485120","state":["exists","up"]},{"osd":5,"uuid":"c47d43ea-abe2-4fe8-af77-699b862309b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6817","nonce":1432062268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6819","nonce":1432062268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6823","nonce":1432062268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1432062268},{"type":"v1","addr":"172.21.15.52:6821","nonce":1432062268}]},"public_addr":"172.21.15.52:6817/1432062268","cluster_addr":"172.21.15.52:6819/1432062268","heartbeat_back_addr":"172.21.15.52:6823/1432062268","heartbeat_front_addr":"172.21.15.52:6821/1432062268","state":["exists","up"]},{"osd":6,"uuid":"069eefff-abdc-4750-86b5-f836d3d6c23b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640674683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6803","nonce":2640674683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6807","nonce":2640674683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2640674683},{"type":"v1","addr":"172.21.15.52:6805","nonce":2640674683}]},"public_addr":"172.21.15.52:6801/2640674683","cluster_addr":"172.21.15.52:6803/2640674683","heartbeat_back_addr":"172.21.15.52:6807/2640674683","heartbeat_front_addr":"172.21.15.52:6805/2640674683","state":["exists","up"]},{"osd":7,"uuid":"e08188d3-3e07-400b-b9ee-21369f465792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6809","nonce":480240003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6811","nonce":480240003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6815","nonce":480240003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":480240003},{"type":"v1","addr":"172.21.15.52:6813","nonce":480240003}]},"public_addr":"172.21.15.52:6809/480240003","cluster_addr":"172.21.15.52:6811/480240003","heartbeat_back_addr":"172.21.15.52:6815/480240003","heartbeat_front_addr":"172.21.15.52:6813/480240003","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.330936+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.943981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.325893+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.261171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.514891+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.127791+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:15.121037+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:10:14.975975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6834/1948696903":"2024-09-11T21:19:10.953909+0000","172.21.15.52:6832/1948696903":"2024-09-11T21:19:10.953909+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:19:18.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc77e00640 1 -- 172.21.15.16:0/886395026 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efc580725e0 msgr2=0x7efc58074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:18.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc77e00640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efc580725e0 0x7efc58074aa0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efc640af000 tx=0x7efc640a7040 comp rx=0 tx=0).stop 2024-09-10T21:19:18.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc77e00640 1 -- 172.21.15.16:0/886395026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc780b1b40 msgr2=0x7efc7806f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:18.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc77e00640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc780b1b40 0x7efc7806f700 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7efc6800c450 tx=0x7efc6800c910 comp rx=0 tx=0).stop 2024-09-10T21:19:18.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc77e00640 1 -- 172.21.15.16:0/886395026 shutdown_connections 2024-09-10T21:19:18.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc77e00640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7efc580725e0 0x7efc58074aa0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc77e00640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7efc780d2820 0x7efc781fb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc77e00640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc780cfd40 0x7efc781fad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc77e00640 1 --2- 172.21.15.16:0/886395026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc780b1b40 0x7efc7806f700 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:18.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc77e00640 1 -- 172.21.15.16:0/886395026 >> 172.21.15.16:0/886395026 conn(0x7efc780d1d00 msgr2=0x7efc7815d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:18.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc77e00640 1 -- 172.21.15.16:0/886395026 shutdown_connections 2024-09-10T21:19:18.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:18.851+0000 7efc77e00640 1 -- 172.21.15.16:0/886395026 wait complete. 2024-09-10T21:19:19.877 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T21:19:19.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:19:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.035+0000 7f2d23200640 1 -- 172.21.15.16:0/1998813108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d240ce9f0 msgr2=0x7f2d240cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.035+0000 7f2d23200640 1 --2- 172.21.15.16:0/1998813108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d240ce9f0 0x7f2d240cfe50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009bc0 tx=0x7f2d1c01cba0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.035+0000 7f2d23200640 1 -- 172.21.15.16:0/1998813108 shutdown_connections 2024-09-10T21:19:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.035+0000 7f2d23200640 1 --2- 172.21.15.16:0/1998813108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d240ce9f0 0x7f2d240cfe50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.035+0000 7f2d23200640 1 --2- 172.21.15.16:0/1998813108 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d240b2a10 0x7f2d240d2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.035+0000 7f2d23200640 1 -- 172.21.15.16:0/1998813108 >> 172.21.15.16:0/1998813108 conn(0x7f2d240d1b70 msgr2=0x7f2d240d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.035+0000 7f2d23200640 1 -- 172.21.15.16:0/1998813108 shutdown_connections 2024-09-10T21:19:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.035+0000 7f2d23200640 1 -- 172.21.15.16:0/1998813108 wait complete. 2024-09-10T21:19:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.035+0000 7f2d23200640 1 Processor -- start 2024-09-10T21:19:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.035+0000 7f2d23200640 1 -- start start 2024-09-10T21:19:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.035+0000 7f2d23200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d240b2a10 0x7f2d241656e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.035+0000 7f2d23200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d240ce610 0x7f2d2415a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d23200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d240ce9f0 0x7f2d2406e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d23200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d240b2460 con 0x7f2d240b2a10 2024-09-10T21:19:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d23200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2d240b22e0 con 0x7f2d240ce9f0 2024-09-10T21:19:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d23200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f2d240b25e0 con 0x7f2d240ce610 2024-09-10T21:19:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d21e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d240b2a10 0x7f2d241656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d21e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d240b2a10 0x7f2d241656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55034/0 (socket says 172.21.15.16:55034) 2024-09-10T21:19:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d21e00640 1 -- 172.21.15.16:0/4079190808 learned_addr learned my addr 172.21.15.16:0/4079190808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d21400640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d240ce610 0x7f2d2415a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d22800640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d240ce9f0 0x7f2d2406e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d21e00640 1 -- 172.21.15.16:0/4079190808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d240ce610 msgr2=0x7f2d2415a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d21e00640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d240ce610 0x7f2d2415a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d21e00640 1 -- 172.21.15.16:0/4079190808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d240ce9f0 msgr2=0x7f2d2406e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d21e00640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d240ce9f0 0x7f2d2406e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d21e00640 1 -- 172.21.15.16:0/4079190808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d24165da0 con 0x7f2d240b2a10 2024-09-10T21:19:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d22800640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d240ce9f0 0x7f2d2406e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d21400640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f2d240ce610 0x7f2d2415a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d21e00640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d240b2a10 0x7f2d241656e0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c4c0 tx=0x7f2d1400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d1a000640 1 -- 172.21.15.16:0/4079190808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140052d0 con 0x7f2d240b2a10 2024-09-10T21:19:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d1a000640 1 -- 172.21.15.16:0/4079190808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d14005490 con 0x7f2d240b2a10 2024-09-10T21:19:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d1a000640 1 -- 172.21.15.16:0/4079190808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14012870 con 0x7f2d240b2a10 2024-09-10T21:19:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d23200640 1 -- 172.21.15.16:0/4079190808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d24166080 con 0x7f2d240b2a10 2024-09-10T21:19:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d23200640 1 -- 172.21.15.16:0/4079190808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d24166360 con 0x7f2d240b2a10 2024-09-10T21:19:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d23200640 1 -- 172.21.15.16:0/4079190808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d240d05a0 con 0x7f2d240b2a10 2024-09-10T21:19:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.039+0000 7f2d1a000640 1 -- 172.21.15.16:0/4079190808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2d14012a10 con 0x7f2d240b2a10 2024-09-10T21:19:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.043+0000 7f2d1a000640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d040725e0 0x7f2d04074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.043+0000 7f2d1a000640 1 -- 172.21.15.16:0/4079190808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f2d140935c0 con 0x7f2d240b2a10 2024-09-10T21:19:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.043+0000 7f2d21400640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d040725e0 0x7f2d04074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.043+0000 7f2d1a000640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] conn(0x7f2d040784a0 0x7f2d0407a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.043+0000 7f2d1a000640 1 -- 172.21.15.16:0/4079190808 --> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d14008e50 con 0x7f2d040784a0 2024-09-10T21:19:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.043+0000 7f2d1a000640 1 -- 172.21.15.16:0/4079190808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7f2d14093970 con 0x7f2d240b2a10 2024-09-10T21:19:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.043+0000 7f2d22800640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] conn(0x7f2d040784a0 0x7f2d0407a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.043+0000 7f2d21400640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d040725e0 0x7f2d04074aa0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2d241fb9b0 tx=0x7f2d10009260 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.043+0000 7f2d22800640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] conn(0x7f2d040784a0 0x7f2d0407a8c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.043+0000 7f2d1a000640 1 -- 172.21.15.16:0/4079190808 <== osd.0 v2:172.21.15.16:6810/1824652127 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d14008e50 con 0x7f2d040784a0 2024-09-10T21:19:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.055+0000 7f2d23200640 1 -- 172.21.15.16:0/4079190808 --> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2d2415d510 con 0x7f2d040784a0 2024-09-10T21:19:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.055+0000 7f2d1a000640 1 -- 172.21.15.16:0/4079190808 <== osd.0 v2:172.21.15.16:6810/1824652127 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2d2415d510 con 0x7f2d040784a0 2024-09-10T21:19:20.063 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:19:20.063 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:19:20.063 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:19:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.059+0000 7f2d23200640 1 -- 172.21.15.16:0/4079190808 >> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] conn(0x7f2d040784a0 msgr2=0x7f2d0407a8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.059+0000 7f2d23200640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.16:6810/1824652127,v1:172.21.15.16:6811/1824652127] conn(0x7f2d040784a0 0x7f2d0407a8c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.059+0000 7f2d23200640 1 -- 172.21.15.16:0/4079190808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d040725e0 msgr2=0x7f2d04074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.059+0000 7f2d23200640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f2d040725e0 0x7f2d04074aa0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2d241fb9b0 tx=0x7f2d10009260 comp rx=0 tx=0).stop 2024-09-10T21:19:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.059+0000 7f2d23200640 1 -- 172.21.15.16:0/4079190808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d240b2a10 msgr2=0x7f2d241656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.059+0000 7f2d23200640 1 --2- 172.21.15.16:0/4079190808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d240b2a10 0x7f2d241656e0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c4c0 tx=0x7f2d1400c980 comp rx=0 tx=0).stop 2024-09-10T21:19:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.059+0000 7f2d22800640 1 -- 172.21.15.16:0/4079190808 reap_dead start 2024-09-10T21:19:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.059+0000 7f2d23200640 1 -- 172.21.15.16:0/4079190808 shutdown_connections 2024-09-10T21:19:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.059+0000 7f2d23200640 1 -- 172.21.15.16:0/4079190808 >> 172.21.15.16:0/4079190808 conn(0x7f2d240d1b70 msgr2=0x7f2d240af190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.059+0000 7f2d23200640 1 -- 172.21.15.16:0/4079190808 shutdown_connections 2024-09-10T21:19:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.059+0000 7f2d23200640 1 -- 172.21.15.16:0/4079190808 wait complete. 2024-09-10T21:19:20.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T21:19:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.223+0000 7feeee800640 1 -- 172.21.15.16:0/3928117686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee8155000 msgr2=0x7feee815bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.223+0000 7feeee800640 1 --2- 172.21.15.16:0/3928117686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee8155000 0x7feee815bbb0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7feed8009840 tx=0x7feed801c7b0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.223+0000 7feeee800640 1 -- 172.21.15.16:0/3928117686 shutdown_connections 2024-09-10T21:19:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.223+0000 7feeee800640 1 --2- 172.21.15.16:0/3928117686 >> v2:172.21.15.52:3300/0 conn(0x7feee80706f0 0x7feee80ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.223+0000 7feeee800640 1 --2- 172.21.15.16:0/3928117686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee8155000 0x7feee815bbb0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.223+0000 7feeee800640 1 -- 172.21.15.16:0/3928117686 >> 172.21.15.16:0/3928117686 conn(0x7feee80cfb20 msgr2=0x7feee80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.223+0000 7feeee800640 1 -- 172.21.15.16:0/3928117686 shutdown_connections 2024-09-10T21:19:20.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.223+0000 7feeee800640 1 -- 172.21.15.16:0/3928117686 wait complete. 2024-09-10T21:19:20.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.223+0000 7feeee800640 1 Processor -- start 2024-09-10T21:19:20.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.223+0000 7feeee800640 1 -- start start 2024-09-10T21:19:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.223+0000 7feeee800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80706f0 0x7feee8152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.223+0000 7feeee800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee8072d40 0x7feee80c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feeee800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee8155000 0x7feee81f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feeee800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feee81679a0 con 0x7feee80706f0 2024-09-10T21:19:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feeee800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feee80cb6b0 con 0x7feee8155000 2024-09-10T21:19:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feeee800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7feee8167b20 con 0x7feee8072d40 2024-09-10T21:19:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feee7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80706f0 0x7feee8152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feee7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80706f0 0x7feee8152b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55064/0 (socket says 172.21.15.16:55064) 2024-09-10T21:19:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feee7400640 1 -- 172.21.15.16:0/3710029417 learned_addr learned my addr 172.21.15.16:0/3710029417 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feee6a00640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee8072d40 0x7feee80c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feee7e00640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee8155000 0x7feee81f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feee7400640 1 -- 172.21.15.16:0/3710029417 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee8072d40 msgr2=0x7feee80c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feee7400640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee8072d40 0x7feee80c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feee7400640 1 -- 172.21.15.16:0/3710029417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee8155000 msgr2=0x7feee81f7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feee7400640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee8155000 0x7feee81f7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feee7400640 1 -- 172.21.15.16:0/3710029417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee80c9300 con 0x7feee80706f0 2024-09-10T21:19:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feee7e00640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee8155000 0x7feee81f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:19:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feee7400640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80706f0 0x7feee8152b90 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7feed8009790 tx=0x7feed8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feedf400640 1 -- 172.21.15.16:0/3710029417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed80bb070 con 0x7feee80706f0 2024-09-10T21:19:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feedf400640 1 -- 172.21.15.16:0/3710029417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feed8005030 con 0x7feee80706f0 2024-09-10T21:19:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feedf400640 1 -- 172.21.15.16:0/3710029417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed80b2040 con 0x7feee80706f0 2024-09-10T21:19:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feeee800640 1 -- 172.21.15.16:0/3710029417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee8158040 con 0x7feee80706f0 2024-09-10T21:19:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feeee800640 1 -- 172.21.15.16:0/3710029417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee8158600 con 0x7feee80706f0 2024-09-10T21:19:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feeee800640 1 -- 172.21.15.16:0/3710029417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee80ca4c0 con 0x7feee80706f0 2024-09-10T21:19:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.227+0000 7feedf400640 1 -- 172.21.15.16:0/3710029417 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feed80b5030 con 0x7feee80706f0 2024-09-10T21:19:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.231+0000 7feedf400640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec80725e0 0x7feec8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.231+0000 7feedf400640 1 -- 172.21.15.16:0/3710029417 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7feed812c760 con 0x7feee80706f0 2024-09-10T21:19:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.231+0000 7feee6a00640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec80725e0 0x7feec8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.231+0000 7feee6a00640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec80725e0 0x7feec8074aa0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feee806fa90 tx=0x7feed4009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.235+0000 7feedf400640 1 -- 172.21.15.16:0/3710029417 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feed80163d0 con 0x7feee80706f0 2024-09-10T21:19:20.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feeee800640 1 -- 172.21.15.16:0/3710029417 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7feee80b51e0 con 0x7feec80725e0 2024-09-10T21:19:20.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feedf400640 1 -- 172.21.15.16:0/3710029417 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7feee80b51e0 con 0x7feec80725e0 2024-09-10T21:19:20.416 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T21:19:20.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feeee800640 1 -- 172.21.15.16:0/3710029417 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec80725e0 msgr2=0x7feec8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feeee800640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec80725e0 0x7feec8074aa0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feee806fa90 tx=0x7feed4009040 comp rx=0 tx=0).stop 2024-09-10T21:19:20.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feeee800640 1 -- 172.21.15.16:0/3710029417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80706f0 msgr2=0x7feee8152b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feeee800640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80706f0 0x7feee8152b90 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7feed8009790 tx=0x7feed8004040 comp rx=0 tx=0).stop 2024-09-10T21:19:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feeee800640 1 -- 172.21.15.16:0/3710029417 shutdown_connections 2024-09-10T21:19:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feeee800640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7feec80725e0 0x7feec8074aa0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feeee800640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feee8155000 0x7feee81f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feeee800640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7feee8072d40 0x7feee80c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feeee800640 1 --2- 172.21.15.16:0/3710029417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feee80706f0 0x7feee8152b90 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feeee800640 1 -- 172.21.15.16:0/3710029417 >> 172.21.15.16:0/3710029417 conn(0x7feee80cfb20 msgr2=0x7feee80cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feeee800640 1 -- 172.21.15.16:0/3710029417 shutdown_connections 2024-09-10T21:19:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.411+0000 7feeee800640 1 -- 172.21.15.16:0/3710029417 wait complete. 2024-09-10T21:19:20.438 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T21:19:20.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:19:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 -- 172.21.15.16:0/1195436113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f497c0cda80 msgr2=0x7f497c15e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 --2- 172.21.15.16:0/1195436113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f497c0cda80 0x7f497c15e500 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f4964009840 tx=0x7f496401c760 comp rx=0 tx=0).stop 2024-09-10T21:19:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 -- 172.21.15.16:0/1195436113 shutdown_connections 2024-09-10T21:19:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 --2- 172.21.15.16:0/1195436113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f497c0cda80 0x7f497c15e500 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 --2- 172.21.15.16:0/1195436113 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f497c0b2b10 0x7f497c15ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 -- 172.21.15.16:0/1195436113 >> 172.21.15.16:0/1195436113 conn(0x7f497c06d1e0 msgr2=0x7f497c0b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 -- 172.21.15.16:0/1195436113 shutdown_connections 2024-09-10T21:19:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 -- 172.21.15.16:0/1195436113 wait complete. 2024-09-10T21:19:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 Processor -- start 2024-09-10T21:19:20.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 -- start start 2024-09-10T21:19:20.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f497c073bc0 0x7f497c1548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f497c0b2b10 0x7f497c14c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f497c0cda80 0x7f497c150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f497c15ad20 con 0x7f497c073bc0 2024-09-10T21:19:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f497c15aba0 con 0x7f497c0b2b10 2024-09-10T21:19:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4982800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f497c15aea0 con 0x7f497c0cda80 2024-09-10T21:19:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f497b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f497c073bc0 0x7f497c1548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f497b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f497c073bc0 0x7f497c1548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55074/0 (socket says 172.21.15.16:55074) 2024-09-10T21:19:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f497b400640 1 -- 172.21.15.16:0/3802957174 learned_addr learned my addr 172.21.15.16:0/3802957174 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f497be00640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f497c0cda80 0x7f497c150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f4973e00640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f497c0b2b10 0x7f497c14c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f497b400640 1 -- 172.21.15.16:0/3802957174 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f497c0cda80 msgr2=0x7f497c150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f497b400640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f497c0cda80 0x7f497c150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f497b400640 1 -- 172.21.15.16:0/3802957174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f497c0b2b10 msgr2=0x7f497c14c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f497b400640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f497c0b2b10 0x7f497c14c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.583+0000 7f497b400640 1 -- 172.21.15.16:0/3802957174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c14cc80 con 0x7f497c073bc0 2024-09-10T21:19:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f497b400640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f497c073bc0 0x7f497c1548f0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f496c00c490 tx=0x7f496c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f4973400640 1 -- 172.21.15.16:0/3802957174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c016020 con 0x7f497c073bc0 2024-09-10T21:19:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f4973400640 1 -- 172.21.15.16:0/3802957174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f496c012940 con 0x7f497c073bc0 2024-09-10T21:19:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f4982800640 1 -- 172.21.15.16:0/3802957174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497c163ee0 con 0x7f497c073bc0 2024-09-10T21:19:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f4973400640 1 -- 172.21.15.16:0/3802957174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c012d90 con 0x7f497c073bc0 2024-09-10T21:19:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f4982800640 1 -- 172.21.15.16:0/3802957174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497c153e10 con 0x7f497c073bc0 2024-09-10T21:19:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f4982800640 1 -- 172.21.15.16:0/3802957174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f497c073fa0 con 0x7f497c073bc0 2024-09-10T21:19:20.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f4973400640 1 -- 172.21.15.16:0/3802957174 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f496c025440 con 0x7f497c073bc0 2024-09-10T21:19:20.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f4973400640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f49580725e0 0x7f4958074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f4973400640 1 -- 172.21.15.16:0/3802957174 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f496c0939a0 con 0x7f497c073bc0 2024-09-10T21:19:20.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f4973e00640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f49580725e0 0x7f4958074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f4973400640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] conn(0x7f49580784a0 0x7f495807a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f4973400640 1 -- 172.21.15.16:0/3802957174 --> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f496c010e50 con 0x7f49580784a0 2024-09-10T21:19:20.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.587+0000 7f4973400640 1 -- 172.21.15.16:0/3802957174 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7f496c093d50 con 0x7f497c073bc0 2024-09-10T21:19:20.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.591+0000 7f497be00640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] conn(0x7f49580784a0 0x7f495807a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.591+0000 7f4973e00640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f49580725e0 0x7f4958074aa0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f49640af000 tx=0x7f496401afa0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:20.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.591+0000 7f497be00640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] conn(0x7f49580784a0 0x7f495807a8c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:20.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.591+0000 7f4973400640 1 -- 172.21.15.16:0/3802957174 <== osd.1 v2:172.21.15.16:6802/4097098606 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f496c010e50 con 0x7f49580784a0 2024-09-10T21:19:20.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.603+0000 7f4982800640 1 -- 172.21.15.16:0/3802957174 --> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f497c14d5a0 con 0x7f49580784a0 2024-09-10T21:19:20.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.603+0000 7f4973400640 1 -- 172.21.15.16:0/3802957174 <== osd.1 v2:172.21.15.16:6802/4097098606 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f497c14d5a0 con 0x7f49580784a0 2024-09-10T21:19:20.610 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:19:20.611 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:19:20.611 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:19:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.603+0000 7f4982800640 1 -- 172.21.15.16:0/3802957174 >> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] conn(0x7f49580784a0 msgr2=0x7f495807a8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.603+0000 7f4982800640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.16:6802/4097098606,v1:172.21.15.16:6803/4097098606] conn(0x7f49580784a0 0x7f495807a8c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.603+0000 7f4982800640 1 -- 172.21.15.16:0/3802957174 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f49580725e0 msgr2=0x7f4958074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.603+0000 7f4982800640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f49580725e0 0x7f4958074aa0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f49640af000 tx=0x7f496401afa0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.603+0000 7f4982800640 1 -- 172.21.15.16:0/3802957174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f497c073bc0 msgr2=0x7f497c1548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.603+0000 7f4982800640 1 --2- 172.21.15.16:0/3802957174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f497c073bc0 0x7f497c1548f0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f496c00c490 tx=0x7f496c00c950 comp rx=0 tx=0).stop 2024-09-10T21:19:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.603+0000 7f497be00640 1 -- 172.21.15.16:0/3802957174 reap_dead start 2024-09-10T21:19:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.603+0000 7f4982800640 1 -- 172.21.15.16:0/3802957174 shutdown_connections 2024-09-10T21:19:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.603+0000 7f4982800640 1 -- 172.21.15.16:0/3802957174 >> 172.21.15.16:0/3802957174 conn(0x7f497c06d1e0 msgr2=0x7f497c164250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.603+0000 7f4982800640 1 -- 172.21.15.16:0/3802957174 shutdown_connections 2024-09-10T21:19:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.603+0000 7f4982800640 1 -- 172.21.15.16:0/3802957174 wait complete. 2024-09-10T21:19:20.630 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T21:19:20.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 -- 172.21.15.16:0/1192515323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6340ce450 msgr2=0x7ff6340ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 --2- 172.21.15.16:0/1192515323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6340ce450 0x7ff6340ce8a0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7ff6340cd5b0 tx=0x7ff61c01db60 comp rx=0 tx=0).stop 2024-09-10T21:19:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 -- 172.21.15.16:0/1192515323 shutdown_connections 2024-09-10T21:19:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 --2- 172.21.15.16:0/1192515323 >> v2:172.21.15.52:3300/0 conn(0x7ff6340ca170 0x7ff6340ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 --2- 172.21.15.16:0/1192515323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6340ce450 0x7ff6340ce8a0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 -- 172.21.15.16:0/1192515323 >> 172.21.15.16:0/1192515323 conn(0x7ff6340cfb20 msgr2=0x7ff6340cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 -- 172.21.15.16:0/1192515323 shutdown_connections 2024-09-10T21:19:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 -- 172.21.15.16:0/1192515323 wait complete. 2024-09-10T21:19:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 Processor -- start 2024-09-10T21:19:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 -- start start 2024-09-10T21:19:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6340ca170 0x7ff6340b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6340ce450 0x7ff6341637a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff63415a270 0x7ff6341fba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff634072b80 con 0x7ff6340ca170 2024-09-10T21:19:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff634072a00 con 0x7ff6340ce450 2024-09-10T21:19:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff63a000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7ff634072d00 con 0x7ff63415a270 2024-09-10T21:19:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff633400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6340ca170 0x7ff6340b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff633400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6340ca170 0x7ff6340b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55090/0 (socket says 172.21.15.16:55090) 2024-09-10T21:19:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff633400640 1 -- 172.21.15.16:0/883658439 learned_addr learned my addr 172.21.15.16:0/883658439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff633400640 1 -- 172.21.15.16:0/883658439 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff63415a270 msgr2=0x7ff6341fba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff633400640 1 --2- 172.21.15.16:0/883658439 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff63415a270 0x7ff6341fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff633400640 1 -- 172.21.15.16:0/883658439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6340ce450 msgr2=0x7ff6341637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff633400640 1 --2- 172.21.15.16:0/883658439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6340ce450 0x7ff6341637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff633400640 1 -- 172.21.15.16:0/883658439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6341fbfa0 con 0x7ff6340ca170 2024-09-10T21:19:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.775+0000 7ff633400640 1 --2- 172.21.15.16:0/883658439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6340ca170 0x7ff6340b3de0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c3c0 tx=0x7ff62400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.779+0000 7ff62be00640 1 -- 172.21.15.16:0/883658439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624017070 con 0x7ff6340ca170 2024-09-10T21:19:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.779+0000 7ff62be00640 1 -- 172.21.15.16:0/883658439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6240053c0 con 0x7ff6340ca170 2024-09-10T21:19:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.779+0000 7ff62be00640 1 -- 172.21.15.16:0/883658439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624003860 con 0x7ff6340ca170 2024-09-10T21:19:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.779+0000 7ff63a000640 1 -- 172.21.15.16:0/883658439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6340b44a0 con 0x7ff6340ca170 2024-09-10T21:19:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.779+0000 7ff63a000640 1 -- 172.21.15.16:0/883658439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6341fb260 con 0x7ff6340ca170 2024-09-10T21:19:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.779+0000 7ff63a000640 1 -- 172.21.15.16:0/883658439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6341fb050 con 0x7ff6340ca170 2024-09-10T21:19:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.779+0000 7ff62be00640 1 -- 172.21.15.16:0/883658439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff624003a00 con 0x7ff6340ca170 2024-09-10T21:19:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.779+0000 7ff62be00640 1 --2- 172.21.15.16:0/883658439 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff610072610 0x7ff610074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.779+0000 7ff62be00640 1 -- 172.21.15.16:0/883658439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7ff624092540 con 0x7ff6340ca170 2024-09-10T21:19:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.779+0000 7ff62aa00640 1 --2- 172.21.15.16:0/883658439 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff610072610 0x7ff610074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:20.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.783+0000 7ff62aa00640 1 --2- 172.21.15.16:0/883658439 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff610072610 0x7ff610074ad0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0b0000 tx=0x7ff61c01a900 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.783+0000 7ff62be00640 1 -- 172.21.15.16:0/883658439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff6341fbfa0 con 0x7ff6340ca170 2024-09-10T21:19:20.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff63a000640 1 -- 172.21.15.16:0/883658439 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7ff6340ca590 con 0x7ff610072610 2024-09-10T21:19:20.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff62be00640 1 -- 172.21.15.16:0/883658439 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff6340ca590 con 0x7ff610072610 2024-09-10T21:19:20.964 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T21:19:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff63a000640 1 -- 172.21.15.16:0/883658439 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff610072610 msgr2=0x7ff610074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff63a000640 1 --2- 172.21.15.16:0/883658439 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff610072610 0x7ff610074ad0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0b0000 tx=0x7ff61c01a900 comp rx=0 tx=0).stop 2024-09-10T21:19:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff63a000640 1 -- 172.21.15.16:0/883658439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6340ca170 msgr2=0x7ff6340b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff63a000640 1 --2- 172.21.15.16:0/883658439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6340ca170 0x7ff6340b3de0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c3c0 tx=0x7ff62400c880 comp rx=0 tx=0).stop 2024-09-10T21:19:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff63a000640 1 -- 172.21.15.16:0/883658439 shutdown_connections 2024-09-10T21:19:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff63a000640 1 --2- 172.21.15.16:0/883658439 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7ff610072610 0x7ff610074ad0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff63a000640 1 --2- 172.21.15.16:0/883658439 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7ff63415a270 0x7ff6341fba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff63a000640 1 --2- 172.21.15.16:0/883658439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6340ce450 0x7ff6341637a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff63a000640 1 --2- 172.21.15.16:0/883658439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6340ca170 0x7ff6340b3de0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff63a000640 1 -- 172.21.15.16:0/883658439 >> 172.21.15.16:0/883658439 conn(0x7ff6340cfb20 msgr2=0x7ff634159140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff63a000640 1 -- 172.21.15.16:0/883658439 shutdown_connections 2024-09-10T21:19:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:20.959+0000 7ff63a000640 1 -- 172.21.15.16:0/883658439 wait complete. 2024-09-10T21:19:20.986 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T21:19:20.986 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:19:21.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 -- 172.21.15.16:0/2807161103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115015be60 msgr2=0x7f1150073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 --2- 172.21.15.16:0/2807161103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115015be60 0x7f1150073cd0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f11500ce200 tx=0x7f114001c770 comp rx=0 tx=0).stop 2024-09-10T21:19:21.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 -- 172.21.15.16:0/2807161103 shutdown_connections 2024-09-10T21:19:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 --2- 172.21.15.16:0/2807161103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115015be60 0x7f1150073cd0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 -- 172.21.15.16:0/2807161103 >> 172.21.15.16:0/2807161103 conn(0x7f115006e430 msgr2=0x7f11500b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 -- 172.21.15.16:0/2807161103 shutdown_connections 2024-09-10T21:19:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 -- 172.21.15.16:0/2807161103 wait complete. 2024-09-10T21:19:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 Processor -- start 2024-09-10T21:19:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 -- start start 2024-09-10T21:19:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115006d2f0 0x7f11501493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1150074260 0x7f115014db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f115015be60 0x7f1150156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f115016a3e0 con 0x7f115006d2f0 2024-09-10T21:19:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f115016a260 con 0x7f1150074260 2024-09-10T21:19:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1155600640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f115016a560 con 0x7f115015be60 2024-09-10T21:19:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f114ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115006d2f0 0x7f11501493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f114ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115006d2f0 0x7f11501493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55112/0 (socket says 172.21.15.16:55112) 2024-09-10T21:19:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f114ea00640 1 -- 172.21.15.16:0/2474004088 learned_addr learned my addr 172.21.15.16:0/2474004088 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f1146000640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1150074260 0x7f115014db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.131+0000 7f114f400640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f115015be60 0x7f1150156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f114ea00640 1 -- 172.21.15.16:0/2474004088 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f115015be60 msgr2=0x7f1150156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f114ea00640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f115015be60 0x7f1150156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f114ea00640 1 -- 172.21.15.16:0/2474004088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1150074260 msgr2=0x7f115014db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f114ea00640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1150074260 0x7f115014db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f114ea00640 1 -- 172.21.15.16:0/2474004088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1150155120 con 0x7f115006d2f0 2024-09-10T21:19:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f1146000640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1150074260 0x7f115014db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f114ea00640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115006d2f0 0x7f11501493a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f1140000c10 tx=0x7f11400a6680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f1147400640 1 -- 172.21.15.16:0/2474004088 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140005140 con 0x7f115006d2f0 2024-09-10T21:19:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f1147400640 1 -- 172.21.15.16:0/2474004088 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11400052e0 con 0x7f115006d2f0 2024-09-10T21:19:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f1155600640 1 -- 172.21.15.16:0/2474004088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1150155400 con 0x7f115006d2f0 2024-09-10T21:19:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f1147400640 1 -- 172.21.15.16:0/2474004088 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11400046d0 con 0x7f115006d2f0 2024-09-10T21:19:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f1155600640 1 -- 172.21.15.16:0/2474004088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11501559c0 con 0x7f115006d2f0 2024-09-10T21:19:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f1155600640 1 -- 172.21.15.16:0/2474004088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1150073cd0 con 0x7f115006d2f0 2024-09-10T21:19:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f1147400640 1 -- 172.21.15.16:0/2474004088 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1140004870 con 0x7f115006d2f0 2024-09-10T21:19:21.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f1147400640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f112c0725e0 0x7f112c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.135+0000 7f1147400640 1 -- 172.21.15.16:0/2474004088 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f114012d530 con 0x7f115006d2f0 2024-09-10T21:19:21.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.139+0000 7f1147400640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] conn(0x7f112c0784a0 0x7f112c07a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.139+0000 7f1146000640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f112c0725e0 0x7f112c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.139+0000 7f1147400640 1 -- 172.21.15.16:0/2474004088 --> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11400c1e50 con 0x7f112c0784a0 2024-09-10T21:19:21.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.139+0000 7f1147400640 1 -- 172.21.15.16:0/2474004088 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7f114012d8e0 con 0x7f115006d2f0 2024-09-10T21:19:21.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.139+0000 7f114f400640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] conn(0x7f112c0784a0 0x7f112c07a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.139+0000 7f1146000640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f112c0725e0 0x7f112c074aa0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f11500b1270 tx=0x7f1138009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:21.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.139+0000 7f114f400640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] conn(0x7f112c0784a0 0x7f112c07a8c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:21.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.139+0000 7f1147400640 1 -- 172.21.15.16:0/2474004088 <== osd.2 v2:172.21.15.16:6826/3506045643 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f11400c1e50 con 0x7f112c0784a0 2024-09-10T21:19:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.151+0000 7f1155600640 1 -- 172.21.15.16:0/2474004088 --> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f11500b4390 con 0x7f112c0784a0 2024-09-10T21:19:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.151+0000 7f1147400640 1 -- 172.21.15.16:0/2474004088 <== osd.2 v2:172.21.15.16:6826/3506045643 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f11500b4390 con 0x7f112c0784a0 2024-09-10T21:19:21.158 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:19:21.159 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:19:21.159 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:19:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.151+0000 7f1155600640 1 -- 172.21.15.16:0/2474004088 >> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] conn(0x7f112c0784a0 msgr2=0x7f112c07a8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.151+0000 7f1155600640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.16:6826/3506045643,v1:172.21.15.16:6827/3506045643] conn(0x7f112c0784a0 0x7f112c07a8c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.151+0000 7f1155600640 1 -- 172.21.15.16:0/2474004088 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f112c0725e0 msgr2=0x7f112c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.151+0000 7f1155600640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f112c0725e0 0x7f112c074aa0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f11500b1270 tx=0x7f1138009040 comp rx=0 tx=0).stop 2024-09-10T21:19:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.151+0000 7f1155600640 1 -- 172.21.15.16:0/2474004088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115006d2f0 msgr2=0x7f11501493a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.151+0000 7f1155600640 1 --2- 172.21.15.16:0/2474004088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115006d2f0 0x7f11501493a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f1140000c10 tx=0x7f11400a6680 comp rx=0 tx=0).stop 2024-09-10T21:19:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.151+0000 7f114f400640 1 -- 172.21.15.16:0/2474004088 reap_dead start 2024-09-10T21:19:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.151+0000 7f1155600640 1 -- 172.21.15.16:0/2474004088 shutdown_connections 2024-09-10T21:19:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.151+0000 7f1155600640 1 -- 172.21.15.16:0/2474004088 >> 172.21.15.16:0/2474004088 conn(0x7f115006e430 msgr2=0x7f115006ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.151+0000 7f1155600640 1 -- 172.21.15.16:0/2474004088 shutdown_connections 2024-09-10T21:19:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.151+0000 7f1155600640 1 -- 172.21.15.16:0/2474004088 wait complete. 2024-09-10T21:19:21.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T21:19:21.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.319+0000 7fa4e8800640 1 -- 172.21.15.16:0/1948798240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4e00d03c0 msgr2=0x7fa4e00ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.319+0000 7fa4e8800640 1 --2- 172.21.15.16:0/1948798240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4e00d03c0 0x7fa4e00ce2b0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009840 tx=0x7fa4d001c820 comp rx=0 tx=0).stop 2024-09-10T21:19:21.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.319+0000 7fa4e8800640 1 -- 172.21.15.16:0/1948798240 shutdown_connections 2024-09-10T21:19:21.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.319+0000 7fa4e8800640 1 --2- 172.21.15.16:0/1948798240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4e00d03c0 0x7fa4e00ce2b0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.319+0000 7fa4e8800640 1 --2- 172.21.15.16:0/1948798240 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa4e00b2860 0x7fa4e0160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.319+0000 7fa4e8800640 1 -- 172.21.15.16:0/1948798240 >> 172.21.15.16:0/1948798240 conn(0x7fa4e00d1d80 msgr2=0x7fa4e00d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:21.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.319+0000 7fa4e8800640 1 -- 172.21.15.16:0/1948798240 shutdown_connections 2024-09-10T21:19:21.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.319+0000 7fa4e8800640 1 -- 172.21.15.16:0/1948798240 wait complete. 2024-09-10T21:19:21.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.319+0000 7fa4e8800640 1 Processor -- start 2024-09-10T21:19:21.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.319+0000 7fa4e8800640 1 -- start start 2024-09-10T21:19:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e8800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa4e00b2860 0x7fa4e0160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e8800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e00cffe0 0x7fa4e014fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e8800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4e00d03c0 0x7fa4e01fb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e8800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e00cd910 con 0x7fa4e00d03c0 2024-09-10T21:19:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e8800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e00cd790 con 0x7fa4e00cffe0 2024-09-10T21:19:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e8800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e00cda90 con 0x7fa4e00b2860 2024-09-10T21:19:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4e00d03c0 0x7fa4e01fb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4e00d03c0 0x7fa4e01fb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55126/0 (socket says 172.21.15.16:55126) 2024-09-10T21:19:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e6800640 1 -- 172.21.15.16:0/1587154566 learned_addr learned my addr 172.21.15.16:0/1587154566 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e5e00640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa4e00b2860 0x7fa4e0160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e5400640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e00cffe0 0x7fa4e014fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e6800640 1 -- 172.21.15.16:0/1587154566 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa4e00b2860 msgr2=0x7fa4e0160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e6800640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa4e00b2860 0x7fa4e0160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e6800640 1 -- 172.21.15.16:0/1587154566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e00cffe0 msgr2=0x7fa4e014fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e6800640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e00cffe0 0x7fa4e014fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e6800640 1 -- 172.21.15.16:0/1587154566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e0158fa0 con 0x7fa4e00d03c0 2024-09-10T21:19:21.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e5400640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e00cffe0 0x7fa4e014fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:21.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e6800640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4e00d03c0 0x7fa4e01fb6b0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00ea50 tx=0x7fa4dc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4da000640 1 -- 172.21.15.16:0/1587154566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00cdc0 con 0x7fa4e00d03c0 2024-09-10T21:19:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4da000640 1 -- 172.21.15.16:0/1587154566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4dc004c40 con 0x7fa4e00d03c0 2024-09-10T21:19:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4da000640 1 -- 172.21.15.16:0/1587154566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc005280 con 0x7fa4e00d03c0 2024-09-10T21:19:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e8800640 1 -- 172.21.15.16:0/1587154566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e0159240 con 0x7fa4e00d03c0 2024-09-10T21:19:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e8800640 1 -- 172.21.15.16:0/1587154566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e0160580 con 0x7fa4e00d03c0 2024-09-10T21:19:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4e8800640 1 -- 172.21.15.16:0/1587154566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e00ce2b0 con 0x7fa4e00d03c0 2024-09-10T21:19:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.323+0000 7fa4da000640 1 -- 172.21.15.16:0/1587154566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa4dc00f040 con 0x7fa4e00d03c0 2024-09-10T21:19:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.327+0000 7fa4da000640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa4c4072610 0x7fa4c4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.327+0000 7fa4da000640 1 -- 172.21.15.16:0/1587154566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7fa4dc0935d0 con 0x7fa4e00d03c0 2024-09-10T21:19:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.327+0000 7fa4e5e00640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa4c4072610 0x7fa4c4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.327+0000 7fa4e5e00640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa4c4072610 0x7fa4c4074ad0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4007fd0 tx=0x7fa4d4007ea0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.331+0000 7fa4da000640 1 -- 172.21.15.16:0/1587154566 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa4dc016030 con 0x7fa4e00d03c0 2024-09-10T21:19:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.503+0000 7fa4e8800640 1 -- 172.21.15.16:0/1587154566 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fa4e00cf950 con 0x7fa4c4072610 2024-09-10T21:19:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.503+0000 7fa4da000640 1 -- 172.21.15.16:0/1587154566 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa4e00cf950 con 0x7fa4c4072610 2024-09-10T21:19:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T21:19:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.503+0000 7fa4e8800640 1 -- 172.21.15.16:0/1587154566 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa4c4072610 msgr2=0x7fa4c4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.503+0000 7fa4e8800640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa4c4072610 0x7fa4c4074ad0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4007fd0 tx=0x7fa4d4007ea0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.503+0000 7fa4e8800640 1 -- 172.21.15.16:0/1587154566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4e00d03c0 msgr2=0x7fa4e01fb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.503+0000 7fa4e8800640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4e00d03c0 0x7fa4e01fb6b0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00ea50 tx=0x7fa4dc00ef10 comp rx=0 tx=0).stop 2024-09-10T21:19:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.503+0000 7fa4e8800640 1 -- 172.21.15.16:0/1587154566 shutdown_connections 2024-09-10T21:19:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.503+0000 7fa4e8800640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa4c4072610 0x7fa4c4074ad0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.503+0000 7fa4e8800640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4e00d03c0 0x7fa4e01fb6b0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.503+0000 7fa4e8800640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4e00cffe0 0x7fa4e014fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.503+0000 7fa4e8800640 1 --2- 172.21.15.16:0/1587154566 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa4e00b2860 0x7fa4e0160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.503+0000 7fa4e8800640 1 -- 172.21.15.16:0/1587154566 >> 172.21.15.16:0/1587154566 conn(0x7fa4e00d1d80 msgr2=0x7fa4e015a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.503+0000 7fa4e8800640 1 -- 172.21.15.16:0/1587154566 shutdown_connections 2024-09-10T21:19:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.507+0000 7fa4e8800640 1 -- 172.21.15.16:0/1587154566 wait complete. 2024-09-10T21:19:21.530 INFO:tasks.ceph:Scrubbing osd.3 2024-09-10T21:19:21.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:19:21.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.675+0000 7fb7bf000640 1 -- 172.21.15.16:0/2612890993 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7b80cca20 msgr2=0x7fb7b815b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.675+0000 7fb7bf000640 1 --2- 172.21.15.16:0/2612890993 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7b80cca20 0x7fb7b815b030 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009870 tx=0x7fb7a801c7a0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.675+0000 7fb7bf000640 1 -- 172.21.15.16:0/2612890993 shutdown_connections 2024-09-10T21:19:21.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.675+0000 7fb7bf000640 1 --2- 172.21.15.16:0/2612890993 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7b80cca20 0x7fb7b815b030 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.675+0000 7fb7bf000640 1 -- 172.21.15.16:0/2612890993 >> 172.21.15.16:0/2612890993 conn(0x7fb7b80d0c20 msgr2=0x7fb7b80d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:21.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.675+0000 7fb7bf000640 1 -- 172.21.15.16:0/2612890993 shutdown_connections 2024-09-10T21:19:21.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.675+0000 7fb7bf000640 1 -- 172.21.15.16:0/2612890993 wait complete. 2024-09-10T21:19:21.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bf000640 1 Processor -- start 2024-09-10T21:19:21.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bf000640 1 -- start start 2024-09-10T21:19:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bf000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b80737c0 0x7fb7b80c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bf000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7b80cca20 0x7fb7b814ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bf000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b80cce00 0x7fb7b80b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bf000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b8154ba0 con 0x7fb7b80cce00 2024-09-10T21:19:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bf000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b8154a20 con 0x7fb7b80737c0 2024-09-10T21:19:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bf000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b8154d20 con 0x7fb7b80cca20 2024-09-10T21:19:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b80cce00 0x7fb7b80b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b80cce00 0x7fb7b80b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55128/0 (socket says 172.21.15.16:55128) 2024-09-10T21:19:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bd000640 1 -- 172.21.15.16:0/102629958 learned_addr learned my addr 172.21.15.16:0/102629958 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7b7e00640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b80737c0 0x7fb7b80c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7af400640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7b80cca20 0x7fb7b814ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bd000640 1 -- 172.21.15.16:0/102629958 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7b80cca20 msgr2=0x7fb7b814ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bd000640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fb7b80cca20 0x7fb7b814ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bd000640 1 -- 172.21.15.16:0/102629958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b80737c0 msgr2=0x7fb7b80c3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bd000640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b80737c0 0x7fb7b80c3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bd000640 1 -- 172.21.15.16:0/102629958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b80c3fb0 con 0x7fb7b80cce00 2024-09-10T21:19:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bd000640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b80cce00 0x7fb7b80b3ab0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000cb70 tx=0x7fb7b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7b4c00640 1 -- 172.21.15.16:0/102629958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0016020 con 0x7fb7b80cce00 2024-09-10T21:19:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7b4c00640 1 -- 172.21.15.16:0/102629958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7b000ab60 con 0x7fb7b80cce00 2024-09-10T21:19:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7b4c00640 1 -- 172.21.15.16:0/102629958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0012420 con 0x7fb7b80cce00 2024-09-10T21:19:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bf000640 1 -- 172.21.15.16:0/102629958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b806d190 con 0x7fb7b80cce00 2024-09-10T21:19:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.679+0000 7fb7bf000640 1 -- 172.21.15.16:0/102629958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b814c230 con 0x7fb7b80cce00 2024-09-10T21:19:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.683+0000 7fb7bf000640 1 -- 172.21.15.16:0/102629958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7b8155250 con 0x7fb7b80cce00 2024-09-10T21:19:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.683+0000 7fb7b4c00640 1 -- 172.21.15.16:0/102629958 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb7b00125c0 con 0x7fb7b80cce00 2024-09-10T21:19:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.683+0000 7fb7b4c00640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7940725e0 0x7fb794074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.683+0000 7fb7b4c00640 1 -- 172.21.15.16:0/102629958 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7fb7b0093aa0 con 0x7fb7b80cce00 2024-09-10T21:19:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.683+0000 7fb7b7e00640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7940725e0 0x7fb794074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.683+0000 7fb7b4c00640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] conn(0x7fb7940784a0 0x7fb79407a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.683+0000 7fb7b4c00640 1 -- 172.21.15.16:0/102629958 --> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb79407af70 con 0x7fb7940784a0 2024-09-10T21:19:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.683+0000 7fb7b4c00640 1 -- 172.21.15.16:0/102629958 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7fb7b0093e50 con 0x7fb7b80cce00 2024-09-10T21:19:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.683+0000 7fb7af400640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] conn(0x7fb7940784a0 0x7fb79407a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.683+0000 7fb7b7e00640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7940725e0 0x7fb794074aa0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80af000 tx=0x7fb7a801afe0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.687+0000 7fb7af400640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] conn(0x7fb7940784a0 0x7fb79407a8c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.687+0000 7fb7b4c00640 1 -- 172.21.15.16:0/102629958 <== osd.3 v2:172.21.15.16:6818/2694071396 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb79407af70 con 0x7fb7940784a0 2024-09-10T21:19:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.699+0000 7fb7bf000640 1 -- 172.21.15.16:0/102629958 --> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb7b8074920 con 0x7fb7940784a0 2024-09-10T21:19:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.699+0000 7fb7b4c00640 1 -- 172.21.15.16:0/102629958 <== osd.3 v2:172.21.15.16:6818/2694071396 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb7b8074920 con 0x7fb7940784a0 2024-09-10T21:19:21.705 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:19:21.706 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:19:21.706 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:19:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.699+0000 7fb7bf000640 1 -- 172.21.15.16:0/102629958 >> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] conn(0x7fb7940784a0 msgr2=0x7fb79407a8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.699+0000 7fb7bf000640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.16:6818/2694071396,v1:172.21.15.16:6819/2694071396] conn(0x7fb7940784a0 0x7fb79407a8c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.699+0000 7fb7bf000640 1 -- 172.21.15.16:0/102629958 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7940725e0 msgr2=0x7fb794074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.699+0000 7fb7bf000640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fb7940725e0 0x7fb794074aa0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80af000 tx=0x7fb7a801afe0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.699+0000 7fb7bf000640 1 -- 172.21.15.16:0/102629958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b80cce00 msgr2=0x7fb7b80b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.699+0000 7fb7bf000640 1 --2- 172.21.15.16:0/102629958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b80cce00 0x7fb7b80b3ab0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000cb70 tx=0x7fb7b000cc60 comp rx=0 tx=0).stop 2024-09-10T21:19:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.699+0000 7fb7bd000640 1 -- 172.21.15.16:0/102629958 reap_dead start 2024-09-10T21:19:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.699+0000 7fb7bf000640 1 -- 172.21.15.16:0/102629958 shutdown_connections 2024-09-10T21:19:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.699+0000 7fb7bf000640 1 -- 172.21.15.16:0/102629958 >> 172.21.15.16:0/102629958 conn(0x7fb7b80d0c20 msgr2=0x7fb7b80cb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.699+0000 7fb7bf000640 1 -- 172.21.15.16:0/102629958 shutdown_connections 2024-09-10T21:19:21.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.699+0000 7fb7bf000640 1 -- 172.21.15.16:0/102629958 wait complete. 2024-09-10T21:19:21.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-10T21:19:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.867+0000 7fa967c00640 1 -- 172.21.15.16:0/1568363407 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa96015ac50 msgr2=0x7fa960073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.867+0000 7fa967c00640 1 --2- 172.21.15.16:0/1568363407 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa96015ac50 0x7fa960073750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa954009840 tx=0x7fa95401c790 comp rx=0 tx=0).stop 2024-09-10T21:19:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.867+0000 7fa967c00640 1 -- 172.21.15.16:0/1568363407 shutdown_connections 2024-09-10T21:19:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.867+0000 7fa967c00640 1 --2- 172.21.15.16:0/1568363407 >> v2:172.21.15.52:3300/0 conn(0x7fa9600cff40 0x7fa960073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.867+0000 7fa967c00640 1 --2- 172.21.15.16:0/1568363407 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa96015ac50 0x7fa960073750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.867+0000 7fa967c00640 1 -- 172.21.15.16:0/1568363407 >> 172.21.15.16:0/1568363407 conn(0x7fa9600d0c10 msgr2=0x7fa9600d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.867+0000 7fa967c00640 1 -- 172.21.15.16:0/1568363407 shutdown_connections 2024-09-10T21:19:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.867+0000 7fa967c00640 1 -- 172.21.15.16:0/1568363407 wait complete. 2024-09-10T21:19:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa967c00640 1 Processor -- start 2024-09-10T21:19:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa967c00640 1 -- start start 2024-09-10T21:19:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa967c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9600cff40 0x7fa9601653e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa967c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9600d06e0 0x7fa9600b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa967c00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa96015ac50 0x7fa960156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa967c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa960167fa0 con 0x7fa9600cff40 2024-09-10T21:19:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa967c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa960167e20 con 0x7fa9600d06e0 2024-09-10T21:19:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa967c00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7fa960168120 con 0x7fa96015ac50 2024-09-10T21:19:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa965200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9600cff40 0x7fa9601653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa965200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9600cff40 0x7fa9601653e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55130/0 (socket says 172.21.15.16:55130) 2024-09-10T21:19:21.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa965200640 1 -- 172.21.15.16:0/3997518619 learned_addr learned my addr 172.21.15.16:0/3997518619 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:21.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa965c00640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa96015ac50 0x7fa960156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa95fe00640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9600d06e0 0x7fa9600b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa965200640 1 -- 172.21.15.16:0/3997518619 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa96015ac50 msgr2=0x7fa960156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa965200640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa96015ac50 0x7fa960156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa965200640 1 -- 172.21.15.16:0/3997518619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9600d06e0 msgr2=0x7fa9600b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa965200640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9600d06e0 0x7fa9600b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa965200640 1 -- 172.21.15.16:0/3997518619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9600cada0 con 0x7fa9600cff40 2024-09-10T21:19:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa965200640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9600cff40 0x7fa9601653e0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa954009790 tx=0x7fa9540a9500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa95cc00640 1 -- 172.21.15.16:0/3997518619 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9540ad050 con 0x7fa9600cff40 2024-09-10T21:19:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa95cc00640 1 -- 172.21.15.16:0/3997518619 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9540052c0 con 0x7fa9600cff40 2024-09-10T21:19:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa95cc00640 1 -- 172.21.15.16:0/3997518619 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954005ca0 con 0x7fa9600cff40 2024-09-10T21:19:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa967c00640 1 -- 172.21.15.16:0/3997518619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9601f86f0 con 0x7fa9600cff40 2024-09-10T21:19:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa967c00640 1 -- 172.21.15.16:0/3997518619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9600b1c80 con 0x7fa9600cff40 2024-09-10T21:19:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.871+0000 7fa967c00640 1 -- 172.21.15.16:0/3997518619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9601f84e0 con 0x7fa9600cff40 2024-09-10T21:19:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.875+0000 7fa95cc00640 1 -- 172.21.15.16:0/3997518619 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa9540ba070 con 0x7fa9600cff40 2024-09-10T21:19:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.875+0000 7fa95cc00640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa9400725e0 0x7fa940074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.875+0000 7fa95cc00640 1 -- 172.21.15.16:0/3997518619 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7fa95412d800 con 0x7fa9600cff40 2024-09-10T21:19:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.875+0000 7fa95fe00640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa9400725e0 0x7fa940074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.875+0000 7fa95fe00640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa9400725e0 0x7fa940074aa0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa9600709d0 tx=0x7fa94c001440 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:21.879+0000 7fa95cc00640 1 -- 172.21.15.16:0/3997518619 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa954016410 con 0x7fa9600cff40 2024-09-10T21:19:22.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.051+0000 7fa967c00640 1 -- 172.21.15.16:0/3997518619 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fa9600d18e0 con 0x7fa9400725e0 2024-09-10T21:19:22.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.051+0000 7fa95cc00640 1 -- 172.21.15.16:0/3997518619 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa9600d18e0 con 0x7fa9400725e0 2024-09-10T21:19:22.058 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 3 to deep-scrub 2024-09-10T21:19:22.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.055+0000 7fa967c00640 1 -- 172.21.15.16:0/3997518619 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa9400725e0 msgr2=0x7fa940074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.055+0000 7fa967c00640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa9400725e0 0x7fa940074aa0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa9600709d0 tx=0x7fa94c001440 comp rx=0 tx=0).stop 2024-09-10T21:19:22.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.055+0000 7fa967c00640 1 -- 172.21.15.16:0/3997518619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9600cff40 msgr2=0x7fa9601653e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.055+0000 7fa967c00640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9600cff40 0x7fa9601653e0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa954009790 tx=0x7fa9540a9500 comp rx=0 tx=0).stop 2024-09-10T21:19:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.055+0000 7fa967c00640 1 -- 172.21.15.16:0/3997518619 shutdown_connections 2024-09-10T21:19:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.055+0000 7fa967c00640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fa9400725e0 0x7fa940074aa0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.055+0000 7fa967c00640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7fa96015ac50 0x7fa960156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.055+0000 7fa967c00640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9600d06e0 0x7fa9600b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.055+0000 7fa967c00640 1 --2- 172.21.15.16:0/3997518619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9600cff40 0x7fa9601653e0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.055+0000 7fa967c00640 1 -- 172.21.15.16:0/3997518619 >> 172.21.15.16:0/3997518619 conn(0x7fa9600d0c10 msgr2=0x7fa96006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.055+0000 7fa967c00640 1 -- 172.21.15.16:0/3997518619 shutdown_connections 2024-09-10T21:19:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.055+0000 7fa967c00640 1 -- 172.21.15.16:0/3997518619 wait complete. 2024-09-10T21:19:22.080 INFO:tasks.ceph:Scrubbing osd.4 2024-09-10T21:19:22.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:19:22.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.223+0000 7f8f29e00640 1 -- 172.21.15.16:0/1694432327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f240b24f0 msgr2=0x7f8f240b2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.223+0000 7f8f29e00640 1 --2- 172.21.15.16:0/1694432327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f240b24f0 0x7f8f240b2980 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f8f240698f0 tx=0x7f8f1c01caf0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.223+0000 7f8f29e00640 1 -- 172.21.15.16:0/1694432327 shutdown_connections 2024-09-10T21:19:22.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.223+0000 7f8f29e00640 1 --2- 172.21.15.16:0/1694432327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f240b24f0 0x7f8f240b2980 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.223+0000 7f8f29e00640 1 --2- 172.21.15.16:0/1694432327 >> v2:172.21.15.52:3300/0 conn(0x7f8f240cd480 0x7f8f240ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.223+0000 7f8f29e00640 1 -- 172.21.15.16:0/1694432327 >> 172.21.15.16:0/1694432327 conn(0x7f8f240d1e60 msgr2=0x7f8f240d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:22.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.223+0000 7f8f29e00640 1 -- 172.21.15.16:0/1694432327 shutdown_connections 2024-09-10T21:19:22.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.223+0000 7f8f29e00640 1 -- 172.21.15.16:0/1694432327 wait complete. 2024-09-10T21:19:22.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f29e00640 1 Processor -- start 2024-09-10T21:19:22.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f29e00640 1 -- start start 2024-09-10T21:19:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f29e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f240b24f0 0x7f8f24156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f29e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f240cd480 0x7f8f241535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f29e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f24161770 0x7f8f2406e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f29e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2416b170 con 0x7f8f240cd480 2024-09-10T21:19:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f29e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2416aff0 con 0x7f8f240b24f0 2024-09-10T21:19:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f29e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2416b2f0 con 0x7f8f24161770 2024-09-10T21:19:22.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f1a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f240cd480 0x7f8f241535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f1a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f240cd480 0x7f8f241535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42620/0 (socket says 172.21.15.16:42620) 2024-09-10T21:19:22.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f1a000640 1 -- 172.21.15.16:0/3242680941 learned_addr learned my addr 172.21.15.16:0/3242680941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:22.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f22a00640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f240b24f0 0x7f8f24156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f23400640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f24161770 0x7f8f2406e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f1a000640 1 -- 172.21.15.16:0/3242680941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f24161770 msgr2=0x7f8f2406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f1a000640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f24161770 0x7f8f2406e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f1a000640 1 -- 172.21.15.16:0/3242680941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f240b24f0 msgr2=0x7f8f24156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f1a000640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f240b24f0 0x7f8f24156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f1a000640 1 -- 172.21.15.16:0/3242680941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f24153b00 con 0x7f8f240cd480 2024-09-10T21:19:22.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f22a00640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f240b24f0 0x7f8f24156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:22.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f23400640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f8f24161770 0x7f8f2406e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:22.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f1a000640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f240cd480 0x7f8f241535c0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00d980 tx=0x7f8f0c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:22.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f1be00640 1 -- 172.21.15.16:0/3242680941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c017070 con 0x7f8f240cd480 2024-09-10T21:19:22.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f1be00640 1 -- 172.21.15.16:0/3242680941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f0c009d20 con 0x7f8f240cd480 2024-09-10T21:19:22.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f1be00640 1 -- 172.21.15.16:0/3242680941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c004280 con 0x7f8f240cd480 2024-09-10T21:19:22.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f29e00640 1 -- 172.21.15.16:0/3242680941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f241faea0 con 0x7f8f240cd480 2024-09-10T21:19:22.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.227+0000 7f8f29e00640 1 -- 172.21.15.16:0/3242680941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f241fb420 con 0x7f8f240cd480 2024-09-10T21:19:22.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.231+0000 7f8f29e00640 1 -- 172.21.15.16:0/3242680941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f240b2980 con 0x7f8f240cd480 2024-09-10T21:19:22.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.231+0000 7f8f1be00640 1 -- 172.21.15.16:0/3242680941 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8f0c003600 con 0x7f8f240cd480 2024-09-10T21:19:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.231+0000 7f8f1be00640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8f000725e0 0x7f8f00074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.231+0000 7f8f1be00640 1 -- 172.21.15.16:0/3242680941 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f8f0c093730 con 0x7f8f240cd480 2024-09-10T21:19:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.231+0000 7f8f22a00640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8f000725e0 0x7f8f00074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.231+0000 7f8f1be00640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] conn(0x7f8f000784a0 0x7f8f0007a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.231+0000 7f8f1be00640 1 -- 172.21.15.16:0/3242680941 --> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f0c0056b0 con 0x7f8f000784a0 2024-09-10T21:19:22.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.231+0000 7f8f1be00640 1 -- 172.21.15.16:0/3242680941 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7f8f0c093ae0 con 0x7f8f240cd480 2024-09-10T21:19:22.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.231+0000 7f8f22a00640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8f000725e0 0x7f8f00074aa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8f240b1030 tx=0x7f8f14009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:22.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.231+0000 7f8f23400640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] conn(0x7f8f000784a0 0x7f8f0007a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.231+0000 7f8f23400640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] conn(0x7f8f000784a0 0x7f8f0007a8c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:22.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.235+0000 7f8f1be00640 1 -- 172.21.15.16:0/3242680941 <== osd.4 v2:172.21.15.52:6824/447485120 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f0c0056b0 con 0x7f8f000784a0 2024-09-10T21:19:22.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.247+0000 7f8f29e00640 1 -- 172.21.15.16:0/3242680941 --> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8f24160d40 con 0x7f8f000784a0 2024-09-10T21:19:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.247+0000 7f8f1be00640 1 -- 172.21.15.16:0/3242680941 <== osd.4 v2:172.21.15.52:6824/447485120 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8f24160d40 con 0x7f8f000784a0 2024-09-10T21:19:22.254 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:19:22.254 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:19:22.254 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:19:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.247+0000 7f8f29e00640 1 -- 172.21.15.16:0/3242680941 >> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] conn(0x7f8f000784a0 msgr2=0x7f8f0007a8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.247+0000 7f8f29e00640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.52:6824/447485120,v1:172.21.15.52:6825/447485120] conn(0x7f8f000784a0 0x7f8f0007a8c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.247+0000 7f8f29e00640 1 -- 172.21.15.16:0/3242680941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8f000725e0 msgr2=0x7f8f00074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.247+0000 7f8f29e00640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f8f000725e0 0x7f8f00074aa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8f240b1030 tx=0x7f8f14009040 comp rx=0 tx=0).stop 2024-09-10T21:19:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.247+0000 7f8f29e00640 1 -- 172.21.15.16:0/3242680941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f240cd480 msgr2=0x7f8f241535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.247+0000 7f8f29e00640 1 --2- 172.21.15.16:0/3242680941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f240cd480 0x7f8f241535c0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00d980 tx=0x7f8f0c00de40 comp rx=0 tx=0).stop 2024-09-10T21:19:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.247+0000 7f8f23400640 1 -- 172.21.15.16:0/3242680941 reap_dead start 2024-09-10T21:19:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.247+0000 7f8f29e00640 1 -- 172.21.15.16:0/3242680941 shutdown_connections 2024-09-10T21:19:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.247+0000 7f8f29e00640 1 -- 172.21.15.16:0/3242680941 >> 172.21.15.16:0/3242680941 conn(0x7f8f240d1e60 msgr2=0x7f8f241601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.247+0000 7f8f29e00640 1 -- 172.21.15.16:0/3242680941 shutdown_connections 2024-09-10T21:19:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.247+0000 7f8f29e00640 1 -- 172.21.15.16:0/3242680941 wait complete. 2024-09-10T21:19:22.274 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-10T21:19:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.415+0000 7f1429800640 1 -- 172.21.15.16:0/2825042713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14240b2860 msgr2=0x7f1424160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.415+0000 7f1429800640 1 --2- 172.21.15.16:0/2825042713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14240b2860 0x7f1424160510 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f14240708f0 tx=0x7f141c01cb80 comp rx=0 tx=0).stop 2024-09-10T21:19:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.415+0000 7f1429800640 1 -- 172.21.15.16:0/2825042713 shutdown_connections 2024-09-10T21:19:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.415+0000 7f1429800640 1 --2- 172.21.15.16:0/2825042713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14240b2860 0x7f1424160510 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.415+0000 7f1429800640 1 --2- 172.21.15.16:0/2825042713 >> v2:172.21.15.52:3300/0 conn(0x7f14240d03c0 0x7f14240ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.415+0000 7f1429800640 1 -- 172.21.15.16:0/2825042713 >> 172.21.15.16:0/2825042713 conn(0x7f14240d1d80 msgr2=0x7f14240d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:22.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.415+0000 7f1429800640 1 -- 172.21.15.16:0/2825042713 shutdown_connections 2024-09-10T21:19:22.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.415+0000 7f1429800640 1 -- 172.21.15.16:0/2825042713 wait complete. 2024-09-10T21:19:22.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1429800640 1 Processor -- start 2024-09-10T21:19:22.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1429800640 1 -- start start 2024-09-10T21:19:22.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1429800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14240b2860 0x7f1424150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1429800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14240cffe0 0x7f142415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1429800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14240d03c0 0x7f14240b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1429800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14240cd9d0 con 0x7f14240b2860 2024-09-10T21:19:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1429800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14240cd850 con 0x7f14240cffe0 2024-09-10T21:19:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1429800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f14240cdb50 con 0x7f14240d03c0 2024-09-10T21:19:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1422a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14240b2860 0x7f1424150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1422a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14240b2860 0x7f1424150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42636/0 (socket says 172.21.15.16:42636) 2024-09-10T21:19:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1422a00640 1 -- 172.21.15.16:0/2371422851 learned_addr learned my addr 172.21.15.16:0/2371422851 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f141be00640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14240cffe0 0x7f142415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1423400640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14240d03c0 0x7f14240b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1422a00640 1 -- 172.21.15.16:0/2371422851 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14240d03c0 msgr2=0x7f14240b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1422a00640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14240d03c0 0x7f14240b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1422a00640 1 -- 172.21.15.16:0/2371422851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14240cffe0 msgr2=0x7f142415ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1422a00640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14240cffe0 0x7f142415ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1422a00640 1 -- 172.21.15.16:0/2371422851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1424166ef0 con 0x7f14240b2860 2024-09-10T21:19:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1423400640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14240d03c0 0x7f14240b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1422a00640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14240b2860 0x7f1424150280 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f141400c500 tx=0x7f141400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f141aa00640 1 -- 172.21.15.16:0/2371422851 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414018070 con 0x7f14240b2860 2024-09-10T21:19:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f141aa00640 1 -- 172.21.15.16:0/2371422851 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1414003600 con 0x7f14240b2860 2024-09-10T21:19:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f141aa00640 1 -- 172.21.15.16:0/2371422851 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414003a90 con 0x7f14240b2860 2024-09-10T21:19:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1429800640 1 -- 172.21.15.16:0/2371422851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14240b4ce0 con 0x7f14240b2860 2024-09-10T21:19:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1429800640 1 -- 172.21.15.16:0/2371422851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14240b52a0 con 0x7f14240b2860 2024-09-10T21:19:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.419+0000 7f1429800640 1 -- 172.21.15.16:0/2371422851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14240ce2b0 con 0x7f14240b2860 2024-09-10T21:19:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.423+0000 7f141aa00640 1 -- 172.21.15.16:0/2371422851 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1414003c30 con 0x7f14240b2860 2024-09-10T21:19:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.423+0000 7f141aa00640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14000725e0 0x7f1400074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.423+0000 7f141aa00640 1 -- 172.21.15.16:0/2371422851 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f14140922b0 con 0x7f14240b2860 2024-09-10T21:19:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.423+0000 7f141be00640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14000725e0 0x7f1400074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.423+0000 7f141be00640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14000725e0 0x7f1400074aa0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14241579b0 tx=0x7f140c00b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.427+0000 7f141aa00640 1 -- 172.21.15.16:0/2371422851 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f141405d670 con 0x7f14240b2860 2024-09-10T21:19:22.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.599+0000 7f1429800640 1 -- 172.21.15.16:0/2371422851 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f14240b3070 con 0x7f14000725e0 2024-09-10T21:19:22.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.603+0000 7f141aa00640 1 -- 172.21.15.16:0/2371422851 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f14240b3070 con 0x7f14000725e0 2024-09-10T21:19:22.607 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 4 to deep-scrub 2024-09-10T21:19:22.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.603+0000 7f1429800640 1 -- 172.21.15.16:0/2371422851 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14000725e0 msgr2=0x7f1400074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.603+0000 7f1429800640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14000725e0 0x7f1400074aa0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14241579b0 tx=0x7f140c00b2e0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.603+0000 7f1429800640 1 -- 172.21.15.16:0/2371422851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14240b2860 msgr2=0x7f1424150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.603+0000 7f1429800640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14240b2860 0x7f1424150280 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f141400c500 tx=0x7f141400c9c0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.603+0000 7f1429800640 1 -- 172.21.15.16:0/2371422851 shutdown_connections 2024-09-10T21:19:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.603+0000 7f1429800640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f14000725e0 0x7f1400074aa0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.603+0000 7f1429800640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f14240d03c0 0x7f14240b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.603+0000 7f1429800640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14240cffe0 0x7f142415ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.603+0000 7f1429800640 1 --2- 172.21.15.16:0/2371422851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14240b2860 0x7f1424150280 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.603+0000 7f1429800640 1 -- 172.21.15.16:0/2371422851 >> 172.21.15.16:0/2371422851 conn(0x7f14240d1d80 msgr2=0x7f142415a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.603+0000 7f1429800640 1 -- 172.21.15.16:0/2371422851 shutdown_connections 2024-09-10T21:19:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.603+0000 7f1429800640 1 -- 172.21.15.16:0/2371422851 wait complete. 2024-09-10T21:19:22.633 INFO:tasks.ceph:Scrubbing osd.5 2024-09-10T21:19:22.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:19:22.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.779+0000 7f6a18000640 1 -- 172.21.15.16:0/4196864808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a100cc520 msgr2=0x7f6a100cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.779+0000 7f6a18000640 1 --2- 172.21.15.16:0/4196864808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a100cc520 0x7f6a100cc970 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f6a1006ed10 tx=0x7f6a0801cb80 comp rx=0 tx=0).stop 2024-09-10T21:19:22.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.779+0000 7f6a18000640 1 -- 172.21.15.16:0/4196864808 shutdown_connections 2024-09-10T21:19:22.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.779+0000 7f6a18000640 1 --2- 172.21.15.16:0/4196864808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a100cc520 0x7f6a100cc970 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.779+0000 7f6a18000640 1 --2- 172.21.15.16:0/4196864808 >> v2:172.21.15.52:3300/0 conn(0x7f6a1015f830 0x7f6a1015fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.779+0000 7f6a18000640 1 -- 172.21.15.16:0/4196864808 >> 172.21.15.16:0/4196864808 conn(0x7f6a1006d1e0 msgr2=0x7f6a100b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:22.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.779+0000 7f6a18000640 1 -- 172.21.15.16:0/4196864808 shutdown_connections 2024-09-10T21:19:22.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.779+0000 7f6a18000640 1 -- 172.21.15.16:0/4196864808 wait complete. 2024-09-10T21:19:22.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.779+0000 7f6a18000640 1 Processor -- start 2024-09-10T21:19:22.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.779+0000 7f6a18000640 1 -- start start 2024-09-10T21:19:22.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.779+0000 7f6a18000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a100cc520 0x7f6a1015c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a18000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a10144ff0 0x7f6a10165fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a18000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a1015f830 0x7f6a10166520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a18000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10168de0 con 0x7f6a100cc520 2024-09-10T21:19:22.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a18000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a100cefd0 con 0x7f6a10144ff0 2024-09-10T21:19:22.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a18000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6a10168f60 con 0x7f6a1015f830 2024-09-10T21:19:22.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a15600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a100cc520 0x7f6a1015c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a15600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a100cc520 0x7f6a1015c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42646/0 (socket says 172.21.15.16:42646) 2024-09-10T21:19:22.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a15600640 1 -- 172.21.15.16:0/410212966 learned_addr learned my addr 172.21.15.16:0/410212966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:22.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a14c00640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a10144ff0 0x7f6a10165fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a16000640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a1015f830 0x7f6a10166520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a15600640 1 -- 172.21.15.16:0/410212966 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a1015f830 msgr2=0x7f6a10166520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a15600640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6a1015f830 0x7f6a10166520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a15600640 1 -- 172.21.15.16:0/410212966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a10144ff0 msgr2=0x7f6a10165fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a15600640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a10144ff0 0x7f6a10165fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a15600640 1 -- 172.21.15.16:0/410212966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a10161d50 con 0x7f6a100cc520 2024-09-10T21:19:22.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a15600640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a100cc520 0x7f6a1015c8f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400c730 tx=0x7f6a0400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:22.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a0d600640 1 -- 172.21.15.16:0/410212966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04017070 con 0x7f6a100cc520 2024-09-10T21:19:22.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a0d600640 1 -- 172.21.15.16:0/410212966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a040051a0 con 0x7f6a100cc520 2024-09-10T21:19:22.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a0d600640 1 -- 172.21.15.16:0/410212966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400a640 con 0x7f6a100cc520 2024-09-10T21:19:22.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a18000640 1 -- 172.21.15.16:0/410212966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a10162030 con 0x7f6a100cc520 2024-09-10T21:19:22.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a18000640 1 -- 172.21.15.16:0/410212966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a101625f0 con 0x7f6a100cc520 2024-09-10T21:19:22.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a18000640 1 -- 172.21.15.16:0/410212966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6a100cc970 con 0x7f6a100cc520 2024-09-10T21:19:22.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.783+0000 7f6a0d600640 1 -- 172.21.15.16:0/410212966 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6a0400a7e0 con 0x7f6a100cc520 2024-09-10T21:19:22.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.787+0000 7f6a0d600640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69f00725e0 0x7f69f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.787+0000 7f6a0d600640 1 -- 172.21.15.16:0/410212966 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f6a04092fe0 con 0x7f6a100cc520 2024-09-10T21:19:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.787+0000 7f6a14c00640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69f00725e0 0x7f69f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.787+0000 7f6a0d600640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] conn(0x7f69f00784a0 0x7f69f007a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.787+0000 7f6a0d600640 1 -- 172.21.15.16:0/410212966 --> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a04012e50 con 0x7f69f00784a0 2024-09-10T21:19:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.787+0000 7f6a0d600640 1 -- 172.21.15.16:0/410212966 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7f6a04093430 con 0x7f6a100cc520 2024-09-10T21:19:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.787+0000 7f6a16000640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] conn(0x7f69f00784a0 0x7f69f007a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.787+0000 7f6a14c00640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69f00725e0 0x7f69f0074aa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6a10163fc0 tx=0x7f69fc00b2e0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.787+0000 7f6a16000640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] conn(0x7f69f00784a0 0x7f69f007a8c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:22.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.787+0000 7f6a0d600640 1 -- 172.21.15.16:0/410212966 <== osd.5 v2:172.21.15.52:6816/1432062268 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a04012e50 con 0x7f69f00784a0 2024-09-10T21:19:22.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.803+0000 7f6a18000640 1 -- 172.21.15.16:0/410212966 --> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6a100b2470 con 0x7f69f00784a0 2024-09-10T21:19:22.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.803+0000 7f6a0d600640 1 -- 172.21.15.16:0/410212966 <== osd.5 v2:172.21.15.52:6816/1432062268 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6a100b2470 con 0x7f69f00784a0 2024-09-10T21:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:19:22.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.803+0000 7f6a18000640 1 -- 172.21.15.16:0/410212966 >> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] conn(0x7f69f00784a0 msgr2=0x7f69f007a8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.803+0000 7f6a18000640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.52:6816/1432062268,v1:172.21.15.52:6817/1432062268] conn(0x7f69f00784a0 0x7f69f007a8c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.803+0000 7f6a18000640 1 -- 172.21.15.16:0/410212966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69f00725e0 msgr2=0x7f69f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.803+0000 7f6a18000640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f69f00725e0 0x7f69f0074aa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6a10163fc0 tx=0x7f69fc00b2e0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.803+0000 7f6a18000640 1 -- 172.21.15.16:0/410212966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a100cc520 msgr2=0x7f6a1015c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.803+0000 7f6a18000640 1 --2- 172.21.15.16:0/410212966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a100cc520 0x7f6a1015c8f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400c730 tx=0x7f6a0400cbf0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.803+0000 7f6a16000640 1 -- 172.21.15.16:0/410212966 reap_dead start 2024-09-10T21:19:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.803+0000 7f6a18000640 1 -- 172.21.15.16:0/410212966 shutdown_connections 2024-09-10T21:19:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.803+0000 7f6a18000640 1 -- 172.21.15.16:0/410212966 >> 172.21.15.16:0/410212966 conn(0x7f6a1006d1e0 msgr2=0x7f6a10143fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.803+0000 7f6a18000640 1 -- 172.21.15.16:0/410212966 shutdown_connections 2024-09-10T21:19:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.803+0000 7f6a18000640 1 -- 172.21.15.16:0/410212966 wait complete. 2024-09-10T21:19:22.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-10T21:19:22.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.967+0000 7f4625000640 1 -- 172.21.15.16:0/4053191165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620155000 msgr2=0x7f462015bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.967+0000 7f4625000640 1 --2- 172.21.15.16:0/4053191165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620155000 0x7f462015bbb0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f4610009900 tx=0x7f461001c870 comp rx=0 tx=0).stop 2024-09-10T21:19:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.967+0000 7f4625000640 1 -- 172.21.15.16:0/4053191165 shutdown_connections 2024-09-10T21:19:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.967+0000 7f4625000640 1 --2- 172.21.15.16:0/4053191165 >> v2:172.21.15.52:3300/0 conn(0x7f46200706f0 0x7f46200ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.967+0000 7f4625000640 1 --2- 172.21.15.16:0/4053191165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620155000 0x7f462015bbb0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.967+0000 7f4625000640 1 -- 172.21.15.16:0/4053191165 >> 172.21.15.16:0/4053191165 conn(0x7f46200cfb20 msgr2=0x7f46200cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.967+0000 7f4625000640 1 -- 172.21.15.16:0/4053191165 shutdown_connections 2024-09-10T21:19:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.967+0000 7f4625000640 1 -- 172.21.15.16:0/4053191165 wait complete. 2024-09-10T21:19:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.967+0000 7f4625000640 1 Processor -- start 2024-09-10T21:19:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.967+0000 7f4625000640 1 -- start start 2024-09-10T21:19:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4625000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200706f0 0x7f46200b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4625000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620072d40 0x7f46200cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4625000640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4620155000 0x7f46201532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4625000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4620167b20 con 0x7f4620072d40 2024-09-10T21:19:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4625000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46201679a0 con 0x7f46200706f0 2024-09-10T21:19:22.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4625000640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f4620167ca0 con 0x7f4620155000 2024-09-10T21:19:22.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4617e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620072d40 0x7f46200cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f461de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200706f0 0x7f46200b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4617e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620072d40 0x7f46200cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42668/0 (socket says 172.21.15.16:42668) 2024-09-10T21:19:22.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4617e00640 1 -- 172.21.15.16:0/3895121416 learned_addr learned my addr 172.21.15.16:0/3895121416 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:22.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f461de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200706f0 0x7f46200b0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.16:48394/0 (socket says 172.21.15.16:48394) 2024-09-10T21:19:22.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f461e800640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4620155000 0x7f46201532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4617e00640 1 -- 172.21.15.16:0/3895121416 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4620155000 msgr2=0x7f46201532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4617e00640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4620155000 0x7f46201532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4617e00640 1 -- 172.21.15.16:0/3895121416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200706f0 msgr2=0x7f46200b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:22.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4617e00640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200706f0 0x7f46200b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:22.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4617e00640 1 -- 172.21.15.16:0/3895121416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46200ae370 con 0x7f4620072d40 2024-09-10T21:19:22.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f461de00640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200706f0 0x7f46200b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:19:22.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4617e00640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620072d40 0x7f46200cf060 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f460800c490 tx=0x7f4608001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:22.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4616000640 1 -- 172.21.15.16:0/3895121416 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608017070 con 0x7f4620072d40 2024-09-10T21:19:22.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4616000640 1 -- 172.21.15.16:0/3895121416 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4608003600 con 0x7f4620072d40 2024-09-10T21:19:22.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4616000640 1 -- 172.21.15.16:0/3895121416 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608003a50 con 0x7f4620072d40 2024-09-10T21:19:22.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4625000640 1 -- 172.21.15.16:0/3895121416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46200adaa0 con 0x7f4620072d40 2024-09-10T21:19:22.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4625000640 1 -- 172.21.15.16:0/3895121416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46200ae630 con 0x7f4620072d40 2024-09-10T21:19:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.971+0000 7f4625000640 1 -- 172.21.15.16:0/3895121416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46200ca4c0 con 0x7f4620072d40 2024-09-10T21:19:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.975+0000 7f4616000640 1 -- 172.21.15.16:0/3895121416 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4608003bf0 con 0x7f4620072d40 2024-09-10T21:19:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.975+0000 7f4616000640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45fc0725e0 0x7f45fc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.975+0000 7f4616000640 1 -- 172.21.15.16:0/3895121416 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f46080920f0 con 0x7f4620072d40 2024-09-10T21:19:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.975+0000 7f461de00640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45fc0725e0 0x7f45fc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.975+0000 7f461de00640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45fc0725e0 0x7f45fc074aa0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f46100af000 tx=0x7f461001b0b0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:22.979+0000 7f4616000640 1 -- 172.21.15.16:0/3895121416 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f460805d3f0 con 0x7f4620072d40 2024-09-10T21:19:23.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.155+0000 7f4625000640 1 -- 172.21.15.16:0/3895121416 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f46200ad850 con 0x7f45fc0725e0 2024-09-10T21:19:23.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.155+0000 7f4616000640 1 -- 172.21.15.16:0/3895121416 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f46200ad850 con 0x7f45fc0725e0 2024-09-10T21:19:23.161 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 5 to deep-scrub 2024-09-10T21:19:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.159+0000 7f4625000640 1 -- 172.21.15.16:0/3895121416 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45fc0725e0 msgr2=0x7f45fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.159+0000 7f4625000640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45fc0725e0 0x7f45fc074aa0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f46100af000 tx=0x7f461001b0b0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.159+0000 7f4625000640 1 -- 172.21.15.16:0/3895121416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620072d40 msgr2=0x7f46200cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.159+0000 7f4625000640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620072d40 0x7f46200cf060 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f460800c490 tx=0x7f4608001e80 comp rx=0 tx=0).stop 2024-09-10T21:19:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.159+0000 7f4625000640 1 -- 172.21.15.16:0/3895121416 shutdown_connections 2024-09-10T21:19:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.159+0000 7f4625000640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45fc0725e0 0x7f45fc074aa0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.159+0000 7f4625000640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f4620155000 0x7f46201532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.159+0000 7f4625000640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620072d40 0x7f46200cf060 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.159+0000 7f4625000640 1 --2- 172.21.15.16:0/3895121416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200706f0 0x7f46200b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.159+0000 7f4625000640 1 -- 172.21.15.16:0/3895121416 >> 172.21.15.16:0/3895121416 conn(0x7f46200cfb20 msgr2=0x7f46200b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:23.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.159+0000 7f4625000640 1 -- 172.21.15.16:0/3895121416 shutdown_connections 2024-09-10T21:19:23.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.159+0000 7f4625000640 1 -- 172.21.15.16:0/3895121416 wait complete. 2024-09-10T21:19:23.184 INFO:tasks.ceph:Scrubbing osd.6 2024-09-10T21:19:23.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:19:23.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.331+0000 7f33b0e00640 1 -- 172.21.15.16:0/1856557940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3390003c40 msgr2=0x7f3390004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.331+0000 7f33b0e00640 1 --2- 172.21.15.16:0/1856557940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3390003c40 0x7f3390004040 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f339c001860 tx=0x7f339c01c840 comp rx=0 tx=0).stop 2024-09-10T21:19:23.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.331+0000 7f33b0e00640 1 -- 172.21.15.16:0/1856557940 shutdown_connections 2024-09-10T21:19:23.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.331+0000 7f33b0e00640 1 --2- 172.21.15.16:0/1856557940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3390003c40 0x7f3390004040 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.331+0000 7f33b0e00640 1 -- 172.21.15.16:0/1856557940 >> 172.21.15.16:0/1856557940 conn(0x7f33ac06d1e0 msgr2=0x7f33ac0b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:23.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.331+0000 7f33b0e00640 1 -- 172.21.15.16:0/1856557940 shutdown_connections 2024-09-10T21:19:23.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.331+0000 7f33b0e00640 1 -- 172.21.15.16:0/1856557940 wait complete. 2024-09-10T21:19:23.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.331+0000 7f33b0e00640 1 Processor -- start 2024-09-10T21:19:23.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.331+0000 7f33b0e00640 1 -- start start 2024-09-10T21:19:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33b0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3390003c40 0x7f33ac131390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33b0e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac1f7e00 0x7f33ac143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33b0e00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f33ac143fa0 0x7f33ac15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33b0e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac0cc520 con 0x7f3390003c40 2024-09-10T21:19:23.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33b0e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac0ceeb0 con 0x7f33ac1f7e00 2024-09-10T21:19:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33b0e00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac0cc6a0 con 0x7f33ac143fa0 2024-09-10T21:19:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3390003c40 0x7f33ac131390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3390003c40 0x7f33ac131390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42690/0 (socket says 172.21.15.16:42690) 2024-09-10T21:19:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a9e00640 1 -- 172.21.15.16:0/248654178 learned_addr learned my addr 172.21.15.16:0/248654178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a3e00640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac1f7e00 0x7f33ac143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33aa800640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f33ac143fa0 0x7f33ac15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a9e00640 1 -- 172.21.15.16:0/248654178 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f33ac143fa0 msgr2=0x7f33ac15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a9e00640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f33ac143fa0 0x7f33ac15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a9e00640 1 -- 172.21.15.16:0/248654178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac1f7e00 msgr2=0x7f33ac143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a9e00640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac1f7e00 0x7f33ac143a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a9e00640 1 -- 172.21.15.16:0/248654178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac0af9d0 con 0x7f3390003c40 2024-09-10T21:19:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a3e00640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac1f7e00 0x7f33ac143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a9e00640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3390003c40 0x7f33ac131390 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f339c01cd20 tx=0x7f339c003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a2000640 1 -- 172.21.15.16:0/248654178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0bc070 con 0x7f3390003c40 2024-09-10T21:19:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a2000640 1 -- 172.21.15.16:0/248654178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f339c003e30 con 0x7f3390003c40 2024-09-10T21:19:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a2000640 1 -- 172.21.15.16:0/248654178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c001480 con 0x7f3390003c40 2024-09-10T21:19:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33b0e00640 1 -- 172.21.15.16:0/248654178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ac1318d0 con 0x7f3390003c40 2024-09-10T21:19:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33b0e00640 1 -- 172.21.15.16:0/248654178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ac144720 con 0x7f3390003c40 2024-09-10T21:19:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33b0e00640 1 -- 172.21.15.16:0/248654178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f33ac0b2b10 con 0x7f3390003c40 2024-09-10T21:19:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.335+0000 7f33a2000640 1 -- 172.21.15.16:0/248654178 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f339c006090 con 0x7f3390003c40 2024-09-10T21:19:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.339+0000 7f33a2000640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33880725e0 0x7f3388074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.339+0000 7f33a2000640 1 -- 172.21.15.16:0/248654178 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f339c12d170 con 0x7f3390003c40 2024-09-10T21:19:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.339+0000 7f33a2000640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] conn(0x7f33880784a0 0x7f338807a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.339+0000 7f33a2000640 1 -- 172.21.15.16:0/248654178 --> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f339c0b2e90 con 0x7f33880784a0 2024-09-10T21:19:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.339+0000 7f33a3e00640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33880725e0 0x7f3388074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.339+0000 7f33a2000640 1 -- 172.21.15.16:0/248654178 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7f339c12d520 con 0x7f3390003c40 2024-09-10T21:19:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.339+0000 7f33aa800640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] conn(0x7f33880784a0 0x7f338807a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.339+0000 7f33a3e00640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33880725e0 0x7f3388074aa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3394000fe0 tx=0x7f33940068b0 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.339+0000 7f33aa800640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] conn(0x7f33880784a0 0x7f338807a8c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.339+0000 7f33a2000640 1 -- 172.21.15.16:0/248654178 <== osd.6 v2:172.21.15.52:6800/2640674683 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f339c0b2e90 con 0x7f33880784a0 2024-09-10T21:19:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.355+0000 7f33b0e00640 1 -- 172.21.15.16:0/248654178 --> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f33ac0b2470 con 0x7f33880784a0 2024-09-10T21:19:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.355+0000 7f33a2000640 1 -- 172.21.15.16:0/248654178 <== osd.6 v2:172.21.15.52:6800/2640674683 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f33ac0b2470 con 0x7f33880784a0 2024-09-10T21:19:23.360 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:19:23.360 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:19:23.360 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:19:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.355+0000 7f33b0e00640 1 -- 172.21.15.16:0/248654178 >> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] conn(0x7f33880784a0 msgr2=0x7f338807a8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.355+0000 7f33b0e00640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.52:6800/2640674683,v1:172.21.15.52:6801/2640674683] conn(0x7f33880784a0 0x7f338807a8c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.355+0000 7f33b0e00640 1 -- 172.21.15.16:0/248654178 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33880725e0 msgr2=0x7f3388074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.355+0000 7f33b0e00640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f33880725e0 0x7f3388074aa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3394000fe0 tx=0x7f33940068b0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.355+0000 7f33b0e00640 1 -- 172.21.15.16:0/248654178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3390003c40 msgr2=0x7f33ac131390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.355+0000 7f33b0e00640 1 --2- 172.21.15.16:0/248654178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3390003c40 0x7f33ac131390 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f339c01cd20 tx=0x7f339c003c40 comp rx=0 tx=0).stop 2024-09-10T21:19:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.355+0000 7f33aa800640 1 -- 172.21.15.16:0/248654178 reap_dead start 2024-09-10T21:19:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.355+0000 7f33b0e00640 1 -- 172.21.15.16:0/248654178 shutdown_connections 2024-09-10T21:19:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.355+0000 7f33b0e00640 1 -- 172.21.15.16:0/248654178 >> 172.21.15.16:0/248654178 conn(0x7f33ac06d1e0 msgr2=0x7f33ac15d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.355+0000 7f33b0e00640 1 -- 172.21.15.16:0/248654178 shutdown_connections 2024-09-10T21:19:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.355+0000 7f33b0e00640 1 -- 172.21.15.16:0/248654178 wait complete. 2024-09-10T21:19:23.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-10T21:19:23.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.523+0000 7f65cce00640 1 -- 172.21.15.16:0/2147262398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c80cdca0 msgr2=0x7f65c80cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.523+0000 7f65cce00640 1 --2- 172.21.15.16:0/2147262398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c80cdca0 0x7f65c80cfe20 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f65c8070190 tx=0x7f65b001c770 comp rx=0 tx=0).stop 2024-09-10T21:19:23.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.523+0000 7f65cce00640 1 -- 172.21.15.16:0/2147262398 shutdown_connections 2024-09-10T21:19:23.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.523+0000 7f65cce00640 1 --2- 172.21.15.16:0/2147262398 >> v2:172.21.15.52:3300/0 conn(0x7f65c80b1c10 0x7f65c80cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.523+0000 7f65cce00640 1 --2- 172.21.15.16:0/2147262398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c80cdca0 0x7f65c80cfe20 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.523+0000 7f65cce00640 1 -- 172.21.15.16:0/2147262398 >> 172.21.15.16:0/2147262398 conn(0x7f65c80d1c10 msgr2=0x7f65c80d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:23.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.523+0000 7f65cce00640 1 -- 172.21.15.16:0/2147262398 shutdown_connections 2024-09-10T21:19:23.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.523+0000 7f65cce00640 1 -- 172.21.15.16:0/2147262398 wait complete. 2024-09-10T21:19:23.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.523+0000 7f65cce00640 1 Processor -- start 2024-09-10T21:19:23.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.523+0000 7f65cce00640 1 -- start start 2024-09-10T21:19:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65cce00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c80b1c10 0x7f65c814c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65cce00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f65c80cdca0 0x7f65c80c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65cce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c80cfa40 0x7f65c815b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65cce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65c80cee20 con 0x7f65c80cfa40 2024-09-10T21:19:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65cce00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f65c80ceca0 con 0x7f65c80b1c10 2024-09-10T21:19:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65cce00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f65c80cefa0 con 0x7f65c80cdca0 2024-09-10T21:19:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65c7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c80cfa40 0x7f65c815b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65c7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c80cfa40 0x7f65c815b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42708/0 (socket says 172.21.15.16:42708) 2024-09-10T21:19:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65c7e00640 1 -- 172.21.15.16:0/1537502349 learned_addr learned my addr 172.21.15.16:0/1537502349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:23.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65bfe00640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f65c80cdca0 0x7f65c80c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65c7400640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c80b1c10 0x7f65c814c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65c7e00640 1 -- 172.21.15.16:0/1537502349 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f65c80cdca0 msgr2=0x7f65c80c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65c7e00640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f65c80cdca0 0x7f65c80c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65c7e00640 1 -- 172.21.15.16:0/1537502349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c80b1c10 msgr2=0x7f65c814c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65c7e00640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c80b1c10 0x7f65c814c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65c7e00640 1 -- 172.21.15.16:0/1537502349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c80c8ce0 con 0x7f65c80cfa40 2024-09-10T21:19:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65c7e00640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c80cfa40 0x7f65c815b660 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f65c000ea20 tx=0x7f65c000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65bf400640 1 -- 172.21.15.16:0/1537502349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c000cde0 con 0x7f65c80cfa40 2024-09-10T21:19:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65bf400640 1 -- 172.21.15.16:0/1537502349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65c0004c40 con 0x7f65c80cfa40 2024-09-10T21:19:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65bf400640 1 -- 172.21.15.16:0/1537502349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0005280 con 0x7f65c80cfa40 2024-09-10T21:19:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65cce00640 1 -- 172.21.15.16:0/1537502349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c80b7240 con 0x7f65c80cfa40 2024-09-10T21:19:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65cce00640 1 -- 172.21.15.16:0/1537502349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c80b7800 con 0x7f65c80cfa40 2024-09-10T21:19:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65cce00640 1 -- 172.21.15.16:0/1537502349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c80cd580 con 0x7f65c80cfa40 2024-09-10T21:19:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.527+0000 7f65bf400640 1 -- 172.21.15.16:0/1537502349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f65c000f040 con 0x7f65c80cfa40 2024-09-10T21:19:23.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.531+0000 7f65bf400640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65a40725e0 0x7f65a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.531+0000 7f65bf400640 1 -- 172.21.15.16:0/1537502349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f65c0093a90 con 0x7f65c80cfa40 2024-09-10T21:19:23.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.531+0000 7f65c7400640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65a40725e0 0x7f65a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.531+0000 7f65c7400640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65a40725e0 0x7f65a4074aa0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f65b8007fd0 tx=0x7f65b8009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:23.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.531+0000 7f65bf400640 1 -- 172.21.15.16:0/1537502349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f65c0016030 con 0x7f65c80cfa40 2024-09-10T21:19:23.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.711+0000 7f65cce00640 1 -- 172.21.15.16:0/1537502349 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f65c80b4610 con 0x7f65a40725e0 2024-09-10T21:19:23.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.711+0000 7f65bf400640 1 -- 172.21.15.16:0/1537502349 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f65c80b4610 con 0x7f65a40725e0 2024-09-10T21:19:23.718 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 6 to deep-scrub 2024-09-10T21:19:23.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.715+0000 7f65cce00640 1 -- 172.21.15.16:0/1537502349 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65a40725e0 msgr2=0x7f65a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.715+0000 7f65cce00640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65a40725e0 0x7f65a4074aa0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f65b8007fd0 tx=0x7f65b8009040 comp rx=0 tx=0).stop 2024-09-10T21:19:23.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.715+0000 7f65cce00640 1 -- 172.21.15.16:0/1537502349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c80cfa40 msgr2=0x7f65c815b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.715+0000 7f65cce00640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c80cfa40 0x7f65c815b660 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f65c000ea20 tx=0x7f65c000eee0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.715+0000 7f65cce00640 1 -- 172.21.15.16:0/1537502349 shutdown_connections 2024-09-10T21:19:23.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.715+0000 7f65cce00640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f65a40725e0 0x7f65a4074aa0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.715+0000 7f65cce00640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c80cfa40 0x7f65c815b660 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.715+0000 7f65cce00640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f65c80cdca0 0x7f65c80c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.715+0000 7f65cce00640 1 --2- 172.21.15.16:0/1537502349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c80b1c10 0x7f65c814c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.715+0000 7f65cce00640 1 -- 172.21.15.16:0/1537502349 >> 172.21.15.16:0/1537502349 conn(0x7f65c80d1c10 msgr2=0x7f65c80d2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:23.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.715+0000 7f65cce00640 1 -- 172.21.15.16:0/1537502349 shutdown_connections 2024-09-10T21:19:23.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.715+0000 7f65cce00640 1 -- 172.21.15.16:0/1537502349 wait complete. 2024-09-10T21:19:23.739 INFO:tasks.ceph:Scrubbing osd.7 2024-09-10T21:19:23.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:19:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.883+0000 7f45d4a00640 1 -- 172.21.15.16:0/2308290515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d0072710 msgr2=0x7f45d00b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.883+0000 7f45d4a00640 1 --2- 172.21.15.16:0/2308290515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d0072710 0x7f45d00b2e40 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f45d006db80 tx=0x7f45b801c600 comp rx=0 tx=0).stop 2024-09-10T21:19:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.883+0000 7f45d4a00640 1 -- 172.21.15.16:0/2308290515 shutdown_connections 2024-09-10T21:19:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.883+0000 7f45d4a00640 1 --2- 172.21.15.16:0/2308290515 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45d00b3380 0x7f45d015e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.883+0000 7f45d4a00640 1 --2- 172.21.15.16:0/2308290515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d0072710 0x7f45d00b2e40 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.883+0000 7f45d4a00640 1 --2- 172.21.15.16:0/2308290515 >> v2:172.21.15.52:3300/0 conn(0x7f45d015bcc0 0x7f45d015b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.883+0000 7f45d4a00640 1 -- 172.21.15.16:0/2308290515 >> 172.21.15.16:0/2308290515 conn(0x7f45d006d1e0 msgr2=0x7f45d00b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.883+0000 7f45d4a00640 1 -- 172.21.15.16:0/2308290515 shutdown_connections 2024-09-10T21:19:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.883+0000 7f45d4a00640 1 -- 172.21.15.16:0/2308290515 wait complete. 2024-09-10T21:19:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45d4a00640 1 Processor -- start 2024-09-10T21:19:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45d4a00640 1 -- start start 2024-09-10T21:19:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45d4a00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45d0072710 0x7f45d0150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45d4a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45d00b3380 0x7f45d0150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45d4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d015bcc0 0x7f45d01f9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45d4a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45d00ccc90 con 0x7f45d015bcc0 2024-09-10T21:19:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45d4a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45d00ccb10 con 0x7f45d00b3380 2024-09-10T21:19:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45d4a00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f45d00cce10 con 0x7f45d0072710 2024-09-10T21:19:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45ce800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d015bcc0 0x7f45d01f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45ce800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d015bcc0 0x7f45d01f9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42718/0 (socket says 172.21.15.16:42718) 2024-09-10T21:19:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45ce800640 1 -- 172.21.15.16:0/513356645 learned_addr learned my addr 172.21.15.16:0/513356645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45c7e00640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45d00b3380 0x7f45d0150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45cde00640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45d0072710 0x7f45d0150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45ce800640 1 -- 172.21.15.16:0/513356645 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45d0072710 msgr2=0x7f45d0150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45ce800640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45d0072710 0x7f45d0150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45ce800640 1 -- 172.21.15.16:0/513356645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45d00b3380 msgr2=0x7f45d0150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45ce800640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45d00b3380 0x7f45d0150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45ce800640 1 -- 172.21.15.16:0/513356645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d00b06d0 con 0x7f45d015bcc0 2024-09-10T21:19:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45cde00640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f45d0072710 0x7f45d0150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45ce800640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d015bcc0 0x7f45d01f9530 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f45c800ea20 tx=0x7f45c800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45c6000640 1 -- 172.21.15.16:0/513356645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c800cde0 con 0x7f45d015bcc0 2024-09-10T21:19:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45c6000640 1 -- 172.21.15.16:0/513356645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45c8004c40 con 0x7f45d015bcc0 2024-09-10T21:19:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45d4a00640 1 -- 172.21.15.16:0/513356645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d01377e0 con 0x7f45d015bcc0 2024-09-10T21:19:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45c6000640 1 -- 172.21.15.16:0/513356645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8005280 con 0x7f45d015bcc0 2024-09-10T21:19:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.887+0000 7f45d4a00640 1 -- 172.21.15.16:0/513356645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d014db10 con 0x7f45d015bcc0 2024-09-10T21:19:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.891+0000 7f45d4a00640 1 -- 172.21.15.16:0/513356645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45d0137e50 con 0x7f45d015bcc0 2024-09-10T21:19:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.891+0000 7f45c6000640 1 -- 172.21.15.16:0/513356645 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f45c800f040 con 0x7f45d015bcc0 2024-09-10T21:19:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.891+0000 7f45c6000640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45ac0725e0 0x7f45ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.891+0000 7f45c6000640 1 -- 172.21.15.16:0/513356645 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f45c8093c30 con 0x7f45d015bcc0 2024-09-10T21:19:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.891+0000 7f45c6000640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] conn(0x7f45ac0784a0 0x7f45ac07a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.891+0000 7f45cde00640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45ac0725e0 0x7f45ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.891+0000 7f45c6000640 1 -- 172.21.15.16:0/513356645 --> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45c8019e70 con 0x7f45ac0784a0 2024-09-10T21:19:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.891+0000 7f45c6000640 1 -- 172.21.15.16:0/513356645 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7f45c8093fe0 con 0x7f45d015bcc0 2024-09-10T21:19:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.891+0000 7f45cde00640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45ac0725e0 0x7f45ac074aa0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f45c0005d90 tx=0x7f45c0005d20 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.891+0000 7f45c7e00640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] conn(0x7f45ac0784a0 0x7f45ac07a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.895+0000 7f45c7e00640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] conn(0x7f45ac0784a0 0x7f45ac07a8c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.895+0000 7f45c6000640 1 -- 172.21.15.16:0/513356645 <== osd.7 v2:172.21.15.52:6808/480240003 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45c8019e70 con 0x7f45ac0784a0 2024-09-10T21:19:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.907+0000 7f45d4a00640 1 -- 172.21.15.16:0/513356645 --> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f45d00cd330 con 0x7f45ac0784a0 2024-09-10T21:19:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.907+0000 7f45c6000640 1 -- 172.21.15.16:0/513356645 <== osd.7 v2:172.21.15.52:6808/480240003 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f45d00cd330 con 0x7f45ac0784a0 2024-09-10T21:19:23.915 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-10T21:19:23.915 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:19:23.915 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-10T21:19:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.907+0000 7f45d4a00640 1 -- 172.21.15.16:0/513356645 >> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] conn(0x7f45ac0784a0 msgr2=0x7f45ac07a8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.907+0000 7f45d4a00640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.52:6808/480240003,v1:172.21.15.52:6809/480240003] conn(0x7f45ac0784a0 0x7f45ac07a8c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.911+0000 7f45d4a00640 1 -- 172.21.15.16:0/513356645 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45ac0725e0 msgr2=0x7f45ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.911+0000 7f45d4a00640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f45ac0725e0 0x7f45ac074aa0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f45c0005d90 tx=0x7f45c0005d20 comp rx=0 tx=0).stop 2024-09-10T21:19:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.911+0000 7f45d4a00640 1 -- 172.21.15.16:0/513356645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d015bcc0 msgr2=0x7f45d01f9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.911+0000 7f45d4a00640 1 --2- 172.21.15.16:0/513356645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d015bcc0 0x7f45d01f9530 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f45c800ea20 tx=0x7f45c800eee0 comp rx=0 tx=0).stop 2024-09-10T21:19:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.911+0000 7f45ce800640 1 -- 172.21.15.16:0/513356645 reap_dead start 2024-09-10T21:19:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.911+0000 7f45d4a00640 1 -- 172.21.15.16:0/513356645 shutdown_connections 2024-09-10T21:19:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.911+0000 7f45d4a00640 1 -- 172.21.15.16:0/513356645 >> 172.21.15.16:0/513356645 conn(0x7f45d006d1e0 msgr2=0x7f45d00b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.911+0000 7f45d4a00640 1 -- 172.21.15.16:0/513356645 shutdown_connections 2024-09-10T21:19:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:23.911+0000 7f45d4a00640 1 -- 172.21.15.16:0/513356645 wait complete. 2024-09-10T21:19:23.933 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-10T21:19:24.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.075+0000 7f6fafc00640 1 -- 172.21.15.16:0/1694487586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa80c42c0 msgr2=0x7f6fa80cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.075+0000 7f6fafc00640 1 --2- 172.21.15.16:0/1694487586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa80c42c0 0x7f6fa80cc960 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8069600 tx=0x7f6f9401c740 comp rx=0 tx=0).stop 2024-09-10T21:19:24.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.075+0000 7f6fafc00640 1 -- 172.21.15.16:0/1694487586 shutdown_connections 2024-09-10T21:19:24.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.075+0000 7f6fafc00640 1 --2- 172.21.15.16:0/1694487586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa80c42c0 0x7f6fa80cc960 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.075+0000 7f6fafc00640 1 --2- 172.21.15.16:0/1694487586 >> v2:172.21.15.52:3300/0 conn(0x7f6fa815be10 0x7f6fa80cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.075+0000 7f6fafc00640 1 -- 172.21.15.16:0/1694487586 >> 172.21.15.16:0/1694487586 conn(0x7f6fa80d0c30 msgr2=0x7f6fa80d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:24.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.075+0000 7f6fafc00640 1 -- 172.21.15.16:0/1694487586 shutdown_connections 2024-09-10T21:19:24.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.075+0000 7f6fafc00640 1 -- 172.21.15.16:0/1694487586 wait complete. 2024-09-10T21:19:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.075+0000 7f6fafc00640 1 Processor -- start 2024-09-10T21:19:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fafc00640 1 -- start start 2024-09-10T21:19:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fafc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa80a1ad0 0x7f6fa815e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fafc00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6fa80c42c0 0x7f6fa8155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fafc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa815be10 0x7f6fa815a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fafc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa8169230 con 0x7f6fa80a1ad0 2024-09-10T21:19:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fafc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa81690b0 con 0x7f6fa815be10 2024-09-10T21:19:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fafc00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa81693b0 con 0x7f6fa80c42c0 2024-09-10T21:19:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fad200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa80a1ad0 0x7f6fa815e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fad200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa80a1ad0 0x7f6fa815e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42742/0 (socket says 172.21.15.16:42742) 2024-09-10T21:19:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fad200640 1 -- 172.21.15.16:0/87905620 learned_addr learned my addr 172.21.15.16:0/87905620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fa7e00640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6fa80c42c0 0x7f6fa8155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fadc00640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa815be10 0x7f6fa815a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fad200640 1 -- 172.21.15.16:0/87905620 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6fa80c42c0 msgr2=0x7f6fa8155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fad200640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6fa80c42c0 0x7f6fa8155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fad200640 1 -- 172.21.15.16:0/87905620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa815be10 msgr2=0x7f6fa815a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fad200640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa815be10 0x7f6fa815a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fad200640 1 -- 172.21.15.16:0/87905620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa8158530 con 0x7f6fa80a1ad0 2024-09-10T21:19:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fad200640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa80a1ad0 0x7f6fa815e490 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00e7d0 tx=0x7f6f9c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fa4c00640 1 -- 172.21.15.16:0/87905620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c00ccc0 con 0x7f6fa80a1ad0 2024-09-10T21:19:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fa4c00640 1 -- 172.21.15.16:0/87905620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f9c010430 con 0x7f6fa80a1ad0 2024-09-10T21:19:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fafc00640 1 -- 172.21.15.16:0/87905620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa8158810 con 0x7f6fa80a1ad0 2024-09-10T21:19:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fa4c00640 1 -- 172.21.15.16:0/87905620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c010860 con 0x7f6fa80a1ad0 2024-09-10T21:19:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fafc00640 1 -- 172.21.15.16:0/87905620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa8158af0 con 0x7f6fa80a1ad0 2024-09-10T21:19:24.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.079+0000 7f6fafc00640 1 -- 172.21.15.16:0/87905620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa80b6bb0 con 0x7f6fa80a1ad0 2024-09-10T21:19:24.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.083+0000 7f6fa4c00640 1 -- 172.21.15.16:0/87905620 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6f9c010a00 con 0x7f6fa80a1ad0 2024-09-10T21:19:24.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.083+0000 7f6fa4c00640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f880725e0 0x7f6f88074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:24.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.083+0000 7f6fa4c00640 1 -- 172.21.15.16:0/87905620 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f6f9c093a50 con 0x7f6fa80a1ad0 2024-09-10T21:19:24.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.083+0000 7f6fa7e00640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f880725e0 0x7f6f88074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:24.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.083+0000 7f6fa7e00640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f880725e0 0x7f6f88074aa0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6f940af000 tx=0x7f6f9401af80 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:24.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.087+0000 7f6fa4c00640 1 -- 172.21.15.16:0/87905620 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6f9c05ecc0 con 0x7f6fa80a1ad0 2024-09-10T21:19:24.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.259+0000 7f6fafc00640 1 -- 172.21.15.16:0/87905620 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f6fa81582e0 con 0x7f6f880725e0 2024-09-10T21:19:24.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.259+0000 7f6fa4c00640 1 -- 172.21.15.16:0/87905620 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6fa81582e0 con 0x7f6f880725e0 2024-09-10T21:19:24.266 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 7 to deep-scrub 2024-09-10T21:19:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.263+0000 7f6fafc00640 1 -- 172.21.15.16:0/87905620 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f880725e0 msgr2=0x7f6f88074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.263+0000 7f6fafc00640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f880725e0 0x7f6f88074aa0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6f940af000 tx=0x7f6f9401af80 comp rx=0 tx=0).stop 2024-09-10T21:19:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.263+0000 7f6fafc00640 1 -- 172.21.15.16:0/87905620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa80a1ad0 msgr2=0x7f6fa815e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.263+0000 7f6fafc00640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa80a1ad0 0x7f6fa815e490 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00e7d0 tx=0x7f6f9c00ec90 comp rx=0 tx=0).stop 2024-09-10T21:19:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.263+0000 7f6fafc00640 1 -- 172.21.15.16:0/87905620 shutdown_connections 2024-09-10T21:19:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.263+0000 7f6fafc00640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f6f880725e0 0x7f6f88074aa0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.263+0000 7f6fafc00640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6fa815be10 0x7f6fa815a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.263+0000 7f6fafc00640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f6fa80c42c0 0x7f6fa8155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.263+0000 7f6fafc00640 1 --2- 172.21.15.16:0/87905620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fa80a1ad0 0x7f6fa815e490 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.263+0000 7f6fafc00640 1 -- 172.21.15.16:0/87905620 >> 172.21.15.16:0/87905620 conn(0x7f6fa80d0c30 msgr2=0x7f6fa815c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.263+0000 7f6fafc00640 1 -- 172.21.15.16:0/87905620 shutdown_connections 2024-09-10T21:19:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.263+0000 7f6fafc00640 1 -- 172.21.15.16:0/87905620 wait complete. 2024-09-10T21:19:24.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:19:24.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:19:24.294 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:19:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.427+0000 7faec4200640 1 -- 172.21.15.16:0/2688047213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faebc0ca170 msgr2=0x7faebc0ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.427+0000 7faec4200640 1 --2- 172.21.15.16:0/2688047213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faebc0ca170 0x7faebc0ca5b0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009bc0 tx=0x7faeb401cb10 comp rx=0 tx=0).stop 2024-09-10T21:19:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec4200640 1 -- 172.21.15.16:0/2688047213 shutdown_connections 2024-09-10T21:19:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec4200640 1 --2- 172.21.15.16:0/2688047213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faebc0ca170 0x7faebc0ca5b0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec4200640 1 -- 172.21.15.16:0/2688047213 >> 172.21.15.16:0/2688047213 conn(0x7faebc0cfb20 msgr2=0x7faebc0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec4200640 1 -- 172.21.15.16:0/2688047213 shutdown_connections 2024-09-10T21:19:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec4200640 1 -- 172.21.15.16:0/2688047213 wait complete. 2024-09-10T21:19:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec4200640 1 Processor -- start 2024-09-10T21:19:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec4200640 1 -- start start 2024-09-10T21:19:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec4200640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faebc0ca170 0x7faebc1f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec4200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faebc0d06b0 0x7faebc151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec4200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faebc15a270 0x7faebc1f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec4200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faebc072ea0 con 0x7faebc0d06b0 2024-09-10T21:19:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec4200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faebc072d20 con 0x7faebc15a270 2024-09-10T21:19:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec4200640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7faebc073020 con 0x7faebc0ca170 2024-09-10T21:19:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faebc0d06b0 0x7faebc151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faebc0d06b0 0x7faebc151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42764/0 (socket says 172.21.15.16:42764) 2024-09-10T21:19:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec0e00640 1 -- 172.21.15.16:0/4124659336 learned_addr learned my addr 172.21.15.16:0/4124659336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec1800640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faebc0ca170 0x7faebc1f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec2200640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faebc15a270 0x7faebc1f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec0e00640 1 -- 172.21.15.16:0/4124659336 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faebc0ca170 msgr2=0x7faebc1f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec0e00640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faebc0ca170 0x7faebc1f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec0e00640 1 -- 172.21.15.16:0/4124659336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faebc15a270 msgr2=0x7faebc1f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec0e00640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faebc15a270 0x7faebc1f85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec0e00640 1 -- 172.21.15.16:0/4124659336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faebc0b50f0 con 0x7faebc0d06b0 2024-09-10T21:19:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.431+0000 7faec0e00640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faebc0d06b0 0x7faebc151ea0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7faea800c480 tx=0x7faea800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.435+0000 7faeb9600640 1 -- 172.21.15.16:0/4124659336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea8012830 con 0x7faebc0d06b0 2024-09-10T21:19:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.435+0000 7faeb9600640 1 -- 172.21.15.16:0/4124659336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faea8017070 con 0x7faebc0d06b0 2024-09-10T21:19:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.435+0000 7faeb9600640 1 -- 172.21.15.16:0/4124659336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea8012d80 con 0x7faebc0d06b0 2024-09-10T21:19:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.435+0000 7faec4200640 1 -- 172.21.15.16:0/4124659336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faebc0b53b0 con 0x7faebc0d06b0 2024-09-10T21:19:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.435+0000 7faec4200640 1 -- 172.21.15.16:0/4124659336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faebc0b2d40 con 0x7faebc0d06b0 2024-09-10T21:19:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.435+0000 7faec4200640 1 -- 172.21.15.16:0/4124659336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faebc0b2b60 con 0x7faebc0d06b0 2024-09-10T21:19:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.435+0000 7faeb9600640 1 -- 172.21.15.16:0/4124659336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faea801c460 con 0x7faebc0d06b0 2024-09-10T21:19:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.435+0000 7faeb9600640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fae9c0725e0 0x7fae9c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.435+0000 7faeb9600640 1 -- 172.21.15.16:0/4124659336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7faea8092d20 con 0x7faebc0d06b0 2024-09-10T21:19:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.435+0000 7faec1800640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fae9c0725e0 0x7fae9c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.439+0000 7faec1800640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fae9c0725e0 0x7fae9c074aa0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7faeb0001870 tx=0x7faeb0001380 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.439+0000 7faeb9600640 1 -- 172.21.15.16:0/4124659336 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faebc0b2b60 con 0x7faebc0d06b0 2024-09-10T21:19:24.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.611+0000 7faec4200640 1 -- 172.21.15.16:0/4124659336 --> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faebc076c50 con 0x7fae9c0725e0 2024-09-10T21:19:24.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.611+0000 7faeb9600640 1 -- 172.21.15.16:0/4124659336 <== mgr.4137 v2:172.21.15.16:6800/462868332 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13426 (secure 0 0 0) 0x7faebc076c50 con 0x7fae9c0725e0 2024-09-10T21:19:24.618 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:24.619 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-10T21:19:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.615+0000 7faec4200640 1 -- 172.21.15.16:0/4124659336 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fae9c0725e0 msgr2=0x7fae9c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.615+0000 7faec4200640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fae9c0725e0 0x7fae9c074aa0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7faeb0001870 tx=0x7faeb0001380 comp rx=0 tx=0).stop 2024-09-10T21:19:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.615+0000 7faec4200640 1 -- 172.21.15.16:0/4124659336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faebc0d06b0 msgr2=0x7faebc151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.615+0000 7faec4200640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faebc0d06b0 0x7faebc151ea0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7faea800c480 tx=0x7faea800c940 comp rx=0 tx=0).stop 2024-09-10T21:19:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.615+0000 7faec4200640 1 -- 172.21.15.16:0/4124659336 shutdown_connections 2024-09-10T21:19:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.615+0000 7faec4200640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7fae9c0725e0 0x7fae9c074aa0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.615+0000 7faec4200640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faebc15a270 0x7faebc1f85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.615+0000 7faec4200640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faebc0d06b0 0x7faebc151ea0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.615+0000 7faec4200640 1 --2- 172.21.15.16:0/4124659336 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7faebc0ca170 0x7faebc1f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.615+0000 7faec4200640 1 -- 172.21.15.16:0/4124659336 >> 172.21.15.16:0/4124659336 conn(0x7faebc0cfb20 msgr2=0x7faebc0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:24.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.615+0000 7faec4200640 1 -- 172.21.15.16:0/4124659336 shutdown_connections 2024-09-10T21:19:24.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.615+0000 7faec4200640 1 -- 172.21.15.16:0/4124659336 wait complete. 2024-09-10T21:19:24.639 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-09-10T21:19:23.580412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":297932,"kb_used_data":15112,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754676788,"statfs":{"total":773094113280,"available":772789030912,"internally_reserved":0,"allocated":15474688,"data_stored":7341936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001804"},"pg_stats":[{"pgid":"1.0","version":"142'74","reported_seq":337,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T21:19:21.854870+0000","last_change":"2024-09-10T21:19:21.854870+0000","last_active":"2024-09-10T21:19:21.854870+0000","last_peered":"2024-09-10T21:19:21.854870+0000","last_clean":"2024-09-10T21:19:21.854870+0000","last_became_active":"2024-09-10T21:10:19.082193+0000","last_became_peered":"2024-09-10T21:10:19.082193+0000","last_unstale":"2024-09-10T21:19:21.854870+0000","last_undegraded":"2024-09-10T21:19:21.854870+0000","last_fullsized":"2024-09-10T21:19:21.854870+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'74","last_scrub_stamp":"2024-09-10T21:19:21.854800+0000","last_deep_scrub":"142'74","last_deep_scrub_stamp":"2024-09-10T21:19:21.854800+0000","last_clean_scrub_stamp":"2024-09-10T21:19:21.854800+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:52.684170+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509551,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37096,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334744,"statfs":{"total":96636764160,"available":96598777856,"internally_reserved":0,"allocated":1785856,"data_stored":770150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509553,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37676,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334164,"statfs":{"total":96636764160,"available":96598183936,"internally_reserved":0,"allocated":2379776,"data_stored":1360518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37096,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334744,"statfs":{"total":96636764160,"available":96598777856,"internally_reserved":0,"allocated":1785856,"data_stored":770150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37100,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1785856,"data_stored":770150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32940,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338900,"statfs":{"total":96636764160,"available":96603033600,"internally_reserved":0,"allocated":1785856,"data_stored":770150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37100,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1785856,"data_stored":770150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542256,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41256,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330584,"statfs":{"total":96636764160,"available":96594518016,"internally_reserved":0,"allocated":1785856,"data_stored":770150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542255,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37668,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334172,"statfs":{"total":96636764160,"available":96598192128,"internally_reserved":0,"allocated":2379776,"data_stored":1360518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:19:24.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T21:19:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.783+0000 7f28eee00640 1 -- 172.21.15.16:0/185693182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc004810 msgr2=0x7f28cc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.783+0000 7f28eee00640 1 --2- 172.21.15.16:0/185693182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc004810 0x7f28cc004c90 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f28d40016c0 tx=0x7f28d401c730 comp rx=0 tx=0).stop 2024-09-10T21:19:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.783+0000 7f28eee00640 1 -- 172.21.15.16:0/185693182 shutdown_connections 2024-09-10T21:19:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.783+0000 7f28eee00640 1 --2- 172.21.15.16:0/185693182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc004810 0x7f28cc004c90 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.783+0000 7f28eee00640 1 -- 172.21.15.16:0/185693182 >> 172.21.15.16:0/185693182 conn(0x7f28e80d1ce0 msgr2=0x7f28e80d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.783+0000 7f28eee00640 1 -- 172.21.15.16:0/185693182 shutdown_connections 2024-09-10T21:19:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.783+0000 7f28eee00640 1 -- 172.21.15.16:0/185693182 wait complete. 2024-09-10T21:19:24.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.783+0000 7f28eee00640 1 Processor -- start 2024-09-10T21:19:24.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.783+0000 7f28eee00640 1 -- start start 2024-09-10T21:19:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28eee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc004810 0x7f28e806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28eee00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80cb430 0x7f28e80cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28eee00640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f28e80c9430 0x7f28e80c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28eee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28e80d04f0 con 0x7f28cc004810 2024-09-10T21:19:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28eee00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f28e80d0370 con 0x7f28e80cb430 2024-09-10T21:19:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28eee00640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f28e80d0670 con 0x7f28e80c9430 2024-09-10T21:19:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc004810 0x7f28e806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc004810 0x7f28e806f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42792/0 (socket says 172.21.15.16:42792) 2024-09-10T21:19:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28e7e00640 1 -- 172.21.15.16:0/2369362317 learned_addr learned my addr 172.21.15.16:0/2369362317 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28ece00640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f28e80c9430 0x7f28e80c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28e7400640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80cb430 0x7f28e80cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28e7e00640 1 -- 172.21.15.16:0/2369362317 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f28e80c9430 msgr2=0x7f28e80c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28e7e00640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f28e80c9430 0x7f28e80c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28e7e00640 1 -- 172.21.15.16:0/2369362317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80cb430 msgr2=0x7f28e80cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28e7e00640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80cb430 0x7f28e80cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28e7e00640 1 -- 172.21.15.16:0/2369362317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e815de60 con 0x7f28cc004810 2024-09-10T21:19:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28e7400640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80cb430 0x7f28e80cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28ece00640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f28e80c9430 0x7f28e80c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28e7e00640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc004810 0x7f28e806f4c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f28d8006fd0 tx=0x7f28d8006740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28dfe00640 1 -- 172.21.15.16:0/2369362317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8015070 con 0x7f28cc004810 2024-09-10T21:19:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28dfe00640 1 -- 172.21.15.16:0/2369362317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28d8011920 con 0x7f28cc004810 2024-09-10T21:19:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28eee00640 1 -- 172.21.15.16:0/2369362317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e80cbf80 con 0x7f28cc004810 2024-09-10T21:19:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28dfe00640 1 -- 172.21.15.16:0/2369362317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8011d70 con 0x7f28cc004810 2024-09-10T21:19:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28eee00640 1 -- 172.21.15.16:0/2369362317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e815e3e0 con 0x7f28cc004810 2024-09-10T21:19:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.787+0000 7f28eee00640 1 -- 172.21.15.16:0/2369362317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e80d0bd0 con 0x7f28cc004810 2024-09-10T21:19:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.791+0000 7f28dfe00640 1 -- 172.21.15.16:0/2369362317 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f28d80103c0 con 0x7f28cc004810 2024-09-10T21:19:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.791+0000 7f28dfe00640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f28c8072610 0x7f28c8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.791+0000 7f28dfe00640 1 -- 172.21.15.16:0/2369362317 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f28d8094130 con 0x7f28cc004810 2024-09-10T21:19:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.791+0000 7f28e7400640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f28c8072610 0x7f28c8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.791+0000 7f28e7400640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f28c8072610 0x7f28c8074ad0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f28e80cd710 tx=0x7f28d4019520 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.795+0000 7f28dfe00640 1 -- 172.21.15.16:0/2369362317 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f28d805fa70 con 0x7f28cc004810 2024-09-10T21:19:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.967+0000 7f28eee00640 1 -- 172.21.15.16:0/2369362317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f28e80704a0 con 0x7f28cc004810 2024-09-10T21:19:24.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.971+0000 7f28dfe00640 1 -- 172.21.15.16:0/2369362317 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v67)=0 v67) ==== 123+0+0 (secure 0 0 0) 0x7f28d805f370 con 0x7f28cc004810 2024-09-10T21:19:24.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.971+0000 7f28dfe00640 1 -- 172.21.15.16:0/2369362317 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f28d805ee80 con 0x7f28cc004810 2024-09-10T21:19:24.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.975+0000 7f28eee00640 1 -- 172.21.15.16:0/2369362317 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f28c8072610 msgr2=0x7f28c8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.975+0000 7f28eee00640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f28c8072610 0x7f28c8074ad0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f28e80cd710 tx=0x7f28d4019520 comp rx=0 tx=0).stop 2024-09-10T21:19:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.975+0000 7f28eee00640 1 -- 172.21.15.16:0/2369362317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc004810 msgr2=0x7f28e806f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.975+0000 7f28eee00640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc004810 0x7f28e806f4c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f28d8006fd0 tx=0x7f28d8006740 comp rx=0 tx=0).stop 2024-09-10T21:19:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.975+0000 7f28eee00640 1 -- 172.21.15.16:0/2369362317 shutdown_connections 2024-09-10T21:19:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.975+0000 7f28eee00640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f28c8072610 0x7f28c8074ad0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.975+0000 7f28eee00640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f28e80c9430 0x7f28e80c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.975+0000 7f28eee00640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80cb430 0x7f28e80cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.975+0000 7f28eee00640 1 --2- 172.21.15.16:0/2369362317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc004810 0x7f28e806f4c0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.975+0000 7f28eee00640 1 -- 172.21.15.16:0/2369362317 >> 172.21.15.16:0/2369362317 conn(0x7f28e80d1ce0 msgr2=0x7f28e80b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.975+0000 7f28eee00640 1 -- 172.21.15.16:0/2369362317 shutdown_connections 2024-09-10T21:19:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:24.975+0000 7f28eee00640 1 -- 172.21.15.16:0/2369362317 wait complete. 2024-09-10T21:19:25.003 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:19:25.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:19:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.143+0000 7f326a800640 1 -- 172.21.15.16:0/705690321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32640b13c0 msgr2=0x7f32640afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.143+0000 7f326a800640 1 --2- 172.21.15.16:0/705690321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32640b13c0 0x7f32640afac0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f32640698f0 tx=0x7f325c01cc60 comp rx=0 tx=0).stop 2024-09-10T21:19:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.143+0000 7f326a800640 1 -- 172.21.15.16:0/705690321 shutdown_connections 2024-09-10T21:19:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.143+0000 7f326a800640 1 --2- 172.21.15.16:0/705690321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32640b13c0 0x7f32640afac0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:25.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.143+0000 7f326a800640 1 --2- 172.21.15.16:0/705690321 >> v2:172.21.15.52:3300/0 conn(0x7f32640716f0 0x7f32640af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:25.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.143+0000 7f326a800640 1 -- 172.21.15.16:0/705690321 >> 172.21.15.16:0/705690321 conn(0x7f326409b130 msgr2=0x7f32640d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:25.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.143+0000 7f326a800640 1 -- 172.21.15.16:0/705690321 shutdown_connections 2024-09-10T21:19:25.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.143+0000 7f326a800640 1 -- 172.21.15.16:0/705690321 wait complete. 2024-09-10T21:19:25.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f326a800640 1 Processor -- start 2024-09-10T21:19:25.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f326a800640 1 -- start start 2024-09-10T21:19:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f326a800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32640716f0 0x7f32640b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f326a800640 1 --2- >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f32640b13c0 0x7f326406ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f326a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f326415f400 0x7f3264164b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f326a800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3264169010 con 0x7f326415f400 2024-09-10T21:19:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f326a800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3264168e90 con 0x7f32640716f0 2024-09-10T21:19:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f326a800640 1 -- --> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] -- mon_getmap magic: 0 -- 0x7f3264169190 con 0x7f32640b13c0 2024-09-10T21:19:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f3263e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f326415f400 0x7f3264164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f3263e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f326415f400 0x7f3264164b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42818/0 (socket says 172.21.15.16:42818) 2024-09-10T21:19:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f3263e00640 1 -- 172.21.15.16:0/3893534327 learned_addr learned my addr 172.21.15.16:0/3893534327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-10T21:19:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f3263400640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32640716f0 0x7f32640b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f325be00640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f32640b13c0 0x7f326406ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f3263e00640 1 -- 172.21.15.16:0/3893534327 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f32640b13c0 msgr2=0x7f326406ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f3263e00640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f32640b13c0 0x7f326406ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f3263e00640 1 -- 172.21.15.16:0/3893534327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32640716f0 msgr2=0x7f32640b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f3263e00640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32640716f0 0x7f32640b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f3263e00640 1 -- 172.21.15.16:0/3893534327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3264164470 con 0x7f326415f400 2024-09-10T21:19:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f3263e00640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f326415f400 0x7f3264164b60 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f325c009bd0 tx=0x7f325c0a5a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f325b400640 1 -- 172.21.15.16:0/3893534327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c0bb070 con 0x7f326415f400 2024-09-10T21:19:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f325b400640 1 -- 172.21.15.16:0/3893534327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f325c004e20 con 0x7f326415f400 2024-09-10T21:19:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f325b400640 1 -- 172.21.15.16:0/3893534327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c0b2040 con 0x7f326415f400 2024-09-10T21:19:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f326a800640 1 -- 172.21.15.16:0/3893534327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3264164750 con 0x7f326415f400 2024-09-10T21:19:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f326a800640 1 -- 172.21.15.16:0/3893534327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326415aa00 con 0x7f326415f400 2024-09-10T21:19:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.147+0000 7f326a800640 1 -- 172.21.15.16:0/3893534327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3264070fe0 con 0x7f326415f400 2024-09-10T21:19:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.151+0000 7f325b400640 1 -- 172.21.15.16:0/3893534327 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f325c0b5030 con 0x7f326415f400 2024-09-10T21:19:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.151+0000 7f325b400640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f32400725e0 0x7f3240074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.151+0000 7f325b400640 1 -- 172.21.15.16:0/3893534327 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4943+0+0 (secure 0 0 0) 0x7f325c12ce90 con 0x7f326415f400 2024-09-10T21:19:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.151+0000 7f3263400640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f32400725e0 0x7f3240074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.151+0000 7f3263400640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f32400725e0 0x7f3240074aa0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3254001640 tx=0x7f3254009040 comp rx=0 tx=0).ready entity=mgr.4137 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.155+0000 7f325b400640 1 -- 172.21.15.16:0/3893534327 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f325c016970 con 0x7f326415f400 2024-09-10T21:19:25.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.367+0000 7f326a800640 1 -- 172.21.15.16:0/3893534327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3264070d70 con 0x7f326415f400 2024-09-10T21:19:25.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.367+0000 7f325b400640 1 -- 172.21.15.16:0/3893534327 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+3915 (secure 0 0 0) 0x7f325c12ef60 con 0x7f326415f400 2024-09-10T21:19:25.374 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-10T21:19:25.374 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":107,"btime":"2024-09-10T21:19:11:956616+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1121375070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6833","nonce":1121375070},{"type":"v1","addr":"172.21.15.52:6835","nonce":1121375070}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7837,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6834/1992189310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1992189310},{"type":"v1","addr":"172.21.15.52:6834","nonce":1992189310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":8247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6836/3443454578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3443454578},{"type":"v1","addr":"172.21.15.16:6836","nonce":3443454578}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":8250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1848135983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6835","nonce":1848135983},{"type":"v1","addr":"172.21.15.16:6837","nonce":1848135983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[]} 2024-09-10T21:19:25.374 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 107 2024-09-10T21:19:25.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.371+0000 7f326a800640 1 -- 172.21.15.16:0/3893534327 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f32400725e0 msgr2=0x7f3240074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:25.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.371+0000 7f326a800640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f32400725e0 0x7f3240074aa0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3254001640 tx=0x7f3254009040 comp rx=0 tx=0).stop 2024-09-10T21:19:25.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.371+0000 7f326a800640 1 -- 172.21.15.16:0/3893534327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f326415f400 msgr2=0x7f3264164b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:25.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.371+0000 7f326a800640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f326415f400 0x7f3264164b60 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f325c009bd0 tx=0x7f325c0a5a40 comp rx=0 tx=0).stop 2024-09-10T21:19:25.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.371+0000 7f326a800640 1 -- 172.21.15.16:0/3893534327 shutdown_connections 2024-09-10T21:19:25.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.371+0000 7f326a800640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.16:6800/462868332,v1:172.21.15.16:6801/462868332] conn(0x7f32400725e0 0x7f3240074aa0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:25.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.371+0000 7f326a800640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f326415f400 0x7f3264164b60 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:25.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.371+0000 7f326a800640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.52:3301/0,v1:172.21.15.52:6790/0] conn(0x7f32640b13c0 0x7f326406ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:25.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.371+0000 7f326a800640 1 --2- 172.21.15.16:0/3893534327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32640716f0 0x7f32640b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:25.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.371+0000 7f326a800640 1 -- 172.21.15.16:0/3893534327 >> 172.21.15.16:0/3893534327 conn(0x7f326409b130 msgr2=0x7f32640d06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:25.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.371+0000 7f326a800640 1 -- 172.21.15.16:0/3893534327 shutdown_connections 2024-09-10T21:19:25.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-10T21:19:25.371+0000 7f326a800640 1 -- 172.21.15.16:0/3893534327 wait complete. 2024-09-10T21:19:25.395 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-10T21:19:25.396 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T21:19:25.396 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:19:25.396 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:25.407 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:19:25.408 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:19:25.408 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:25.462 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:19:25.462 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:19:25.462 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:25.477 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:19:25.477 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:19:25.477 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:25.536 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:19:25.537 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T21:19:25.537 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T21:19:25.537 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:25.627 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T21:19:25.628 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T21:19:25.628 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:25.728 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T21:19:25.729 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T21:19:25.729 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:25.826 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T21:19:25.827 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-10T21:19:25.827 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:30.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-10T21:19:31.928 INFO:tasks.ceph.osd.3:Stopped 2024-09-10T21:19:31.929 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-10T21:19:31.929 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:36.461 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-10T21:19:36.461 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-10T21:19:38.030 INFO:tasks.ceph.osd.4:Stopped 2024-09-10T21:19:38.031 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-10T21:19:38.031 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:38.094 INFO:tasks.ceph.osd.5:Stopped 2024-09-10T21:19:38.094 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-10T21:19:38.094 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:38.175 INFO:tasks.ceph.osd.6:Stopped 2024-09-10T21:19:38.175 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-10T21:19:38.175 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:38.265 INFO:tasks.ceph.osd.7:Stopped 2024-09-10T21:19:38.266 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T21:19:38.266 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-10T21:19:38.266 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:38.332 INFO:tasks.ceph.mgr.y:Stopped 2024-09-10T21:19:38.332 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T21:19:38.333 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:38.390 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T21:19:38.390 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T21:19:38.391 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:19:38.391 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:38.412 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:19:38.412 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T21:19:38.412 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:38.429 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T21:19:38.429 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T21:19:38.429 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:19:38.488 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T21:19:38.488 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T21:19:38.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2024-09-10T21:19:38.534 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:19:38.534 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T21:19:38.879 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:19:38.879 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T21:19:39.028 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:19:39.028 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T21:19:39.177 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:19:39.177 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-10T21:19:39.444 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:19:39.444 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-10T21:19:39.928 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:19:39.928 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-10T21:19:40.185 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:19:40.185 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-10T21:19:40.383 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:19:40.383 DEBUG:teuthology.orchestra.run.smithi052:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-10T21:19:40.616 INFO:tasks.ceph:Archiving mon data... 2024-09-10T21:19:40.617 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899658/data/mon.a.tgz 2024-09-10T21:19:40.617 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-10T21:19:40.627 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.wtJsgqxR0Z 2024-09-10T21:19:40.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.wtJsgqxR0Z 2024-09-10T21:19:40.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.wtJsgqxR0Z 2024-09-10T21:19:40.848 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.wtJsgqxR0Z is 411KB 2024-09-10T21:19:40.913 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.wtJsgqxR0Z 2024-09-10T21:19:40.922 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899658/data/mon.b.tgz 2024-09-10T21:19:40.922 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-09-10T21:19:40.931 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.aqjenXaxnr 2024-09-10T21:19:40.931 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.aqjenXaxnr 2024-09-10T21:19:41.077 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0666 /tmp/tmp.aqjenXaxnr 2024-09-10T21:19:41.151 DEBUG:teuthology.orchestra.remote:smithi052:/tmp/tmp.aqjenXaxnr is 413KB 2024-09-10T21:19:41.213 DEBUG:teuthology.orchestra.run.smithi052:> rm -fr /tmp/tmp.aqjenXaxnr 2024-09-10T21:19:41.222 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899658/data/mon.c.tgz 2024-09-10T21:19:41.222 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-09-10T21:19:41.271 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.7uMKeZ7hQg 2024-09-10T21:19:41.272 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.7uMKeZ7hQg 2024-09-10T21:19:41.411 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0666 /tmp/tmp.7uMKeZ7hQg 2024-09-10T21:19:41.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-10T21:19:41.488 DEBUG:teuthology.orchestra.remote:smithi052:/tmp/tmp.7uMKeZ7hQg is 414KB 2024-09-10T21:19:41.553 DEBUG:teuthology.orchestra.run.smithi052:> rm -fr /tmp/tmp.7uMKeZ7hQg 2024-09-10T21:19:41.562 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T21:19:41.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T21:19:41.565 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T21:19:41.635 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T21:19:41.704 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T21:19:41.704 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T21:19:41.706 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899658/remote/smithi016/crash 2024-09-10T21:19:41.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T21:19:41.728 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899658/remote/smithi052/crash 2024-09-10T21:19:41.729 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T21:19:41.765 INFO:tasks.ceph:Shutting down logrotate 2024-09-10T21:19:41.765 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:19:41.773 DEBUG:teuthology.orchestra.run.smithi052:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:19:41.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T21:19:41.877 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T21:19:41.917 INFO:tasks.ceph:Compressing logs... 2024-09-10T21:19:41.918 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T21:19:41.921 DEBUG:teuthology.orchestra.run.smithi052:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T21:19:41.956 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28880.log 2024-09-10T21:19:41.957 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44537.log 2024-09-10T21:19:41.957 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43077.log 2024-09-10T21:19:41.957 INFO:teuthology.orchestra.run.smithi016.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.28880.log.gz 2024-09-10T21:19:41.958 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47123.log 2024-09-10T21:19:41.958 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44537.log.gz 2024-09-10T21:19:41.959 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48123.log 2024-09-10T21:19:41.959 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43077.log.gz 2024-09-10T21:19:41.961 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46863.log 2024-09-10T21:19:41.961 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47123.log.gz 2024-09-10T21:19:41.961 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35441.log 2024-09-10T21:19:41.961 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48123.log.gz 2024-09-10T21:19:41.961 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39313.log 2024-09-10T21:19:41.962 INFO:teuthology.orchestra.run.smithi016.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46863.log.gz 2024-09-10T21:19:41.963 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46294.log 2024-09-10T21:19:41.964 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35441.log.gz 2024-09-10T21:19:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28892.log 2024-09-10T21:19:41.964 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39313.log.gz 2024-09-10T21:19:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49556.log 2024-09-10T21:19:41.965 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46294.log.gz 2024-09-10T21:19:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41878.log 2024-09-10T21:19:41.965 INFO:teuthology.orchestra.run.smithi016.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28892.log.gz 2024-09-10T21:19:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39156.log 2024-09-10T21:19:41.966 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49556.log.gz 2024-09-10T21:19:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49013.log 2024-09-10T21:19:41.967 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41878.log.gz 2024-09-10T21:19:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49866.log 2024-09-10T21:19:41.967 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39156.log.gz 2024-09-10T21:19:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41904.log 2024-09-10T21:19:41.969 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49013.log.gz 2024-09-10T21:19:41.969 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36996.log 2024-09-10T21:19:41.969 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49866.log.gz 2024-09-10T21:19:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38035.log 2024-09-10T21:19:41.970 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41904.log.gz 2024-09-10T21:19:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36937.log 2024-09-10T21:19:41.971 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36996.log.gz 2024-09-10T21:19:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47713.log 2024-09-10T21:19:41.972 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38035.log.gz 2024-09-10T21:19:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37957.log 2024-09-10T21:19:41.973 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36937.log.gz 2024-09-10T21:19:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47713.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-10T21:19:41.973 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47713.log.gz 2024-09-10T21:19:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46837.log 2024-09-10T21:19:41.974 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37957.log.gz 2024-09-10T21:19:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37175.log 2024-09-10T21:19:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45171.log 2024-09-10T21:19:41.976 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46837.log.gz 2024-09-10T21:19:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34404.log 2024-09-10T21:19:41.977 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37175.log.gz 2024-09-10T21:19:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36405.log 2024-09-10T21:19:41.978 INFO:teuthology.orchestra.run.smithi016.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.45171.log.gz 2024-09-10T21:19:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34404.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34404.log.gz 2024-09-10T21:19:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48330.log 2024-09-10T21:19:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36304.log 2024-09-10T21:19:41.980 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36405.log.gz 2024-09-10T21:19:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39182.log 2024-09-10T21:19:41.980 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48330.log.gz 2024-09-10T21:19:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37879.log 2024-09-10T21:19:41.981 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36304.log.gz 2024-09-10T21:19:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39182.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39182.log.gz 2024-09-10T21:19:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35637.log 2024-09-10T21:19:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28736.log 2024-09-10T21:19:41.983 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37879.log.gz 2024-09-10T21:19:41.983 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44785.log 2024-09-10T21:19:41.983 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35637.log.gz 2024-09-10T21:19:41.984 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35743.log 2024-09-10T21:19:41.984 INFO:teuthology.orchestra.run.smithi016.stderr: 74.5% -- replaced with /var/log/ceph/ceph-client.admin.28736.log.gz 2024-09-10T21:19:41.985 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28871.log 2024-09-10T21:19:41.985 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40221.log 2024-09-10T21:19:41.986 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28919.log 2024-09-10T21:19:41.986 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44785.log.gz 2024-09-10T21:19:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37415.log 2024-09-10T21:19:41.986 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35743.log.gz 2024-09-10T21:19:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43285.log 2024-09-10T21:19:41.989 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40221.log.gz 2024-09-10T21:19:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44988.log 2024-09-10T21:19:41.989 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37415.log.gz 2024-09-10T21:19:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44383.log 2024-09-10T21:19:41.989 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43285.log.gz 2024-09-10T21:19:41.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.28871.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-10T21:19:41.990 INFO:teuthology.orchestra.run.smithi052.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.28871.log.gz 2024-09-10T21:19:41.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.28919.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-10T21:19:41.990 INFO:teuthology.orchestra.run.smithi052.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.28919.log.gz 2024-09-10T21:19:41.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29046.log 2024-09-10T21:19:41.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29022.log 2024-09-10T21:19:41.991 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.29046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28950.log 2024-09-10T21:19:41.991 INFO:teuthology.orchestra.run.smithi052.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29046.log.gz 2024-09-10T21:19:41.991 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.29022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34539.log 2024-09-10T21:19:41.991 INFO:teuthology.orchestra.run.smithi052.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29022.log.gz 2024-09-10T21:19:41.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.28950.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28950.log.gz 2024-09-10T21:19:41.994 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-10T21:19:41.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.1.34539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34253.log 2024-09-10T21:19:41.994 INFO:teuthology.orchestra.run.smithi052.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34539.log.gz 2024-09-10T21:19:41.994 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33669.log 2024-09-10T21:19:41.995 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.1.34253.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34253.log.gz 2024-09-10T21:19:41.995 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33944.log 2024-09-10T21:19:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40195.log 2024-09-10T21:19:41.997 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44988.log.gz 2024-09-10T21:19:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38744.log 2024-09-10T21:19:41.997 INFO:teuthology.orchestra.run.smithi016.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44383.log.gz 2024-09-10T21:19:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34120.log 2024-09-10T21:19:41.997 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40195.log.gz 2024-09-10T21:19:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47791.log 2024-09-10T21:19:41.997 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38744.log.gz 2024-09-10T21:19:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33832.log 2024-09-10T21:19:41.997 INFO:teuthology.orchestra.run.smithi016.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34120.log.gz 2024-09-10T21:19:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44331.log 2024-09-10T21:19:41.997 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47791.log.gz 2024-09-10T21:19:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49530.log 2024-09-10T21:19:41.997 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33832.log.gz 2024-09-10T21:19:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42118.log 2024-09-10T21:19:41.998 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44331.log.gz 2024-09-10T21:19:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45614.log 2024-09-10T21:19:41.998 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49530.log.gz 2024-09-10T21:19:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40546.log 2024-09-10T21:19:42.002 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42118.log.gz 2024-09-10T21:19:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47176.log 2024-09-10T21:19:42.002 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45614.log.gz 2024-09-10T21:19:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40546.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T21:19:42.002 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40546.log.gz 2024-09-10T21:19:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41499.log 2024-09-10T21:19:42.002 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47176.log.gz 2024-09-10T21:19:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45380.log 2024-09-10T21:19:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43233.log 2024-09-10T21:19:42.003 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41499.log.gz 2024-09-10T21:19:42.003 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34663.log 2024-09-10T21:19:42.003 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.1.33669.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33669.log.gz 2024-09-10T21:19:42.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.1.33944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33944.log.gz 2024-09-10T21:19:42.004 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T21:19:42.004 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28974.log 2024-09-10T21:19:42.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.1.34663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34663.log.gz 2024-09-10T21:19:42.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T21:19:42.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.28974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28895.log 2024-09-10T21:19:42.004 INFO:teuthology.orchestra.run.smithi052.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28974.log.gz 2024-09-10T21:19:42.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T21:19:42.004 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.28895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33806.log 2024-09-10T21:19:42.004 INFO:teuthology.orchestra.run.smithi052.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.28895.log.gz 2024-09-10T21:19:42.005 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T21:19:42.007 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.1.33806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33806.log.gz 2024-09-10T21:19:42.007 INFO:teuthology.orchestra.run.smithi052.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T21:19:42.007 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28998.log 2024-09-10T21:19:42.007 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-10T21:19:42.007 INFO:teuthology.orchestra.run.smithi052.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T21:19:42.007 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.28998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34090.log 2024-09-10T21:19:42.007 INFO:teuthology.orchestra.run.smithi052.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.28998.log.gz 2024-09-10T21:19:42.010 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43645.log 2024-09-10T21:19:42.010 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45380.log.gz 2024-09-10T21:19:42.010 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36728.log 2024-09-10T21:19:42.010 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43233.log.gz 2024-09-10T21:19:42.010 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37121.log 2024-09-10T21:19:42.011 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43645.log.gz 2024-09-10T21:19:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44024.log 2024-09-10T21:19:42.011 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36728.log.gz 2024-09-10T21:19:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48642.log 2024-09-10T21:19:42.011 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37121.log.gz 2024-09-10T21:19:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47389.log 2024-09-10T21:19:42.011 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44024.log.gz 2024-09-10T21:19:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45067.log 2024-09-10T21:19:42.011 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48642.log.gz 2024-09-10T21:19:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33360.log 2024-09-10T21:19:42.011 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47389.log.gz 2024-09-10T21:19:42.013 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-10T21:19:42.013 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.1.34090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34090.log.gz 2024-09-10T21:19:42.013 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-mds.b.log: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T21:19:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33268.log 2024-09-10T21:19:42.015 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45067.log.gz 2024-09-10T21:19:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33360.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33360.log.gz 2024-09-10T21:19:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38962.log 2024-09-10T21:19:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36006.log 2024-09-10T21:19:42.015 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33268.log.gz 2024-09-10T21:19:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38962.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38962.log.gz 2024-09-10T21:19:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48278.log 2024-09-10T21:19:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50074.log 2024-09-10T21:19:42.015 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36006.log.gz 2024-09-10T21:19:42.017 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T21:19:42.017 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34809.log 2024-09-10T21:19:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46941.log 2024-09-10T21:19:42.018 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48278.log.gz 2024-09-10T21:19:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38995.log 2024-09-10T21:19:42.018 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50074.log.gz 2024-09-10T21:19:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46385.log 2024-09-10T21:19:42.018 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46941.log.gz 2024-09-10T21:19:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33517.log 2024-09-10T21:19:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.1.34809.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-osd.4.log -- replaced with /var/log/ceph/ceph-client.1.34809.log.gz 2024-09-10T21:19:42.021 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-09-10T21:19:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33265.log 2024-09-10T21:19:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.1.33517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33517.log.gz 2024-09-10T21:19:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34397.log 2024-09-10T21:19:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42968.log 2024-09-10T21:19:42.022 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38995.log.gz 2024-09-10T21:19:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40093.log 2024-09-10T21:19:42.022 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46385.log.gz 2024-09-10T21:19:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47415.log 2024-09-10T21:19:42.022 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42968.log.gz 2024-09-10T21:19:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.40093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40093.log.gz 2024-09-10T21:19:42.022 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46915.log 2024-09-10T21:19:42.023 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.1.33265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33265.log.gz 2024-09-10T21:19:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33369.log 2024-09-10T21:19:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39025.log 2024-09-10T21:19:42.024 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47415.log.gz 2024-09-10T21:19:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39469.log 2024-09-10T21:19:42.025 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46915.log.gz 2024-09-10T21:19:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39866.log 2024-09-10T21:19:42.026 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.1.34397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34397.log.gz 2024-09-10T21:19:42.026 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39025.log.gz 2024-09-10T21:19:42.026 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40416.log 2024-09-10T21:19:42.026 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39469.log.gz 2024-09-10T21:19:42.027 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42578.log 2024-09-10T21:19:42.027 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39866.log.gz 2024-09-10T21:19:42.027 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.3.33369.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33369.log.gz 2024-09-10T21:19:42.028 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38770.log 2024-09-10T21:19:42.028 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40416.log.gz 2024-09-10T21:19:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37571.log 2024-09-10T21:19:42.029 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42578.log.gz 2024-09-10T21:19:42.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38770.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38770.log.gz 2024-09-10T21:19:42.030 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49295.log 2024-09-10T21:19:42.030 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45699.log 2024-09-10T21:19:42.031 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37571.log.gz 2024-09-10T21:19:42.032 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49295.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49295.log.gz 2024-09-10T21:19:42.032 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38165.log 2024-09-10T21:19:42.032 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37387.log 2024-09-10T21:19:42.033 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45699.log.gz 2024-09-10T21:19:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45920.log 2024-09-10T21:19:42.034 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38165.log.gz 2024-09-10T21:19:42.034 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29180.log 2024-09-10T21:19:42.035 INFO:teuthology.orchestra.run.smithi016.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37387.log.gz 2024-09-10T21:19:42.036 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47739.log 2024-09-10T21:19:42.036 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45920.log.gz 2024-09-10T21:19:42.037 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36084.log 2024-09-10T21:19:42.037 INFO:teuthology.orchestra.run.smithi016.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29180.log.gz 2024-09-10T21:19:42.038 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44922.log 2024-09-10T21:19:42.038 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47739.log.gz 2024-09-10T21:19:42.039 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34300.log 2024-09-10T21:19:42.039 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36084.log.gz 2024-09-10T21:19:42.040 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37983.log 2024-09-10T21:19:42.040 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44922.log.gz 2024-09-10T21:19:42.041 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39339.log 2024-09-10T21:19:42.041 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34300.log.gz 2024-09-10T21:19:42.042 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47849.log 2024-09-10T21:19:42.042 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37983.log.gz 2024-09-10T21:19:42.043 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49231.log 2024-09-10T21:19:42.043 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39339.log.gz 2024-09-10T21:19:42.044 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33334.log 2024-09-10T21:19:42.044 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47849.log.gz 2024-09-10T21:19:42.045 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39788.log 2024-09-10T21:19:42.045 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49231.log.gz 2024-09-10T21:19:42.046 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41769.log 2024-09-10T21:19:42.046 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33334.log.gz 2024-09-10T21:19:42.046 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40520.log 2024-09-10T21:19:42.047 INFO:teuthology.orchestra.run.smithi016.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.39788.log.gz 2024-09-10T21:19:42.048 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29321.log 2024-09-10T21:19:42.048 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41769.log.gz 2024-09-10T21:19:42.049 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35558.log 2024-09-10T21:19:42.049 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40520.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40520.log.gz 2024-09-10T21:19:42.050 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35253.log 2024-09-10T21:19:42.050 INFO:teuthology.orchestra.run.smithi016.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29321.log.gz 2024-09-10T21:19:42.051 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35558.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35558.log.gz 2024-09-10T21:19:42.051 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47259.log 2024-09-10T21:19:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43697.log 2024-09-10T21:19:42.052 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35253.log.gz 2024-09-10T21:19:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48779.log 2024-09-10T21:19:42.053 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47259.log.gz 2024-09-10T21:19:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42734.log 2024-09-10T21:19:42.054 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43697.log.gz 2024-09-10T21:19:42.055 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48779.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48779.log.gz 2024-09-10T21:19:42.055 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39892.log 2024-09-10T21:19:42.056 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35506.log 2024-09-10T21:19:42.056 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42734.log.gz 2024-09-10T21:19:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33386.log 2024-09-10T21:19:42.057 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39892.log.gz 2024-09-10T21:19:42.058 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39651.log 2024-09-10T21:19:42.058 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35506.log.gz 2024-09-10T21:19:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43311.log 2024-09-10T21:19:42.059 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33386.log.gz 2024-09-10T21:19:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37931.log 2024-09-10T21:19:42.060 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39651.log.gz 2024-09-10T21:19:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35119.log 2024-09-10T21:19:42.061 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43311.log.gz 2024-09-10T21:19:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37597.log 2024-09-10T21:19:42.062 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37931.log.gz 2024-09-10T21:19:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39078.log 2024-09-10T21:19:42.063 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35119.log.gz 2024-09-10T21:19:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50126.log 2024-09-10T21:19:42.064 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.37597.log.gz 2024-09-10T21:19:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40860.log 2024-09-10T21:19:42.065 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39078.log.gz 2024-09-10T21:19:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.50126.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42656.log 2024-09-10T21:19:42.066 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50126.log.gz 2024-09-10T21:19:42.067 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33414.log 2024-09-10T21:19:42.067 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40860.log.gz 2024-09-10T21:19:42.068 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46740.log 2024-09-10T21:19:42.068 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42656.log.gz 2024-09-10T21:19:42.069 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38113.log 2024-09-10T21:19:42.069 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33414.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33414.log.gz 2024-09-10T21:19:42.070 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50022.log 2024-09-10T21:19:42.070 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46740.log.gz 2024-09-10T21:19:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37028.log 2024-09-10T21:19:42.071 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38113.log.gz 2024-09-10T21:19:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49374.log 2024-09-10T21:19:42.072 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50022.log.gz 2024-09-10T21:19:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41931.log 2024-09-10T21:19:42.073 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37028.log.gz 2024-09-10T21:19:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36261.log 2024-09-10T21:19:42.074 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49374.log.gz 2024-09-10T21:19:42.074 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41931.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41931.log.gz 2024-09-10T21:19:42.074 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43541.log 2024-09-10T21:19:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.36261.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.0.36261.log.gz -5 2024-09-10T21:19:42.075 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T21:19:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43181.log 2024-09-10T21:19:42.076 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43541.log.gz 2024-09-10T21:19:42.077 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34869.log 2024-09-10T21:19:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34113.log 2024-09-10T21:19:42.078 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43181.log.gz 2024-09-10T21:19:42.079 INFO:teuthology.orchestra.run.smithi016.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-10T21:19:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35876.log 2024-09-10T21:19:42.079 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34869.log.gz 2024-09-10T21:19:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36488.log 2024-09-10T21:19:42.080 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34113.log.gz 2024-09-10T21:19:42.081 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43129.log 2024-09-10T21:19:42.081 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35876.log.gz 2024-09-10T21:19:42.082 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49265.log 2024-09-10T21:19:42.082 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36488.log.gz 2024-09-10T21:19:42.083 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33901.log 2024-09-10T21:19:42.083 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43129.log.gz 2024-09-10T21:19:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49265.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49265.log.gz 2024-09-10T21:19:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49996.log 2024-09-10T21:19:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48018.log 2024-09-10T21:19:42.085 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33901.log.gz 2024-09-10T21:19:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35532.log 2024-09-10T21:19:42.085 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49996.log.gz 2024-09-10T21:19:42.086 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33494.log 2024-09-10T21:19:42.086 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48018.log.gz 2024-09-10T21:19:42.087 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35532.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35532.log.gz 2024-09-10T21:19:42.087 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49400.log 2024-09-10T21:19:42.088 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45640.log 2024-09-10T21:19:42.088 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33494.log.gz 2024-09-10T21:19:42.089 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35227.log 2024-09-10T21:19:42.089 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49400.log.gz 2024-09-10T21:19:42.090 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49117.log 2024-09-10T21:19:42.090 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45640.log.gz 2024-09-10T21:19:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49426.log 2024-09-10T21:19:42.093 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35227.log.gz 2024-09-10T21:19:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47058.log 2024-09-10T21:19:42.093 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49117.log.gz 2024-09-10T21:19:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49426.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49426.log.gz 2024-09-10T21:19:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36431.log 2024-09-10T21:19:42.094 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28772.log 2024-09-10T21:19:42.094 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47058.log.gz 2024-09-10T21:19:42.094 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46559.log 2024-09-10T21:19:42.095 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.36431.log.gz 2024-09-10T21:19:42.095 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28772.log: 75.5% -- replaced with /var/log/ceph/ceph-client.admin.28772.log.gz 2024-09-10T21:19:42.096 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45786.log 2024-09-10T21:19:42.096 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40995.log 2024-09-10T21:19:42.096 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46559.log.gz 2024-09-10T21:19:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37493.log 2024-09-10T21:19:42.097 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45786.log.gz 2024-09-10T21:19:42.098 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40995.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T21:19:42.098 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40995.log.gz 2024-09-10T21:19:42.099 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36136.log 2024-09-10T21:19:42.099 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37493.log.gz 2024-09-10T21:19:42.100 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42708.log 2024-09-10T21:19:42.101 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36136.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36136.log.gz 2024-09-10T21:19:42.101 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42994.log 2024-09-10T21:19:42.102 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39502.log 2024-09-10T21:19:42.102 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42708.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42708.log.gz 2024-09-10T21:19:42.103 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38139.log 2024-09-10T21:19:42.103 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42994.log.gz 2024-09-10T21:19:42.104 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.34789.log 2024-09-10T21:19:42.104 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39502.log.gz 2024-09-10T21:19:42.105 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38139.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38139.log.gz 2024-09-10T21:19:42.105 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44758.log 2024-09-10T21:19:42.106 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36598.log 2024-09-10T21:19:42.106 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.2.34789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.34789.log.gz 2024-09-10T21:19:42.107 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44758.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44758.log.gz 2024-09-10T21:19:42.107 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41473.log 2024-09-10T21:19:42.108 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37467.log 2024-09-10T21:19:42.108 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36598.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36598.log.gz 2024-09-10T21:19:42.109 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41473.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41473.log.gz 2024-09-10T21:19:42.109 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37335.log 2024-09-10T21:19:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42630.log 2024-09-10T21:19:42.110 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37467.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37467.log.gz 2024-09-10T21:19:42.111 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37335.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37335.log.gz 2024-09-10T21:19:42.111 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41305.log 2024-09-10T21:19:42.112 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44601.log/var/log/ceph/ceph-client.admin.42630.log: 2024-09-10T21:19:42.112 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42630.log.gz 2024-09-10T21:19:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41305.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41305.log.gz 2024-09-10T21:19:42.113 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35357.log 2024-09-10T21:19:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43829.log 2024-09-10T21:19:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44601.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44601.log.gz 2024-09-10T21:19:42.115 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35357.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35357.log.gz 2024-09-10T21:19:42.115 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39918.log 2024-09-10T21:19:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48961.log 2024-09-10T21:19:42.116 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43829.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43829.log.gz 2024-09-10T21:19:42.117 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39918.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39918.log.gz 2024-09-10T21:19:42.117 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42760.log 2024-09-10T21:19:42.118 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33235.log 2024-09-10T21:19:42.118 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48961.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48961.log.gz 2024-09-10T21:19:42.119 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34973.log 87.5% 2024-09-10T21:19:42.119 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42760.log.gz 2024-09-10T21:19:42.120 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.33235.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36572.log 2024-09-10T21:19:42.120 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33235.log.gz 2024-09-10T21:19:42.121 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34973.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34973.log.gz 2024-09-10T21:19:42.121 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38282.log 2024-09-10T21:19:42.122 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33129.log 2024-09-10T21:19:42.122 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36572.log: /var/log/ceph/ceph-client.admin.38282.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36572.log.gz 2024-09-10T21:19:42.123 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38282.log.gz 2024-09-10T21:19:42.123 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37147.log 2024-09-10T21:19:42.124 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50100.log 2024-09-10T21:19:42.124 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33129.log.gz 2024-09-10T21:19:42.125 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37147.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37147.log.gz 2024-09-10T21:19:42.125 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37827.log 2024-09-10T21:19:42.126 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47205.log 2024-09-10T21:19:42.126 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50100.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50100.log.gz 2024-09-10T21:19:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37827.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37827.log.gz 2024-09-10T21:19:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41073.log 2024-09-10T21:19:42.128 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-osd.4.log: 93.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-10T21:19:42.128 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41177.log 2024-09-10T21:19:42.129 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47205.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47205.log.gz 2024-09-10T21:19:42.129 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41073.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41073.log.gz 2024-09-10T21:19:42.129 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48356.log 2024-09-10T21:19:42.130 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41177.log: /var/log/ceph/ceph-client.admin.50048.log 2024-09-10T21:19:42.131 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41177.log.gz 2024-09-10T21:19:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48356.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48356.log.gz 2024-09-10T21:19:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41125.log 2024-09-10T21:19:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48486.log 2024-09-10T21:19:42.132 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50048.log.gz 2024-09-10T21:19:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49814.log 2024-09-10T21:19:42.133 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41125.log.gz 2024-09-10T21:19:42.134 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47987.log 2024-09-10T21:19:42.134 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48486.log.gz 2024-09-10T21:19:42.135 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49609.log/var/log/ceph/ceph-client.admin.49814.log: 2024-09-10T21:19:42.135 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49814.log.gz 2024-09-10T21:19:42.136 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36032.log 2024-09-10T21:19:42.136 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47987.log.gz 2024-09-10T21:19:42.137 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43155.log 2024-09-10T21:19:42.137 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49609.log.gz 2024-09-10T21:19:42.138 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37711.log 2024-09-10T21:19:42.138 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36032.log.gz 2024-09-10T21:19:42.139 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34118.log 2024-09-10T21:19:42.139 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43155.log.gz 2024-09-10T21:19:42.140 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33468.log 2024-09-10T21:19:42.140 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37711.log.gz 2024-09-10T21:19:42.141 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.34118.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34703.log 2024-09-10T21:19:42.142 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34118.log.gz 2024-09-10T21:19:42.142 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40254.log 2024-09-10T21:19:42.142 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33468.log.gz 2024-09-10T21:19:42.143 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43434.log 2024-09-10T21:19:42.143 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34703.log.gz 2024-09-10T21:19:42.144 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.40254.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35383.log 2024-09-10T21:19:42.145 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40254.log.gz 2024-09-10T21:19:42.146 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.43434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43434.log.gz 2024-09-10T21:19:42.146 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34947.log 2024-09-10T21:19:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35383.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35383.log.gz 2024-09-10T21:19:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47363.log 2024-09-10T21:19:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.34947.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49583.log 2024-09-10T21:19:42.148 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34947.log.gz 2024-09-10T21:19:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46889.log 2024-09-10T21:19:42.149 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47363.log.gz 2024-09-10T21:19:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49583.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.49583.log.gz --verbose 2024-09-10T21:19:42.150 INFO:teuthology.orchestra.run.smithi016.stderr: -- /var/log/ceph/ceph-client.admin.40572.log 2024-09-10T21:19:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35409.log 2024-09-10T21:19:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46889.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46889.log.gz 2024-09-10T21:19:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37095.log 2024-09-10T21:19:42.152 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40572.log.gz 2024-09-10T21:19:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43777.log 2024-09-10T21:19:42.153 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35409.log.gz 2024-09-10T21:19:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45671.log 2024-09-10T21:19:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37095.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37095.log.gz 2024-09-10T21:19:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50230.log 2024-09-10T21:19:42.155 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43777.log.gz 2024-09-10T21:19:42.156 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33650.log 2024-09-10T21:19:42.156 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45671.log.gz 2024-09-10T21:19:42.157 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35980.log 2024-09-10T21:19:42.157 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50230.log.gz 2024-09-10T21:19:42.158 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42225.log 2024-09-10T21:19:42.158 INFO:teuthology.orchestra.run.smithi016.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.33650.log.gz 2024-09-10T21:19:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49167.log 2024-09-10T21:19:42.159 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35980.log.gz 2024-09-10T21:19:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.42225.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28796.log 2024-09-10T21:19:42.161 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42225.log.gz 2024-09-10T21:19:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39104.log 2024-09-10T21:19:42.162 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49167.log.gz 2024-09-10T21:19:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28796.log.gz 2024-09-10T21:19:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41395.log 2024-09-10T21:19:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34119.log 2024-09-10T21:19:42.164 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39104.log.gz 2024-09-10T21:19:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48883.log 2024-09-10T21:19:42.165 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41395.log.gz 2024-09-10T21:19:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34456.log 2024-09-10T21:19:42.166 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34119.log.gz 2024-09-10T21:19:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48883.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48883.log.gz 2024-09-10T21:19:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46190.log 2024-09-10T21:19:42.168 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35025.log 2024-09-10T21:19:42.168 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34456.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34456.log.gz 2024-09-10T21:19:42.169 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43671.log 2024-09-10T21:19:42.169 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46190.log.gz 2024-09-10T21:19:42.170 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39680.log 2024-09-10T21:19:42.170 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35025.log.gz 2024-09-10T21:19:42.171 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40702.log 2024-09-10T21:19:42.171 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43671.log: /var/log/ceph/ceph-client.admin.39680.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43671.log.gz 2024-09-10T21:19:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45588.log 2024-09-10T21:19:42.172 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39680.log.gz 2024-09-10T21:19:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49065.log 2024-09-10T21:19:42.173 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40702.log.gz 2024-09-10T21:19:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35824.log 2024-09-10T21:19:42.175 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45588.log: /var/log/ceph/ceph-client.admin.49065.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45588.log.gz 2024-09-10T21:19:42.175 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42942.log 2024-09-10T21:19:42.175 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49065.log.gz 2024-09-10T21:19:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35824.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35824.log.gz 2024-09-10T21:19:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38614.log 2024-09-10T21:19:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34661.log 2024-09-10T21:19:42.178 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42942.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42942.log.gz 2024-09-10T21:19:42.179 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34326.log 2024-09-10T21:19:42.179 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38614.log.gz 2024-09-10T21:19:42.179 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.34661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.34661.log.gz 2024-09-10T21:19:42.180 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39625.log 2024-09-10T21:19:42.180 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5/var/log/ceph/ceph-client.admin.34326.log: --verbose -- /var/log/ceph/ceph-client.admin.46475.log 2024-09-10T21:19:42.181 INFO:teuthology.orchestra.run.smithi016.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34326.log.gz 2024-09-10T21:19:42.182 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48408.log 2024-09-10T21:19:42.182 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39625.log.gz 2024-09-10T21:19:42.183 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39970.log 2024-09-10T21:19:42.183 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46475.log.gz 2024-09-10T21:19:42.183 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42501.log 2024-09-10T21:19:42.184 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48408.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48408.log.gz 2024-09-10T21:19:42.185 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28760.log 2024-09-10T21:19:42.185 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39970.log.gz 2024-09-10T21:19:42.186 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42501.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42501.log.gz -5 2024-09-10T21:19:42.186 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44409.log 2024-09-10T21:19:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36833.log 2024-09-10T21:19:42.187 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28760.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.28760.log.gz 2024-09-10T21:19:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50152.log 2024-09-10T21:19:42.188 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44409.log.gz 2024-09-10T21:19:42.189 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33858.log 2024-09-10T21:19:42.189 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36833.log.gz 2024-09-10T21:19:42.190 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39840.log 2024-09-10T21:19:42.190 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50152.log.gz 2024-09-10T21:19:42.191 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40162.log 2024-09-10T21:19:42.191 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33858.log.gz 2024-09-10T21:19:42.192 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35954.log 2024-09-10T21:19:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39840.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39840.log.gz 2024-09-10T21:19:42.193 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28904.log 2024-09-10T21:19:42.193 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40162.log.gz 2024-09-10T21:19:42.194 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36278.log 2024-09-10T21:19:42.195 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35954.log.gz 2024-09-10T21:19:42.195 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48045.log 2024-09-10T21:19:42.196 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28904.log: 77.2% -- replaced with /var/log/ceph/ceph-client.admin.28904.log.gz 2024-09-10T21:19:42.196 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46164.log 2024-09-10T21:19:42.197 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36278.log.gz 2024-09-10T21:19:42.198 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46138.log 2024-09-10T21:19:42.198 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48045.log.gz 2024-09-10T21:19:42.198 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41691.log 2024-09-10T21:19:42.199 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46164.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46164.log.gz 2024-09-10T21:19:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46138.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46138.log.gz 2024-09-10T21:19:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33598.log 2024-09-10T21:19:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42449.log 2024-09-10T21:19:42.201 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41691.log.gz 2024-09-10T21:19:42.202 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34112.log 2024-09-10T21:19:42.202 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33598.log.gz 2024-09-10T21:19:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37801.log 2024-09-10T21:19:42.203 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42449.log.gz 2024-09-10T21:19:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41352.log 2024-09-10T21:19:42.204 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34112.log.gz 2024-09-10T21:19:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43881.log 2024-09-10T21:19:42.205 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37801.log.gz 2024-09-10T21:19:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.41352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41352.log.gz 2024-09-10T21:19:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-10T21:19:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41021.log 2024-09-10T21:19:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43881.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43881.log.gz 2024-09-10T21:19:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34841.log 2024-09-10T21:19:42.209 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45354.log 2024-09-10T21:19:42.209 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41021.log.gz 2024-09-10T21:19:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40598.log 2024-09-10T21:19:42.210 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34841.log.gz 2024-09-10T21:19:42.211 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34430.log 2024-09-10T21:19:42.211 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45354.log.gz 2024-09-10T21:19:42.212 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49504.log 2024-09-10T21:19:42.213 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40598.log.gz 2024-09-10T21:19:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34999.log 2024-09-10T21:19:42.214 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34430.log.gz 2024-09-10T21:19:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48909.log 2024-09-10T21:19:42.215 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49504.log.gz 2024-09-10T21:19:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42682.log 2024-09-10T21:19:42.216 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34999.log.gz 2024-09-10T21:19:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39286.log 2024-09-10T21:19:42.217 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48909.log.gz 2024-09-10T21:19:42.218 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35771.log 2024-09-10T21:19:42.218 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42682.log.gz 2024-09-10T21:19:42.219 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43337.log 2024-09-10T21:19:42.219 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39286.log.gz 2024-09-10T21:19:42.220 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34096.log 2024-09-10T21:19:42.221 INFO:teuthology.orchestra.run.smithi016.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35771.log.gz 2024-09-10T21:19:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49970.log 2024-09-10T21:19:42.222 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34096.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43337.log.gz 2024-09-10T21:19:42.223 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40136.log 2024-09-10T21:19:42.223 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34096.log.gz 2024-09-10T21:19:42.224 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47928.log 2024-09-10T21:19:42.224 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49970.log.gz 2024-09-10T21:19:42.225 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45842.log 2024-09-10T21:19:42.225 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40136.log.gz 2024-09-10T21:19:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39760.log 2024-09-10T21:19:42.226 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47928.log.gz 2024-09-10T21:19:42.227 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49710.log 2024-09-10T21:19:42.228 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45842.log.gz 2024-09-10T21:19:42.228 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40046.log 2024-09-10T21:19:42.229 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39760.log.gz 2024-09-10T21:19:42.229 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44896.log 2024-09-10T21:19:42.229 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49710.log.gz 2024-09-10T21:19:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35149.log 2024-09-10T21:19:42.231 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40046.log.gz 2024-09-10T21:19:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45119.log 2024-09-10T21:19:42.232 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44896.log.gz 2024-09-10T21:19:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42092.log 2024-09-10T21:19:42.233 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35149.log.gz 2024-09-10T21:19:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34104.log 2024-09-10T21:19:42.234 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45119.log.gz 2024-09-10T21:19:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45458.log 2024-09-10T21:19:42.235 INFO:teuthology.orchestra.run.smithi016.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42092.log.gz 2024-09-10T21:19:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34895.log 2024-09-10T21:19:42.236 INFO:teuthology.orchestra.run.smithi016.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34104.log.gz 2024-09-10T21:19:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45145.log 2024-09-10T21:19:42.237 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45458.log.gz 2024-09-10T21:19:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33904.log 2024-09-10T21:19:42.239 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34895.log.gz 2024-09-10T21:19:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41988.log 2024-09-10T21:19:42.240 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45145.log.gz 2024-09-10T21:19:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39944.log 2024-09-10T21:19:42.241 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33904.log.gz 2024-09-10T21:19:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44144.log 2024-09-10T21:19:42.242 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41988.log.gz 2024-09-10T21:19:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33442.log 2024-09-10T21:19:42.243 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39944.log.gz 2024-09-10T21:19:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37905.log 2024-09-10T21:19:42.244 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44144.log.gz 2024-09-10T21:19:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45523.log 2024-09-10T21:19:42.246 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33442.log.gz 2024-09-10T21:19:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40110.log 2024-09-10T21:19:42.247 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37905.log.gz 2024-09-10T21:19:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48551.log 2024-09-10T21:19:42.248 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45523.log.gz 2024-09-10T21:19:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46112.log 2024-09-10T21:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40110.log.gz 2024-09-10T21:19:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48149.log 2024-09-10T21:19:42.250 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48551.log.gz 2024-09-10T21:19:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46048.log 2024-09-10T21:19:42.251 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46112.log.gz 2024-09-10T21:19:42.252 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44279.log 2024-09-10T21:19:42.252 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48149.log.gz 2024-09-10T21:19:42.253 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35585.log 2024-09-10T21:19:42.253 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46048.log.gz 2024-09-10T21:19:42.254 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42172.log 2024-09-10T21:19:42.254 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44279.log.gz 2024-09-10T21:19:42.255 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36624.log 2024-09-10T21:19:42.256 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35585.log.gz 2024-09-10T21:19:42.256 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41611.log 2024-09-10T21:19:42.257 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42172.log.gz 2024-09-10T21:19:42.258 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34542.log 2024-09-10T21:19:42.258 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36624.log.gz 2024-09-10T21:19:42.259 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49322.log 2024-09-10T21:19:42.259 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41611.log.gz 2024-09-10T21:19:42.260 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34542.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.34542.log.gz 2024-09-10T21:19:42.260 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43593.log 2024-09-10T21:19:42.261 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49322.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T21:19:42.261 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49322.log.gz 2024-09-10T21:19:42.263 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40969.log 2024-09-10T21:19:42.263 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43593.log.gz 2024-09-10T21:19:42.263 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37441.log 2024-09-10T21:19:42.265 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37545.log 2024-09-10T21:19:42.265 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40969.log.gz 2024-09-10T21:19:42.266 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37441.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37441.log.gz 2024-09-10T21:19:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41369.log 2024-09-10T21:19:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37545.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37545.log.gz 2024-09-10T21:19:42.268 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39052.log 2024-09-10T21:19:42.269 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35175.log 2024-09-10T21:19:42.269 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41369.log.gz 2024-09-10T21:19:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39052.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39052.log.gz 2024-09-10T21:19:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28832.log 2024-09-10T21:19:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42786.log 2024-09-10T21:19:42.272 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35175.log.gz 2024-09-10T21:19:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33892.log 2024-09-10T21:19:42.274 INFO:teuthology.orchestra.run.smithi016.stderr: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.28832.log.gz 2024-09-10T21:19:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42786.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42786.log.gz 2024-09-10T21:19:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42552.log 2024-09-10T21:19:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45041.log 2024-09-10T21:19:42.277 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33892.log.gz 2024-09-10T21:19:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38428.log 2024-09-10T21:19:42.278 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42552.log.gz 2024-09-10T21:19:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45041.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45041.log.gz 2024-09-10T21:19:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40468.log 2024-09-10T21:19:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38428.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38428.log.gz 2024-09-10T21:19:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.23861.log 2024-09-10T21:19:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44487.log 2024-09-10T21:19:42.282 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40468.log.gz 2024-09-10T21:19:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.tmp-client.admin.23861.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.23861.log.gz 2024-09-10T21:19:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44487.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44487.log.gz 2024-09-10T21:19:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37744.log 2024-09-10T21:19:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37205.log 2024-09-10T21:19:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36702.log 2024-09-10T21:19:42.288 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37744.log.gz 2024-09-10T21:19:42.289 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37205.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37205.log.gz 2024-09-10T21:19:42.290 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36702.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36702.log.gz 2024-09-10T21:19:42.290 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45224.log 2024-09-10T21:19:42.291 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45015.log 2024-09-10T21:19:42.293 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42604.log 2024-09-10T21:19:42.293 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45224.log.gz 2024-09-10T21:19:42.293 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45015.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45015.log.gz 2024-09-10T21:19:42.295 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42604.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42604.log.gz 2024-09-10T21:19:42.295 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41795.log 2024-09-10T21:19:42.296 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46533.log 2024-09-10T21:19:42.300 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47337.log 2024-09-10T21:19:42.300 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41795.log.gz 2024-09-10T21:19:42.301 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39443.log 2024-09-10T21:19:42.301 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46533.log.gz 2024-09-10T21:19:42.301 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47337.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47337.log.gz 2024-09-10T21:19:42.301 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49684.log 2024-09-10T21:19:42.301 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39443.log.gz 2024-09-10T21:19:42.302 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35663.log 2024-09-10T21:19:42.303 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28726.log 2024-09-10T21:19:42.303 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49684.log.gz 2024-09-10T21:19:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35663.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35663.log.gz 2024-09-10T21:19:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28726.log: 76.7% -- replaced with /var/log/ceph/ceph-client.admin.28726.log.gz 2024-09-10T21:19:42.306 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36650.log 2024-09-10T21:19:42.307 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47150.log 2024-09-10T21:19:42.308 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33676.log 2024-09-10T21:19:42.309 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36650.log.gz 2024-09-10T21:19:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48668.log 2024-09-10T21:19:42.309 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47150.log.gz 2024-09-10T21:19:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33676.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33676.log.gz 2024-09-10T21:19:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41229.log 2024-09-10T21:19:42.312 INFO:teuthology.orchestra.run.smithi052.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T21:19:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48668.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48668.log.gz 2024-09-10T21:19:42.313 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37231.log 2024-09-10T21:19:42.314 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41229.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41229.log.gz 2024-09-10T21:19:42.314 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47618.log 2024-09-10T21:19:42.315 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34815.log 2024-09-10T21:19:42.315 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37231.log.gz 2024-09-10T21:19:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.47618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47618.log.gz 2024-09-10T21:19:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37774.log 2024-09-10T21:19:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43259.log 2024-09-10T21:19:42.317 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34815.log.gz 2024-09-10T21:19:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46637.log 2024-09-10T21:19:42.319 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37774.log.gz 2024-09-10T21:19:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43259.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43259.log.gz 2024-09-10T21:19:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39599.log 2024-09-10T21:19:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46637.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46637.log.gz 2024-09-10T21:19:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45734.log 2024-09-10T21:19:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39734.log 2024-09-10T21:19:42.323 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39599.log.gz 2024-09-10T21:19:42.324 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45734.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45734.log.gz 2024-09-10T21:19:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39734.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39734.log.gz 2024-09-10T21:19:42.326 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43387.log 2024-09-10T21:19:42.327 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43567.log 2024-09-10T21:19:42.328 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46326.log 2024-09-10T21:19:42.328 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43387.log.gz 2024-09-10T21:19:42.329 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46506.log 2024-09-10T21:19:42.330 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43567.log.gz 2024-09-10T21:19:42.331 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46326.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46326.log.gz 2024-09-10T21:19:42.331 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41849.log 2024-09-10T21:19:42.332 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46506.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46506.log.gz 2024-09-10T21:19:42.332 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34352.log 2024-09-10T21:19:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44461.log 2024-09-10T21:19:42.334 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41849.log.gz 2024-09-10T21:19:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34352.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34352.log.gz 2024-09-10T21:19:42.336 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44461.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44461.log.gz 2024-09-10T21:19:42.336 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42475.log 2024-09-10T21:19:42.338 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40442.log 2024-09-10T21:19:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47954.log 2024-09-10T21:19:42.339 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42475.log.gz 2024-09-10T21:19:42.341 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40442.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40442.log.gz 2024-09-10T21:19:42.341 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47954.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47954.log.gz 2024-09-10T21:19:42.342 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35305.log 2024-09-10T21:19:42.343 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46411.log 2024-09-10T21:19:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48252.log 2024-09-10T21:19:42.345 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35305.log.gz 2024-09-10T21:19:42.345 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35715.log 2024-09-10T21:19:42.345 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46411.log.gz 2024-09-10T21:19:42.347 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48252.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48252.log.gz 2024-09-10T21:19:42.347 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41743.log 2024-09-10T21:19:42.348 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35715.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35715.log.gz 2024-09-10T21:19:42.348 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48434.log 2024-09-10T21:19:42.350 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33572.log 2024-09-10T21:19:42.350 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41743.log.gz 2024-09-10T21:19:42.351 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48434.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48434.log.gz 2024-09-10T21:19:42.352 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33572.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33572.log.gz 2024-09-10T21:19:42.352 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48460.log 2024-09-10T21:19:42.353 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46268.log 2024-09-10T21:19:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45328.log 2024-09-10T21:19:42.355 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48460.log.gz 2024-09-10T21:19:42.356 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46268.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46268.log.gz 2024-09-10T21:19:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45328.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45328.log.gz 2024-09-10T21:19:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47902.log 2024-09-10T21:19:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47635.log 2024-09-10T21:19:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40886.log 2024-09-10T21:19:42.360 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47902.log.gz 2024-09-10T21:19:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47635.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47635.log.gz 2024-09-10T21:19:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40886.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40886.log.gz 2024-09-10T21:19:42.363 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34482.log 2024-09-10T21:19:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42344.log 2024-09-10T21:19:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37257.log 2024-09-10T21:19:42.366 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1%/var/log/ceph/ceph-client.admin.42344.log: -- replaced with /var/log/ceph/ceph-client.admin.34482.log.gz 2024-09-10T21:19:42.366 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42344.log.gz 2024-09-10T21:19:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37257.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37257.log.gz 2024-09-10T21:19:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44732.log 2024-09-10T21:19:42.369 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38718.log 2024-09-10T21:19:42.370 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35850.log 2024-09-10T21:19:42.371 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44732.log.gz 2024-09-10T21:19:42.372 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38718.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38718.log.gz 2024-09-10T21:19:42.373 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35850.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35850.log.gz 2024-09-10T21:19:42.373 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28685.log 2024-09-10T21:19:42.374 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37853.log 2024-09-10T21:19:42.376 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28685.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T21:19:42.376 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.28685.log.gz 2024-09-10T21:19:42.377 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42279.log 2024-09-10T21:19:42.377 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37853.log.gz 2024-09-10T21:19:42.380 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.42279.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42279.log.gz 2024-09-10T21:19:42.380 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47519.log 2024-09-10T21:19:42.381 INFO:teuthology.orchestra.run.smithi016.stderr: 86.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T21:19:42.381 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33702.log 2024-09-10T21:19:42.382 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33806.log 2024-09-10T21:19:42.383 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0%/var/log/ceph/ceph-client.admin.33702.log: -- replaced with /var/log/ceph/ceph-client.admin.47519.log.gz 2024-09-10T21:19:42.383 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33702.log.gz 2024-09-10T21:19:42.384 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37309.log 2024-09-10T21:19:42.385 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33806.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33806.log.gz 2024-09-10T21:19:42.385 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40624.log 2024-09-10T21:19:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38536.log 2024-09-10T21:19:42.387 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37309.log.gz 2024-09-10T21:19:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40624.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40624.log.gz 2024-09-10T21:19:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38536.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38536.log.gz 2024-09-10T21:19:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47493.log 2024-09-10T21:19:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43959.log 2024-09-10T21:19:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40364.log 2024-09-10T21:19:42.391 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47493.log.gz 2024-09-10T21:19:42.392 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43959.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43959.log.gz 2024-09-10T21:19:42.393 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40364.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40364.log.gz 2024-09-10T21:19:42.394 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36963.log 2024-09-10T21:19:42.395 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41584.log 2024-09-10T21:19:42.396 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44870.log 2024-09-10T21:19:42.397 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36963.log.gz 2024-09-10T21:19:42.398 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41584.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41584.log.gz 2024-09-10T21:19:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48805.log 2024-09-10T21:19:42.399 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44870.log.gz 2024-09-10T21:19:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48226.log 2024-09-10T21:19:42.401 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46611.log 2024-09-10T21:19:42.402 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48805.log.gz 2024-09-10T21:19:42.403 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48226.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48226.log.gz 2024-09-10T21:19:42.404 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46611.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46611.log.gz 2024-09-10T21:19:42.404 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42397.log 2024-09-10T21:19:42.405 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45946.log 2024-09-10T21:19:42.407 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44173.log 2024-09-10T21:19:42.407 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42397.log.gz 2024-09-10T21:19:42.408 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33906.log 2024-09-10T21:19:42.408 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45946.log.gz 2024-09-10T21:19:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45250.log 2024-09-10T21:19:42.409 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44173.log.gz 2024-09-10T21:19:42.411 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33906.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.33906.log.gz 2024-09-10T21:19:42.411 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47467.log 2024-09-10T21:19:42.412 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45250.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45250.log.gz 2024-09-10T21:19:42.413 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42812.log 2024-09-10T21:19:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38061.log 2024-09-10T21:19:42.414 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47467.log.gz 2024-09-10T21:19:42.415 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44201.log 2024-09-10T21:19:42.416 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42812.log.gz 2024-09-10T21:19:42.416 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41255.log 2024-09-10T21:19:42.416 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38061.log.gz 2024-09-10T21:19:42.418 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44201.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44201.log.gz 2024-09-10T21:19:42.418 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44227.log 2024-09-10T21:19:42.419 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39996.log 2024-09-10T21:19:42.419 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41255.log.gz 2024-09-10T21:19:42.420 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33624.log 2024-09-10T21:19:42.421 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44227.log.gz 2024-09-10T21:19:42.422 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39996.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.39996.log.gz 2024-09-10T21:19:42.423 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33624.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33624.log.gz 2024-09-10T21:19:42.423 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44118.log 2024-09-10T21:19:42.424 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36058.log 2024-09-10T21:19:42.426 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47233.log 2024-09-10T21:19:42.426 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44118.log.gz 2024-09-10T21:19:42.427 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36058.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36058.log.gz 2024-09-10T21:19:42.428 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47233.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47233.log.gz 2024-09-10T21:19:42.429 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36885.log 2024-09-10T21:19:42.430 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35059.log 2024-09-10T21:19:42.432 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38217.log 2024-09-10T21:19:42.432 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36885.log.gz 2024-09-10T21:19:42.433 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47311.log 2024-09-10T21:19:42.433 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35059.log.gz 2024-09-10T21:19:42.434 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45276.log 2024-09-10T21:19:42.434 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38217.log.gz 2024-09-10T21:19:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47311.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47311.log.gz 2024-09-10T21:19:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43514.log 2024-09-10T21:19:42.436 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45276.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45276.log.gz 2024-09-10T21:19:42.437 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38640.log 2024-09-10T21:19:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28748.log 2024-09-10T21:19:42.438 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43514.log.gz 2024-09-10T21:19:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38640.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38640.log.gz 2024-09-10T21:19:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28748.log: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.28748.log.gz 2024-09-10T21:19:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45760.log 2024-09-10T21:19:42.441 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36214.log 2024-09-10T21:19:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45972.log 2024-09-10T21:19:42.443 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45760.log.gz 2024-09-10T21:19:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36214.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36214.log.gz 2024-09-10T21:19:42.445 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45972.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45972.log.gz 2024-09-10T21:19:42.445 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36164.log 2024-09-10T21:19:42.446 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28916.log 2024-09-10T21:19:42.449 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42527.log 2024-09-10T21:19:42.449 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.36164.log.gz 2024-09-10T21:19:42.449 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28916.log: 76.5% -- replaced with /var/log/ceph/ceph-client.admin.28916.log.gz 2024-09-10T21:19:42.450 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42527.log: 83.7% -- replaced with /var/log/ceph/ceph-client.admin.42527.log.gz 2024-09-10T21:19:42.450 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46216.log 2024-09-10T21:19:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41717.log 2024-09-10T21:19:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43051.log 2024-09-10T21:19:42.454 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46216.log.gz 2024-09-10T21:19:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48175.log 2024-09-10T21:19:42.454 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41717.log.gz 2024-09-10T21:19:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43051.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43051.log.gz 2024-09-10T21:19:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42313.log 2024-09-10T21:19:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48175.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48175.log.gz 2024-09-10T21:19:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33728.log 2024-09-10T21:19:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43803.log 2024-09-10T21:19:42.459 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42313.log.gz 2024-09-10T21:19:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33728.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33728.log.gz 2024-09-10T21:19:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43803.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43803.log.gz 2024-09-10T21:19:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39708.log 2024-09-10T21:19:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42040.log 2024-09-10T21:19:42.464 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38250.log 2024-09-10T21:19:42.464 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39708.log.gz 2024-09-10T21:19:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38692.log 2024-09-10T21:19:42.465 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42040.log.gz 2024-09-10T21:19:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38250.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38250.log.gz 2024-09-10T21:19:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43484.log 2024-09-10T21:19:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38692.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38692.log.gz 2024-09-10T21:19:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35090.log 2024-09-10T21:19:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42371.log 2024-09-10T21:19:42.469 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43484.log.gz 2024-09-10T21:19:42.470 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35090.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35090.log.gz 2024-09-10T21:19:42.471 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42371.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42371.log.gz 2024-09-10T21:19:42.472 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39814.log 2024-09-10T21:19:42.473 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28820.log 2024-09-10T21:19:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48382.log 2024-09-10T21:19:42.474 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39814.log.gz 2024-09-10T21:19:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28820.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.28820.log.gz 2024-09-10T21:19:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48382.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48382.log.gz 2024-09-10T21:19:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40390.log 2024-09-10T21:19:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40676.log 2024-09-10T21:19:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50204.log 2024-09-10T21:19:42.480 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40390.log.gz 2024-09-10T21:19:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40676.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40676.log.gz 2024-09-10T21:19:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50204.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50204.log.gz 2024-09-10T21:19:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41447.log 2024-09-10T21:19:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38087.log 2024-09-10T21:19:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33910.log 2024-09-10T21:19:42.485 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41447.log.gz 2024-09-10T21:19:42.485 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38087.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38087.log.gz 2024-09-10T21:19:42.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33910.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33910.log.gz 2024-09-10T21:19:42.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45998.log 2024-09-10T21:19:42.488 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46357.log 2024-09-10T21:19:42.489 INFO:teuthology.orchestra.run.smithi052.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T21:19:42.490 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46811.log 2024-09-10T21:19:42.490 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46357.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45998.log.gz 2024-09-10T21:19:42.491 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46357.log.gz 2024-09-10T21:19:42.492 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46811.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46811.log.gz 2024-09-10T21:19:42.492 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47569.log 2024-09-10T21:19:42.493 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44817.log 2024-09-10T21:19:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33911.log 2024-09-10T21:19:42.496 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47569.log.gz 2024-09-10T21:19:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45093.log 2024-09-10T21:19:42.496 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44817.log.gz 2024-09-10T21:19:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33911.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33911.log.gz 2024-09-10T21:19:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35331.log 2024-09-10T21:19:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42066.log 2024-09-10T21:19:42.499 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45093.log.gz 2024-09-10T21:19:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41421.log 2024-09-10T21:19:42.500 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35331.log.gz 2024-09-10T21:19:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42066.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42066.log.gz 2024-09-10T21:19:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41960.log 2024-09-10T21:19:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41421.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41421.log.gz 2024-09-10T21:19:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36755.log 2024-09-10T21:19:42.504 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46785.log 2024-09-10T21:19:42.504 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41960.log.gz 2024-09-10T21:19:42.505 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36755.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36755.log.gz 2024-09-10T21:19:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46785.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46785.log.gz 2024-09-10T21:19:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49214.log 2024-09-10T21:19:42.508 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37361.log 2024-09-10T21:19:42.509 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.49214.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49214.log.gz 2024-09-10T21:19:42.509 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39365.log 2024-09-10T21:19:42.510 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37361.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37361.log.gz 2024-09-10T21:19:42.511 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39365.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39365.log.gz 2024-09-10T21:19:42.512 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43907.log 2024-09-10T21:19:42.513 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41663.log 2024-09-10T21:19:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44957.log 2024-09-10T21:19:42.515 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43907.log.gz 2024-09-10T21:19:42.515 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48519.log 2024-09-10T21:19:42.515 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41663.log.gz 2024-09-10T21:19:42.517 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44957.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44957.log.gz 2024-09-10T21:19:42.517 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T21:19:42.518 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48519.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48519.log.gz 2024-09-10T21:19:42.518 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28844.log 2024-09-10T21:19:42.520 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49944.log 2024-09-10T21:19:42.521 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28844.log: 76.2% -- replaced with /var/log/ceph/ceph-client.admin.28844.log.gz 2024-09-10T21:19:42.522 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49944.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49944.log.gz 2024-09-10T21:19:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43751.log 2024-09-10T21:19:42.523 INFO:teuthology.orchestra.run.smithi016.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T21:19:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-10T21:19:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40285.log 2024-09-10T21:19:42.525 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43751.log.gz 2024-09-10T21:19:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44584.log 2024-09-10T21:19:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40285.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40285.log.gz 2024-09-10T21:19:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46689.log 2024-09-10T21:19:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.44584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44584.log.gz 2024-09-10T21:19:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49892.log 2024-09-10T21:19:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46689.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.46689.log.gz 2024-09-10T21:19:42.532 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39208.log 2024-09-10T21:19:42.532 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49892.log.gz 2024-09-10T21:19:42.532 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40912.log 2024-09-10T21:19:42.534 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39208.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39208.log.gz 2024-09-10T21:19:42.534 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-10T21:19:42.535 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40912.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40912.log.gz 2024-09-10T21:19:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36807.log 2024-09-10T21:19:42.541 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41637.log 2024-09-10T21:19:42.542 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46437.log 2024-09-10T21:19:42.542 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36807.log.gz 2024-09-10T21:19:42.543 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38796.log 2024-09-10T21:19:42.543 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41637.log.gz 2024-09-10T21:19:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46437.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46437.log.gz 2024-09-10T21:19:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28868.log 2024-09-10T21:19:42.547 INFO:teuthology.orchestra.run.smithi016.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-10T21:19:42.549 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.38796.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38796.log.gz/var/log/ceph/ceph-client.admin.28868.log: 2024-09-10T21:19:42.549 INFO:teuthology.orchestra.run.smithi016.stderr: -5 74.0% -- replaced with /var/log/ceph/ceph-client.admin.28868.log.gz 2024-09-10T21:19:42.549 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45816.log 2024-09-10T21:19:42.551 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38191.log 2024-09-10T21:19:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45816.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45816.log.gz 2024-09-10T21:19:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38191.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38191.log.gz 2024-09-10T21:19:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36911.log 2024-09-10T21:19:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48857.log 2024-09-10T21:19:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36911.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36911.log.gz 2024-09-10T21:19:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48857.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48857.log.gz 2024-09-10T21:19:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-09-10T21:19:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48097.log 2024-09-10T21:19:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42198.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-09-10T21:19:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48097.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48097.log.gz 2024-09-10T21:19:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47817.log 2024-09-10T21:19:42.563 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48697.log 2024-09-10T21:19:42.565 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47817.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47817.log.gz 2024-09-10T21:19:42.566 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48697.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48697.log.gz 2024-09-10T21:19:42.566 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41099.log 2024-09-10T21:19:42.567 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47765.log 2024-09-10T21:19:42.569 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41099.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41099.log.gz 2024-09-10T21:19:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47765.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47765.log.gz 2024-09-10T21:19:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45491.log 2024-09-10T21:19:42.571 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49918.log 2024-09-10T21:19:42.573 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45491.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45491.log.gz 2024-09-10T21:19:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49918.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49918.log.gz 2024-09-10T21:19:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28784.log 2024-09-10T21:19:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49452.log 2024-09-10T21:19:42.577 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28784.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.28784.log.gz 2024-09-10T21:19:42.578 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49452.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49452.log.gz 2024-09-10T21:19:42.578 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44305.log 2024-09-10T21:19:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33520.log 2024-09-10T21:19:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44305.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44305.log.gz 2024-09-10T21:19:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33520.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33520.log.gz 2024-09-10T21:19:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42014.log 2024-09-10T21:19:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45432.log 2024-09-10T21:19:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42014.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42014.log.gz 2024-09-10T21:19:42.586 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45432.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45432.log.gz 2024-09-10T21:19:42.586 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44628.log 2024-09-10T21:19:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38898.log 2024-09-10T21:19:42.590 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44628.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44628.log.gz 2024-09-10T21:19:42.590 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38898.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38898.log.gz 2024-09-10T21:19:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42845.log 2024-09-10T21:19:42.592 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33912.log 2024-09-10T21:19:42.594 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42845.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42845.log.gz 2024-09-10T21:19:42.594 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33912.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33912.log.gz 2024-09-10T21:19:42.595 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39234.log 2024-09-10T21:19:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36546.log 2024-09-10T21:19:42.598 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39234.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39234.log.gz 2024-09-10T21:19:42.598 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36546.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36546.log.gz 2024-09-10T21:19:42.599 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40793.log 2024-09-10T21:19:42.600 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41557.log 2024-09-10T21:19:42.602 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40793.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40793.log.gz 2024-09-10T21:19:42.602 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41557.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41557.log.gz 2024-09-10T21:19:42.603 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43933.log 2024-09-10T21:19:42.604 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47687.log 2024-09-10T21:19:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43933.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43933.log.gz 2024-09-10T21:19:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47687.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47687.log.gz 2024-09-10T21:19:42.607 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37519.log 2024-09-10T21:19:42.609 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36676.log 2024-09-10T21:19:42.610 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37519.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37519.log.gz 2024-09-10T21:19:42.611 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47026.log 2024-09-10T21:19:42.611 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36676.log.gz 2024-09-10T21:19:42.612 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36330.log 2024-09-10T21:19:42.614 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47026.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47026.log.gz 2024-09-10T21:19:42.614 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36330.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36330.log.gz 2024-09-10T21:19:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35798.log 2024-09-10T21:19:42.616 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49736.log 2024-09-10T21:19:42.618 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35798.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35798.log.gz 2024-09-10T21:19:42.619 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49736.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49736.log.gz 2024-09-10T21:19:42.619 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49348.log 2024-09-10T21:19:42.620 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47661.log 2024-09-10T21:19:42.622 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49348.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49348.log.gz 2024-09-10T21:19:42.622 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47661.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47661.log.gz 2024-09-10T21:19:42.623 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40338.log 2024-09-10T21:19:42.624 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50180.log 2024-09-10T21:19:42.626 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40338.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40338.log.gz 2024-09-10T21:19:42.626 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50180.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50180.log.gz 2024-09-10T21:19:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38562.log 2024-09-10T21:19:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48935.log 2024-09-10T21:19:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38562.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38562.log.gz 2024-09-10T21:19:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48935.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48935.log.gz 2024-09-10T21:19:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33780.log 2024-09-10T21:19:42.632 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36110.log 2024-09-10T21:19:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33780.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33780.log.gz 2024-09-10T21:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36110.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36110.log.gz 2024-09-10T21:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37694.log 2024-09-10T21:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43103.log 2024-09-10T21:19:42.638 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.37694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37694.log.gz 2024-09-10T21:19:42.639 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44680.log 2024-09-10T21:19:42.639 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43103.log.gz 2024-09-10T21:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48071.log 2024-09-10T21:19:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44680.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44680.log.gz 2024-09-10T21:19:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48071.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48071.log.gz 2024-09-10T21:19:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35928.log 2024-09-10T21:19:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44706.log 2024-09-10T21:19:42.646 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35928.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35928.log.gz 2024-09-10T21:19:42.646 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44706.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44706.log.gz 2024-09-10T21:19:42.647 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37647.log 2024-09-10T21:19:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38373.log 2024-09-10T21:19:42.650 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37647.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37647.log.gz 2024-09-10T21:19:42.650 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38373.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38373.log.gz 2024-09-10T21:19:42.651 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39391.log 2024-09-10T21:19:42.653 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42423.log 2024-09-10T21:19:42.653 INFO:teuthology.orchestra.run.smithi052.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-10T21:19:42.654 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39391.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39391.log.gz 2024-09-10T21:19:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42423.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42423.log.gz 2024-09-10T21:19:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43855.log 2024-09-10T21:19:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38484.log 2024-09-10T21:19:42.658 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43855.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43855.log.gz 2024-09-10T21:19:42.659 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38484.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38484.log.gz 2024-09-10T21:19:42.659 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46242.log 2024-09-10T21:19:42.660 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46714.log 2024-09-10T21:19:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46242.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46242.log.gz 2024-09-10T21:19:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46714.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46714.log.gz 2024-09-10T21:19:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39130.log 2024-09-10T21:19:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40650.log 2024-09-10T21:19:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39130.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39130.log.gz 2024-09-10T21:19:42.667 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40650.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40650.log.gz 2024-09-10T21:19:42.667 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47441.log 2024-09-10T21:19:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35689.log 2024-09-10T21:19:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47441.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47441.log.gz 2024-09-10T21:19:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35689.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35689.log.gz 2024-09-10T21:19:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41047.log 2024-09-10T21:19:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38848.log 2024-09-10T21:19:42.674 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41047.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41047.log.gz 2024-09-10T21:19:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38848.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38848.log.gz 2024-09-10T21:19:42.676 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38399.log 2024-09-10T21:19:42.677 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33546.log 2024-09-10T21:19:42.679 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38399.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38399.log.gz 2024-09-10T21:19:42.680 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33546.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33546.log.gz 2024-09-10T21:19:42.681 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38510.log 2024-09-10T21:19:42.682 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41203.log 2024-09-10T21:19:42.684 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38510.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38510.log.gz 2024-09-10T21:19:42.684 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41203.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41203.log.gz 2024-09-10T21:19:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35902.log 2024-09-10T21:19:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38822.log 2024-09-10T21:19:42.688 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35902.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35902.log.gz 2024-09-10T21:19:42.688 INFO:teuthology.orchestra.run.smithi052.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-10T21:19:42.689 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38822.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38822.log.gz 2024-09-10T21:19:42.689 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36519.log 2024-09-10T21:19:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40728.log 2024-09-10T21:19:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36519.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36519.log.gz 2024-09-10T21:19:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40728.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40728.log.gz 2024-09-10T21:19:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39417.log 2024-09-10T21:19:42.694 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48725.log 2024-09-10T21:19:42.696 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39417.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39417.log.gz 2024-09-10T21:19:42.697 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48725.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48725.log.gz 2024-09-10T21:19:42.697 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38456.log 2024-09-10T21:19:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47876.log 2024-09-10T21:19:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38456.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38456.log.gz 2024-09-10T21:19:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47876.log: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T21:19:42.701 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47876.log.gz 2024-09-10T21:19:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34105.log 2024-09-10T21:19:42.703 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39260.log 2024-09-10T21:19:42.704 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49658.log 2024-09-10T21:19:42.704 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34105.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34105.log.gz 2024-09-10T21:19:42.705 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39260.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39260.log.gz 2024-09-10T21:19:42.706 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49658.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49658.log.gz 2024-09-10T21:19:42.706 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44654.log 2024-09-10T21:19:42.707 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38588.log 2024-09-10T21:19:42.709 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35611.log 2024-09-10T21:19:42.709 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44654.log.gz 2024-09-10T21:19:42.710 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48831.log 2024-09-10T21:19:42.710 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38588.log.gz 2024-09-10T21:19:42.711 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35611.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35611.log.gz 2024-09-10T21:19:42.711 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42251.log 2024-09-10T21:19:42.712 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48831.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48831.log.gz 2024-09-10T21:19:42.712 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49091.log 2024-09-10T21:19:42.714 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42146.log 2024-09-10T21:19:42.714 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42251.log.gz 2024-09-10T21:19:42.715 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49091.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49091.log.gz 2024-09-10T21:19:42.715 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41525.log 2024-09-10T21:19:42.716 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42146.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.42146.log.gz 2024-09-10T21:19:42.716 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33754.log 2024-09-10T21:19:42.717 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40494.log 2024-09-10T21:19:42.717 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41525.log.gz 2024-09-10T21:19:42.719 INFO:teuthology.orchestra.run.smithi052.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-10T21:19:42.719 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40312.log 2024-09-10T21:19:42.719 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33754.log.gz 2024-09-10T21:19:42.720 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40494.log: gzip -5 --verbose -- 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40494.log.gz 2024-09-10T21:19:42.720 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.38666.log 2024-09-10T21:19:42.721 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41821.log 2024-09-10T21:19:42.721 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40312.log.gz 2024-09-10T21:19:42.723 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38666.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38666.log.gz 2024-09-10T21:19:42.723 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47285.log 2024-09-10T21:19:42.724 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41821.log.gz 2024-09-10T21:19:42.724 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49762.log 2024-09-10T21:19:42.726 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46993.log 2024-09-10T21:19:42.726 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47285.log.gz 2024-09-10T21:19:42.727 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49762.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49762.log.gz 2024-09-10T21:19:42.727 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36859.log 2024-09-10T21:19:42.728 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46993.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46993.log.gz 2024-09-10T21:19:42.728 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45894.log 2024-09-10T21:19:42.729 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37283.log 2024-09-10T21:19:42.729 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36859.log.gz 2024-09-10T21:19:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46967.log 2024-09-10T21:19:42.731 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45894.log.gz 2024-09-10T21:19:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37283.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37283.log.gz 2024-09-10T21:19:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46967.log: /var/log/ceph/ceph-client.admin.36379.log 2024-09-10T21:19:42.733 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46967.log.gz 2024-09-10T21:19:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34378.log 2024-09-10T21:19:42.735 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43023.log 2024-09-10T21:19:42.735 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36379.log.gz 2024-09-10T21:19:42.736 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34378.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34378.log.gz 2024-09-10T21:19:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49788.log 2024-09-10T21:19:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43023.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43023.log.gz 2024-09-10T21:19:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41151.log 2024-09-10T21:19:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49788.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49788.log.gz -5 2024-09-10T21:19:42.739 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.28856.log 2024-09-10T21:19:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28808.log 2024-09-10T21:19:42.740 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41151.log.gz 2024-09-10T21:19:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28856.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.28856.log.gz 2024-09-10T21:19:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28808.log: 74.8% -- replaced with /var/log/ceph/ceph-client.admin.28808.log.gz 2024-09-10T21:19:42.743 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40941.log 2024-09-10T21:19:42.744 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45302.log 2024-09-10T21:19:42.746 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46095.log 2024-09-10T21:19:42.746 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40941.log.gz 2024-09-10T21:19:42.746 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38009.log 2024-09-10T21:19:42.746 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45302.log.gz 2024-09-10T21:19:42.747 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.46095.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.46095.log.gz 2024-09-10T21:19:42.748 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48201.log 2024-09-10T21:19:42.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38009.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38009.log.gz 2024-09-10T21:19:42.749 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44435.log 2024-09-10T21:19:42.750 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48201.log: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.48201.log.gz 2024-09-10T21:19:42.750 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36781.log 2024-09-10T21:19:42.751 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.44435.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49478.log 2024-09-10T21:19:42.752 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44435.log.gz 2024-09-10T21:19:42.753 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36781.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36781.log.gz 2024-09-10T21:19:42.753 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35279.log 2024-09-10T21:19:42.754 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49478.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49478.log.gz 2024-09-10T21:19:42.754 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29234.log 2024-09-10T21:19:42.755 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35279.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35279.log.gz 2024-09-10T21:19:42.756 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43619.log 2024-09-10T21:19:42.757 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43451.log 2024-09-10T21:19:42.757 INFO:teuthology.orchestra.run.smithi016.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29234.log.gz 2024-09-10T21:19:42.758 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48751.log 2024-09-10T21:19:42.758 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43619.log.gz 2024-09-10T21:19:42.759 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46663.log 2024-09-10T21:19:42.760 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43451.log.gz 2024-09-10T21:19:42.761 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48751.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48751.log.gz 2024-09-10T21:19:42.761 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48987.log 2024-09-10T21:19:42.762 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46663.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46663.log.gz 2024-09-10T21:19:42.762 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48304.log 2024-09-10T21:19:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34921.log 2024-09-10T21:19:42.765 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48987.log.gz 2024-09-10T21:19:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48304.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48304.log.gz 2024-09-10T21:19:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33897.log 2024-09-10T21:19:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34921.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34921.log.gz 2024-09-10T21:19:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48616.log 2024-09-10T21:19:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43992.log 2024-09-10T21:19:42.769 INFO:teuthology.orchestra.run.smithi016.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.33897.log.gz 2024-09-10T21:19:42.769 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48616.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48616.log.gz 2024-09-10T21:19:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44253.log 2024-09-10T21:19:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43723.log 2024-09-10T21:19:42.771 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43992.log.gz 2024-09-10T21:19:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44357.log 2024-09-10T21:19:42.772 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44253.log.gz 2024-09-10T21:19:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45196.log 2024-09-10T21:19:42.773 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43723.log.gz 2024-09-10T21:19:42.775 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44357.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44357.log.gz 2024-09-10T21:19:42.775 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46585.log 2024-09-10T21:19:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38945.log 2024-09-10T21:19:42.777 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45196.log.gz 2024-09-10T21:19:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43207.log 2024-09-10T21:19:42.777 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46585.log.gz 2024-09-10T21:19:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.38945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38945.log.gz 2024-09-10T21:19:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44089.log 2024-09-10T21:19:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44844.log 2024-09-10T21:19:42.780 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43207.log.gz 2024-09-10T21:19:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38347.log 2024-09-10T21:19:42.781 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44089.log.gz 2024-09-10T21:19:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44844.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44844.log.gz 2024-09-10T21:19:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45868.log 2024-09-10T21:19:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38347.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38347.log.gz 2024-09-10T21:19:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40761.log 2024-09-10T21:19:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39534.log 2024-09-10T21:19:42.786 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45868.log.gz 2024-09-10T21:19:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40761.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40761.log.gz 2024-09-10T21:19:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39534.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39534.log.gz 2024-09-10T21:19:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49840.log 2024-09-10T21:19:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45406.log 2024-09-10T21:19:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42877.log 2024-09-10T21:19:42.791 INFO:teuthology.orchestra.run.smithi016.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.49840.log.gz 2024-09-10T21:19:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45406.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45406.log.gz 2024-09-10T21:19:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42877.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42877.log.gz 2024-09-10T21:19:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35201.log 2024-09-10T21:19:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49039.log 2024-09-10T21:19:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35201.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35201.log.gz 2024-09-10T21:19:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49039.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49039.log.gz 2024-09-10T21:19:42.811 INFO:teuthology.orchestra.run.smithi016.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T21:19:42.826 INFO:teuthology.orchestra.run.smithi016.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-10T21:19:42.875 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T21:19:42.918 INFO:teuthology.orchestra.run.smithi052.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-10T21:19:43.010 INFO:teuthology.orchestra.run.smithi016.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-10T21:19:43.024 INFO:teuthology.orchestra.run.smithi016.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T21:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-10T21:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m1.101s 2024-09-10T21:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m4.597s 2024-09-10T21:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m1.336s 2024-09-10T21:19:44.732 INFO:teuthology.orchestra.run.smithi052.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-10T21:19:44.734 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-09-10T21:19:44.734 INFO:teuthology.orchestra.run.smithi052.stderr:real 0m2.769s 2024-09-10T21:19:44.734 INFO:teuthology.orchestra.run.smithi052.stderr:user 0m6.161s 2024-09-10T21:19:44.734 INFO:teuthology.orchestra.run.smithi052.stderr:sys 0m0.555s 2024-09-10T21:19:44.735 INFO:tasks.ceph:Archiving logs... 2024-09-10T21:19:44.735 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899658/remote/smithi016/log 2024-09-10T21:19:44.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T21:19:45.618 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899658/remote/smithi052/log 2024-09-10T21:19:45.618 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T21:19:46.278 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T21:19:46.287 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T21:19:46.287 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T21:19:46.290 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T21:19:46.395 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T21:19:46.403 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T21:19:46.403 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:19:46.407 DEBUG:teuthology.orchestra.run.smithi052:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:19:46.419 INFO:teuthology.orchestra.run.smithi016.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T21:19:46.420 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================== 2024-09-10T21:19:46.420 INFO:teuthology.orchestra.run.smithi016.stdout:*hv01.front.sepi 67.205.162.81 3 u 62 64 377 0.081 -1.928 1.243 2024-09-10T21:19:46.420 INFO:teuthology.orchestra.run.smithi016.stdout:+hv02.front.sepi 96.245.170.99 3 u 62 64 377 0.066 -3.464 0.750 2024-09-10T21:19:46.420 INFO:teuthology.orchestra.run.smithi016.stdout:+hv03.front.sepi 74.6.168.72 3 u 62 64 377 0.097 +0.226 0.991 2024-09-10T21:19:46.420 INFO:teuthology.orchestra.run.smithi016.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-10T21:19:46.422 INFO:teuthology.orchestra.run.smithi052.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T21:19:46.422 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================== 2024-09-10T21:19:46.422 INFO:teuthology.orchestra.run.smithi052.stdout:*hv01.front.sepi 67.205.162.81 3 u 5 64 377 0.086 -0.868 1.082 2024-09-10T21:19:46.422 INFO:teuthology.orchestra.run.smithi052.stdout:+hv02.front.sepi 96.245.170.99 3 u 1 64 377 0.082 -2.960 0.948 2024-09-10T21:19:46.423 INFO:teuthology.orchestra.run.smithi052.stdout:+hv03.front.sepi 74.6.168.72 3 u 62 64 377 0.083 +0.378 0.900 2024-09-10T21:19:46.423 INFO:teuthology.orchestra.run.smithi052.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-10T21:19:46.423 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T21:19:46.432 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T21:19:46.432 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T21:19:46.439 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T21:19:46.448 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T21:19:46.455 INFO:teuthology.task.internal:Duration was 1301.850473 seconds 2024-09-10T21:19:46.456 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T21:19:46.464 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T21:19:46.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T21:19:46.468 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T21:19:46.572 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T21:19:46.573 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-09-10T21:19:46.573 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T21:19:46.594 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi052.front.sepia.ceph.com 2024-09-10T21:19:46.595 DEBUG:teuthology.orchestra.run.smithi052:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T21:19:46.616 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T21:19:46.616 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T21:19:46.646 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T21:19:46.699 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T21:19:46.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T21:19:46.733 DEBUG:teuthology.orchestra.run.smithi052:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T21:19:46.829 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T21:19:47.170 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T21:19:47.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T21:19:47.195 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T21:19:47.221 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T21:19:47.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:19:47.410 DEBUG:teuthology.orchestra.run.smithi052:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:19:47.426 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-09-10T21:19:47.430 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = core 2024-09-10T21:19:47.449 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:19:47.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:19:47.494 DEBUG:teuthology.orchestra.run.smithi052:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:19:47.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:19:47.503 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T21:19:47.578 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T21:19:47.580 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899658/remote/smithi016 2024-09-10T21:19:47.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T21:19:47.610 DEBUG:teuthology.misc:Transferring archived files from smithi052:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899658/remote/smithi052 2024-09-10T21:19:47.610 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T21:19:47.637 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T21:19:47.637 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T21:19:47.649 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T21:19:47.687 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T21:19:47.697 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T21:19:47.697 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T21:19:47.706 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T21:19:47.706 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T21:19:47.709 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T21:19:47.715 INFO:teuthology.orchestra.run.smithi016.stdout: 655464 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 21:19 /home/ubuntu/cephtest 2024-09-10T21:19:47.743 INFO:teuthology.orchestra.run.smithi052.stdout: 658141 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 21:19 /home/ubuntu/cephtest 2024-09-10T21:19:47.745 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T21:19:47.755 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T21:19:47.775 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/client-limits} duration: 1301.8504734039307 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-10T21:19:47.775 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T21:19:47.856 INFO:teuthology.run:pass